KR100501414B1 - Method of and apparatus for logging and restoring the meta data in file system - Google Patents

Abstract

The present invention relates to a method for restoring file system metadata to a consistent state when a system failure occurs in a computer. The metadata of the file system is information that configures and manages the file system. The loss of metadata due to a system failure has an important effect on the file system operation. Therefore, it is very important to restore the file system's metadata to a consistent state. According to the present invention, when a log of changed metadata is recorded and a system failure occurs, the recorded log can be used to restore the metadata of the file system to a consistent state. Such logs may be stored on the same storage device as the file system or on another storage device. In addition, the log space in which logs are stored allows repetitive reuse of a fixed amount of contiguous space. In this case, the log space should be kept only the minimum log needed for recovery, so that when a system failure occurs, the log space can be scanned more quickly and the file system metadata can be restored to a consistent state.

Description

Method and device for logging and restoring the meta data in file system}

The present invention relates to a file system of a computer, and more particularly, to a method for logging metadata of a file system and a method and apparatus for recovering the metadata of a file system to a consistent state by the logged data when a system failure occurs. It is about.

In general, the metadata of the file system is information for configuring and managing the file system, and the loss of the metadata due to a system failure has an important effect on the file system operation. Therefore, it is very important to restore the file system's metadata to a consistent state.

That is, the metadata of the file system is information for configuring and managing the file system such as a super block, a bitmap, an inode, a directory, and the like. If a system failure occurs, the metadata is restored to a consistent state. Failure to do so can cause serious problems in future file operations and can even render the entire file system incapable. Therefore, in recent file systems, the updated metadata is stored in a log to ensure the consistency of the metadata, and when a system failure occurs, the metadata is restored to a consistent state by executing the log again. Methods are being developed.

The present invention was devised in view of the above situation, and it is possible to efficiently reuse a fixed size of log space, and at least when a system failure occurs while maintaining only the minimum log necessary for metadata recovery in the log space. It is an object of the present invention to provide a logging and recovery method and apparatus for recovering metadata of a file system that can quickly recover meta data to a consistent state by scanning a log space of a single time.

In order to achieve the above object, the present invention relates to a method of recovering metadata of a file system from logged data, the method comprising: reading a log from an LSN of a minimum valid log at the time of a system failure; Checking whether the read log is a valid login; If the read log is a valid log, determining a log redo procedure according to the log type; If the log type is 'meta data log', the actual metadata of the log is read from the disk into the buffer, and the metadata version stored in the log is compared by comparing the metadata version of the log with the metadata version stored on the disk. If large, redoing the metadata loaded in the buffer by overwriting the metadata of the log; If the log type is a 'transaction completion log', unfixing a buffer in which all metadata logs for a transaction are re-run because the log is stored by a normally completed transaction; If the log type is a 'transaction withdrawal log', the log is written by an abnormally completed transaction, and thus, ignoring the contents of the buffer that has been re-executed for the metadata log of the transaction and unfixing the buffers; Changing the LSN value of the minimum valid log to the LSN value of the next log to be read when all processing is completed according to the log type; And repeating the above process for all valid logs in the log space, and when the log is re-executed, storing all the metadata buffers that have been re-executed on disk.

In addition, in order to achieve the above object, the system of the present invention, in the file system for storing and managing files in the file storage space, by changing the metadata by performing file operations for changing the metadata of the file system on a transaction basis A transaction manager module for supporting recovery of the operation; A metadata buffer manager module configured to manage a buffer space in which operations on metadata are performed according to a request of the transaction; A log manager module for storing a log of meta data changed in the transaction in a global log buffer unit in a log space in a meta data buffer; And a recovery manager module for recovering the metadata of the file system to a consistent state by using the log stored in the log space through the log manager module when a system failure occurs.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of modules related to recovery in a file system to which the present invention is applied. In the file system according to the present invention, a transaction manager module 112, a metadata buffer manager module 114, and a file storage space are associated with file operations. 116, there is a recovery manager module 122, a log manager module 124, and a log space 126 with respect to the recovery operation.

Referring to FIG. 1, the transaction manager module 112 supports recovery of a metadata change operation by performing file operations that change metadata of a file system on a transaction basis. The metadata buffer manager module 114 manages the buffer space in which the operation on the metadata is performed, and the log manager module 124 logs the log space for the metadata changed in the transaction directly in the metadata buffer. ). The recovery manager module 122 uses the log stored in the log space 126 through the log manager module 124 to restore the metadata of the file system to a consistent state when a system failure occurs. File storage space 116 and log space 126 may be on the same device or on different devices.

2 is a schematic diagram showing the relationship between the global log buffers 220-1 and 220-2 of the recovery manager module 122 and the transaction log buffer 210 allocated for each transaction in the file system to which the present invention is applied.

Referring to FIG. 2, the logs are stored in the global log buffer unit 220 in the log space 126 through the global log buffers 220-1 and 220-2. The global log buffers 220-1 and 220-2 are placed in order to improve the concurrency of the transactions recording the log. Thus, when one global log buffer is stored, transactions can continue to write logs to another global log buffer. The transaction log buffer 210 is used for each transaction. The transaction log buffer 210 records information about the metadata log each time a transaction records the metadata log. This metadata log information will be used later for recovery due to system failure. The contents of the transaction log buffer 210 are written to the global log buffers 220-1, 220-2 and stored in the log space 126 when completing or withdrawing a transaction.

3 is a format of log data stored in log space 126 in a file system to which the present invention is applied.

Referring to FIG. 3, the log includes a metadata log 310 and a transaction termination or withdrawal log 320. The metadata log 310 includes header information 312 and changed metadata blocks 314 for the log. It is composed. The transaction completion or withdrawal log 320 is composed of a log header 322 and information 324 about the metadata log recorded by the transaction.

4 is a schematic diagram showing a relationship between a global log buffer and a log space in a file system to which the present invention is applied.

Referring to FIG. 4, the log is stored in the log space 126 in the global log buffer unit 220. Each log has a Log Sequence Number (LSN), which is the only value that can be distinguished from other logs. This log sequence number (LSN) consists of a sequence number representing the number of times log space 126 has been reused and a relative offset from the start of log space 126, which is the log space. The location of the log may be indicated at 126.

The log manager module 124 maintains three pieces of information for storing and managing these logs. First, the log sequence number (LSN) value 400 of the currently added log is maintained. This value indicates the starting position of the log that is currently added to the global log buffer 220 and allocates the LSN value of the next added log, incrementing each time a transaction adds the log. Secondly, the LSN value 401 of the minimum valid log is maintained. The LSN value 401 of the least valid log is the smallest LSN value among the valid logs required to recover the metadata to a consistent state from a system failure, and is a starting position to access the log space 126 for system recovery. Thirdly, the LSN value 402 of the log most recently stored in the log space 126 is maintained. The LSN value 402 of the recent log points to the beginning of the log last stored in the log space 126, and also to the beginning of the last valid log. Therefore, only a log between the LSN 401 of the minimum valid log and the LSN 402 of the most recently stored log needs to be rerun to recover from a system failure.

5 is a schematic diagram illustrating a concept of repeatedly reusing log space in a file system to which the present invention is applied.

Referring to FIG. 5, in order to efficiently reuse a fixed size log space 126, an association between the metadata log 310 stored in the log space 126 and the metadata in which the actual metadata is changed is described. You need to keep it. In other words, if the changed metadata is stored from the buffer to disk, the log for that metadata is no longer valid. Thus, these logs do not need to be kept in log space 126 continuously.

In the present invention, the valid log table 500 is maintained in the global log buffer unit 220 stored in the log space 126. In addition, the valid log table 500 records the reference value of the metadata buffer, which is the number of metadata for the metadata log 310 stored in the log, which is not yet stored in the buffer to disk. In addition, the header 503 of the meta data buffer maintains valid log table information of a log space in which a log of meta data loaded in the buffer is stored, and an LSN value in which the meta data log 310 is stored. The metadata buffer reference value of the valid log table 500 is incremented 501 after storing the metadata log when the transaction completes. And later, when the metadata buffer is saved to disk, it is reduced (502).

6 is a flow chart for logging when completing a transaction in a file system to which the present invention is applied.

Referring to FIG. 6, the transaction performs steps 600 to 603 on all metadata buffers changed by the transaction upon completion. First, the metadata log 310 is written to the global log buffers 220-1 and 220-2, and the reference value of the metadata buffer is increased in the valid log table 500 corresponding to the global log buffers 220-1 and 220-2. (600). In operation 601, information about the meta data log 310 is recorded in the transaction log buffer 210. Next, information about the valid log table 500 and the LSN value in which the metadata log 310 is recorded are set in the header 503 of the metadata buffer (602). At this time, if the metadata log has been recorded before, the LSN value of the previously recorded metadata log is compared with the LSN value of the currently recorded metadata log. In step 603, the metadata buffer is stored to be stored after the data storage is stored. After this processing is performed for all changed metadata, the transaction completion log 320 of the transaction log buffer 210 is written to the global log buffers 220-1 and 220-2 (604), and the global log buffer 220- 1,220-2 is stored in the log space 126 (605). And after storing (606) the metadata buffer marked for storage on disk in step 603, release the latch on all changed metadata buffers so that other transactions can access the modified metadata buffer (607). .

7 is a flowchart illustrating management of log space when a metadata buffer is stored on a disk in the file system to which the present invention is applied.

Changes to the metadata are made on the metadata buffer. Therefore, the transaction loads the metadata into the metadata buffer and changes the metadata in order to change the metadata. At this point, the transaction latches the changed metadata so that no other transaction can access it, and then releases the latch when it completes or withdraws the transaction. Therefore, metadata stored on disk is written only by transactions that have completed normally. The metadata buffer is stored on disk when the buffer is selected to be written to disk by the buffer management policy or when the metadata buffer is changed on the buffer without being reflected on the disk for a long time. Also, when you unmount the file system, all changed metadata buffers are flushed to disk. Here, storing the metadata buffer to disk means that the metadata log is no longer valid. Therefore, when reflecting the metadata buffer to disk, the log space can be recycled and reused by appropriately handling the log space.

Referring to FIG. 7, the operation of storing the meta data buffer on the disk first checks whether the meta data loaded on the meta data buffer has changed (700), and immediately stores the meta data buffer on the disk (701). If the difference between the LSN 401 of the minimum valid log of the current log space and the LSN 402 of the recently stored log is greater than the size of the global log buffer (702), the valid log table indicated by the metadata buffer header 503 At (500), 1 is subtracted from the reference value of the metadata buffer (703).

8 is a flowchart illustrating the management of log space when storing a log buffer in a file system to which the present invention is applied.

The log buffer is stored in the log space 126 when the log buffer is full or when the log recorded while completing a transaction is not reflected in the log space 126. At this time, a process for maintaining only the minimum valid log in the log space 126 is performed.

First, after storing the log buffer, the difference between the LSN value 401 of the current minimum valid log and the LSN value 402 of the recently stored log is compared with the size of the global log buffer (800). If the difference is smaller than the size of the global log buffer, the minimum valid log LSN value 401 is not changed since the log may still be written by another transaction in the global log buffer. Otherwise, if the difference is greater than the size of the global log buffer, the valid log table 500 corresponding to the current minimum valid log LSN 401 compares the value referenced by the metadata buffer (801). If the value is "0", the metadata log written to the log corresponding to the minimum valid log LSN 401 is no longer valid. This is because the metadata has already been reflected on disk. Therefore, the minimum valid log LSN value 401 is changed to the LSN value corresponding to the next valid log table. And the difference between the minimum valid log LSN 401 and the last stored log LSN 402 is less than the global log buffer size, or the metadata buffer reference value of the valid log table corresponding to the minimum valid log LSN 401 is greater than " 0 ". Repeat steps 800-802 until large.

9 is a flowchart illustrating management of log space when a file system is normally unmounted in a file system to which the present invention is applied.

Referring to FIG. 9, if the file system is normally unmounted, all metadata changed by the file operation performed while the file system is mounted are reflected to the disk. Thus, logs recorded by previous file operations no longer need to be maintained. In the present invention, even if the file system is remounted while repeatedly reusing a fixed size log space, the log can be recorded without initializing the previously written log.

To this end, in the present invention, when the file system is unmounted, it is checked whether there are recently stored logs after storing all changed metadata on the disk (900). If there is a recently stored log, the LSN value 401 of the minimum valid log is changed to the LSN value of the recently stored log (901). If the changed displacement of the minimum valid log LSN 401 reaches the end of the log space 126, the sequence number of the LSN is increased by one (902). Thus, again, the log stored from the beginning of the log space 126 can have a unique LSN value.

10 is a flowchart illustrating a process of restoring the metadata of the file system to a consistent state when a system failure occurs in the file system to which the present invention is applied.

First, the log is read from the LSN 401 of the minimum valid log at the time of a system failure (1000). It checks if the log read is a valid login. That is, the log read is the last 1001 or the previously stored login location 1002 is checked. If the read log is a valid log, the log re-execution procedure is determined according to the log type (1003). There can be three re-processing steps depending on the type of log.

First, in the case of the 'meta data log' 310 having information on the meta data actually changed, the meta data of the log is read from the disk into the buffer (1004). The metadata version of the log is compared with the metadata version stored in the disk. If the version of the metadata stored in the log is large (1007), a system failure occurs before the changed metadata is stored on the disk. Redo is performed by overwriting the metadata with the metadata of the log (1008).

Secondly, if the transaction completion log 320 is a log stored by a normally completed transaction, the log has information about a metadata log recorded by the transaction. Therefore, in this case, the buffer which re-performed all the metadata logs for the transaction is unfixed (1005).

Third, the transaction withdrawal log 320 is a log written by an abnormally completed transaction. The log has information of a metadata log recorded before the transaction is withdrawn. Therefore, the processing for this log ignores the contents of the buffer that was previously rerun for the metadata log of this transaction and unfixes the buffers (1006) so that the result of the rerun by the withdrawn transaction is not left.

After processing for the three cases according to the type of log read as described above, the LSN value 401 of the minimum valid log is changed to the LSN value of the next log to be read (1009). The process of steps 1000 to 1009 is repeated for all valid logs in the log space. When the log is rerun, all metadata buffers that have been rerun are saved to disk (1010) to make the metadata consistent.

As described above, the present invention maintains a minimum number of logs in the log space and restores the file system metadata in a single log scan in order to recover the file system metadata to a consistent state more quickly when a system failure occurs. You can recover to a consistent state. In addition, log space can be efficiently managed by reusing fixed size log space.

1 is a block diagram of modules related to recovery in a file system to which the present invention can be applied;

2 is a schematic diagram showing a relationship between a log buffer of a recovery manager module and a log buffer of transactions in a file system to which the present invention is applied;

3 is a format of log data stored in log space in a file system to which the present invention is applied;

4 is a conceptual diagram illustrating a relationship between a log buffer and a log space in a file system to which the present invention is applied;

5 is a schematic diagram illustrating a concept for repeatedly reusing log space in a file system to which the present invention is applied;

6 is a flow chart for logging when completing a transaction in a file system to which the present invention is applied;

7 is a flowchart illustrating management of log space when a metadata buffer is stored on a disk in a file system to which the present invention is applied;

8 is a flowchart illustrating management of log space when a log buffer is stored in log space in a file system to which the present invention is applied;

9 is a flowchart for managing log space when unmounting a file system in the file system to which the present invention is applied;

10 is a flowchart illustrating a recovery process when a system failure occurs in a file system to which the present invention is applied.

<Explanation of symbols for the main parts of the drawings>

112: transaction manager module 114: metadata buffer manager module

116: file storage 122: recovery manager module

124: Log Manager Module 126: Log Space

210: Transaction log buffer 220, 220-1,220-2: Global log buffer

310: Metadata Log 322: Transaction End / Cancel Log

Claims (12)

In the file system for storing and managing files in the file storage space, A transaction manager module for supporting recovery of the metadata change operation by performing file operations for changing the metadata of the file system on a transaction basis; A metadata buffer manager module configured to manage a buffer space in which operations on metadata are performed according to a request of the transaction; A log manager module for storing a log of meta data changed in the transaction in a global log buffer unit in a log space in a meta data buffer; And File system metadata comprising a recovery manager module for recovering the metadata of the file system to a consistent state by using the log stored in the log space through the log manager module when a system failure occurs Logging and recovery system for recovery. The method of claim 1, wherein the log manager module, To improve the concurrency of the transactions that record the logs, two global log buffers are provided so that when one global log buffer is stored, the transactions are kept written to another global log buffer. Logging and recovery system for data recovery. The method of claim 1, wherein the transaction manager module, A transaction log buffer is used for each transaction, and the transaction log buffer records and stores information on the metadata log every time a transaction writes the metadata log. . The method of claim 3, wherein the transaction log buffer, Logging and recovery system for metadata recovery of a file system, characterized in that when the transaction is completed or withdrawn, it is written to the global log buffer and stored in log space. The method of claim 1, The log is divided into a metadata log and a transaction termination / retraction log, and the metadata log is composed of a log header and a changed metadata block, and the transaction completion / retraction log is a log header and a metadata log recorded by a transaction. A logging and retrieval system for file system metadata recovery, comprising information. The method of claim 5, The log has a Log Sequence Number (LSN), which is the only value that can be distinguished from other logs, and the Log Sequence Number (LSN) is a sequence number representing the number of times log space is reused, and a log. A logging and recovery system for metadata recovery of a file system, characterized by a relative offset from the start of space. The method of claim 1, wherein the log manager module It maintains a valid log table in units of global log buffers that are stored in log space for log storage and management.The log sequence number (LSN) value of the currently added log, the LSN value of the minimum valid log, and the recent log space The LSN value of the log stored in A logging and recovery system for recovering metadata of a file system, characterized in that only logs between the LSN of the least valid log and the LSN of the most recent stored log are rerun to recover from a system failure. Writing the meta data log to a global log buffer and increasing a reference value of the meta data buffer in a valid log table corresponding to the global log buffer; Writing information about the meta data log into a transaction log buffer; Setting a log sequence number (LSN) value in which information about a valid log table and a metadata log are recorded in a header of the metadata buffer; If the metadata log has been recorded before, the log sequence number (LSN) value of the previously recorded metadata log is compared with the log sequence number (LSN) value of the currently recorded metadata log and the value is greater than or equal to the threshold. If so, marking all the logs in a log space and then storing a metadata buffer; After the process is performed on all changed metadata, recording the transaction completion log of the transaction log buffer in the global log buffer and storing the global log buffer in the log space; And After storing the metadata buffer indicated on the disk to be stored on disk, unlocking all changed metadata buffers so that other transactions can access the changed metadata buffer. Logging method for recovery. The method of claim 8, wherein the metadata buffer is stored in a disk. Checking whether the metadata loaded in the buffer has been changed, and if the metadata has been changed, immediately storing the metadata in the disk; If the difference between the log sequence number (LSN) of the minimum valid log in the current log space and the recently stored log sequence number (LSN) is greater than the size of the global log buffer, the metadata in the valid log table pointed to by the metadata buffer header And subtracting 1 from the reference value of the buffer. The method of claim 8, wherein when storing the log buffer Comparing the size of the log buffer with the difference between the LSN of the current minimum valid log and the LSN of the most recently stored log after storing the log buffer; If the difference is less than the size of the global log buffer, maintaining the minimum valid log LSN value; Comparing the value referenced by the metadata buffer with a valid log table corresponding to the current minimum valid log LSN if the difference is greater than the size of the global log buffer; If the referenced value is "0", changing the minimum valid log LSN value to an LSN value corresponding to the next valid log table since the metadata log written to the log corresponding to the minimum valid log LSN is no longer valid; And Repeating the above steps until the difference between the minimum valid log LSN and the most recent stored log LSN is less than the log buffer size or the metadata buffer reference value of the valid log table corresponding to the minimum valid log LSN is greater than "0". Logging method for recovering metadata of the file system, characterized in that provided. The method of claim 8 wherein the file system is normally unmounted, Checking whether there are recently stored logs after storing all changed metadata on the disk; If there is a recently stored log, changing the LSN value of the minimum valid log to the LSN value of the recently stored log; And And increasing the sequential number of the LSN by one when the changed displacement of the minimum valid log LSN reaches the end of the log space. A method for recovering metadata of a file system from data logged by the method of claim 8, Reading a log from the LSN of the minimum valid log at the time of a system failure; Checking whether the read log is a valid login; If the read log is a valid log, determining a log redo procedure according to the log type; If the log type is 'meta data log', the actual metadata of the log is read from the disk into the buffer, and the metadata version stored in the log is compared by comparing the metadata version of the log with the metadata version stored on the disk. If large, redoing the metadata loaded in the buffer by overwriting the metadata of the log; If the log type is a 'transaction completion log', unfixing a buffer in which all metadata logs for a transaction are re-run because the log is stored by a normally completed transaction; If the log type is a 'transaction withdrawal log', since the log is written by an abnormally completed transaction, ignoring the contents of the buffer that has been re-executed for the metadata log of the transaction and unfixing the buffers; Changing the LSN value of the minimum valid log to the LSN value of the next log to be read when all processing is completed according to the log type; And Repeating the above process for all valid logs in the log space, and if the log has been rerun, storing all the metadata buffers that have been rerun to the disk; .