KR100611739B1 - PARALLEL RESOURCE Allocation AND freeing METHOD IN JOURNALING FILE SYSTEM OF MULTI-PROCESSOR ENVIRONMENT - Google Patents

Abstract

An object of the present invention is to enable the early return of a lock before the completion of a transaction without compromising consistency and resilience, thereby eliminating the complexity and additional cost of handling lock deadlocks, since lock deadlocks between allocation groups do not inherently occur. In order to achieve the object of the present invention, a parallel resource allocation and return method in a journaling file system in a multiprocessor environment is divided into a plurality of allocation groups in a file system supporting journaling operating in a multiprocessor environment. A method for allocating and returning a block resource of a disk by using metadata, the method allocating a resource from a corresponding allocation group for a resource allocation request for a specific allocation group requested from a transaction and assigning an identifier of the allocated resource to a transaction table Resource allocation to register in the resource list And a resource return step of registering an identifier of a corresponding resource in a return resource list of a transaction table for a request for return of a specific resource requested from the transaction, and waiting for the actual resource return procedure until the transaction is completed. .

Journal file system, metadata, allocation group, parallel processing

Description

PARALLEL RESOURCE Allocation AND freeing METHOD IN JOURNALING FILE SYSTEM OF MULTI-PROCESSOR ENVIRONMENT}             

1 is a flow diagram of a method of allocating and returning resources in a conventional Unconditional Commit Duration lock technique.

2 is an exemplary view showing the problem of FIG.

3 is a flow diagram of a resource allocation and return method in a conventional conditional Commit Duration lock technique.

4 is an exemplary view showing the problem of FIG.

5 is a configuration diagram of a parallel resource allocation and return system to which an embodiment of the present invention is applied.

6 is an exemplary diagram of a parallel resource allocation method in a journaling file system of a multiprocessor environment in accordance with an embodiment of the present invention.

7 is an exemplary diagram of a parallel resource return method in a journaling file system in a multiprocessor environment according to an embodiment of the present invention.

8 is an exemplary view of locking deadlock avoidance according to an embodiment of the present invention.

9 is a flowchart of a transaction initiation method according to an embodiment of the present invention.

10 is a flowchart of a resource allocation method according to an embodiment of the present invention.

11 is a flowchart of a resource return method according to an embodiment of the present invention.

12 is a flowchart of a transaction completion method according to an embodiment of the present invention.

13 is a flow chart of a transaction withdrawal method according to an embodiment of the present invention.

The present invention relates to a parallel processing method for metadata in a journal file system. In particular, the metadata representing the allocation status of all blocks in a storage device is divided into a plurality of allocation groups, In a file system that acquires independent locks for each allocation group to increase access parallelism, parallel resource allocation and return methods can be used in a journaling file system in a multiprocessor environment that enables secure early return of locks even before transaction completion. It is about.

At present, with the rapid development of computer and networking environments, the amount of data to be stored and the processing speed required are rapidly increasing. As the environment changes, existing file systems represented by ufs and ext2 exhibit various limitations, and new file systems such as ext3 and XFS are emerging. These recent file system-related technologies are large file support functions that exceed the existing 32-bit size, large directory support for fast access when a large number of files exist in a specific directory, and files. Allocating contiguous blocks within the same allocation group, existing block-based allocation methods, along with capacity-related techniques such as large number of file support to eliminate the number of files in the system. High locality between blocks belonging to the same file, such as extent-based allocation schemes, fast crash recovery using logging, and symmetric multi-processor High parallelism support using special data structures and algorithms that make the best use of features It is related technology.

Indeed, ext2 splits metadata into multiple allocation groups, but when you change it, you acquire a single lock on the entire allocation group. This is not a problem in a single process environment, but in a multiprocess environment, other processes that access the same file system have to wait for locks, which hinders parallelism.

This limitation continues in ext3. Ext3 is a file system that adds journaling to the basic functionality of ext2, and has all the limitations of ext2. Ext3 does not retain the lock until the transaction completes, but immediately returns it.

While ext3's locking mechanism is known to cause cascading abort and unrecoverable problems in traditional transaction processing technology, ext3 is limited in the nature of operations such as resource allocation and return.

As can be seen in steps S101-S106 of FIG. 1, in traditional transaction processing techniques such as ARIES, all shared locks and exclusive locks are maintained until transaction completion by default to ensure consistency and isolation. However, for certain applications where low isolation levels are not a problem, the point at which the shared lock is returned can be moved before transaction completion. If you do not follow this method, the transaction may become unrecoverable, or when a specific transaction is canceled, another transaction may have to be cascading abort. Ext3 does not follow this locking method, so this problem can occur in principle.However, due to the nature of allocation and return operations, when a transaction that returns resources has to be withdrawn after a new transaction allocates resources returned by a transaction that has not yet completed. Only the above problem occurs. To solve this problem, ext3 keeps a buffer in which the version of the metadata block that is changed is the only version that has been written by the completed transaction as well as the current version. The return of resources is recorded in the current version, and the allocation of resources uses the method of allocating the resources that are marked as free resources in both the current and completed versions. This approach has the problem that although ext3 has a short locking period for metadata, the burden of maintaining two metadata in the buffer for this purpose is not completely free from lock collisions in a multiprocessor environment.

This problem with ext3 has been largely solved in XFS.

XFS solves the same problem that new transactions can allocate resources returned by incomplete transactions, such as ext3, without maintaining a separate copy. This problem occurs because the lockout period is not until synchronous completion, which is when the actual log record is written to the log device, but until asynchronous completion, which is when it is written to the log buffer in memory. By setting the BUSY flag up to the actual synchronization completion point, use this method to prohibit the allocation of the resource, or use the method of forcibly writing the log of the transaction that returned this resource to the log device. The problem is solved efficiently without having a copy.

Also, unlike ex3, XFS only acquires locks on metadata blocks for allocation groups that are actually accessed to increase parallelism in multiprocess environments. As a result, file systems that access the same file system but access different allocation groups do not inherently experience a lock wait, which greatly improves parallelism.

However, these individual locks between allocation groups have to be a commit duration lock by the transaction concept introduced in XFS for journaling, thereby preventing the locks from being deadlocked. In other words, locks on an allocation group in XFS are returned after a transaction has been asynchronously committed, which allows the same data to match the order of the logs on the log device when multiple transactions change at the same time. Constraints This is a constraint expressed as strict execution of a transaction in a traditional database. As a result, XFS's lock operation by allocation group significantly improves parallelism in multiprocessor environments, compared to ext2 and ext3's locking, but has a new problem of possible deadlock between allocation groups.

That is, if a particular transaction needs to allocate or return resources across multiple allocation groups (such as write (), unlink (), or truncate () on large data), the lock period is until asynchronous completion, so You cannot release a lock on a previously allocated group before obtaining a lock on the metadata for the group. As a result, lock deadlocks can occur for locks already acquired between two or more transactions. For example, as shown in FIG. 2, when transaction T1 is acquiring a lock on allocation group 1, and attempts to acquire a lock on allocation group 2, transaction T2 acquires a lock on allocation group 2. If you try to acquire a lock on Allocation Group 1 in a state, a lock deadlock occurs between these two transactions.

Therefore, XFS needs a special procedure to prevent lock deadlock, and uses such procedures as lock ordering and conditional locking (using the XFS_BMI_TRYLOCK flag) for allocation group locks. Allocation group access ordering is to order the lock request for a later allocation group than the allocation group for which the lock has already been acquired. However, this ordering is not always possible, especially if the last allocation group has been accessed but the allocation and return remain to be processed, XFS cycles access back to the first allocation group. In this case the lock ordering is broken and conditional locking is used, as shown in Figs. 3 and 4, to detect the possibility of lock deadlock in advance. In this way, XFS can avoid lock deadlocks, but these techniques have to be applied, which increases the complexity of the allocation and return process and increases the cost.

Conventional techniques that use period locks that cannot be freed from lock deadlocks are a generalized way to ensure the consistency and isolation of these resources when accessing multiple resources simultaneously. In this case, the generalized method means that the above lock operation method can easily guarantee the consistency and isolation of resources regardless of the data to be protected and the characteristics of the operation.

The transaction management method results in serializing all operations and logging in metadata units for each allocation group for the resource allocation and return procedure through the allocation group. That is, maintaining a completion point lock on metadata of a changed allocation group serializes metadata change and log record records of all transactions for the same allocation group in metadata change units.

Such serialization of metadata units is excessive serialization for the following reasons, thereby causing a problem of paying more synchronization costs than necessary. That is, a block different from an already allocated block may be allocated to a new transaction regardless of whether or not the previous transaction is completed. In addition, blocks that are already allocated are not duplicated in other transactions even if they do not use a separate completion period lock in the metadata. In addition, if allocation-related logging is performed by operational logging rather than before image logging for all of the metadata, the ordering of the logs does not need to be serialized over the entire metadata, only for that block. Therefore, maintaining completion period locks on allocations is an unnecessary synchronization and can also avoid lock deadlocks.

In addition, in case of returning a block, if it waits until the completion of a transaction and returns the actual block after completing the completion log, it is possible to prevent another transaction from allocating a block returned by an incomplete transaction.

Accordingly, an object of the present invention is to solve the lock deadlock problem of the completion period lock by using the characteristics of the allocation and return operation, and to divide and manage the metadata to a plurality of allocation groups to improve the parallelism In the journaling file system that performs locks for each allocation group, the locks can be returned early without transactional consistency and resilience before transaction completion, so that lock deadlocks between allocation groups do not occur. The purpose of the present invention is to provide a parallel resource allocation and return method in a journaling file system in a multiprocessor environment that can eliminate the complexity and the additional cost.

In addition, an object of the present invention is to use a metadata partition management structure through a plurality of allocation groups, which is a structure generally employed in a journaling file system, and each transaction using characteristics of operations thereof. Changes instead of the commit duration locking, which acquires locks only on the metadata of the allocation group that actually changes when the disk resource is allocated and released, and also keeps the duration of the lock until the transaction completes. Unlocking immediately upon completion does not impede the consistency and resilience of the metadata, improving parallelism by multiple processes, and avoiding fundamentally deadlock-to-lock deadlocks that must be addressed by existing procedures. How parallel resources are allocated and returned in journaled file systems in a processor environment To provide.

In order to achieve the object of the present invention, a parallel resource allocation and return method in a multiprocessor journaling file system uses metadata divided into a plurality of allocation groups in a file system supporting journaling operating in a multiprocessor environment. In the method of allocating and returning block resources of a disk, a transaction identifier is allocated in a transaction table for a transaction start request from a plurality of transactions, and an allocation resource list, a return resource list, and a mounting buffer list included in the identifier are allocated. A transaction start processing step of initializing, a resource allocation step of allocating a resource from the allocation group for a resource allocation request for a specific allocation group from the transaction and registering an identifier of the allocated resource in the allocation resource list of the transaction table; A resource return step that registers an identifier of the resource in the return resource list of the transaction table for a request for return of a specific resource requested from a transaction and waits for the actual resource return procedure until the transaction is completed, and allocates a resource for a transaction completion request from the transaction. A transaction completion processing step of discarding the list, releasing all resources in the return resource list, releasing all buffers in the mounting buffer list, and returning the corresponding transaction identifier, and in the allocation resource list for a request to withdraw a transaction from the transaction. And returning all resources, discarding the returned resource list, releasing all buffers in the mounting buffer list, and then canceling the transaction.

5 is a block diagram of a file system to which an embodiment of the present invention is applied.

As shown in FIG. 5, a file system to which an embodiment of the present invention is applied includes a transaction manager 200 that accesses the file system, and a transaction manager 200 that is responsible for starting, completing, and withdrawing the transaction 100. ), A resource manager 300 for allocating and returning block resources of the disk, and a buffer manager 400 for providing a buffer function for reading or changing a block of the disk.

An operation of a file system to which an embodiment of the present invention configured as described above is applied will be described in detail with reference to FIGS. 6 to 13.

First, the transaction manager 200 is allocated and initialized one transaction identifier further including a resource allocation list, a resource return list, and a buffer mounting list 220 in the transaction table 210 at the start of a transaction.

In addition, the disk 500 is composed of a plurality of allocation groups 510, and each allocation group is composed of one metadata block 520 and a plurality of data blocks 530. The metadata block 520 is a bitmap indicating that all data blocks in the same allocation group are allocated. The metadata block 520 has 1 bit when allocated and 0 bits when not allocated.

6 illustrates an example of a resource allocation method from a plurality of allocation groups in a file system according to an embodiment of the present invention.

In the file system shown in FIG. 5, all resource allocations within a transaction 100 mount a metadata block 520 of the allocation group 510 to perform the allocation into a buffer and acquire an unconditional lock on it, and within this buffer. Searches the bitmap and allocates a free block to mark it, releases the lock on that buffer and continues allocation from the next allocation group. The lock deadlock does not occur because the lock on the previous allocation group is released before continuing the block allocation for the next allocation group.

7 illustrates an example of a method of returning resources from a plurality of allocation groups in a file system according to an embodiment of the present invention.

In the file system shown in FIG. 5, all resource returns in the transaction 100 do not process the return immediately for the return request, but instead register the corresponding return request in the return resource list in the corresponding transaction identifier. The registered resource return request is processed at the same time when the transaction is completed. This prevents other transactions from allocating resources returned by incomplete transactions.

8 is an exemplary view showing that lock deadlock does not occur in a file system according to an exemplary embodiment of the present invention.

In the file system illustrated in FIG. 5, a lock is acquired only when each allocation group 500 is accessed, and the lock is released before the next allocation group is accessed. Therefore, one transaction 100 is allocated to the allocation group 500 one at a time. Only locks are obtained, so no deadlock between these locks occurs.

9 illustrates a flowchart of a transaction initiation method in a file system according to an embodiment of the present invention.

When the transaction manager 200 illustrated in FIG. 5 receives a transaction start request, first, the transaction manager 200 is allocated a free transaction identifier 220 from the transaction table 210 (S301). The allocated transaction identifier further includes a return resource list, an allocation resource list, and a mounting buffer list, and initialization thereof is performed (S302 to S304). With this process complete, the transaction is ready to run.

10 is a flowchart illustrating a resource allocation method in a file system according to an embodiment of the present invention.

When the resource manager 300 receives a resource allocation request for a specific allocation group from the transaction 100 shown in FIG. 5, the resource manager 300 first loads the metadata block of the allocation group in a buffer (S401). The mounted buffer identifier is registered in the mounted buffer list in the transaction identifier (S402). The resource manager 300 requests an exclusive lock on the corresponding buffer (S403), and checks whether there is a free resource by searching the mounted metadata (S404).

If there are free resources, the resource manager 300 indicates in the metadata that the corresponding resources are allocated (S405), registers in the allocation resource list in the transaction identifier (S406), and releases the obtained buffer lock (S407). .

If there is no free resource, the resource manager 300 releases the buffer lock (S408) and releases the loaded buffer (S409).

11 is a flowchart illustrating a resource return method in a file system according to an embodiment of the present invention.

When the resource manager 300 receives a return request for a specific resource from the transaction 100 shown in FIG. 5, the resource manager 300 registers the request in the return resource list in the transaction identifier and then completes the return request. It becomes (S501).

12 is a flowchart illustrating a transaction completion method in a file system according to an embodiment of the present invention.

When the transaction manager 200 receives the completion request from the transaction 100 shown in FIG. 5, the transaction manager 200 discards the allocation resource list in the transaction identifier (S601) and checks whether there is a return request in the return resource list. (S602).

If there is, the transaction manager 200 mounts the metadata block of the allocation group to which the return resource belongs to the buffer for all return requests (S603), and acquires a lock on the mounted buffer (S604). The transaction manager 200 displays the resource return in the metadata (S605), unlocks the corresponding buffer (S606), and returns the loaded buffer (S607). If there are no more resources to return, the transaction manager 200 releases the buffer for all the buffers registered in the mounting buffer list and completes (S608).

13 illustrates a flowchart of a transaction withdrawal method in a file system according to an embodiment of the present invention.

When the transaction manager 200 receives the withdrawal request from the transaction 100 shown in FIG. 5, the transaction manager 200 discards the return resource list in the transaction identifier (S701) and checks whether there is an allocated resource in the allocated resource list. It is made (S702).

If there is, the transaction manager 200 mounts the metadata block of the allocation group to which the resource belongs to the buffer for all allocated resources (S703), and acquires a lock on the mounted buffer (S704). The transaction manager 200 displays the resource return in the metadata (S705), unlocks the corresponding buffer (S706), and returns the loaded buffer (S707). If there are no more resources allocated, the transaction manager 200 releases the buffer for all the buffers registered in the mounting buffer list and completes (S708).

As such, the present invention returns locks on individual allocation groups early before the transaction is completed, so that lock deadlocks between lock allocation groups do not occur fundamentally. It can solve the problem of complicated file system implementation through processing, etc. In particular, the parallelism can be maximized in a multiprocessor environment, and thus, excellent time delay characteristics can be obtained.

In addition, the present invention fundamentally eliminates the problem of having to do a variety of avoidance design in order to avoid this, because there is a possibility of lock deadlock between the existing methods when there are transactions that change a large number of allocation groups, in particular operating in a multi-processor environment By increasing the parallelism of the file system and eliminating the avoidance cost due to lock deadlock, the file system is simplified and the time delay is improved.

Claims (6)

In a method for allocating and returning block resources of a disk by using metadata divided into a plurality of allocation groups, A transaction start processing step of allocating one transaction identifier in a transaction table for a transaction start request from a plurality of transactions, and initializing an allocation resource list, a return resource list, and a mounting buffer list included in the identifier; A resource allocation step of allocating a resource from the allocation group for a resource allocation request for a specific allocation group from the transaction and registering an identifier of the allocated resource in the allocation resource list of the transaction table; A resource return step of registering an identifier of a corresponding resource in a return resource list of a transaction table in response to a return request of a specific resource requested from the transaction, and waiting for an actual resource return procedure until the transaction is completed; A transaction completion processing step of discarding an allocation resource list for the transaction completion request from the transaction, returning all resources in the return resource list, releasing all buffers in the mounting buffer list, and returning the corresponding transaction identifier; A transaction withdrawal step of returning all resources in the allocation resource list, discarding the return resource list, releasing all buffers in the mounting buffer list, and returning the corresponding transaction identifier for the transaction withdrawal request from the transaction. Parallel resource allocation and return method in a journaling file system of a multi-processor environment comprising a. delete The method of claim 1, wherein the transaction completion step and transaction withdrawal step, And dismounting each buffer in the mount buffer list. The method of claim 1, wherein the transaction completion step, Mounting and locking the metadata of the allocation group to which the resource belongs to the buffer for each resource in the returned resource list; A second step of recording the fact that the resource is returned in the metadata; And a third process of releasing the mounted buffer and releasing the buffer. The parallel resource allocation and return method in a journaling file system in a multiprocessor environment. The method of claim 1, wherein the transaction withdrawal step, Mounting and locking metadata of a allocation group to which a resource belongs to a buffer for each resource in the allocation resource list; Changing the metadata to return the allocated resources; And a third process of releasing the mounted buffer and releasing the buffer. The parallel resource allocation and return method in a journaling file system in a multiprocessor environment. The method of claim 1, wherein the resource allocation step, A first step of mounting and locking metadata of a corresponding allocation group in a buffer in response to a resource allocation request for a specific allocation group requested from the transaction; A second step of checking whether there is a spare resource by searching the mounted metadata; If there is a free resource, it is changed to the allocated state on the metadata, and registers the identifier of the allocated resource in the allocation resource list of the transaction table and then releases only the lock without dismounting the buffer. How to allocate and return parallel resources in a journaling file system in your environment.

Images