KR102199575B1 - Computing system and method for data consistency - Google Patents

Abstract

A buffer cache and method related to the file system and more specifically ensuring data consistency are provided. The buffer cache according to an embodiment includes a first area in which first data is cached and a second area in which second data including an eigenvalue given to the first data when the first data is generated is cached. It may include a volatile memory, and a controller that determines a location where the first data is cached based on the eigenvalue when an operation on the first data is requested from an application program.

Description

Buffer cache and method for data consistency {COMPUTING SYSTEM AND METHOD FOR DATA CONSISTENCY}

A buffer cache and method related to the file system and more specifically ensuring data consistency are provided.

Volatile RAM (RAM) used in the file system loses data when the system crashes. Therefore, guaranteeing the consistency or atomicity of the operation processing used for data update is not a big issue.

On the other hand, when a non-volatile memory (NVRAM: Non-volatile Random Access Memory) is used for data storage, data is not lost even when the system crashes. To reuse the data that has not been lost, the data is consistent or atomic. This should be guaranteed.

For example, in response to a cache flush call of a central processing unit (CPU), there is a need to maintain consistency of data stored in nonvolatile memory. Conventionally, a data structure in a nonvolatile memory is implemented as a B+ tree structure, and a logging method or entry that stores data to be newly stored in the logging area in advance and performs an operation on a cache flush call. A shadowing method of changing points after copying and updating the data contained in) was used as a solution. However, in the B+ tree structure, since the size of the key and pointer in the entry exceeds 8 bytes in which atomicity is maintained, there is a possibility that the consistency of data may be broken in the event of a sudden crash. There is a need for a new solution.

The buffer cache according to an embodiment includes a first area in which first data is cached and a second area in which second data including an eigenvalue given to the first data when the first data is generated is cached. Volatile memory; And a controller configured to determine a location at which the first data is cached based on the eigenvalue when an operation on the first data is requested from an application program.

The first region may have a 2-way structure.

The first area may be configured in chunk units, and each of the chunks may form a slot consisting of a pair of chunks according to the 2-way structure.

When receiving a request for an operation on the first data from the application program, the controller may determine the index of the slot in which the first data is cached based on the eigenvalue and the number of chunks constituting the first area. have.

The second data may further include a bit indicating a state of the first data.

The bit may include a first bit indicating validity of the first data and a second bit indicating a pre-and-after relationship of caching in two chunks of the slot.

The first bit may be changed after the operation on the first data is completed.

The bit may further include a third bit indicating a dirty state of the first data.

The controller may control the first bit to have a first logical value when the first data is valid, and the second bit to have a first logical value in the case of a cache cached later among the two chunks of the slot. .

When an update operation on the first data is requested from the application program, the controller may copy the first data to a toggle chunk of the chunk in which the first data is cached.

When receiving a request for a write operation on the first data, the controller may flush the first data through a background process and perform a next operation without waiting for a response from the storage.

When the controller receives a request for an operation on new file data in the chunk in which the first data is cached, the controller may pending the operation on the new file data until the write operation on the first data is completed. have.

The buffer cache may exist on a virtual file system layer.

When the size of the first data is larger than the size of the unit chunk, the controller determines an index area of the slot in which the first data is cached, and based on the index area, the first data is stored in a plurality of chunks. Can be cached.

When the size of the first data is larger than the size of a unit chunk and a write operation on the first data is requested, the first data is written to all of the plurality of chunks when the write operation of the first data is completed. The first data may be flushed through a background process.

The bit includes a fourth bit indicating whether the write operation of the first data is completed when the size of the first data is larger than the size of the unit chunk and a write operation is requested for the first data. I can.

When the write operation of the first data is completed in a last chunk among a plurality of chunks, the fourth bit of the last chunk may have a first logical value.

The controller may determine an offset bit of a first chunk among the plurality of chunks, and initialize the offset bit when the write operation of the first data is completed in a last chunk of the plurality of chunks.

After initializing the offset bit, the controller may change the fourth bit to a second logical value.

A method of operating a buffer cache according to an embodiment includes the steps of: receiving a request for an operation on first data from an application program; And determining a location where the first data is cached based on the eigenvalue given to the first data, wherein the first data is cached in a first area of a nonvolatile memory, and includes the eigenvalue. The second data is cached in the second area of the nonvolatile memory.

The determining of the location where the first data is cached may include the first data based on the eigenvalue and the number of chunks constituting the first area when an operation on the first data is requested from the application program. It may include determining the index of the slot to be cached.

In the method of operating a buffer cache according to an embodiment, when the first data is valid, the first bit has a first logical value, and in the case of a cache cached later among the two chunks of the slot, the second bit is a first bit. It may further include controlling to have a logical value.

In the method of operating a buffer cache according to an embodiment, when an update operation on the first data is requested from an application program, the first data is transferred to a toggle chunk of the chunk in which the first data is cached. It may further include the step of copying.

In the method of operating the buffer cache according to an embodiment, when a write operation on the first data is requested, the first data is flushed through a background process, and the next operation is performed without waiting for a response from storage. It may further include a step.

In the method of operating a buffer cache according to an embodiment, when an operation for new file data is requested in a chunk in which the first data is cached, the new file data is read until the write operation of the first data is completed. It may further include the step of pending the operation (pending).

Determining the location where the first data is cached includes determining an index area of the slot in which the first data is cached when the size of the first data is larger than the size of the unit chunk, and one implementation The method of operating the buffer cache according to the example may further include caching the first data in a plurality of chunks based on the index area.

In the step of caching the first data in a plurality of chunks, when the size of the first data is larger than the size of a unit chunk and a write operation is requested for the first data, the first data is included in all of the plurality of chunks. When the write operation of is completed, flushing the first data on the first data through a background process.

The bit includes a fourth bit indicating whether the write operation of the first data is completed when the size of the first data is larger than the size of the unit chunk and a write operation is requested for the first data. I can.

When the write operation of the first data is completed in a last chunk among a plurality of chunks, the fourth bit of the last chunk may have a first logical value.

The caching of the first data in a plurality of chunks may include determining an offset bit of a first chunk among the plurality of chunks; And when the write operation of the first data is completed in a last chunk among a plurality of chunks, initializing the offset bit.

The method of operating the buffer cache according to an embodiment may further include changing the fourth bit to a second logical value after initializing the offset bit.

1 is a diagram illustrating a file system using a persistent buffer cache (PBC) according to an embodiment.
2 is a diagram illustrating a structure of a persistent buffer cache according to an embodiment.
3 is a diagram illustrating a state of a pair of chunks in an indexed slot according to an embodiment.
4 is a diagram illustrating a case-by-case method of processing an operation using chunks and tags according to an exemplary embodiment.
FIG. 5 is a diagram for explaining that contents of a persistent buffer cache can always be recovered from an error when a system failure occurs according to an embodiment.
6 is a diagram for describing a background flush technique according to an embodiment.
FIG. 7 is a diagram for describing handling of a case in which the size of first data is larger than the size of a unit chunk according to an exemplary embodiment.
8 is a diagram illustrating a method of operating a persistent buffer cache according to an embodiment.

Specific structural or functional descriptions of the embodiments according to the concept of the present invention disclosed in this specification are exemplified only for the purpose of describing the embodiments according to the concept of the present invention, and embodiments according to the concept of the present invention They may be implemented in various forms and are not limited to the embodiments described herein.

Since the embodiments according to the concept of the present invention can apply various changes and have various forms, the embodiments will be illustrated in the drawings and described in detail herein. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosed forms, and includes changes, equivalents, or substitutes included in the spirit and scope of the present invention.

Terms such as first or second may be used to describe various elements, but the elements should not be limited by the terms. The above terms are only for the purpose of distinguishing one component from other components, for example, without departing from the scope of rights according to the concept of the present invention, the first component may be named as the second component, Similarly, the second component may also be referred to as a first component.

When a component is referred to as being “connected” or “connected” to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. Should be. On the other hand, when a component is referred to as being “directly connected” or “directly connected” to another component, it should be understood that there is no other component in the middle. Expressions that describe the relationship between components, for example, “between” and “just between” or “directly adjacent to” should be interpreted as well.

The terms used in the present specification are only used to describe specific embodiments and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as "comprise" or "have" are intended to designate that the specified features, numbers, steps, actions, components, parts, or combinations thereof exist, but one or more other features or numbers, It is to be understood that the presence or addition of steps, actions, components, parts, or combinations thereof, does not preclude the possibility of preliminary exclusion.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this specification. Does not.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The same reference numerals in each drawing indicate the same members.

1 is a diagram illustrating a file system using a persistent buffer cache (PBC) according to an exemplary embodiment.

Referring to FIG. 1, a file system according to an embodiment may use a persistent buffer cache.

Buffer caching techniques are widely used to compensate for the difference in speed between the memory layer and the storage layer in a computer system. Because reading data from storage is slower than reading data from memory, frequent readings of the same area of storage over and over for short periods of time can slow down the computer system. If the information read once from the storage is stored in memory for a considerable amount of time, the first time it is read only takes some time and the overall speed can be increased, and the memory used for this purpose may be a buffer cache. When a read or write operation occurs again for data that already exists in the buffer cache, the data can be served directly from the buffer cache to reduce the number of times the storage is accessed.

File system buffer cache can greatly contribute to improving file I/O performance by reducing the number of slow storage accesses, but since general buffer caches are outdated as volatile memory, it crashes if the data modified in the buffer cache is not reflected in the file system on the storage for a long time. When this occurs, the latest data may be lost or data consistency may be broken. Crashes can include system failures such as system crashes or abnormal shutdowns.

To prevent this, multi-versioning may be used in which modified dirty data is periodically reflected in storage. Dirty data may be data that has been updated only in the buffer cache and not in the storage. As an example of multi-versioning, there may be a journaling technique.

Through the journaling technique, data can be recorded in a separate storage area and then reflected in the original location. The journaling technique may be a technique that ensures file system consistency and high stability of data by periodically recording the facts in a specific journal space each time the file system is updated and then reflecting the updated contents to the actual file system.

In the journaling technique, file data must always be synchronized with the storage, so overhead may increase. For example, the journaling technique may have a limitation that the performance of the file system is deteriorated due to an additional recording process, and the recovery time after the system crash is lengthened as the journal space increases. If the journaling technique is used, higher reliability can be maintained, but since a large amount of additional writes are generated, in general, a journaling technique can be used only for metadata and a flush can be applied for file data. The journaling technique can significantly reduce the effectiveness of the buffer cache because a significant portion of the write operation leads to disk I/O without insufficient cache space.

The file system according to an embodiment can maintain the consistency of the file system even if a system failure occurs by using a persistent buffer cache in which a buffer cache is implemented as a nonvolatile memory. The consistency of the file system may mean consistency between buffer cache data and storage data. Data may include file data as well as metadata.

The persistent buffer cache may exist on a virtual file system layer. The virtual file system is an abstraction layer above the actual file system, allowing client applications to access multiple file systems in the same way. For example, if a virtual file system is used, a client application program may not feel the difference between a local and a network because a client application program can directly access a storage device on a network even to a local storage device. Since the persistent buffer cache exists on the virtual file system layer, data coming from the user may not ride a deep stack. Since data does not ride on a deep stack, a critical path for data to be stored to have persistence is short, and thus, it can have higher performance than a buffer cache implemented with a conventional volatile memory.

Before describing a method of operating a persistent buffer cache according to an embodiment, a general method of operating a buffer cache will be briefly described.

There may be a limit on the size of the buffer cache. Since the buffer cache is not large enough to hold all the necessary data, when the buffer cache is full, data that has not been used for a long time is discarded, and the empty space can be filled with new data. The buffer cache can use several algorithms to get the best efficiency within that limit. For example, the LRU algorithm (Least Recently Used Algorism) may be an algorithm that removes data that has not been used for the longest time. The basic hypothesis of the LRU algorithm may be that the data that has not been used for the longest time is unlikely to be used in the future.

The existing buffer cache management system may not be suitable in situations where data must be immediately and persistently guaranteed. For example, since the conventional buffer cache precisely manages inconsistencies caused by delay differences between the storage and the memory used as the buffer cache, it may be expensive to find the optimal victim space.

Since the nonvolatile memory has a short access and response speed, the existing buffer cache mechanism may be wasted in a persistent buffer cache that uses nonvolatile memory as a buffer cache. This situation can be analogous to a sophisticated sacrificial space selection mechanism in the central processing unit (CPU) cache adversely affecting the overall performance. The persistent buffer cache according to an embodiment may reduce overhead due to placement and ensure consistency by using a simple replacement algorithm. A method of operating the persistent buffer cache will be described in detail below with reference to FIGS. 2 to 7.

2 is a diagram illustrating a structure of a persistent buffer cache according to an embodiment.

Referring to FIG. 2, the persistent buffer cache 200 according to an embodiment includes a nonvolatile memory 210 and a controller 220.

Types of the nonvolatile memory 210 described in an embodiment may include, for example, a phase change RAM (PCRAM), a magnetoresistive RAM (MRAM), a ferroelectric RAM (FeRAM), or the like. However, the types of the nonvolatile memory 210 are also only some embodiments of the present invention, and the present invention should not be limitedly interpreted by these embodiments.

The nonvolatile memory 210 may include a first area in which first data is cached and a second area in which second data including a unique value given to the first data when the first data is generated is cached. The first data in the present specification may be file data, and the second data may be meta data. Meta data is secondary data for structure management for managing file data structurally existing on the disk, and may be data generated according to file creation, deletion, directory creation and deletion, increase or decrease in file size, and the like. For example, the meta data may be data including information on changes reflected in the file system. Caching may refer to the process of temporarily storing data in memory rather than storage.

The nonvolatile memory 210 may include a first region and a second region. In the present specification, the first area may include a chunk area, and the second area may include a tag area. The chunk area may be configured in chunk units. For example, a total of N consecutive chunks may exist in the chunk area, and the corresponding chunks may be a unit of load/storage into storage. The chunk region may have a 2-way structure. Each chunk may constitute a slot consisting of a pair of chunks according to a 2-way structure, which may be similar to the design of a 2-way association cache used in the central processing unit cache.

The tag area may be configured in a tag unit, and a chunk and a slot of the tag area may have a one-to-one correspondence relationship. Each of the N tags can be managed in association with one chunk. The tag can be composed of a unique value that can identify which file data is included in the corresponding chunk and a bit indicating the status of the chunk. Hereinafter, the unique value may be referred to as a key value. The key value is a number assigned by the persistent buffer cache 200 when creating a file, and may be a unique number connected to each file. The key value is a global value managed by the persistent buffer cache 200, and when all files are created from an initial value of 0, a value incremented by a predetermined stride value may be assigned to reduce collisions between files. The number of files that can be created can be limited by the type of the variable containing the key value. For example, since the variable used in the current implementation is an unsigned long, you can assign a key to a value that this type can contain.

In a situation where a file is accessed, file data may need to be fetched from the persistent buffer cache 200. In a conventional buffer cache, data could reside in a location determined by an alternate algorithm running in the cache. The actually placed blocks could be repositioned each time they were reloaded by the replacement algorithm. In contrast, in the case of the persistent buffer cache 200, a simple replacement algorithm that statically fixes the indexing system for replacement can be used. The replacement algorithm may be performed by the controller 220.

When an operation on the first data is requested from an application program, the controller 220 may determine a location in which the first data is cached based on the eigenvalue. Specifically, when receiving an operation on the first data from an application program, the controller 220 may determine the index of the slot in which the first data is cached based on the eigenvalue and the number of chunks constituting the first region. .

For example, the slot index may be calculated as in Equation 1.

a MOD b denotes a remainder obtained by dividing a and b, and N may be the number of chunks in the persistent buffer cache 200. Since the destination is determined in the persistent buffer cache 200 for all file data through Equation 1, there may be little overhead due to arrangement. For example, in the case of data having a number of chunks of 20 and a key value of 5, a caching position may be determined as slot 5 by Equation 1. In the case of data having a number of chunks of 20 and a key value of 7, a caching location may be determined as slot 7. In the case of data having a number of chunks of 20 and a key value of 15, a caching location may be determined as slot 5.

In addition, since the destination is determined in the persistent buffer cache 200 for all file data through Equation 1, when the persistent buffer cache 200 is full and the existing file data is replaced with new file data, the existing file data to be replaced You may not even need a replacement algorithm to find the file data. For example, when file data with key values 5 and 15 is cached in slot 5, and when file data with key value 25 is loaded, file data with key value 25 is slot 5 by Equation 1 Since the destination is determined by the method, a complex replacement algorithm such as the existing LRU algorithm may not be required.

The chunk replacement indexing technique of the persistent buffer cache 200 may have disadvantages compared to replacement indexing techniques such as the existing LRU algorithm. For example, according to the chunk replacement indexing technique of the persistent buffer cache 200, since frequently used file data and infrequently used file data are unconditionally and sequentially replaced, the situation of a cache miss where there is no desired data in the chunk Occurrence may occur frequently, and this may result in frequent flushing. Flush can refer to the process of exporting data from the buffer cache to storage. If the chunk replacement indexing technique of the persistent buffer cache 200 is followed, overhead may increase due to frequent flushing.

At first glance, the chunk replacement indexing technique of the persistent buffer cache 200 may cause frequent flushing, but the emergence of recent storage devices with shorter wait times causes the existing I/O path to affect the overall system performance. It can be harmful. When using nonvolatile memory with very low latency, I/O-bound applications can be regarded as central processing unit-bound applications, as well as a small number of instructions in the execution critical path. The increase can continue to accumulate and adversely affect system performance. In a medium having a very short latency, such as a nonvolatile memory, instead of a complex replacement algorithm, a simple chunk replacement indexing technique of the persistent buffer cache 200 may be effective. In addition, a method for compensating for frequent flushing of the persistent buffer cache 200 will be described in detail below.

3 is a diagram illustrating a state of a pair of chunks in an indexed slot according to an embodiment.

The tag can be composed of a unique value that can identify which file data is included in the corresponding chunk and a bit indicating the status of the chunk. The bit indicating the status of the chunk may include a first bit indicating whether the file data is valid, and a second bit indicating a pre-and-after relationship of caching in two chunks of the slot. When the file data is valid, the first bit may have a first logical value, and in the case of a cache cached later of the two chunks of the slot, the second bit may have a first logical value. For example, if the file data is valid, it may have a '1' bit, and if the file data is not valid, it may have a '0' bit. In addition, among the two chunks of the slot, a later cached chunk may have a '1' bit, and a first cached chunk may have a '0' bit.

Referring to FIG. 3, a pair of chunks according to an embodiment may be in one of three states.

State 310 can be represented by S[C(0, 0), C(0, 0)], in which case it can be seen that the slot is empty because neither chunk is valid. Data may be empty or invalid in all chunks of that slot.

State 320 may be represented by S[C(1, 1), C(0, 0)], and in this case, it can be seen that only one chunk has valid data.

State 330 can be represented as S[C(1, 1), C(1, 0)], in which case both chunks have valid data, and the chunk in which the file data'L' is cached is the chunk cached later It can be seen that

The bit may further include a third bit indicating a dirty state of the first data. When the first data is valid, the first bit may have a first logical value, and in the case of a cache cached later of the two chunks of the slot, the second bit may have a first logical value.

4 is a diagram illustrating a case-by-case method of processing an operation using chunks and tags according to an exemplary embodiment.

Referring to FIG. 4, a method of processing an operation according to an exemplary embodiment may be divided into six cases and described.

The case 410 may have an initial state when both chunks of the slot are not valid ([C(0, 0), C(0, 0)]). In this case, any chunk can be used in the slot. When requesting an update write operation for K (Write(K')), if file data for complete K exists in the storage, the data may be first loaded into the persistent buffer cache 200. Thereafter, K'is written, and ordering and durability of data writing to the persistent buffer cache 200 may be guaranteed through clush and sfence commands. Next, the tag C(0, 0) is modified to C(1, 1), and the request can be completed after all are set. In order to ensure the persistence of writing to the persistent buffer cache 200, a technique using clush and sfence may have to be performed every time a write occurs. Hereinafter, descriptions of overlapping contents are omitted. It also implies that modifying the flag bits of the tag without further mention is performed atomically. Performing atomically means that the first bit is changed after actually all operations are completed, and will be described in detail with reference to FIG. 5.

Case 420 and case 430 have an initial state in which one chunk has a valid object K, and the other chunk is not valid (S[C(1, 1), C(0, 0)]) I can. Two cases can be considered in this situation. The case 420 may be a case where K is modified, and the case 430 may be a case where a new object is written.

In the case of case 420, when K is updated (Write(K')), K is still valid, but the second bit representing the most recent chunk can be cleared (C(1, 1) -> C( 1, 0)). After clearing the second bit, K is copied to the toggle chunk in the state C(0, 0), and the update request can be reflected in the chunk. Toggle chunks can mean chunks that are not themselves in the same slot. The state of the tag is modified from C(0, 0) to C(1, 1), and finally, the state of the original chunk C(1, 0) can be modified to C(0, 0). When such a series of procedures are completed, the application program may receive that the Write(K') request has been completed. When an update request Write(K') occurs for valid data K, the valid K can be moved to the toggle chunk and then K'can be overwritten there. This is because the update request may be a partial write including K. In this case, if writing is performed without copying to the toggle chunk, it may become unrecoverable when the write fails. In the case of case 420, since a 2-way chunk structure is used, when K is updated, a toggle chunk exists and can be copied to the toggle chunk, but when using a 1-way chunk, K is flushed because there is no toggle chunk. And you may need to update K'with a new one. As in the case 420, when the toggle chunk is empty and the file data is updated, the flush is reduced by using a 2-way chunk structure, thereby compensating for the reduced efficiency by using the chunk replacement indexing technique of the persistent buffer cache 200. .

In the case of the case 430, the object L'may be recorded in a slot including a chunk in which the object K exists. Since the chunk in which K exists is no longer the recently accessed chunk, C(1, 1) can be modified first to C(1, 0). If the write to the L'object is overwrite, the L'object may be overwritten in the corresponding chunk after loading L from storage first. The state at this time is C(0, 0), and after overwriting is completed, C(0, 0) can be modified to C(1, 1).

Cases 440, 450, and 460 may be in a situation where the initial state is when two chunks in the slot are valid, that is, S[C(1, 1), C(1, 0)]. Assuming that the slot includes K and L, referring to the second bit, it can be seen that the chunk including L is a cached more recently.

Cases 440 and 450 may indicate when an update request for an object K or L is performed. In the case of cases 440 and 450, a chunk containing an object that is not updated may be first flushed to storage.

In the case of the case 440, the chunk containing L is flushed to the storage, and a signal indicating that the flush is completed may be waited for. After the completion of reception is confirmed, the state C(1, 1) is changed to C(0, 0), and the situation at this time can be the same as that of the first stage of case 420, and proceeds from thereafter. The process may be performed in the same manner as the case 420.

In case 450 in which an update request for object L is performed, the chunk containing K is first flushed, and after completion reception is confirmed, the state C(1, 0) of the corresponding chunk is C(0, 0) It may be changed to, and the situation at this time may also be the same as the state of the first stage of the case 420. Even after this process, similarly, it may proceed to the process after the case 420.

The case 460 may be when a write or update operation is requested for an object that does not exist in the persistent buffer cache 200 (Write (M')). As in the case of cases 440 and 450, one of the two chunks should be flushed, but the previously written chunk may be flushed. In the case of the case 460, the chunk including the K object may be flushed. When the flushing is confirmed, the state C(1, 0) is changed to C(0, 0), the initial state of the case 430 becomes, and the next step may be the same as the subsequent process of the case 430. .

The flush operation can occur when a write operation is requested, and because it occurs synchronously, performance overhead can occur. A background flush technique may be used to reduce overhead, and a description thereof will be described in detail below.

The read operation process can be similar to the write process. To find the slot to be read, the key value can be used the same as when writing. When there is data to be read in the chunk of the corresponding slot, the data can be read from the persistent buffer cache 200. If there is a cache miss situation where there is no desired data in the chunk, the data can be loaded and read from storage. The chunk selected as a space to contain the data to be loaded is first, an invalid chunk is used, and if there is no invalid chunk, an old chunk can be selected. When a chunk is selected, data originally in the chunk is flushed, data is loaded there, and finally, the data can be read into the user's user buffer. During the read operation, the tag value is also changed atomically, and similarly, clush and sfence commands can be used for persistency.

FIG. 5 is a diagram for explaining that contents of a persistent buffer cache can always be recovered from an error when a system failure occurs according to an embodiment.

Referring to FIG. 5, the persistent buffer cache 200 according to an exemplary embodiment may be always recoverable except when the contents of the persistent buffer cache 200 itself are damaged due to a failure of the media itself. The persistent buffer cache 200 can always return to a consistent state even if an allowable failure occurs in any case.

Since the first bit indicating whether the file data is valid or not is changed after the state change occurs, the state of the tag value is changed to one of the initial states of each case even if an error occurs at any stage of the cases (410 to 460). Can be. The write request itself has not been completed, but no matter what state it stops, the state C value is intact, and this state becomes the initial value of a specific case immediately after the system is restored, so there is no problem in the consistency of the file system.

For example, it may be assumed that a failure occurs in step 1 520 of the case 450. If a failure occurs while the chunk containing K is being flushed, the flushing fails, but the state may be the same as S[C(1, 0), C(1, 1)], which is the initial state 510 of the case 450 have. If the flush is completed, but a failure occurs before receiving completion, it can be recovered by the background flush technique. The background flush technique is described in detail with reference to FIG. 6 below. The K in the persistent buffer cache 200 chunk may simply be a copy of the data in the storage device. If the flush is not completed, the existing file system simply deletes the write request as failed, but the K object in the persistent buffer cache 200 chunk may still be in an active state. In both cases, the task can be restarted from the initial state. When the user recognizes that the request for K has failed and executes the task again, flushing occurs again.In this case, it is possible to overwrite what has already been written from the point of view of the storage device or to write a new data space that is not present. .

After receipt of the flush completion confirmation, a failure may occur before the state C(1, 0) changes to C(0, 0) between, for example, step 1 520 and step 2 530. The case may also be the same as the situation described above. If an error occurs after step 2 530, the state of the slot becomes S[C(1, 1), C(0, 0)], which may be the same as the initial state of the case 420. I can. In addition, since the process of performing the second step 530 is guaranteed to have atomicity, an error cannot occur during the execution.

In the case of the state S[C(1, 0), C(0, 0)] that can occur in cases 420, 430, 440, the state of the tag value may be different from the initial state of each case. In this case, since one state is C(0, 0), the valid states of the states C(1, 0) and C(1, 1) can be considered to be actually the same. When state C(1, 0) is regarded as C(1, 1), the initial state of cases 420 and 430 may be the same.

Explaining the 4th step of case 420, this situation may be substantially the same as the initial state of cases 440, 450, 460, but chunks refer to the same block of the same file, and one of the previous You are referencing data, and the other may refer to the most recent data. During recovery, old data should be invalid, but it may remain in a valid state of C(1, 0). Even in this case, since the flush for K'will always occur before K according to the algorithm, it may not be a problem in terms of consistency. Due to the characteristics of the persistent buffer cache 300 that guarantees the consistency of the file system in all situations, a recovery mechanism that must be additionally performed for recovery may not be required.

The persistent buffer cache 200 may be a buffer cache that replaces a buffer cache generally used in a file system. Except for read and write requests (in the case of file deletion, the tag contents of the persistent buffer cache 200 are invalidated), the consistency mechanism provided by the file system can be used as it is for the consistency problem that occurs in the part that interacts with the file system. . It is possible to guarantee a consistency level that can be provided by additionally cooperating with the consistency mechanism provided by the file system to store data in the persistent buffer cache 200. For example, the Ext4 file system to which the persistent buffer cache 200 is applied can basically provide ordered mode journaling that guarantees the consistency of metadata, and if the persistent buffer cache 200 is additionally disposed, the persistent buffer cache Since data residing in (200) can also be preserved at an appropriate level, consistency can be guaranteed similar to data mode journaling, which provides the highest level of consistency in the existing Ext4 file system.

Considering the location where the consistency of the file system may be broken in the existing file system, the existing volatile buffer cache is only used for improving performance, and data stored therein may not be preserved at all. Although the data stored in the buffer cache is synchronized to the storage device according to an explicit command or an appropriate period, a consistency inconsistency problem may occur before and during synchronization. Although journaling is used to preserve the consistency of the file system, intrinsically, the time interval at which volatile information can be lost can be important to maintain consistency. If the persistent buffer cache 200 is used, the amount of data that may be lost may be significantly reduced. All records are immediately written to the persistent buffer cache 200, and since they are permanently stored in a non-volatile medium, they can be maintained in case of failure. Although data in a valid state of the persistent buffer cache 200 may not be synchronized to the storage device, all reads and writes of the latest data in the persistent buffer cache 200 are processed only by the persistent buffer cache 200, and eventually Since the persistent buffer cache 200 is synchronized to the storage device through flushing, it may not be a problem. In addition, since the metadata is also adjusted when data is flushed to the file system, the consistency of the metadata is guaranteed by the consistency mechanism of the lower file system, and additionally, the data stored in the persistent buffer cache 200 is relatively High-level data consistency can be guaranteed.

6 is a diagram for describing a background flush technique according to an embodiment.

According to the chunk replacement indexing technique of the persistent buffer cache 200, since frequently used file data and infrequently used file data are unconditionally and sequentially replaced without distinction, efficiency may decrease. For example, if a cache miss situation occurs where there is no desired data in the chunk, the data in the cache may need to be flushed and then loaded from storage and read. The cache miss rate may be (1-cache hit rate), and the cache hit rate may be (number of hits to the cache/number of total storage accesses). When a cache miss occurs, flushing occurs, and the flush operation may incur performance overhead. To reduce the overhead, you can use a background flush technique.

When using a conventional nonvolatile memory, data can be written to the storage at the same time as the data is written to the cache using a write through technique. In the case of using the write-through technique, the next operation can be performed after receiving a write completion signal from the storage. According to the write-through technique, performance may be degraded because it takes time to wait for the operation of writing data to the slow storage to complete.

Referring to FIG. 6, according to the background flush method according to an embodiment, when a write operation on first data is requested, the controller flushes the first data through a background process, and sends a response from the storage. You can proceed to the next operation without waiting. For example, the controller may receive a request for a write operation on data A, flush data A through a background process, and perform a read operation on data B before receiving a write completion signal from the storage. Background processes can be run using, for example, workqueue api. Since the persistent buffer cache 200 uses a nonvolatile memory, even if the next operation is performed without waiting for a response from the storage, even if the system crashes, there may be no problem in recovery.

Overhead due to a cache miss occurring due to the structure of the persistent buffer cache 200 may be compensated by a background flush technique. When a cache miss situation occurs where there is no desired data, the data in the cache may need to be flushed and then loaded from storage and read. In the case of using the background flush technique according to an embodiment, data is immediately flushed through a background process, so if a cache miss occurs, the data need only be loaded without flushing, so that the overhead due to flushing can be reduced. have.

In general, the controller can proceed with the next operation without waiting for a response from the storage according to the background flush technique, but when an operation for new file data is requested in the chunk in which the first data is cached, the write operation of the first data is completed. The operation on new file data can be pending until it becomes available. For example, case 460 of FIG. 4 is a case where an operation for new M data is requested in a chunk in which K data is cached, and a write operation for M data is performed without waiting for a response from the storage to flush the K data in the background. If you proceed, there may be problems in recovery in case of system failure. When an operation on new file data is requested in a cached chunk of the first data, the operation on the new file data must be held until a write operation on the first data is completed to maintain consistency.

FIG. 7 is a diagram for explaining handling when the size of first data is larger than the size of a unit chunk according to an exemplary embodiment.

Referring to FIG. 7, when the size of 1 data according to an embodiment is larger than the size of a unit chunk, the first data may be cached in a plurality of chunks. A chunk to be cached first among a plurality of chunks may be determined based on the key value. The required number of chunks can be determined based on the size of the file data. For example, when the chunk size is 4 KB and the file data size is 12 KB, three chunks, the second chunk, and the fourth chunk may be determined to cache the file data. Hereinafter, for convenience of description, an example in which first data is cached in chunks 2 to 4 will be described. In addition, although the chunk according to an embodiment is configured as a 2-way, for convenience of description, only one of the 2-way chunks is illustrated in FIG. 7.

When an update occurs in multiple chunks, a fourth bit for indicating a state when the update is performed for each chunk may be added to the tag. The fourth bit may be expressed as C (Complete). A tag according to an embodiment has a size of 1 byte, a first bit (V) indicating whether the first data is valid, a second bit (N) indicating a pre-and-after relationship of caching in two chunks of a slot, and dirty. A third bit (D) indicating a dirty state, a fourth bit (C) indicating a state when updated for each chunk, and four offsets indicating the offset of the chunk to which the last data will be written according to the updated size May contain bits.

Referring to drawing 710, after 4KB of 12KB of data is written to the second chunk, the offset of the fourth chunk where 8KB to 12KB of data will be written can be displayed as '0010' in the offset bit area of the tag. have. The offset can be calculated as the difference between the position of the last chunk and the position of the first chunk.

According to the mechanism when data is written or updated in a single chunk, background flush occurs immediately after data is written to the chunk, and the data is transferred to the storage device below.If data is written to multiple chunks, it can be pending. have. The reason for pending the background flush may be to prevent a situation in which, if a system failure occurs before all 12 KB of data is written, data newly written to the chunk before that is partially applied to the storage device through background flush.

Referring to the drawing 720, the background flush may be pending after data of 4 KB to 8 KB is written to the third chunk, similarly to the second chunk. Also, no change may be made to the tag for the 3rd chunk.

Referring to FIG. 730, the background flush may be pending after the remaining 4KB of data is written to the fourth chunk. When a write operation of the first data is completed in the last chunk among the plurality of chunks, the fourth bit of the last chunk may have a first logical value. For example, after the remaining 4KB of data is written to the 4th chunk, the 4th bit (C bit) of the tag for the 4th chunk may be marked as '1'. The fact that the fourth bit has a first logical value may mean that the data has been completely written to the chunk. After that, the offset bit '0010' displayed in the second chunk can be initialized again. The reason for initializing the offset bit of chunk 2 is that in the recovery process in case of system failure, it traverses the chunks to determine the example cases as in the present, and finds the chunks whose offset bit in front of the tag is not initialized to '0000'. By initializing write and update actions that have been completely executed, unnecessary inspection during the recovery process can be avoided. Next, after changing the fourth bit (C bit) of the 4th chunk to '0', background flushes of the previously pending chunks 2-4 may be released.

According to the method for caching multiple chunks according to an embodiment, a recovery process in the event of a failure may be very simple. If a system failure occurs while data is being updated in multiple chunks, the offset of the last chunk to be written will be written in the tag of the chunk where the first write is started through the runtime process described above. Since the corresponding update action has not been completed in the tag, the fourth bit ('C' bit) will be marked as '0'. Finding the area marked in this state is all of the performance load that can occur during the recovery process, and the system consistency can be guaranteed by initializing all the tags of the chunks in the area where this state occurs to 0. The reason for making this possible is that if all updates are not fully reflected in the chunk in the runtime process, partial updates will not be applied to the storage device below by pending background flush.

8 is a diagram illustrating a method of operating a persistent buffer cache according to an embodiment. Referring to FIG. 8, steps 810 and 820 may be operated by the persistent buffer cache 200.

In step 810, the persistent buffer cache 200 receives a request for an operation on the first data from the application program.

In step 820, the persistent buffer cache 200 determines the location where the first data is cached based on the eigenvalue assigned to the first data.

The first data is cached in a first area of the nonvolatile memory, and second data including an eigenvalue is cached in a second area of the nonvolatile memory.

The embodiments described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices, methods, and components described in the embodiments include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions, such as one or more general purpose computers or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to behave as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodyed in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of the program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

As described above, although the embodiments have been described by the limited drawings, a person of ordinary skill in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order from the described method, and/or components such as a system, structure, device, circuit, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and claims and equivalents fall within the scope of the claims to be described later.

Claims (37)

A nonvolatile memory including a first area in which first data is cached and a second area in which second data including a eigenvalue given to the first data when the first data is generated is cached; And
When receiving a request for an operation on the first data from an application program, a controller that determines a location where the first data is cached based on the eigenvalue
Including,
The second data further includes a bit indicating a state of the first data, wherein the bit indicates a first bit indicating validity of the first data and a first bit indicating a precedence relationship between caching in two chunks of the slot. Including 2 bits, wherein the bit indicates whether the write operation of the first data is completed when the size of the first data is larger than the size of the unit chunk and a write operation is requested for the first data. Buffer cache containing the fourth bit.
The method of claim 1,
The first area has a 2-way structure, the buffer cache.
The method of claim 2,
The first area is configured in chunk units, and each of the chunks constitutes a slot consisting of a pair of chunks according to the 2-way structure.
The method of claim 3,
The controller is
When receiving a request for an operation on the first data from the application program, the buffer cache determines the index of the slot in which the first data is cached based on the eigenvalue and the number of chunks constituting the first area .
delete delete The method of claim 1,
The first bit is changed after the operation on the first data is completed, the buffer cache
The method of claim 1,
Wherein the bit further comprises a third bit indicating a dirty state of the first data.
The method of claim 1,
The controller is
The buffer cache, wherein the first bit has a first logical value when the first data is valid, and the second bit has a first logical value in the case of a cached later of the two chunks of the slot.
The method of claim 1,
The controller is
When an update operation on the first data is requested from the application program, the first data is copied to a toggle chunk of the chunk in which the first data is cached.
The method of claim 1,
The controller is
When a write operation on the first data is requested, the first data is flushed through a background process, and the next operation is performed without waiting for a response from the storage.
The method of claim 11,
The controller is
When an operation on new file data is requested in the cached chunk of the first data, the operation on the new file data is pending until the write operation on the first data is completed.
The method of claim 1,
The buffer cache resides on a virtual file system layer.
The method of claim 1,
The controller is
When the size of the first data is larger than the size of the unit chunk, an index area of a slot in which the first data is cached is determined,
Caching the first data in a plurality of chunks based on the index area.
The method of claim 14,
The controller is
When the size of the first data is larger than the size of a unit chunk and a write operation on the first data is requested, when the write operation of the first data is completed in all of the plurality of chunks, the first data is 1 Buffer cache, flushing data.
delete The method of claim 1,
When the write operation of the first data is completed in a last chunk among a plurality of chunks, the fourth bit of the last chunk has a first logical value.
The method of claim 14,
The controller is
Determining an offset bit of the first chunk among the plurality of chunks,
When a write operation of the first data is completed in a last chunk among a plurality of chunks, the offset bit is initialized.
The method of claim 18,
The controller is
After initializing the offset bit, a buffer cache for changing a fourth bit to a second logical value.
Receiving a request for an operation on the first data from an application program;
Determining a location where the first data is cached based on an eigenvalue given to the first data
Including,
The first data is cached in a first area of the nonvolatile memory, and second data including the eigenvalue is cached in a second area of the nonvolatile memory,
The determining of a location where the first data is cached includes determining an index area of a slot in which the first data is cached when the size of the first data is larger than the size of a unit chunk,
Caching the first data in a plurality of chunks based on the index area, further comprising:
The caching of the first data in a plurality of chunks may include determining an offset bit of a first chunk among the plurality of chunks; And initializing the offset bit when the write operation of the first data is completed in a last chunk among a plurality of chunks.
The method of claim 20,
The first region has a 2-way structure, a method of operating a buffer cache.
The method of claim 21,
The first area is configured in a chunk unit, and each of the chunks constitutes a slot consisting of a pair of chunks according to the 2-way structure.
The method of claim 22,
The step of determining the location where the first data is cached
When receiving an operation request for the first data from the application program, determining an index of the slot in which the first data is cached based on the eigenvalue and the number of chunks constituting the first area
Including a method of operating the buffer cache.
The method of claim 20,
The second data further includes a bit indicating a state of the first data.
The method of claim 24,
The bit includes a first bit indicating validity of the first data and a second bit indicating a pre-and-after relationship of caching in two chunks of the slot.
The method of claim 25,
The first bit is changed after an operation on the first data is completed.
The method of claim 25,
Controlling the first bit to have a first logical value when the first data is valid, and the second bit to have a first logical value in the case of a cached later of the two chunks of the slot
The method of operation of the buffer cache further comprising.
The method of claim 20,
When an update operation on the first data is requested from an application program, copying the first data to a toggle chunk of the chunk in which the first data is cached
The method of operation of the buffer cache further comprising.
The method of claim 20,
When receiving a request for a write operation on the first data, flushing the first data through a background process and performing a next operation without waiting for a response from the storage
The method of operation of the buffer cache further comprising.
The method of claim 29,
When an operation on new file data is requested in the cached chunk of the first data, pending an operation on the new file data until the write operation on the first data is completed.
The method of operation of the buffer cache further comprising.
delete The method of claim 20,
Caching the first data in a plurality of chunks
When the size of the first data is larger than the size of a unit chunk and a write operation on the first data is requested, when the write operation of the first data is completed in all of the plurality of chunks, the first data is 1 Steps to flush the data
Including, a method of operating a buffer cache.
The method of claim 25,
The bit includes a fourth bit indicating whether the write operation of the first data is completed when the size of the first data is larger than the size of the unit chunk and a write operation is requested for the first data. , How the buffer cache works.
The method of claim 33,
When the write operation of the first data is completed in a last chunk among a plurality of chunks, the fourth bit of the last chunk has a first logical value.
delete The method of claim 20,
After initializing the offset bit, changing the fourth bit to a second logical value
The method of operation of the buffer cache further comprising.
A computer program stored in a computer-readable recording medium for executing the method of any one of claims 20 to 30, 32 to 34, and 36 in combination with hardware.

Images