WO2022259493A1

WO2022259493A1 - Journal log control system, journal log control method, and journal log control program

Info

Publication number: WO2022259493A1
Application number: PCT/JP2021/022211
Authority: WO
Inventors: 孝治佐藤
Original assignee: 日本電信電話株式会社
Priority date: 2021-06-10
Filing date: 2021-06-10
Publication date: 2022-12-15

Abstract

A persistent memory (1200) stores data, metadata, and a journal log that is an update history of the data and metadata. If a write request is issued, a journal log control unit (1110) controls the execution of a journaling process for storing an update history of data and metadata in association with the write request in the journal log, a commit process for committing the update history stored in the journal log, and a checkpoint process for executing either an immediate checkpoint process that causes the committed update history stored in the journal log to be reflected in the data and metadata, or a delayed checkpoint process that delays the immediate checkpoint process.

Description

JOURNAL LOG CONTROL SYSTEM, JOURNAL LOG CONTROL METHOD AND JOURNAL LOG CONTROL PROGRAM

The present invention relates to a journal log control system, a journal log control method, and a journal log control program.

In recent years, persistent memory has been put into practical use. Persistent memory is fast, non-volatile, and byte-addressable. Examples of persistent memory technologies include Phase Change Memory (PCM), Spin Transfer Torque Magnetic Random Access Memory (STT-MRAM), Resistive Random Access Memory (ReRAM) )and so on. Examples of persistent memory products include Non-Volatile Dual In-line Memory Module (NVDIMM) and Intel(R) Optane(TM) Persistent Memory (Non-Patent Document 7).

Conventional file systems are based on the premise of using a hard disk drive (hereinafter referred to as HDD) or a solid state drive (hereinafter referred to as SSD). It is not suitable for using memory.Therefore, file systems for persistent memory have been proposed.Examples of file systems for persistent memory include PMFS (see Non-Patent Document 1), BPFS (see Non-Patent Document 2), NOVA (see Non-Patent Document 3), Shortcut-JFS (see Non-Patent Document 4), Ext4 (see Non-Patent Document 5), XFS (see Non-Patent Document 6), and the like.

The file system for persistent memory mentioned above uses one or more of journaling, copy-on-write, and log structuring to ensure the consistency of file system data and metadata.

Journaling writes updates to the journal log before updating the data. After the update has been written to the journal log, a process called checkpoint is executed to reflect the update in the file system itself. In the event of a failure, the journal log can be used to restore the file system to a consistent state.

With copy-on-write, the block to be updated is copied to a newly allocated block, and the copied block is updated. Since the file system is never overwritten, the integrity of the file system can always be ensured.

In log structuring, the data structure of the file system is saved as a continuous log. Filesystem updates are not overwritten, but appended to the logical end of the log, ensuring filesystem integrity at all times.

When a file system for persistent memory writes to persistent memory, if the cache of the Central Processing Unit (hereinafter referred to as CPU) (hereinafter referred to as CPU cache) is a write-back method, even if an instruction to write data to persistent memory is executed, Data is only written to the CPU cache and not to persistent memory. If the server fails due to a state that has been written to the CPU cache but not to persistent memory, the data written to the CPU cache will be lost. To ensure that it writes to persistent memory, a persistent memory-oriented file system either executes instructions to flush the CPU cache after writing, or writes directly to persistent memory, or automatically flushes the CPU cache on failure. It is necessary to use a server equipped with Enhanced Asynchronous DRAM Refresh (hereafter, eADR) (see Non-Patent Document 8) for flashing dynamically. In addition, since recent CPUs sometimes change the execution order of writing to memory in order to improve performance, it is necessary to execute a memory barrier instruction to guarantee the execution order of writing to persistent memory.

JP 2019-003288 A

File system consistency can be ensured by using journaling, copy-on-write, log structuring, or multiple technologies. However, with journaling, updates must be written twice, once in the journal log and reflected in the file system itself, and with copy-on-write and log structuring, cascading updates from the updated data block to the file system root is required, there is a problem that the amount of writing increases and the writing performance of the file system decreases.

Also, when writing to persistent memory, it is necessary to execute flush instructions and memory barrier instructions for the write area, but since flush instructions and memory barrier instructions are time-consuming processes, the write performance of the file system decreases. There is a problem that Using a server with eADR eliminates the need to execute flush instructions, but not all servers with persistent memory have eADR available, and using a server with eADR does not. , the memory barrier instruction must be executed.

In order to solve the above-described problems, the journal log control system of the present invention provides data, metadata, and a journal log, which is an update history of the data and the metadata, stored in a persistent memory. A journal log control system for controlling a journal log, comprising journaling processing for storing update history of data and metadata associated with the write request in the journal log when a write request is made; and storing in the journal log. a commit process for confirming the updated history, an immediate checkpoint process for reflecting the confirmed update history stored in the journal log in the data and the metadata, and a delayed checkpoint process for delaying the immediate checkpoint process. and a journal log control unit for controlling the execution of checkpoint processing.

According to the present invention, journaling is used to ensure the consistency of both file system data and metadata in the event of a failure, reduce the amount of writes associated with journaling, and use flush commands and memory barriers. By reducing the number of command executions, it is possible to improve the write performance of the file system.

FIG. 1 is a diagram showing an example of the configuration of the entire system to which the present invention is applied in the first embodiment. FIG. 2 is a diagram illustrating an example of a journal log configuration according to the first embodiment. FIG. 3 is a diagram showing an example of the configuration of log entries according to the first embodiment. FIG. 4 is a diagram showing an example of the structure of a data write log entry according to the first embodiment. FIG. 5 is a diagram illustrating an example of the configuration of a log page end log entry according to the first embodiment. FIG. 6 is a flowchart illustrating an example of write processing operations in the first embodiment. FIG. 7 is a flowchart illustrating an example of journaling processing operations in the first embodiment. FIG. 8 is a flowchart illustrating an example of the operation of data write log entry addition processing according to the first embodiment. FIG. 9 is a flow chart showing an example of the operation of the data additional writing process according to the first embodiment. FIG. 10 is a flowchart illustrating an example of the operation of log entry area allocation processing according to the first embodiment. FIG. 11 is a flowchart illustrating an example of the operation of log page end log entry addition processing according to the first embodiment. FIG. 12 is a flowchart illustrating an example of commit processing operations in the first embodiment. FIG. 13 is a flowchart illustrating an example of checkpoint processing operations in the first embodiment. FIG. 14 is a flow chart showing an example of the operation of checkpoint processing for data write log entries in the first embodiment. FIG. 15 is a flowchart illustrating an example of the operation of immediate checkpoint processing of data write log entries in the first embodiment. FIG. 16 is a flow chart showing an example of operation of delayed checkpoint processing of data write log entries in the first embodiment. FIG. 17 is a flowchart illustrating an example of operation of insert pointer update processing according to the first embodiment.

Embodiments of a journal log control system, a journal log control method, and a journal log control program according to the present invention will be described below with reference to the accompanying drawings. In addition, this invention is not limited by this embodiment.

[Example of overall system configuration to which the present invention is applied in the first embodiment]
FIG. 1 is a diagram showing an example of the configuration of the entire system to which the present invention is applied in the first embodiment. As shown in FIG. 1, a system 1000 embodying the present invention has a file system 1100 and persistent memory 1200 . The file system 1100 is implemented, for example, by the CPU executing processes according to a given program. Also, the journal log control system of the present invention includes at least the journal log control unit 1110 .

The file system 1100 is composed of a journal log control unit 1110 and a free space management unit 1120. The journal log control unit 1110 writes updates to the journal log in response to requests from users such as file writing and file attribute changes, commits processing to make permanent and confirm the update contents in the journal log, and and a checkpoint process for reflecting the update contents of the file system body. The free space management unit 1120 executes allocation processing and release processing of the permanent memory 1200 . The area of the persistent memory 1200 is divided into units of a certain size called pages (hereinafter referred to as page size) and managed. The free area management unit 1120 allocates and releases areas in the persistent memory 1200 in units of pages. Note that FIG. 1 shows only the components necessary for explaining the first embodiment of the present invention, and the file system 1100 may have other components.

In conventional journaling file systems for HDDs and SSDs, it is common to statically allocate contiguous areas on HDDs or SSDs as journal logs when the file system is created. On the other hand, since persistent memory can be accessed at high speed compared to HDD and SSD, in the first embodiment of the present invention, pages in the persistent memory area are dynamically allocated and released, and the journal log is It should be configured as a linked list of pages. Each page that constitutes the journal log (hereinafter referred to as a log page) holds a log entry indicating update content. It should be noted that a specific area of persistent memory may be statically allocated as a journal log when a file system is created, like a conventional journaling file system on an HDD or SSD.

FIG. 2 is a diagram showing an example of the structure of a journal log in the first embodiment. In the example of FIG. 2, the journal log 2000 consists of two log page lists 2100, 2200. The journal log control unit 1110 alternately uses the log page lists 2100 and 2200 for each update process such as file writing and file attribute change. The journal log control unit 1110 alternates between the log page lists 2100 and 2200 for each update process, such as using the log page list 2100 in the first write process and using the log page list 2200 in the next write process. used for In the example of FIG. 2, the log page list 2100 shows the state used in the previous update process, and the log page list 2200 shows the state used in the currently executing update process.

Log page lists 2100 and 2200 are linked lists of log pages. In the example of FIG. 2, the log page list 2100 is made up of log pages 2110, 2120, and 2130, and the log page list 2200 is made up of log pages 2210, 2220, and 2230. FIG. At the end of the log pages 2110 and 2120 other than the last log page in the log page list 2100 and at the end of the log pages 2210 and 2220 other than the last log page in the log page list 2200, the address of the next log page is held. do. At the end of the last log page 2130 of the log page list 2100 and at the end of the last log page 2230 of the log page list 2200, a null indicating that the next log page does not exist is held. The log page holds log entries that describe updates such as file writes and file attribute changes.

The journal log 2000 manages log page lists 2100 and 2200 using

start pointers

2300 and 2400, end pointer 2500 and insertion pointer 2600. The

start point pointers

2300, 2400 and the end point pointer 2500 are held on permanent memory. Alternatively, the insert pointer 2600 may be held in volatile memory.

The initial values of the

start pointers

2300, 2400, the end pointer 2500, and the insertion pointer 2600 are null, and when log entries are added to the journal log 2000, log pages are dynamically allocated as needed, as described above.

A starting point pointer 2300 indicates the top position of the log page list 2100 , and a starting point pointer 2400 indicates the top position of the log page list 2200 . The end point pointer 2500 indicates the position of the log entry for which commit processing has been completed in the log page list. In the example of FIG. 2, log entries 2111, 2112, 2121, 2122, and 2131 between the start point pointer 2300 and the end point pointer 2500 are log entries for which commit processing has been completed.

When an update process is started and a log entry associated with the update process is added to the journal log 2000, it is added to the position of the insertion pointer 2600, and the insertion pointer 2600 is updated to point to the next position of the log entry. be. If the log page at the position of the insertion pointer 2600 does not have enough free space to hold the log entry, the journal log control unit 1110 requests the free space management unit 1120 to allocate a free page to create a new free space. Get page. Next, the journal log control unit 1110 retains the address of the new log page at the end of the log page, joins the new log page to the log page list, and inserts the log page at the position of the insertion pointer 2600. Add a special log entry to indicate that there are no log entries in the following regions. Finally, the log entry is added to the top of the new log page and the insert pointer 2600 is updated to point to the next position of the log entry. In the example of FIG. 2, log entries 2211, 2212, 2221, 2222, and 2231 between the start pointer 2400 and the insertion pointer 2600 are log entries that have been added to the journal log 2000 but have not yet been committed. .

When the addition of all log entries associated with the update processing is completed, the journal log control unit 1110 executes commit processing. In the example of FIG. 2, the journal log control unit 1110 executes a flush instruction and a memory barrier instruction for the log page area between the start pointer 2400 and the insertion pointer 2600 where the log entry is written. Next, the end point pointer 2500 is updated to the position indicated by the insertion pointer 2600, and the flush instruction for the area holding the end point pointer 2500 and the memory barrier instruction are executed. As a result, the log entries 2111, 2112, 2121, 2122, and 2131 held by the log page list 2100 become invalid, and the log entries 2211, 2212, 2221, 2222, and 2231 held by the log page list 2200 become the committed logs. valid as an entry.

Also, in the commit process, the journal log control unit 1110 sets the starting point pointer 2300 to null, executes a flush instruction and a memory barrier instruction for the area holding the starting point pointer 2300, and sends the log to the free area management unit 1120. All log pages in page list 2100 may be requested to be released.

When the commit processing is completed, the journal log control unit 1110 executes checkpoint processing and reflects the contents of the committed log entries in the journal log 2000 to the main body of the file system.

When the checkpoint processing is completed, the journal log control unit 1110 updates the insertion pointer 2600 to the position indicated by the start point pointer of the log page list that is not the log page list used in the update processing, and ends the update processing. In the example of FIG. 2, the insert pointer 2600 is updated to the position indicated by the starting point pointer 2300 .

As described above, the journal log control unit 1110 alternately uses the log page lists 2100 and 2200 in the journal log 2000 for each update process. Even if the checkpoint process in the update process is completed and the log entry associated with the update process becomes unnecessary, the update of the end point pointer 2500 for invalidating the log entry, the flush instruction, and the memory barrier instruction are not performed in the update process. By executing the end point pointer 2500 update, flush instruction, and memory barrier instruction in the log entry commit process in the next update process, invalidation of the log entry in the update process and execution of the next By executing commit processing of the log entries in the update processing of 1 at once, it is possible to reduce the number of executions of the update of the end point pointer 2500, the flush instruction, and the memory barrier instruction.

FIG. 3 is a diagram showing an example of the configuration of log entries in the first embodiment. A log entry 3000 is composed of a header portion 3100 and a data portion 3200 . The header portion 3100 holds the type of log entry (hereinafter referred to as log entry type). For example, log entry types include a data write log entry, which is a log entry for writing data in a file, and a metadata update log entry, which is a log entry for updating metadata. There is also an end-of-log-page log entry, which is a log entry indicating that there are no log entries in subsequent areas in the log page. Other log entry types may be prepared. Header portion 3100 may also hold other information about the log entry. The data part 3200 may have different information and sizes depending on the log entry type held in the header part 3100 of the log entry. Also, the data section 3200 may not exist.

FIG. 4 is a diagram showing an example of the structure of a data write log entry in the first embodiment. In the first embodiment, write data uses one data write log entry 4000 for one page. Embodiments can also be configured to use one data write log entry 4000 for multiple consecutive pages in the write data. A header portion 4100 of the data write log entry 4000 holds data write log entry identification information 4110 that is information indicating that the log entry type is a data write log entry. The data part 4200 of the data write log entry 4000 allocates a write data page 4300 separately from the data write log entry 4000 and holds the write data in the write data page 4300 .

The data part 4200 of the data write log entry 4000 includes the address 4210 of the write data page 4300 that holds the write data (hereinafter referred to as the write data page address) and the offset 4220 from the beginning of the file in the write data (hereinafter referred to as the file offset). , the size 4230 of the write data (hereinafter referred to as write data size), and the address (hereinafter referred to as the reflection destination data page address) of the data page to which the write data is reflected in the file system (hereinafter referred to as the reflection destination data page). The address 4240 (hereinafter referred to as the reflection destination data page address holding address) of the position where the reflection destination data page address is held in the metadata page to be processed is held. Data write log entry 4000 may hold other information.

FIG. 5 is a diagram showing an example of the configuration of a log page end log entry in the first embodiment. The header portion 5100 of the log page end log entry 5000 holds log page end log entry identification information 5110 which is information indicating that the log entry type is the log page end log entry. In the example of FIG. 5, the end of log page log entry 5000 does not have a data portion.

Since the first embodiment of the present invention does not depend on the structure and content of metadata, the description of the structure and content of the metadata update log entry is omitted.

[An example of write processing in the first embodiment]
The write processing in the first embodiment consists of journaling processing, commit processing, and checkpoint processing. An outline of the write process is described below. In the journaling process, data write log entries are used to keep written data in the journal log. Metadata update log entries are also used to keep updates to file system metadata in the journal log. In the commit process, the log entry associated with the write process is made permanent to confirm the write process. In checkpoint processing, the contents of the log entries held in the journal log are reflected in the file system. At this time, when the write data to the file has not reached the end of the write data page of the data write log entry, delay checkpoint processing is executed to delay the reflection of the data write log entry to the file system. Deferred checkpointing saves the location of the data write log entry.

In the journaling process for subsequent writes, write data for which delayed checkpointing of a data write log entry has been performed in the previous write process, and checkpointing has been delayed by the delayed checkpointing of the data write log entry. If the range of write data of the subsequent write process is continuous with respect to the end of the range, write in the subsequent write process to the write data page of the data write log entry, and write the data Add a new data write log entry to the journal log that holds the write data page address of the log entry as the write data page address. On the other hand, in the journaling process of the subsequent write process, the delayed checkpoint process of the data write log entry is executed in the previous write process, but the checkpoint process is delayed by the delayed checkpoint process of the data write log entry. If the range of write data for the subsequent write operation is not contiguous with the end of the range of write data being written, immediately checkpoint the data write log entry and then journal the subsequent write operation. Execute the process.

When metadata is updated along with write processing, journaling processing of metadata update log entries, commit processing, and checkpoint processing are required. Since the description of the structure and contents of the metadata is omitted, the details of the metadata update log entry journaling process, commit process, and checkpoint process will be omitted in the following description.

FIG. 6 is a flow chart showing an example of write processing operations in the first embodiment. In the write process, the journal log control unit 1110 receives the write data, the size of the entire write data (hereinafter, total write data size), and the position within the file to write the write data (hereinafter, write position).

First, the journal log control unit 1110 executes journaling processing (step S6001). Journaling processing will be described later. When the journaling process is completed, the journal log control unit 1110 executes commit process (step S6002). Commit processing will be described later. When the commit processing is completed, the journal log control unit 1110 executes checkpoint processing (step S6003). Checkpoint processing will be described later.

[Example of journaling processing in the first embodiment]
FIG. 7 is a flowchart illustrating an example of journaling processing operations in the first embodiment. First, the journal log control unit 1110 transitions to step S7002 if the position in the journal log of the delayed data write log entry is saved, otherwise transitions to step S7007 (step S7001).

When the position in the journal log of the delayed data write log entry is saved, the journal log control unit 1110 acquires the data write log entry from the position in the journal log of the delayed data write log entry. However, the write range in the write process is continuous to the end of the range of write data held by the delayed data write log entry, that is, the write position in the write process is within the file of the delayed data write log entry. If equal to the sum of the offset value and the write data size value of the data write log entry, go to step S7003; otherwise, go to step S7006 (step S7002).

The write range is continuous to the end of the range of write data held by the delayed data write log entry, that is, the write position in the write process is the offset value in the file of the delayed data write log entry and the data When it is equal to the sum of the write data size values of the write log entries, the journal log control unit 1110 executes data append write processing (step S7003). The data append write process will be described later.

Next, the journal log control unit 1110 adds the data size added in step S7003 to the write position (step S7004), and subtracts the added data size from the total write size (step S7005).

The write range is not continuous to the end of the range of write data held by the delayed data write log entry, i.e., the write position is the value of the offset in the file of the delayed data write log entry and the delayed data When not equal to the sum of the write data size values of the write log entries, the journal log control unit 1110 sets the delayed checkpoint flag to false and executes checkpoint processing of the delayed data write log entries (step S7006). ). The delayed checkpoint flag is a flag that indicates whether to delay the checkpoint processing of the data write log entry. Checkpointing of data write log entries will be described later.

The journal log control unit 1110 executes the processing from step S7007 to step S7011 for each data page within the write range after the write position. If the total write size is 0, the journal log control unit 1110 proceeds to step S7012; otherwise, to step S7008 (step S7007).

When the total write size is not 0, the journal log control unit 1110 executes metadata update log entry addition processing for metadata holding the address of the data page at the write position (step S7008). In the first embodiment of the present invention, the metadata update log entry addition process is assumed to update metadata accompanying a change in the address of the data page at the write position. As described above, the details of the metadata update log entry addition process, including whether or not the addition process is necessary, will not be described.

The journal log control unit 1110 executes data write log entry addition processing for the data page at the write position (step S7009). The data write log entry addition process will be described later.

Next, the journal log control unit 1110 adds the data size written in step S7009 to the write position (step S7010), and subtracts the data size from the total write size (step S7011).

Finally, when the total write size is 0, the journal log control unit 1110 executes metadata update log entry addition processing for the metadata of the entire file (step S7012). In the first embodiment of the present invention, the metadata update log entry addition process is assumed to update the file update time. , details of the metadata update log entry addition process will be omitted.

[Data write log entry addition processing in the first embodiment]
FIG. 8 is a flowchart illustrating an example of the operation of data write log entry addition processing according to the first embodiment. First, the journal log control unit 1110 executes log entry area securing processing to secure an empty area for adding a data write log entry in the journal log (step S8001). The log entry area reservation process will be described later.

Next, the journal log control unit 1110 acquires information held in the data part of the data write log entry in steps S8002 to S8005. The journal log control unit 1110 acquires a free page by requesting the free space management unit 1120 to allocate a free page. is the write data page address (step S8002).

The journal log control unit 1110 searches the metadata of the file to be written, and acquires the address of the position that holds the address of the data page in the metadata page that holds the address of the data page corresponding to the write position. , the address is set as the reflection destination data page address holding address of the data write log entry (step S8003).

The journal log control unit 1110 sets the write position to the file offset of the data write log entry (step S8004).

The journal log control unit 1110 calculates, from the page size, the remainder obtained by dividing the offset in the file of the data write log entry acquired in step S8004 by the page size, and the minimum value of the total write size. The value is set as the write data size of the data write log entry (step S8005).

The journal log control unit 1110 writes write data equal to the write data size of the data write log entry to the position of the remainder value calculated in step S8005 in the write data page of the write log entry. is executed (step S8006).

The journal log control unit 1110 stores data write log entry identification information indicating that the log entry type is a data write log entry, the write data page address of the write log entry, and the reflection destination data of the write log entry. A data write log entry is created based on the page address holding address, the offset in the file of the write log entry, and the write data size of the write log entry, and the data write log entry is written at the position indicated by the insertion pointer 2600. (Step S8007). Finally, the journal log control unit 1110 sets the insertion pointer 2600 to the position next to the data write log entry written in step S8007 (step S8008).

[Additional data writing process in the first embodiment]
FIG. 9 is a flow chart showing an example of the operation of the data additional writing process according to the first embodiment. First, the journal log control unit 1110 executes log entry area securing processing to secure an empty area for adding a data write log entry in the journal log (step S9001). The log entry area reservation process will be described later.

Next, the journal log control unit 1110 acquires information held in the data part of the data write log entry in steps S9002 to S9005. The journal log control unit 1110 uses the data write data page address of the delayed data write log entry acquired in step S7002 as the data write data page address of the data write log entry for the data additional write (step S9002).

The journal log control unit 1110 sets the reflection destination data page address holding address of the delayed data write log entry acquired in step S7002 as the reflection destination data page address holding address of the data write log entry for the data append write ( step S9003).

The journal log control unit 1110 sets the file offset of the delayed data write log entry acquired in step S7002 as the file offset of the data write log entry for the data append write (step S9004).

The journal log control unit 1110 calculates, from the page size, the remainder obtained by dividing the file offset of the delayed data write log entry acquired in step S7002 by the page size, and the write data size of the delayed data write log entry. Calculate the minimum value of the value obtained by subtracting the total and the current total write size, and calculate the sum of the write data size of the delayed data write log entry and the minimum value of the data write log entry for the data append write. The write data size is set (step S9005).

The journal log control unit 1110 divides the in-file offset of the delayed data write log entry by the page size, in the write data page indicated by the write data page address of the data write log entry for the data append write, and the delay The value obtained by subtracting the write data size of the delayed data write log entry from the write data size of the data write log entry corresponding to the data append write is placed at the position of the sum of the write data size of the delayed data write log entry and the write data size of the delayed data write log entry , writes the write data, and executes the flash command for the written area (step S9006).

The journal log control unit 1110 includes data write log entry identification information indicating that the log entry type is a data write log entry, the write data page address of the write log entry for the data append write, and the data write log entry. The data page address to which the write log entry is reflected, the offset in the file of the write log entry for the data appending write, and the write data size of the write log entry for the data appending write. An entry is created, and a data write log entry for the data additional write is written at the position indicated by the insertion pointer 2600 (step S9007). Finally, the journal log control unit 1110 sets the insertion pointer 2600 to the position next to the data write log entry for the data additional write written in step S9007 (step S9008).

[Log entry area reservation processing in the first embodiment]
FIG. 10 is a flowchart illustrating an example of the operation of log entry area allocation processing according to the first embodiment. First, the journal log control unit 1110 transitions to step S10007 if the insertion pointer is null, otherwise transitions to step S10002 (step S10001).

When the insert pointer 2600 is not null, the journal log control unit 1110 reserves a log entry area if the log page indicated by the insert pointer 2600 has an area sufficient to hold the log entry to be added. End the process. Otherwise, the journal log control unit 1110 transitions to step S10003 (step S10002).

When the log page at the position indicated by the insertion pointer 2600 does not have enough free space to hold the log entry to be added, the journal log control unit 1110 requests the free space management unit 1120 to allocate a free page. to obtain an empty page (step S10003).

Next, the journal log control unit 1110 writes the address of the free page acquired in step S10003 to the end of the log page at the position indicated by the insertion pointer 2600, and joins the free page to the log page list of the journal log ( step S10004). The journal log control unit 1110 also executes log page end log entry addition processing (step S10005). The log page end log entry addition processing will be described later.

The journal log control unit 1110 sets the insertion pointer 2600 to the top position of the free page acquired in step S10003 (step S10006).

When the insert pointer 2600 is null, the journal log control unit 1110 requests allocation of a free page to the free space management unit 1120 and acquires a free page (step S10007).

The journal log control unit 1110 sets the head pointer indicating the head of the log page list used in the current update process to the head position of the empty page acquired in step S10007, and sets the head pointer to the area holding the head pointer. A flash instruction and a memory barrier instruction are executed (step S10008). Also, the journal log control unit 1110 sets the insertion pointer 2600 to the top position of the empty page acquired in step S10007 (step S10009).

[Log page end log entry addition processing in the first embodiment]
FIG. 11 is a flow chart showing an example of the operation of log page end log entry addition processing according to the first embodiment. The journal log control unit 1110 creates a log page end log entry using the log page end log entry identification information, which is information indicating that the log entry type is a log page end log entry, and inserts the log page end log entry at the position indicated by the insertion pointer 2600. A log page end log entry is written (step S11001).

[Commit processing in the first embodiment]
FIG. 12 is a flowchart illustrating an example of commit processing operations in the first embodiment. First, the journal log control unit 1110 sets a log entry pointer indicating a log entry in the journal log to the position of the head pointer of the log page list used in the journaling process of the write process (step S12001).

If the log entry pointer and the insertion pointer 2600 indicate the same log page, the journal log control unit 1110 proceeds to step S12005, otherwise proceeds to step S12003 (step S12002).

When the log entry pointer and the insertion pointer 2600 do not indicate the same log page, the journal log control unit 1110 executes a flush instruction for the entire log page indicated by the log entry pointer (step S12003).

Next, the journal log control unit 1110 sets the log entry pointer to the head position of the next log page in the log page list held at the end of the log page indicated by the log entry pointer (step S12004). ).

When the log entry pointer and the insertion pointer 2600 indicate the same log page, the journal log control unit 1110 executes a flush instruction for the area from the log entry pointer to the insertion pointer 2600, and then executes a memory barrier instruction. Execute (step S12005).

The journal log control unit 1110 sets the end point pointer 2500 to the position of the insertion pointer 2600, and executes the flush command and memory barrier command for the area holding the end point pointer 2500 (step S12006).

The journal log control unit sets the log entry pointer to the position indicated by the head pointer of the log page list not used in the write process (step S12007).

If the log entry pointer is null, the journal log control unit 1110 ends commit processing. Otherwise, the journal log control unit 1110 transitions to step S12009 (step S12008).

When the log entry pointer is not null, the journal log control unit 1110 saves the value of the log entry pointer, and then stores the log entry pointer at the end of the log page at the position indicated by the log entry pointer. The next log page in the log page list is set at the head position (step S12009).

The journal log control unit 1110 requests the free space management unit 1120 to release the log page indicated by the value of the log entry pointer saved in step S12009, and releases the log page (step S12010).

[Checkpoint processing in the first embodiment]
FIG. 13 is a flowchart illustrating an example of checkpoint processing operations in the first embodiment. The journal log control unit 1110 performs step S13001 on each committed log entry, that is, each entry between the start point pointer indicating the head position of the log page list used in the update process and the end point pointer 2500. , the process of step S13008 is executed. First, the journal log control unit 1110 sets the log entry pointer indicating the log entry to be processed in the journal log to the position indicated by the start point pointer of the log page list used in the update process (step S13001).

If the log entry pointer is at the same position as the end point pointer 2500, the journal log control unit 1110 proceeds to step S13009; otherwise, to step S13003 (step S13002).

When the log entry pointer is not at the same position as the end point pointer 2500, the journal log control unit 1110 proceeds to step S13004 if the log entry type of the log entry indicated by the log entry pointer is a data write log entry; otherwise, step S13006. (step S13003).

When the log entry type of the log entry indicated by the log entry pointer is the data write log entry, the journal log control unit 1110 sets the delay checkpoint flag to true and executes checkpoint processing for the data write log entry (step S13004). . Checkpointing of data write log entries will be described later.

Next, the journal log control unit 1110 adds the log entry size corresponding to the log entry type of the log entry indicated by the log entry pointer to the log entry pointer (step S13005).

When the log entry type of the log entry indicated by the log entry pointer is not a data write log entry, the journal log control unit 1110 proceeds to step S13007 if the log entry type of the log entry indicated by the log entry pointer is a metadata update log entry. Otherwise, the process proceeds to step S13008 (step S13006).

When the log entry type of the log entry indicated by the log entry pointer is metadata update log entry, the journal log control unit 1110 executes checkpoint processing for the metadata update log entry (step S13007). As described above, the detailed description of the metadata update log entry checkpoint process is omitted.

When the log entry type of the log entry indicated by the log entry pointer is not a metadata update log entry, the journal log control unit 1110 stores the log entry pointer at the end of the log page at the position indicated by the log entry pointer. The next log page in the log page list is set to the head position (step S13008). In one example of the checkpoint processing operation in the first embodiment, the log entry type is assumed to be one of data write log entry, metadata update log entry, and log page end log entry. . When other log entry types exist, it is necessary to determine that the log entry type is the other log entry type and to execute checkpoint processing corresponding to the other log entry.

Finally, when the log entry pointer is at the same position as the end point pointer, the journal log control unit 1110 executes insert pointer update processing (step S13009). Insert pointer update processing will be described later.

[Checkpoint processing of data write log entries in the first embodiment]
FIG. 14 is a flow chart showing an example of the operation of checkpoint processing for data write log entries in the first embodiment. The journal log control unit 1110 determines whether the delayed checkpoint flag is false or the write has reached the end of the write data page, that is, the remainder obtained by dividing the offset in the file of the data write log entry by the page size and the data write log If the total write data size of the entry is the same value as the page size, the process proceeds to step S14002; otherwise, the process proceeds to step S14003 (step S14001).

The deferred checkpoint flag is false, or the write has reached the end of the write data page, that is, the remainder of dividing the offset in the file of the data write log entry by the page size and the write data size of the data write log entry When the total is the same value as the page size, the journal log control unit 1110 executes immediate checkpoint processing of the data write log entry (step S14002). Immediate checkpointing of data write log entries is described below.

The deferred checkpoint flag is true and the write has not reached the end of the write data page, i.e. the remainder of dividing the offset in the file of the data write log entry by the page size and the write data size of the data write log entry When the total is less than the page size, the journal log control unit 1110 executes delayed checkpoint processing of data write log entries (step S14003). Delayed checkpointing of data write log entries is described below.

[Immediate checkpoint processing of data write log entries in the first embodiment]
Immediate checkpointing of data write log entries can reduce the amount of data written associated with checkpointing compared to checkpointing in conventional journaling.

For example, if the write data size of the data write log entry is less than a predetermined threshold, the write data held in the write data page of the data write log entry is transferred to the reflection destination data page address holding address of the data write log entry. In the data page indicated by the target data page address held at the location, write to the location of the remainder obtained by dividing the offset in the file of the relevant data write log entry by the page size, and execute the flush command for the written area. do.

Further, for example, if the write data size of the data write log entry is equal to or greater than a predetermined threshold, the write data page of the data write log entry is transferred to an area other than the range where the write data is held. Data is written from the data page indicated by the reflection destination data page address held at the position of the reflection destination data page address holding address, and the flash instruction of the written area is executed. Furthermore, the write data page address of the data write log entry is written at the position of the reflection destination data page address holding address of the data write log entry, and the flush instruction of the written area is executed. The predetermined threshold is, for example, 1/2 of the page size.

FIG. 15 is a flow chart showing an example of the operation of immediate checkpoint processing of data write log entries in the first embodiment. First, the journal log control unit 1110 detects that the reflection destination data page address held at the position of the reflection destination data page address holding address of the data write log entry is null, that is, if the reflection destination data page does not exist, To step S15011, otherwise, to step S15002 (step S15001).

When the reflection destination data page address is not null, that is, when the reflection destination data page exists, the journal log control unit 1110 proceeds to step S15003 if the write data size of the data write log entry is less than a predetermined threshold. Otherwise, the process proceeds to step S15004 (step S15002).

When the write data size of the data write log entry is less than the predetermined threshold, the journal log control unit 1110 writes the write data held in the write data page indicated by the write data page address of the data write log entry to the The remainder obtained by dividing the offset in the file of the data write log entry by the page size in the reflection destination data page indicated by the reflection destination data page address held at the position of the reflection destination data page address holding address of the data write log entry The write data size of the data write log entry is written to the position of the value, and the flush command of the written area is executed (step S15003).

When the write data size of the data write log entry is not less than the predetermined threshold, the journal log control unit 1110 determines that the write is not from the beginning of the write data page of the data write log entry, that is, within the file of the write log entry. If the remainder obtained by dividing the offset by the page size is not 0, the process proceeds to step S15005; otherwise, the process proceeds to step S15006 (step S15004).

When writing is not from the beginning of the write data page of the data write log entry, that is, when the remainder obtained by dividing the offset in the file of the data write log entry by the page size is not 0, the journal log control unit 1110 starts the data write. From the beginning of the reflection destination data page indicated by the reflection destination data page address held at the position of the reflection destination data page address holding address of the log entry, to the write data page indicated by the write data page address of the data write log entry Data is written to the top by the remainder obtained by dividing the offset in the file of the data write log entry by the page size, and the flush command for the written area is executed (step S15005).

The journal log control unit 1110 determines that the write has not reached the end of the write data page of the data write log entry, that is, the remainder obtained by dividing the offset in the file of the data write log entry by the page size, and the data write log entry is smaller than the page size, the process proceeds to step S15007; otherwise, the process proceeds to step S15008 (step S15006).

The write has not reached the end of the write data page for this data write log entry, i.e., the sum of the remainder of dividing the offset in file of this data write log entry by the page size plus the write data size for this data write log entry is smaller than the page size, the journal log control unit 1110, in the reflection destination data page indicated by the reflection destination data page address held at the position of the reflection destination data page address holding address of the data write log entry, Data from the position of the sum of the remainder obtained by dividing the file offset of the relevant data write log entry by the page size and the write data size of the relevant data write log entry to the end of the relevant reflection destination data page In the write data page indicated by the write data page address of the write log entry, the position of the sum of the remainder obtained by dividing the file offset of the data write log entry by the page size and the write data size of the data write log entry , and executes the flash command for the written area (step S15007).

The journal log control unit 1110 saves the reflection destination data page address held at the position of the reflection destination data page address holding address of the data write log entry (step S15008).

The journal log control unit 1110 writes the write data page address of the data write log entry to the position of the reflection destination data page address holding address of the data write log entry, and executes the flush command for the written area (step S15009). ).

The journal log control unit 1110 requests the free space management unit 1120 to release the reflection destination data page indicated by the reflection destination data page address saved in step S15008, and releases the reflection destination data page (step S15010). ).

When the reflection destination data page address held at the position of the reflection destination data page address holding address of the data write log entry is null, that is, when the reflection destination data page does not exist, the journal log control unit 1110 determines that the write If not from the beginning of the write data page of the data write log entry, that is, if the remainder obtained by dividing the file offset of the data write log entry by the page size is not 0, go to step S15012; otherwise, go to step S15013. Transition (step S15011).

When writing is not from the beginning of the write data page of the data write log entry, that is, when the remainder obtained by dividing the offset in the file of the data write log entry by the page size is not 0, the journal log control unit 1110 starts the data write. In the write data page indicated by the write data page address of the log entry, 0 is written from the beginning of the write data page by the remainder obtained by dividing the offset in the file of the data write log entry by the page size. A flush command for the area is executed (step S15012).

The journal log control unit 1110 determines that the write has not reached the end of the write data page of the data write log entry, that is, the remainder obtained by dividing the offset in the file of the data write log entry by the page size, and the data write log entry is smaller than the page size, the process proceeds to step S15014; otherwise, the process proceeds to step S15015 (step S15013).

The write has not reached the end of the write data page for this data write log entry, i.e., the sum of the remainder of dividing the offset in file of this data write log entry by the page size plus the write data size for this data write log entry is smaller than the page size, the journal log control unit 1110 divides the offset in the file of the data write log entry by the page size in the write data page indicated by the write data page address of the data write log entry. , 0 is written from the position of the total value of the write data size of the data write log entry to the end of the write data page, and the flush command for the written area is executed (step S15014).

The journal log control unit 1110 writes the write data page address of the data write log entry to the position of the reflection destination data page address holding address of the data write log entry, and executes the flush command for the written area (step S15015). ).

[Delayed checkpoint processing of data write log entries in the first embodiment]
FIG. 16 is a flow chart showing an example of operation of delayed checkpoint processing of data write log entries in the first embodiment. The journal log control unit 1110 saves the position of the data write log entry indicated by the log entry pointer as the position of the delayed data write log entry (step S16001). The area that holds the location of the delayed data write log entry indicated by the log entry pointer may be volatile memory.

[Insertion Pointer Update Processing in First Embodiment]
FIG. 17 is a flowchart illustrating an example of operation of insert pointer update processing according to the first embodiment. The journal log control unit 1110 sets the insertion pointer 2600 to the position indicated by the start point pointer of the log page list that is not the log page list currently indicated by the insertion pointer 2600 (step S17001). For example, if the insertion pointer 2600 points to the log page list 2100, the insertion pointer 2600 is moved to the position of the start pointer 2400. set.

[Effects of the first embodiment]
In a first embodiment, the persistent memory 1200 stores data, metadata, and journal logs, which are update histories of data and metadata. Further, when a write request is received, the journal log control unit 1110 performs journaling processing for storing the update history of data and metadata associated with the write request in the journal log, and commit processing for finalizing the update history stored in the journal log. Checkpoint processing that executes either immediate checkpoint processing that reflects the fixed update history stored in the journal log in data or metadata, or delayed checkpoint processing that delays the immediate checkpoint processing; controls the execution of

For example, according to the first embodiment, in checkpoint processing, when the write data has not reached the end of the write data page of the data write log entry, the write of the data write log entry is performed. It can be delayed until the end of the data page is reached. For this reason, according to the first embodiment, checkpoint processing in multiple write processing can be collectively performed. By ensuring consistency and reducing the amount of writing associated with journaling, the write performance of the file system can be improved.

Further, if the size of the write data in the log entry in the journal log for which checkpoint processing is delayed is less than a predetermined threshold, the journal log control unit 1110 transfers the write data in the log entry in the journal log to a file. , and if it is equal to or greater than the predetermined threshold, the page storing the write data in the log entry in the journal log is used as the page storing the file data. As a result, the amount of writing from the journal log to the file can be reduced.

Furthermore, by alternately using two log page lists in the journal log, even if the checkpoint processing in the write processing is completed and the log entry for which the checkpoint processing is completed becomes unnecessary, the checkpoint processing is completed. Sometimes, without updating the endpoint pointer, flush instruction, and memory barrier instruction to invalidate the log entry, update the endpoint pointer, flush instruction, and memory barrier in commit processing in the next write processing By executing the instruction, invalidation of the log entry associated with the write processing and commit processing of the log entry associated with the next write processing are performed at the same time, thereby updating the end point pointer, executing the flush instruction, and executing the memory barrier instruction. The number of times can be reduced.

1000 system 1100 file system 1110 journal log control unit 1120 free area management unit 1200 persistent memory 2000 journal log 2100 log page list 2110 log page 2111, 2112 log entry 2120 log page 2121, 2122 log entry 2130 log page 2131 log entry 2200 log page List 2210 Log pages 2211, 2212 Log entries 2220 Log pages 2221, 2222 Log entries 2230 Log pages 2231

Log entries

2300, 2400 Start pointer 2500 End pointer 2600 Insert pointer 3000 Log entries 3100 Header 3200 Data 4000 Data write log entry 4100 Header Part 4110 Data write log entry identification information 4200 Data part 4210 Write data page address 4220 File offset 4230 Write data size 4240 Reflection target data page address holding address 4300 Write data page 5000 Log page end log entry 5100 Header part 5110 Log page end log Entry identification information

Claims

A journal log control system for controlling the journal log stored in a persistent memory that stores data, metadata, and a journal log that is an update history of the data and the metadata, wherein a write request is received. journaling processing for storing update history of data and metadata associated with the write request in the journal log; commit processing for finalizing the update history stored in the journal log; A journal that controls the execution of checkpoint processing that executes either immediate checkpoint processing for reflecting a confirmed update history on the data or metadata, or delayed checkpoint processing that delays the immediate checkpoint processing. A journal log control system comprising a log control unit.
When the size of the write data in the log entry in the journal log for which checkpoint processing is delayed is less than a predetermined threshold, the journal log control unit writes the write data in the log entry in the journal log to a file. 2. The method according to claim 1, characterized in that, if the result is reflected and is equal to or greater than a predetermined threshold, a page storing write data in the log entry in the journal log is used as a page storing file data. journal log control system.
The journal log control unit prepares a plurality of storage locations for the update history to be stored in the journal log, uses the plurality of storage locations properly for each execution of journaling processing, and executes checkpoint processing to store unnecessary journal logs. 2. The journal log control system according to claim 1, wherein invalidation of the stored update history and commit processing for the next update are executed simultaneously.
A journal log control method executed by a computer that controls the journal log stored in a persistent memory that stores data, metadata, and a journal log that is an update history of the data and the metadata, wherein ,
a journaling process of storing an update history of data and metadata associated with the write request in the journal log when a write request is made;
a commit process for finalizing the update history stored in the journal log;
Immediate checkpoint processing for reflecting the confirmed update history stored in the journal log in the data and the metadata;
a checkpoint process that performs either a deferred checkpoint process that delays the immediate checkpoint process;
A journal log control method comprising:
A computer that controls the journal log stored in a permanent memory that stores data, metadata, and a journal log that is an update history of the data and the metadata,
a journaling process of storing an update history of data and metadata associated with the write request in the journal log when a write request is made;
a commit process for finalizing the update history stored in the journal log;
Immediate checkpoint processing for reflecting the confirmed update history stored in the journal log in the data and the metadata;
a checkpoint process that performs either a deferred checkpoint process that delays the immediate checkpoint process;
A journal log control program characterized by executing