KR101881039B1 - Method for asynchronous atomic update of memory mapped files stored in non-volatile memory and control apparatus thereof - Google Patents

Abstract

According to the present invention, disclosed is a file update method and a control apparatus thereof for providing a more safe and efficient memory mapped file I/O in an in-memory file system of a non-volatile memory. According to the file update method of the present invention, when update occurs in an original file, a process for real memory synchronization by a memory synchronization request asynchronously processes the update in a kernel area, and the process utilizes process resources to use only low latency time and work in a user area.

Description

Technical Field [0001] The present invention relates to an asynchronous atomic update method and apparatus for a memory mapping file stored in a nonvolatile memory,

The present invention relates to an update method and apparatus for updating a memory mapping file, and more particularly, to an asynchronous atomic update method and apparatus capable of ensuring consistency and reliability of a memory mapping file stored in a nonvolatile main memory will be.

Recently, nonvolatile memory devices have similar performance to dynamic random access memory (DRAM), and are expected to be the main memory of the future. As non-volatile dual in-line memory modules (NVDIMMs) have been developed, nonvolatile memories are increasingly used as main memory of computers. With nonvolatile main memory, files can be stored continuously in main memory, enabling fast file I / O without secondary storage latency. In particular, since the memory mapped file I / O technique maps a file located in main memory to a process address space, the process accesses the file directly using the load / store command without the help of the operating system . This technique is attracting attention as an important file input / output technique because it can minimize the software overhead which is the biggest overhead in the system where the secondary storage delay time is lost.

However, even a system using non-volatile memory has a volatile cache and a buffer in the processor, so that when a system crash occurs, some data is lost. The processor also uses out-of-order execution techniques to improve performance and uses cache management techniques to take advantage of data locality, so it is difficult to know when updated data is reflected in memory . Therefore, if a system error occurs while the processor is updating the file, some of the files will be updated and some will not be updated. In particular, memory mapping file I / O has a problem that the operating system can not guarantee the consistency and reliability of the file because the process directly accesses the actual memory.

The memory synchronization system function (call) provided by the operating system can be used to perform synchronization of the memory mapping file. However, there is still the problem that the consistency of the file can not be guaranteed because the updated contents can not be atomically performed on the original.

Korean Patent Publication No. 10-2015-0082010

The present specification is intended to provide an asynchronous atomic update method and apparatus for a memory mapping file that can ensure consistency and reliability.

The solutions described herein are not limited to those mentioned above, and other solutions not mentioned may be clearly understood by those skilled in the art from the following description.
According to an aspect of the present invention, there is provided a method for updating a memory mapping file, the method comprising: a nonvolatile memory having a process address space, a page table, and a file system area; (Hereinafter, referred to as an 'original file') stored in the file system area and mapped to a process address space, the method comprising the steps of: (a) Allocating a new page (hereinafter referred to as an 'update page') to the file system area and copying the contents of the page where the original file is updated to the update page when the update occurs; (b) changing, by the first process, a mapping address of a page of the original file stored in the page table in which the update occurs, to a mapping address of the update page; (c) the first process stores contents (hereinafter, referred to as " connection information ") describing the connection relationship between the page where the update occurred and the update page, and stores the update content in the update page step; (d) the first process locks the original file and the update page by a memory synchronization request and returns a memory synchronization request; (e) when the second process is storing the connection information, locking the original file and copying the update page to a page of the original file where the update occurred; And (f) changing, by the second process, the mapping address of the page table to a page in the original file that the update occurred, and releasing the lock setting of the original file.

delete

According to an embodiment of the present invention, the non-volatile memory has a general area, and the step (c) may be a step in which the first process stores the connection information in the general area. In this case, the step (d) may further include, after the first process locks the original file, storing the connection information in the general area to the file system area.

The memory mapping file updating method according to the present invention may further include: (d-1) storing the data stored in the cache in the file system area by the first process.

According to an embodiment of the present invention, in the step (f), the second process deletes connection information when the update page is in use, and when the update page is not in use, adds the update page to a delete queue And deleting the connection information; And terminating the memory synchronization when the second process does not include the update page in the deletion wait queue, and deleting the update page and terminating the memory synchronization when the update page exists in the deletion wait queue.

According to one embodiment of the present disclosure, the steps (e) and (f) for performing memory synchronization may be parallelized into multiple processes or threads.

The method of updating a memory mapping file according to the present specification may take the form of a computer program recorded on a computer-readable recording medium which is created to perform each step in the computer.

According to an aspect of the present invention, there is provided a memory mapping file update apparatus including a nonvolatile memory having a process address space, a page table and a file system area, a first process of processing file update, (Hereinafter referred to as " original file ") that is stored in the file system area and mapped in the process address space, and the first process is a step of, when an update occurs on a page of the original file, The update page is allocated to the file system area, the updated content is stored in the update page, and the mapping address of the page of the original file stored in the page table is updated to a mapping address of the update page The page of the original file, (Hereinafter referred to as " connection information ") describing a connection relationship between the page where the update occurs and the update page, stores the update content in the update page, and stores the original file and the update page Locks and returns a memory synchronization request, and the second process locks the original file when the connection information is stored, copies the update page to the page where the update occurred, , The mapping address of the page table may be changed to the page where the update occurred in the page of the original file, and the lock setting of the original file may be canceled.

According to the file update method and control apparatus of the present disclosure, a secure memory-mapped file I / O (I / O) is possible in an in-memory file system. In addition, memory-mapped file I / O (I / O) in an in-memory file system can make full use of the characteristics of a high-performance memory storage device.

The effects described in the present specification are not limited to those mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

1 is a block diagram of an architecture of a computer system.
FIGS. 2 to 6 are reference views showing page storage and movement of a file over time in order to explain a synchronous atomic file update method. FIG.
7 is a process execution conceptual diagram of a synchronous memory mapping file updating apparatus and method.
8 is a process execution conceptual diagram of an asynchronous memory mapping file updating apparatus and method.
9 is a flow chart for processing a memory synchronization request.
10 is a flow chart for processing a memory synchronization task.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. Wherever possible, the same or similar parts are denoted using the same reference numerals in the drawings.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms as used herein include plural forms as long as the phrases do not expressly express the opposite meaning thereto.

Means that a particular feature, region, integer, step, operation, element and / or component is specified and that other specific features, regions, integers, steps, operations, elements, components, and / It does not exclude the existence or addition of a group.

All terms including technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Predefined terms are further interpreted as having a meaning consistent with the relevant technical literature and the present disclosure, and are not to be construed as ideal or very formal meanings unless defined otherwise.

Before describing the asynchronous file update method and the control apparatus according to the present invention, a synchronous file update method and a control apparatus thereof will be described. It is to be understood that the description of the non-asynchronous synchronous method is to be understood more easily.

1 is a block diagram of an architecture of a computer system.

Referring to FIG. 1, a computer system 100 including a processor 120 and a non-volatile memory 140 may be identified.

The non-volatile memory 140 may have a process address space 142, a page table 144, and a file system area 146. The process address space 142, the page table 144, and the file system area 146 are well-known to those skilled in the art, and detailed description of each configuration will be omitted .

The processor 120 may update a file (hereinafter, referred to as a 'source file') stored in the file system area and mapped to the process address space 142. To that end, the processor 120 may include a core 121 and a cache 122. Since the core 121 and the cache 122 are well known to those skilled in the art, a detailed description of each configuration will be omitted.

The non-volatile memory 140 and the processor 120 may be connected through a memory bus (I / O bus) 160. The memory bus 160 is also a well-known term to those skilled in the art, so that a detailed description of the configuration will be omitted.

FIGS. 2 to 6 are reference views showing page storage and movement of a file over time to describe a synchronous file update method.

Referring to FIG. 2, it can be seen that a file in the non-volatile memory is stored. The file system area is a physical area where files are actually stored. The file stored in the file system area may be an 'original file', and the file may be composed of a plurality of pages (A to F). The process address space is a virtual area in which a process recognizes that a file is stored. The process address space is a virtual area used by a process, and is an independent area for each process. The process can directly access the original file by mapping the original file to the process address space.

The page table stores information for mapping the processor address space, which is a virtual area, with the file system area, which is an actual area. Accordingly, when the processor accesses a specific file (or a specific page, memory), the processor accesses the process address space, identifies a mapping address stored in the page table, and stores the address in the physical file (or page) Approach.

In the example shown in FIG. 2, it can be confirmed through the page table that virtual addresses '1 to 6' corresponding to pages 'A to F' stored in the file system area are stored in the processor address space.

In one example, the file system area may be an Ext4-DAX format.

The synchronous memory mapping file update method according to the present invention can be processed by a processor that processes an instruction to update a file. Therefore, in describing the present specification, a subject which copies, moves, and stores a file can be considered as a processor. Also, in this specification, what is referred to as a processor is a name that refers to a device that performs a process, and it is apparent that a single processor can mount two or more processes. Therefore, when it is necessary to distinguish two or more processes, it will be referred to as a 'first process' and a 'second process', respectively.

Referring to FIG. 3, it can be confirmed that the update occurs in the page 'E' of the page corresponding to the virtual address '5'. The processor allocates a new page (hereinafter referred to as an 'update page') to the file system area when an update occurs on a page of the original file, and copies the contents of the page where the update of the original file occurred to the update page .

The processor may change a mapping address of a page of the original file stored in the page table to a mapping address of the update page. Referring to FIG. 3, it can be seen that the physical address of the virtual address '5' of the page table is changed from 'E' to 'G'.

The processor may store contents (hereinafter referred to as " connection information ") describing a connection relationship between the page where the update occurred and the update page among pages of the original file.

And the processor may store the update content in the update page.

The processor may store the connection information in a process address space. Referring to FIG. 3, connection information described as 'E-G' can be confirmed in the processor address space. The process address space in which the connection information is stored is not a space to which the file system area is mapped.

Upon completion of the process, the processor performs the update that occurred first. The processor performs an update to the original virtual address, but the actual page on which the update actually occurs is the update page.

The above process can be repeated every time a plurality of updates occur until a synchronization request of a memory mapping file occurs. Referring to FIG. 4, an update occurs in a page stored in physical addresses 'B' and 'D', and updates contents are stored as 'H' and 'I' after allocation of an update page, Connection information is added.

5 is a situation where a synchronization request of a memory mapping file is executed.

The system calls that cause the memory mapping file to be synchronized correspond to 'msync' on Linux OS or Unix-like OS and 'FlushFileBuffers' on Windows OS. However, the above example is merely an example according to the operating system, and it should be considered as a system call corresponding to the present specification depending on whether or not the memory mapping file is synchronized regardless of the operating system.

First, the processor can lock the original file by a memory synchronization request. The locking of the original file means that the reading / writing is prohibited. The read / write prohibition can prevent the first process and the second process from accessing the original file that has not been updated before the update is completed.

According to an embodiment of the present invention, the processor may store the connection information in the file system area in the process address space after setting the original file lock. A journaling effect can be obtained by simply storing the connection information in the file system area. If the connection information is stored in the process address space, a memory synchronization request has not yet occurred. In this case, when a system error occurs and the system is rebooted, the connection information disappears, and the updated contents are not reflected in the original file. When the connection information is stored in the file system area, a memory synchronization request is generated. Even if a system error occurs during the memory synchronization, the connection information, the update page and the original page remain in the file system area. Therefore, the memory synchronization operation can be completed through the connection information remaining in the file system area. In addition, since the size of the connection information is small, the burden on the processor can be reduced as compared with general journaling.

The processor may store data stored in the cache of the processor in the file system area. The update may not be immediately stored in the file system area and may remain in the processor's cache. Thus, the update is nonvolatile, which is a continuously storable area, so that the memory can be reliably stored in the remaining file system area without being initialized even after a reboot.

Next, the processor can refer to the connection information and copy the update page to the page where the update occurred among the pages of the original file.

After the copying of the update file is completed, as shown in FIG. 6, the processor changes the mapping address of the page table to the page of the original file that the update occurred, and the process address space and the file system area Lt; / RTI > The processor may also delete the update page stored in the file system area. The processor can then unlock the original file.

According to an apparatus and method for updating a synchronous memory mapping file, the processor stores the connection information in a file system area when starting a memory synchronization operation, and when the memory synchronization is completed, the processor deletes the connection information, Atomic updates are possible even during a system failure. This is because, after rebooting, the connection information remaining in the file system area is referred to, and the memory synchronization operation that can not be completed due to a system error can be re-executed. If there is no remaining connection information, the current file system can be determined to be in a state of maintaining consistency.

A major advantage of the synchronous memory mapping file updating apparatus and method is that the updated contents are reflected in the original so that other processes can access the updated contents. However, as a result of analyzing the workload of the actual processor, it can be said that other processes rarely access the updated file. In other words, it is not physically impossible for another process to access an updated file, but in most real devices, accessing a file that is being processed by another process occurs with a low probability, and even if it happens, It is questionable whether synchronous updating of files in order to take advantage of an effective process is inevitable.

7 is a process execution conceptual diagram of a synchronous memory mapping file updating apparatus and method.

Referring to FIG. 7, the memory synchronization request can confirm that the process is waiting while the memory synchronization proceeds. The waiting time of the process is the time from memory synchronization request to memory synchronization return.

8 is a process execution conceptual diagram of an asynchronous memory mapping file updating apparatus and method.

Referring to FIG. 8, it can be seen that the waiting time of the process is much shorter than the synchronous memory mapping file updating apparatus and method, from the memory synchronization request to the memory synchronization return. In other words, the process of real memory synchronization by the memory synchronization request is processed asynchronously in the kernel domain, and the process uses the process resource to use only the short wait time and to operate in the user domain.

Hereinafter, an apparatus and method for updating an asynchronous memory mapping file according to the present invention will be described. However, the repetitive description will not be repeated for the components described in the above-described synchronous memory mapping file updating apparatus and method, and differences will be mainly described.

The apparatus and method for updating an asynchronous memory mapping file according to the present specification are divided into processing of file update processing and memory synchronization, as mentioned in FIG. In the present specification, a subject to process the file update will be referred to as a first process, and a subject to process the memory synchronization will be referred to as a second process.

The contents of the file update process of the first process are the same as those of the synchronous memory mapping file updating apparatus and method shown in FIG. 2 to FIG. Therefore, the repetitive description of the file update process of the first process will be omitted.

At this time, the non-volatile memory may have a general area, and the first process may store the connection information in the general area.

On the other hand, the first process may lock the original file and the update page and return a memory synchronization request when there is a memory synchronization request.

9 is a flow chart for processing a memory synchronization request.

Referring to FIG. 9, there is shown a process for processing when a first process has a memory synchronization request.

The first process may first lock the original page. This is to prevent other processes from accessing the original page.

And the first process can check whether the update page is stored in the file system. This is the so-called cache flush, in which the first process stores the data stored in the cache in the file system area. This makes it possible to prevent the files that existed only in the cache area from disappearing even if a problem occurs in the computer system suddenly.

And the first process may store the connection information in the file system area in the general area. This prevents sudden loss of connection information that existed only in the general area even if a problem occurs in the computer system.

Meanwhile, regarding the thread related to the second process to be described later, the first process may output a signal (Thread Wake-UP signal) indicating that the connection information is stored in the thread after storing the connection information. On the other hand, the scheduler may periodically execute the thread to check whether the connection information is stored in the file system area.

And the first process can write-protect the update page. This is an asynchronous feature that is divided into processing of file update processing and memory synchronization. New update contents may occur on the same page after the file update process is performed by the first process. In this case, it is possible to generate a new update page according to a new update content, instead of modifying the existing update page with a new update content. That is, a first update page according to an update content generated before a memory synchronization operation, a second update page based on an update content generated later, a third update page, a fourth update page, and the like may occur.

Then, the first process releases the locking of the original page and terminates the synchronization request. Thereafter, the first process returns to the original operation and processes the work in the user area. A new update page may occur repeatedly depending on the task occurring at this time. At this time, the process proceeds according to the file update method shown in FIG. 2 to FIG.

Meanwhile, in the apparatus and method for updating an asynchronous memory mapping file according to the present specification, a memory synchronization task that the second process processes is stored in a thread area. That is, the content for starting the memory synchronization operation exists in the thread area, and the memory synchronization task is allocated to the second process by the scheduler that manages the task of the thread.

On the other hand, the first process and the second process are generally processed in different cores. However, if the CPU usage is high, the first process and the second process may be processed in the same core.

10 is a flow chart for processing a memory synchronization task.

Referring to FIG. 10, the second process determines whether the connection information exists. If there is no connection information, since the update has not occurred, the second process is switched to the standby state. On the other hand, if there is connection information, the update page is reflected on the original page. At this time, the 'reading and locking of the connection information', 'locking the original page', 'copying the update page to the original page' and 'unlocking the original page' shown in FIG. Therefore, the repetitive description will be omitted.

On the other hand, the asynchronous method differs from the synchronous method in the deletion of the connection information and the update page shown in FIG. This is because the update page may be used by another process.

Accordingly, the second process deletes the connection information when the update page is in use, and when the update page is not in use, adds the update page to the delete queue and deletes the connection information. The second process may terminate the memory synchronization when there is no update page in the delete queue, and may delete the update page and terminate the memory synchronization if there is an update page in the delete queue.

Described embodiments can be implemented in the form of a computer program written on a recording medium readable by a computer to perform each step of the memory mapping file update method described above.

The embodiments and the accompanying drawings described in the present specification are merely illustrative of some of the technical ideas included in the present invention. Accordingly, the embodiments disclosed herein are for the purpose of describing rather than limiting the technical spirit of the present invention, and it is apparent that the scope of the technical idea of the present invention is not limited by these embodiments. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

100: Computer system
120: Processor
121: Core
122: Cache
140: Nonvolatile memory
142: Process address space
144: Page table
146: File system area

Claims (11)

A computer system, comprising: a non-volatile memory having a process address space, a page table and a file system area; a first process for processing file update processing; and a second process for processing memory synchronization, A method for updating a source file mapped to a source file,
(a) allocating a new update page to the file system area when an update occurs on a page of the original file, and copying the contents of a page where the update of the original file occurred to the update page ;
(b) changing, by the first process, a mapping address of a page in the original file stored in the page table in which the update occurs, to a mapping address of the update page;
(c) storing, in the process address space, connection information in which the first process describes the connection relationship between the page where the update occurred and the update page among the pages of the original file, and storing the update content in the update page ;
(d) the first process locks the original file and the update page by a memory synchronization request and returns the memory synchronization request;
(e) when the second process is storing the connection information, locking the original file and copying the update page to a page of the original file where the update occurred; And
(f) changing, by the second process, the mapping address of the page table to a page where the update occurred in the page of the original file, and releasing the lock setting of the original file,
The non-volatile memory having a general area;
The step (c) may include: the first process storing the connection information in the general area;
Wherein the step (d) further comprises, after the first process locks the original file, storing the connection information in the general area to the file system area Way. delete delete The method according to claim 1,
(d-1) storing the data stored in the cache in the file system area by the first process. The method according to claim 1,
The step (f) includes deleting the connection information when the update page is in use, adding the update page to the delete queue and deleting the connection information when the update page is not in use ; And
And terminating the memory synchronization when the second process does not include the update page in the deletion wait queue and deleting the update page when the update page exists in the deletion wait queue and terminating the memory synchronization To update the memory mapping file. A computer program recorded on a computer-readable recording medium, the computer program being written in the computer to perform the steps of the memory mapping file update method according to any one of claims 1 to 4. A nonvolatile memory having a process address space, a page table, and a file system area, a first process for processing file update processing, and a second process for processing memory synchronization are stored in the file system area, As an apparatus for updating,
Wherein the first process allocates a new update page to the file system area when an update occurs on a page of the original file, copies the contents of the page where the update of the original file occurred to the update page, A mapping address of a page of the original file stored in the table is changed to a mapping address of the update page and a link indicating a connection relationship between the page where the update occurs and the update page Storing the update information in the update page, locking the original file and the update page by a memory synchronization request, returning the memory synchronization request,
Wherein the second process locks the original file when the connection information is stored and copies the update page to a page where the update occurred among the pages of the original file, Changing the page of the file to the page where the update occurred, and unlocking the original file;
The non-volatile memory having a general area;
Wherein the first process stores the connection information in the general area;
Wherein the first process stores the connection information in the file system area in the general area after locking the original file. delete delete The method of claim 7,
Wherein the first process stores data stored in the cache in the file system area. The method of claim 7,
Wherein the second process deletes the connection information when the update page is in use and adds the update page to the delete queue and deletes the connection information when the update page is not in use, Wherein when the update page does not exist in the delete queue, the memory synchronization is terminated, and when the update page exists in the delete queue, the update page is deleted and the memory synchronization is terminated.

Images