KR102049417B1 - Data storing and restoring method based on In-memory database using NVDIMM - Google Patents

Abstract

The present invention relates to an in-memory database. More specifically, the main memory is implemented as a non-volatile storage medium (Non Volatile Dual In-line Memory Module), and a database image and UNDO log stored in the main memory during data recovery. By using only, it is possible to perform data restoration more quickly, as well as a technology that can improve system performance by minimizing disk I / O.
In the in-memory database-based data storage and restore method using the NVDIMM according to the present invention, in addition to storing the data in the main memory consisting of NVDIMM in the operating device, the UNDO log of the transaction corresponding to the change data is loaded on the disk, UN In the DO log, the database storage step of additionally recording and storing the location information of the NVDIMM in which the corresponding UNDO target data is stored, and capturing a database image by capturing the last working page stored in main memory during system recovery, is included in the UNDO log. A database restoration step of performing data restoration processing by using the database image obtained from main memory and the UNDO log by acquiring transaction UNDO log information corresponding to the final working position of the database image from the disk based on the location information of the NVDIMM; So It is characterized in that the configuration.

Description

Data storing and restoring method based on In-memory database using NVDIMM}

The present invention relates to an in-memory database. More specifically, the main memory is implemented as a non-volatile storage medium (Non Volatile Dual In-line Memory Module), and a database image and UNDO log stored in the main memory during data recovery. By using only, it is possible to perform data restoration more quickly, as well as a technology that can improve system performance by minimizing disk I / O.

As the demand for big data increases and rapid analysis processing increases, the research on in-memory database is actively being conducted.

In-Memory Database is a database that builds and manages a permanent database in main memory. In other words, by managing part or all of the database in the main memory, it is a technology that enables efficient transaction processing by processing data directly with only memory access without access to the disk.

In general, a database management system (DBMS) performs logging of data changes to ensure the stability of the database. Logging is the basic function of DBMS to record the insertion / deletion / change of the data in real time in a safe storage such as disk, and to utilize it in case of failure to return to the final database state.

Since in-memory database is built using volatile memory as main memory, if a failure occurs, the data stored in main memory is lost. Therefore, disk I / O is inevitable due to logging.

1 is a view for explaining the structure of an in-memory database system, and includes an operating device 10 and a main memory 20 and a disk 30 formed of a volatile storage medium such as a RAM.

The operating device 10 basically stores the data in the main memory 20, and also performs a logging operation to write down the changed record when the data is changed to the disk 30, and the data stored on the main memory 20 at regular intervals. By performing a checkpoint operation that writes down the backup data to the disk 30, the data is preserved while maintaining the performance of the database.

At this time, the logging operation automatically generates a redo log for redo and an UNDO log for reversion in the main memory 20 in response to the working state of the data, and stores them in the disk 30.

In addition, when restarting after the shutdown due to a normal shutdown or failure, the operating device 10 loads the backup data of the last state stored as a checkpoint in the main memory, and then changes the changes that have occurred since the last checkpoint through a log that is a change record. By reflecting, you restore the last state just before the database shuts down. This is to ensure the durability of the database that once a transaction is successfully completed (COMMIT), it should ensure that the transaction processing results are reflected in the database under any system error.

At this time, the operating apparatus 10 maintains the log (REDO) and the log cancel (UNDO) that the user has completed the storage separately to perform a series of operations occurring since the last checkpoint in the reverse order to recover.

2 is a view for explaining a data restoration process in a general in-memory database, (A) is a database image of the driving state just before termination, (B) is a database image of the redrive state, (C) Database image in a restored state.

Referring to FIG. 2, (A) is a database image immediately before the termination of the database, and shows a state in which the database is terminated due to a system failure. That is, the UNDO target data and the REDO target data for the working data exist in the main memory 20.

At this time, the backup data 31 is stored in the previous storage state, that is, the data of the committed state, the database image finally stored in the backup data 31 is the UNDO target data and RE target data as shown in (B) is lost It becomes a state.

Accordingly, the operating apparatus restores the REDO data area using the REDO log, and then returns to the transaction state just before the termination using the UNDO log, thereby restoring the database image to the state of completion just before the termination as shown in (C). Restore

Therefore, in the conventional in-memory system, as shown in FIG. 1, the backup data 31, the REDO log 32, and the UNDO log 33 must be stored in the disk 20, respectively, and a large number of disk I / Os are essential. Is caused by.

However, the generation of I / O to the disk, which is the slowest medium, introduces a delay in running the database, resulting in a problem of degrading the overall system performance.

Prior Art 1 (Korean Patent Publication No. 10-2017-0089194) discloses a NVDIMM for temporarily backing up a device drive for controlling a memory disk and peripheral devices used as main memory and a table working on the memory disk, and data of the NVDIMM. Disclosed is a memory disk configuration apparatus using a nonvolatile dual inline memory module (NVDIMM) including an HDD / SDD for permanently storing the data.

However, the aforementioned document 1 improves the read / write performance of the disk by transferring the use of HDD / SDD with a slow data read / write speed to some NVDIMMs. Since REDO logs are required, there is still a problem of system performance.

1. Korean Patent Publication No. 10-2017-0089194 (Name: Memory disk driving method and apparatus using a nonvolatile dual inline memory module (NVDIMM))

Accordingly, the present invention has been made in view of the above-described circumstances, and a database is constructed by configuring a NVDIMM (Volatile Dual In-line Memory Module) as a nonvolatile storage medium as a main memory, and logging only UNDO logs to disk to disk I Its technical objective is to provide an in-memory database-based data storage and retrieval method using NVDIMM that can reliably guarantee data persistence while minimizing / O and to perform data restoration operations quickly.

According to an aspect of the present invention for achieving the above object, in addition to storing data in the main memory consisting of non-volatile memory nonvolatile memory (NVDIMM) in the operating device, the transaction corresponding to the change data Only load UNDO logs to disk, but UNDO In the log, the database storage step of additionally recording and storing the location information of NVDIMM corresponding to the target UNDO target data, and the data restoration process using only the last stored information in main memory and the UNDO log stored on disk during system recovery by the operating system The database restoration step includes performing a database restoration step, wherein the database restoration step captures a work page including REDO target data and UNDO target data finally stored in main memory to obtain a database image, and backs up data for database restoration processing. In the first step of setting the backup data and the backup target data, REDO target data is not restored, and transaction UNDO log information corresponding to the final working position of the backup data is stored based on the location information of the NVDIMM included in the UNDO log. Obtained from The method for storing and restoring data based on in-memory database using NVDIMM is configured to include a second step of performing a restore operation to a previously committed state by using an UNDO log only for UNDO target data. Is provided.

In addition, the database storage step is in the state that the main memory is mounted in the RAM slot, the operating device is activated in the DAX mode to operate the in-memory database-based data storage using NVDIMM, characterized in that to operate directly A restoration method is provided.

According to the present invention, since the main memory is composed of NVDIMM, which is a nonvolatile storage medium, and the database image of the last storage state stored in the NVDIMM is used as the data for restoring as it is, restoring of the redo log is unnecessary, so that data restoration can be performed more quickly. Not only can it be done, it is possible to improve system-wide performance by generating only a minimum disk I / O for storing UNDO logs on disk.

1 is a diagram for explaining the structure of a general in-memory database system.
FIG. 2 is a diagram for explaining a data restoration process in the in-memory database system shown in FIG.
3 is a diagram for explaining the configuration of an in-memory database system using NVDIMM to which the present invention is applied;
4 is a diagram illustrating an in-memory database-based data storage and restoration process using NVDIMMs according to a first embodiment of the present invention.
FIG. 5 is a diagram for explaining a data restoration process in FIG. 4; FIG.

Hereinafter, with reference to the accompanying drawings will be described in detail the present invention. It should be noted that the same elements in the figures are denoted by the same reference signs wherever possible. On the other hand, the terms or words used in the present specification and claims are not to be construed as limiting the ordinary or dictionary meanings, the inventors should use the concept of the term in order to explain the invention in the best way. It should be interpreted as meanings and concepts corresponding to the technical idea of the present invention based on the principle that it can be properly defined. Therefore, the embodiments described in the present specification and the configuration shown in the drawings are only the most preferred embodiments of the present invention, and do not represent all of the technical ideas of the present invention, and various alternatives may be substituted at the time of the present application. It should be understood that there may be equivalents and variations.

3 is a view for explaining the configuration of an in-memory database system using NVDIMM to which the present invention is applied.

As shown in FIG. 3, the in-memory database system using the NVDIMM to which the present invention is applied includes an operating device 100 and a main memory 200 and a disk 300 formed of NVDIMMs.

In the present invention, the main memory 200 is implemented using a non-volatile storage medium (Non Volatile Dual In-line Memory Module), and the disk 300 stores only UNDO logs.

The disk 300 is a backup database, and stores only the UNDO log (return log) automatically generated by a transaction in the main memory 200.

In this case, the main memory 200, that is, the NVDIMM may be mounted in a RAM slot of an existing in-memory database system as a hardware.

In this case, in order to use the NVDIMM as the main memory, the operating apparatus 100 must be activated in the direct access storage (DAX) mode to access the NVDIMM through the existing RAM slot.

The operating apparatus 100 is a processor for operating the database as a whole and may be provided in the main memory 100 in hardware to form an in-memory database system.

The operating apparatus 100 basically stores data in the main memory 200, and captures and stores UNDO logs of the changed data, that is, the committed transactions, on the disk 300.

The UNDO log is a copy of the original data before the change, which is captured for every transaction that changes the data and is preserved at least until the transaction ends. This UNDO data supports read consistency and Flashback Query, which prevents a failed transaction recovery if a transaction fails, for example, because the user session terminates abnormally before the user decides to commit or roll back the transaction. Used for.

At this time, the operating apparatus 100 is configured to include a memory map manager 110 for recording the NVDIMM position of the data to be UNDO in the UNDO log for synchronization between the current drive position of the main memory 200 and the UNDO log.

In addition, the operating apparatus 100 continuously monitors the occurrence of a failure and performs a restoration process on the main memory 200 when the failure occurs or when the system is restarted after the shutdown.

In this case, the operating apparatus 100 performs a database restoration process by using the database image immediately before the termination stored in the main memory 200 and the UNDO log stored in the disk 300.

That is, in the present invention, another big feature is that the database image stored in the main memory 200 is used as the database image for restoration.

4 is a diagram illustrating an in-memory database-based data storage and restoration method using NVDIMM according to the present invention.

The present invention is applied to an in-memory database system in which the CPU 100 is located in the main memory 200, and the CPU 100 performs overall control of the in-memory database system.

Referring to FIG. 4, the in-memory database system includes a data storage process ST100 and a data restoration process ST200.

At this time, the data storage process ST100 is repeatedly performed by driving the database, and the data restoration process ST200 is performed when a failure occurs or when the system is restarted after the shutdown. Hereinafter, a process of restoring data when a failure occurs will be described.

Looking at the data storage process (ST100),

First, the operating apparatus 100 is activated in the DAX mount mode for using the NVDIMM which is the main memory 200 installed in the existing RAM slot (ST110). DAX mode is a communication protocol applied to support the special environment of the NVDIMM in the operating device 100 is applied, and provides an environment in which the operating device 100 can directly access the data in bytes to the NVDIMM.

In the above state, the operating apparatus 100 performs a series of data storage operations for storing data in the main memory 200, that is, NVDIMM in response to the data operation request (ST120). At this time, the data is read from the NVDIMM or the data is written to the NVDIMM using the pmemory library, and the data storage includes all data change operations such as writing / modifying / deleting data into the main memory 200, that is, the NVDIMM. .

In addition, a transaction is generated in response to a data change of the main memory 200, and an UNDO log is automatically generated whenever each query included in the transaction is processed, and the UNDO log is stored on the disk 300. Logging operation is performed in real time (ST130). At this time, the UNDO log is stored in the disk 300 with the NVDIMM location of the data subject to UNDO being automatically included.

That is, only UNDO logs are loaded on the disk 300. Accordingly, both the backup data, the REDO log, and the UNDO log are loaded on the conventional disk, and the number of disk I / Os can be significantly reduced as compared to the process of storing a separate backup database image.

Next, the data restoration process (ST100) will be described.

If the operating apparatus 100 continuously monitors the operation of the database and a failure occurs while performing the data storage process as described above (ST210), the operating apparatus 100 stores the main memory 200 stored in a state immediately before the failure occurs. Capture and obtain a database image (ST220). At this time, since the main memory 200 is implemented as NVDIMM, which is a nonvolatile storage medium, data of a previous working state is maintained as it is when the system is shut down due to a failure.

In addition, the operating apparatus 100 obtains a UNDO log for a transaction that has finished execution due to a failure in the disk 300 (ST230).

Subsequently, the operating apparatus 100 performs data restoration using the database image acquired from the main memory 200 and the UNDO log for the last committed transaction (ST240).

5 is a diagram illustrating a recovery process of an in-memory database using NVDIMM. (X) is a database image stored in the main memory 200 immediately before termination due to a failure, (Y) is a database image in a restarted state, and (Z) is a database image in a restored state.

In the main memory 200, as shown in FIG. 5 (X), the data of the last state immediately before the end is always stored in response to the driving of the database, and the database image includes the REDO target block corresponding to the current work position with respect to the state immediately before the end, UNDO target block exists. At this time, only the UNDO log corresponding to the finally committed transaction for the UNDO target block is stored in the disk 300.

When the database is re-driven in the above state, the data for the REDO target block and the UNDO target block are maintained as shown in FIG.

That is, in the present invention, since the data finally stored in the main memory 200 is used as the backup data as it is, there is no need to perform a restore operation on the REDO target block.

However, in the present invention, only the revert restore operation to the previously committed state is performed using the UNDO log for the last stored transaction in which the commit is not completed when the drive is terminated.

That is, according to the above embodiment, since the final state immediately before recovery is always stored by using the NVDIMM which is a nonvolatile storage medium, the database image of the last storage state stored in the NVDIMM is used as the restoration data as it is. During in-memory database recovery, it is not necessary to load the last checkpoint into memory or to restore the redo logs performed since the last checkpoint.

Therefore, in the present invention, it is not necessary to restore the redo log so that the data restoration can be performed more quickly, and the system-wide performance is improved due to the minimum disk I / O occurrence that stores only the UNDO log on the disk. It is possible.

100: operating device, 110: memory map manager,
200: main memory (NVDIMM), 300: disk.

Claims (2)

In addition to storing data in the main memory consisting of non-volatile memory, Non Volatile Dual In-line Memory Module (NVDIMM), the operating system loads only UNDO logs of transactions corresponding to the changed data onto disk. A database storage step of additionally recording and storing the location information of the NVDIMM corresponding to the UNDO target data;
When the system recovers from the operating system, it includes a database restoration step of performing data restoration processing using only the information last stored in the main memory and the UNDO log stored on the disk.
The database restoration step may include a first step of capturing a work page including REDO target data and UNDO target data finally stored in main memory to obtain a database image, and setting it as backup data for database restoration processing;
Only the UNDO target data can be obtained from the disk by acquiring the transaction UNDO log information corresponding to the final working position of the backup data based on the location information of the NVDIMM included in the UNDO log. An in-memory database-based data storage and restoration method comprising a second step of performing a restore operation to a previously committed state by using the UNDO log.
The method of claim 1,
In the database storing step, an in-memory database-based data storage and restoration method using an NVDIMM, wherein the operating device is activated in a DAX mode and directly accesses the main memory while the main memory is mounted in a RAM slot. .

Images