KR101750744B1 - Apparatus and Method for Operating Write-Ahead-Logging - Google Patents

Apparatus and Method for Operating Write-Ahead-Logging Download PDF

Info

Publication number
KR101750744B1
KR101750744B1 KR1020150148998A KR20150148998A KR101750744B1 KR 101750744 B1 KR101750744 B1 KR 101750744B1 KR 1020150148998 A KR1020150148998 A KR 1020150148998A KR 20150148998 A KR20150148998 A KR 20150148998A KR 101750744 B1 KR101750744 B1 KR 101750744B1
Authority
KR
South Korea
Prior art keywords
data
storage medium
area
copy command
log
Prior art date
Application number
KR1020150148998A
Other languages
Korean (ko)
Other versions
KR20170048084A (en
Inventor
이은지
반효경
한유일
김백엽
Original Assignee
충북대학교 산학협력단
이화여자대학교 산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 충북대학교 산학협력단, 이화여자대학교 산학협력단 filed Critical 충북대학교 산학협력단
Priority to KR1020150148998A priority Critical patent/KR101750744B1/en
Publication of KR20170048084A publication Critical patent/KR20170048084A/en
Application granted granted Critical
Publication of KR101750744B1 publication Critical patent/KR101750744B1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0804—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches with main memory updating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1405—Saving, restoring, recovering or retrying at machine instruction level
    • G06F11/141—Saving, restoring, recovering or retrying at machine instruction level for bus or memory accesses
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0866—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches for peripheral storage systems, e.g. disk cache
    • G06F12/0871—Allocation or management of cache space

Abstract

The present invention discloses an apparatus and method for pre-write logging. The pre-write logging operation apparatus according to an embodiment of the present invention has a log area for storing data to be updated according to a commit operation and a storage space logically divided into a data area for storing original data, A storage medium providing the storage medium; And a controller for instructing an internal copy command to update the data to be updated to the original data position of the data area at the time of the checkpoint operation to the storage medium.

Description

[0001] Apparatus and Method for Operating Write-Ahead-Logging [

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data logging technique, and more particularly, to a pre-write logging operation apparatus and method including a commit and a checkpoint operation.

In general, the file system uses the Write-Ahead-Logginf technique to prevent data consistency from being corrupted by unexpected errors.

Specifically, when an update (change) of data occurs in an atomic transaction, the change is not directly applied to the original data, but is recorded in the log area. When the transaction is completed, the change is reflected in the original data.

A conventional file system is a hierarchical memory structure composed of a main memory and a storage of a host. In the storage area, an area in which original data is stored and an area in which a log file is stored are allocated in a divided manner.

In this way, the main memory is basically managed by the operating system (kernel) in the form of a page cache, and utilizes a part of the storage area as a unique buffer area. At this time, the buffer area is used to load the data to be updated before updating the data. Since the buffer area is divided and managed in units of pages, the loaded data is also loaded in page units.

Modification of the original data in the file system is performed by dividing into two steps: commit and checkpoint.

First, the commit means writing the changed page of the operating system to the buffer area of the kernel page cache (DB buffer), and when the recording is completed, the log file of the secondary storage (log file) file) is recorded again.

Next, the point at which the data in the log file is written directly to the data source file is called a checkpoint.

When a checkpoint occurs, the operating system loads the log file of the secondary storage into the buffer area of the kernel page cache (DB buffer) and writes it again to the file of the secondary storage (Checkpoint operation).

When attempting to load log file data into main memory, the operating system must read the remaining data from main memory back into main memory via physical physical storage access. This series of processes leads to double input / output to the secondary storage, causing considerable overhead. In other words, although the log file data is in the secondary storage at the checkpoint time, the operating system loads the log file data into the buffer area of the kernel page cache (DB buffer) And then re-read it to write to the secondary storage.

Accordingly, the conventional file system may cause problems such as data access speed deterioration, power consumption, and cache pollution due to the inefficiency.

In this way, there is a problem that inefficiency occurs because the journaling technique is used in the file system.

Korean Patent Laid-Open Publication No. 2014-0118519 (published date October 08, 2014)

SUMMARY OF THE INVENTION It is an object of the present invention to provide a pre-write logging operation apparatus and method using an improved check point operation.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

The pre-write logging operation apparatus according to an embodiment of the present invention has a log area for storing data to be updated according to a commit operation and a storage space logically divided into a data area for storing original data, A storage medium providing the storage medium; And a controller for instructing an internal copy command to update the data to be updated to the original data position of the data area at the time of the checkpoint operation to the storage medium.

The internal copy command may include a first logical block address at which the data to be updated is stored, a second logical block address at the original data location, and information of original data corresponding to the data to be updated.

The storage medium may update the original data stored in the second logical block address by copying the updated data stored in the first logical block address to the second logical block address.

The controller may wait until the copying completion is received from the storage medium after transmitting the internal copy command, or may perform another transaction from the storage medium without waiting for the completion of the copying.

On the other hand, when the controller receives the failure report for the internal copy command from the storage medium, the controller can retransmit the internal copy command.

According to another aspect of the present invention, there is provided a pre-write logging operation method using a storage medium having a self-copy function by a control unit, the method comprising: Confirming the address of the original data in the data area of the data area; And transmitting an internal copy command for instructing a data copy from the storage address of the data to be updated to the storage address of the original data to the storage medium, wherein the internal copy command includes data in one address of the storage medium To another address, to the other address.

The internal copy command may include a logical block address of the data to be updated in the log area, a logical block address of the original data in the data area corresponding to the logical block address, and a copy command thereof.

In addition, the above-described steps may include waiting after transmitting the command and reporting completion of copying from the storage medium; Or performing another transaction without waiting for a copy completion report from the storage medium after transmitting the command.

According to the present invention, it is possible to reduce the software load at the time of check point calculation.

FIG. 1 illustrates a conventional pre-write logging calculation process. FIG.
BACKGROUND OF THE INVENTION 1. Field of the Invention [0001]
FIG. 3A illustrates a commit operation process according to an embodiment of the present invention; FIG.
FIG. 3B illustrates a checkpoint calculation process according to an embodiment of the present invention; FIG.
FIG. 4 is a flowchart illustrating a pre-write logging operation method according to an embodiment of the present invention; FIG.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, advantages and features of the present invention and methods of achieving them will be apparent from the following detailed description of embodiments thereof taken in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. As used herein, the terms " comprises, " and / or "comprising" refer to the presence or absence of one or more other components, steps, operations, and / Or additions.

Embodiments of the present invention will now be described in detail with reference to the accompanying drawings. FIG. 3 is a flowchart illustrating a commit operation process according to an embodiment of the present invention, and FIG. 3B is a flowchart illustrating a write operation according to an embodiment of the present invention. And FIG.

2, the pre-write logging computing device according to an embodiment of the present invention includes a storage medium 230, a main memory 220, and a control unit 210. [ Here, the apparatus for performing the pre-write logging operation may be a file system using a journaling technique.

The storage medium 230 includes a data area 232 in which original data is stored and a log area 231 in which log data is stored. Here, the log data may be data in which the content of the data to be updated including the content to be changed in the original data is recorded in page units.

The storage medium 230 may be storage means for providing a copy function between internal logical blocks according to an internal copy command.

For example, the storage medium 230 may be a data storage device based on a hard disk, a solid state disk (SSD), or a flash memory, including a processor capable of performing a copy function between internal logical blocks.

The main memory 220 is a memory including the buffer area 221 and may be a volatile element (or a nonvolatile element) capable of high-speed random access such as RAM.

Here, the buffer area 221 may be an area used for changing the original data in the data area 232 of the storage medium 230. Normally, the buffer area 221 is also referred to as a kernel page cache.

The control unit 210 may be at least one processing unit that performs a transaction including a commit operation and a checkpoint operation. The control unit 210 may include at least one processing unit. For example, the processing unit may include a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, an application specific integrated circuit (ASIC), field programmable gate arrays (FPGA) Lt; / RTI > core. The memory may be a volatile memory (e.g., RAM, etc.), a non-volatile memory (e.g., ROM, flash memory, etc.), or a combination thereof.

Here, the control unit 210 may perform a pre-write logging operation by executing an operating system or an application program.

At this time, the control unit 210 performs a write transaction to update the original data in the storage medium 230 in two steps of a commit operation and a checkpoint operation.

3A, when the commit operation is performed, the controller 210 loads the changed contents of the storage medium 230 into the buffer area 221 of the main memory 220 in units of pages, 221), it is corrected to the new requested data. If a request is made to permanently store data after a predetermined time has elapsed since the data change, the control unit 210 performs a commit operation. The control unit 210 requests the contents of the buffer area 221 to be written into the log area 231 in the storage medium 231 according to the commit operation. This is the same as or similar to the conventional commit operation, so that further description thereof will be omitted.

Here, the controller 210 determines a logical block address (eg, a unit of a sector) of the original data and a logical block address of data to be updated in the log area 231 before or during the commit operation .

At the time of a check point calculation for updating (applying) the data of the log area 231 to the position of the original data of the data area 232 after the completion of the commit operation, the control unit 210 reads the log area 231 ) To the storage medium 230 to update the data to be updated to the original data position of the data storage area.

The internal copy command transmitted to the storage medium 230 includes a logical block address A1 of the data to be updated, a logical block address A2 of the original data, a correspondence relationship between the data to be updated and the original data Address A1 and logical block address A2) (Fig. 3B).

For example, when a, b, and c stored in the first to third logical block addresses in the log area 231 are updated to A, B, and C stored in the fourth to sixth logical block addresses in the data area 232 , The controller 210 copies the data a of the first address to the fourth logical block address, copies the data b of the second address to the fifth logical block address, and writes the data c of the third address to the sixth logical block address The internal copy command may be transmitted to the external device.

As described above, according to the embodiment of the present invention, the log data stored in the storage medium is written back to the main memory at the time of the conventional checkpoint operation using the internal storage copying operation at the time of the checkpoint operation and then transferred to the storage medium, It is possible to improve the problem that traffic is generated.

In addition, the embodiment of the present invention improves storage performance by reducing unnecessary I / O traffic in the check point calculation, reduces energy consumption used for data transmission and reception, and reduces software overhead.

Hereinafter, a pre-write logging operation method according to an embodiment of the present invention will be described with reference to FIG. 4 is a flowchart illustrating a pre-write logging operation method according to an embodiment of the present invention.

Referring to FIG. 4, the controller 210 determines whether a write transaction requiring update of data stored in the file system occurs (S410).

When a write transaction occurs, the control unit 210 loads the data to be changed into the buffer area 221 of the main memory 220 in units of pages (S420).

When the data to be changed is loaded in the buffer area 221 of the main memory 220, the data is changed according to the write request. When a predetermined time has elapsed or an explicit commit request is generated, To the log area 231 of the storage medium 230 at step S430.

When the data in the buffer area 221 is written into the log area 231, the control unit 210 changes the data to be changed in the buffer area 221 to the original data position in the data area 232 To the storage medium 230 (S440). At this time, the check point can be executed after the writing completion time in the log area 231, and may be executed after a predetermined time.

Thereafter, the control unit 210 waits for a response (command execution result) corresponding to the internal copy command from the storage medium 230 (S450). Here, the control unit 210 can perform another operation (asynchronous method) or wait (synchronous manner) without performing another operation until it receives report of completion of copying from the storage medium 230. [

If the received command execution result indicates failure, the controller 210 determines that the write transaction has failed and retransmits the internal copy command to the storage medium 230. At this time, the control unit 210 may re-execute from the commit operation.

As described above, the embodiment of the present invention improves the performance of the storage medium and reduces the energy consumption and the software overhead as the data input / output traffic is reduced in the checkpoint calculation.

While the present invention has been described in detail with reference to the accompanying drawings, it is to be understood that the invention is not limited to the above-described embodiments. Those skilled in the art will appreciate that various modifications, Of course, this is possible. Accordingly, the scope of protection of the present invention should not be limited to the above-described embodiments, but should be determined by the description of the following claims.

210: control unit 220: main memory 230: storage medium
221: buffer area 231: log area 232: data area

Claims (7)

In a pre-write logging computing device in a file system that stores and manages data in file form,
A main memory which is a memory including a buffer area;
A storage medium having a storage area logically divided into a log area in which log data is stored and a data area in which original data is stored and a copy function between internal logical blocks according to an internal copy command; And
When a write transaction occurs, new data is loaded in a buffer area of the main memory on a page-by-page basis, and new data of the buffer area is sequentially written to the log area of the storage medium according to a commit operation To the storage medium, an internal copy command instructing to change the position of the new data in the log area to the original data position in the data area according to a checkpoint operation when the commit operation is completed, / RTI >
The control unit recognizes a first logical block address (LBA) of the new data in the log area and a second logical block address of the original data, and determines the first logical block address And transmits the internal copy command to the storage medium in such a manner that new data is copied to the second logical block address. delete 2. The apparatus of claim 1,
Wherein the controller is configured to wait until the copying completion is reported from the storage medium after transmitting the internal copy command or to perform another transaction without waiting for the report of completion of copying from the storage medium. 2. The apparatus of claim 1,
And upon receipt of a failure report for the internal copy command from the storage medium, retransmits the internal copy command. delete delete delete
KR1020150148998A 2015-10-26 2015-10-26 Apparatus and Method for Operating Write-Ahead-Logging KR101750744B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020150148998A KR101750744B1 (en) 2015-10-26 2015-10-26 Apparatus and Method for Operating Write-Ahead-Logging

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020150148998A KR101750744B1 (en) 2015-10-26 2015-10-26 Apparatus and Method for Operating Write-Ahead-Logging

Publications (2)

Publication Number Publication Date
KR20170048084A KR20170048084A (en) 2017-05-08
KR101750744B1 true KR101750744B1 (en) 2017-06-23

Family

ID=59283391

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020150148998A KR101750744B1 (en) 2015-10-26 2015-10-26 Apparatus and Method for Operating Write-Ahead-Logging

Country Status (1)

Country Link
KR (1) KR101750744B1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11556268B2 (en) 2021-04-22 2023-01-17 Western Digital Technologies, Inc. Cache based flow for a simple copy command
US11733920B2 (en) 2020-09-10 2023-08-22 Western Digital Technologies, Inc. NVMe simple copy command support using dummy virtual function

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102098253B1 (en) * 2018-02-13 2020-04-07 재단법인대구경북과학기술원 System and method for large-scale frequent pattern mining on distributed systems
KR102132387B1 (en) * 2018-10-19 2020-07-09 한양대학교 산학협력단 Method and apparatus for logging based on barrier

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11733920B2 (en) 2020-09-10 2023-08-22 Western Digital Technologies, Inc. NVMe simple copy command support using dummy virtual function
US11556268B2 (en) 2021-04-22 2023-01-17 Western Digital Technologies, Inc. Cache based flow for a simple copy command

Also Published As

Publication number Publication date
KR20170048084A (en) 2017-05-08

Similar Documents

Publication Publication Date Title
US11216214B2 (en) Memory system and operation method thereof
KR102395360B1 (en) A device controller and method for performing a plurality of write transactions atomically within a nonvolatile data storage device
KR102403034B1 (en) Dynamic garbage collection p/e policies for redundant storage blocks and distributed software stacks
CN109426619B (en) Method for accessing flash memory module, related flash memory controller and electronic device
US9244839B2 (en) Methods and apparatus for supporting persistent memory
KR101750744B1 (en) Apparatus and Method for Operating Write-Ahead-Logging
US9940201B2 (en) Disk array having mirror configuration and rebuilding method therefor
US20130111103A1 (en) High-speed synchronous writes to persistent storage
EP3446221B1 (en) Adapted block translation table (btt)
TW202001573A (en) Memory system
US10769074B2 (en) Computer memory content movement
US20140304487A1 (en) Information processing apparatus, memory control device, and data transfer control method
JP4745465B1 (en) Semiconductor memory device and method for controlling semiconductor memory device
US20170160940A1 (en) Data processing method and apparatus of solid state disk
CN111026325B (en) Flash memory controller, control method of flash memory controller and related electronic device
JP4892812B2 (en) Cache control and data processing system and processing program therefor
WO2018139223A1 (en) Information processing device, control program and information processing method
US20210271600A1 (en) Data storage device and operating method thereof
US10714176B2 (en) Read-write cycle execution based on history
WO2023245942A1 (en) Ssd finite window data deduplication identification method and apparatus, and computer device
CN104978280B (en) Data storage system and specific instruction execution method thereof
JP2011175666A (en) Semiconductor storage device, and method for controlling the same
CN108984117B (en) Data reading and writing method, medium and equipment
WO2015141219A1 (en) Storage system, control device, memory device, data access method, and program recording medium
JP2013109404A (en) Information processing device

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E701 Decision to grant or registration of patent right