KR20170048084A - Apparatus and Method for Operating Write-Ahead-Logging - Google Patents

Apparatus and Method for Operating Write-Ahead-Logging Download PDF

Info

Publication number
KR20170048084A
KR20170048084A KR1020150148998A KR20150148998A KR20170048084A KR 20170048084 A KR20170048084 A KR 20170048084A KR 1020150148998 A KR1020150148998 A KR 1020150148998A KR 20150148998 A KR20150148998 A KR 20150148998A KR 20170048084 A KR20170048084 A KR 20170048084A
Authority
KR
South Korea
Prior art keywords
data
storage medium
updated
address
area
Prior art date
Application number
KR1020150148998A
Other languages
Korean (ko)
Other versions
KR101750744B1 (en
Inventor
이은지
반효경
한유일
김백엽
Original Assignee
충북대학교 산학협력단
이화여자대학교 산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 충북대학교 산학협력단, 이화여자대학교 산학협력단 filed Critical 충북대학교 산학협력단
Priority to KR1020150148998A priority Critical patent/KR101750744B1/en
Publication of KR20170048084A publication Critical patent/KR20170048084A/en
Application granted granted Critical
Publication of KR101750744B1 publication Critical patent/KR101750744B1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0804—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches with main memory updating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1405—Saving, restoring, recovering or retrying at machine instruction level
    • G06F11/141—Saving, restoring, recovering or retrying at machine instruction level for bus or memory accesses
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0866—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches for peripheral storage systems, e.g. disk cache
    • G06F12/0871—Allocation or management of cache space

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Retry When Errors Occur (AREA)

Abstract

The present invention discloses an apparatus and method for pre-write logging. The pre-write logging operation apparatus according to an embodiment of the present invention has a log area for storing data to be updated according to a commit operation and a storage space logically divided into a data area for storing original data, A storage medium providing the storage medium; And a controller for instructing an internal copy command to update the data to be updated to the original data position of the data area at the time of the checkpoint operation to the storage medium.

Description

[0001] Apparatus and Method for Operating Write-Ahead-Logging [

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data logging technique, and more particularly, to a pre-write logging operation apparatus and method including a commit and a checkpoint operation.

In general, a database or file system uses the Write-Ahead-Logginf technique to prevent corruption of data consistency due to unexpected errors.

Specifically, when an update (change) of data occurs in an atomic transaction, the change is not directly applied to the original data, but is recorded in the log area. When the transaction is completed, the change is reflected in the original data.

A conventional database system is a hierarchical memory structure composed of a main memory and a storage of a host. In the storage area, an area in which original data is stored and an area in which a log file is stored are allocated in a divided manner.

In this way, the main memory is basically managed by the operating system (kernel) in the form of a page cache, and utilizes a part of the storage area as a unique buffer area. At this time, the buffer area is used to load the data to be updated before updating the data in the database. Since the buffer area is managed in units of pages, the loaded data is also loaded in page units.

In the database system, modification of the original data is performed in two steps: commit and checkpoint.

First, the commit means writing the changed page of the operating system into the database buffer area of the kernel page cache, and when the recording is completed, the log file of the storage area of the secondary storage Log file) is recorded again.

Next, the point at which the data in the log file is written directly to the data source file is called a checkpoint.

When a checkpoint occurs, the operating system loads the log file of the secondary storage into the database buffer area of the kernel page cache and then writes the database file of the secondary storage ) (Check point operation).

When attempting to load log file data into main memory, the operating system must read the remaining data from main memory back into main memory via physical physical storage access. This series of processes leads to double input / output to the secondary storage, causing considerable overhead. In other words, although the log file data is in the secondary storage at the checkpoint time, the operating system loads the log file data into the buffer area of the kernel page cache (DB buffer) And then re-read it to write to the secondary storage.

Therefore, the conventional database system may cause problems such as data access speed, power consumption, and cache pollution due to the inefficiency.

Similar inefficiencies can result from using journaling techniques in the same way in the file system as well as in the database system.

Korean Patent Laid-Open Publication No. 2014-0118519 (published date October 08, 2014)

SUMMARY OF THE INVENTION It is an object of the present invention to provide a pre-write logging operation apparatus and method using an improved check point operation.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

The pre-write logging operation apparatus according to an embodiment of the present invention has a log area for storing data to be updated according to a commit operation and a storage space logically divided into a data area for storing original data, A storage medium providing the storage medium; And a controller for instructing an internal copy command to update the data to be updated to the original data position of the data area at the time of the checkpoint operation to the storage medium.

The internal copy command may include a first logical block address at which the data to be updated is stored, a second logical block address at the original data location, and information of original data corresponding to the data to be updated.

The storage medium may update the original data stored in the second logical block address by copying the updated data stored in the first logical block address to the second logical block address.

The controller may wait until the copying completion is received from the storage medium after transmitting the internal copy command, or may perform another transaction from the storage medium without waiting for the completion of the copying.

On the other hand, when the controller receives the failure report for the internal copy command from the storage medium, the controller can retransmit the internal copy command.

According to another aspect of the present invention, there is provided a pre-write logging operation method using a storage medium having a self-copy function by a control unit, the method comprising: Confirming the address of the original data in the data area of the data area; And transmitting an internal copy command for instructing a data copy from the storage address of the data to be updated to the storage address of the original data to the storage medium, wherein the internal copy command includes data in one address of the storage medium To another address, to the other address.

The internal copy command may include a logical block address of the data to be updated in the log area, a logical block address of the original data in the data area corresponding to the logical block address, and a copy command thereof.

In addition, the above-described steps may include waiting after transmitting the command and reporting completion of copying from the storage medium; Or performing another transaction without waiting for a copy completion report from the storage medium after transmitting the command.

According to the present invention, it is possible to reduce the software load at the time of check point calculation.

FIG. 1 illustrates a conventional pre-write logging calculation process. FIG.
BACKGROUND OF THE INVENTION 1. Field of the Invention [0001]
FIG. 3A illustrates a commit operation process according to an embodiment of the present invention; FIG.
FIG. 3B illustrates a checkpoint calculation process according to an embodiment of the present invention; FIG.
FIG. 4 is a flowchart illustrating a pre-write logging operation method according to an embodiment of the present invention; FIG.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, advantages and features of the present invention and methods of achieving them will be apparent from the following detailed description of embodiments thereof taken in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. As used herein, the terms " comprises, " and / or "comprising" refer to the presence or absence of one or more other components, steps, operations, and / Or additions.

Embodiments of the present invention will now be described in detail with reference to the accompanying drawings. FIG. 3 is a flowchart illustrating a commit operation process according to an embodiment of the present invention, and FIG. 3B is a flowchart illustrating a write operation according to an embodiment of the present invention. And FIG.

2, the pre-write logging computing device according to an embodiment of the present invention includes a storage medium 230, a main memory 220, and a control unit 210. [ Here, the device for performing the pre-write logging operation may be a database system or a file system using a journaling technique.

The storage medium 230 includes a data area 232 in which original data is stored and a log area 231 in which log data is stored. Here, the log data may be data in which the content of the data to be updated including the content to be changed in the original data is recorded in page units.

The storage medium 230 may be storage means for providing a copy function between internal logical blocks according to an internal copy command.

For example, the storage medium 230 may be a data storage device based on a hard disk, a solid state disk (SSD), or a flash memory, including a processor capable of performing a copy function between internal logical blocks.

The main memory 220 is a memory including the buffer area 221 and may be a volatile element (or a nonvolatile element) capable of high-speed random access such as RAM.

Here, the buffer area 221 may be an area used for changing the original data in the data area 232 of the storage medium 230. Normally, the buffer area 221 is also referred to as a kernel page cache.

The control unit 210 may be at least one processing unit that performs a transaction including a commit operation and a checkpoint operation. The control unit 210 may include at least one processing unit. For example, the processing unit may include a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, an application specific integrated circuit (ASIC), field programmable gate arrays (FPGA) Lt; / RTI > core. The memory may be a volatile memory (e.g., RAM, etc.), a non-volatile memory (e.g., ROM, flash memory, etc.), or a combination thereof.

Here, the control unit 210 may perform a pre-write logging operation by executing an operating system or an application program.

At this time, the control unit 210 performs a write transaction to update the original data in the storage medium 230 in two steps of a commit operation and a checkpoint operation.

3A, when the commit operation is performed, the controller 210 loads the changed contents of the storage medium 230 into the buffer area 221 of the main memory 220 in units of pages, 221), it is corrected to the new requested data. If a request is made to permanently store data after a predetermined time has elapsed since the data change, the control unit 210 performs a commit operation. The control unit 210 requests the contents of the buffer area 221 to be written into the log area 231 in the storage medium 231 according to the commit operation. This is the same as or similar to the conventional commit operation, so that further description thereof will be omitted.

Here, the controller 210 determines a logical block address (eg, a unit of a sector) of the original data and a logical block address of data to be updated in the log area 231 before or during the commit operation .

At the time of a check point calculation for updating (applying) the data of the log area 231 to the position of the original data of the data area 232 after the completion of the commit operation, the control unit 210 reads the log area 231 ) To the storage medium 230 to update the data to be updated to the original data position of the data storage area.

The internal copy command transmitted to the storage medium 230 includes a logical block address A1 of the data to be updated, a logical block address A2 of the original data, a correspondence relationship between the data to be updated and the original data Address A1 and logical block address A2) (Fig. 3B).

For example, when a, b, and c stored in the first to third logical block addresses in the log area 231 are updated to A, B, and C stored in the fourth to sixth logical block addresses in the data area 232 , The controller 210 copies the data a of the first address to the fourth logical block address, copies the data b of the second address to the fifth logical block address, and writes the data c of the third address to the sixth logical block address The internal copy command may be transmitted to the external device.

As described above, according to the embodiment of the present invention, the log data stored in the storage medium is written back to the main memory at the time of the conventional checkpoint operation using the internal storage copying operation at the time of the checkpoint operation and then transferred to the storage medium, It is possible to improve the problem that traffic is generated.

In addition, the embodiment of the present invention improves storage performance by reducing unnecessary I / O traffic in the check point calculation, reduces energy consumption used for data transmission and reception, and reduces software overhead.

Hereinafter, a pre-write logging operation method according to an embodiment of the present invention will be described with reference to FIG. 4 is a flowchart illustrating a pre-write logging operation method according to an embodiment of the present invention.

Referring to FIG. 4, the control unit 210 determines whether a write transaction requiring update of data stored in the database system or the file system occurs (S410).

When a write transaction occurs, the control unit 210 loads the data to be changed into the buffer area 221 of the main memory 220 in units of pages (S420).

When the data to be changed is loaded in the buffer area 221 of the main memory 220, the data is changed according to the write request. When a predetermined time has elapsed or an explicit commit request is generated, To the log area 231 of the storage medium 230 at step S430.

When the data in the buffer area 221 is written into the log area 231, the control unit 210 changes the data to be changed in the buffer area 221 to the original data position in the data area 232 To the storage medium 230 (S440). At this time, the check point can be executed after the writing completion time in the log area 231, and may be executed after a predetermined time.

Thereafter, the control unit 210 waits for a response (command execution result) corresponding to the internal copy command from the storage medium 230 (S450). Here, the control unit 210 can perform another operation (asynchronous method) or wait (synchronous manner) without performing another operation until it receives report of completion of copying from the storage medium 230. [

If the received command execution result indicates failure, the controller 210 determines that the write transaction has failed and retransmits the internal copy command to the storage medium 230. At this time, the control unit 210 may re-execute from the commit operation.

As described above, the embodiment of the present invention improves the performance of the storage medium and reduces the energy consumption and the software overhead as the data input / output traffic is reduced in the checkpoint calculation.

While the present invention has been described in detail with reference to the accompanying drawings, it is to be understood that the invention is not limited to the above-described embodiments. Those skilled in the art will appreciate that various modifications, Of course, this is possible. Accordingly, the scope of protection of the present invention should not be limited to the above-described embodiments, but should be determined by the description of the following claims.

210: control unit 220: main memory 230: storage medium
221: buffer area 231: log area 232: data area

Claims (7)

A storage medium having a storage area logically divided into a log area storing data to be updated according to a commit operation and a data area storing original data and providing an internal data copying function; And
And a controller for instructing an internal copy command to update the data to be updated to the original data position of the data area at the time of a checkpoint calculation to the storage medium. The method of claim 1,
Wherein the internal copy command includes information of a first logical block address where the data to be updated is stored, a second logical block address of the original data position, and original data corresponding to the data to be updated,
Wherein the storage medium updates the original data stored in the second logical block address by copying the data to be updated stored in the first logical block address to the second logical block address. 2. The apparatus of claim 1,
Wherein the controller is configured to wait until the copying completion is reported from the storage medium after transmitting the internal copy command or to perform another transaction without waiting for the report of completion of copying from the storage medium. 2. The apparatus of claim 1,
And upon receipt of a failure report for the internal copy command from the storage medium, retransmits the internal copy command. A pre-write logging operation method using a storage medium having a self-copy function by a control unit,
Confirming the address of the data to be updated in the log area of the storage medium and the address of the original data of the data area of the storage medium upon completion of the commit operation; And
And transmitting an internal copy command for instructing a data copy from the storage address of the data to be updated to the storage address of the original data to the storage medium,
Wherein the internal copy command is a command for instructing a function of copying data in one address of the storage medium to another address. 6. The method of claim 5,
A logical block address of the original data in the data area corresponding to the logical block address of the data to be updated in the log area, and a copy command thereof. The method of claim 5,
Waiting for a completion of copying from the storage medium after receiving the command; or,
Performing another transaction without waiting for a copy completion report from the storage medium after transmitting the command
Write pre-write operation.
KR1020150148998A 2015-10-26 2015-10-26 Apparatus and Method for Operating Write-Ahead-Logging KR101750744B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020150148998A KR101750744B1 (en) 2015-10-26 2015-10-26 Apparatus and Method for Operating Write-Ahead-Logging

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020150148998A KR101750744B1 (en) 2015-10-26 2015-10-26 Apparatus and Method for Operating Write-Ahead-Logging

Publications (2)

Publication Number Publication Date
KR20170048084A true KR20170048084A (en) 2017-05-08
KR101750744B1 KR101750744B1 (en) 2017-06-23

Family

ID=59283391

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020150148998A KR101750744B1 (en) 2015-10-26 2015-10-26 Apparatus and Method for Operating Write-Ahead-Logging

Country Status (1)

Country Link
KR (1) KR101750744B1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20190103918A (en) * 2018-02-13 2019-09-05 재단법인대구경북과학기술원 System and method for large-scale frequent pattern mining on distributed systems
KR20200044660A (en) * 2018-10-19 2020-04-29 한양대학교 산학협력단 Method and apparatus for logging based on barrier

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11733920B2 (en) 2020-09-10 2023-08-22 Western Digital Technologies, Inc. NVMe simple copy command support using dummy virtual function
US11556268B2 (en) 2021-04-22 2023-01-17 Western Digital Technologies, Inc. Cache based flow for a simple copy command

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20190103918A (en) * 2018-02-13 2019-09-05 재단법인대구경북과학기술원 System and method for large-scale frequent pattern mining on distributed systems
KR20200044660A (en) * 2018-10-19 2020-04-29 한양대학교 산학협력단 Method and apparatus for logging based on barrier

Also Published As

Publication number Publication date
KR101750744B1 (en) 2017-06-23

Similar Documents

Publication Publication Date Title
US11216214B2 (en) Memory system and operation method thereof
KR102403034B1 (en) Dynamic garbage collection p/e policies for redundant storage blocks and distributed software stacks
KR102395360B1 (en) A device controller and method for performing a plurality of write transactions atomically within a nonvolatile data storage device
CN109426619B (en) Method for accessing flash memory module, related flash memory controller and electronic device
US7921339B2 (en) Flash storage device with data correction function
US9244839B2 (en) Methods and apparatus for supporting persistent memory
US10860246B2 (en) Persistent memory updating
JP6470300B2 (en) Method and processor for data processing
KR101750744B1 (en) Apparatus and Method for Operating Write-Ahead-Logging
JP2015201234A (en) Host system and host controller
US9940201B2 (en) Disk array having mirror configuration and rebuilding method therefor
EP3446221B1 (en) Adapted block translation table (btt)
US10769074B2 (en) Computer memory content movement
KR102287677B1 (en) Data accessing method, apparatus, device, and storage medium
US20170160940A1 (en) Data processing method and apparatus of solid state disk
US20160103613A1 (en) Tagging for storage device regions
CN105760218A (en) Online migration method and device for virtual machine
CN111026325B (en) Flash memory controller, control method of flash memory controller and related electronic device
WO2022144646A1 (en) Migrating pages of memory accessible by input-output devices
JP4892812B2 (en) Cache control and data processing system and processing program therefor
US20210271600A1 (en) Data storage device and operating method thereof
US8856468B2 (en) Memory device capable of improving write processing speed and memory control method
CN108519860B (en) SSD read hit processing method and device
WO2015141219A1 (en) Storage system, control device, memory device, data access method, and program recording medium
KR20210022260A (en) Operating method of memory controller, memory controller, and storage device

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E701 Decision to grant or registration of patent right