KR20130057789A - Storage system and method using double writing based on flash memory - Google Patents

Abstract

PURPOSE: A storage system using double writing based on flash memory and a method thereof are provided to process a part which performs synchronous IO by using an SSD(Solid State Drive), thereby reducing response time. CONSTITUTION: A first storage device(420) stores original data received from a host device through a double writing buffer(414). A second storage device(430) interfaces with the first storage device. The second storage device receives the data from the first storage device and stores the same. A control unit includes the double write buffer and controls the first storage device and the second storage device through a data mapping table. When the capacity of the first storage device is full, the control unit overwrites the data stored in the first storage device in the second storage device. When errors are generated in the second storage device, the first storage device performs data recovery by using the stored original data.

Description

Storage system and method using dual writing based on flash memory {STORAGE SYSTEM AND METHOD USING DOUBLE WRITING BASED ON FLASH MEMORY}

The present invention relates to a storage system and method, and more particularly, to a storage system and method using dual writing based on an SSD (flash memory).

Flash memory Solid State Drive (SSD) has many advantages over HDD, which is the existing storage device, such as fast read / write speed and low power consumption through rapid development of related technologies. However, due to the high price / capacity and reliability verification, the HDD is not completely replaced in the enterprise market. In addition, with the activation of smartphones and SNSs, the production speed of content is exploding and flash memory SSDs and HDDs are expected to continue to complement each other.

The read / write unit of flash memory is a page, and it is divided into small blocks and large blocks according to the page size of the chip. Single-level cell (SLC) and multicell ( Multi-level cell, MLC). In general, SLC has a shorter lifespan and write speed than MLC, and the read speed is similar.

SSDs have multiple chips on multiple data buses, so they have internal parallelism during data read / write. This results in higher performance when accessing multiple data pages on an SSD simultaneously. It also shows good performance in the case of sequential data access because many channels are activated at the same time by utilizing the internal SDRAM buffer. In general, SSD's sequential read performance is close to the bandwidth of the host interface, and several models on the market recently show sequential read / write performance close to the full bandwidth.

1 is a diagram illustrating the structure of a typical two-channel two-way flash memory SSD. As shown in FIG. 1, a high-density NAND Flash memory chip consists of a plurality of blocks, and one block again has several pages. Since each channel has two ways, a program can be executed on the A-0 way and a program can be performed on the A-1 way. Therefore, if a large number of write operations are performed at the same time, slow write performance can be compensated for.

However, in the conventional researches that utilize a flash memory SSD in a database, there is a problem in that the write performance is not maximized by focusing only on random access of the SSD.

(Patent 1) Republic of Korea open patent KR 10-2009-0129095 ("High-speed data processing system using a double buffer", Myung Information Technology, 2009.12.16 published)

An object of the present invention for solving the above problems is to provide a storage system and method using a double write based on the flash memory showing the high efficiency compared to the investment cost by using the SSD as a cache between the memory and the HDD.

Another object of the present invention is to modify the dual write function that prevents the partial write problem when the system is down and apply to the SSD cache, it is possible to apply the existing dual write structure without significant modification, to share the data storage load, and to require a fast response speed The present invention provides a storage system and method using dual writing based on a flash memory that includes only a part of the SSD and allows other tasks to be processed in parallel.

According to an aspect of the present invention, there is provided a storage system using double writing, comprising: a first storage device storing original data received from a host device through a double writing buffer; A second storage device interfacing with the first storage device to receive and store data from the first storage device; And the dual write buffer, and controls the first storage device and the second storage device through a data mapping table, and when the capacity of the first storage device is full, the data stored in the first storage device is stored. And a control unit for controlling to overwrite the second storage device, wherein the first storage device is a flash memory and performs data recovery through the stored original data when an error occurs in the second storage device. Can be.

The input / output pattern of the first storage device may be in the form of first input first output (FIFO) and random read. The first storage device may have a capacity of one gigabyte, two gigabytes, and four gigabytes. The controller may further include a data buffer.

The mapping table may map information of all data existing in the first storage device and location information inside the first storage device.

When receiving a data read request from the host device, first, the requested data is retrieved and read from the data buffer, and if there is no requested data in the data buffer, the data is present in the first storage device through the mapping table. Can read data.

The mapping table may be composed of a data area offset of the original data, location information in the first storage device, and a pointer configuring a hash chain.

The second storage device may be a magnetic hard disk.

According to an aspect of the present invention, there is provided a storage method using double write, comprising: storing original data received from a host device through a double write buffer in a first storage device which is a flash memory; Interfacing with the first storage device and receiving data from the first storage device and storing the data in the second storage device; And controlling, by the controller, the first storage device and the second storage device through a data mapping table, wherein the controlling step is performed when the capacity of the first storage device is full. Controlling to overwrite data on the second storage device; And when an error occurs in the second storage device, performing data recovery through original data stored in the first storage device.

The input / output pattern of the first storage device may be in the form of first input first output (FIFO) and random read. The first storage device may have a capacity of one gigabyte, two gigabytes, and four gigabytes. The controller may further include a data buffer. The double write buffer may temporarily store data to be input to the first and second storage devices.

The control unit may include a mapping table that maps information of all data existing in the first storage device and location information inside the first storage device.

The control step may include: searching for and reading requested data from the data buffer when receiving a data read request from the host device; And reading the data existing in the first storage device through the mapping table when the requested data is not present in the data buffer.

The mapping table may be composed of a data area offset of the original data, location information in the first storage device, and a pointer configuring a hash chain.

The second storage device may be a magnetic hard disk.

According to the storage system and method using the dual write based on the flash memory according to the present invention, by processing the portion performing the synchronous IO to the SSD, the response time is reduced.

In addition, according to the storage system and method using the double write based on the flash memory according to the present invention, while the conventional double write is to be performed inevitably for the sake of stability, rather than by utilizing the double write as caching It can be used as a means of improving performance, and only a small amount of data is used as a cache, but compared to the data, the performance of the system can be greatly improved.

Therefore, it is expected to contribute to DBMS (DataBase Management System) tuning related fields.

1 is a view showing the structure of a typical two-channel two-way flash memory SSD,
2 is a view for explaining a double-write technique of a conventional Inno DB,
3 is a conceptual diagram illustrating a concept of a storage system using dual writing according to an embodiment of the present invention;
4 is a block diagram schematically illustrating a storage system using dual writing according to an embodiment of the present invention;
5 is a view for explaining a write operation to a dual write area of a storage system using dual writes according to an embodiment of the present invention;
6 is a view for explaining a read operation to a dual write area of a storage system using dual writes according to an embodiment of the present invention;
7 is a flowchart schematically illustrating a storage system using dual writing according to an embodiment of the present invention;
8 is a flowchart illustrating a method of performing a read command of a storage method using double writing according to an embodiment of the present invention;
FIG. 9A illustrates a state in which contents are replaced after asynchronous writing is completed before contents of the dual writing region are replaced in the conventional double writing technique. FIG.
FIG. 9B is a diagram illustrating an example of parallel processing of input / output commands by delaying a synchronization period by expanding a size of a double write area in a storage system using double writes according to an embodiment of the present invention; FIG.
FIG. 10 is a graph illustrating performance comparison according to an increase in the number of concurrent users between a storage system using dual writing and a storage system using a different method according to an embodiment of the present invention;
FIG. 11 is a graph illustrating a change in performance and hit ratio according to a size of a cache region through a flash memory in a storage system using dual writes according to an embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail.

It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. In order to facilitate the understanding of the present invention, the same reference numerals are used for the same constituent elements in the drawings and redundant explanations for the same constituent elements are omitted.

2 is a view for explaining a double-write technique of a conventional Inno DB. As shown in FIG. 2, the conventional MySQL InnoDB storage engine uses a write method known as double writing to solve a partial-page write problem.

The InnoDB storage engine may include a RAM 210 and an HDD 220. The HDD 220 is used as a data table space, and the RAM 210 may include a buffer pool 212 and a double write buffer 214. First, the InnoDB storage engine copies data pages to a main memory area called double write buffer 214. The write buffer 214 is composed of two extents, and the pages of the write buffer 214 are simultaneously written to the write area of the system table space by the buffer flush caused by the buffer being full or later. Is written in sequential writing.

Referring to FIG. 2, the double-write area is located at the front of the data table space of the HDD 220. When a write request to the double-write area is successful, all pages of the double-write buffer 214 are IO threads. Is rewritten into each tablespace with Asynchronous IO (AIO). The double-write technique writes the replaced page twice, which can increase the IO burden.

With this two-phase write, InnoDB Storage Manager can recover partial-write pages even in the event of a system crash caused by a power failure.

If a system error occurs, the system checks the validity of pages in the transaction log and double-write area when the system starts. If a page in a double-write area is partially written, it may not have been saved in the data tablespace to which the page belongs. Therefore, the page in the data tablespace remains before the update.

Therefore, the original page of the data tablespace is used instead of the page in the double-write area during the recovery process using the WAL log after a system error.

As mentioned above, double-write in InnoDB is a necessary feature, but because it writes double-write pages and writes original data pages, it doubles the workload and can adversely affect storage and overall system performance. The double write area is collected and written down at once, and the data area is divided into asynchronous writes to reduce the burden. However, there is a drawback of waiting for the completion of an asynchronous write before overwriting the double write area.

3 is a conceptual diagram illustrating a concept of a storage system using dual writes according to an embodiment of the present invention. As shown in FIG. 3, a storage system according to an embodiment of the present invention may include an SSD 320 and an HDD 330.

Referring to FIG. 3, in the storage system according to an embodiment of the present invention, a dual write table space is separately formed in the SSD 320 in a partial region of a data table space of the HDD 310. In addition, only the data table space is configured in the HDD 330, and the dual write area is separated and extended and allocated to the SSD 320. Therefore, the new storage device (SSD: 320) shared the load on the data storage device by separating the dual write area from the data tablespace.

In other words, by expanding the size of the dual write area and filling it in sequence, the fast sequential write characteristics of the SSD 320 can be utilized, and as the overwriting period becomes longer, the synchronization period can be delayed to improve the output history processing of the data storage device. Can be.

In addition, when the page is loaded, the random access characteristic of the SSD 320 superior to the HDD 330 is utilized by reading from the extended double write area if possible, and thus, the double structure is not changed significantly without changing the structure of the conventional storage system. By modifying / adding the write area, you can greatly increase system performance.

4 is a block diagram schematically illustrating a storage system using dual writing according to an embodiment of the present invention. As illustrated in FIG. 4, a storage system according to an embodiment of the present invention may include a RAM 410, a dual write table space 420 (SSD), and a data table space 430 (HDD).

Referring to FIG. 4, the conventional InnoDB dual write area is replaced with the flash memory SSD 420, and the IO pattern for the SSD 420 is converted into sequential writes and random reads, thereby improving performance.

First, the RAM 410 may include a buffer pool 412, a double write buffer 414, a double write hash 416, and a double write mapping table 418. The RAM 410 may control the dual write table space 420 and the data table space 430 through the dual write mapping table 418.

The buffer pool 412 is a space temporarily stored when data is received from a host device (not shown). In addition, the double write buffer 414 may temporarily store data input from the host device copied by the RAM 410.

The double-write hash 416 is a component of the double-write mapping table 418 and is used to quickly check the existence of pages included in the double-write table space 420. The hash 416 may be composed of a data area offset of the original page, position information dw_offset in the double-write area, and a pointer constituting the hash chain.

The dual write mapping table 418 is a table for managing the dual write table space 420, and maps information of all data stored in the dual write table space 420 with location information inside the dual write table space 420. Doing.

The double write table space 420 stores data received from the host device from the double write buffer 414. The dual write table space 420 is controlled by the RAM 410. Therefore, control is based on the mapping table 418. When the capacity of the double write table space 420 is full, data stored in the double write table space 420 may be overwritten by the data table space 430. The double write table space 420 is a device for recovering data through original data in the event of a system error. Accordingly, when there is a system error, the RAM 410 may validate data pages existing in the double-write table space 420 based on the mapping table 418 and recover data based on the test result. Can be.

In a preferred embodiment of the present invention, the dual write table space 420 is a flash memory SSD. In addition, the input / output pattern of the flash memory is preferably sequential write and random read. Double write saves multiple data pages in one synchronous IO command and then makes an asynchronous IO request to the data table space for that page. Therefore, the dual write table space 420 and the data table space 430 are synchronized in the form of write-through. Therefore, since InnoDB uses a plurality of IO threads, the IO requests of the two regions may overlap.

Furthermore, the capacity of the double write table space 420 is preferably 1 gigabyte or more. This is because it is advantageous in terms of synchronization period with the data tablespace 430.

The data table space 430 receives and stores data through the double write table space 420. The data table space 430 may also operate under the control of the RAM 410. The data table space 430 is preferably an HDD. When a system error occurs, an error may occur in the data stored in the data table space 430. In this case, the data in the data table space 430 may be recovered through the data in the double-write table space 420. can do.

FIG. 5 is a diagram illustrating a write operation to a dual write area of a storage system using dual writes according to an embodiment of the present invention. As illustrated in FIG. 5, data received from a host device (not shown) is temporarily stored in the double write buffer 514 in the buffer pool 512, and the double write table space 520 from the double write buffer 514. ) And the data stored in the double write table space 520 is written to the data table space 530 in duplicate.

A storage system according to an embodiment of the present invention manages data through a file space structure. One space can be connected to multiple file nodes. By default, space 0 manages system information and data, and space 1 manages log areas. Duplicate table spaces are contained in the header part of space 0, and are separated into separate file spaces. Create a new file space number 9, connect the file nodes, and specify the path as a separate storage device. Thereafter, the I / O load of the storage device can be distributed by allowing the dual write I / O to pass through the 9th file space.

The dual write table space 520 is originally composed of two extents. Thus, by overwriting the first extent each time, the double-write table space 520 is extended to determine which extent to refer to each access. Put a variable to manage this and manage to use the next extent sequentially.

Referring to FIG. 5, upon receiving data input from a host device, the received data is stored in the buffer pool 512. Next, the received data is sent to the double write buffer 514, and the double write buffer 514 stores the data received from the buffer pool 512 in the double write table space 520. Next, data stored in the double write table space 520 is stored in the data table space 530 in duplicate. In this case, a sequential write (FIFO) method is preferably used in the double write table space 520. When stored in the double write table space 520, the location stored in the double write mapping table 518 and the stored data are mapped, which may form a hash 516 structure.

FIG. 6 is a diagram illustrating a read operation of a dual write area of a storage system using dual writes according to an embodiment of the present invention. As shown in FIG. 6, when receiving a read request from a host device (not shown), first, the requested data is looked up in the buffer pool 612, and if there is no data, a double write hash 616 is used. The data is found and retrieved from the double write table space 620 through the double write mapping table 618.

Referring to FIG. 6, it is checked whether the requested data is in the buffer pool 612 when a desired data page request is made. If the requested data exists in the buffer pool 612, the data is fetched and the read request related operation is completed. However, if there is no desired data in the buffer pool 612, the data is fetched from the double-write data space 620. That is, the mapping table 618 is referred to using the double-write hash 616. Since the mapping table 618 has a mapping relationship between all data in the write table space 620 and a storage location in the write table space 620, the mapping table 618 searches for the requested data. As a result of the search, if there is data, it is withdrawn from the double-write table space 620 to complete the read request related operation. If there is no data, the data table space 630 is searched to retrieve the requested data.

As described above, since the double write table space 620 may be quickly fetched by the SSD, that is, the flash memory, it is efficient to first determine whether data exists in the double write table space 620.

7 is a flowchart schematically illustrating a storage method using double writing according to an embodiment of the present invention. As shown in FIG. 7, the storage method according to an embodiment of the present invention includes storing the original data received through the double write buffer in a double write table space that is a flash memory (710), and data from the double write table space. The method may include receiving 720 and storing the data table space in the data table space and controlling the double-write table space and the data table space through the data mapping table 730.

Referring to FIG. 7, the first storage device may be a dual-write table space, and the second storage device may be a data table space. Therefore, the data input from the host device (not shown) is first stored in the double write table space (710). It can be used for data recovery in case of system failure. Next, in operation 720, data stored in the double-write table space is stored in the data table space. The controller may control the dual write table space and the data table space through the data mapping table (730). The control step 730 may include controlling to overwrite the data table space with data stored in the double write table space when the capacity of the double write table space is full, and when the error occurs in the data table space. The method may include performing data recovery through the original data stored in the table space.

Here, it is preferable that the device for the double-write table space is a flash memory. This is because fast access to flash memory is effective for double-write table spaces.

8 is a flowchart illustrating a method of performing a read command of a storage method using double writing according to an embodiment of the present invention. As shown in FIG. 8, an operation according to a read command of a storage method according to an exemplary embodiment of the present invention may be performed by retrieving and reading requested data from a data buffer (not shown) (810) and a mapping table. The method may include searching for and reading data existing in the double-write table space (820).

Referring to FIG. 8, the method checks whether data requested in a data page request is in a data buffer (not shown) (810). If the requested data exists in the data buffer, the data is fetched and the read request related operation is completed. However, if there is no desired data in the data buffer, step 820 of extracting data from the double-write data space is performed. In other words, the mapping table is referenced using a double-write hash. The mapping table has a mapping relationship between all data in the double-write table space and the storage location inside the double-write table space, so look up it to see if the requested data exists. As a result of the search, if there is data, it is fetched from the double-write table space to complete the operation related to the read request. If there is no data, the data table space is searched and the requested data is retrieved.

FIG. 9A is a diagram illustrating a state in which contents are replaced after asynchronous writing is completed before contents of the dual writing region are replaced in the conventional double writing technique. As shown in FIG. 9A, the storage device 910 using double writing stores data in the double writing area by using a double writing area in one storage device, and stores the stored data again in the data area. Double overwriting was used.

Referring to FIG. 9A, one column in the drawing symbolically represents a plurality of pages, and the number of pages in the buffer may include 512 pages. When converting the data into the size of the data, the double-write area becomes full when the 2MB data is stored in the double-write area (920). Accordingly, the next data is in the form of overwriting the data in the existing double write area with the data area, generating the next new double write area, and storing the data again in the newly created double write area (930). In this case, the synchronization needs to be performed every 2MB, and the more frequently the write write request is, the more the calculation amount is and the storage efficiency is lowered.

FIG. 9B is a diagram illustrating an example of parallel processing of an input / output command by delaying a synchronization period by expanding a size of a double write area in a storage system using double writes according to an embodiment of the present invention.

Referring to FIG. 9B, since the dual write area 950 is separately secured as a flash memory, the first 4 spaces, that is, 1 MB of data, may be stored in the data area 960 in the same manner as the conventional method. In the case of storing data of 2MB or more, in the case of a storage system according to an exemplary embodiment of the present invention, there is a storage space in the dual-write area 970 so that data can be stored, and the data can be stored without requesting synchronization of the dual-write area. Data may be stored in the area 980. In fact, the dual write area 970 may be modified to be synchronized with each 1GB according to the capacity of the flash memory to increase parallelism. Moreover, the capacity of the flash memory may be 1 GB or more. You can also use flash memory with a capacity of 2GB or 4GB. In this case, the output parallelism can be further enhanced by increasing the synchronization period of the dual write area.

FIG. 10 is a graph illustrating performance comparison according to an increase in the number of concurrent users between a storage system using dual writing and a storage system using a different method according to an embodiment of the present invention. Here, Read Cache refers to a case in which a flash memory SSD is used as a cache, and DW-seperate refers to a case in which the double-write area is not expanded but is separated from the conventional one, and Original refers to a storage device using a conventional double-write method. Say. The horizontal axis indicates concurrent users, and the vertical axis shows the average Transaction Per Second (TPS) during execution time. By separating the dual write area into other storage devices, the overall performance improved by about 3%. When the read cache function is applied, the performance is greatly increased. This is because the random read performance is basically 10 times better than the HDD used in the experiment, and the parallelism of data page writing is increased.

FIG. 11 is a graph illustrating a change in performance and hit ratio according to a size of a cache region through a flash memory in a storage system using dual writes according to an embodiment of the present invention. As shown in FIG. 11, as the number of concurrent users is fixed to 16, and benchmarking is performed while increasing the size of the cache area from 1 GB to 4 GB, the TPS and the hit ratio increase as the size of the cache area increases. can see. Compared with the result of not using the cache, we can see that the performance of 1GB cache is improved from 47% up to 54% on 30GB data.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the inventions as defined by the following claims It will be understood that various modifications and changes may be made thereto without departing from the spirit and scope of the invention.

Claims (17)

A first storage device storing original data received from a host device through a double write buffer;
A second storage device interfacing with the first storage device to receive and store data from the first storage device; And
The first storage device and the second storage device are controlled through the data mapping table, and when the capacity of the first storage device is full, the data stored in the first storage device is stored. A control unit for controlling to overwrite the second storage device,
The first storage device is a flash memory, and when an error occurs in the second storage device, a storage system using double write through the flash memory, characterized in that for performing data recovery through the stored original data.
The method of claim 1,
The input / output pattern of the first storage device has a first write first output (FIFO) and random (Random) read form, the storage system using dual write through the flash memory.
The method of claim 1,
The first storage device has a capacity of any one of 1 gigabyte, 2 gigabyte, and 4 gigabyte.
The method of claim 1,
And the control unit further comprises a data buffer.
The method of claim 1,
And the mapping table maps information of all data existing in the first storage device and location information inside the first storage device.
The method of claim 1,
When receiving a data read request from the host device, first, the requested data is retrieved and read from the data buffer,
And when the requested data is not in the data buffer, reading data existing in the first storage device through the mapping table.
The method of claim 1,
And the mapping table comprises a data area offset of the original data, location information in the first storage device, and a pointer constituting a hash chain.
The method of claim 1,
And said second storage device is a magnetic hard disk.
Storing original data received from the host device through a double write buffer in a first storage device which is a flash memory;
Interfacing with the first storage device and receiving data from the first storage device and storing the data in the second storage device; And
And controlling the controller to control the first storage device and the second storage device through a data mapping table.
The controlling may include controlling to overwrite data stored in the first storage device with the second storage device when the capacity of the first storage device is full; And
And performing data recovery through original data stored in the first storage device when an error occurs in the second storage device.
The method of claim 9,
The input / output pattern of the first storage device is a first write first output (FIFO) and random (random) read form, characterized in that the dual write through the flash memory.
The method of claim 9,
The first storage device has a capacity of any one of 1 gigabyte, 2 gigabyte, and 4 gigabyte.
The method of claim 9,
And the control unit further comprises a data buffer.
The method of claim 9,
And the double write buffer temporarily stores data to be input to the first and second storage devices.
The method of claim 9,
And the control unit includes a mapping table that maps information of all data existing in the first storage device and location information in the first storage device.
The method of claim 14, wherein the controlling step,
Retrieving the requested data from the data buffer when receiving a data read request from the host device; And
And reading data existing in the first storage device through the mapping table when the requested data is not present in the data buffer.
15. The method of claim 14,
And the mapping table comprises a data area offset of the original data, location information in the first storage device, and a pointer constituting a hash chain.
The method of claim 9,
And the second storage device is a magnetic hard disk.

Images