KR20190027279A - Method And Apparatus for Recovery in Flash Storage Using Selective Erasure Coding - Google Patents

Abstract

Disclosed are a method for recovering a data loss using selective erasure coding in a flash storage apparatus, and a device thereof. According to an aspect of the embodiment, the method for recovering a data loss using selective erasure coding in a flash storage apparatus comprises: a process of receiving a data write request from a host; a process of determining a data storage method of either an erasure coding method or a replication method, based on a size of data requested to be written; a process of generating metadata including information on the data requested to be written and recording the generated metadata in a memory; and a process of recording the data requested to be written in the memory in accordance with a determined data storage mode.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a method and an apparatus for restoring data loss using selective erasure coding in a flash storage device,

This embodiment relates to a flash memory based storage device.

The contents described in this section merely provide background information on the present embodiment and do not constitute the prior art.

One of the important techniques in storage devices is to restore original data when data loss occurs. Failure to restore the data can result in economic loss. As a method for restoring data loss, there is a replication method which creates a copy of the original data and stores the copy in a physical space different from the original. In such a copying method, since the original data is copied as it is, additional storage space of the same size as the original data is required. Therefore, it is used in a relatively cheap HDD or tape device.

In recent years, as cost and storage space efficiency become more important, erasure coding is being introduced. The erasure coding scheme is a method of encoding data to generate parity, distributing data and parity, and recovering original data through a decoding process using parity in case of data loss. This erasure coding method is a method that can restore original data even if a part of original data stored is lost. Since it is possible to restore data by maintaining only some restoration information without generating a copy of original data, It is advantageous from the side and has excellent error restoration performance and is used in expensive devices such as SSD. According to the erase coding scheme, the original data is divided and the divided data is divided into several storage spaces. In this case, when the size of the divided data is smaller than the logical page size of the flash, the internal fragmentation problem of the flash page . In order to retrieve the page where the internal fragmentation occurs, additional data read / write process is required. However, since the flash memory has a limited number of write operations due to its physical characteristics, there is a problem that the memory life can be shortened.

Embodiments of the present invention have a main object to provide a method for solving the problem of shortening the life of a flash memory that can occur due to internal fragmentation in applying an erasure coding scheme for data loss recovery of a flash storage device .

According to an aspect of the present invention, a method of determining a data storage method of an erasure coding method and a replication method is determined based on a size of data requested to be written, Generating metadata including information on data requested to be written, recording the generated metadata in a memory, and writing data requested to be written according to a determined data storage method to a memory, Thereby providing a method for restoring data loss of the storage device.

The embodiment of the data loss recovery method may further include one or more of the following features.

The process of determining the data storage method includes a process of comparing the data chunk size divided by the erasure coding scheme and the logical page size of the memory, and when the data chunk size is equal to or larger than the logical page size, The data storage method can be determined. If the data chunk size is smaller than the logical page size, the data storage method can be determined by the replication method.

The process of writing the meta data into the memory may generate the meta data including the information about the data storage method corresponding to the data requested to be written.

According to an aspect of the present invention, there is provided a method for managing a data read request, the method comprising: receiving a data read request from a host; checking metadata of data requested to be read; A step of reading a plurality of original data chunks from a memory and a step of reading either original data or copied data from the memory when the data storing method is a duplication method. ≪ / RTI >

As described above, according to the embodiment of the present invention, the internal page fragmentation problem can be solved by selectively applying erasure coding in consideration of the logical page size of the flash memory, and the problems of the input / output performance degradation and the shortening of the life of the flash memory It is possible to solve the problem.

In addition, according to the embodiments of the present invention, it is possible to provide a method and an apparatus for restoring data even if a loss occurs in stored data due to damage to the flash memory, thereby ensuring reliability of data stored in the flash storage device have.

1 is a diagram for explaining an erasure coding scheme.
Fig. 2 is a diagram for explaining a copying method.
3 is a configuration diagram of a flash storage device according to an embodiment of the present invention.
4 is a flowchart illustrating a method of writing data in a flash storage device using a data loss recovery method according to an embodiment of the present invention.
5 and 6 are diagrams for explaining a data writing method according to an embodiment of the present invention.
7 is a flowchart illustrating a method of reading data in a flash storage device using a data loss recovery method according to an embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail with reference to the accompanying drawings. In describing the present invention, a flash storage device refers to a storage device implemented based on a flash memory, and may be a storage device such as an SSD. Further, the flash memory is a concept including a plurality of flash memory chips, and each flash memory chip corresponds to a physically different storage space. The flash memory is composed of a plurality of blocks, and each block includes a plurality of pages. Writing (program or write) and reading (reading) in the flash memory are performed page by page, and erasing is performed block by block.

1 is a diagram for explaining an erasure coding scheme. FIG. 1 illustrates a case in which 4 KB data is written using a 4 + 2 erase code in a flash storage device having a logical page size of 4 KB.

In erase coding, the original data chunks generated by dividing the original data and the parity chunks generated through the encoding process are stored in different spaces, so that even if a loss occurs in a part of the original data chunks, the lost data can be restored by using the parity chunks There is a way. In general, the Reed Solomon code is often used as the erase code, and is expressed in the form of K + N. In this case, K denotes the number of original data to be divided, N denotes the number of parities to be generated for restoring original data, and K + N denotes the total number of storage spaces for storing divided data and parity. If data is stored using a K + N erasure code, the original data can be reconstructed even if there is a loss in N divided data or parity. Hereinafter, a process of writing data to the flash memory by applying the erasure coding scheme will be described as an example using the Reed Solomon code.

When a data write request occurs, a K + N erasure code is applied to divide the original data into K original data chunks, and N parity chunks are generated from these original data chunks. Here, the original data chunk refers to a data chunk generated by dividing original data, and the parity chunk refers to a data chunk generated for restoring original data. It is desirable to store the generated original data chunks and parity chunks physically in different spaces in case the storage space is damaged.

Referring to FIG. 1, when a data write request of a size of 4 KB occurs, four original data chunks each having a size of 1 KB are generated using a 4 + 2 erase code, and two parity chunks for original data chunks are generated do. When applying these erasure codes in the course of writing data, the original data can be reconstructed if there are only four chunks, regardless of the original data chunk or the parity chunk. As shown in the figure, original data chunks and parity chunks generated by applying an erasure code are recorded in different flash memories. In this case, data writing is performed on a page-by-page basis. Therefore, if a logical page size is 4 KB, recording of 1 KB chunks results in internal fragmentation of 3 KB. The page where the internal fragmentation occurred is re-read by the internal fragmentation elimination routine in the future. Then, the data recorded on the pages where the internal fragmentation occurs is merged and the new page is written to another page. And is left as an idle space. This process is called Write Amplification Factor (WAF). As the WAF increases, the parallel operation becomes impossible, the input / output performance is lowered, and the lifetime of the flash memory is also shortened.

Fig. 2 is a diagram for explaining a copying method. FIG. 2 illustrates a case where a 4-KB data is written by applying a 3-way replication method to a flash storage device having a logical page size of 4 KB.

In order to prepare for the loss of data, the duplication method creates duplicated data by making a copy of the original data and stores the original data and the duplicated data in different spaces, so that even if any data is lost, It is a method that can be maintained as it is. N-way means that the total number of original data and copied data is N, and in the case of the 3-way copying method illustrated in FIG. 2, two copies of original data are generated. When the data is stored by applying the replication method, original data can be restored even if a loss occurs in N-1 data.

When a data write request occurs, it generates N original data and replicated data. Here, the original data and the replicated data are substantially identical, so that the division is virtually meaningless, and the division is described below for the sake of explanation. In case the storage space is damaged, it is desirable that the original data and the replicated data are physically stored in different spaces.

Referring to FIG. 2, when a data write request of a size of 4 KB occurs, two copies of write-requested data are generated, and the original data and the copied data are written to different flash memories. In this case, data writing is performed on a page-by-page basis. Therefore, if the logical page size is 4 KB, the original data of 4 KB size and the duplicated page are written, so that the internal fragmentation problem does not occur. However, in this case, there is a disadvantage that the capacity required for recording data in the storage device is very large. In both cases shown in FIGS. 1 and 2, it is possible to prepare for a maximum of two disc failures. However, the erasure coding scheme of FIG. 1 requires a capacity of 6 KB, and the replication scheme of FIG. 2 requires a capacity of 12 KB Which is disadvantageous in terms of capacity efficiency of the storage device.

The present invention proposes a method of selectively applying an erase coding scheme and a replication scheme in consideration of a logical page size of a flash memory in applying erasure coding in a flash-based storage device. Specifically, when the size of the data chunks generated by dividing the original data is smaller than the logical page size of the flash, no unnecessary data internal fragmentation problem is caused by applying the data replication method without applying erasure coding.

In addition, since the method of storing data is determined according to the size of the data requested to be written, in order to determine the data storage method in the data reading process and to read the data, We propose a method to generate meta data with different information. The generated metadata can be recorded in the memory together with the data, and the data storage method can be determined through the process of checking the metadata in the reading process.

3 is a configuration diagram of a flash storage device according to an embodiment of the present invention. Referring to FIG. 3, a flash storage device according to an embodiment of the present invention includes a storage controller 100 and a flash memory 200.

The storage controller 100 controls communication with the host and data transmission / reception with the flash memory 200. The storage controller 100 may store data in the flash memory 200 or may read and erase the stored data. In the embodiment of the present invention, the storage controller 100 controls the process of storing data in the flash memory 200 and provides a data writing method for recovering data in preparation for data loss. In addition, in the embodiment of the present invention, the storage controller 100 controls the process of reading data from the flash memory 200 and provides a data reading method for recovering original data when data loss occurs. The storage controller 100 may include a microprocessor and the like for driving firmware for a flash memory application and for controlling the process of reading and writing data.

The storage controller 100 may include a host interface (not shown), a flash memory controller (not shown), and the like. The host interface provides an interface between the storage device and an external host computer. The host interface may be a communication standard such as PATA (Parallel ATA), SATA (Seral ATA), or USB (Universal Serial Bus). A flash storage device according to an embodiment of the present invention receives data access commands such as reading data from a host and writing data through a host interface. The flash memory controller performs data transfer, bus control, and the like of the flash memory 200 under the control of the storage controller 100.

The flash memory 200 includes a plurality of flash memory chips. The flash memory 200 is constituted by a plurality of blocks which are basic units of the data erase operation, and the blocks are composed of a plurality of pages which are basic units of read and write operations.

4 is a flowchart illustrating a method of writing data in a storage device using a data loss recovery method according to an embodiment of the present invention.

5 and 6 are diagrams for explaining a data writing method according to an embodiment of the present invention.

Hereinafter, a method of storing data in a flash storage device according to an embodiment of the present invention will be described with reference to FIGS.

Referring to FIG. 4, when the flash storage device according to the embodiment of the present invention receives a data write request from the host at step S410, the storage controller 100 determines whether or not the flash memory Determine which method of replication will be used to store the data. Specifically, the storage controller 100 compares the size of the data chunk generated when the erase coding scheme is applied to the data requested to be written to determine the data storage mode, and the logical page size of the memory (S420).

As a result of the comparison, if the data chunk size is equal to or greater than the logical page size, it is determined to apply the erasure coding scheme (S430). When the data storage method is determined, metadata including information on the data storage method corresponding to the data requested to be written is generated (S440). It is necessary to include information on the determined data storage method in the metadata in order to reconstruct the original data by reading all of the plurality of original data chunks stored in different spaces. The generated metadata is recorded in the memory, and the original data chunks and the parity chunk are written into the flash memory 200 (S450).

Referring to FIG. 5, a case of writing data of 16 KB size using a 4 + 2 erase code to a flash storage device having a logical page size of 4 KB will be described as an example. In the example shown in FIG. 5, when the erasure coding scheme is applied to the 16 KB data, the size of the divided original data chunk is 4 KB. Since the data chunk size is equal to the logical page size, it is determined that the erasure coding method is applied to the data storage method of the write-requested data. When the determined data storage scheme is an erasure coding scheme, metadata including the number of original data chunks, parity chunks, data chunk size, and parity chunks is generated, and the generated metadata Into memory. In the figure, the original data chunks generated through the 4 + 2 erasure coding process are denoted as a, b, c, and d, respectively, and the parity chunks for the original data chunks are denoted as p1 and p2. A total of six original data chunks (a, b, c, d) and parity chunks (p1, p2) are physically recorded in different storage spaces and only 8KB are used for restoring data loss of 16KB in size.

On the other hand, if it is determined that the data chunk size is smaller than the logical page size as a result of comparing the size of the data chunk and the size of the logical page of the memory, it is determined to apply the copying method (S460). When the data storage method is determined, metadata including information on the data storage method corresponding to the data requested to be written is generated (S470). When there is a data read request, it is necessary to include information on the determined data storage method in the metadata in order to read either the original data or the duplicated data and return the original data. The generated metadata is recorded in the memory, and the original data and the copied data are recorded in the flash memory 200 (S480).

Referring to FIG. 6, a 3-way replication method is applied to a flash storage device having a logical page size of 4 KB to write data of 8 KB size. 6, when the erase coding scheme is applied to the 8 KB data, the size of the divided original data chunk is 2 KB. Since the data chunk size is smaller than the logical page size, it is determined that the replication method is applied to the data storage method of the write-requested data. When the determined data storage method is the replication method, metadata including the original data and the location information of the copied data is generated, and the generated metadata is stored in the memory before storing the requested data. In the figure, the original data is denoted by o and the replicated data is denoted by o '. Since the data writing process is performed page by page, as shown in the figure, the data is divided into 4 KB and recorded in the memory, and the data are written as a and b. The original data and the replicated data are physically recorded in different storage spaces, so that the internal fragmentation problem does not occur.

7 is a flowchart illustrating a method of reading data in a flash storage device using a data loss recovery method according to an embodiment of the present invention.

Referring to FIG. 7, when the flash storage device according to the embodiment of the present invention receives a data read request from the host (S710), the storage controller 100 checks metadata of the read requested data (S720) It is determined whether the storage scheme is an erasure coding scheme or a replication scheme (S730).

If the data storage scheme is the erasure coding scheme, the original data chunks are read from the address by referring to the location information of the original data chunks included in the metadata (S730). If it is determined that there is a data loss in one or more data chunks of the original data chunks (S740), the parity chunk is read from the corresponding address by referring to the position information of the parity chunk included in the meta data, To restore the original data (S750).

If the data storage method is the replication method, the original data is read from the address by referring to the location information of the original data included in the metadata, or the copied data is read with reference to the location information of the replicated data (S760).

According to the data storage method, the original data is read or restored using different methods, and the original data is returned to the host (S770).

The foregoing description is merely illustrative of the technical idea of the present embodiment, and various modifications and changes may be made to those skilled in the art without departing from the essential characteristics of the embodiments. Therefore, the present embodiments are to be construed as illustrative rather than restrictive, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be construed according to the following claims, and all technical ideas within the scope of equivalents thereof should be construed as being included in the scope of the present invention.

100: Storage controller
200: flash memory

Claims (16)

Receiving a data write request from a host;
Determining a data storage method of either an erasure coding method or a replication method based on a size of data requested to be written;
Generating metadata including information on the write-requested data, and recording the generated metadata in a memory; And
And writing the requested data to the memory in accordance with the determined data storage mode
The method comprising the steps of: The method according to claim 1,
The method of claim 1,
And comparing the size of the data chunk to be divided with the logical page size of the memory by applying the erasure coding scheme,
Determining a data storage scheme using an erasure coding scheme when the data chunk size is equal to or greater than the logical page size and determining a data storage scheme using a replication scheme when the data chunk size is smaller than the logical page size, A method for restoring data loss in a flash storage device. The method according to claim 1,
Wherein the step of recording the metadata in a memory comprises:
And generating metadata including information on a data storage method corresponding to the write-requested data. The method of claim 3,
Wherein the step of recording the metadata in a memory comprises:
Generating metadata including a data chunk size and position information of a parity chunk when the determined data storage scheme is an erasure coding scheme and generating metadata including location information of the replicated data when the determined data storage scheme is a replication scheme, Wherein the flash memory device is a flash memory device. Receiving a data read request from a host;
Confirming a data storage mode of the read-requested data by referring to metadata of the read-requested data;
Reading a plurality of original data chunks from a memory when the data storage method is an erasure coding method; And
When the data storage method is a replication method, a process of reading either original data or copied data from the memory
The method comprising the steps of: 6. The method of claim 5,
Reading parity chunks from the memory when data loss occurs in at least one of the plurality of original data chunks; And
A process of restoring lost data using the parity chunk
The method comprising the steps of: Upon receiving the data write request from the host, the data storage method of either the erase coding method or the replication method is determined based on the size of the data requested to be written, and the metadata including the information about the write requested data is generated And writing the requested data into the memory according to the determined data storage mode; And
The method comprising: receiving a data read request from a host, checking a data storage mode of the read-requested data by referring to the metadata of the requested data; if the data storage scheme is an erasure-coding scheme, And reading the original data and the duplicated data from the memory when the data storage method is a duplication method
The method comprising the steps of: 8. The method of claim 7,
And writing the requested data to the memory,
The data chunk size of the memory is compared with the logical page size of the memory by applying the erasure coding scheme and if the data chunk size is equal to or greater than the logical page size, And if the chunk size is smaller than the logical page size, the data storage method is determined by the replication method. 8. The method of claim 7,
Wherein when a data loss occurs in at least one of the plurality of original data chunks, parity chunks are read from the memory, and the lost data is recovered using the parity chunk. Way. A storage controller that receives a data write request from a host and determines a data storage method of either an erase coding method or a replication method based on a size of data requested to be written; And
Wherein the write request data is recorded in the flash memory according to a determined data storage format,
/ RTI > 11. The method of claim 10,
The storage controller comprising:
The data chunk size of the flash storage device is compared with the logical page size of the flash storage device when the erasure coding method is applied and the data storage method is determined by the erasure coding method when the data chunk size is equal to or greater than the logical page size, Wherein the data storage method is determined by a replication method when the data chunk size is smaller than the logical page size. 11. The method of claim 10,
The metadata includes:
And information on a data storage mode corresponding to the write-requested data. 13. The method of claim 12,
The metadata includes:
And location information of a chunk of data when the determined data storage scheme is an erasure coding scheme, and location information of the replicated data when the determined data storage scheme is a replication scheme. 11. The method of claim 10,
The storage controller comprising:
Wherein the flash memory device reads data from the flash memory by checking the data storage method of the read requested data by referring to the meta data of the read requested data when receiving a data read request from the host. 15. The method of claim 14,
The storage controller comprising:
Wherein the data storage method reads a plurality of original data chunks from the flash memory when the data storage method is an erasure coding method and reads either original data or copied data from the flash memory when the data storage method is a replication method . 16. The method of claim 15,
The storage controller comprising:
Reads a parity chunk from the flash memory when data loss occurs in at least one of the plurality of original data chunks, and restores the lost data using the parity chunk.

Images