KR101731832B1 - Method and Apparatus of Encoding and Decoding for Data Recovery in Storage System - Google Patents

Abstract

The present invention relates to an encoding and decoding method for recovering data lost in a storage system, and to an apparatus therefor. The storage system supports a function of recovering nodes that stores data in a distributed storage environment when the nodes are lost. More specifically, the present invention relates to an encoding and decoding method for recovering data lost in a distributed storage system, and to an apparatus therefor. The encoding method for recovering data lost in a storage system according to one embodiment of the present invention comprises: a division step of dividing data to be stored into a predetermined number of data blocks; an encoding step of generating information symbols according to the divided data blocks, selecting two different data blocks among the divided data blocks, and generating parity symbols by using the selected two data blocks; and a storage step of storing each of the generated symbols in each node of a storage system. The encoding step enables to select two different data blocks among the divided data blocks and to encode the parity symbols by using the selected two data blocks in order to maintain the number of the nodes required for recovering symbols stored in the lost nodes and the number of the parity symbols generated in the encoding step at a predetermined limit.

Description

[0001] The present invention relates to an encoding and decoding method and apparatus for recovering data loss in a storage system,

The present invention relates to a coding and decoding method and apparatus for recovering lost data in a storage system and a storage system supporting a function of recovering a lost node in a distributed storage environment. In particular, the present invention relates to an encoding and decoding method for data loss recovery in a distributed storage system and an apparatus therefor.

A cloud storage system for secure storage and recovery of Big Data has been introduced and is actively being used. For example, companies like Google, Facebook, Amazon, and Microsoft, which provide services such as search and data delivery around the world, use cloud storage systems, or distributed storage systems, to store vast amounts of data.

The amount of information stored and distributed in the distributed storage system is rapidly increasing due to factors such as the increase of service users and the quality of contents. However, in a distributed storage system, data is lost due to a defect in the equipment in the distributed node, software, or hardware update. Therefore, various types of coding techniques have been developed and used to recover lost data in response to such data loss.

For example, there is a repetition coding method as the most basic existing coding method for recovering lost data in a distributed storage system. In this method, multiple identical copies of data are created and stored in multiple distributed nodes, so that any one of the duplicated nodes can be recovered using the remaining nodes, even if one of them is lost.

An important index in the coding scheme for data loss recovery is a locality value indicating whether at least some other nodes can be used to recover lost nodes when some of the nodes of the distributed storage system are lost. Also, the capacity or coding rate (coded rate) of coded data by data loss recovery coding to support the above specific localities is also an important index.

However, existing coding and decoding schemes for data loss recovery suggest only a method of guaranteeing the locality, which is the number of nodes required to recover a specified number of nodes when they are lost, There is a limitation in that it can not guarantee the integrated localizedness for the node loss. In addition, existing coding and decoding techniques for data loss recovery have a limit in that they can not consider a coding rate optimization method with two or more specified number of node losses.

Japanese Patent Application Laid-Open No. 10-2011-0051527 (May 18, 2011)

SUMMARY OF THE INVENTION It is an object of the present invention to provide a method and apparatus for maintaining a node within a predetermined number of nodes to be used for recovering a node lost in the case of two or more specified number of node losses, And at the same time to provide a coding and decoding technique for optimizing a coding rate and a distributed storage system accordingly.

According to an aspect of the present invention, there is provided a method for restoring data loss in a storage system, the method comprising: dividing data to be stored into a predetermined number of data blocks; Generating an information symbol according to the divided data block, selecting two different data blocks from the divided data blocks, and using the selected two data blocks to generate a parity symbol (Parity Symbol) Encoding step; And a storing step of storing each symbol generated in each node of the storage system.

Wherein the encoding step comprises: inputting an input matrix having input symbols representing each of the divided data blocks as an element; generating a matrix having an element value of 1 or 0 and the number of 1s included in each column vector is 2 or less; Wherein an element corresponding to one input symbol in the output matrix is the information symbol, and an element corresponding to a sum of two input symbols in the output matrix is multiplied by the parity symbol (Generator Matrix) Respectively.

Wherein the generator matrix has a unit matrix of the number of elements of the input matrix and all column vectors having two elements of which the row sequences are adjacent to each other is 1, wherein the generator matrix has a length according to the number of elements of the input matrix, And the matrix is a matrix in which

Wherein the generating matrix has a length corresponding to the number of elements of the input matrix and the generating matrix has a unit matrix of the number of elements of the input matrix and five columns of two elements of which row- And a matrix in which all the combinations of the column vectors of any one element included in any one column vector of the five column vectors and one element not included in the five column vectors are connected, can do.

Wherein the column vector of the generator matrix has a length corresponding to the number of elements of the input matrix and the generator matrix has a unit matrix of the number of elements of the input matrix and a column of two And a matrix in which all the combinations of column vectors in which one element included in the one column vector and one element not included in the one column vector are 1 are connected.

Wherein, when a node of the storage system is lost, the node can be recovered by using the remaining nodes other than the lost node, and the encoding step includes the steps of: The method comprising: selecting two different data blocks from the divided data blocks to maintain the number of parity symbols generated in the encoding step to a predetermined limit; Can be encoded.

Here, the encoding step may code each of the parity symbols by selecting and using two data blocks so that each data block is used only twice to generate two parity symbols for all the data blocks .

Here, if one node corresponding to any one of the information symbols of the storage system is lost, the lost node may be recovered using two nodes other than the lost node, When two nodes corresponding to any two information symbols of the nodes of the system are lost, the lost two nodes can be recovered by using four nodes other than the lost node .

Here, when the data to be stored is divided into k data blocks, the coding step generates k pieces of the information symbols and k pieces of the parity symbols using k pieces of the data blocks twice .

Wherein the encoding step includes the steps of: selecting a first data block, a second data block, a third data block, and a fourth data block among the data blocks; a pair of the first data block and the second data block; Coding the parity symbols for each pair using a pair of the first data block, the second data block and the third data block, and the third data block and the fourth data block, For each of the remaining data blocks, a pair of each of the remaining data blocks and the first data block, each of the remaining data blocks and the second data block for each of the remaining data blocks, And encoding the symbol.

Here, if one node corresponding to any one of the information symbols of the storage system is lost, the lost node may be recovered using two nodes other than the lost node, When two nodes corresponding to any two information symbols of the nodes of the system are lost, it is possible to recover the two lost nodes using three nodes other than the lost node, If one of the nodes of the storage system is lost, the lost node can be recovered using two nodes other than the lost node, and if any two nodes of the storage system are lost, It is possible to recover the lost two nodes by using four nodes other than the lost node.

Here, when the data to be stored is divided into k data blocks, the encoding step may generate k pieces of the information symbols and generate 2k-5 pieces of the parity symbols.

The encoding step may include a step of selecting a first data block and a second data block of the data block, a step of encoding the parity symbol using a pair of the first data block and the second data block, The remaining data blocks and the first data blocks, and the remaining data blocks and the second data blocks are used for the remaining data blocks except for the data block selected as the block in the remaining data blocks And encoding the patterned symbols by the pair.

If one node corresponding to any one of the information symbols of the storage system is lost, the lost node can be recovered using two nodes other than the lost node, If two nodes corresponding to any two information symbols among the nodes of the storage system are lost, the lost two nodes can be recovered using three nodes other than the lost node, When one arbitrary node is lost, the lost node can be recovered using two nodes other than the lost node, and if any two nodes of the storage system are lost, It is possible to recover the lost two nodes by using three nodes other than the lost node.

Here, when the data to be stored is divided into k data blocks, the encoding step may generate k pieces of the information symbols and generate 2k-3 pieces of the parity symbols.

According to an aspect of the present invention, there is provided a decoding method for recovering data loss in a storage system according to one aspect of the present invention includes an information symbol generated for a data block obtained by dividing data to be stored into a predetermined number, When a parity symbol generated by using two different data blocks among the divided data blocks is lost in a node of a storage system stored in each node, The method comprising the steps of: storing a number of nodes needed to recover a symbol stored in the lost node and a number of the parity symbols generated by the encoder; To calculate the symbol stored in the node among the remaining nodes Node selection step of selecting is at least one or more of the nodes as possible to obtain the symbol stored in the lost node; And recovering the symbol stored in the lost node using the selected node.

Wherein the storage system is configured to use the parity symbol encoded by selecting and using two data blocks so that each data block is used only twice to generate two parity symbols for all the data blocks, And when one node corresponding to any one of the information symbols of the nodes of the storage system is lost, the node selecting step selects the symbol stored in the node among the remaining nodes other than the lost node And selecting the two nodes capable of acquiring the symbol stored in the lost node, the recovering step restores the lost node by using the selected node, When two nodes corresponding to the two information symbols of the node < RTI ID = 0.0 > The selecting step selects the four nodes capable of operating the symbol stored in the node among the remaining nodes other than the lost node to obtain the symbol stored in the lost node, And recovering the lost node by using the node.

Wherein the storage system is configured to select a first data block, a second data block, a third data block, and a fourth data block among the data blocks, a pair of the first data block and the second data block, Encoding the parity symbols for each pair using a pair of the data block and the third data block, the third data block and the fourth data block, For each of the remaining data blocks, a pair of the remaining data block and the first data block, each of the remaining data blocks and the second data block for each of the remaining data blocks, And storing the parity symbols encoded in the encoding process in each node, When one node corresponding to any one of the information symbols is lost, the node selecting step calculates the symbol stored in the node among the remaining nodes other than the lost node, and stores the symbol stored in the lost node Selecting one of the two nodes capable of acquiring the symbol, restoring the lost node by using the selected node, and selecting a node corresponding to any two of the information symbols of the nodes of the storage system The node selecting step comprises the steps of: computing a symbol stored in the node, which is stored in the node, of the remaining nodes other than the lost node, And the recovery step restores the lost node by using the selected node . ≪ / RTI >

Here, the storage system may include the steps of selecting a first data block and a second data block of the data block, encoding the parity symbol using a pair of the first data block and the second data block, A pair of the remaining data blocks and the first data blocks and a pair of the remaining data blocks and the second data blocks for the remaining data blocks, , Storing the parity symbols encoded by coding the parity symbol on the pair basis in each node, and when one node corresponding to any one of the information symbols of the storage system is lost , The node selecting step may include a step of selecting a node among the remaining nodes other than the lost node The node selecting two nodes capable of calculating the symbol stored in the lost node and obtaining the symbol stored in the lost node, and the recovering step restores the lost node using the selected node, When two nodes corresponding to any two of the information symbols of the nodes of the system are lost, the node selecting step calculates the symbols stored in the node among the remaining nodes other than the lost node, The node selecting three nodes capable of acquiring the symbol stored in the node, and the restoring step restores the lost node by using the selected node.

According to an aspect of the present invention, there is provided an encoding apparatus for recovering data loss in a storage system, the apparatus comprising: a storage unit for storing data to be stored in a storage system; An encoding unit for generating a parity symbol using the selected two data blocks, selecting two different data blocks from the divided data blocks, and generating a parity symbol using the selected two data blocks; And a data processing unit for storing the generated symbols in each node of the storage system, wherein when the node of the storage system is lost, the encoding unit updates the symbols stored in the lost node, And the number of parity symbols generated by the encoder is set to a predetermined limit, the data block selecting unit selects two different data blocks from the divided data blocks, and uses the selected two data blocks to generate the parity symbol And a second encoding unit.

According to one aspect of the present invention, there is provided a decoding apparatus for recovering data loss in a storage system, comprising: an information symbol generated for a data block obtained by dividing data to be stored into a predetermined number; When a parity symbol generated by using two different data blocks among the divided data blocks is lost in a node of a storage system stored in each node, Wherein the number of nodes required to recover a symbol stored in the lost node and the number of parity symbols generated by the encoder are maintained at a predetermined limit, To calculate the symbol stored in the node among the remaining nodes The stored in the lost-node selection unit that selects at least one or more of the nodes as possible to obtain the symbols; And a recovery unit for recovering the symbols stored in the lost node using the selected node.

According to the encoding and decoding method for recovering data loss in the distributed storage system according to the present invention, the number of nodes to be used for restoring lost nodes in the case of two or more specified number of node losses is maintained within a predetermined number The coding rate can be optimized at the same time while guaranteeing a predetermined joint locality.

1A to 1C are reference views for explaining an existing distributed storage system.
FIG. 2 is a reference diagram showing a manner in which a conventional iterative encoding method operates.
3 is a flowchart of a method for recovering data loss in a distributed storage system according to an embodiment of the present invention.
4 is a reference diagram for explaining a coding method according to the ring code technique according to the present invention.
5 is a reference diagram showing a generation matrix according to a ring code technique according to the present invention.
6 is a reference diagram for explaining features of the ring code technique for optimizing the coding rate.
7 is a reference diagram for explaining a coding method according to the crown code technique according to the present invention.
8 is a reference diagram showing a generation matrix according to the crown code technique according to the present invention.
9 is a reference diagram for explaining a feature of the crown code technique for optimizing the coding rate.
FIG. 10 is a reference diagram for explaining a coding method according to the tiara code technique according to the present invention.
11 is a reference diagram showing a generation matrix according to the tiara code technique according to the present invention.
12 is a flowchart of a decoding method for data loss recovery in a distributed storage system according to an embodiment of the present invention.
13 is a block diagram of an encoding apparatus for data loss recovery in a distributed storage system according to an embodiment of the present invention.
14 is a block diagram of a decoding apparatus for recovering data loss in a distributed storage system according to an embodiment of the present invention.
15 is a reference view showing a distributed storage system including an encoding apparatus and a decoding apparatus according to an embodiment of the present invention.
16 is a reference view showing a distributed storage system including a node management server according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same or similar components throughout the drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. In addition, the preferred embodiments of the present invention will be described below, but it is needless to say that the technical idea of the present invention is not limited thereto and can be variously modified by those skilled in the art.

As the age of big data comes, a distributed storage system using several nodes on the network has been developed and used to store a large amount of big data. Distributed storage system is a storage system that divides original data into several blocks, stores them in nodes of distributed cloud through encoding, and acquires original data by collecting data stored in each node when original data is needed do.

An important point in such a distributed storage system is to distribute data to a plurality of nodes and to restore the data to the original data. However, as the total amount of storage space used in the distributed storage system is greatly increased, some of the storage nodes are lost or lost due to errors in the storage hardware device, equipment in the storage system, software update, . For example, Facebook has dozens of petabytes (1024 times as many terabytes) of storage space for photo files, and dozens of storage nodes are periodically lost among thousands of storage nodes.

Therefore, in the distributed storage system, it is impossible to cope with the loss of the storage node by simply dividing the original data, distributing the original data to a plurality of nodes, storing the same, and then restoring the original data. Various coding methods have been proposed for dividing original data and distributing the original data to nodes in order to cope with the loss of the storage node. In the conventional distributed storage system, when original data is divided, separate data for data recovery is encoded and stored in each node together with original data. Then, when a lost node occurs, Or recovering the lost node or restoring the contents of the lost node to the alternative node.

FIGS. 1A to 1C illustrate a case where a conventional distributed storage system divides and encodes original data and stores the divided data in distributed nodes on the network, collects them again to acquire original data, FIG. 6 is a reference diagram showing a process of recovering a node. FIG.

FIG. 1A is a reference diagram showing a process in which original data is divided into a predetermined number of data blocks and then stored in distributed nodes through an encoding process. Also, FIG. 1B is a reference diagram showing a process of selecting a part (k) of nodes (n) dispersed as shown in FIG. 1A and collecting data from selected nodes to acquire original data. FIG. 1C illustrates a method of selecting a part (d) of remaining nodes D2 to Dn when one of the nodes (for example, D1) is lost, collecting data from the selected nodes, (D1) is restored and stored in an alternative node. That is, by using such a coding method for recovering the data loss, even if a part of a plurality of nodes dividing and storing the original data is lost, the lost node can be recovered by using the remaining nodes, Similarly, the lost nodes can be periodically recovered to obtain the raw data needed by the managed distributed storage system.

For example, there is a repetition coding method as the most basic existing coding method for recovering lost data in a distributed storage system. In this method, when the original data is divided, several identical copies are created and stored in various distributed nodes, and if one of the copied nodes is lost, the remaining nodes are used to recover the lost node.

Fig. 2 is a reference diagram showing a manner in which such a repetition coding method operates.

As an index for evaluating the performance of the coding method for recovering lost data in the distributed storage system, there are an acquisition success rate of original data, a recovery success rate of lost data, a storage capacity of a distributed node, a recovery bandwidth or a coding rate, And the number of nodes. Here, the recovery bandwidth is an index indicating the amount of data to be downloaded in order to recover the lost node.

In particular, here, the number of access nodes for recovery refers to the minimum number of nodes that must be used to recover any lost node, and is expressed as a locality. In a distributed storage system, this locality is the number of the minimum number of nodes that must be connected and used in the process of recovering any lost node. Here, the number of the minimum number of nodes means the minimum number of nodes that can recover the lost node if any node stored in the storage system is lost. However, since all of the nodes need to be applicable to the arbitrary node, the maximum value among the minimum number of nodes required for restoration upon loss of each node stored in the storage system becomes the locality value. In more detail, when the node N _i stored in the storage system is lost, the minimum number of nodes required to recover the node N _i is K _i , i is the index of the node, If M, the locality of the encoded code can be defined as max {K _i }, i = 1, ..., M.

Also here, if the dog is any node l loss minimum number of nodes that must be used to recover the loss of the node l l - it can be defined as a locality (r _l).

For example, if any one of the n nodes is lost and at least three nodes are needed to recover it, then ( l = 1) - the locality is 3. Such locality is an important index that determines the performance of the distributed storage system because it is an index that determines the number of nodes needed to recover the lost node. If a coding method that guarantees less locality is used, it is advantageous to recover lost nodes faster and reduce network traffic because fewer nodes are required to recover lost nodes.

However, in the existing distributed storage system, a method of guaranteeing the locality (or l - locality), which is the number of nodes required to recover a specified number of nodes, has been proposed. However, since there are various cases in which nodes are lost on a real distributed storage system, a coding method that guarantees an optimized locality must be devised.

Accordingly, a coding method for data loss recovery according to the present invention is defined as a joint locality which together represents the number of minimum nodes necessary for restoring two or more specified number of node losses, Disclosed is an encoding method for data loss recovery that can optimize a coding rate while ensuring a certain number of localities.

For example, when a ring code scheme or a crown code scheme, which will be described below, is used, the encoding method for data loss recovery according to the present invention guarantees localities 2 and 4 when one node is lost and two nodes are lost, respectively And a coding method. In this case, the joint localities are expressed as ((r ₁ , r ₂ ) = (2, 4)). Also, when using the tiara code technique to be described below, the encoding method for data loss recovery according to the present invention also provides a coding method for guaranteeing localities 2 and 3, respectively, when one node is lost and two nodes are lost . In this case, the joint localities are expressed as ((r ₁ , r ₂ ) = (2, 3)).

In the present invention, the above-described joint localities in the case where nodes storing information symbols to be described below are lost are defined as Joint Information Locality. In other words, there are an information symbol generated for a data block obtained by dividing data to be stored and a parity symbol generated by encoding the data blocks, and the symbols thus generated are stored in a storage system The joint locality is defined as the joint information locality when the node storing the information symbol is lost. Therefore, the joint information locality is the minimum number of nodes needed to recover it from the loss of two or more specific information nodes (nodes storing information symbols).

For example, in the case of using a ring coding technique to be described below, the coding method for data loss recovery according to the present invention is a coding method for guaranteeing locality 2 and 4, respectively, when one information node is lost and two information nodes are lost ≪ / RTI > In this case, the joint information locality is expressed as ((r ₁ , r ₂ ) _info = (2, 4)). When a coded code scheme or a tiara code scheme, which will be described below, is used, a coding scheme for data loss recovery according to the present invention is a coding scheme for ensuring localities 2 and 3 when a node is lost, Method. In this case, the joint information locality is expressed as ((r ₁ , r ₂ ) _info = (2, 3)).

Hereinafter, a coding method, an apparatus, a decoding method, and a device for recovering data loss in a distributed storage system according to the present invention will be described in detail. Hereinafter, the description of the distributed storage system is equally applied to a storage system that stores data in at least one or more nodes. Hereinafter, a storage system storing data in at least one node will be referred to as a distributed storage system, and a method and apparatus for encoding and decoding the data for loss recovery will be described.

3 is a flowchart of a method for recovering data loss in a distributed storage system according to an embodiment of the present invention.

The encoding method for data loss recovery in the storage system according to an exemplary embodiment of the present invention may include a storage target data segmentation step S100, an encoding step S200, and a storage step S300.

The operation of the storage target data segmentation step S100, the encoding step S200, and the storage step S300, which will be described in detail below, may be performed by the encoding device 10 connected to the storage system. Here, the encoding device 10 may be a server device connected to the storage system, a computer device, or an embedded system for performing a coding function.

The storage target data segmenting step S100 divides the storage target data into a predetermined number of data blocks.

The encoding step S200 generates an information symbol according to the divided data block, selects two different data blocks from the divided data blocks, and uses the selected two data blocks to generate parity And generates a symbol (Parity Symbol).

Herein, the information symbol and the parity symbol generated as above are collectively referred to as a symbol.

In the encoding step S200, the encoding device 10 generates each data block as an information symbol, and generates a parity symbol for each combination of two data blocks. For example, when the data to be stored is divided into three data blocks A, B, and C, the encoding device 10 generates information symbols S_A, S_B, and S_C according to A, B, and C blocks, (A, B), S_ (B, C) as parity symbols for each combination of (A, B), (B, C) , And S_ (A, C). Hereinafter, X and Y are arbitrary data blocks, S_X denotes an information symbol generated for an X data block, S_ (X, Y) denotes an X data block and 2 Y data blocks, Quot; and " parity symbol "

The storage step (S300) stores each symbol generated in each node of the storage system.

That is, the symbols including the information symbol and the parity symbol generated in the encoding step S200 may be stored in each node on the distributed storage system. Wherein each node of the distributed storage system may be a node formed in an information storage device in a distributed storage system and may be interconnected on a network.

Here, the node storing the information symbol is an information node, and the node storing the parity symbol is a parity node. (A, B), S_ (B, C), S_ (A, C) generated in the coding step S200 are transmitted to each node of the distributed storage system Lt; / RTI >

Here, the node of the distributed storage system has a property that, when a specific node is lost, it can be restored by using the remaining nodes other than the lost node. This has the same meaning as that in the generated symbol, if a specific symbol is lost, it can be recovered using the remaining symbols other than the lost symbol. Therefore, in describing a method of recovering a specific symbol lost or a specific node is lost, the description of the symbol can be applied to the node where the corresponding symbol is stored. The same holds true for stored symbols. Recovering the lost node may then be to recover the symbol stored in the lost node. At this time, the recovered data may be stored in the original storage node, or may be stored as an alternative node in another storage location.

Hereinafter, the encoding step S200 will be described in more detail. In the encoding step S200, the encoding device 10 generates an information symbol according to the divided data block, selects two different data blocks from the divided data blocks, A parity symbol is generated using the data block. Here, the encoding step S200 may include a step of encoding the number of nodes needed to recover the symbol stored in the lost node and the number of parity symbols generated in the encoding step S200 to a predetermined coding rate , It is possible to select two data blocks of a part of the divided data blocks and to encode the parity symbols using the selected two data blocks.

As will be described in detail below, the encoding step (S200) comprises, for the node stored in the storage system, accessing up to four or less of the node losses in all cases of not more than two node losses, And generates the parity symbol so as to guarantee recovery.

Here, the encoding step S200 may generate a parity symbol using any one of a ring code technique, a crown code technique, and a tiara code technique, which will be described in more detail below. When the coding is performed using the above techniques, the storage system for storing the symbols and the nodes according to the coding scheme maintains a predetermined joint locality or joint information locality value, and at the same time, the joint locality or the joint information locality value Lt; / RTI >

Here, the coding rate (code rate) can be defined as the number of information symbols divided by the total number of symbols. Since the number of information symbols is determined according to the number of divided pieces of data to be stored, The coding rate is changed according to the number of parity symbols generated by coding. In this case, the smaller the number of generated parity symbols, the more the coding rate is improved. This means that the coding rate is improved by reducing the number of generated parity symbols. This means that when the lost node is recovered, the total number of nodes storing the parity symbols to be connected is reduced, It can mean to reduce the number. That is, according to the coding step S200 according to the present invention, the number of parity symbols generated in the coding step S200 is reduced while maintaining the number of nodes required for recovering symbols stored in the lost node to a predetermined level The coding rate can be optimized.

Where information symbols are generated corresponding to the data blocks. That is, the information symbol includes all the contents of the data block and is generated corresponding to each data block so as to acquire the contents of the data block using the information symbol. The parity symbol is generated by encoding using a plurality of data blocks as described above.

Here, in encoding step S200, when parity symbols are encoded using two data blocks, parity symbols may be encoded by adding two data blocks. Here, the method of encoding a parity symbol is not limited to adding two data blocks, and various methods of generating parity symbols using a plurality of data blocks in the distributed storage system can be used.

In the encoding step S200, when the parity symbol is generated, a parity symbol is generated using the selected data blocks by selecting a part of the data blocks into which the storage target data is divided, A code block, a coded code block, a tier code block, and a parity symbol can be generated according to the selected block.

First, a description will be given of a method of encoding a parity symbol using a ring coding scheme in a coding step (S200).

Here, the encoding step S200 selects and uses the two data blocks so that each data block is used only twice to generate two parity symbols for all the data blocks, and the parity symbols are encoded .

4 is a reference diagram for explaining a coding method according to a ring code technique. The operation of the encoding method using the ring code scheme can be expressed as a graph as shown in FIG. Referring to FIG. 4A, for example, when the number k of divided data blocks is 3, the graph node transmits the information symbol / node (here, the graph node is a concept different from the node of the distributed storage system) (The parity symbol is generated using the two graph nodes connected to the edge), and the symbols generated in the encoding step S200 are the graphs of FIG. 4 (a) Can be expressed as 4 (b) when k is 4, and (c) when k is 8 can be expressed as shown in FIG. 4 (c). And for arbitrary k, graph nodes can also be connected to the edge in the form of a ring. Thus, in the ring code scheme, an information symbol / node is connected to an edge in the form of a ring, and a parity symbol is encoded according to an edge generated due to the above connection.

In the present invention, the coding method expressed by the graph of FIG. 4 is referred to as a coding method using a ring code technique.

The operation of the encoding step S200 using the ring code technique can be expressed as a generator matrix as follows.

Here, the encoding step S200 includes an input matrix in which each element is an input symbol representing the divided data blocks, and an input matrix in which the value of the element is either 1 or 0, and the number of 1s included in each column vector is 2 or less The output matrix can be generated by multiplying the generator matrix by the generator matrix. The condition of the generation matrix is a common condition in the crown code technique and the tiara code technique to be described below. Herein, the input symbol refers to a symbol representing each data block, and is referred to as an 'input' symbol and an 'input' matrix in the sense that an input matrix, which is a matrix formed by these input symbols, is multiplied by the generator matrix to generate an output matrix. At this time, the column vector of the generator matrix has a length corresponding to the number of elements of the input matrix.

을 곱하여 출력 행렬로 [A, B, C, A+B, A+C, B+C]를 생성할 수 있다. 이는 도 4의 (a)의 경우 이용되는 생성 행렬이다. For example, an input matrix [A, B, C] in which each element represents A, B, and C representing each divided data block, and an input matrix [ Generation matrix with two or fewer

A, B, C, A + B, A + C, B + C] as the output matrix. This is the generation matrix used in case of FIG. 4 (a).

Here, the encoding step S200 can generate the information symbol and the parity symbol according to the output matrix generated using the generator matrix as described above. First, an element corresponding to one input symbol in the output matrix is generated as the information symbol. And an element corresponding to the sum of the two input symbols in the output matrix is generated as the parity symbol. For example, the elements A, B, and C corresponding to one input symbol are generated as the information symbols S_A, S_B, and S_C, and the elements corresponding to the sum of the two input symbols A + B, A + C, B + C can be generated by each parity symbol S_ (A, B), S_ (A, C), S_ (B, C).

Also, the generator matrix may include another matrix for reordering the columns of the generator matrix. In other words, the generator matrix may include all the matrices that can be rearranged between the column vectors of the generator matrix, i.e., all possible column-permutation matrices, . The contents of generating the information symbol and the parity symbol according to the output matrix and the content of the column-mixing are also applied to the crown code technique and the tiara code technique to be described below.

Here, the generation matrix when the ring code scheme is used is a matrix formed by concatenating a unit matrix of the number of elements of the input matrix and all column vectors of two elements whose adjacent rows are adjacent to each other. In this case, the generation matrix may be a matrix formed by concatenating all column vectors whose two adjacent elements in the row order are 1, and column vectors whose first and last two elements are 1 in the row order.

5 is a reference diagram showing a generation matrix according to the ring code technique described above. In this way, the generation matrix according to the ring code scheme can be composed of a k-sized unit matrix and a remaining portion equal to the number of divided data blocks, and the remaining column vectors include all column vectors having two adjacent elements of 1 . Here, when the row order is the first and last row as in the rightmost column vector in FIG. 5, it is assumed that the two elements are adjacent to each other.

When the ring code scheme is used, when one node corresponding to any one of the information symbols of the nodes of the storage system is lost, using the two nodes other than the lost node, Can be recovered. In addition, when two nodes corresponding to any two of the information symbols of the storage system are lost, the lost nodes can be recovered using four nodes other than the lost node. That is, when encoding is performed using the ring code technique, the joint information locality becomes ((r ₁ , r ₂ ) _info = (2, 4)).

Also, when a ring code scheme is used, and if any one of the nodes of the storage system is lost, the lost node can be recovered by using two nodes other than the lost node, If any two nodes of the system are lost, the lost two nodes can be recovered using four nodes other than the lost node. That is, when encoding is performed using the ring code technique, the joint localities are also ((r ₁ , r ₂ ) = (2, 4)).

The code is encoded by using a ring code technique is a joint information locality is _{((r 1, r 2)} info = (2, 4)), the joint is locality ((r _1, and r ₂₎ = _(2, 4)) encode, the code having the best coding rate is obtained.

6 is a reference diagram for explaining features of the ring code technique for optimizing the coding rate. 6 (a), when the parity symbol is encoded according to the edge connecting the information symbols, the joint information locality is ((r ₁ , r ₂ ) _info = (2, 4) It is not optimized code. On the other hand, when the parity symbol is coded according to the edge connecting the information symbols according to the ring code scheme as shown in FIG. 6 (b), the joint information locality is ((r ₁ , r ₂ ) _info = (2, 4) And the code rate is optimized at the same time.

Here, the minimum distance can be defined as an index indicating the performance of the encoded code. Even if a node (symbol) is lost up to k - 1, the lost node (symbol) If recoverable, the minimum distance d, which is an indicator of the number of recoverable loss nodes (symbols), can be defined as k. In other words, even if the node is lost by one less than the minimum distance d, the lost node can be recovered by using another node.

According to this, even if up to two nodes (symbols) are lost in the code coded using the ring code technique, the lost nodes (symbols) can be recovered by using other nodes (symbols). In this case, the minimum distance d, which is an index indicating the number of recoverable loss nodes (symbols), is 3.

Assuming that a code block using the coding method in which the data block divided into k blocks is generated with n symbols (including information symbols and parity symbols) and the minimum distance is d is expressed as [n, k, d] ₂ , The code according to the ring code technique is [2k, k, 3] ₂ code. That is, when the data to be stored is divided into k data blocks, the encoding step (S200) generates k pieces of the information symbols and generates k pieces of the parity symbols using all k pieces of the data blocks twice . In this case, the coding rate can be defined as k / n, which is the ratio between the number of data blocks and the number of symbols, and the coding rate is 1/2 in the case of codes encoded using the ring coding technique.

Next, a description will be made of a method of encoding the parity symbol using the coded code scheme in the encoding step (S200).

7 is a reference diagram for explaining a coding method according to the crown code technique. The operation of the encoding method using the crown code technique can be expressed as a graph as shown in FIG. Referring to FIG. 7A, for example, when the number k of divided data blocks is 5, when the graph node represents an information symbol / node and the graph edge represents a parity symbol / node, S200 may be expressed as a graph of FIG. 7 (a). 7 (b) when k is 6, and FIG. 7 (c) when k is 9. Similarly, for all cases where k is greater than or equal to 5, edges are concatenated to form a pentagon with respect to five graph nodes as shown in FIG. 7, and then two graph nodes are selected, You can create a graph node to connect to both graph nodes and edges. In the crown code scheme, an information symbol / node is connected to an edge in the form of a crown, and a parity symbol is encoded according to an edge generated due to the above connection.

In the present invention, the coding method expressed by the graph of FIG. 7 is referred to as a coding method using a Crown Code technique.

Here, when encoding is performed using the crown code technique, the encoding step (S200) can operate according to the following procedure.

Here, the encoding step S200 may include a step of selecting the first data block, the second data block, the third data block, and the fourth data block among the data blocks. Referring to FIG. 7B, information symbols S_1, S_2, S_3, and S_4 corresponding to each data block can be selected.

Here, the encoding step S200 may include a pair of the first data block and the second data block, a pair of the second data block and the third data block, and a pair of the third data block and the fourth data block, respectively And encoding the parity symbols by the pair. Referring to FIG. 7B, a parity symbol S_ (1, 2) is generated using a pair of S_1 information symbol and S_2 information symbol, and a parity symbol S_ (1, 2) is generated using a pair of S_2 information symbol and S_3 information symbol. 2, 3), and generates a parity symbol S_ (3, 4) using the pair of the S_3 information symbol and the S_4 information symbol.

Herein, the encoding step (S200) may include, for the remaining data blocks except for the data block selected as the gender of the data block, a pair of the remaining data blocks and the first data blocks, And encoding the pattern data using the first and second data block pairs. Referring to FIG. 7B, a parity symbol S_ (1, 5) is generated using an S_5 information symbol and a S_1 information symbol pair for an S_5 information symbol corresponding to one of the remaining data blocks, and S_5 information And the parity symbol S_ (2, 5) can be generated using a pair of S_2 information symbols. Also, for the S_6 information symbol corresponding to the other data block, the parity symbol S_ (1, 6) is generated by using the pair of the S_6 information symbol and the S_1 information symbol, and the pair of the S_6 information symbol and the S_2 information symbol To generate parity symbols S_ (2, 6).

The operation of the encoding step S200 using the crown code technique can be expressed as a generation matrix as follows.

To this end, the coding step S200 includes an input matrix in which each element of the input symbol represents the divided data block, and a matrix in which the value of the element is either 1 or 0, and the number of 1s included in each column vector is 2 (Generator Matrix) to generate an output matrix. And an element corresponding to one input symbol in the output matrix is the information symbol, and an element corresponding to a sum of two input symbols in the output matrix is the parity symbol. At this time, the column vector of the generator matrix has a length corresponding to the number of elements of the input matrix.

Here, the generation matrix in the case of using the crown code scheme is composed of a unit matrix of the number of elements of the input matrix, five column vectors of two elements whose row order is mutually adjacent, and one column of the five column vectors And a combination of all the combinations of column vectors in which one element included in the vector and one element not included in the five column vectors are 1 is connected.

8 is a reference diagram showing a generation matrix according to the crown code technique. The generation matrix according to the crown code technique can be composed of a k-sized unit matrix equal to the number of divided data blocks and the remaining part. At this time, if k is greater than 5, the remaining part is divided into a first matrix part and a second matrix part, and if k is 5, the remaining part can be represented by only the first matrix part. Here, the first matrix part may include five column vectors selected from five rows among the k rows as shown in FIG. 8, and two adjacent elements in the column direction in the selected five rows. Here, even when the row order is first and last, it is assumed that the two elements are adjacent to each other. Next, the second matrix part is a matrix included in the remaining part when k is larger than 5, selects two rows from the five selected rows as shown in FIG. 8, and selects one of the two selected rows as 1 And all of the column vectors consisting of elements whose elements are either 1 or not included in the selected 5 rows.

In the case of using the crown code technique, when one node corresponding to any one of the information symbols of the storage system is lost, the lost node, using two nodes other than the lost node, Can be recovered. Also, when two nodes corresponding to any two information symbols of the nodes of the storage system are lost, the lost nodes can be recovered using three nodes other than the lost node. That is, when coding is performed using the crown code technique, the joint information locality becomes ((r ₁ , r ₂ ) _info = (2, 3)).

In addition, when any one of the nodes of the storage system is lost, it is possible to recover the lost node by using two nodes other than the lost node, and any two of the nodes of the storage system If the node is lost, the lost node can be recovered using four nodes other than the lost node. That is, when coding is performed using the crown code technique, the joint localities are ((r ₁ , r ₂ ) = (2, 4)).

The crown code code used for encoding technique is a joint information locality is _{((r 1, r 2)} info = (2, 3)), r 1, the joint locality ((a r ₂₎ = _(2, 4)) encode, the code having the best coding rate is obtained.

9 is a reference diagram for explaining a feature of the crown code technique for optimizing the coding rate. 9 (a), when the parity symbol is encoded according to the edge connecting the information symbols, the joint information locality becomes ((r ₁ , r ₂ ) _info = (2, 3) It is not optimized code. On the other hand, when the parity symbol is encoded according to the edge connecting the information symbols according to the crown code scheme as shown in FIG. 9 (b), the joint information locality is ((r ₁ , r ₂ ) _info = (2, 3) And the code rate is optimized at the same time.

Also, by encoding the parity symbols as described above, even if up to two nodes (symbols) are lost, the lost nodes (symbols) can be recovered by using other nodes (symbols). In this case, the minimum distance d, which is an index indicating the number of recoverable loss nodes (symbols), is 3.

Here, the code according to the crown code technique is [3k - 5, k, 3] ₂ code. That is, when the data to be stored is divided into k data blocks, the encoding step (S200) generates k information symbols and generates 2k-5 parity symbols. And the minimum distance d is 3. And the coding rate is k / (3k - 5).

Next, a description will be made of a method of encoding a parity symbol using the tiara code scheme in the encoding step (S200).

10 is a reference diagram for explaining a coding method according to the tiara code technique. The operation of the coding method using the tiara code technique can be expressed as a graph as shown in FIG. Referring to FIG. 10A, when the number k of divided data blocks is 3, for example, when the graph node represents an information symbol / node and the graph edge represents a parity symbol / node, S200 may be represented as a graph of FIG. 10 (a). 10 (b) when k is 4, and FIG. 10 (c) when k is 7. Similarly, for the other k values, the edges are connected to form a triangle with respect to the three graph nodes as shown in FIG. 10, and then the two graph nodes are selected so that the remaining graph nodes are connected to the selected two graph nodes and the edge You can create a graph node. In the tiara code scheme, an information symbol / node is connected to an edge in the form of a tiaras, and a parity symbol is encoded according to an edge generated due to the above connection.

In the present invention, the coding method expressed by the graph of FIG. 10 is referred to as a coding method using the Tiara Code technique.

In the case of using the tiara code scheme, the encoding step (S200) can code the parity symbol through the following process.

Here, the encoding step S200 may include a step of selecting a first data block and a second data block among the data blocks. Referring to FIG. 10B, information symbols S_1 and S_2 corresponding to each data block can be selected.

Here, the coding step S200 may include coding the parity symbol using the pair of the first data block and the second data block. Referring to FIG. 10 (b), a parity symbol S_ (1, 2) can be generated using a pair of S_1 information symbols and S_2 information symbols.

Herein, the encoding step (S200) may include, for the remaining data blocks except for the data block selected as the gender of the data block, a pair of the remaining data blocks and the first data blocks, And encoding the pattern data using the first and second data block pairs. Referring to FIG. 10B, a parity symbol S_ (1, 3) is generated using an S_3 information symbol and a S_1 information symbol pair for an S_3 information symbol corresponding to one of the remaining data blocks, and S_3 information And the parity symbol S_ (2, 3) can be generated using the pair of S_2 information symbols. Also, a parity symbol S_ (1, 4) is generated using the pair of S_4 information symbol and S_l information symbol for the S_4 information symbol corresponding to the other data block, and a pair of S_4 information symbol and S_2 information symbol is generated To generate parity symbols S_ (2, 4).

The operation of the encoding step S200 using the above-mentioned tiara code technique can be represented by a generation matrix as follows.

To this end, the coding step S200 includes an input matrix in which each element of the input symbol represents the divided data block, and a matrix in which the value of the element is either 1 or 0, and the number of 1s included in each column vector is 2 (Generator Matrix) to generate an output matrix. And an element corresponding to one input symbol in the output matrix is the information symbol, and an element corresponding to a sum of two input symbols in the output matrix is the parity symbol. At this time, the column vector of the generator matrix has a length corresponding to the number of elements of the input matrix.

Here, the generation matrix when the tiara code scheme is used is defined by a unit matrix of the number of elements of the input matrix, a column vector of two elements whose row order is mutually adjacent, and one column vector And a matrix of all combinations of column vectors in which an element and an element not included in the one column vector are 1, respectively.

11 is a reference diagram showing a generation matrix according to the above-described tiara code technique. The generation matrix according to the tiara code scheme can be composed of a k-sized unit matrix equal to the number of divided data blocks and the remaining part. In this case, if k is greater than 3, the remaining part is divided into a first matrix part and a second matrix part, and if k is 3, the remaining part can be represented by only the first matrix part. Here, the first matrix portion may be a column vector with two elements selected from the k rows selected as shown in FIG. 11, and the elements of the selected two rows may be one. Next, the second matrix part is a matrix included in the remaining part when k is greater than 3, and is a matrix including an element having one of the two selected rows as shown in FIG. 11 and a row not included in the selected two rows And contains all column vectors consisting of elements with one being one.

In a case where coding is performed using the tiara code scheme, when one node corresponding to any one of the information symbols of the nodes of the storage system is lost, using the two nodes in addition to the lost node, And if two nodes corresponding to any two of the information symbols of the storage system are lost, then three nodes other than the lost node are used to recover the lost 2 Nodes can be recovered. That is, when coding is performed using the tiara code technique, the joint information locality becomes ((r ₁ , r ₂ ) _info = (2, 3)).

In addition, when any one of the nodes of the storage system is lost, it is possible to recover the lost node by using two nodes other than the lost node, and any two of the nodes of the storage system If the node is lost, the lost node can be recovered using three nodes other than the lost node. That is, when coding is performed using the tiara code technique, the joint locality becomes ((r ₁ , r ₂ ) = (2, 3)).

The Tierra code encoded using the techniques are joint information locality is _{((r 1, r 2)} info = (2, 3)) is a joint locality _{((r 1, r 2)} = (2, 3)) encode, the code having the best coding rate is obtained.

Also, by encoding the parity symbols as described above, even if up to two nodes (symbols) are lost, the lost nodes (symbols) can be recovered by using other nodes (symbols). In this case, the minimum distance d, which is an index indicating the number of recoverable loss nodes (symbols), is 3.

Here, the code according to the tiara code technique is [3k - 3, k, 3] ₂ code. That is, when the data to be stored is divided into k data blocks, the encoding step (S200) generates k information symbols and 2k-3 parity symbols. And the coding rate is k / (3k - 3).

In the case where symbols encoded in the encoding step S200 are stored in each node of the distributed storage system in the storage step S300, when symbols are lost, the symbols recovered using the remaining symbols are recovered in the same manner , And when each node is lost, the remaining node can be used to recover the lost node. That is, as described repeatedly, the above-described parts of the symbols may be similarly applied to nodes of the distributed storage system stored in the storage step S300.

Yet another embodiment of the present invention may be a decoding method for recovering data loss in a storage system including symbols and nodes generated and stored through the above-described encoding process. The decoding method for recovering data loss in the storage system according to the present invention is characterized in that when a node of the distributed storage system is lost, at least one of the nodes other than the lost node is selected And recover the lost node using the selected node. The decoding method according to the present invention can recover the lost node using the encoded nodes in the manner described in detail in the encoding method according to the present invention.

Here, the decoding method according to the present invention can be performed by the decoding device 20 connected to the storage system. Here, the decryption apparatus 20 may be a server apparatus or a computer apparatus connected to the storage system, or an embedded system for performing a data recovery function.

Here, the encoding apparatus 10 and the decrypting apparatus 20 may be included in one server apparatus or a computer apparatus, if necessary. At this time, one server device may be the node management server 1. [

15 is a reference diagram showing a distributed storage system including the coding apparatus 10 and the decoding apparatus 20 as described above. FIG. 16 is a diagram showing a node management for performing both the operations of the coding apparatus 10 and the decoding apparatus 20 1 is a reference view showing a distributed storage system including a server 1. Fig.

Here, the decoding method for data loss recovery according to the present invention may include a node selecting step (S1000) and a recovering step (S2000).

12 is a flowchart of a decoding method for data loss recovery in a distributed storage system according to an embodiment of the present invention.

The decoding method for data loss recovery according to the present invention includes an information symbol generated for a data block obtained by dividing data to be stored into a predetermined number, A parity symbol generated by using a block is a decoding method for recovering a symbol stored in the lost node when a node of a storage system stored in each node is lost.

In this case, in order to maintain the number of nodes required to recover the symbols stored in the lost node and the number of parity symbols generated by the encoder, a node selection step (S1000) And selecting at least one or more nodes capable of obtaining the symbol stored in the lost node by operating the symbol stored in the node among the remaining nodes.

In the recovery step S2000, the symbol stored in the lost node is recovered using the selected node.

A decoding method for data loss recovery in a storage system configured by nodes storing coded symbols using a ring code technique according to the present invention will be described.

Wherein the storage system is configured to use the parity symbol encoded by selecting and using two data blocks so that each data block is used only twice to generate two parity symbols for all the data blocks, .

Wherein, when one node corresponding to any one of the information symbols of the storage system is lost, the node selection step (S1000) includes a step of selecting, among the remaining nodes other than the lost node, And the recovery step S2000 can recover the lost node by using the selected node. The recovery node S2000 can recover the lost node by selecting the two nodes capable of obtaining the symbol stored in the lost node.

In addition, when two nodes corresponding to any two information symbols among the nodes of the storage system are lost, the node selection step (S1000) may include a step of selecting, among the remaining nodes other than the lost node, To select the four nodes capable of obtaining the symbol stored in the lost node, and the recovering step (S2000) can recover the lost node using the selected node.

Here, when any one of the nodes of the storage system is lost, the node selection step (S1000) may calculate the symbol stored in the node among the remaining nodes other than the lost node, The two nodes capable of acquiring the stored symbol are selected, and the recovery step (S2000) can recover the lost node using the selected node.

In addition, when any two nodes among the nodes of the storage system are lost, the node selection step (S1000) calculates the symbol stored in the node among the remaining nodes other than the lost node, Selects four nodes capable of acquiring the stored symbols, and the restoring step (S2000) can recover the lost node by using the selected node.

Next, a decoding method for data loss recovery in a storage system configured by nodes storing coded symbols using the crown code technique according to the present invention will be described.

Wherein the storage system is configured to select a first data block, a second data block, a third data block, and a fourth data block among the data blocks, a pair of the first data block and the second data block, Encoding the parity symbols for each pair using a pair of a block, a third data block, and a third data block and a fourth data block, Wherein each of the remaining data blocks and the first data block and each of the remaining data blocks and the second data block are used for each of the remaining data blocks, And stores the parity symbol encoded by the encoding process in each node.

Wherein, when one node corresponding to any one of the information symbols of the storage system is lost, the node selection step (S1000) includes a step of selecting, among the remaining nodes other than the lost node, And the recovery step S2000 can recover the lost node by using the selected node. The recovery node S2000 can recover the lost node by selecting the two nodes capable of obtaining the symbol stored in the lost node.

In addition, when two nodes corresponding to any two information symbols among the nodes of the storage system are lost, the node selection step (S1000) may include a step of selecting, among the remaining nodes other than the lost node, And selects the three nodes capable of acquiring the symbol stored in the lost node, and the recovering step (S2000) can recover the lost node using the selected node.

Here, when any one of the nodes of the storage system is lost, the node selection step (S1000) may calculate the symbol stored in the node among the remaining nodes other than the lost node, The two nodes capable of acquiring the stored symbol are selected, and the recovery step (S2000) can recover the lost node using the selected node.

In addition, when any two nodes among the nodes of the storage system are lost, the node selection step (S1000) calculates the symbol stored in the node among the remaining nodes other than the lost node, Selects four nodes capable of acquiring the stored symbols, and the restoring step (S2000) can recover the lost node by using the selected node.

Next, a decoding method for data loss recovery in a storage system configured by nodes storing coded symbols using the tiara code technique according to the present invention will be described.

Here, the storage system may include the steps of selecting a first data block and a second data block of the data block, encoding the parity symbol using a pair of the first data block and the second data block, A pair of the remaining data blocks and the first data blocks and a pair of the remaining data blocks and the second data blocks for the remaining data blocks, And stores the parity symbols encoded by coding the parity symbol on a pair basis in each node.

Wherein, when one node corresponding to any one of the information symbols of the storage system is lost, the node selection step (S1000) includes a step of selecting, among the remaining nodes other than the lost node, And the recovery step S2000 can recover the lost node by using the selected node. The recovery node S2000 can recover the lost node by selecting the two nodes capable of obtaining the symbol stored in the lost node.

In addition, when two nodes corresponding to any two information symbols among the nodes of the storage system are lost, the node selection step (S1000) may include a step of selecting, among the remaining nodes other than the lost node, And selects the three nodes capable of acquiring the symbol stored in the lost node, and the recovering step (S2000) can recover the lost node using the selected node.

Here, when any one of the nodes of the storage system is lost, the node selection step (S1000) may calculate the symbol stored in the node among the remaining nodes other than the lost node, The two nodes capable of acquiring the stored symbol are selected, and the recovery step (S2000) can recover the lost node using the selected node.

In addition, when any two nodes among the nodes of the storage system are lost, the node selection step (S1000) calculates the symbol stored in the node among the remaining nodes other than the lost node, The three nodes capable of acquiring the stored symbols are selected, and the recovery step (S2000) can recover the lost node using the selected node.

Here, the decoding method according to the present invention can be expressed by a method using a generation matrix as described in the encoding method. The decoding method according to the present invention multiplies the parity check matrix corresponding to the inverse matrix of the generator matrix described in the coding step S200 for each technique of the ring code, the crown code and the tiar code, The information symbol can be decoded.

Here, a recovery set, which is a set of nodes capable of recovering a specific node, can be preset. Such a recovery set can be determined according to the generation matrix. Therefore, in the decoding method according to the present invention, the symbols necessary for recovery are selected in consideration of the information of the generation matrix according to the symbols stored in the lost node, and the lost nodes can be recovered by accessing the nodes storing the corresponding symbols .

That is, in the encoding step (S200) according to the present invention, a recovery set can be preset and stored for all recoverable node losses. If a node loss occurs, in the decoding method according to the present invention, You can connect to the nodes contained in the recovery set, and recover lost nodes using the connected nodes.

Yet another embodiment of the present invention may be an encoding device 10 for recovering data loss in a storage system. The encoding apparatus 10 may include an encoding unit 100 and a data processing unit 200. Here, the encoding apparatus 10 may operate in the same manner as the encoding method for data loss recovery according to the present invention described above. The overlapping portions will be omitted and briefly described.

13 is a block diagram of an encoding apparatus 10 for recovering data loss in a distributed storage system according to an embodiment of the present invention.

The encoding unit 100 generates an information symbol according to a data block obtained by dividing data to be stored in a storage system into a predetermined number and divides the data blocks into two different data blocks And generates a parity symbol using the selected two data blocks.

The data processing unit 200 stores each symbol generated in each node of the storage system.

Here, when the node of the storage system is lost, the encoding unit 100 maintains the number of nodes required to recover the symbol stored in the lost node and the number of the parity symbols generated by the encoder to a predetermined limit And selects the two different data blocks from among the divided data blocks and codes the parity symbols using the selected two data blocks.

Yet another embodiment of the present invention may be a decryption device 20 for recovering data loss in a storage system. Here, the decoding apparatus 20 may include a node selecting unit 1000 and a restoring unit 2000. Here, the decryption apparatus 20 can operate in the same manner as the decryption method for data loss recovery according to the present invention described above. The overlapping portions will be omitted and briefly described.

14 is a block diagram of a decoding apparatus 20 for recovering data loss in a distributed storage system according to an embodiment of the present invention.

Here, the decoding apparatus 20 generates an information symbol (Information Symbol) for a data block obtained by dividing data to be stored into a predetermined number and a data block generated by using two different data blocks among the divided data blocks The parity symbol is a decoding device 20 for recovering a symbol stored in the lost node when a node of the storage system stored in each node is lost.

In order to maintain the number of nodes required for recovering symbols stored in the lost node and the number of parity symbols generated by the encoder to a predetermined limit, And selecting at least one or more nodes capable of obtaining the symbol stored in the lost node by operating the symbol stored in the node among the remaining nodes.

Here, the restoration unit 2000 restores the symbols stored in the lost node using the selected node.

Yet another embodiment of the present invention can be a computer program stored in a medium for performing a coding method or a decoding method for data loss recovery in the distributed storage system described above.

It is to be understood that the present invention is not limited to these embodiments, and all elements constituting the embodiment of the present invention described above are described as being combined or operated in one operation. That is, within the scope of the present invention, all of the components may be selectively coupled to one or more of them.

In addition, although all of the components may be implemented as one independent hardware, some or all of the components may be selectively combined to perform a part or all of the functions in one or a plurality of hardware. As shown in FIG. In addition, such a computer program may be stored in a computer readable medium such as a USB memory, a CD disk, a flash memory, etc., and read and executed by a computer to implement an embodiment of the present invention. As the recording medium of the computer program, a magnetic recording medium, an optical recording medium, a carrier wave medium, and the like can be included.

Furthermore, all terms including technical or scientific terms have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined in the Detailed Description. Commonly used terms, such as predefined terms, should be interpreted to be consistent with the contextual meanings of the related art, and are not to be construed as ideal or overly formal, unless expressly defined to the contrary.

It will be apparent to those skilled in the art that various modifications, substitutions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. will be. Therefore, the embodiments disclosed in the present invention and the accompanying drawings are intended to illustrate and not to limit the technical spirit of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments and the accompanying drawings . The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

S100: Storage target data segmentation step
S200: encoding step
S300: storage step
S1000: Node selection step
S2000: Recovery Steps
100:
200:
1000: Node selection unit
2000:
10: Encoding device
20: Decryption device
1: Node management server

Claims (21)

Dividing data to be stored into a predetermined number of data blocks;
Generating an information symbol according to the divided data block, selecting two different data blocks from the divided data blocks, and using the selected two data blocks to generate a parity symbol (Parity Symbol) Encoding step; And
And storing the generated symbols in respective nodes of the storage system,
Wherein the encoding step comprises the steps of: for the node stored in the storage system, to connect to up to four or less of the nodes in all cases of loss of two or less of the nodes in all cases, Generates a parity symbol,
Wherein the encoding step comprises the steps of: inputting an input matrix having input symbols representing the divided data blocks as elements, generating matrixes having element values of 1 or 0 and the number of 1s included in each column vector is 2 or less Generator Matrix) to generate an output matrix,
Wherein an element corresponding to one input symbol in the output matrix is the information symbol and an element corresponding to a sum of two input symbols in the output matrix is the parity symbol. A method for restoring data loss in a mobile station. delete The method according to claim 1,
Wherein the column vector of the generator matrix has a length according to the number of elements of the input matrix,
Wherein the generation matrix is a matrix formed by concatenating a unit matrix of the number of elements of the input matrix and all column vectors of two elements adjacent to each other in the row direction. . The method according to claim 1,
Wherein the column vector of the generator matrix has a length according to the number of elements of the input matrix,
Wherein the generation matrix includes a unit matrix of the number of elements of the input matrix, five column vectors of two elements adjacent to each other in the row direction, and one element of one of the five column vectors And a matrix of all combinations of column vectors of which one element not included in the five column vectors is 1 is connected. The method according to claim 1,
Wherein the column vector of the generator matrix has a length according to the number of elements of the input matrix,
Wherein the generation matrix includes a unit matrix of the number of elements of the input matrix, one column vector of two elements adjacent to each other in the row direction, one column vector of one column vector, Is a matrix obtained by concatenating all combinations of column vectors, each of which is an element not included in the first column. The method according to claim 1,
If a node of the storage system is lost, it can be recovered using the remaining nodes other than the lost node,
Wherein the coding step comprises the steps of: storing the number of nodes required to recover a symbol stored in the lost node and the number of parity symbols generated in the encoding step to a predetermined limit; Wherein the parity symbol generator generates the parity symbols using the selected two data blocks and selects the different data blocks. The method according to claim 1,
Wherein the encoding step generates each parity symbol by selecting and using two data blocks so that each data block is used only twice to generate two parity symbols for all the data blocks. The method comprising the steps < RTI ID = 0.0 > of: < / RTI > 8. The method of claim 7,
When one node corresponding to any one of the information symbols of the storage system is lost, the lost node can be recovered using two nodes other than the lost node,
When two nodes corresponding to any two information symbols of the nodes of the storage system are lost, the lost two nodes can be recovered using four nodes other than the lost node , A coding method for recovering data loss in a storage system. 8. The method of claim 7,
When the storage target data is divided into k data blocks,
Wherein the coding step generates k pieces of the information symbols and generates k pieces of the parity symbols using all k pieces of the data blocks twice. 2. The method according to claim 1,
Selecting a first data block, a second data block, a third data block, and a fourth data block among the data blocks;
A pair of the first data block and the second data block, a pair of the second data block and the third data block, and a pair of the third data block and the fourth data block, The process of generating a symbol,
A pair of each of the remaining data blocks and the first data block, a pair of each of the remaining data blocks and the second data block for each of the remaining data blocks excluding the selected data block among the data blocks, And generating the parity symbol for each pair by using the parity symbols, respectively. 11. The method of claim 10,
When one node corresponding to any one of the information symbols of the storage system is lost, the lost node can be recovered using two nodes other than the lost node,
If two nodes corresponding to any two information symbols of the nodes of the storage system are lost, the lost two nodes can be recovered using three nodes other than the lost node,
If one of the nodes of the storage system is lost, the lost node can be recovered using two nodes other than the lost node,
Characterized in that if any two nodes of the storage system are lost then it is possible to recover the two lost nodes using four nodes in addition to the lost node, / RTI > 11. The method of claim 10,
When the storage target data is divided into k data blocks,
Wherein the coding step generates k pieces of the information symbols and generates 2k to 5 pieces of the parity symbols. 2. The method according to claim 1,
Selecting a first data block and a second data block among the data blocks,
Generating the parity symbol using a pair of the first data block and the second data block,
A pair of each of the remaining data blocks and the first data block, a pair of each of the remaining data blocks and the second data block for each of the remaining data blocks excluding the selected data block among the data blocks, And generating the parity symbol for each pair by using the parity symbols, respectively. 14. The method of claim 13,
When one node corresponding to any one of the information symbols of the storage system is lost, the lost node can be recovered using two nodes other than the lost node,
If two nodes corresponding to any two information symbols of the nodes of the storage system are lost, the lost two nodes can be recovered using three nodes other than the lost node,
If one of the nodes of the storage system is lost, the lost node can be recovered using two nodes other than the lost node,
Characterized in that if any two of the nodes of the storage system are lost then the lost two nodes can be recovered using three nodes in addition to the lost node, / RTI > 14. The method of claim 13,
When the storage target data is divided into k data blocks,
Wherein the encoding step generates k pieces of the information symbols and generates 2k to 3 pieces of the parity symbols. An information symbol (Information Symbol) generated for a data block obtained by dividing data to be stored into a predetermined number and a parity symbol generated using two different data blocks among the divided data blocks A decoding method for recovering a symbol stored in a lost node when a node of a storage system stored in each node is lost,
And for storing the number of nodes and the number of parity symbols required for restoring symbols stored in the lost node to a predetermined limit, calculating the symbol stored in the node among remaining nodes other than the lost node A node selecting step of selecting at least one or more nodes capable of obtaining the symbol stored in the lost node; And
And recovering symbols stored in the lost node using the selected node,
The generated information symbol and the generated parity symbol are each composed of an input matrix in which each element is an input symbol representing each divided data block, and an input matrix in which the value of the element is either 1 or 0, Wherein an element corresponding to one of the input symbols in the output matrix is multiplied by the information symbol and a sum of two input symbols in the output matrix is multiplied by a generator matrix having a number of 2 or less, Are generated by the parity symbol, respectively, in the storage system. 17. The method of claim 16,
The storage system stores the parity symbols encoded by selecting and using two data blocks so that each data block is used only twice to generate two parity symbols for all the data blocks, and,
When one node corresponding to any one of the information symbols among the nodes of the storage system is lost, the node selecting step calculates the symbol stored in the node among the remaining nodes other than the lost node, Selecting the two nodes capable of acquiring the symbol stored in the lost node, the recovering step restoring the lost node using the selected node,
When two nodes corresponding to any two information symbols among the nodes of the storage system are lost, the node selecting step calculates the symbol stored in the node among the remaining nodes other than the lost node, Selecting the four nodes capable of obtaining the symbol stored in the lost node and recovering the lost two nodes using the selected node. Decryption method for recovery. 17. The method of claim 16,
Wherein the storage system is configured to select a first data block, a second data block, a third data block, and a fourth data block among the data blocks, a pair of the first data block and the second data block, Generating the parity symbol for each pair using a pair of the block, the third data block, and the third data block and the fourth data block, The parity symbol is generated for each pair by using the pair of the remaining data block and the first data block and the pair of the remaining data block and the second data block for each of the remaining data blocks, And storing the parity symbols encoded by the parity symbol encoding unit in each node,
When one node corresponding to any one of the information symbols among the nodes of the storage system is lost, the node selecting step calculates the symbol stored in the node among the remaining nodes other than the lost node, Selecting the two nodes capable of acquiring the symbol stored in the lost node, the recovering step restoring the lost node using the selected node,
When two nodes corresponding to any two information symbols among the nodes of the storage system are lost, the node selecting step calculates the symbol stored in the node among the remaining nodes other than the lost node, Selecting the three nodes capable of obtaining the symbol stored in the lost node and recovering the lost node using the selected node. Decryption method for recovery. 17. The method of claim 16,
Wherein the storage system selects a first data block and a second data block among the data blocks,
Generating the parity symbol by using the pair of the first data block and the second data block; calculating, for each of the remaining data blocks excluding the selected data block, The parity symbol encoded by generating the parity symbol for each pair using the pair of the block and the first data block and the pair of the remaining data block and the second data block, Store,
When one node corresponding to any one of the information symbols among the nodes of the storage system is lost, the node selecting step calculates the symbol stored in the node among the remaining nodes other than the lost node, Selecting the two nodes capable of acquiring the symbol stored in the lost node, the recovering step restoring the lost node using the selected node,
When two nodes corresponding to any two information symbols among the nodes of the storage system are lost, the node selecting step calculates the symbol stored in the node among the remaining nodes other than the lost node, Selecting the three nodes capable of obtaining the symbol stored in the lost node and recovering the lost node using the selected node. Decryption method for recovery. A storage system comprising: an information symbol generating unit configured to generate an information symbol according to a predetermined number of data blocks to be stored in a storage system, select two different data blocks from the divided data blocks, A coding unit for generating a parity symbol using a data block; And
And a data processor for storing the generated symbols in each node of the storage system,
Wherein the encoding unit is configured to divide the number of nodes required to recover a symbol stored in the lost node and the number of parity symbols generated by the encoding unit into a predetermined limit when the node of the storage system is lost, Selects two different data blocks from among one data block, generates the parity symbols using the selected two data blocks,
Wherein the encoding unit comprises: an input matrix having input symbols each representing an input symbol representing the divided data blocks; and a generator matrix having a value of an element of 1 or 0 and the number of 1's included in each column vector is 2 or less Matrix) to generate an output matrix,
And an element corresponding to one input symbol in the output matrix is the information symbol and an element corresponding to a sum of two input symbols in the output matrix is the parity symbol. Wherein the storage device is a storage device. An information symbol (Information Symbol) generated for a data block obtained by dividing data to be stored into a predetermined number and a parity symbol generated using two different data blocks among the divided data blocks A decoding apparatus for recovering a symbol stored in a lost node when a node of a storage system stored in each node is lost,
And for storing the number of nodes and the number of parity symbols required for restoring symbols stored in the lost node to a predetermined limit, calculating the symbol stored in the node among remaining nodes other than the lost node A node selector for selecting at least one or more nodes capable of obtaining the symbol stored in the lost node; And
And a recovering unit for recovering symbols stored in the lost node using the selected node,
The generated information symbol and the generated parity symbol are each composed of an input matrix in which each element is an input symbol representing each divided data block, and an input matrix in which the value of the element is either 1 or 0, Wherein an element corresponding to one of the input symbols in the output matrix is multiplied by the information symbol and a sum of two input symbols in the output matrix is multiplied by a generator matrix having a number of 2 or less, Are generated by the parity symbol, respectively, in the storage system.

Images