WO2017186871A1 - Data protection coding technique - Google Patents

Data protection coding technique Download PDF

Info

Publication number
WO2017186871A1
WO2017186871A1 PCT/EP2017/060114 EP2017060114W WO2017186871A1 WO 2017186871 A1 WO2017186871 A1 WO 2017186871A1 EP 2017060114 W EP2017060114 W EP 2017060114W WO 2017186871 A1 WO2017186871 A1 WO 2017186871A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
sub
blocks
redundant
block
Prior art date
Application number
PCT/EP2017/060114
Other languages
French (fr)
Inventor
Rune Erlend JENSEN
Original Assignee
Memoscale As
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Memoscale As filed Critical Memoscale As
Publication of WO2017186871A1 publication Critical patent/WO2017186871A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1076Parity data used in redundant arrays of independent storages, e.g. in RAID systems

Definitions

  • Embodiments structure encoding of source data for encoding within redundant data. Advantages include one or more of improved verification of data and improved recovery from data errors. Background
  • RAID Redundant Array of Independent Disks
  • SSDs solid-state drives
  • mechanical storage media because they have faster access times and are not as vulnerable to mechanical problems.
  • SSDs are more vulnerable to data corruption than mechanical drives as their medium is more volatile.
  • data loss In a data storage system there are many possible causes of data loss. For example, failures of data drives may occur, cabling may be faulty, controller mistakes may cause data to be written in the wrong place and stored data may be corrupted.
  • Data erasures and data errors are different types of data loss in a data storage system. For data erasures, it is known with full certainty where the data loss has occurred. For example, if a data drive fails, the data within that data drive has been lost. A controller of the data storage system will determine that the data drive has failed, due to the lack of response of the failed data drive, and so it is known where the data loss has occurred.
  • Data errors are sometimes referred to as silent data corruption and it is more difficult to recover data loss from errors than from erasures.
  • redundant data such as parity data.
  • properties of the stored data such as checksums, and to store the properties in metadata.
  • the number of errors that can be detected and/or recovered from depends on what redundant data and metadata is generated.
  • the additional storage requirements due to the redundant data is often significant.
  • the metadata of a data storage system is also a significant overhead and may increase the data storage requirements by up to 4%.
  • FIGURE 1 Shows a configuration with 3 data and 1 redundancy nodes, using pairs of 2 modified redundancy codewords according to an embodiment
  • FIGURE 2 Shows a configuration with 3 data and 2 redundancy nodes, using pairs of 2 modified redundancy codewords according to an embodiment
  • FIGURE 3 Shows a configuration with 3 data and 1 redundancy nodes, using groups of 3 modified redundancy codewords according to an embodiment
  • FIGURE 4 Shows a configuration with 3 data and 1 redundancy nodes, using 2 modified redundancy codewords according to an embodiment
  • FIGURE 5 Shows a configuration with 3 data and 1 redundancy nodes, using 3 modified redundancy codewords according to an embodiment
  • FIGURE 6 Shows a configuration with 3 data and 2 redundancy nodes, using 6 modified redundancy codewords according to an embodiment
  • Embodiments provide a data coding technique that improves the redundant data determinable from source data. Advantages of encoding the data include one or more of improved determination of whether or not the stored data comprises errors, an increase in the number of errors that can be detected and the number of errors that can be corrected increased.
  • a large capacity storage system is typically divided into a plurality of nodes.
  • Each node may represent any of different data drives, stripes, sectors or parts of the data storage system.
  • Figure 1 shows systematically coded data in the exemplary application of a data storage system.
  • source data is stored in source nodes without further coding applied. Redundant data is generated in dependence on the source data and stored in redundant nodes.
  • Each node may be a data drive or block of a data storage system.
  • Each node comprises a plurality of sub-blocks of data.
  • D1 i is individual sub-block of node D1.
  • the total number of sub-blocks comprised by each node is 'x' and all of the nodes comprise the same number of sub-blocks.
  • the entire data stored in node D1 is stored in the sub-blocks D1 i to D1x.
  • the entire data stored in node D2 is stored in the sub-blocks D2i to D2x.
  • the entire data stored in node D3 is stored in the sub-blocks D3i to D3x.
  • the entire data stored in node Dr1 is stored in the sub-blocks Dr1 i to Dr1x.
  • the size of a sub-block may be any size, such as 1 byte or a word, but preferably is determined in dependence on the finite field size of the underlying code.
  • the term sub-block is used interchangeably herein with the terms word, dataword and codeword, and the term sub-block includes all of a word, dataword and codeword.
  • the number of datawords included in the computation of redundant codewords is increased by a factor E.
  • E For a storage system with N data nodes, with X datawords comprised by each node, we a total of (N * X * E) datawords are encoded into an redundant node with X codewords. This is E times more than required by maximum distance separable (MDS) codes, such as Reed-Solomon (RS).
  • MDS maximum distance separable
  • FIG. 1 A first implementation of the present embodiment is shown in Figure 1.
  • each of the columns, labelled D1 , D2, D3 and Dr1 represents a different node.
  • D1 , D2 and D3 are source nodes that store source data.
  • Dr1 is a redundant node that stores redundant data.
  • Such operations are known, for example, from the RS coding of a plurality of redundant nodes.
  • the '+' operator represents an XOR addition of data units. Two datawords from each data node are encoded to generate each pair of redundancy codewords.
  • Dr1 i D1 i + D2 + D3i
  • Dr1 2 D1 2 + D2 2 + D3 2
  • the present embodiment allows such an error to be both detected and corrected as more than one equation is provided. It is possible to both detect and correct a single error in any dataword within each group.
  • Dr1 i D1 i + D2 + D3i + D1 2 + D2 2 + D3 2
  • Dr1 2 D1 -, + D2-, 2 + D3 ⁇ 3 + D1 2 4 + D2 2 5 + D3 2 6
  • each group has properties equivalent to a (8, 6) RS code. More generally, each group according to the present embodiment has properties equivalent to a ((N+R)*E, N*E) code.
  • FIG. 2 shows another implementation according to the present embodiment.
  • embodiments include any number of redundant nodes being used, i.e. R redundant nodes, with the coding technique for generating each redundant node according to the present embodiment being extended so that twice as many datawords contribute to each redundant codeword than in standard RS coding. This enables error detection and correction for RAID6 even when one node is lost. In addition the number of correctable errors is improved over RAID6 when no nodes are lost.
  • FIG 2 the shown finite field coefficients are purely exemplary.
  • a sufficiently large finite field for instance GF(256)
  • error(s) can be detected and recovered using known error correction techniques, such as a Berlekamp- Massey decoder or with Soft-decoding, as described in, for example, Koetter, Ralf; Vardy, Alexander (2003). "Algebraic soft-decision decoding of Reed- Solomon codes”. IEEE Transactions on Information Theory 49 (11): 2809-2825. doi:10.1109/TIT.2003.819332.
  • each node set contained X/E separate groups, with each group comprising an equation system that is dependent on more than one row of source data.
  • the number of separate groups, i.e. equation systems is E for the entire set of data nodes.
  • Dr1x_i is generated by combining, using finite field encoding operations that are preferably XOR additions, the datawords from the data nodes with indexes X-1 and X.
  • Dr1x is generated by combining the datawords from all of the data nodes, using finite field encoding operations that are different from those used to generate Dr1x-i. According to the present embodiment, it is possible to combine, preferably XOR, all redundant codewords Dr1 i to Dr1x_i to form an equation for a codeword Dr1 * :
  • Dr1 * contains all datawords from all nodes XOR'ed together.
  • Dr1x also contains all datawords from each data node, encoded with finite field multiplications other than XOR.
  • the Dr1 * and Dr1x codewords can therefore be used as a standard known Reed- Solomon code with block length (3*X)+2. This can also be referred to as a (((3 * X)+2), (3 * X)) code.
  • embodiments provide a (((N * X)+2), N * X) code, i.e. a Reed-Solomon code with 2 redundancy codewords. It is therefore possible to detect 2 errors and correct 1 error for each node set when no nodes are lost.
  • the present embodiment also includes any of the redundant codewords being extended as described for Dr1 x- and Dr1 x above and are not restricted to the extended codewords being the last two codewords in the redundant node.
  • codewords Dr1x_i and Dr1x are fully extended while the codeword Dr1x -2 is partially extended.
  • this design it is possible to construct a standard Reed- Solomon code with block length ((3 * X)+3), or a (((N * X)+3), N * X) code.
  • Embodiments also include increasing the number of fully extended codewords until X-1 codewords in the redundancy node are fully extended.
  • Embodiments provide a (((N * X)+E), N * X) code, with E from 2 to X inclusive.
  • Embodiments include choosing finite field coefficients that are linearly independent when constructing Dr1x_i and Dr1x in order to improve the recovery of a lost data node.
  • Embodiments also include first constructing a Reed- Solomon code of size (((N*X)+E), N*X), and then reordering the dataword layout in order to obtain suitable finite field coefficients for the system.
  • Embodiments include choosing finite field coefficients that are linearly independent when constructing DMX-1 and Dr1X in order to improve the recovery of a lost data node.
  • Embodiments also include first constructing a Reed-Solomon code of size (((N*X)+E), N*X), and then reordering the dataword layout in order to obtain suitable finite field coefficients for the system.
  • the number of redundant nodes is not limited to 1 , and the number of redundant nodes can be any number.
  • An example of how to another redundant node can be provided is shown in Figure 6. This is similar to a RAID6 configuration.
  • E 3 codewords have been extended on each redundancy node, namely Dr1 x-2 , Dr1 x- , Dr1 x , Dr2 x-2 , Dr2 x- i and Dr2 x .
  • Codewords Dr1 x-2 and Dr2 x-2 are partially extended. It is possible to design a Dr2 * codeword in a corresponding way to the Dr1 * codeword in Figure 5.
  • codewords Dr2-i to Dr2 x-3 should be encoded with different finite field coefficients/operations in order for the constructed Dr2 * codeword to have the property of being MDS.
  • the code according to the present embodiment corresponds to a (((N*X)+(3*2)), N*X) code.
  • the number of extended codewords are not limited to 3, and can be from 1 to X-1. More generally, embodiments can provide a (((N * X)+(E * R)), N * X) code, with X number of codewords for each data block, N data blocks, R redundancy blocks and E extended codewords.
  • the number of data nodes may be 1 to N.
  • all of the embodiments presented herein may be combined/rearranged with known interleaved codes. This can be done so that datawords with the same finite field coefficients become spatially sequential, similar to existing known methods. This enables the more efficient usage of usage of SIMD vector instructions.
  • the RAID5 and RAID6 systems have been used to present the techniques of embodiments.
  • embodiments are also applicable to other types of storage system.
  • the embodiment is also applicable in software defined storage systems, determining how to encode data packets in packet based communication (like UDP/IP), solid state disks (SSDs), random access memory and RAIDz with standard erasure coding as well as other use cases.
  • the choice of using the last indexes for storing modified codewords is used as an example, and is not required.
  • the modified codewords may be stored at any index by rearranging the data layout.
  • CEPH storage system products An application in which the coding of data according to embodiments is particularly preferable is as a Plugln for CEPH storage system products. Details of CEPH storage system products can be found at: http://ceph.com/; as viewed on 25th April 2017.
  • CEPH there are some versions of CEPH that do not support, or have limited support of, the use of metadata to aid the detection and recovery from errors in stored data. More specifically, there is no support for the use checksums generated from data stored using erasure coding. Moreover, due to overhead of storing checksums/hashes, they are not used on small blocks of data and only used on large amounts of data. Embodiments allow a CEPH system to be modified so that additional parity/redundancy is introduced without there being a substantial increase, or any increase, in metadata.
  • CEPH allows known reading/data processing operations to be modified and it is possible to use CEPH to implement the embodiments described herein.
  • embodiments improve both the detection and correction of errors in read operations by CEPH systems. No external error handling support is required by the modified storage system. Accordingly, it is fast and efficient to implement.
  • Embodiments can also be implemented in other types of system than CEPH.
  • the embodiments described throughout the present document can also be used in combination with other coding techniques.
  • embodiments can also be used in combination with techniques that calculate data characterising values, such as checksums and hashes, in order to aid error detection and recovery.
  • embodiments can be used in combination with the techniques as disclosed in either of the patent applications GB1604648.4 and PCT/IB2017/000354, the entire contents of which are incorporated herein by reference.
  • the combination of techniques allows data characterising values to be used quickly determine if errors have occurred and then error correcting operation to only be applied when errors are detected.
  • Figure 7 is a flowchart of a process according to an embodiment.
  • step 701 the process begins.
  • step 703 the process generates redundant data of source data, wherein there are N source nodes of source data and R redundant nodes of redundant data such that there are a plurality of (N+R) nodes, wherein each of the (N+R) nodes comprises a plurality of sub-blocks of data, wherein a block of data comprises E rows, with each of the E rows comprising (N+R) sub-blocks and each of the (N+R) sub-blocks of a row of a block are comprised by a different one of the (N+R) nodes, such that each block comprises (ExN) sub-blocks of source data and (ExR) sub-blocks of redundant data, and the number of blocks of data is B, the method comprising, for each of the B blocks: generating each of the sub- blocks of redundant data of the block in dependence on all of the sub-blocks of source data of the block in accordance with an MDS coding technique such that the sub-blocks of the block provide an
  • Figure 8 is a flowchart of a process according to an embodiment.
  • step 801 the process begins.
  • step 803 the process generates redundant data of source data, wherein there are N source nodes of source data and R redundant nodes of redundant data such that there are a plurality of (N+R) nodes, wherein each of the (N+R) nodes comprises X sub-blocks of data, wherein a block of data comprises E rows, with each of the E rows comprising (N+R) sub-blocks and each of the (N+R) sub- blocks of a row of a block are comprised by a different one of the (N+R) nodes, such that the block comprises (ExN) sub-blocks of source data and (ExR) sub- blocks of redundant data, the method comprising: generating, for each of the (X- E) rows that are not comprised by the block, each redundant sub-block of the row in dependence on all of the sub-blocks of source data of the row in accordance with an MDS coding technique; generating, for one of the rows comprised by the block, each redundant sub-block of the row in dependence on all of
  • step 805 the process ends.
  • Embodiments include a number of modifications and variations to the techniques described above.
  • one or more, but not all of the redundant nodes may be constructed as described.
  • some of the nodes can be constructed as known RS codes.
  • Such redundant nodes do not have extra repair capabilities, but do not require additional computing resources to construct.
  • Embodiments also include all combinations of the techniques presented for each of the different implementations of embodiments herein.
  • embodiments include the techniques as shown in Figure 2 being used in combination with those shown in Figure 3.
  • each coding of source and redundant nodes that are generated according to embodiments is tested to determine that it has the property of being MDS.
  • This preferably is part of an iterative process with each iteration changing one or more of the coefficients or operations when generating the redundant node(s). The iterative process would be stopped as soon as the overall coding was determined to have the property of being MDS.
  • N and R can be flexibly chosen and can cover all disk array sizes.
  • the underlying finite field may be GF(256).
  • the codes according to embodiments can be constructed over any GF field of size GF(2n) so long as n is large enough for a MDS code to be generated.
  • a typical implementation would have a GF field size of GF(16) or GF(256).
  • the actual generation of coded data in dependence on source data according to embodiments can be performed with known techniques and using known hardware. The processes required to use the techniques according to embodiments to generate a plurality of source nodes and redundant nodes in a data storage system/network of a data centre storing the coded data would be a straightforward task for the skilled person.
  • the skilled person would also be able to use known hardware to reconstruct one or more source nodes in order to implement embodiments.
  • the nodes according to embodiments include single data disks, or drives, or groups or data disks, or drives.
  • a node includes any form of data storage element, a part of such an element or multiple such elements.
  • a node can be any logical entity where data can be stored, and can be anything from a whole, a group of or parts of physical storage devices or locations including but not limited to memory based storage devices such as RAM and SSDs, hard drives, tape storage, optical storage devices, servers and data centers.
  • the method according to embodiments may be performed within a single SSD disk.
  • the method according to embodiments may be performed between chips inside a SSD, or between banks inside (flash) chips.
  • the storage of the data in a data storage system is not limited to the data storage system having nodes, i.e. data drives or sections of a data drive, that are only for use as a store of source data node or redundant data.
  • a mapping may be introduced so that a data drive may store redundant data within a source data node and vice-versa. This interleaving of data changes the mapping of coded data to stored data and can be used to control the read operations from a data storage system, for example to ensure that the network traffic is balanced across the data storage system.
  • data storage is a particularly preferable application for the coding techniques disclosed herein embodiments include the generation of codes for any application, such as data transmission.
  • the nodes and/or blocks may correspond to data packets for transmission over a network.
  • the embodiments can be used to determine data packets for transmission as TCP/IP or UDP packets, as well as other form of data packets used for data transmission over a network.
  • Methods and processes described herein can be embodied as code (e.g., software code) and/or data. Such code and data can be stored on one or more computer-readable media, which may include any device or medium that can store code and/or data for use by a computer system.
  • code and data can be stored on one or more computer-readable media, which may include any device or medium that can store code and/or data for use by a computer system.
  • the computer system When a computer system reads and executes the code and/or data stored on a computer-readable medium, the computer system performs the methods and processes embodied as data structures and code stored within the computer-readable storage medium.
  • a processor e.g., a processor of a computer system or data storage system.
  • Computer-readable media include removable and non-removable structures/devices that can be used for storage of information, such as computer-readable instructions, data structures, program modules, and other data used by a computing system/environment.
  • a computer-readable medium includes, but is not limited to, volatile memory such as random access memories (RAM, DRAM, SRAM); and non-volatile memory such as flash memory, various read-only-memories (ROM, PROM, EPROM, EEPROM), magnetic and ferromagnetic/ferroelectric memories (MRAM, FeRAM), phase- change memory and magnetic and optical storage devices (hard drives, magnetic tape, CDs, DVDs); network devices; or other media now known or later developed that is capable of storing computer-readable information/data.
  • Computer-readable media should not be construed or interpreted to include any propagating signals.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Techniques For Improving Reliability Of Storages (AREA)

Abstract

Disclosed herein is a computer-implemented method of generating redundant data of source data, wherein there are N source nodes of source data and R redundant nodes of redundant data such that there are a plurality of (N+R) nodes, wherein each of the (N+R) nodes comprises a plurality of sub-blocks of data, wherein a block of data comprises E rows, with each of the E rows comprising (N+R) sub-blocks and each of the (N+R) sub-blocks of a row of a block are comprised by a different one of the (N+R) nodes, such that each block comprises (ExN) sub-blocks of source data and (ExR) sub-blocks of redundant data, and the number of blocks of data is B, the method comprising, for each of the B blocks: generating each of the sub-blocks of redundant data of the block in dependence on all of the sub-blocks of source data of the block in accordance with an MDS coding technique such that the sub-blocks of the block provide an MDS code with (ExN) sub-blocks of source data and (ExR) sub-blocks of redundant data.

Description

Data Protection Coding Technique
Field
The field of the invention is the coding of data. Embodiments structure encoding of source data for encoding within redundant data. Advantages include one or more of improved verification of data and improved recovery from data errors. Background
An ever increasing amount of data is being stored in large capacity distributed data storage systems. RAID (Redundant Array of Independent Disks) are well known configurations of distributed data storage systems. The different configurations are described by RAID levels. RAID levels of 1 and above introduce redundant data into stored data. Redundancy allows a loss of data to be detected and, in some circumstances, the lost data can be recovered.
Data drives have traditionally been provided by mechanical storage media. These can be unreliable because the stored data within mechanical storage media can be corrupted due to, for example, data corruption caused by radiation, physical defects or transition error. SSDs (solid-state drives) are preferable over mechanical storage media because they have faster access times and are not as vulnerable to mechanical problems. However, SSDs are more vulnerable to data corruption than mechanical drives as their medium is more volatile.
In a data storage system there are many possible causes of data loss. For example, failures of data drives may occur, cabling may be faulty, controller mistakes may cause data to be written in the wrong place and stored data may be corrupted. Data erasures and data errors are different types of data loss in a data storage system. For data erasures, it is known with full certainty where the data loss has occurred. For example, if a data drive fails, the data within that data drive has been lost. A controller of the data storage system will determine that the data drive has failed, due to the lack of response of the failed data drive, and so it is known where the data loss has occurred. For data errors, it can be determined that there is an error in the data but the location of the error is harder to determine with full certainty, and it may not be possible to determine the location of the error. For example, if data within an SSD is corrupted due to electro-magnetic radiation, a parity check of the stored data in an array of SSDs including the SSD will determine that an error has occurred in one of the SSDs. However, there has been no failure of any part of the SSD and so it is not possible to determine in which SSD of the array of SSDs the error has occurred. The parity check alone therefore only informs a system operator that there is an error in the data and does not provide sufficient information for the error to be corrected. Data errors are sometimes referred to as silent data corruption and it is more difficult to recover data loss from errors than from erasures. In order to improve the handling of errors in stored data, it is known to generate and store redundant data, such as parity data. It is also known to calculate properties of the stored data, such as checksums, and to store the properties in metadata. The number of errors that can be detected and/or recovered from depends on what redundant data and metadata is generated. The additional storage requirements due to the redundant data is often significant. However, in some implementations, the metadata of a data storage system is also a significant overhead and may increase the data storage requirements by up to 4%. Known implementations of data storage systems differ in the amount of redundant data storage, how the redundant data is calculated, what properties of the metadata are calculated and how the properties of the metadata are calculated. Without using any metadata it is known that an error correcting code can correct half as many errors as there are redundant symbols. It is also known that soft- decoding can extend this limit to some extent. There is a need to improve known coding techniques so as to improve error data recovery in storage systems. For RAID5 systems silent data errors can only be detected, leading to loss of data or incorrect data. A RAID6 system that has lost a disk degrades to have the same properties as a RAID5. A method enabling improved error detection and correction is needed for RAID systems.
More generally, there is a need to improve coding techniques so as to improve error handling in all data applications, in particular the coding of data for data storage and data communication applications.
Summary
Aspects of the invention are as set out in the appended independent claims. List of Figures
FIGURE 1 Shows a configuration with 3 data and 1 redundancy nodes, using pairs of 2 modified redundancy codewords according to an embodiment; FIGURE 2 Shows a configuration with 3 data and 2 redundancy nodes, using pairs of 2 modified redundancy codewords according to an embodiment;
FIGURE 3 Shows a configuration with 3 data and 1 redundancy nodes, using groups of 3 modified redundancy codewords according to an embodiment;
FIGURE 4 Shows a configuration with 3 data and 1 redundancy nodes, using 2 modified redundancy codewords according to an embodiment;
FIGURE 5 Shows a configuration with 3 data and 1 redundancy nodes, using 3 modified redundancy codewords according to an embodiment;
FIGURE 6 Shows a configuration with 3 data and 2 redundancy nodes, using 6 modified redundancy codewords according to an embodiment; Description
Embodiments provide a data coding technique that improves the redundant data determinable from source data. Advantages of encoding the data include one or more of improved determination of whether or not the stored data comprises errors, an increase in the number of errors that can be detected and the number of errors that can be corrected increased.
Embodiments are described in more detail below.
A large capacity storage system is typically divided into a plurality of nodes. Each node may represent any of different data drives, stripes, sectors or parts of the data storage system. Figure 1 shows systematically coded data in the exemplary application of a data storage system. For systematically coded data, source data is stored in source nodes without further coding applied. Redundant data is generated in dependence on the source data and stored in redundant nodes. Each node may be a data drive or block of a data storage system.
Each node comprises a plurality of sub-blocks of data. For example, D1 i is individual sub-block of node D1. The total number of sub-blocks comprised by each node is 'x' and all of the nodes comprise the same number of sub-blocks. The entire data stored in node D1 is stored in the sub-blocks D1 i to D1x.
The entire data stored in node D2 is stored in the sub-blocks D2i to D2x.
The entire data stored in node D3 is stored in the sub-blocks D3i to D3x.
The entire data stored in node Dr1 is stored in the sub-blocks Dr1 i to Dr1x.
The size of a sub-block may be any size, such as 1 byte or a word, but preferably is determined in dependence on the finite field size of the underlying code. The term sub-block is used interchangeably herein with the terms word, dataword and codeword, and the term sub-block includes all of a word, dataword and codeword. In the present embodiment, the number of datawords included in the computation of redundant codewords is increased by a factor E. For a storage system with N data nodes, with X datawords comprised by each node, we a total of (N*X*E) datawords are encoded into an redundant node with X codewords. This is E times more than required by maximum distance separable (MDS) codes, such as Reed-Solomon (RS).
A first implementation of the present embodiment is presented below.
A first implementation of the present embodiment is shown in Figure 1. This shows an example layout according to an embodiment with N=3 data nodes and R=1 redundancy node with E=2. Since E = 2, there are twice as many datawords contributing to each redundant codeword than if RS coding had been used. In Figure 1 , each of the columns, labelled D1 , D2, D3 and Dr1 , represents a different node. D1 , D2 and D3 are source nodes that store source data. Dr1 is a redundant node that stores redundant data. There are X codewords, as shown by the subscript index. Coefficients/operations in finite fields are indicated by the superscript notation. The superscript notation indicates any mathematical operation that may be performed on a dataword for ensuring linear independence. Such operations are known, for example, from the RS coding of a plurality of redundant nodes. The '+' operator represents an XOR addition of data units. Two datawords from each data node are encoded to generate each pair of redundancy codewords.
In a known RAID5 system the redundancy is encoded according to RS coding and is as follows:
Dr1 i = D1 i + D2 + D3i Dr12 = D12 + D22 + D32
With the above known RAID5 coding it is not possible to correct a single silent data error, only detect that it exists if:
DM -,≠ D1 -I + D2-, + D3-,
Advantageously, the present embodiment allows such an error to be both detected and corrected as more than one equation is provided. It is possible to both detect and correct a single error in any dataword within each group.
In case of an error we evaluate each group of ((N+R)*E) codewords separately. From Figure 1 the redundancy codewords in the first group are:
Dr1 i = D1 i + D2 + D3i + D12 + D22 + D32
Dr12 = D1 -, + D2-,2 + D3^3 + D12 4 + D22 5 + D32 6
The above equation system according to the present embodiment allows (E/2+1 ) errors to be detected and (E/2) errors to be corrected for each group of (N+R)*E words. In this example the first group of datawords and codewords is {D1 i , D2i , D3i , D12 , D22 , D32 , Dr1 -i , Dr12 }. Embodiments can use known techniques for encoding and decoding the data, such as the Berlekamp-Massey algorithm. In the present implementation according to an embodiment, each group has properties equivalent to a (8, 6) RS code. More generally, each group according to the present embodiment has properties equivalent to a ((N+R)*E, N*E) code. The coding of data according to the present embodiment allows erased/lost data to be recovered using known techniques. Figure 2 shows another implementation according to the present embodiment. In the present implementation, E=2 and R=2 and it is therefore similar to a RAID6 design. There are twice as many datawords contributing to each redundant codeword than if standard RS coding is used. Although shown with two redundant nodes in Figure 2, embodiments include any number of redundant nodes being used, i.e. R redundant nodes, with the coding technique for generating each redundant node according to the present embodiment being extended so that twice as many datawords contribute to each redundant codeword than in standard RS coding. This enables error detection and correction for RAID6 even when one node is lost. In addition the number of correctable errors is improved over RAID6 when no nodes are lost.
In Figure 2 the shown finite field coefficients are purely exemplary. By using a sufficiently large finite field, for instance GF(256), it is possible to find coefficients for an MDS code. It is possible to select random coefficients and then check if the system is MDS. It is also possible to use standard coefficients from a ((N+R)*E, N*E) Reed-Solomon code as a basis for selecting coefficients. Figure 3 is another implementation of the present embodiment. The implementation in Figure 3, is similar to that in Figure 1 but with a larger value of E and so the group size is increased. In the shown implementation, E=3 and so there are three times as many datawords contributing to each redundant codeword than in standard RS coding. More generally, embodiments include E being any value larger than 2.
In all of the implementations of embodiments, error(s) can be detected and recovered using known error correction techniques, such as a Berlekamp- Massey decoder or with Soft-decoding, as described in, for example, Koetter, Ralf; Vardy, Alexander (2003). "Algebraic soft-decision decoding of Reed- Solomon codes". IEEE Transactions on Information Theory 49 (11): 2809-2825. doi:10.1109/TIT.2003.819332.
A second embodiment is shown in Figure 4. In the first embodiment each node set contained X/E separate groups, with each group comprising an equation system that is dependent on more than one row of source data. In the second embodiment, the number of separate groups, i.e. equation systems, is E for the entire set of data nodes. For the specific implementation in which X=E, the implementation is the same as the first embodiment. The system shown in Figure 4 has 3 data nodes and 1 redundant node. In the shown implementation of the present embodiment, E = 2 and so only two redundant codewords have been extended, namely Dr1x_i and Dr1x.
Dr1x_i is generated by combining, using finite field encoding operations that are preferably XOR additions, the datawords from the data nodes with indexes X-1 and X. Dr1x is generated by combining the datawords from all of the data nodes, using finite field encoding operations that are different from those used to generate Dr1x-i. According to the present embodiment, it is possible to combine, preferably XOR, all redundant codewords Dr1 i to Dr1x_i to form an equation for a codeword Dr1 *:
DM * = Dr1 i + Dr12 + ... + Dr1x-2 + Dr1x-1
DM * = (D1 -, + D2 + D3i) + (D12 + D22 + D32) + ... + (D1x-2 + D2x-2 + D3x-2)
+ (D1x-i + D2x-i + D3x-i + D1x + D2X + D3X) Accordingly, Dr1 * contains all datawords from all nodes XOR'ed together. According to the present embodiment Dr1x, also contains all datawords from each data node, encoded with finite field multiplications other than XOR. The Dr1 * and Dr1x codewords can therefore be used as a standard known Reed- Solomon code with block length (3*X)+2. This can also be referred to as a (((3*X)+2), (3*X)) code. More generally, with N data nodes, embodiments provide a (((N*X)+2), N*X) code, i.e. a Reed-Solomon code with 2 redundancy codewords. It is therefore possible to detect 2 errors and correct 1 error for each node set when no nodes are lost. The present embodiment also includes any of the redundant codewords being extended as described for Dr1x- and Dr1x above and are not restricted to the extended codewords being the last two codewords in the redundant node. The number of extended codewords are not limited to E = 2, and E can be any value from 2 to X inclusive. Another implementation according to the present embodiment is shown in Figure 5 in which E = 3. In the present implementation, codewords Dr1x_i and Dr1x are fully extended while the codeword Dr1x-2 is partially extended. With this design it is possible to construct a standard Reed- Solomon code with block length ((3*X)+3), or a (((N*X)+3), N*X) code.
Embodiments also include increasing the number of fully extended codewords until X-1 codewords in the redundancy node are fully extended. Embodiments provide a (((N*X)+E), N*X) code, with E from 2 to X inclusive.
Embodiments include choosing finite field coefficients that are linearly independent when constructing Dr1x_i and Dr1x in order to improve the recovery of a lost data node. Embodiments also include first constructing a Reed- Solomon code of size (((N*X)+E), N*X), and then reordering the dataword layout in order to obtain suitable finite field coefficients for the system.
Embodiments include choosing finite field coefficients that are linearly independent when constructing DMX-1 and Dr1X in order to improve the recovery of a lost data node. Embodiments also include first constructing a Reed-Solomon code of size (((N*X)+E), N*X), and then reordering the dataword layout in order to obtain suitable finite field coefficients for the system.
In embodiments the number of redundant nodes is not limited to 1 , and the number of redundant nodes can be any number. An example of how to another redundant node can be provided is shown in Figure 6. This is similar to a RAID6 configuration. In the shown implementation E = 3, and so 3 codewords have been extended on each redundancy node, namely Dr1x-2, Dr1x- , Dr1x, Dr2x-2, Dr2x-i and Dr2x. Codewords Dr1x-2 and Dr2x-2 are partially extended. It is possible to design a Dr2* codeword in a corresponding way to the Dr1* codeword in Figure 5. However, codewords Dr2-i to Dr2x-3 should be encoded with different finite field coefficients/operations in order for the constructed Dr2* codeword to have the property of being MDS. The code according to the present embodiment corresponds to a (((N*X)+(3*2)), N*X) code. The number of extended codewords are not limited to 3, and can be from 1 to X-1. More generally, embodiments can provide a (((N*X)+(E*R)), N*X) code, with X number of codewords for each data block, N data blocks, R redundancy blocks and E extended codewords.
For both embodiments the number of data nodes may be 1 to N. For N=1 and R=1 the design will enable duplication like storage with error detection and correction abilities. For N=1 and R=2 the design will enable triplication like storage with extended error detection and correction abilities.
In high performance implementations, all of the embodiments presented herein may be combined/rearranged with known interleaved codes. This can be done so that datawords with the same finite field coefficients become spatially sequential, similar to existing known methods. This enables the more efficient usage of usage of SIMD vector instructions.
In the above-described embodiments the RAID5 and RAID6 systems have been used to present the techniques of embodiments. However, embodiments are also applicable to other types of storage system. The embodiment is also applicable in software defined storage systems, determining how to encode data packets in packet based communication (like UDP/IP), solid state disks (SSDs), random access memory and RAIDz with standard erasure coding as well as other use cases. Also, in all embodiments, the choice of using the last indexes for storing modified codewords is used as an example, and is not required. The modified codewords may be stored at any index by rearranging the data layout.
Embodiments have been explained with coefficients indicated by the superscript notation. These coefficients represent any finite field operation, including weighting, that may be performed to ensure the MDS properties of the code are obtained.
An application in which the coding of data according to embodiments is particularly preferable is as a Plugln for CEPH storage system products. Details of CEPH storage system products can be found at: http://ceph.com/; as viewed on 25th April 2017.
There are some versions of CEPH that do not support, or have limited support of, the use of metadata to aid the detection and recovery from errors in stored data. More specifically, there is no support for the use checksums generated from data stored using erasure coding. Moreover, due to overhead of storing checksums/hashes, they are not used on small blocks of data and only used on large amounts of data. Embodiments allow a CEPH system to be modified so that additional parity/redundancy is introduced without there being a substantial increase, or any increase, in metadata. CEPH allows known reading/data processing operations to be modified and it is possible to use CEPH to implement the embodiments described herein. Advantageously, embodiments improve both the detection and correction of errors in read operations by CEPH systems. No external error handling support is required by the modified storage system. Accordingly, it is fast and efficient to implement.
Embodiments can also be implemented in other types of system than CEPH. The embodiments described throughout the present document can also be used in combination with other coding techniques. In particular, embodiments can also be used in combination with techniques that calculate data characterising values, such as checksums and hashes, in order to aid error detection and recovery. In particular, embodiments can be used in combination with the techniques as disclosed in either of the patent applications GB1604648.4 and PCT/IB2017/000354, the entire contents of which are incorporated herein by reference. The combination of techniques allows data characterising values to be used quickly determine if errors have occurred and then error correcting operation to only be applied when errors are detected.
Figure 7 is a flowchart of a process according to an embodiment.
In step 701 , the process begins. In step 703, the process generates redundant data of source data, wherein there are N source nodes of source data and R redundant nodes of redundant data such that there are a plurality of (N+R) nodes, wherein each of the (N+R) nodes comprises a plurality of sub-blocks of data, wherein a block of data comprises E rows, with each of the E rows comprising (N+R) sub-blocks and each of the (N+R) sub-blocks of a row of a block are comprised by a different one of the (N+R) nodes, such that each block comprises (ExN) sub-blocks of source data and (ExR) sub-blocks of redundant data, and the number of blocks of data is B, the method comprising, for each of the B blocks: generating each of the sub- blocks of redundant data of the block in dependence on all of the sub-blocks of source data of the block in accordance with an MDS coding technique such that the sub-blocks of the block provide an MDS code with (ExN) sub-blocks of source data and (ExR) sub-blocks of redundant data. In step 705, the process ends.
Figure 8 is a flowchart of a process according to an embodiment.
In step 801 , the process begins.
In step 803, the process generates redundant data of source data, wherein there are N source nodes of source data and R redundant nodes of redundant data such that there are a plurality of (N+R) nodes, wherein each of the (N+R) nodes comprises X sub-blocks of data, wherein a block of data comprises E rows, with each of the E rows comprising (N+R) sub-blocks and each of the (N+R) sub- blocks of a row of a block are comprised by a different one of the (N+R) nodes, such that the block comprises (ExN) sub-blocks of source data and (ExR) sub- blocks of redundant data, the method comprising: generating, for each of the (X- E) rows that are not comprised by the block, each redundant sub-block of the row in dependence on all of the sub-blocks of source data of the row in accordance with an MDS coding technique; generating, for one of the rows comprised by the block, each redundant sub-block of the row in dependence on all of the sub-blocks of source data of the rows comprised by the block in accordance with an MDS coding technique; and generating, for each of the other rows comprised by the block, each redundant sub-block of the row in dependence on all of the (NxX) sub-blocks of source data in accordance with an MDS coding technique such that an MDS code is generated with (NxX) sub- blocks of source data and (ExR) sub-blocks of redundant data.
In step 805, the process ends.
Embodiments include a number of modifications and variations to the techniques described above.
For example, one or more, but not all of the redundant nodes may be constructed as described. Thus, some of the nodes can be constructed as known RS codes. Such redundant nodes do not have extra repair capabilities, but do not require additional computing resources to construct.
Embodiments also include all combinations of the techniques presented for each of the different implementations of embodiments herein. For example, embodiments include the techniques as shown in Figure 2 being used in combination with those shown in Figure 3.
Preferably, each coding of source and redundant nodes that are generated according to embodiments is tested to determine that it has the property of being MDS. This preferably is part of an iterative process with each iteration changing one or more of the coefficients or operations when generating the redundant node(s). The iterative process would be stopped as soon as the overall coding was determined to have the property of being MDS.
The values of N and R according to embodiments can be flexibly chosen and can cover all disk array sizes. The underlying finite field may be GF(256). The codes according to embodiments can be constructed over any GF field of size GF(2n) so long as n is large enough for a MDS code to be generated. A typical implementation would have a GF field size of GF(16) or GF(256). The actual generation of coded data in dependence on source data according to embodiments can be performed with known techniques and using known hardware. The processes required to use the techniques according to embodiments to generate a plurality of source nodes and redundant nodes in a data storage system/network of a data centre storing the coded data would be a straightforward task for the skilled person. The skilled person would also be able to use known hardware to reconstruct one or more source nodes in order to implement embodiments. The nodes according to embodiments include single data disks, or drives, or groups or data disks, or drives. A node includes any form of data storage element, a part of such an element or multiple such elements. In particular, a node can be any logical entity where data can be stored, and can be anything from a whole, a group of or parts of physical storage devices or locations including but not limited to memory based storage devices such as RAM and SSDs, hard drives, tape storage, optical storage devices, servers and data centers. The method according to embodiments may be performed within a single SSD disk. The method according to embodiments may be performed between chips inside a SSD, or between banks inside (flash) chips.
The storage of the data in a data storage system is not limited to the data storage system having nodes, i.e. data drives or sections of a data drive, that are only for use as a store of source data node or redundant data. A mapping may be introduced so that a data drive may store redundant data within a source data node and vice-versa. This interleaving of data changes the mapping of coded data to stored data and can be used to control the read operations from a data storage system, for example to ensure that the network traffic is balanced across the data storage system. Although data storage is a particularly preferable application for the coding techniques disclosed herein embodiments include the generation of codes for any application, such as data transmission. For example, the nodes and/or blocks may correspond to data packets for transmission over a network. In particular, the embodiments can be used to determine data packets for transmission as TCP/IP or UDP packets, as well as other form of data packets used for data transmission over a network.
The flow charts and descriptions thereof herein should not be understood to prescribe a fixed order of performing the method steps described therein. Rather, the method steps may be performed in any order that is practicable. Although the present invention has been described in connection with specific exemplary embodiments, it should be understood that various changes, substitutions, and alterations apparent to those skilled in the art can be made to the disclosed embodiments without departing from the spirit and scope of the invention as set forth in the appended claims.
Methods and processes described herein can be embodied as code (e.g., software code) and/or data. Such code and data can be stored on one or more computer-readable media, which may include any device or medium that can store code and/or data for use by a computer system. When a computer system reads and executes the code and/or data stored on a computer-readable medium, the computer system performs the methods and processes embodied as data structures and code stored within the computer-readable storage medium. In certain embodiments, one or more of the steps of the methods and processes described herein can be performed by a processor (e.g., a processor of a computer system or data storage system). It should be appreciated by those skilled in the art that computer-readable media include removable and non-removable structures/devices that can be used for storage of information, such as computer-readable instructions, data structures, program modules, and other data used by a computing system/environment. A computer-readable medium includes, but is not limited to, volatile memory such as random access memories (RAM, DRAM, SRAM); and non-volatile memory such as flash memory, various read-only-memories (ROM, PROM, EPROM, EEPROM), magnetic and ferromagnetic/ferroelectric memories (MRAM, FeRAM), phase- change memory and magnetic and optical storage devices (hard drives, magnetic tape, CDs, DVDs); network devices; or other media now known or later developed that is capable of storing computer-readable information/data. Computer-readable media should not be construed or interpreted to include any propagating signals.

Claims

Claims:
A computer-implemented method of generating redundant data of source data, wherein there are N source nodes of source data and R redundant nodes of redundant data such that there are a plurality of (N+R) nodes, wherein each of the (N+R) nodes comprises a plurality of sub-blocks of data, wherein a block of data comprises E rows, with each of the E rows comprising (N+R) sub-blocks and each of the (N+R) sub-blocks of a row of a block are comprised by a different one of the (N+R) nodes, such that each block comprises (ExN) sub-blocks of source data and (ExR) sub- blocks of redundant data, and the number of blocks of data is B, the method comprising, for each of the B blocks: generating each of the sub-blocks of redundant data of the block in dependence on all of the sub-blocks of source data of the block in accordance with an MDS coding technique such that the sub-blocks of the block provide an MDS code with (ExN) sub-blocks of source data and (ExR) sub-blocks of redundant data.
The method according to claim 1 , wherein E is 2 or more.
The method according to claim 1 or 2, wherein the number of sub-blocks comprised by each node is X and:
1 < B < (X/E).
The method according to any preceding claim, wherein R = 1.
5. The method according to any of claims 1 to 3, wherein R is 2 or more.
6. The method according to any preceding claim, wherein N is 2 or more.
7. The method according to any preceding claim, wherein each node is data drive of a data storage system.
8. The method according to any preceding claim, wherein each row of sub- blocks corresponds to a data packet for transmission over a network.
9. A method for storing data in a data storage system, wherein the data storage system comprises N source nodes of source data and R redundant nodes of redundant data such that there are a plurality of (N+R) nodes, wherein each of the (N+R) nodes comprises a plurality of sub-blocks of data, wherein a block of data comprises E rows, with each of the E rows comprising (N+R) sub-blocks and each of the (N+R) sub- blocks of a row of a block are comprised by a different one of the (N+R) nodes, such that each block comprises (ExN) sub-blocks of source data and (ExR) sub-blocks of redundant data, and the number of blocks of data is B, the method comprising, for each of the B blocks: generating each of the sub-blocks of redundant data of the block in dependence on all of the sub-blocks of source data of the block in accordance with an MDS coding technique such that the sub-blocks of the block provide an MDS code with (ExN) sub-blocks of source data and (ExR) sub-blocks of redundant data; and storing the source data and the redundant data in the data storage nodes of the data storage system.
10. The method according to claim 9, further comprising performing a mapping operation on the source data and redundant data such that one or more of the nodes stores both source data and redundant data.
11. A method of coding source data, the method comprising: obtaining source data; and encoding the source data in accordance with the method of any preceding claim.
12. A coding technique for generating coded data from source data, the coding technique being equivalent to generating the coded data in dependence on the method of any preceding claim.
13. A generator matrix for defining how to generate coded data from source data, the generator matrix defining a code that is equivalent to a code that has been generated by the method of any preceding claim. 14. A computing system configured to perform the method of any of claims 1 to 12.
15. A computer program that, when executed by a computing system, causes the computing system to perform the method of any of claims 1 to 12.
16. A computer readable medium comprising instructions that, when executed, cause a computing device to perform the method of any of claims 1 to 12. 17. A data storage system configured to perform the method of claim 9 or 10.
18. A computer-implemented method of detecting one or more errors in data, and/or correcting one or more errors in data, by using detection and/or decoding techniques of MDS codes, wherein the method is performed on data coded according to the method of any of claims 1 to 12.
19. A computing system configured to perform the method of claim 18.
20. A computer-implemented method of generating redundant data of source data, wherein there are N source nodes of source data and R redundant nodes of redundant data such that there are a plurality of (N+R) nodes, wherein each of the (N+R) nodes comprises X sub-blocks of data, wherein a block of data comprises E rows, with each of the E rows comprising (N+R) sub-blocks and each of the (N+R) sub-blocks of a row of a block are comprised by a different one of the (N+R) nodes, such that the block comprises (ExN) sub-blocks of source data and (ExR) sub- blocks of redundant data, the method comprising: generating, for each of the (X-E) rows that are not comprised by the block, each redundant sub-block of the row in dependence on all of the sub-blocks of source data of the row in accordance with an MDS coding technique; generating, for one of the rows comprised by the block, each redundant sub-block of the row in dependence on all of the sub-blocks of source data of the rows comprised by the block in accordance with an MDS coding technique; and generating, for each of the other rows comprised by the block, each redundant sub-block of the row in dependence on all of the (NxX) sub- blocks of source data in accordance with an MDS coding technique such that an MDS code is generated with (NxX) sub-blocks of source data and (ExR) sub-blocks of redundant data.
21. The method according to claim 20, wherein 2 < E < X, preferably 2 < E (X-1 ).
22. The method according to claim 20 or 21 , wherein R = 1.
23. The method according to any of claims 20 or 21 , wherein R is 2 or more.
24. The method according to any of claims 20 to 23, wherein N is 2 or more.
25. The method according to any of claims 20 to 24, wherein each node is data drive of a data storage system.
26. The method according to any of claims 20 to 25, wherein each row of sub-blocks corresponds to a data packet for transmission over a network.
27. A method for storing data in a data storage system, wherein the data storage system comprises N source nodes of source data and R redundant nodes of redundant data such that there are a plurality of (N+R) nodes, wherein each of the (N+R) nodes comprises X sub-blocks of data, wherein a block of data comprises E rows, with each of the E rows comprising (N+R) sub-blocks and each of the (N+R) sub-blocks of a row of a block are comprised by a different one of the (N+R) nodes, such that the block comprises (ExN) sub-blocks of source data and (ExR) sub-blocks of redundant data, the method comprising: generating, for each of the (X-E) rows that are not comprised by the block, each redundant sub-block of the row in dependence on all of the sub-blocks of source data of the row in accordance with an MDS coding technique; generating, for one of the rows comprised by the block, each redundant sub-block of the row in dependence on all of the sub-blocks of source data of the rows comprised by the block in accordance with an MDS coding technique; generating, for each of the other rows comprised by the block, each redundant sub-block of the row in dependence on all of the (NxX) sub- blocks of source data in accordance with an MDS coding technique such that an MDS code is generated with (NxX) sub-blocks of source data and (ExR) sub-blocks of redundant data; and storing the source data and the redundant data in the data storage nodes of the data storage system.
28. The method according to claim 27, further comprising performing a mapping operation on the source data and redundant data such that one or more of the nodes stores both source data and redundant data.
29. A method of coding source data, the method comprising: obtaining source data; and encoding the source data in accordance with the method of any of claims 20 to 28.
30. A coding technique for generating coded data from source data, the coding technique being equivalent to generating the coded data in dependence on the method of any of claims 20 to 29.
31. A generator matrix for defining how to generate coded data from source data, the generator matrix defining a code that is equivalent to a code that has been generated by the method of any of claims 20 to 30.
32. A computing system configured to perform the method of any of claims 20 to 30.
33. A computer program that, when executed by a computing system, causes the computing system to perform the method of any of claims 20 to 30.
34. A computer readable medium comprising instructions that, when executed, cause a computing device to perform the method of any of claims 20 to 30.
35. A data storage system configured to perform the method of claim 27 or 28.
36. A computer-implemented method of detecting one or more errors in data, and/or correcting one or more errors in data, by using detection and/or decoding techniques of MDS codes, wherein the method is performed on data coded according to the method of any of claims 20 to 30.
37. A computing system configured to perform the method of claim 36.
PCT/EP2017/060114 2016-04-27 2017-04-27 Data protection coding technique WO2017186871A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB201607325 2016-04-27
GB1607325.6 2016-04-27

Publications (1)

Publication Number Publication Date
WO2017186871A1 true WO2017186871A1 (en) 2017-11-02

Family

ID=58640888

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2017/060114 WO2017186871A1 (en) 2016-04-27 2017-04-27 Data protection coding technique

Country Status (1)

Country Link
WO (1) WO2017186871A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110302369A1 (en) * 2010-06-03 2011-12-08 Buffalo Inc. Storage apparatus and control method therefor
US8327080B1 (en) * 2010-09-28 2012-12-04 Emc Corporation Write-back cache protection
US20150303949A1 (en) * 2014-04-21 2015-10-22 The Regents Of The University Of California [01] cost-efficient repair for storage systems using progressive engagement

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110302369A1 (en) * 2010-06-03 2011-12-08 Buffalo Inc. Storage apparatus and control method therefor
US8327080B1 (en) * 2010-09-28 2012-12-04 Emc Corporation Write-back cache protection
US20150303949A1 (en) * 2014-04-21 2015-10-22 The Regents Of The University Of California [01] cost-efficient repair for storage systems using progressive engagement

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
KOETTER, RALF; VARDY, ALEXANDER: "Algebraic soft-decision decoding of Reed-Solomon codes", IEEE TRANSACTIONS ON INFORMATION THEORY, vol. 49, no. 11, 2003, pages 2809 - 2825

Similar Documents

Publication Publication Date Title
US10218789B2 (en) Erasure correcting coding using temporary erasure data
US10572345B2 (en) First responder parities for storage array
JP6153541B2 (en) Method, system and program for storing data in storage array using erasure error correction code
CN103392172B (en) Correct the erasing in storage array
US9417963B2 (en) Enabling efficient recovery from multiple failures together with one latent error in a storage array
US8522122B2 (en) Correcting memory device and memory channel failures in the presence of known memory device failures
US10901839B2 (en) Common high and low random bit error correction logic
US9058291B2 (en) Multiple erasure correcting codes for storage arrays
JP2008117395A (en) Detection and correction of block-level data corruption in fault-tolerant data storage system
EP2929436A1 (en) Stopping criteria for layered iterative error correction
US20120017140A1 (en) Non-mds erasure codes for storage systems
US10417088B2 (en) Data protection techniques for a non-volatile memory array
JP2019057752A (en) Memory system
Lastras-Montaño et al. A new class of array codes for memory storage
US11042440B2 (en) Data checksums without storage overhead
JP7249719B2 (en) Common high random bit error and low random bit error correction logic
WO2017186871A1 (en) Data protection coding technique
US10902154B2 (en) Data security
US11831333B2 (en) Encoding for data recovery in storage systems
WO2017194780A1 (en) Balanced locally repairable erasure codes
Vardy et al. Codes for RAID Solutions based upon SSDs
KR20170067656A (en) Turbo product codes for nand flash

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17720119

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17720119

Country of ref document: EP

Kind code of ref document: A1