US20080022150A1 - Method and system for improved buffer utilization for disk array parity updates - Google Patents

Method and system for improved buffer utilization for disk array parity updates Download PDF

Info

Publication number
US20080022150A1
US20080022150A1 US11867407 US86740707A US2008022150A1 US 20080022150 A1 US20080022150 A1 US 20080022150A1 US 11867407 US11867407 US 11867407 US 86740707 A US86740707 A US 86740707A US 2008022150 A1 US2008022150 A1 US 2008022150A1
Authority
US
Grant status
Application
Patent type
Prior art keywords
parity
buffer
value
disk
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US11867407
Inventor
Carl Forhan
Robert Galbraith
Adrian Gerhard
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1076Parity data used in redundant arrays of independent storages, e.g. in RAID systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2211/00Indexing scheme relating to details of data-processing equipment not covered by groups G06F3/00 - G06F13/00
    • G06F2211/10Indexing scheme relating to G06F11/10
    • G06F2211/1002Indexing scheme relating to G06F11/1076
    • G06F2211/1057Parity-multiple bits-RAID6, i.e. RAID 6 implementations

Abstract

During a parity update of a parity stripe in a disk array, constant values used in finite field arithmetic are algebraically combined in order to reduce the number of buffers and steps needed to update multiple parity values when a change in data occurs. In one implementation, for example, the contents of a buffer that stores the product of a delta value associated with the change in data and a first constant, which is used to update a first parity value, are multiplied by a value representative of the ratio of a second constant, which is used to update a second parity value, and the first constant.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS
  • This application is a divisional of U.S. Patent Application Ser. No. 10/994,086 filed on Nov. 19, 2004 by Carl Edward Forhan et al. (ROC920040177US1), and is related to the following U.S. Patent Applications all filed on Nov. 19, 2004 by Carl Edward Forhan, Robert Edward Galbraith and Adrian Cuenin Gerhard: application Ser. No. 10/994,088, entitled “METHOD AND SYSTEM FOR ENHANCED ERROR IDENTIFICATION WITH DISK ARRAY PARITY CHECKING” (ROC920040141US 1), application Ser. No. 10/994,099, entitled “RAID ENVIRONMENT INCORPORATING HARDWARE-BASED FINITE FIELD MULTIPLIER FOR ON-THE-FLY XOR” (ROC920040176US1), application Ser. No. 10/994,098, entitled “METHOD AND SYSTEM FOR INCREASING PARALLELISM OF DISK ACCESSES WHEN RESTORING DATA IN A DISK ARRAY SYSTEM” (ROC920040178US 1), and application Ser. No. 10/994,097, entitled “METHOD AND SYSTEM FOR RECOVERING FROM ABNORMAL INTERRUPTION OF A PARITY UPDATE OPERATION IN A DISK ARRAY SYSTEM” (ROC920040179US1). Each of these applications is incorporated by reference herein.
  • FIELD OF THE INVENTION
  • The present invention relates to data protection methods for data storage and, more particularly, to systems implementing RAID-6 and similar data protection and recovery strategies.
  • BACKGROUND OF THE INVENTION
  • RAID stands for Redundant Array of Independent Disks and is a taxonomy of redundant disk array storage schemes which define a number of ways of configuring and using multiple computer disk drives to achieve varying levels of availability, performance, capacity and cost while appearing to the software application as a single large capacity drive. Typical RAID storage subsystems can be implemented in either hardware or software. In the former instance, the RAID algorithms are packaged into separate controller hardware coupled to the computer input/output (“I/O”) bus and, although adding little or no central processing unit (“CPU”) overhead, the additional hardware required nevertheless adds to the overall system cost. On the other hand, software implementations incorporate the RAID algorithms into system software executed by the main processor together with the operating system, obviating the need and cost of a separate hardware controller, yet adding to CPU overhead.
  • Various RAID levels have been defined from RAID-0 to RAID-6, each offering tradeoffs in the previously mentioned factors. RAID-0 is nothing more than traditional striping in which user data is broken into chunks which are stored onto the stripe set by being spread across multiple disks with no data redundancy. RAID-1 is equivalent to conventional “shadowing” or “mirroring” techniques and is the simplest method of achieving data redundancy by having, for each disk, another containing the same data and writing to both disks simultaneously. The combination of RAID-0 and RAID-1 is typically referred to as RAID-0+1 and is implemented by striping shadow sets resulting in the relative performance advantages of both RAID levels. RAID-2, which utilizes Hamming Code written across the members of the RAID set is not now considered to be of significant importance.
  • In RAID-3, data is striped across a set of disks with the addition of a separate dedicated drive to hold parity data. The parity data is calculated dynamically as user data is written to the other disks to allow reconstruction of the original user data if a drive fails without requiring replication of the data bit-for-bit. Error detection and correction codes (“ECC”) such as Exclusive-OR (“XOR”) or more sophisticated Reed-Solomon techniques may be used to perform the necessary mathematical calculations on the binary data to produce the parity information in RAID-3 and higher level implementations. While parity allows the reconstruction of the user data in the event of a drive failure, the speed of such reconstruction is a function of system workload and the particular algorithm used.
  • As with RAID-3, the RAID scheme known as RAID-4 consists of N data disks and one parity disk wherein the parity disk sectors contain the bitwise XOR of the corresponding sectors on each data disk. This allows the contents of the data in the RAID set to survive the failure of any one disk. RAID-5 is a modification of RAID-4 which stripes the parity across all of the disks in the array in order to statistically equalize the load on the disks.
  • The designation of RAID-6 has been used colloquially to describe RAID schemes that can withstand the failure of two disks without losing data through the use of two parity drives (commonly referred to as the “P” and “Q” drives) for redundancy and sophisticated ECC techniques. Although the term “parity” is used to describe the codes used in RAID-6 technologies, the codes are more correctly a type of ECC code rather than simply a parity code. Data and ECC information are striped across all members of the RAID set and write performance is generally lower than with RAID-5 because three separate drives must each be accessed twice during writes. However, the principles of RAID-6 may be used to recover a number of drive failures depending on the number of “parity” drives that are used.
  • Some RAID-6 implementations are based upon Reed-Solomon algorithms, which depend on Galois Field arithmetic. A complete explanation of Galois Field arithmetic and the mathematics behind RAID-6 can be found in a variety of sources and, therefore, only a brief overview is provided below as background. The Galois Field arithmetic used in these RAID-6 implementations takes place in GF(2N). This is the field of polynomials with coefficients in GF(2), modulo some generator polynomial of degree N. All the polynomials in this field are of degree N-I or less, and their coefficients are all either 0 or 1, which means they can be represented by a vector of N coefficients all in {0,1}; that is, these polynomials “look” just like N-bit binary numbers. Polynomial addition in this Field is simply N-bit XOR, which has the property that every element of the Field is its own additive inverse, so addition and subtraction are the same operation. Polynomial multiplication in this Field, however, can be performed with table lookup techniques based upon logarithms or with simple combinational logic.
  • Each RAID-6 check code (i.e., P and Q) expresses an invariant relationship, or equation, between the data on the data disks of the RAID-6 array and the data on one or both of the check disks. If there are C check codes and a set of F disks fail, F≦C, the failed disks can be reconstructed by selecting F of these equations and solving them simultaneously in GF(2N) for the F missing variables. In the RAID-6 systems implemented or contemplated today there are only 2 check disks—check disk P, and check disk Q. It is worth noting that the check disks P and Q change for each stripe of data and parity across the array such that parity data is not written to a dedicated disk but is, instead, striped across all the disks.
  • Even though RAID-6 has been implemented with varying degrees of success in different ways in different systems, there remains an ongoing need to improve the efficiency and costs of providing RAID-6 protection for data storage. The mathematics of implementing RAID-6 involve complicated calculations that are also repetitive. Accordingly, efforts to improve the simplicity of circuitry, the cost of circuitry and the efficiency of the circuitry needed to implement RAID-6 remains a priority today and in the future.
  • For example, one drawback with conventional RAID-6 designs relates to the throughput of parity updates in such designs due to comparatively higher buffer requirements for performing such updates. A parity update, in this context, refers to updating the parity information stored in a given parity stripe in a disk array in response to a change in the data stored in the parity stripe.
  • By way of comparison, in a RAID-5 design, a parity update operation typically requires, first, that the old data to be updated be retrieved from the appropriate disk and compared to the new data to calculate a delta value Δ, e.g., by performing an exclusive-OR (XOR) operation with the old and new data. This delta value Δ is then used to update the parity, e.g., by performing an XOR operation with the old parity value and the delta value Δ.
  • Of note, the two XOR operations performed during a RAID-5 parity update can be implemented using only two buffers. Specifically, a first buffer can be used to initially store the new data to be written, with a second buffer used to store the delta value Δ generated from the XOR of the new data stored in the first buffer and old data retrieved from the appropriate disk (which can be fed directly to XOR logic, and thus does not need to be buffered). Once the new data is written to the appropriate disk, the first buffer can then be reused to store the result of the XOR operation between the delta value A stored in the second buffer and the old parity value retrieved from the appropriate disk (which also can be fed directly to XOR logic without buffering). After this second XOR operation, the first buffer stores the new parity value, which can then be read out of the first buffer and written to the appropriate disk.
  • On the other hand, in a RAID-6 environment, where two disks include parity data for each parity stripe, parity update operations typically require at least one additional buffer to store interim data associated with such operations. In particular, while RAID-6 parity updates still require the calculation of a delta value Δ, updating both of the parity values for a given parity stripe require that the delta value Δ be multiplied or scaled by different constants that are respectively associated with the different parity stripe equations that relate the parity values to the data in the parity stripe equation. These different constants are conventionally designated as constants K1 and K2, and as such, one parity value P for a parity stripe is typically calculated by performing an XOR of the old parity value P and the product of constant K1 and delta value Δ (i.e., K1Δ), while the other parity value Q for the same parity stripe is typically calculated by performing an XOR of the old parity value Q and the product of constant K2 and delta value Δ (i.e., K2Δ).
  • In a conventional RAID-6 implementation, three or more buffers are used, with a first buffer initially storing the new data to be written, and with a second buffer used to store the delta value Δ generated from the XOR of the new data stored in the first buffer and old data retrieved from the appropriate disk. Since the delta value Δ is required for both parity values, a third buffer is used to store the product K1Δ of the delta value Δ stored in the second buffer and constant K1. Also, similar to a RAID-5 implementation, after the new data is written to the appropriate disk, the first buffer is reused to store the result of an XOR operation between the product K1Δ stored in the third buffer and the old parity value P retrieved from the appropriate disk, which result is then written back out to the appropriate disk as the new parity value P. To update parity value Q, after the parity value P is updated, the third buffer is then reused to store the product K2Δ of the delta value Δ stored in the second buffer and constant K2, and the first buffer is then reused to store the result of an XOR operation between the product K2Δ now stored in the third buffer and the old parity value Q retrieved from the appropriate disk. This result, now stored in the first buffer, is then written back out to the appropriate disk as the new parity value Q.
  • Utilizing three buffers for a RAID-6 parity update, however, increases buffer requirements by 50% over the two buffers required for RAID-5 designs. As a result, in situations where the number of buffers is constrained, a RAID-6 design typically can have only 2/3rd the number of parity update operations in progress in comparison to a RAID-5 design, thus reducing throughput and limiting overall performance.
  • SUMMARY OF THE INVENTION
  • The invention addresses these and other problems associated with the prior art by algebraically combining multiple constant values used in updating multiple parity values for a parity stripe such that a product used in the generation of one parity value is effectively derived from the product used in the generation of the other parity value. By doing so, the delta value that is common to both products need not be separately stored in a buffer to calculate each product, thus eliminating the need for such a buffer. As a result, parity updates made to multiple parity values may be performed with a reduced number of required buffers, thus limiting the potential bottlenecks associated with the buffers, and improving overall throughput and performance.
  • By way of example, in one exemplary RAID-6 implementation consistent with the invention, where P and Q parity values are calculated by XOR'ing old P and Q values with separate products K1Δ and K2Δ, the product K2Δ is derived from the product K1Δ by multiplying the product K1Δ by a value (designated herein as K3) representative of the ratio of K2 and K1 (i.e., K2/K1). As a result, the need for a separate buffer to store the delta value Δ for use in calculating each of the products K1Δ and K2Δ is eliminated.
  • Therefore, consistent with the invention, parity values for a parity stripe in a disk array are updated in association with storing new data in the parity stripe, where the disk array is of the type where the parity stripe includes at least first and second parity values related to data stored in the parity stripe by at least one parity stripe equation, and where the first parity value is capable of being updated based upon the new data using a first constant derived from the at least one parity stripe equation, and the second parity value is capable of being updated based upon the new data using a second constant derived from the at least one parity stripe equation. The parity values are updated by storing in a first buffer a first value representative of a product of the first constant and a delta value representative of a difference between the new data and old data previously stored in the parity stripe, updating the first parity value using the first value stored in the first buffer, storing in a second buffer a second value representative of a product of the first value stored in the first buffer and a third constant representative of a ratio of the second constant to the first constant, and updating the second parity value using the second value stored in the second buffer.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a block diagram of an exemplary computer system that can implement a RAID-6 storage controller in accordance with the principles of the present invention.
  • FIG. 2 is a block diagram illustrating the principal components of a RAID controller of FIG. 1.
  • FIG. 3 is a flowchart of an exemplary parity update operation consistent with the invention.
  • DETAILED DESCRIPTION
  • The embodiments discussed hereinafter reduce the number of buffers required to perform parity updates in RAID-6 and similar environments by algebraically combining multiple constant values used in updating multiple parity values for a parity stripe such that a product used in the generation of one parity value is effectively derived from the product used in the generation of the other parity value. By doing so, the need to separately store a delta value, representative of the difference between new and old data in a parity stripe, is often eliminated, thus reducing buffer requirements and improving performance.
  • Presented hereinafter are a number of embodiments of a disk array environment implementing the parity update operations consistent with the invention. However, prior to discussing such embodiments, a brief background on RAID-6 is provided, followed by a description of an exemplary hardware environment within which parity update operations consistent with the invention may be implemented.
  • General RAID-6 Background
  • The nomenclature used herein to describe RAID-6 storage systems conforms to the most readily accepted standards for this field. In particular, there are N drives of which any two are considered to be the parity drives, P and Q. Using Galois Field arithmetic, two independent equations can be written:
    α0 d 00 d 10 d 2+ . . . +α0 d N-1=0   (1)
    α0 d 01 d 12 d 2+ . . . +αN-1 d N-1=0   (2)
    where the “+” operator used herein represents an Exclusive-OR (XOR) operation.
  • In these equations, αx is an element of the finite field and dx is data from the xth disk. While the P and Q disk can be any of the N disks for any particular stripe of data, they are often noted as dP and dQ. When data to one of the disks (i.e., dX) is updated, the above two equations resolve to:
    Δ=(old d X)+(new d X)   (3)
    (new d P)=(old d P)+((αQX)/(αPQ))Δ  (4)
    (new d Q)=(old d Q)+((αPX)/(αPQ))Δ  (5)
  • In each of the last two equations the term to the right of the addition sign is a constant multiplied by the change in the data (i.e., Δ). These terms in equations (4) and (5) are often denoted as K1Δ and K2Δ, respectively.
  • In the case of one missing, or unavailable drive, simple XOR'ing can be used to recover the drive's data. For example, if d1 fails then d1 can be restored by
    d 1 =d 0 +d 2 +d 3+  (6)
  • In the case of two drives failing, or being “exposed”, the above equations can be used to restore a drive's data. For example, given drives 0 through X and assuming drives A and B have failed, the data for either drive can be restored from the remaining drives. If for example, drive A was to be restored, the above equations reduce to:
    d A=((αB0)/(αBA))d 0+((αB1)/(αBA))d 1+ . . . +(αBX)/(αBA))d X   (7)
    Exemplary Hardware Environment
  • With this general background of RAID-6 in mind, attention can be turned to the drawings, wherein like numbers denote like parts throughout the several views. FIG. 1 illustrates an exemplary computer system in which a RAID-6, or other disk array, may be implemented. For the purposes of the invention, apparatus 10 may represent practically any type of computer, computer system or other programmable electronic device, including a client computer, a server computer, a portable computer, a handheld computer, an embedded controller, etc. Moreover, apparatus 10 may be implemented using one or more networked computers, e.g., in a cluster or other distributed computing system. Apparatus 10 will hereinafter also be referred to as a “computer”, although it should be appreciated the term “apparatus” may also include other suitable programmable electronic devices consistent with the invention.
  • Computer 10 typically includes at least one processor 12 coupled to a memory 14. Processor 12 may represent one or more processors (e.g., microprocessors), and memory 14 may represent the random access memory (RAM) devices comprising the main storage of computer 10, as well as any supplemental levels of memory, e.g., cache memories, non-volatile or backup memories (e.g., programmable or flash memories), read-only memories, etc. In addition, memory 14 may be considered to include memory storage physically located elsewhere in computer 10, e.g., any cache memory in a processor 12, as well as any storage capacity used as a virtual memory, e.g., as stored on the disk array 34 or on another computer coupled to computer 10 via network 18 (e.g., a client computer 20).
  • Computer 10 also typically receives a number of inputs and outputs for communicating information externally. For interface with a user or operator, computer 10 typically includes one or more user input devices 22 (e.g., a keyboard, a mouse, a trackball, a joystick, a touchpad, and/or a microphone, among others) and a display 24 (e.g., a CRT monitor, an LCD display panel, and/or a speaker, among others). Otherwise, user input may be received via another computer (e.g., a computer 20) interfaced with computer 10 over network 18, or via a dedicated workstation interface or the like.
  • For additional storage, computer 10 may also include one or more mass storage devices accessed via a storage controller, or adapter, 16, e.g., removable disk drive, a hard disk drive, a direct access storage device (DASD), an optical drive (e.g., a CD drive, a DVD drive, etc.), and/or a tape drive, among others. Furthermore, computer 10 may include an interface with one or more networks 18 (e.g., a LAN, a WAN, a wireless network, and/or the Internet, among others) to permit the communication of information with other computers coupled to the network. It should be appreciated that computer 10 typically includes suitable analog and/or digital interfaces between processor 12 and each of components 14, 16, 18, 22 and 24 as is well known in the art.
  • In accordance with the principles of the present invention, the mass storage controller 16 advantageously implements RAID-6 storage protection within an array of disks 34.
  • Computer 10 operates under the control of an operating system 30, and executes or otherwise relies upon various computer software applications, components, programs, objects, modules, data structures, etc. (e.g., software applications 32). Moreover, various applications, components, programs, objects, modules, etc. may also execute on one or more processors in another computer coupled to computer 10 via a network 18, e.g., in a distributed or client-server computing environment, whereby the processing required to implement the functions of a computer program may be allocated to multiple computers over a network.
  • In general, the routines executed to implement the embodiments of the invention, whether implemented as part of an operating system or a specific application, component, program, object, module or sequence of instructions, or even a subset thereof, will be referred to herein as “computer program code,” or simply “program code.” Program code typically comprises one or more instructions that are resident at various times in various memory and storage devices in a computer, and that, when read and executed by one or more processors in a computer, cause that computer to perform the steps necessary to execute steps or elements embodying the various aspects of the invention. Moreover, while the invention has and hereinafter will be described in the context of fully functioning computers and computer systems, those skilled in the art will appreciate that the various embodiments of the invention are capable of being distributed as a program product in a variety of forms, and that the invention applies equally regardless of the particular type of computer readable signal bearing media used to actually carry out the distribution. Examples of computer readable signal bearing media include but are not limited to recordable type media such as volatile and non-volatile memory devices, floppy and other removable disks, hard disk drives, magnetic tape, optical disks (e.g., CD-ROM's, DVD's, etc.), among others, and transmission type media such as digital and analog communication links.
  • In addition, various program code described hereinafter may be identified based upon the application within which it is implemented in a specific embodiment of the invention. However, it should be appreciated that any particular program nomenclature that follows is used merely for convenience, and thus the invention should not be limited to use solely in any specific application identified and/or implied by such nomenclature. Furthermore, given the typically endless number of manners in which computer programs may be organized into routines, procedures, methods, modules, objects, and the like, as well as the various manners in which program functionality may be allocated among various software layers that are resident within a typical computer (e.g., operating systems, libraries, API's, applications, applets, etc.), it should be appreciated that the invention is not limited to the specific organization and allocation of program functionality described herein.
  • FIG. 2 illustrates a block diagram of the control subsystem of a disk array system, e.g., a RAID-6 compatible system. In particular, the mass storage controller 16 of FIG. 1 is shown in more detail to include a RAID controller 202 that is coupled through a system bus 208 with the processor 12 and through a storage bus 210 to various disk drives 212-218. As known to one of ordinary skill, these buses may be proprietary in nature or conform to industry standards such as SCSI-1, SCSI-2, etc. The RAID controller includes a microcontroller 204 that executes program code that implements the RAID-6 algorithm for data protection, and that is typically resident in memory located in the RAID controller. In particular, data to be stored on the disks 212-218 is used to generate parity data and then broken apart and striped across the disks 212-218. The disk drives 212-218 can be individual disk drives that are directly coupled to the controller 202 through the bus 210 or may include their own disk drive adapters that permit a string a individual disk drives to be connected to the storage bus 210. In other words, a disk drive 212 may be physically implemented as 4 or 8 separate disk drives coupled to a single controller connected to the bus 210. As data is exchanged between the disk drives 212-218 and the RAID controller 202, in either direction, buffers 206 are provided to assist in the data transfers. The utilization of the buffers 206 can sometimes produce a bottle neck in data transfers and the inclusion of numerous buffers may increase cost, complexity and size of the RAID controller 202. Thus, certain embodiments of the present invention relate to provision and utilizing these buffers 206 in an economical and efficient manner.
  • It will be appreciated that the embodiment illustrated in FIGS. 1 and 2 is merely exemplary in nature. For example, it will be appreciated that the invention may be applicable to other disk array environments where multiple parity values are associated with each parity stripe. It will also be appreciated that a disk array environment consistent with the invention may utilize a completely software-implemented control algorithm resident in the main storage of the computer, or that some functions handled via program code in a computer or controller can be implemented in hardware logic circuits, and vice versa. Therefore, the invention should not be limited to the particular embodiments discussed herein.
  • Buffer Reduction for RAID-6 Parity Updates
  • In a RAID-6 system, six I/O operations are generated for every normal parity update. These I/O operations can be summarized as:
      • 1) read an old data, dX from a disk;
      • 2) write new data, dX to the disk;
      • 3) read old parity, dP from a disk;
      • 4) write new parity, dP to the disk;
      • 5) read old parity, dQ from a disk, and
      • 6) write new parity, dQ to the disk.
  • The new data and parity values are calculated according to the equations (3), (4) and (5). Typically, these operations require a number of buffers as shown in Table I below. The column refers to three different buffers A, B, C and the rows are different sequential steps of the operation of updating the parity.
    TABLE I
    Buffer A Buffer B Buffer C
    get new data, dx
    Δ = (old dX) + A
    Write new data, dx
    K1 Δ = K1 * B
    dP = (old dP) + C
    Write new parity, dP
    K2 Δ = K2 * B
    dQ = (old dQ) + C
    write new parity, dQ
  • In the above table, the addition (or XOR) steps are performed via an XOR engine in which data that is not in a buffer is read directly from a drive into the XOR engine. Compared to RAID-5, the parity update of RAID-6 utilizes additional buffers and, therefore, performs at about 2/3 the throughput in a buffer constrained environment.
  • Accordingly embodiments of the present invention include a different buffer arrangement that improves throughput of the parity update operation in a RAID-6 environment. In particular, rather than keep the delta value Δ in buffer B the entire time, simple algebra may be used to calculate the delta value Δ when generating new parity, dQ. Table II shown below, which represents a modification of Table I, illustrates how the values or products K1Δ and K2Δ (refer to equations (4) and (5)) can be manipulated to reduce the reliance on buffers in RAID-6 parity updates.
    TABLE II
    Buffer A Buffer B Buffer C
    get new data, dx
    Δ = (old dX) + A
    write new data, dx NOT
    K1 Δ = K1 * B USED
    dP = (old dP) + A
    write new parity, dp
    K2 Δ = K3*A
    dQ = (old dQ) + B
    write new parity, dQ
  • As can be seen from the table, a value K3 is defined and stored such that K3=K2/K1. When the value K3 is multiplied with the product K1Δ, the result is equal to K2Δ. Thus, by simple algebraic manipulation according to the above table, the RAID-6 parity update is accomplished using only two buffers, which will be appreciated can increase performance and throughput by limiting bottlenecks due to constrained buffer requirements.
  • FIG. 3 illustrates, similar to Table II, the steps involved in performing a parity update operation in an exemplary RAID-6 environment. In step 302, the new data for disk dx is retrieved into buffer A, and in step 304, the delta value Δ is calculated by XOR'ing the old data retrieved from disk dx with the contents of buffer A, and stored in buffer B. In step 306, the new data is written from buffer A to disk dx, and in step 308, the first value or product K1Δ is calculated by multiplying the contents of buffer B by the constant K1, and stored in buffer A.
  • In step 310, the new parity P is calculated by XOR'ing the old parity P retrieved from disk dP with the contents of buffer A, and in step 312, the new parity P is written to disk dP. Then, in step 314, the second value or product K2Δ is calculated by multiplying the contents of buffer A by the constant K3, which is set to equal K2/K1, and stored in buffer B. Then, in step 316, the new parity Q is calculated by XOR'ing the old parity Q retrieved from disk dQ with the contents of buffer B, and in step 318, the new parity Q is written to disk dQ. The parity update operation is then complete.
  • Thus, embodiments of the present invention provide a method and system, within a RAID-6 or similar environment where multiple parity values are associated with each parity stripe, that perform parity updates utilizing a reduced number of buffers. Various modifications may be made to the illustrated embodiments without departing from the spirit and scope of the invention. Therefore, the invention lies in the claims hereinafter appended.

Claims (3)

  1. 1. A method for updating first and second parity values associated with a parity stripe in a disk array in association with storing new data for a parity stripe to a data disk dx from the disk array, wherein the first and second parity values are respectively stored on first and second parity disks dP, dQ in the disk array, the method comprising:
    storing in a first buffer the new data for the parity stripe;
    storing in a second buffer a delta value A calculated by XOR'ing the new data stored in the first buffer with old data for the parity stripe retrieved from data disk dx;
    writing the new data for the parity stripe from the first buffer to data disk dx;
    after writing the new data to data disk dx, storing in the first buffer a first value K1Δ calculated by multiplying the delta value Δ stored in the second buffer by a first constant K1, where the first constant K1 is derived from at least one parity stripe equation associated with the parity stripe;
    after storing the first value K1Δ in the first buffer, storing in the second buffer a new first parity value calculated by XOR'ing the first value K1Δ stored in the first buffer with an old first parity value for the parity stripe retrieved from parity disk dP;
    writing the new first parity value stored in the second buffer to parity disk dP;
    calculating a third constant K3 by dividing a second constant K2, derived from the at least one parity stripe equation associated with the parity stripe, by the first constant K1;
    after writing the new first parity value to parity disk dP, storing in the second buffer a second value K2Δ calculated by multiplying the first value K1Δ stored in the first buffer by the third constant K3;
    after storing the second value K2Δ in the second buffer, storing in the first buffer a new second parity value calculated by XOR'ing the second value K2Δ stored in the second buffer with an old second parity value for the parity stripe retrieved from parity disk dQ; and
    writing the new second parity value stored in the first buffer to parity disk dQ.
  2. 2. An apparatus, comprising:
    first and second buffers; and
    a disk array controller configured to update first and second parity values associated with a parity stripe in a disk array in association with storing new data for a parity stripe to a data disk dx from the disk array, wherein the first and second parity values are respectively stored on first and second parity disks dP, dQ in the disk array, the disk array controller configured to:
    store in the first buffer the new data for the parity stripe;
    store in the second buffer a delta value A calculated by XOR'ing the new data stored in the first buffer with old data for the parity stripe retrieved from data disk dx;
    write the new data for the parity stripe from the first buffer to data disk dx;
    after writing the new data to data disk dx, store in the first buffer a first value K1Δ calculated by multiplying the delta value Δ stored in the second buffer by a first constant K1, where the first constant K1 is derived from at least one parity stripe equation associated with the parity stripe;
    after storing the first value K1Δ in the first buffer, store in the second buffer a new first parity value calculated by XOR'ing the first value K1Δ stored in the first buffer with an old first parity value for the parity stripe retrieved from parity disk dP;
    write the new first parity value stored in the second buffer to parity disk dP;
    calculate a third constant K3 by dividing a second constant K2, derived from the at least one parity stripe equation associated with the parity stripe, by the first constant K1;
    after writing the new first parity value to parity disk dP, store in the second buffer a second value K2Δ calculated by multiplying the first value K1Δ stored in the first buffer by the third constant K3;
    after storing the second value K2A in the second buffer, store in the first buffer a new second parity value calculated by XOR'ing the second value K2Δ stored in the second buffer with an old second parity value for the parity stripe retrieved from parity disk dQ; and
    write the new second parity value stored in the first buffer to parity disk dQ.
  3. 3. A program product, comprising:
    a computer readable medium; and
    program code stored on the computer readable medium and configured to update first and second parity values associated with a parity stripe in a disk array in association with storing new data for a parity stripe to a data disk dx from the disk array, wherein the first and second parity values are respectively stored on first and second parity disks dP, dQ in the disk array, the program code configured to:
    store in a first buffer the new data for the parity stripe;
    store in a second buffer a delta value A calculated by XOR'ing the new data stored in the first buffer with old data for the parity stripe retrieved from data disk dx;
    write the new data for the parity stripe from the first buffer to data disk dx;
    after writing the new data to data disk dx, store in the first buffer a first value K1Δ calculated by multiplying the delta value Δ stored in the second buffer by a first constant K1, where the first constant K1 is derived from at least one parity stripe equation associated with the parity stripe;
    after storing the first value K1A in the first buffer, store in the second buffer a new first parity value calculated by XOR'ing the first value K1Δ stored in the first buffer with an old first parity value for the parity stripe retrieved from parity disk dP;
    write the new first parity value stored in the second buffer to parity disk dP;
    calculate a third constant K3 by dividing a second constant K2, derived from the at least one parity stripe equation associated with the parity stripe, by the first constant K1;
    after writing the new first parity value to parity disk dP, store in the second buffer a second value K2Δ calculated by multiplying the first value K1Δ stored in the first buffer by the third constant K3;
    after storing the second value K2Δ in the second buffer, store in the first buffer a new second parity value calculated by XOR'ing the second value K2Δ stored in the second buffer with an old second parity value for the parity stripe retrieved from parity disk dQ; and
    write the new second parity value stored in the first buffer to parity disk dQ.
US11867407 2004-11-19 2007-10-04 Method and system for improved buffer utilization for disk array parity updates Abandoned US20080022150A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US10994086 US7290199B2 (en) 2004-11-19 2004-11-19 Method and system for improved buffer utilization for disk array parity updates
US11867407 US20080022150A1 (en) 2004-11-19 2007-10-04 Method and system for improved buffer utilization for disk array parity updates

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US11867407 US20080022150A1 (en) 2004-11-19 2007-10-04 Method and system for improved buffer utilization for disk array parity updates

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US10994086 Division US7290199B2 (en) 2004-11-19 2004-11-19 Method and system for improved buffer utilization for disk array parity updates

Publications (1)

Publication Number Publication Date
US20080022150A1 true true US20080022150A1 (en) 2008-01-24

Family

ID=36575775

Family Applications (2)

Application Number Title Priority Date Filing Date
US10994086 Expired - Fee Related US7290199B2 (en) 2004-11-19 2004-11-19 Method and system for improved buffer utilization for disk array parity updates
US11867407 Abandoned US20080022150A1 (en) 2004-11-19 2007-10-04 Method and system for improved buffer utilization for disk array parity updates

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US10994086 Expired - Fee Related US7290199B2 (en) 2004-11-19 2004-11-19 Method and system for improved buffer utilization for disk array parity updates

Country Status (2)

Country Link
US (2) US7290199B2 (en)
CN (1) CN100390746C (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080010243A1 (en) * 2006-06-02 2008-01-10 Salesforce.Com, Inc. Method and system for pushing data to a plurality of devices in an on-demand service environment
US20090054625A1 (en) * 2002-11-01 2009-02-26 Alk-Abello A/S Recombinant protein variants
US20110035549A1 (en) * 2009-08-04 2011-02-10 Samsung Electronics Co., Ltd. Data storage device

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060123271A1 (en) * 2004-11-19 2006-06-08 International Business Machines Corporation RAID environment incorporating hardware-based finite field multiplier for on-the-fly XOR
US7392458B2 (en) * 2004-11-19 2008-06-24 International Business Machines Corporation Method and system for enhanced error identification with disk array parity checking
US7343546B2 (en) * 2004-12-23 2008-03-11 Intel Corporation Method and system for syndrome generation and data recovery
WO2007012920A1 (en) * 2005-07-27 2007-02-01 Adaptec, Inc. Method and system for improving the performance of reed-solomon parity operations in redundant array of inexpensive disks
US7930586B1 (en) * 2008-02-07 2011-04-19 At&T Intellectual Property Ii, L.P. Error rate reduction for memory arrays
US8370717B1 (en) * 2008-04-08 2013-02-05 Marvell International Ltd. Method and apparatus for flexible buffers in an XOR engine
US8954668B2 (en) 2011-05-23 2015-02-10 International Business Machines Corporation Writing of data of a first block size in a raid array that stores and mirrors data in a second block size
US9043543B2 (en) * 2011-05-23 2015-05-26 International Business Machines Corporation Writing of new data of a first block size in a raid array that stores both parity and data in a second block size
US9213486B2 (en) * 2012-02-22 2015-12-15 International Business Machines Corporation Writing new data of a first block size to a second block size using a write-write mode
US20170093426A1 (en) * 2015-09-25 2017-03-30 HGST Netherlands B.V. Cache Oblivious Algorithm for Butterfly Code
US9793922B2 (en) * 2015-09-25 2017-10-17 HGST Netherlands B.V. Repair-optimal parity code

Citations (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3688265A (en) * 1971-03-18 1972-08-29 Ibm Error-free decoding for failure-tolerant memories
US5134619A (en) * 1990-04-06 1992-07-28 Sf2 Corporation Failure-tolerant mass storage system
US5140592A (en) * 1990-03-02 1992-08-18 Sf2 Corporation Disk array system
USRE34100E (en) * 1987-01-12 1992-10-13 Seagate Technology, Inc. Data error correction system
US5390187A (en) * 1990-10-23 1995-02-14 Emc Corporation On-line reconstruction of a failed redundant array system
US5448719A (en) * 1992-06-05 1995-09-05 Compaq Computer Corp. Method and apparatus for maintaining and retrieving live data in a posted write cache in case of power failure
US5488731A (en) * 1992-08-03 1996-01-30 International Business Machines Corporation Synchronization method for loosely coupled arrays of redundant disk drives
US5499253A (en) * 1994-01-05 1996-03-12 Digital Equipment Corporation System and method for calculating RAID 6 check codes
US5530948A (en) * 1993-12-30 1996-06-25 International Business Machines Corporation System and method for command queuing on raid levels 4 and 5 parity drives
US5537534A (en) * 1995-02-10 1996-07-16 Hewlett-Packard Company Disk array having redundant storage and methods for incrementally generating redundancy as data is written to the disk array
US5537567A (en) * 1994-03-14 1996-07-16 International Business Machines Corporation Parity block configuration in an array of storage devices
US5617530A (en) * 1991-01-04 1997-04-01 Emc Corporation Storage device array architecture with copyback cache
US5673412A (en) * 1990-07-13 1997-09-30 Hitachi, Ltd. Disk system and power-on sequence for the same
US5720025A (en) * 1996-01-18 1998-02-17 Hewlett-Packard Company Frequently-redundant array of independent disks
US5754563A (en) * 1995-09-11 1998-05-19 Ecc Technologies, Inc. Byte-parallel system for implementing reed-solomon error-correcting codes
US5948110A (en) * 1993-06-04 1999-09-07 Network Appliance, Inc. Method for providing parity in a raid sub-system using non-volatile memory
US5956524A (en) * 1990-04-06 1999-09-21 Micro Technology Inc. System and method for dynamic alignment of associated portions of a code word from a plurality of asynchronous sources
US6092215A (en) * 1997-09-29 2000-07-18 International Business Machines Corporation System and method for reconstructing data in a storage array system
US6101615A (en) * 1998-04-08 2000-08-08 International Business Machines Corporation Method and apparatus for improving sequential writes to RAID-6 devices
US6279050B1 (en) * 1998-12-18 2001-08-21 Emc Corporation Data transfer apparatus having upper, lower, middle state machines, with middle state machine arbitrating among lower state machine side requesters including selective assembly/disassembly requests
US6408400B2 (en) * 1997-11-04 2002-06-18 Fujitsu Limited Disk array device
US20020166078A1 (en) * 2001-03-14 2002-11-07 Oldfield Barry J. Using task description blocks to maintain information regarding operations
US6480944B2 (en) * 2000-03-22 2002-11-12 Interwoven, Inc. Method of and apparatus for recovery of in-progress changes made in a software application
US20020194427A1 (en) * 2001-06-18 2002-12-19 Ebrahim Hashemi System and method for storing data and redundancy information in independent slices of a storage device
US6567891B2 (en) * 2001-03-14 2003-05-20 Hewlett-Packard Development Company, L.P. Methods and arrangements for improved stripe-based processing
US6570839B2 (en) * 1991-05-10 2003-05-27 Discovision Associates Optical data system and optical disk relating thereto
US6687872B2 (en) * 2001-03-14 2004-02-03 Hewlett-Packard Development Company, L.P. Methods and systems of using result buffers in parity operations
US20040049632A1 (en) * 2002-09-09 2004-03-11 Chang Albert H. Memory controller interface with XOR operations on memory read to accelerate RAID operations
US6836820B1 (en) * 2002-02-25 2004-12-28 Network Appliance, Inc. Flexible disabling of disk sets
US20050108613A1 (en) * 2003-11-17 2005-05-19 Nec Corporation Disk array device, parity data generating circuit for raid and galois field multiplying circuit
US6944791B2 (en) * 2002-07-18 2005-09-13 Lsi Logic Corporation Method of handling unreadable blocks during write of a RAID device
US7028136B1 (en) * 2002-08-10 2006-04-11 Cisco Technology, Inc. Managing idle time and performing lookup operations to adapt to refresh requirements or operational rates of the particular associative memory or other devices used to implement the system
US7065609B2 (en) * 2002-08-10 2006-06-20 Cisco Technology, Inc. Performing lookup operations using associative memories optionally including selectively determining which associative memory blocks to use in identifying a result and possibly propagating error indications
US7082492B2 (en) * 2002-08-10 2006-07-25 Cisco Technology, Inc. Associative memory entries with force no-hit and priority indications of particular use in implementing policy maps in communication devices
US7206946B2 (en) * 2003-10-09 2007-04-17 Hitachi, Ltd. Disk drive system for starting destaging of unwritten cache memory data to disk drive upon detection of DC voltage level falling below predetermined value
US7426611B1 (en) * 2003-08-18 2008-09-16 Symantec Operating Corporation Method and system for improved storage system performance using cloning of cached data

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5623595A (en) 1994-09-26 1997-04-22 Oracle Corporation Method and apparatus for transparent, real time reconstruction of corrupted data in a redundant array data storage system

Patent Citations (41)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3688265A (en) * 1971-03-18 1972-08-29 Ibm Error-free decoding for failure-tolerant memories
USRE34100E (en) * 1987-01-12 1992-10-13 Seagate Technology, Inc. Data error correction system
US5140592A (en) * 1990-03-02 1992-08-18 Sf2 Corporation Disk array system
US5274645A (en) * 1990-03-02 1993-12-28 Micro Technology, Inc. Disk array system
US5134619A (en) * 1990-04-06 1992-07-28 Sf2 Corporation Failure-tolerant mass storage system
US5956524A (en) * 1990-04-06 1999-09-21 Micro Technology Inc. System and method for dynamic alignment of associated portions of a code word from a plurality of asynchronous sources
US5285451A (en) * 1990-04-06 1994-02-08 Micro Technology, Inc. Failure-tolerant mass storage system
US5673412A (en) * 1990-07-13 1997-09-30 Hitachi, Ltd. Disk system and power-on sequence for the same
US5390187A (en) * 1990-10-23 1995-02-14 Emc Corporation On-line reconstruction of a failed redundant array system
US5617530A (en) * 1991-01-04 1997-04-01 Emc Corporation Storage device array architecture with copyback cache
US5911779A (en) * 1991-01-04 1999-06-15 Emc Corporation Storage device array architecture with copyback cache
US6570839B2 (en) * 1991-05-10 2003-05-27 Discovision Associates Optical data system and optical disk relating thereto
US5448719A (en) * 1992-06-05 1995-09-05 Compaq Computer Corp. Method and apparatus for maintaining and retrieving live data in a posted write cache in case of power failure
US5488731A (en) * 1992-08-03 1996-01-30 International Business Machines Corporation Synchronization method for loosely coupled arrays of redundant disk drives
US5948110A (en) * 1993-06-04 1999-09-07 Network Appliance, Inc. Method for providing parity in a raid sub-system using non-volatile memory
US5530948A (en) * 1993-12-30 1996-06-25 International Business Machines Corporation System and method for command queuing on raid levels 4 and 5 parity drives
US5499253A (en) * 1994-01-05 1996-03-12 Digital Equipment Corporation System and method for calculating RAID 6 check codes
US5537567A (en) * 1994-03-14 1996-07-16 International Business Machines Corporation Parity block configuration in an array of storage devices
US5537534A (en) * 1995-02-10 1996-07-16 Hewlett-Packard Company Disk array having redundant storage and methods for incrementally generating redundancy as data is written to the disk array
US5754563A (en) * 1995-09-11 1998-05-19 Ecc Technologies, Inc. Byte-parallel system for implementing reed-solomon error-correcting codes
US5720025A (en) * 1996-01-18 1998-02-17 Hewlett-Packard Company Frequently-redundant array of independent disks
US6092215A (en) * 1997-09-29 2000-07-18 International Business Machines Corporation System and method for reconstructing data in a storage array system
US6408400B2 (en) * 1997-11-04 2002-06-18 Fujitsu Limited Disk array device
US6101615A (en) * 1998-04-08 2000-08-08 International Business Machines Corporation Method and apparatus for improving sequential writes to RAID-6 devices
US6279050B1 (en) * 1998-12-18 2001-08-21 Emc Corporation Data transfer apparatus having upper, lower, middle state machines, with middle state machine arbitrating among lower state machine side requesters including selective assembly/disassembly requests
US6480944B2 (en) * 2000-03-22 2002-11-12 Interwoven, Inc. Method of and apparatus for recovery of in-progress changes made in a software application
US20020166078A1 (en) * 2001-03-14 2002-11-07 Oldfield Barry J. Using task description blocks to maintain information regarding operations
US6567891B2 (en) * 2001-03-14 2003-05-20 Hewlett-Packard Development Company, L.P. Methods and arrangements for improved stripe-based processing
US6687872B2 (en) * 2001-03-14 2004-02-03 Hewlett-Packard Development Company, L.P. Methods and systems of using result buffers in parity operations
US7111227B2 (en) * 2001-03-14 2006-09-19 Hewlett-Packard Development Company, L.P. Methods and systems of using result buffers in parity operations
US20020194427A1 (en) * 2001-06-18 2002-12-19 Ebrahim Hashemi System and method for storing data and redundancy information in independent slices of a storage device
US6836820B1 (en) * 2002-02-25 2004-12-28 Network Appliance, Inc. Flexible disabling of disk sets
US6944791B2 (en) * 2002-07-18 2005-09-13 Lsi Logic Corporation Method of handling unreadable blocks during write of a RAID device
US7028136B1 (en) * 2002-08-10 2006-04-11 Cisco Technology, Inc. Managing idle time and performing lookup operations to adapt to refresh requirements or operational rates of the particular associative memory or other devices used to implement the system
US7065609B2 (en) * 2002-08-10 2006-06-20 Cisco Technology, Inc. Performing lookup operations using associative memories optionally including selectively determining which associative memory blocks to use in identifying a result and possibly propagating error indications
US7082492B2 (en) * 2002-08-10 2006-07-25 Cisco Technology, Inc. Associative memory entries with force no-hit and priority indications of particular use in implementing policy maps in communication devices
US6918007B2 (en) * 2002-09-09 2005-07-12 Hewlett-Packard Development Company, L.P. Memory controller interface with XOR operations on memory read to accelerate RAID operations
US20040049632A1 (en) * 2002-09-09 2004-03-11 Chang Albert H. Memory controller interface with XOR operations on memory read to accelerate RAID operations
US7426611B1 (en) * 2003-08-18 2008-09-16 Symantec Operating Corporation Method and system for improved storage system performance using cloning of cached data
US7206946B2 (en) * 2003-10-09 2007-04-17 Hitachi, Ltd. Disk drive system for starting destaging of unwritten cache memory data to disk drive upon detection of DC voltage level falling below predetermined value
US20050108613A1 (en) * 2003-11-17 2005-05-19 Nec Corporation Disk array device, parity data generating circuit for raid and galois field multiplying circuit

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090054625A1 (en) * 2002-11-01 2009-02-26 Alk-Abello A/S Recombinant protein variants
US20080010243A1 (en) * 2006-06-02 2008-01-10 Salesforce.Com, Inc. Method and system for pushing data to a plurality of devices in an on-demand service environment
US9201939B2 (en) * 2006-06-02 2015-12-01 Salesforce.Com, Inc. Method and system for pushing data to a plurality of devices in an on-demand service environment
US20110035549A1 (en) * 2009-08-04 2011-02-10 Samsung Electronics Co., Ltd. Data storage device
US8321631B2 (en) * 2009-08-04 2012-11-27 Samsung Electronics Co., Ltd. Parity calculation and journal generation in a storage device with multiple storage media

Also Published As

Publication number Publication date Type
CN100390746C (en) 2008-05-28 grant
US7290199B2 (en) 2007-10-30 grant
US20060123268A1 (en) 2006-06-08 application
CN1776636A (en) 2006-05-24 application

Similar Documents

Publication Publication Date Title
US5487160A (en) Concurrent image backup for disk storage system
US8775868B2 (en) Adaptive RAID for an SSD environment
US6067635A (en) Preservation of data integrity in a raid storage device
US5479611A (en) Disk array apparatus
US7890795B1 (en) Auto-adapting cache memory system and memory
US6289471B1 (en) Storage device array architecture with solid-state redundancy unit
US6041423A (en) Method and apparatus for using undo/redo logging to perform asynchronous updates of parity and data pages in a redundant array data storage environment
US7024586B2 (en) Using file system information in raid data reconstruction and migration
US5581690A (en) Method and apparatus for preventing the use of corrupt data in a multiple disk raid organized storage system
US7681104B1 (en) Method for erasure coding data across a plurality of data stores in a network
US5488701A (en) In log sparing for log structured arrays
US5289478A (en) Method and means for verification of write data
US6883131B2 (en) XOR processing incorporating error correction code data protection
US5533190A (en) Method for maintaining parity-data consistency in a disk array
US6981171B2 (en) Data storage array employing block verification information to invoke initialization procedures
US7577866B1 (en) Techniques for fault tolerant data storage
US6775791B2 (en) Replaceable memory modules with parity-based data recovery
US5208813A (en) On-line reconstruction of a failed redundant array system
US6161192A (en) Raid array data storage system with storage device consistency bits and raidset consistency bits
US5303244A (en) Fault tolerant disk drive matrix
US6334168B1 (en) Method and system for updating data in a data storage system
US20120079190A1 (en) Offset protection data in a raid array
US20090228648A1 (en) High performance disk array rebuild
US20100115335A1 (en) Simulator For Determining Data Loss In A Fault Tolerant System
Baek et al. Reliability and performance of hierarchical RAID with multiple controllers