RU2835373C9 - Method of arranging data in raid arrays for balanced load distribution during array recovery - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: invention relates to a method of arranging data in a RAID array for balanced load distribution during array recovery. Method comprises steps of creating a new RAID array using a procedure for generating a stripe map; number of free disks N, length of current stripe L and the length of the stripe arrangement map R is passed at the input of the procedure for generate a stripe map and based on the obtained data, a stripe map consisting of R concatenated permutations of the set {1, …, N} is formed; matrix of combinations M of size N×N is initialized, during which all its elements are assigned values of 0, the current stripe is initialized with an empty list; permutation is generated in the stripe map by calling the generate permutation procedure with the following input parameters: current value of combination matrix M, list of occupied disks in current stripe, length of current stripe L, number of disks in RAID array N; to generate one permutation based on input parameters, auxiliary structures are initialized, namely: list of free disks is initialized with numbers from 1 to N, list of disks in current permutation is initialized with empty list; iterative selection of a disk which is contained in the list of free disks, but is not contained in the list of occupied disks in the current stripe, using the search for the minimum sum of the elements of the combination matrix corresponding to the disk, and occupied disks in the current stripe; adding a disc to a list of occupied discs in the current stripe and to the end of the list of discs in the current permutation; once the length of the list of occupied disks in the current stripe has reached the length of the current stripe L, updating the combination matrix and assigning the list of occupied disks in the current stripe to an empty list value; procedure for generation of a permutation in a stripe arrangement map is repeated until the number of permutations reaches R.

EFFECT: faster recovery of a RAID array due to a data arrangement scheme which ensures uniform distribution of the read load across all disks during recovery of the array.

1 cl, 9 dwg

Description

AREA OF TECHNOLOGY

The claimed technical solution generally relates to the field of computing technology, and in particular to a method for placing data in RAID arrays for balanced load distribution during array recovery.

LEVEL OF TECHNOLOGY

One of the most important characteristics of data storage systems is the availability of stored data. In other words, the ability to continuously work with data over a long period of time. When using standard storage devices such as hard drives (HDDs) or solid-state drives (SSDs), there is a high risk of device failure and loss of access to data. One universal way to increase the availability of data stored on devices is to combine these devices into a RAID array with data redundancy. Data redundancy in a RAID array can be achieved by storing full copies of data (RAID-1, RAID-10) or storing checksums for certain data fragments (RAID-5, RAID-6, RAID-50, RAID-60). Let's take a closer look at the second option.

RAID arrays using parity sums divide all stored data into consecutive fragments called stripes. A stripe consists of data stripes of equal size. To provide redundancy, parity sum stripes are added to the data stripes (1 for RAID-5, 2 for RAID-6). Each stripe is stored on a single physical device, and no device is included in the same stripe twice. The more disks in a RAID array, the greater the risk of multiple disk failures. Therefore, the number of disks with parity sums is increased. To scale RAID-6 to a large number of disks, RAID-60 technology is used, in which the disks are divided into several groups. Each group forms a similar RAID-6, but the data is interleaved between the groups. That is, if there are two groups of disks, the odd stripes will be on the first group of disks, and the even stripes on the second.

RAID array recovery with checksums is stripe-by-strip. If for some reason a stripe containing data cannot be read, all information from other stripes is read instead, including those containing checksums. Using all the information stored within a stripe and the checksum recovery algorithm, data can be accurately recovered as long as the number of damaged (missing) stripes does not exceed the number of stripes containing checksums.

When installing a new drive to replace a damaged one, a similar procedure is performed, but the data is not only calculated but also written to the new drive. This process is called RAID rebuilding. When using RAID 60, only the drives in the same group as the failed drive are read to rebuild the drives; drives in other groups are not used, which can lead to prolonged drive overload and further failures.

One solution to this problem is the alternative data storage methods proposed in the stated solution.

The described RAID array logic can be implemented in two ways.

The first method is to use a hardware RAID controller to manage the array. In this case, configuration occurs before the operating system boots, and the underlying storage devices are invisible to the operating system.

The second method is to write a device driver in the operating system kernel; such devices are called software-defined or virtual. In this case, the disks are visible to the operating system kernel, and after the driver loads, a new device is created that operates on the assigned storage devices, forwarding requests to the underlying devices according to the specified logic.

The prior art includes patent US9841908B1 "Declustered array of storage devices with chunk groups and support for multiple erasure schemes" by Western Digital Technologies Inc., published on December 12, 2017. This solution describes a method for generating balanced incomplete block designs (BIBD), a method for generating partial balanced incomplete block designs (PBIBD), and a method for using the generated block designs to create a chunk group mapping table (chunk group mapping table in the patent terminology). BIBD generation is only possible for configurations described by the formula N= ^k2 , where N is the number of disks in the RAID array and k is the number of disks in a stripe. The generation uses a randomly generated permutation and successive rotational operations. PBIBD generation is used only for those configurations for which BIBD cannot be generated. The algorithm uses sequential pseudo-random generation of permutations with estimation of block design parameters at each step.

The disadvantages of the described method are:

1. The applicability of the basic generation algorithm is only for a part of the possible configurations.

2. Using pseudo-random generation requires storing additional information on disks.

3. The running time and output length of the PBIBD generation algorithm are unpredictable and depend on how successfully the permutation is generated at each iteration.

4. The block design parameters estimated in the patent, such as the number of blocks containing any point and the number of blocks containing any two points, evaluate the generated block design only globally, which may lead to locally overloaded elements.

Also known in the prior art is patent EP2921960A2, "Method of, and apparatus for, accelerated data recovery in a storage system," owned by Seagate Systems UK Ltd. This patent describes a data layout method based on two parameters: width and number of repetitions, allowing for optimized RAID array recovery by using read-ahead from base devices. The approach consists of three main steps. In the first step, the width and number of repetitions are determined. Next, a matrix is formed based on the selected parameters, the number of disks, and the number of disks per stripe. In the final step, the matrix columns are shuffled using a randomly generated permutation.

The main disadvantages of this approach are:

1. When optimizing using read-ahead, some disks become overloaded with requests, which can slow down the processing of user requests and lead to an increased likelihood of failure of the disks from which recovery is performed.

2. Using pseudo-random permutation at the end of generation requires storing additional information on disks.

Unfortunately, hard drives, which are the primary data storage device today, aren't as reliable as we'd like. Protecting your files to avoid data recovery is a pressing issue.

ESSENCE OF THE INVENTION

The disadvantages of the prior art are overcome and advantages are achieved by providing a computer-implemented method for placing data in RAID arrays for balanced load distribution during array rebuild.

The technical result achieved by solving this problem is to ensure a high level of data availability, data storage reliability, and an increase in the speed of RAID array recovery due to a data arrangement scheme that ensures uniform distribution of the read load across all disks during array recovery.

Furthermore, the developed data layout enabled us to achieve a maximum imbalance ratio of no more than 1.5 across all possible configurations (up to 1024 disks). By forming stripes using a combination matrix that stores data on how many times each pair of disks occurred in a single stripe, we achieve a balanced stripe construction. For example, during data recovery, reads are made from those disks that are in the same stripe as the disk being recovered.

Additional effects include RAM consumption of no more than 2 MB (in the worst case, 1024 disks) per RAID array. While the RAID array is operating, only the stripe map is stored in RAM. With a fixed stripe map length of 1024, it stores a number of elements equal to 1024 times the number of disks. For the maximum number of disks, this is 1024 * 1024 elements. The element size is 16 bits, since this is enough to store numbers from 1 to 1024. Thus, the maximum memory consumption is 1024 * 1024 * 16 = 16,777,216 bits = 2 MB.

The specified technical result is achieved by implementing a method for placing data in RAID arrays for balanced load distribution during array recovery, comprising the following stages:

- create a new RAID array using the generate stripe map procedure;

- the number of free disks N, the length of the current stripe L and the length of the stripe placement map R are passed to the input of the stripe placement map generation procedure and, based on the received data, a stripe placement map is formed consisting of R concatenated permutations of the set {1, …, N};

- initialize the matrix of combinations M of size NxN, during which all its elements are assigned the value 0, the current stripe is initialized with an empty list;

- generate a permutation in the stripe allocation map by calling the generate permutation procedure with the following input parameters: the current value of the combination matrix M, the list of occupied disks in the current stripe, the length of the current stripe L, the number of disks in the RAID array N;

- to generate one permutation based on the input parameters, the auxiliary structures are initialized, namely: the list of free disks is initialized with numbers from 1 to N, the list of disks in the current permutation is initialized with an empty list;

- iteratively select a disk that is contained in the list of free disks, but is not contained in the list of occupied disks in the current stripe, using the search for the minimum sum of the elements of the matrix of combinations corresponding to the disk and the occupied disks in the current stripe;

- add the disk to the list of occupied disks in the current stripe and to the end of the list of disks in the current permutation;

- as soon as the length of the list of occupied disks in the current stripe has reached the length of the current stripe L, the combination matrix is updated and the list of occupied disks in the current stripe is assigned the value of the empty list;

- repeat the procedure of generating a permutation in the stripe placement map until the number of permutations reaches R.

BRIEF DESCRIPTION OF DRAWINGS

The features and advantages of the present technical solution will become apparent from the detailed description below and the attached drawings.

Fig. 1 - illustrates a block diagram of the implementation of the claimed method;

Fig. 2 - illustrates the states of the combination matrix after generating three permutations in the stripe placement map;

Fig. 3 - illustrates the requirements for the stripe placement map;

Fig. 4 - illustrates an example of iteration in permutation generation;

Fig. 5 - illustrates the use of a stripe placement map to calculate the physical placement of stripes;

Fig. 6 - illustrates a block diagram of the permutation generation procedure;

Fig. 7 - illustrates a flow chart of the procedure for searching for a candidate for addition to the stripe;

Fig. 8 - illustrates a flow chart of the procedure for updating the combination matrix;

Fig. 9 - illustrates a general example of a computing device.

DETAILED DESCRIPTION OF THE INVENTION

The following detailed description of the invention includes numerous implementation details intended to provide a clear understanding of the present invention. However, one skilled in the art will readily appreciate how the present invention may be utilized with or without these implementation details. In other instances, well-known methods, procedures, and components have not been described in detail to avoid unnecessarily obscuring the features of the present invention.

Furthermore, it will be clear from the foregoing description that the invention is not limited to the embodiment described. Numerous possible modifications, changes, variations, and substitutions, while preserving the spirit and form of the present invention, will be apparent to those skilled in the art.

Below we will describe the concepts and terms necessary for understanding this technical solution.

RAID or RAID array (English: Redundant Array of Independent Disks) is a set of several block non-volatile storage devices, such as disks (SSD, HDD), combined into a single logical block device in such a way that the failure of one or more block devices in the RAID does not cause the failure of the array itself and does not lead to data loss.

Strip - a sequential section of a basic data storage device (RAID disk) of a fixed size, for example 16 kilobytes.

A stripe is a set of stripes located on different underlying storage devices (RAID array disks) and together forming a sequential section of a virtual storage device (RAID array). Each stripe contains a set of data and, optionally, checksums calculated from the stripe's data set. If checksums are stored, separate stripes are allocated for them (based on the number of different checksums). Stripe depth is the size of a single stripe within a stripe. Stripe width is the amount of data contained in each stripe.

So, if the stripe depth is 64 KB, then we can calculate the stripe width by multiplying this value by the number of stripes with data in the stripe.

The imbalance ratio is a metric used to evaluate the efficiency of load distribution across disks during RAID array rebuild. It is calculated for a fixed number of failed disks as the ratio of the number of I/O operations performed on the most and least loaded disks during RAID array rebuild. The minimum, average, and maximum imbalance ratio values among all possible disk failures are used to evaluate the RAID array as a whole.

A stripe map is a structure that stores information about which base device (RAID disk) each stripe of each stripe in the RAID array is located on. To conserve resources, stripe maps are used whose length is significantly shorter than the number of stripes in the RAID array. Access to the stripe map is performed modulo the division by the stripe map length.

A combination matrix is a square table of numbers with sides equal to the number of disks in the RAID array. The number in the i-th column and j-th row of the table indicates how many times disk i and disk j appear in the same stripe, according to the stripe map.

This technical solution can be implemented on a computer, in the form of an automated information system (AIS), a distributed computer system, or a machine-readable medium containing instructions for performing the above-mentioned method of placing data in RAID arrays for balanced load distribution during array recovery using computing means (for example, a processor).

Fig. 1 shows a block diagram of the implementation of the claimed method (100) of placing data in RAID arrays for balanced load distribution during array recovery.

In the first step (101), a new RAID array is created using the generate stripe map procedure.

At step (102), the number of free disks N, the length of the current stripe L and the length of the stripe placement map R are passed to the input of the stripe placement map generation procedure, and based on the data received, a stripe placement map is formed, consisting of R concatenated permutations of the set {1, …, N}.

The stripe placement map is an array of length k*N, where N is the number of disks and k is a fixed coefficient, in our case equal to 1024.

The following requirements apply to the stripe placement map:

The stripe placement map can be thought of as a concatenation of k permutations of the set {1,...,N}, which guarantees uniform disk usage since each disk n occurs exactly k times.

The stripe placement map can be thought of as a concatenation of (k*N)/M permutations of the set {1,...,M}, where M is the stripe length, which guarantees a well-formed (without repetitions) stripe.

Figure 3 shows an example of a requirement for a stripe allocation map, where N is the number of disks, M is the stripe length, d _i is the disk number from the set {1, …, N}, and i is the index in the stripe allocation map from the set {1, …, k*N}.

At step (103), the matrix of combinations M of size NxN is initialized, where N is the number of disks, during which all its elements are assigned the value 0, the current stripe is initialized with an empty list.

Initializing a permutation matrix M means assigning all its elements the value 0. The current stripe is initialized to an empty list. The current stripe is the name of an auxiliary structure of the list type. The elements of the list are disk numbers. At any given moment, the list may contain from 0 to L (the stripe length parameter passed as input) elements. The current stripe list is populated during permutation generation. Once its length reaches L, the list is reset. It is not stored in itself, but serves as an auxiliary structure for permutation generation.

For example, the element of the combination matrix M[i][j] corresponds to the number of times disk i and disk j occur in the same stripe. Since the relationship of occurrence in the same stripe is symmetric, the combination matrix M is symmetric with respect to its main diagonal. To optimize memory usage, only the upper triangular part of the combination matrix M can be stored. For simplicity, the description of the algorithm uses the full version of the combination matrix M, in which M[i][j]=M[j][i] for all i, j from the set {1,...,N}.

The value of the imbalance coefficient linearly depends on the ratio of the minimum and maximum (excluding elements on the main diagonal) elements in the combination matrix M.

The data placement method presented above creates a stripe map by trying to balance the elements of the combination matrix: it directly uses the current data about the balance of disk combinations in stripes and tries to find the best combination based on this.

The combination matrix and stripe allocation map are located in RAM. The combination matrix is a temporary object, allocated memory only while the stripe allocation map is being filled. The stripe allocation map, on the other hand, is persistent and permanently resides in RAM. It determines which disk a given block of information is located on.

At step (104), a permutation is generated in the stripe placement map (Fig. 6) by calling the generate permutation procedure with the following input parameters: the current value of the combination matrix M, the list of occupied disks in the current stripe, the length of the current stripe L, the number of disks in the RAID array N.

The stripe allocation map consists of K concatenated permutations. A permutation is an arbitrary ordered set of all elements of the set of disks without repetitions. For example, the permutations of the set {1,2,3} are 1, 2, 3; 3, 1, 2, etc.

This generation method was chosen because using permutation concatenation ensures that all disks appear in the stripe allocation map with equal frequency. This, in turn, ensures uniform space utilization across all disks.

Passing input parameters allows us to generate a permutation that, taking into account previously generated permutations, will yield a good imbalance coefficient for the disks, which is also the goal of this solution. The most important factors here are the combination matrix M and the list of disks in the current stripe; these are used to select a new disk. This new disk is then added to the stripe and to the permutation.

At step (105), to generate one permutation based on the input parameters, the auxiliary structures are initialized, namely: the list of free disks is initialized with numbers from 1 to N, the list of disks in the current permutation is initialized with an empty list.

At step (106), a disk that is on the free disk list but not on the occupied disk list in the current stripe is iteratively selected. This search involves finding the minimum sum of the elements of the combination matrix corresponding to the disk and the occupied disks in the current stripe. The minimum sum of the combination matrix elements is calculated during the permutation generation step. At each step of permutation generation, one disk is added. The disk is selected based on the minimum sum in the combination matrix. The elements to be summed are selected according to the following logic. The row number is always the number of the candidate disk. The column number changes in a loop that iterates through all the elements of the current stripe's array.

The disk selection procedure (Fig. 7) receives as input a combination matrix M, a list of disks in the current stripe S, and a non-empty list of available disks A. First, we initialize the auxiliary variables: the number of the disk with the smallest local sum, and the first element of the list of available disks is assigned to it. The minimum of local sums is assigned the maximum value of the INT type, and it is assigned the minimum of local sums. Next, we iterate over all elements of list A. At the beginning of each iteration, we assign a local sum value equal to zero. We iterate over all elements of the list of disks in the current stripe S and add to the local sum the value of the combination matrix M, located in the cell (the value of the current element of the non-empty list A, the value of the current element of the non-empty list S). After iterating over list S, we obtain the value of the local sum. If it is less than the value of the current minimum of local sums, then we store the disk number. The procedure returns the disk from list A with the smallest local sum value.

At step (107), the disk is added to the list of occupied disks in the current stripe and to the end of the list of disks in the current permutation.

At step (108), as soon as the length of the list of occupied disks in the current stripe has reached the length of the current stripe L, the combination matrix is updated and the list of occupied disks in the current stripe is assigned the value of the empty list and the procedure for generating a permutation in the stripe allocation map is repeated until the number of permutations has reached R.

The combination matrix update procedure (Fig. 8) receives the combination matrix and the current stripe as input. Iteratively iterates through all possible values of the tuple (d1, d2), where d1 and d2 are the disk numbers from the list S, and increments M[d1][d2] by 1. After processing all tuples, the procedure returns the updated combination matrix.

After all the above stages of the cycle are completed, the stripe placement map is returned, which will continue to be stored in RAM for the entire duration of the RAID array operation.

As follows from the above, the declared solution ensures a high level of data storage reliability and increases the speed of RAID array recovery due to the data arrangement scheme, which ensures uniform distribution of the read load across all disks during data recovery.

Fig. 9 shows a general example of a computing device (900), which may be, for example, a computer, a server, a laptop, a smartphone, a SoC (System-on-a-Chip), etc. The device (900) may be used for the full or partial implementation of the claimed method (100).

In general, the device (900) contains components such as: one or more processors (901), at least one random access memory (902), a permanent data storage means (903), input/output interfaces (904) including relay outputs for connection to belt conveyor motion controllers, an I/O means (905), and network interaction means (906).

The processor (901) of the device performs the basic computing operations necessary for the operation of the device (900) or the functionality of one or more of its components. The processor (901) executes the necessary machine-readable instructions contained in the RAM (902).

Memory (902) is typically implemented as RAM and contains the necessary software logic to provide the required functionality. Data storage (903) can be implemented as HDD, SSD, RAID array, network storage, flash memory, optical storage (CD, DVD, MD, Blue-Ray discs), etc. Data storage (903) enables long-term storage of various types of information, such as magnetic tape recordings, request processing history (logs), user IDs, camera data, images, etc.

Interfaces (904) are standard means for connecting and working with computing devices. Interfaces (904) may be, for example, relay connections, USB, RS232/422/485 or others, RJ45, LPT, UART, COM, HDMI, PS/2, Lightning, Fire Wire, etc. for work, including, via Modbus protocols and Probfibus, Profinet networks or other types of networks. The choice of interfaces (904) depends on the specific design of the device (900), which may be a computing unit (computing module), for example, based on a CPU (one or more processors), a microcontroller, etc., a personal computer, a mainframe, a server cluster, a thin client, a smartphone, a laptop, etc., as well as connected third-party devices.

The following can be used as I/O data means (905): keyboard, joystick, display (touch display), projector, touchpad, mouse, trackball, light pen, speakers, microphone, etc.

The network interaction means (906) are selected from a device that provides network reception and transmission of data, for example, an Ethernet card, a WLAN/Wi-Fi module, a Bluetooth module, a BLE module, an NFC module, an IrDa module, an RFID module, a GSM modem, etc. With the help of the means (906), the organization of data exchange is ensured via a wired or wireless data transmission channel, for example, a WAN, PAN, LAN, Intranet, Internet, WLAN, WMAN or GSM, a quantum (fiber optic) data transmission channel, satellite communication, etc. The components of the device (900), as a rule, are connected via a common data transmission bus.

A program is a sequence of instructions intended for execution by a computer control unit or command processing device.

These application materials present a preferred disclosure of the implementation of the claimed technical solution, which should not be used as limiting other, particular embodiments of its implementation that do not go beyond the scope of the requested scope of legal protection and are obvious to specialists in the relevant field of technology.

Claims (10)

A computer-implemented method for placing data in RAID arrays for balanced load distribution during array recovery, comprising the steps of: - create a new RAID array using the generate stripe map procedure; - the number of free disks N, the length of the current stripe L and the length of the stripe placement map R are passed to the input of the stripe placement map generation procedure and, based on the received data, a stripe placement map is formed consisting of R concatenated permutations of the set {1, …, N}; - initialize the matrix of combinations M of size N×N, during which all its elements are assigned the value 0, the current stripe is initialized with an empty list; - generate a permutation in the stripe allocation map by calling the generate permutation procedure with the following input parameters: the current value of the combination matrix M, the list of occupied disks in the current stripe, the length of the current stripe L, the number of disks in the RAID array N; - to generate one permutation based on the input parameters, the auxiliary structures are initialized, namely: the list of free disks is initialized with numbers from 1 to N, the list of disks in the current permutation is initialized with an empty list; - iteratively select a disk that is contained in the list of free disks, but is not contained in the list of occupied disks in the current stripe, using the search for the minimum sum of the elements of the matrix of combinations corresponding to the disk and the occupied disks in the current stripe; - add the disk to the list of occupied disks in the current stripe and to the end of the list of disks in the current permutation; - as soon as the length of the list of occupied disks in the current stripe has reached the length of the current stripe L, the combination matrix is updated and the list of occupied disks in the current stripe is assigned the value of the empty list; - repeat the procedure of generating a permutation in the stripe placement map until the number of permutations reaches R.