RU2835373C1 - Method of arranging data in raid arrays for balanced load distribution during array recovery - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: invention relates to a method of arranging data in a RAID array for balanced load distribution during array recovery. Method comprises steps of creating a new RAID array using a procedure for generating a stripe map; number of free disks N, length of current stripe L and the length of the stripe arrangement map R is passed at the input of the procedure for generate a stripe map and based on the obtained data, a stripe map consisting of R concatenated permutations of the set {1, …, N} is formed; matrix of combinations M of size N×N is initialized, during which all its elements are assigned values of 0, the current stripe is initialized with an empty list; permutation is generated in the stripe map by calling the generate permutation procedure with the following input parameters: current value of combination matrix M, list of occupied disks in current stripe, length of current stripe L, number of disks in RAID array N; to generate one permutation based on input parameters, auxiliary structures are initialized, namely: list of free disks is initialized with numbers from 1 to N, list of disks in current permutation is initialized with empty list; iterative selection of a disk which is contained in the list of free disks, but is not contained in the list of occupied disks in the current stripe, using the search for the minimum sum of the elements of the combination matrix corresponding to the disk, and occupied disks in the current stripe; adding a disc to a list of occupied discs in the current stripe and to the end of the list of discs in the current permutation; once the length of the list of occupied disks in the current stripe has reached the length of the current stripe L, updating the combination matrix and assigning the list of occupied disks in the current stripe to an empty list value; procedure for generation of a permutation in a stripe arrangement map is repeated until the number of permutations reaches R.

EFFECT: faster recovery of a RAID array due to a data arrangement scheme which ensures uniform distribution of the read load across all disks during recovery of the array.

1 cl, 9 dwg

Description

AREA OF TECHNOLOGY

The claimed technical solution generally relates to the field of computing technology, and in particular to a method of placing data in RAID arrays for balanced load distribution during array recovery.

LEVEL OF TECHNOLOGY

One of the most important characteristics of data storage systems is the availability of stored data. In other words, the ability to continuously work with data for a long time. When using standard data storage devices such as hard disk drives (HDD) or solid-state drives (SSD), there is a high probability of the device breaking down and losing access to data. One of the universal ways to increase the availability of data stored on devices is to combine these devices into a RAID array with data storage redundancy. Data redundancy in a RAID array can be provided by storing full copies of data (RAID-1, RAID-10) or storing checksums for some data fragments (RAID-5, RAID-6, RAID-50, RAID-60). Let's consider the second option in more detail.

RAID arrays using checksums divide all stored data into consecutive fragments called stripes. A stripe consists of data stripes of equal size. To ensure redundancy, stripes of checksums are also added to the data stripes (1 for RAID-5, 2 for RAID-6). Each stripe is stored on a physical device, and no device is included in the same stripe twice. The more disks in a RAID array, the greater the likelihood of several disks failing at the same time, so the number of disks with checksums is increased. To scale RAID-6 to a large number of disks, RAID-60 technology is used, in which the disks are divided into several groups. Each group forms a semblance of RAID-6, but the data is alternated between the groups. That is, if there are 2 groups of disks, then the odd stripes will be on the first group of disks, and the even ones on the second.

Recovery of a RAID array with checksums is performed stripe by stripe. If for some reason it is impossible to read a strip with data, then all information from other stripes is read instead, including stripes with checksums. Using all the information stored inside the stripe and the algorithm for recovery from checksums, it is possible to accurately recover data as long as the number of damaged (missing) stripes does not exceed the number of strips with checksums.

When installing a new disk to replace a damaged one, a similar procedure is performed, but the data is not only calculated, but also written to the new disk. This process is called the RAID array recovery process. In the case of using RAID-60 to restore disks, only those disks that were in the same group as the broken disk are read, disks of other groups are not used, which can lead to long-term disk overload and new failures.

One solution to this problem is the alternative methods of data placement proposed in the stated solution.

The described RAID array logic can be implemented in two ways.

The first method is to use a hardware RAID controller to manage the array. In this case, the configuration is done before the operating system is loaded, and the operating system does not see the underlying storage devices.

The second method is to write a device driver in the operating system kernel; such devices are called software-defined or virtual. In this case, the disks are visible to the operating system kernel, and after loading the driver, a new device is created that works with the data storage devices transferred to it, implementing the redirection of requests to the base devices according to the specified logic.

The prior art includes patent US9841908B1 "Declustered array of storage devices with chunk groups and support for multiple erasure schemes", patent holder Western Digital Technologies Inc, published on 12/12/2017. This solution describes a method for generating balanced incomplete block designs (BIBD), a method for generating partial balanced incomplete block designs (PBIBD), and a method for using the generated block designs to create a chunk group mapping table (in the patent terminology). BIBD generation is only possible for configurations described by the formula N=k ² , where N is the number of disks in the RAID array, k is the number of disks in the stripe. During the generation, a randomly generated permutation and successful rotational operations are used. PBIBD generation is used only for those configurations for which BIBD cannot be generated. The algorithm uses sequential pseudo-random generation of permutations with estimation of block design parameters at each step.

The disadvantages of the described method are:

1. The applicability of the basic generation algorithm is only for a part of possible configurations.

2. Using pseudo-random generations requires storing additional information on disks.

3. The running time and output length of the PBIBD generation algorithm are unpredictable and depend on how successfully the permutation is generated at each iteration.

4. The block design parameters estimated in the patent, such as the number of blocks containing any point and the number of blocks containing any two points, evaluate the generated block design only globally, which may lead to locally overloaded elements.

In addition, the prior art includes patent EP2921960A2 "Method of, and apparatus for, accelerated data recovery in a storage system", patent holder Seagate Systems UK Ltd, published on 23.12.2015. This solution describes a data layout method based on two parameters: width and number of repetitions, allowing to optimize RAID array recovery by using anticipatory reading from base devices. The approach consists of three main steps. In the first step, the parameters of the width and number of repetitions are determined. Then, a matrix is formed according to the selected parameters, the number of disks and the number of disks in the stripe. In the last step, the matrix columns are shuffled using a randomly generated permutation.

The main disadvantages of this approach are:

1. When optimizing using read-ahead, some disks become overloaded with requests, which can slow down the processing of user requests and lead to an increased probability of failure of the disks from which recovery is performed.

2. Using pseudo-random permutation at the end of generation requires storing additional information on disks.

Unfortunately, hard drives, which are the main data storage today, are not as reliable as we would like. And the problem of protecting your files so that you do not have to resort to data recovery is quite acute.

ESSENCE OF THE INVENTION

The disadvantages of the prior art are overcome and advantages are achieved by providing a computer-implementable method for placing data in RAID arrays for balanced load distribution during array rebuild.

The technical result achieved by solving this problem is to ensure a high level of data availability, data storage reliability and increase the speed of RAID array recovery due to the data arrangement scheme, which ensures uniform distribution of the reading load across all disks during array recovery.

In addition, the developed data layout scheme made it possible to achieve a maximum imbalance coefficient of no more than 1.5 among all permissible configurations (up to 1024 disks). By forming stripes using a combination matrix that stores data on how many times each pair of disks was found in one stripe, balanced stripe construction is achieved. For example, during data recovery, reading occurs from those disks that are included in the stripe with the disk that is being recovered.

Additional effects include the consumption of RAM in volumes of no more than 2 MB (in the worst case, 1024 disks) per RAID array. During RAID operation, only the stripe allocation map is stored in RAM. With a fixed stripe allocation map length of 1024, it stores a number of elements equal to 1024 multiplied by the number of disks. For the maximum number of disks, this is 1024 * 1024 elements. The element size is 16 bits, since this is enough to store numbers from 1 to 1024. Thus, the maximum memory consumption is `1024 * 1024 * 16 = 16,777,216 bits = 2 MB.

The specified technical result is achieved by implementing a method for placing data in RAID arrays for balanced load distribution during array recovery, containing stages in which:

- create a new RAID array using the generate stripe map procedure;

- the number of free disks N, the length of the current stripe L and the length of the stripe placement map R are passed to the input of the stripe placement map generation procedure and, based on the data received, a stripe placement map is formed consisting of R concatenated permutations of the set {1, …, N};

- initialize the matrix of combinations M of size NxN, during which all its elements are assigned the value 0, the current stripe is initialized with an empty list;

- generate a permutation in the stripe placement map by calling the generate permutation procedure with the following input parameters: the current value of the combination matrix M, the list of occupied disks in the current stripe, the length of the current stripe L, the number of disks in the RAID array N;

- to generate one permutation based on the input parameters, the auxiliary structures are initialized, namely: the list of free disks is initialized with numbers from 1 to N, the list of disks in the current permutation is initialized with an empty list;

- iteratively select a disk that is contained in the list of free disks, but is not contained in the list of occupied disks in the current stripe, using the search for the minimum sum of the elements of the matrix of combinations corresponding to the disk and the occupied disks in the current stripe;

- add a disk to the list of occupied disks in the current stripe and to the end of the list of disks in the current permutation;

- as soon as the length of the list of occupied disks in the current stripe reaches the length of the current stripe L, the combination matrix is updated and the list of occupied disks in the current stripe is assigned the value of the empty list;

- repeat the procedure of generating a permutation in the stripe placement map until the number of permutations reaches R.

BRIEF DESCRIPTION OF DRAWINGS

The features and advantages of the present technical solution will become apparent from the following detailed description and the attached drawings.

Fig. 1 - illustrates a block diagram of the implementation of the claimed method;

Fig. 2 - illustrates the states of the combination matrix after generating three permutations in the stripe placement map;

Fig. 3 - illustrates the requirements for the stripe placement map;

Fig. 4 - illustrates an example of iteration in permutation generation;

Fig. 5 - illustrates the use of a stripe placement map to calculate the physical placement of stripes;

Fig. 6 - illustrates a block diagram of the permutation generation procedure;

Fig. 7 - illustrates a flow chart of the procedure for searching for a candidate for addition to a stripe;

Fig. 8 - illustrates a block diagram of the procedure for updating the combination matrix;

Fig. 9 - illustrates a general example of a computing device.

DETAILED DESCRIPTION OF THE INVENTION

In the following detailed description of the embodiment of the invention, numerous implementation details are set forth in order to provide a clear understanding of the present invention. However, it will be apparent to one skilled in the art how the present invention may be used with or without these implementation details. In other instances, well-known methods, procedures, and components have not been described in detail in order not to unnecessarily obscure the features of the present invention.

In addition, it will be clear from the above description that the invention is not limited to the embodiment shown. Numerous possible modifications, changes, variations and substitutions, preserving the essence and form of the present invention, will be obvious to those skilled in the art.

Below we will describe the concepts and terms necessary for understanding this technical solution.

RAID or RAID array (English: Redundant Array of Independent Disks - redundant array of independent (independent) disks) is a set of several block non-volatile storage devices, such as disks (SSD, HDD), combined into a single logical block device in such a way that the failure of one or more block devices in the RAID does not cause the failure of the array itself, and does not lead to data loss.

Strip - a sequential section of a basic data storage device (RAID disk array) of a fixed size, for example 16 kilobytes.

Stripe - a set of stripes located on different basic storage devices (RAID array disks) and together forming a sequential section of a virtual storage device (RAID array). Each stripe contains a data set and, optionally, checksums calculated from the stripe data set. In the case of storing checksums, separate stripes are allocated for them (according to the number of different checksums). Stripe depth is the size of one strip included in the stripe. Stripe width is the volume of data contained in each stripe.

So if the stripe depth is 64 KB, then we can calculate the stripe width by multiplying this value by the number of strips with data in the stripe.

The imbalance ratio is a metric used to evaluate the efficiency of load distribution across disks during RAID array recovery. It is calculated for fixed numbers of failed disks as the ratio of the number of I/O operations for the most and least loaded disks during RAID array recovery. The minimum, average, and maximum imbalance ratio values among all possible disk failures are used to evaluate the RAID array as a whole.

Stripe map - a structure that stores information about which base device (RAID array disk) each stripe of each stripe of the RAID array is located on. To save resources, maps are used whose length is significantly less than the number of stripes in the RAID array, and the stripe map is accessed modulo the division by the length of the stripe map.

The combination matrix is a square table of numbers with the side length equal to the number of disks in the RAID array. The number in the table in the i-th column, j-th row indicates how many times disk i and disk j were found in one stripe according to the stripe placement map.

This technical solution can be implemented on a computer, in the form of an automated information system (AIS), a distributed computer system, or a machine-readable medium containing instructions for performing the above-mentioned method of placing data in RAID arrays for balanced load distribution during array recovery using computing means (for example, a processor).

Fig. 1 shows a block diagram of the implementation of the claimed method (100) of placing data in RAID arrays for balanced load distribution during array recovery.

In the first step (101), a new RAID array is created using the generate stripe map procedure.

At stage (102), the number of free disks N, the length of the current stripe L and the length of the stripe placement map R are passed to the input of the stripe placement map generation procedure, and based on the data received, a stripe placement map is formed, consisting of R concatenated permutations of the set {1, …, N}.

The stripe placement map is an array of length k*N, where N is the number of disks and k is a fixed coefficient, in our case equal to 1024.

The following requirements apply to the stripe placement map:

The stripe placement map can be thought of as a concatenation of k permutations of the set {1,...,N}, which guarantees uniform disk usage since each disk n occurs exactly k times.

The stripe placement map can be perceived as a concatenation of (k*N)/M permutations of the set {1,...,M}, where M is the stripe length, which guarantees a correctly formed (without repetitions) stripe.

Figure 3 shows an example of a requirement for a stripe allocation map, where N is the number of disks, M is the stripe length, d _i is the disk number from the set {1, …, N}, and i is the index in the stripe allocation map from the set {1, …, k*N}.

At step (103), the matrix of combinations M of size NxN is initialized, where N is the number of disks, during which all its elements are assigned the value 0, and the current stripe is initialized with an empty list.

Initialization of the matrix of combinations M is the assignment of the value 0 to all its elements. The current stripe is initialized with an empty list. The current stripe is the name of an auxiliary structure of the list type. The elements of the list are the disk numbers. At different moments, the list may contain from 0 to L (the stripe length parameter passed to the input) elements. The filling of the list "current stripe" occurs during the generation of the permutation. As soon as its length reaches L, the list is reset. It is not saved by itself, but plays the role of an auxiliary structure for generating the permutation.

For example, the element of the combination matrix M[i][j] is responsible for how many times disk i and disk j occurred in one stripe. Since the relationship of being in one stripe is symmetric, the combination matrix M is symmetric with respect to its main diagonal. In order to optimize memory usage, only the upper triangular part of the combination matrix M can be stored. For simplicity of presentation, the description of the algorithm uses the full version of the combination matrix M, in which M[i][j]=M[j][i] for all i, j from the set {1,...,N}.

The value of the imbalance coefficient depends linearly on the ratio of the minimum and maximum (except for elements on the main diagonal) elements in the combination matrix M.

The above data layout method creates a stripe map by trying to balance the elements of the combination matrix: it directly uses the current data on the balance of disk combinations in stripes and tries to find the best combination based on this.

The combination matrix and stripe placement map are located in RAM. The combination matrix is a temporary object, the memory for which is allocated only for the time the stripe placement map is filled. The stripe placement map, on the contrary, is a persistent object and is located in RAM permanently. It is the one that determines on which disk a particular block of information is located.

At step (104), a permutation is generated in the stripe placement map (Fig. 6) by calling the generate permutation procedure with the following input parameters: the current value of the combination matrix M, the list of occupied disks in the current stripe, the length of the current stripe L, the number of disks in the RAID array N.

The stripe allocation map consists of K concatenated permutations. A permutation is an arbitrary ordered set of all elements of the set of disks without repetitions. For example, the permutations of the set {1,2,3} are 1, 2, 3; 3, 1, 2, etc.

The choice of this particular generation method is due to the fact that using the concatenation of permutations it is guaranteed that all disks occur in the stripe placement map with the same frequency. In turn, this guarantees uniform use of the space of all disks.

The input parameters allow generating a permutation that, taking into account the previously generated permutations, will allow obtaining a good imbalance coefficient for the disks, which is also the task of this solution. The most important here are the combination matrix M and the list of disks in the current stripe, which is what the new disk is selected for. Then this new disk is added to the stripe and to the permutation.

At step (105), to generate one permutation based on the input parameters, the auxiliary structures are initialized, namely: the list of free disks is initialized with numbers from 1 to N, the list of disks in the current permutation is initialized with an empty list.

At step (106), a disk that is contained in the list of free disks but is not contained in the list of occupied disks in the current stripe is iteratively selected using the search for the minimum sum of the elements of the combination matrix corresponding to the disk and the occupied disks in the current stripe. The minimum sum of the elements of the combination matrix is calculated at the permutation generation step. At each step in the permutation generation, one disk is added. The disk is selected based on the minimum of the sums in the combination matrix. The elements to be summed are selected according to the following logic, the row number is always the number of the candidate disk. The column number changes in a cycle that goes through all the elements of the current stripe array.

The disk selection procedure (Fig. 7) receives as input a matrix of combinations M, a list of disks in the current stripe S, and a non-empty list of available disks A. First, we initialize the auxiliary variables: the number of the disk with the smallest local sum, and the first element of the list of available disks is assigned to it. The minimum of local sums is assigned to it, and the maximum value of the INT type is assigned to it. Then, we iteratively go through all elements of list A. At the beginning of each iteration, we assign a local sum value equal to zero. We go through all elements of the list of disks in the current stripe S and add to the local sum the value of the matrix of combinations M, which is in the cell (the value of the current element of the non-empty list A, the value of the current element of the non-empty list S). After going through the list S, we obtain the value of the local sum. If it is less than the value of the current minimum of local sums, then we remember the disk number. The procedure returns the disk from list A with the smallest value of the local sum.

At step (107), the disk is added to the list of occupied disks in the current stripe and to the end of the list of disks in the current permutation.

At step (108), as soon as the length of the list of occupied disks in the current stripe has reached the length of the current stripe L, the combination matrix is updated and the list of occupied disks in the current stripe is assigned the value of the empty list and the procedure for generating a permutation in the stripe allocation map is repeated until the number of permutations has reached R.

The procedure for updating the combination matrix (Fig. 8) receives the combination matrix and the current stripe as input. We iteratively go through all possible values of the tuple (d1, d2), where d1 and d2 are the disk numbers from the list S, and increase the value of M[d1][d2] by 1. After processing all tuples, the procedure returns the updated combination matrix.

After all the above stages of the cycle are completed, the stripe placement map is returned, which will continue to be stored in RAM for the entire duration of the RAID array operation.

As follows from the above, the declared solution allows for a high level of data storage reliability and an increase in the speed of RAID array recovery due to the data arrangement scheme, which ensures uniform distribution of the reading load across all disks during data recovery.

Fig. 9 shows a general example of a computing device (900), which may be, for example, a computer, a server, a laptop, a smartphone, a SoC (System-on-a-Chip), etc. The device (900) may be used for the full or partial implementation of the claimed method (100).

In general, the device (900) comprises components such as: one or more processors (901), at least one random access memory (902), a permanent data storage means (903), input/output interfaces (904) including relay outputs for connection to belt conveyor motion controllers, an I/O means (905), and network interaction means (906).

The processor (901) of the device performs the basic computing operations necessary for the functioning of the device (900) or the functionality of one or more of its components. The processor (901) executes the necessary machine-readable commands contained in the RAM (902).

Memory (902) is usually implemented as RAM and contains the necessary software logic that provides the required functionality. Data storage facility (903) can be implemented as HDD, SSD disks, RAID array, network storage, flash memory, optical storage devices (CD, DVD, MD, BlueRay disks), etc. Facility (903) allows long-term storage of various types of information, such as recording magnetograms, request processing history (logs), user identifiers, camera data, images, etc.

Interfaces (904) are standard means for connecting and working with computing devices. Interfaces (904) can be, for example, relay connections, USB, RS232/422/485 or others, RJ45, LPT, UART, COM, HDMI, PS/2, Lightning, Fire Wire, etc. for work, including, according to Modbus protocols and Probfibus, Profinet networks or networks of another type. The choice of interfaces (904) depends on the specific design of the device (900), which can be a computing unit (computing module), for example based on a CPU (one or more processors), a microcontroller, etc., a personal computer, a mainframe, a server cluster, a thin client, a smartphone, a laptop, etc., as well as connected third-party devices.

The following can be used as I/O data means (905): keyboard, joystick, display (touch display), projector, touchpad, mouse, trackball, light pen, speakers, microphone, etc.

Network interaction means (906) are selected from a device that provides network reception and transmission of data, for example, an Ethernet card, a WLAN/Wi-Fi module, a Bluetooth module, a BLE module, an NFC module, IrDa, an RFID module, a GSM modem, etc. With the help of the means (906), data exchange is organized via a wired or wireless data transmission channel, for example, WAN, PAN, LAN, Intranet, Internet, WLAN, WMAN or GSM, a quantum (fiber optic) data transmission channel, satellite communication, etc. The components of the device (900), as a rule, are connected via a common data transmission bus.

A program is a sequence of instructions intended for execution by a computer control device or a command processing device.

The present application materials present a preferred disclosure of the implementation of the claimed technical solution, which should not be used as limiting other, particular embodiments of its implementation that do not go beyond the scope of the requested scope of legal protection and are obvious to specialists in the relevant field of technology.

Claims (10)

A computer-implemented method for placing data in RAID arrays for balanced load distribution during array recovery, comprising the steps of: - create a new RAID array using the generate stripe map procedure; - the number of free disks N, the length of the current stripe L and the length of the stripe placement map R are passed to the input of the stripe placement map generation procedure and, based on the data received, a stripe placement map is formed consisting of R concatenated permutations of the set {1, …, N}; - initialize the matrix of combinations M of size N×N, during which all its elements are assigned the value 0, the current stripe is initialized with an empty list; - generate a permutation in the stripe placement map by calling the generate permutation procedure with the following input parameters: the current value of the combination matrix M, the list of occupied disks in the current stripe, the length of the current stripe L, the number of disks in the RAID array N; - to generate one permutation based on the input parameters, the auxiliary structures are initialized, namely: the list of free disks is initialized with numbers from 1 to N, the list of disks in the current permutation is initialized with an empty list; - iteratively select a disk that is contained in the list of free disks, but is not contained in the list of occupied disks in the current stripe, using the search for the minimum sum of the elements of the matrix of combinations corresponding to the disk and the occupied disks in the current stripe; - add a disk to the list of occupied disks in the current stripe and to the end of the list of disks in the current permutation; - as soon as the length of the list of occupied disks in the current stripe reaches the length of the current stripe L, the combination matrix is updated and the list of occupied disks in the current stripe is assigned the value of the empty list; - repeat the procedure of generating a permutation in the stripe placement map until the number of permutations reaches R.