WO2020094134A1 - 磁盘分配方法、装置和可读存储介质 - Google Patents

磁盘分配方法、装置和可读存储介质 Download PDF

Info

Publication number
WO2020094134A1
WO2020094134A1 PCT/CN2019/116783 CN2019116783W WO2020094134A1 WO 2020094134 A1 WO2020094134 A1 WO 2020094134A1 CN 2019116783 W CN2019116783 W CN 2019116783W WO 2020094134 A1 WO2020094134 A1 WO 2020094134A1
Authority
WO
WIPO (PCT)
Prior art keywords
disk
disk group
erasure code
group
data
Prior art date
Application number
PCT/CN2019/116783
Other languages
English (en)
French (fr)
Inventor
曹力
夏伟强
吴小鹏
Original Assignee
杭州海康威视系统技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视系统技术有限公司 filed Critical 杭州海康威视系统技术有限公司
Publication of WO2020094134A1 publication Critical patent/WO2020094134A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1008—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices
    • G06F11/1012—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices using codes or arrangements adapted for a specific type of error
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1008—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices
    • G06F11/1064—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices in cache or content addressable memories
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers

Definitions

  • This application relates to the field of computer technology, and in particular, to a disk allocation method, device, and readable storage medium.
  • erasure codes In the field of large-scale data storage, erasure codes (Erasure Coding, EC) are used to encode data to obtain data blocks and check blocks, that is, erasure code data, to improve the fault tolerance of the data.
  • the data blocks and check blocks in the erasure code data can be stored in different disks to further improve the security of the data.
  • different types of erasure code data may be stored in different disks, so disk allocation needs to be performed for multiple types of erasure code data.
  • the type of erasure code data refers to the type of erasure code used when encoding to obtain erasure code data.
  • a distributed storage system is used to store data, and the distributed storage system includes multiple disks.
  • an initial disk group is determined from the distributed storage system through a hash algorithm.
  • the initial disk group includes multiple storage devices, and each storage device includes multiple disks. Determine the usage heat and reference heat of each storage device in the initial disk group, and allocate the final disk for the erasure code data corresponding to the target data according to the difference between the usage heat and reference heat of each storage device.
  • the characteristics of the hash algorithm will result in uneven distribution of the initial disk group distribution among all disks in the entire distributed storage system, resulting in uneven distribution of the final allocated disks among all disks in the entire distributed storage system ; And after the initial disk group is determined, the type of erasure code data that each storage device in the initial disk group can store is also fixed. Therefore, for new types of erasure code data, the erasure code data may not be successfully stored, Therefore, the flexibility of this disk allocation method is poor.
  • Embodiments of the present application provide a disk allocation method, device, and readable storage medium to overcome the problem of poor flexibility in the disk allocation manner in the related art.
  • the technical solution is as follows:
  • a disk allocation method includes:
  • the first erasure code is used to encode the target data to obtain erasure code data including a data block and a check block, and different types of erasure codes
  • the parameter value corresponding to each type of erasure code is used to indicate the number of data blocks and the number of check blocks coded according to the corresponding erasure code
  • a first disk group is allocated in the storage space according to the first erasure code, and there are one or more first disk groups for storing the erasure code data.
  • the allocating the first disk group in the storage space according to the first erasure code includes:
  • the first disk group is created in the storage space according to the first erasure code
  • the second disk group is used to store the erasure code data obtained according to the second erasure code encoding, then according to the first erasure code and the A second erasure code allocates the first disk group in the storage space, wherein the type of the second erasure code is different from the first erasure code.
  • the allocating the first disk group in the storage space according to the first erasure code and the second erasure code includes:
  • the second disk group is used as the first disk group
  • the first disk group is created in the storage space according to the first erasure code
  • the first disk group is allocated in the second disk group
  • the disk allocation information corresponding to the second erasure code is backed up and redistributed The first disk group and the second disk group.
  • the creating the first disk group in the storage space according to the first erasure code includes:
  • any first disk group sequentially select the disks in the storage space as one disk in the first disk group, and the number of disks in the first disk group is equal to the data corresponding to the first erasure code
  • the sum of the number of blocks and the number of check blocks, at least one disk is spaced between the starting disks of the plurality of first disk groups.
  • the allocating the first disk group in the second disk group includes:
  • Target disk group is one or more
  • the selecting a target disk group from the second disk group includes:
  • the overlapping second disk group is the second including any disk included in the previously selected target disk group Disk group.
  • the reallocating the first disk group and the second disk group includes:
  • the second disk group is allocated according to the disk allocation information corresponding to the second erasure code backup
  • the second disk group is allocated in the storage space based on the first disk group and the second erasure code
  • the second disk group is allocated according to the disk allocation information corresponding to the second erasure code that is backed up;
  • the disk allocation information corresponding to the backed-up erasure codes of the second type is deleted.
  • a disk distribution device is also provided.
  • the device includes:
  • the acquisition module is used to acquire the target data to be stored
  • a determining module configured to determine a first erasure code corresponding to the target data, and the first erasure code is used to encode the target data to obtain erasure code data including a data block and a check block, different
  • the types of erasure codes correspond to different parameter values, and the parameter values corresponding to each type of erasure codes are used to indicate the number of data blocks and the number of check blocks that are encoded according to the corresponding erasure codes;
  • An allocation module configured to allocate a first disk group in a storage space according to the first erasure code, and the first disk group is one or more, used to store the erasure code data.
  • the distribution module is used to:
  • the first disk group is created in the storage space according to the first erasure code
  • the second disk group is used to store the erasure code data obtained according to the second erasure code encoding, then according to the first erasure code and the A second erasure code allocates the first disk group in the storage space, wherein the type of the second erasure code is different from the first erasure code.
  • the distribution module is used to:
  • the second disk group is used as the first disk group
  • the first disk group is created in the storage space according to the first erasure code
  • the first disk group is allocated in the second disk group
  • the disk allocation information corresponding to the second erasure code is backed up and redistributed The first disk group and the second disk group.
  • the allocation module is configured to select, in order for any first disk group, the disks in the storage space as one disk in the first disk group, and the number of disks in the first disk group It is equal to the sum of the number of data blocks and the number of check blocks corresponding to the first erasure code, and there is at least one disk between the starting disks of the plurality of first disk groups.
  • the allocation module is configured to select a target disk group from the second disk group, and the target disk group is one or more; select at least one disk from the selected target disk group as A disk in the first disk group.
  • the allocation module is configured to skip overlapping second disk groups, and select a target disk group from the remaining second disk groups in sequence, and the overlapping second disk group includes the previously selected target disk The second disk group of any disk included in the group.
  • the distribution module is used to:
  • the second disk group is allocated according to the disk allocation information corresponding to the second erasure code backup
  • the second disk group is allocated in the storage space based on the first disk group and the second erasure code
  • the second disk group is allocated according to the disk allocation information corresponding to the second erasure code that is backed up;
  • the disk allocation information corresponding to the backed-up erasure codes of the second type is deleted.
  • a disk allocation device includes a processor and a memory. At least one instruction is stored in the memory, and the instruction is loaded and executed by the processor to implement any of the above disk allocations. method.
  • a computer-readable storage medium where at least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement any of the above-mentioned disk allocation methods.
  • the first disk group is allocated in the storage space according to the first erasure code corresponding to the target data, so as to store the target data according to the first erasure code through the disks included in the first disk group Erasure code data obtained by encoding data. Since different types of erasure codes correspond to different parameter values, the parameter values corresponding to each type of erasure codes are used to indicate the number of data blocks and the number of check blocks that are encoded according to the corresponding erasure codes, thus It can be seen that the technical solutions provided by the embodiments of the present application can dynamically adjust the allocated disks according to different types of erasure codes, thereby improving the flexibility of disk allocation.
  • FIG. 1 shows an architectural diagram of an object storage system provided by an embodiment of the present application
  • FIG. 2 shows a schematic diagram of writing erasure code data written to a disk according to an embodiment of the present application
  • FIG. 3 shows a flowchart of a disk allocation method provided by an embodiment of the present application
  • FIG. 4 shows a flowchart of a disk allocation method provided by an embodiment of the present application
  • FIG. 5 shows a structural block diagram of a disk distribution device provided by an embodiment of the present application
  • FIG. 6 shows a structural block diagram of a disk distribution device provided by an embodiment of the present application.
  • Erasure coding is a method of processing data in order to protect the data. It divides the data into data blocks and encodes the data blocks to obtain redundant check blocks, so that the correlation between the data is generated to achieve fault tolerance.
  • the erasure code includes at least two core parameters, respectively N and M, which means that N data blocks are encoded by the erasure code to obtain M check blocks, thereby obtaining N + M block erasure code data.
  • erasure codes correspond to different parameter values, for example, the value of N for the first erasure code is N1, and the value of M for the first erasure code is M1, indicating that N1 data blocks are encoded by the first erasure code M1 check blocks are obtained; the value of N of the second erasure code is N2, and the value of M of the second erasure code is M2, indicating that N2 data blocks are encoded by the second erasure code to obtain M2 check blocks.
  • the data block and the check block in the encoded erasure code data can be stored on different disks to prevent the data from being damaged due to damage to one disk.
  • different types of erasure code data may be stored in the storage space, for example, erasure code data obtained through second erasure code encoding and erasure code data obtained through first erasure code encoding.
  • the disk allocation method of erasure code data involved in the present application can be used for disk allocation in a storage space.
  • multiple data blocks and check blocks of erasure coded data can be stored on disks in different object storage devices (Object-based Storage Devices, OSDs) or on the same OSD In different disks.
  • OSDs Object-based Storage Devices
  • the following describes an example of the method for allocating erasure code data in a disk of the target storage system to store the erasure code data in different disks of the same storage system as an example.
  • the object storage system includes at least a metadata server, an object storage server, a client, and multiple disks.
  • the client provides a user interaction interface, which can be used to obtain data that has not been processed by erasure coding.
  • the object storage server includes a disk distribution component, an EC computing component, and a disk read-write component.
  • the object storage server receives the data obtained by the client; the data is processed by the EC computing component to obtain erasure code data; the erasure code data is stored in multiple disks in the form of a stripe.
  • a Stripe is composed of N + M stripe blocks, N indicates that there are N data blocks in the Stripe, and M indicates that there are M check blocks in the Stripe ,
  • Each data block or parity block can be called a stripe block, the disk allocation component allocates N + M disks for the N + M stripe blocks; the disk read-write component separates the N + M stripe blocks Write to the allocated N + M disks.
  • the metadata server generates a stripe information table according to the storage status of Stripe in the object storage server, and stores the stripe information table in the metadata server.
  • the Stripe information table includes but is not limited to one or more of the following information: Stripe ID (Stripe Identifier, Stripe ID), the globally unique name (wwn) of the hard disk corresponding to the stripe block, and the type of erasure code. See Table 1, which shows an example of the Stripe information table.
  • the data obtained from the client is processed by the EC computing component to obtain four data blocks and a check block, which are data block 1, data block 2, data block 3, data block 4, and check Block 1, to obtain a Stripe
  • the Stripe includes stripe block 1, stripe block 2, stripe block 3, stripe block 4 and stripe block 5, respectively corresponding to data block 1, data block 2, data block 3, Data block 4 and parity block 1
  • the disk allocation component allocates disk 1, disk 2, disk 3, disk 4, and disk 5 to the stripe
  • the disk read-write component combines stripe block 1, stripe block 2, and stripe block 3 , Stripe block 4 and stripe block 5 are written to disk 1, disk 2, disk 3, disk 4 and disk 5, respectively; and store the Stripe Stripe information table in the metadata server.
  • FIG. 3 shows a flowchart of a disk allocation method for erasure code data provided by an embodiment of the present application.
  • the method will be described below by using an object storage server as an example.
  • the method includes:
  • Step 301 Obtain target data to be stored.
  • the client When there is data to be stored, the client sends the target data to be stored to the object storage server, and the object storage server thus obtains the target data to be stored, and triggers the subsequent disk allocation process.
  • Step 302 Determine a first erasure code corresponding to the target data.
  • the first erasure code is used to encode the target data to obtain erasure code data including a data block and a check block;
  • the target data when storing the target data, the target data is divided into data blocks, and the erasure codes are used to encode the data blocks to obtain redundant check blocks, so that the correlation between the data is generated to achieve fault tolerance the goal of.
  • the storage of the target data is achieved by storing the data block and the check block.
  • the erasure code since the erasure code is used when encoding the data block, the erasure code is different, and the check block obtained by the encoding is also different. Therefore, in order to achieve subsequent disk allocation, the method provided in the embodiment of the present application needs to first determine the corresponding The determination method of the first erasure code may be limited according to the actual situation, which is not limited in the embodiments of the present application.
  • Step 303 Allocate a first disk group in the storage space according to the first erasure code, and the first disk group is one or more for storing erasure code data.
  • the method provided by the embodiment of the present application includes, but is not limited to, the following two cases when allocating the first disk group in the storage space according to the first erasure code:
  • the first case if there is no allocated disk group in the storage space, the first disk group is created in the storage space according to the first erasure code;
  • the second case if there is an allocated second disk group in the storage space, and the second disk group is used to store the erasure code data obtained according to the second erasure code encoding, then according to the first erasure code and the second correction
  • the erasure code allocates the first disk group in the storage space, where the type of the second erasure code is different from the first erasure code.
  • allocating the first disk group in the storage space according to the first erasure code and the second erasure code includes any one of the following four processing methods:
  • the first way if the number of check blocks corresponding to the first erasure code is equal to the number of check blocks corresponding to the second erasure code, and the number of data blocks corresponding to the first erasure code is equal to the second The number of data blocks corresponding to the erasure code, the second disk group is used as the first disk group;
  • the first disk group is created in the storage space according to the first erasure code
  • creating the first disk group in the storage space according to the first erasure code includes:
  • any first disk group select the disks in the storage space in turn as a disk in the first disk group.
  • the number of disks in the first disk group is equal to the number of data blocks corresponding to the first erasure code and the calibration
  • Third method If the number of check blocks corresponding to the first erasure code is greater than the number of check blocks corresponding to the second erasure code, the first disk group is allocated in the second disk group;
  • Target disk group from the second disk group, the target disk group is one or more;
  • selecting the target disk group from the second disk group includes:
  • the overlapping second disk group is the second disk group including any disks included in the first selected target disk group .
  • Fourth method If the number of check blocks corresponding to the first erasure code is less than the number of check blocks corresponding to the second erasure code, then back up the disk allocation information corresponding to the second erasure code and reallocate the first One disk group and the second disk group.
  • Reassign the first disk group and the second disk group including:
  • the second disk group is allocated according to the disk allocation information corresponding to the backup second erasure code
  • the second disk group is allocated in the storage space based on the first disk group and the second erasure code
  • the second disk group is allocated according to the disk allocation information corresponding to the backup second erasure code
  • the disk allocation information corresponding to the backup type 2 erasure code is deleted.
  • the first disk group is allocated in the storage space according to the first erasure code corresponding to the target data, so as to store the target data according to the first erasure code through the disks included in the first disk group Erasure code data obtained by encoding data. Since different types of erasure codes correspond to different parameter values, the parameter values corresponding to each type of erasure codes are used to indicate the number of data blocks and the number of check blocks that are encoded according to the corresponding erasure codes, thus It can be seen that the technical solutions provided by the embodiments of the present application can dynamically adjust the allocated disks according to different types of erasure codes, thereby improving the flexibility of disk allocation.
  • the data blocks and check blocks corresponding to the erasure code data can be stored relatively evenly in the disks in the storage space, which improves the utilization of the storage space.
  • FIG. 4 shows a flowchart of a disk allocation method provided by an embodiment of the present application.
  • the method will be described below by using an object storage server as an example.
  • Step 401 Allocate a second disk group corresponding to the second erasure code in the storage space.
  • the storage space includes X disks; the second erasure code is used to generate M2 check blocks based on N2 data blocks; the second disk group is used to store N2 data blocks obtained by encoding the second erasure code and M2 parity blocks, the storage space includes at least one second disk group, each second disk group includes N2 + M2 disks out of X disks in the storage space, X, N2, and M2 are positive integers, and X Not less than N2 + M2.
  • step 401 includes:
  • a plurality of second disk groups are allocated according to this sequential selection method, where the starting disks of the plurality of second disk groups are separated by K1 disks, K1 is a non-negative integer, and K1 is less than X.
  • the disks included in the second disk group are: the X-1th disk Disk, the Xth disk, the first disk, ..., the N2 + M2-2 disk.
  • one of the foregoing OSDs includes 12 disks, that is, the value of X is 12, and the disk numbers are D1 to D12; the value of N2 of the second erasure code is 2, and the value of M2 is 1, according to the above
  • the disks included in the multiple second disk groups allocated in the implementation manner are shown in Table 2.
  • There is one disk between the first disk that is, the value of K1 is 1.
  • Step 402 Obtain target data to be stored, determine a first erasure code corresponding to the target data, and allocate a first disk group in the storage space according to the first erasure code.
  • the first disk group is used to store data blocks and check blocks in the erasure code data obtained by encoding the target data according to the first erasure code, for example, the first erasure code is used to generate based on N1 data blocks M1 check blocks.
  • the storage space includes at least one first disk group, and each first disk group includes N1 + M1 disks out of X disks in the storage space, N1 and M1 are positive integers, and X is not less than N1 + M1. Optionally, if X is less than N1 + M1, the allocation of the first disk group fails.
  • the first disk group is allocated based on the disk groups already allocated in the storage space, and if there are multiple disk groups corresponding to multiple erasure codes in the storage space, the multiple erasure codes may be selected The erasure code whose parameter M is the smallest and (N + M) is the smallest, and the first disk group is allocated based on the disk group corresponding to the erasure code.
  • the following takes the second disk group corresponding to the second erasure code in the storage space and assigns the first disk group based on the second disk group as an example for description.
  • the parameters of the first erasure code and the second erasure code May include the following.
  • assigning a first disk group includes the following steps:
  • Step A Select a disk in a second disk group as a base disk (baseDisk), the number of the baseDisk in the storage space is Db, and use the baseDisk as the first disk in the first disk group. For example, the disk with the number 1 can be selected as the baseDisk of the first disk group. If M2 is less than 2, step A ends; if M2 is not less than 2, then in the second disk group where baseDisk is located, select the M2-1 disks numbered Db + 1 to Db + M2-1 after baseDisk in sequence As the second to M2th disks in the first disk group, step A ends. Repeat step B until the number of disks in the first disk group is M1 + N1.
  • Step B Select the disk with the disk number Db + N2 + M2 as the M2 + 1th disk in the first disk group. If M2 is less than 2, step B ends; if M2 is not less than 2, then sequentially select the disks numbered Db + N2 + M2 + 1 to Db + N2 + 2 * M2-1 as the disks in the first disk group, Step B ends. Where, if any of the numbers Db + N2 + M2 + 1 to Db + N2 + 2 * M2-1 is greater than X, the value greater than X is subtracted from X, and the updated number value is selected Disk.
  • step B if the number of disks in the first disk group reaches M1 + N1, step B ends.
  • a first disk group is allocated through steps A and B, and other first disk groups can also be acquired through steps A and B.
  • the selected baseDisk and The baseDisks of one or more first disk groups that have been allocated are different. For example, you can select the disk with the number 2 as the baseDisk of the other first disk group, and perform steps A and B in sequence to obtain the other first disk group.
  • a first disk group allocated through steps A and B includes four disks numbered 4, 7, 10, and 1, and the first disk group has been allocated with a first disk group (including the number 1 , 4, 7 and 10 disks) are the same, the allocation of the first disk group ends.
  • assigning a first disk group includes the following steps:
  • Step 1 In the process of allocating disks for a first disk group, first select a second disk group from the second disk group corresponding to the second erasure code, and then select the next second disk group in sequence. If a second disk group is an overlapping second disk group, the overlapping second disk group is skipped, and the second disk group continues to be selected after the overlapping second disk group until P second disk groups are obtained.
  • the overlapping second disk group is a second disk group including any disk included in the second disk group selected first.
  • the second disk group numbered 1 in Table 2 has been selected, and the second disk group including the disks with disk numbers D1, D2, or D3 is skipped, that is, skipped For the second disk group numbered 2 and 3, continue to allocate the first disk group from the second disk group numbered 4.
  • Step 2 Select M2 disks from the plurality of second disk groups that have been selected in step 1, as a disk in the first disk group, until the first disk group includes N1 + M1 disks.
  • M2 disks are selected in sequence from the selected second disk group.
  • the starting disk in the selection process is the H1th disk, and H1 is a non-negative integer.
  • H1 is a non-negative integer.
  • the value of the starting disk number H1 may be the same or different.
  • step 1 and step 2 you can assign a disk to a first disk group.
  • M2 disks are used as disks in the other first disk group until the first disk group includes N1 + M1 disks, and the other M2 disks do not overlap with the M2 disks selected in step two. For example, select M2 disks in sequence from the selected second disk group.
  • the starting disk in the selection process is the H2th disk, H2 is a non-negative integer, the value of H2 is different from the value of H1, and the value of H2 and H1
  • the difference is M2 to ensure that the other M2 disks do not overlap with the M2 disks selected in step two.
  • the 4 disks are selected from the second disk groups in Table 2 and selected from the 4 second disks.
  • Select M2 1 disk in the group, so as to obtain 4 disks that make up a first disk group.
  • the second disk group numbered 1 is preferred, and the disk numbered D1 is selected as the first disk group numbered 1 One disk; because the second disk group numbered 1 also includes disks numbered D2 and D3, skip the second disk group numbers 2 and 3 that include disks numbered D1, D2, or D3, and select the number Is the second disk group of 4, and selects the disk with the number D4 as the second disk of the first disk group with the number 1; and so on, and assigns multiple first disk groups. It can be seen that the disks of the first disk group and the second disk group are more evenly distributed in the storage space, so that the data of the second erasure code and the data of the first erasure code can be more evenly distributed in the storage space, improving storage Space utilization
  • step one selects the disks from the selected multiple second disk groups and assign the first disk group, if the selected multiple second disk groups The disks of are selected as the disks in multiple first disk groups, then look for the second disk group in the storage space that is not assigned as the first disk group, starting with the second disk group, repeat steps one and step Second, assign other first disk groups.
  • the storage space includes 48 disks, including 48 second disk groups
  • three first disk groups are obtained based on the first 12 second disk groups.
  • the first 12 second disk groups All of the disks are assigned as the first disk group, you can start from the 13th second disk group and continue to allocate the 4th first disk group.
  • Case 4 If M1 is less than M2, the disk allocation information corresponding to the second erasure code is backed up, and the first disk group and the second disk group are reallocated.
  • the second disk group if the first disk group is allocated successfully, then the second disk group is allocated based on the first disk group according to the method shown in case one in step 402, and if the second disk group fails to be allocated, then the second disk group is allocated
  • the disk allocation information corresponding to the erasure code restores the second disk group among the X disks in the storage space; if the allocation of the second disk group is successful, the disk allocation information corresponding to the backup second erasure code is deleted.
  • the failure to allocate the first disk group may be due to X being less than N1 + M1; the failure to allocate the second disk group may be due to X being less than N2 + M2.
  • the metadata server may also generate a Stripe information table according to the first disk group and the second disk group allocated by the object storage server in steps 401 and 402, respectively. And store the Stripe information table generated in the metadata server.
  • the first disk group is allocated in the storage space according to the first erasure code corresponding to the target data, so as to store the target data according to the first erasure code through the disks included in the first disk group Erasure code data obtained by encoding data. Since different types of erasure codes correspond to different parameter values, the parameter values corresponding to each type of erasure codes are used to indicate the number of data blocks and the number of check blocks that are encoded according to the corresponding erasure codes, thus It can be seen that the technical solutions provided by the embodiments of the present application can dynamically adjust the allocated disks according to different types of erasure codes, thereby improving the flexibility of disk allocation.
  • the data blocks and check blocks corresponding to the erasure code data can be stored relatively evenly in the disks in the storage space, which improves the utilization of the storage space.
  • FIG. 5 shows a structural block diagram of a disk distribution device 500 for erasure code data provided by an embodiment of the present application.
  • the device includes: an acquisition module 510, a determination module 520, and an allocation module 530.
  • the obtaining module 510 is used to obtain target data to be stored.
  • the determination module 520 is used to determine the first erasure code corresponding to the target data.
  • the first erasure code is used to encode the target data to obtain erasure code data including a data block and a check block.
  • Different types of erasure codes The codes correspond to different parameter values, and the parameter values corresponding to each type of erasure codes are used to indicate the number of data blocks and the number of check blocks that are encoded according to the corresponding erasure codes.
  • the allocation module 530 is configured to allocate a first disk group in the storage space according to the first erasure code, and the first disk group is one or more, used to store erasure code data.
  • the distribution module 530 is used to:
  • the first disk group is created in the storage space according to the first erasure code
  • the second disk group is used to store the erasure code data obtained according to the second erasure code encoding, then the first erasure code and the second erasure code are stored in the storage space
  • the first disk group is allocated in, where the type of the second erasure code is different from the first erasure code.
  • the distribution module 530 is used to:
  • the number of check blocks corresponding to the first erasure code is equal to the number of check blocks corresponding to the second erasure code
  • the number of data blocks corresponding to the first erasure code is equal to that corresponding to the second erasure code
  • the number of check blocks corresponding to the first erasure code is equal to the number of check blocks corresponding to the second erasure code, and the number of data blocks corresponding to the first erasure code is not equal to that corresponding to the second erasure code
  • the number of data blocks then create the first disk group in the storage space according to the first erasure code
  • the first disk group is allocated in the second disk group
  • the disk allocation information corresponding to the second erasure code is backed up, and the first disk group and the first Two disk groups.
  • the distribution module 530 is used to:
  • any first disk group select the disks in the storage space in turn as a disk in the first disk group.
  • the number of disks in the first disk group is equal to the number of data blocks corresponding to the first erasure code and the calibration
  • the distribution module 530 is used to:
  • Select the target disk group from the second disk group, the target disk group is one or more; select at least one disk from the selected target disk group as a disk in the first disk group.
  • the distribution module 530 is used to:
  • the overlapping second disk group is the second disk including any disk included in the first selected target disk group group.
  • the distribution module 530 is used to:
  • the second disk group is allocated according to the disk allocation information corresponding to the backup second erasure code
  • the second disk group is allocated in the storage space based on the first disk group and the second erasure code
  • the second disk group is allocated according to the disk allocation information corresponding to the backup second erasure code
  • the disk allocation information corresponding to the backup type 2 erasure code is deleted.
  • the data blocks and check blocks corresponding to the erasure code data can be stored relatively evenly in the disks in the storage space, which improves the utilization of the storage space.
  • FIG. 6 shows a schematic structural diagram of a disk distribution device provided by an embodiment of the present application.
  • the device may be a server or a terminal, specifically:
  • the computing system 600 includes a central processing unit (CPU) 601, a system memory 604 including a random access memory (RAM) 602 and a read only memory (ROM) 603, and a system bus 605 connecting the system memory 604 and the central processing unit 601.
  • the computing system 600 also includes a basic input / output system (I / O system) 606 that helps transfer information between various devices in the computer, and a mass storage device for storing the operating system 613, application programs 614, and other program modules 615 607.
  • I / O system basic input / output system
  • the basic input / output system 606 includes a display 608 for displaying information and an input device 609 such as a mouse and a keyboard for a user to input information.
  • the display 608 and the input device 609 are both connected to the central processing unit 601 through the input and output controller 610 connected to the system bus 605.
  • the basic input / output system 606 may also include an input-output controller 610 for receiving and processing input from a number of other devices such as a keyboard, mouse, or electronic stylus.
  • the input output controller 610 also provides output to a display screen, printer, or other type of output device.
  • the mass storage device 607 is connected to the central processing unit 601 through a mass storage controller (not shown) connected to the system bus 605.
  • Mass storage device 607 and its associated computer-readable medium provide non-volatile storage for computing system 600. That is, the mass storage device 607 may include a computer-readable medium (not shown) such as a hard disk or CD-ROM drive.
  • Computer-readable media may include computer storage media and communication media.
  • Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
  • Computer storage media include RAM, ROM, EPROM, EEPROM, flash memory, or other solid-state storage technologies, CD-ROM, DVD, or other optical storage, tape cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices.
  • RAM random access memory
  • ROM read-only memory
  • EPROM Erasable programmable read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • flash memory or other solid-state storage technologies
  • CD-ROM, DVD or other optical storage
  • tape cassettes magnetic tape
  • magnetic disk storage or other magnetic storage devices.
  • computer storage medium is not limited to the above types.
  • the above-mentioned system memory 604 and mass storage device 607 may be collectively referred to as a memory.
  • the computing system 600 may also be operated by a remote computer connected to the network through a network such as the Internet. That is, the computing system 600 can be connected to the network 612 through the network interface unit 611 connected to the system bus 605, or the network interface unit 611 can also be used to connect to other types of networks or remote computer systems (not shown).
  • the above memory also includes one or more programs.
  • One or more programs are stored in the memory and configured to be executed by the CPU.
  • the one or more programs include instructions for performing a disk allocation method for erasure code data provided in any of FIGS. 3 and 4.
  • An embodiment of the present application also provides a non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the computing system, the computing system can execute any of the provided in FIG. 3 and FIG. 4 Disk allocation method for erasure code data.
  • a computer program product containing instructions that, when run on a computer, causes the computer to execute instructions for performing a disk allocation method for erasure code data provided in any of FIGS. 3 and 4.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Human Computer Interaction (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)

Abstract

一种磁盘分配方法、装置和可读存储介质，属于计算机技术领域，该方法包括：获取待存储的目标数据(301)，确定所述目标数据对应的第一纠删码，所述第一纠删码用于对所述目标数据进行编码，得到包括数据块和校验块的纠删码数据(302)，根据所述第一纠删码在存储空间中分配第一磁盘组，所述第一磁盘组为一个或者多个，用于存储所述纠删码数据(303)。上述方法实现了可根据纠删码的不同，来动态调整分配的磁盘，提高了磁盘分配的灵活性。

Description

磁盘分配方法、装置和可读存储介质
本申请要求于2018年11月09日提交的申请号为201811332967.4、发明名称为“磁盘分配方法、装置和可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，特别涉及一种磁盘分配方法、装置和可读存储介质。
背景技术
在大规模数据存储领域，纠删码(Erasure Coding，EC)用于将数据编码得到数据块和校验块，即纠删码数据，以提高数据的容错能力。纠删码数据中的数据块和校验块可以分别存储于不同的磁盘中以进一步提高数据的安全性。对于大规模数据存储，不同类型的纠删码数据可能存储于不同的磁盘中，因此需要针对多类型的纠删码数据进行磁盘分配。其中，纠删码数据的类型是指编码得到纠删码数据时所使用的纠删码的类型。
相关技术中，采用分布式存储系统来存储数据，分布式存储系统包括多个磁盘。针对待存储的目标数据，通过哈希算法从分布式存储系统中确定初始磁盘组，初始磁盘组包括多个存储设备，每个存储设备包括多个磁盘。确定该初始磁盘组中每个存储设备的使用热度和参考热度，根据每个存储设备的使用热度和参考热度之间的差值，为目标数据对应的纠删码数据分配最终的磁盘。
哈希算法的特性会导致分配的初始磁盘组在整个分布式存储系统中的所有磁盘中的分布不均匀，从而导致最终分配的磁盘在整个分布式存储系统中的所有磁盘中的分布也不均匀；且在初始磁盘组确定后，初始磁盘组中各个存储设备能够存储的纠删码数据的类型也固定，因此，对于新的类型的纠删码数据，可能导致纠删码数据无法成功存储，因而该种磁盘分配方式的灵活性较差。
发明内容
本申请实施例提供了一种磁盘分配的方法、装置和可读存储介质，以克服相关技术中存在的磁盘分配方式灵活性较差的问题。所述技术方案如下：
一方面，提供了一种磁盘分配方法，该方法包括：
获取待存储的目标数据；
确定所述目标数据对应的第一纠删码，所述第一纠删码用于对所述目标数据进行编码，得到包括数据块和校验块的纠删码数据，不同类型的纠删码对应不同的参数值，每种类型的纠删码对应的参数值用于指示按照相应纠删码编码得到的数据块的个数和校验块的个数；
根据所述第一纠删码在存储空间中分配第一磁盘组，所述第一磁盘组为一个或者多个，用于存储所述纠删码数据。
可选地，所述根据所述第一纠删码在存储空间中分配第一磁盘组，包括：
若所述存储空间中不存在已分配的磁盘组，则根据所述第一纠删码在所述存储空间中创建所述第一磁盘组；
若所述存储空间中存在已分配的第二磁盘组，所述第二磁盘组用于存储根据第二纠删码编码得到的纠删码数据，则根据所述第一纠删码及所述第二纠删码在所述存储空间中分配所述第一磁盘组，其中，所述第二纠删码的类型不同于所述第一纠删码。
可选地，所述根据所述第一纠删码及所述第二纠删码在所述存储空间中分配所述第一磁盘组，包括：
若所述第一纠删码对应的校验块的个数等于所述第二纠删码对应的校验块的个数，且所述第一纠删码对应的数据块的个数等于所述第二纠删码对应的数据块的个数，则将所述第二磁盘组作为所述第一磁盘组；
若所述第一纠删码对应的校验块的个数等于所述第二纠删码对应的校验块的个数，且所述第一纠删码对应的数据块的个数不等于所述第二纠删码对应的数据块的个数，则根据所述第一纠删码在所述存储空间中创建所述第一磁盘组；
若所述第一纠删码对应的校验块的个数大于所述第二纠删码对应的校验块的个数，则在所述第二磁盘组中分配所述第一磁盘组；
若所述第一纠删码对应的校验块的个数小于所述第二纠删码对应的校验块的个数，则备份所述第二纠删码对应的磁盘分配信息，重新分配所述第一磁盘组和所述第二磁盘组。
可选地，所述根据所述第一纠删码在所述存储空间中创建所述第一磁盘组， 包括：
对于任一第一磁盘组，依次选择存储空间中的磁盘作为所述第一磁盘组中的一个磁盘，所述第一磁盘组中的磁盘的个数等于所述第一纠删码对应的数据块的个数和校验块的个数之和，多个所述第一磁盘组的起始磁盘之间至少间隔有一个磁盘。
可选地，所述在所述第二磁盘组中分配所述第一磁盘组，包括：
从所述第二磁盘组中选择目标磁盘组，所述目标磁盘组为一个或多个；
从所选择的目标磁盘组中分别选择至少一个磁盘，作为一个第一磁盘组中的磁盘。
可选地，所述从所述第二磁盘组中选择目标磁盘组，包括：
跳过重叠的第二磁盘组，从剩余的第二磁盘组中依次选择目标磁盘组，所述重叠的第二磁盘组是包括在先选择的目标磁盘组中所包括的任一磁盘的第二磁盘组。
可选地，所述重新分配所述第一磁盘组和所述第二磁盘组，包括：
在所述存储空间中分配所述第一磁盘组；
若分配所述第一磁盘组失败，则根据备份的所述第二纠删码对应的磁盘分配信息分配第二磁盘组；
若分配所述第一磁盘组成功，则基于所述第一磁盘组和所述第二纠删码，在所述存储空间中分配所述第二磁盘组；
若分配所述第二磁盘组失败，则根据备份的所述第二纠删码对应的磁盘分配信息分配第二磁盘组；
若分配所述第二磁盘组成功，则删除备份的所述第二类型纠删码对应的磁盘分配信息。
一方面，还提供了一种磁盘分配装置，所述装置包括：
获取模块，用于获取待存储的目标数据；
确定模块，用于确定所述目标数据对应的第一纠删码，所述第一纠删码用于对所述目标数据进行编码，得到包括数据块和校验块的纠删码数据，不同类型的纠删码对应不同的参数值，每种类型的纠删码对应的参数值用于指示按照相应纠删码编码得到的数据块的个数和校验块的个数；
分配模块，用于根据所述第一纠删码在存储空间中分配第一磁盘组，所述 第一磁盘组为一个或者多个，用于存储所述纠删码数据。
可选地，所述分配模块，用于：
若所述存储空间中不存在已分配的磁盘组，则根据所述第一纠删码在所述存储空间中创建所述第一磁盘组；
若所述存储空间中存在已分配的第二磁盘组，所述第二磁盘组用于存储根据第二纠删码编码得到的纠删码数据，则根据所述第一纠删码及所述第二纠删码在所述存储空间中分配所述第一磁盘组，其中，所述第二纠删码的类型不同于所述第一纠删码。
可选地，所述分配模块，用于：
若所述第一纠删码对应的校验块的个数等于所述第二纠删码对应的校验块的个数，且所述第一纠删码对应的数据块的个数等于所述第二纠删码对应的数据块的个数，则将所述第二磁盘组作为所述第一磁盘组；
若所述第一纠删码对应的校验块的个数等于所述第二纠删码对应的校验块的个数，且所述第一纠删码对应的数据块的个数不等于所述第二纠删码对应的数据块的个数，则根据所述第一纠删码在所述存储空间中创建所述第一磁盘组；
若所述第一纠删码对应的校验块的个数大于所述第二纠删码对应的校验块的个数，则在所述第二磁盘组中分配所述第一磁盘组；
若所述第一纠删码对应的校验块的个数小于所述第二纠删码对应的校验块的个数，则备份所述第二纠删码对应的磁盘分配信息，重新分配所述第一磁盘组和所述第二磁盘组。
可选地，所述分配模块，用于对于任一第一磁盘组，依次选择存储空间中的磁盘作为所述第一磁盘组中的一个磁盘，所述第一磁盘组中的磁盘的个数等于所述第一纠删码对应的数据块的个数和校验块的个数之和，多个所述第一磁盘组的起始磁盘之间至少间隔有一个磁盘。
可选地，所述分配模块，用于从所述第二磁盘组中选择目标磁盘组，所述目标磁盘组为一个或多个；从所选择的目标磁盘组中分别选择至少一个磁盘，作为一个第一磁盘组中的磁盘。
可选地，所述分配模块，用于跳过重叠的第二磁盘组，从剩余的第二磁盘组中依次选择目标磁盘组，所述重叠的第二磁盘组是包括在先选择的目标磁盘组中所包括的任一磁盘的第二磁盘组。
可选地，所述分配模块，用于：
在所述存储空间中分配所述第一磁盘组；
若分配所述第一磁盘组失败，则根据备份的所述第二纠删码对应的磁盘分配信息分配第二磁盘组；
若分配所述第一磁盘组成功，则基于所述第一磁盘组和所述第二纠删码，在所述存储空间中分配所述第二磁盘组；
若分配所述第二磁盘组失败，则根据备份的所述第二纠删码对应的磁盘分配信息分配第二磁盘组；
若分配所述第二磁盘组成功，则删除备份的所述第二类型纠删码对应的磁盘分配信息。
另一方面，提供了一种磁盘分配装置，所述装置包括处理器和存储器，所述存储器中存储有至少一条指令，所述指令由所述处理器加载并执行以实现上述任一种磁盘分配方法。
另一方面，提供了一种计算机可读存储介质，该存储介质中存储有至少一条指令，所述指令由处理器加载并执行以实现上述任一种磁盘分配方法。
另一方面，提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行进行上述任一种磁盘分配方法的指令。
本申请实施例提供的技术方案至少包括以下有益效果：
本申请获取待存储的目标数据后，根据该目标数据对应的第一纠删码在存储空间中分配第一磁盘组，以通过第一磁盘组包括的磁盘来存储根据第一纠删码对目标数据进行编码得到的纠删码数据。由于不同类型的纠删码对应不同的参数值，每种类型的纠删码对应的参数值用于指示按照相应纠删码编码得到的数据块的个数和校验块的个数，由此可见，本申请实施例提供的技术方案可根据纠删码类型的不同，来动态调整分配的磁盘，提高了磁盘分配的灵活性。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本申请。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出了本申请实施例提供的一个对象存储系统的架构图；
图2示出了本申请实施例提供的纠删码数据写入磁盘的示意图；
图3示出了本申请实施例提供的一种磁盘分配方法的流程图；
图4示出了本申请实施例提供的一种磁盘分配方法的流程图；
图5示出了本申请实施例提供的磁盘分配装置的结构方框图；
图6示出了本申请实施例提供的磁盘分配装置的结构方框图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
在对本申请进行详细的解释说明之前，先对本申请涉及的应用场景及相关技术予以介绍。
纠删码是一种为了保护数据而对数据进行处理的方法，它将数据分割成数据块，对数据块进行编码得到冗余的校验块，使得数据之间产生关联性，以达到容错的目的。纠删码至少包括两个核心参数，分别为N和M，其含义为N个数据块经过该纠删码编码得到M个校验块，从而得到N+M块纠删码数据。不同类型的纠删码对应不同的参数值，例如，第一纠删码的N的值为N1，第一纠删码的M的值为M1，表示N1个数据块经过第一纠删码编码得到M1个校验块；第二纠删码的N的值为N2，第二纠删码的M的值为M2，表示N2个数据块经过第二纠删码编码得到M2个校验块。
为了保证数据的安全，可以将编码后的纠删码数据中的数据块和校验块分别存储于不同的磁盘，以防止因一个磁盘发生损坏而导致的数据无法恢复。此外，在实际应用中，存储空间中可能存储有不同类型的纠删码数据，例如通过第二纠删码编码得到的纠删码数据和通过第一纠删码编码得到的纠删码数据。本申请所涉及的纠删码数据的磁盘分配方法可以用于在存储空间中进行磁盘分配。
针对基于对象存储技术的存储系统，纠删码数据的多个数据块和校验块可以存储于不同的对象存储设备(Object-based Storage Device，OSD)中的磁盘中，也可以存储于同一OSD的不同磁盘中。以下以纠删码数据存储于对象存储系统的同一OSD的不同磁盘中为例，对本申请所涉及的纠删码数据的磁盘分配方法进行说明。
参见图1，其示出了本申请实施例的一个对象存储系统。如图1所示，该对象存储系统至少包括一个元数据服务器、一个对象存储服务器、一个客户端和多个磁盘。
其中，客户端提供用户交互接口，可以用于获取未经纠删码处理的数据。
对象存储服务器包括磁盘分配组件、EC计算组件和磁盘读写组件。对象存储服务器接收客户端获取到的数据；数据经过EC计算组件处理得到纠删码数据；该纠删码数据以条带(Stripe)的形式存储于多个磁盘中。例如，对于参数为N和M的类型的纠删码数据，一个Stripe由N+M个条带块组成，N表示该Stripe中有N个数据块，M表示该Stripe中有M个校验块，每个数据块或者校验块可以称为一个条带块，磁盘分配组件为该N+M个条带块分配N+M个磁盘；磁盘读写组件将该N+M个条带块分别写入所分配的N+M个磁盘中。
元数据服务器根据对象存储服务器中Stripe的存储情况生成条带(Stripe)信息表，并将Stripe信息表存储于元数据服务器中。Stripe信息表包含但不限于以下信息中的一个或者多个：Stripe的标识(Stripe Identifier，Stripe ID)、条带块对应的硬盘的全球唯一名字(worldwide name，wwn)和纠删码的类型。参见表1，其示出了Stripe信息表的一个示例。
表1Stripe信息表的示例
Figure PCTCN2019116783-appb-000001
示例性地，参见图2，从客户端获取的数据经过EC计算组件处理得到四个数据块和一个校验块，分别为数据块1、数据块2、数据块3、数据块4和校验块1，从而得到一个Stripe，该Stripe包括条带块1、条带块2、条带块3、条带块4和条带块5，分别对应数据块1、数据块2、数据块3、数据块4和校验块1；磁盘分配组件为该Stripe分配磁盘1、磁盘2、磁盘3、磁盘4和磁盘5；磁盘读写组件将条带块1、条带块2、条带块3、条带块4和条带块5分别写入磁盘1、磁盘2、磁盘3、磁盘4和磁盘5中；并将该Stripe的Stripe信息表存储在元数据服务器中。
以下对为纠删码数据(包括数据块和校验块)分配磁盘的方法进行说明。
参见图3，其示出了本申请实施例提供的一种纠删码数据的磁盘分配方法的流程图，以下以该方法由对象存储服务器执行为例进行说明，该方法包括：
步骤301：获取待存储的目标数据。
当有数据需要存储时，客户端向对象存储服务器发送待存储的目标数据，则对象存储服务器由此获取到待存储的目标数据，并触发后续的磁盘分配流程。
步骤302：确定目标数据对应的第一纠删码，第一纠删码用于对目标数据进行编码，得到包括数据块和校验块的纠删码数据；
为了保证目标数据的安全性，在存储目标数据时，将目标数据分割成数据块，采用纠删码对数据块进行编码得到冗余的校验块，使得数据之间产生关联性，以达到容错的目的。在此基础上，通过对数据块及校验块进行存储，来实现对目标数据的存储。其中，由于对数据块进行编码时采用纠删码，纠删码不同，编码得到的校验块也不同，因而为了实现后续的磁盘分配，本申请实施例提供的方法需要先确定目标数据对应的第一纠删码，确定方式可根据实际情况进行限定，本申请实施例对此不加以限定。
步骤303：根据第一纠删码在存储空间中分配第一磁盘组，第一磁盘组为一个或者多个，用于存储纠删码数据。
针对该步骤，考虑到存储空间的容量均衡，本申请实施例提供的方法在根据第一纠删码在存储空间中分配第一磁盘组时，包括但不限于分为如下两种情况：
第一种情况：若存储空间中不存在已分配的磁盘组，则根据第一纠删码在存储空间中创建第一磁盘组；
第二种情况：若存储空间中存在已分配的第二磁盘组，第二磁盘组用于存储根据第二纠删码编码得到的纠删码数据，则根据第一纠删码及第二纠删码在存储空间中分配第一磁盘组，其中，第二纠删码的类型不同于第一纠删码。
可选地，根据第一纠删码及第二纠删码在存储空间中分配第一磁盘组，包括如下四种处理方式中的任一种：
第一种方式：若第一纠删码对应的校验块的个数等于第二纠删码对应的校验块的个数，且第一纠删码对应的数据块的个数等于第二纠删码对应的数据块的个数，则将第二磁盘组作为第一磁盘组；
第二种方式：若第一纠删码对应的校验块的个数等于第二纠删码对应的校验块的个数，且第一纠删码对应的数据块的个数不等于第二纠删码对应的数据块的个数，则根据第一纠删码在存储空间中创建第一磁盘组；
针对该种方式，根据第一纠删码在存储空间中创建第一磁盘组，包括：
对于任一第一磁盘组，依次选择存储空间中的磁盘作为第一磁盘组中的一个磁盘，第一磁盘组中的磁盘的个数等于第一纠删码对应的数据块的个数和校验块的个数之和，多个第一磁盘组的起始磁盘之间至少间隔有一个磁盘。
第三种方式：若第一纠删码对应的校验块的个数大于第二纠删码对应的校验块的个数，则在第二磁盘组中分配第一磁盘组；
在第二磁盘组中分配第一磁盘组，包括：
从第二磁盘组中选择目标磁盘组，目标磁盘组为一个或多个；
从所选择的目标磁盘组中分别选择至少一个磁盘，作为一个第一磁盘组中的磁盘。
进一步地，从第二磁盘组中选择目标磁盘组，包括：
跳过重叠的第二磁盘组，从剩余的第二磁盘组中依次选择目标磁盘组，重叠的第二磁盘组是包括在先选择的目标磁盘组中所包括的任一磁盘的第二磁盘组。
第四种方式：若第一纠删码对应的校验块的个数小于第二纠删码对应的校验块的个数，则备份第二纠删码对应的磁盘分配信息，重新分配第一磁盘组和第二磁盘组。
重新分配第一磁盘组和第二磁盘组，包括：
在存储空间中分配第一磁盘组；
若分配第一磁盘组失败，则根据备份的第二纠删码对应的磁盘分配信息分 配第二磁盘组；
若分配第一磁盘组成功，则基于第一磁盘组和第二纠删码，在存储空间中分配第二磁盘组；
若分配第二磁盘组失败，则根据备份的第二纠删码对应的磁盘分配信息分配第二磁盘组；
若分配第二磁盘组成功，则删除备份的第二类型纠删码对应的磁盘分配信息。
本申请获取待存储的目标数据后，根据该目标数据对应的第一纠删码在存储空间中分配第一磁盘组，以通过第一磁盘组包括的磁盘来存储根据第一纠删码对目标数据进行编码得到的纠删码数据。由于不同类型的纠删码对应不同的参数值，每种类型的纠删码对应的参数值用于指示按照相应纠删码编码得到的数据块的个数和校验块的个数，由此可见，本申请实施例提供的技术方案可根据纠删码类型的不同，来动态调整分配的磁盘，提高了磁盘分配的灵活性。
此外，通过根据纠删码来分配磁盘组，可使得纠删码数据对应的数据块和校验块能够相对均匀地存储于该存储空间的磁盘中，提高了存储空间的利用率。
为了便于理解，在上述实施例基础上，参见图4，其示出了本申请实施例提供的一种磁盘分配方法的流程图，以下以该方法由对象存储服务器执行为例进行说明，该方法包括：
步骤401、在存储空间中分配第二纠删码对应的第二磁盘组。
其中，该存储空间包括X个磁盘；第二纠删码用于基于N2个数据块生成M2个校验块；第二磁盘组用于存储通过第二纠删码编码得到的N2个数据块和M2个校验块，该存储空间中包括至少一个第二磁盘组，每个第二磁盘组包括存储空间的X个磁盘中的N2+M2个磁盘，X、N2和M2为正整数，且X不小于N2+M2。
在一种可能的实现方式中，步骤401包括：
依次选择存储空间的X个磁盘中的N2+M2个磁盘作为一个第二磁盘组中的磁盘；
按照该依次选择的方式分配多个第二磁盘组，其中，多个第二磁盘组的起始磁盘之间间隔K1个磁盘，K1为非负整数，且K1小于X。
此外，在分配第二磁盘组的过程中，若起始磁盘与到存储空间中第X个磁 盘(即最后一个磁盘)的磁盘的个数小于N2+M2，则可以轮转到该存储空间中的第一个磁盘继续分配第二磁盘组，例如，一个第二磁盘组的起始磁盘为存储空间中的第X-1个磁盘，则该第二磁盘组包括的磁盘为：第X-1个磁盘、第X个磁盘、第一个磁盘、……、第N2+M2-2个磁盘。
示例性地，一个前述的OSD中包括12个磁盘，即X的值为12，磁盘编号分别是D1～D12；第二纠删码的N2的值为2，M2的值为1，则按照上述实现方式分配得到的多个第二磁盘组所包括的磁盘如表2所示。其中，每个第二磁盘组包括N2+M2＝3个磁盘，该3个磁盘是12个磁盘中顺次的3个磁盘，且序号为1～12的12个第二磁盘组的12个起始磁盘之间间隔1个磁盘，即K1的值为1。例如，序号为1的第二磁盘组的起始磁盘D1与序号为2的第二磁盘组的起始磁盘D2之间间隔1个磁盘。
表2第二磁盘组的分配示例
Figure PCTCN2019116783-appb-000002
步骤402、获取待存储的目标数据，确定目标数据对应的第一纠删码，根据第一纠删码在存储空间中分配第一磁盘组。
其中，第一磁盘组用于存储根据第一纠删码对目标数据编码后得到的纠删码数据中的数据块和校验块，例如，第一纠删码用于基于N1个数据块生成M1个校验块。该存储空间中包括至少一个该第一磁盘组，每个第一磁盘组包括存 储空间的X个磁盘中的N1+M1个磁盘，N1和M1为正整数，且X不小于N1+M1。可选地，若X小于N1+M1，则分配第一磁盘组失败。
在一种可能的实现方式中，基于存储空间中已经分配的磁盘组分配第一磁盘组，若该存储空间中存在多个纠删码对应的磁盘组，则可以选择该多个纠删码中其参数M最小且(N+M)最小的纠删码，并基于该纠删码对应的磁盘组分配第一磁盘组。
以下以该存储空间中存在有第二纠删码对应的第二磁盘组，并基于第二磁盘组分配第一磁盘组为例进行说明，根据第一纠删码和第二纠删码的参数，可能包括如下情况。
情况一：若M1大于M2，且ceil(X/(N2+M2)*M2)≥(N1+M1)，则根据M1、N1、M2和N2在存储空间中分配第一磁盘组。
以下针对情况一，对第一磁盘组的分配示例进行说明。
在第一示例中，分配一个第一磁盘组包括以下步骤：
步骤A、选择一个第二磁盘组中的一个磁盘作为基准磁盘(baseDisk)，该baseDisk在存储空间中的编号为Db，并将该baseDisk作为该第一磁盘组中的第一个磁盘。例如，可以选择编号为1的磁盘作为该第一磁盘组的baseDisk。若M2小于2，则步骤A结束；若M2不小于2，则在baseDisk所在的第二磁盘组中，依次选择baseDisk之后的、编号为Db+1至Db+M2-1的M2-1个磁盘作为该第一磁盘组中的第2个至第M2个磁盘，步骤A结束。重复执行步骤B，直至该第一磁盘组中的磁盘个数为M1+N1。
步骤B、选择磁盘编号为Db+N2+M2的磁盘作为该第一磁盘组中的第M2+1个磁盘。若M2小于2，则步骤B结束；若M2不小于2，则依次选择编号为Db+N2+M2+1至Db+N2+2*M2-1的磁盘作为该第一磁盘组中的磁盘，步骤B结束。其中，若编号为Db+N2+M2+1至Db+N2+2*M2-1中的任一值大于X，则将该大于X的值减去X，并以此更新后的编号值选择磁盘。例如，X＝12，Db+N2+M2+1＝15，则将编号为Db+N2+M2+1-12＝3的磁盘作为该第一磁盘组中的磁盘。在执行步骤B的过程中，若该第一磁盘组中的磁盘个数达到M1+N1，则步骤B结束。
在第一示例中，通过步骤A和B分配得到一个第一磁盘组，还可以通过步骤A和B得到其他第一磁盘组，在分配得到其他第一磁盘组的过程中，所选择的baseDisk与已经分配的一个或者多个第一磁盘组的baseDisk不同。例如，可 以选择编号为2的磁盘作为该其他第一磁盘组的baseDisk，并依次执行步骤A和B分配得到该其他第一磁盘组。
此外，在通过步骤A和B分配得到其他第一磁盘组后，若该其他第一磁盘组与已经分配的一个或者多个第一磁盘组相同，则基于第二磁盘组分配第一磁盘组结束。例如，某次通过步骤A和B分配得到的第一磁盘组包括编号为4、7、10和1的四个磁盘，该第一磁盘组与已经分配的一个第一磁盘组(包括编号为1、4、7和10的磁盘)相同，则分配第一磁盘组结束。
在第二示例中，首先选择第二纠删码对应的第二磁盘组中的P个第二磁盘组，并从所选择的各个第二磁盘组中依次分别选择M2个磁盘，作为一个第一磁盘组中的磁盘，直至该第一磁盘组包括N1+M1个磁盘，其中P＝(N1+M1)/M2。按照该分配方式为多个第一磁盘组分配磁盘。在该示例中，分配一个第一磁盘组包括以下步骤：
步骤一、在为一个第一磁盘组分配磁盘的过程中，首先从第二纠删码对应的第二磁盘组中选择一个第二磁盘组，再顺次选择下一个第二磁盘组，若下一个第二磁盘组是重叠的第二磁盘组，则跳过该重叠的第二磁盘组，从重叠的第二磁盘组之后继续选择第二磁盘组，直至得到P个第二磁盘组。其中，重叠的第二磁盘组为包括在先选择的第二磁盘组中所包括的任一磁盘的第二磁盘组。P需要满足的条件为：P＝(N1+M1)/M2。
例如，在分配第一磁盘组的过程中，已经选择了表2中编号为1的第二磁盘组，则跳过包括磁盘编号为D1、D2或者D3的磁盘的第二磁盘组，即跳过编号为2和3的第二磁盘组，从编号为4的第二磁盘组继续为分配该第一磁盘组。
步骤二、从步骤一已选择的多个第二磁盘组中分别选择M2个磁盘，作为一个第一磁盘组中的磁盘，直至该第一磁盘组包括N1+M1个磁盘。例如，从已选择的第二磁盘组中依次选择M2个磁盘，选择过程中的起始磁盘为第H1个磁盘，H1为非负整数。对于从已选择的多个第二磁盘组中选择磁盘，起始磁盘的编号H1的值可以相同，也可以不同。
根据步骤一和步骤二，可以为一个第一磁盘组分配磁盘，在为另一个第一磁盘组分配磁盘的过程中，则可以从步骤一中已选择的多个第二磁盘组中分别选择另外M2个磁盘，作为该另一个第一磁盘组中的磁盘，直至该第一磁盘组包括N1+M1个磁盘，该另外M2个磁盘与步骤二中所选择的M2个磁盘不重叠。例如，从已选择的第二磁盘组中依次选择M2个磁盘，选择过程中的起始磁盘为 第H2个磁盘，H2为非负整数，H2的值与H1的值不同，且H2与H1之差为M2，以保证该另外M2个磁盘与步骤二中所选择的M2个磁盘不重叠。
示例性地，若N1＝2，M1＝2，基于表1所示的第二磁盘组分配第一磁盘组得到的结果如表3所示。其中每个第一磁盘组包括N1+M1＝4个磁盘，该4个磁盘是从表2中的多个第二磁盘组中选择4个第二磁盘组，并分别从该4个第二磁盘组中选择M2＝1个磁盘，从而得到组成一个第一磁盘组的4个磁盘。例如，在为编号为1的第一磁盘组选择第二磁盘组的过程中，首选选择编号为1的第二磁盘组，从中选择编号为D1的磁盘作为编号为1的第一磁盘组的第一个磁盘；由于编号为1的第二磁盘组还包括编号为D2和D3的磁盘，则跳过包括编号为D1、D2或者D3的磁盘的编号为2和3的第二磁盘组，选择编号为4的第二磁盘组，并从中选择编号为D4的磁盘作为编号为1的第一磁盘组的第二个磁盘；依次类推，分配多个第一磁盘组。可见，第一磁盘组和第二磁盘组的磁盘较为均匀地分布于存储空间中，使得第二纠删码的数据和第一纠删码的数据能够较为均匀地分布于存储空间中，提高存储空间的利用率
表3第二磁盘组的分配示例
Figure PCTCN2019116783-appb-000003
进一步地，根据步骤一选择的多个第二磁盘组，并重复步骤二从已选择的多个第二磁盘组中选择磁盘并分配第一磁盘组，若已选择的多个第二磁盘组中的磁盘都被选择作为多个第一磁盘组中的磁盘，则在存储空间中向后寻找没有分配为第一磁盘组的第二磁盘组，以该第二磁盘组开始，重复步骤一和步骤二，再分配其他第一磁盘组。
例如，存储空间中包括48个磁盘，其中包括48个第二磁盘组，则基于前12个第二磁盘组得到3个第一磁盘组，如表3所示，前12个第二磁盘组中的磁盘都被分配为第一磁盘组，则可以从第13个第二磁盘组开始，继续分配第4个第一磁盘组。
若所有的第一磁盘组中的磁盘都分配为第二磁盘组的磁盘，则分配第二磁盘组结束。
情况二：若M1等于M2，且N1等于N2，则多个第一磁盘组包括的磁盘与第二磁盘组所包括的磁盘相同，即，为第一磁盘组分配与第二磁盘组相同的磁盘。
情况三：若M1等于M2，且N1不等于N2，则按照步骤401中所述的方法为分配第一磁盘组。
情况四：若M1小于M2，则备份第二纠删码对应的磁盘分配信息，并重新分配第一磁盘组和第二磁盘组。
示例性地，按照步骤401中所述的方法首先分配第一磁盘组，若分配第一磁盘组失败，则根据备份的第二纠删码对应的磁盘分配信息在存储空间的X个磁盘中还原第二磁盘组；若分配第一磁盘组成功，则再按照步骤402中情况一所示的方法基于第一磁盘组分配第二磁盘组，若分配第二磁盘组失败，则根据备份的第二纠删码对应的磁盘分配信息在存储空间的X个磁盘中还原第二磁盘组；若分配第二磁盘组成功，则删除备份的第二纠删码对应的磁盘分配信息。
可选地，分配第一磁盘组失败可能由于X小于N1+M1；分配第二磁盘组失败可能由于X小于N2+M2。
此外，元数据服务器还可以根据对象存储服务器在步骤401和402中所分配的第一磁盘组和第二磁盘组分别生成Stripe信息表。并将生成的Stripe信息表存储于元数据服务器中。
本申请获取待存储的目标数据后，根据该目标数据对应的第一纠删码在存储空间中分配第一磁盘组，以通过第一磁盘组包括的磁盘来存储根据第一纠删码对目标数据进行编码得到的纠删码数据。由于不同类型的纠删码对应不同的参数值，每种类型的纠删码对应的参数值用于指示按照相应纠删码编码得到的数据块的个数和校验块的个数，由此可见，本申请实施例提供的技术方案可根据纠删码类型的不同，来动态调整分配的磁盘，提高了磁盘分配的灵活性。
此外，通过根据纠删码来分配磁盘组，可使得纠删码数据对应的数据块和校验块能够相对均匀地存储于该存储空间的磁盘中，提高了存储空间的利用率。
下述为本申请装置实施例，对于装置实施例中未详尽描述的细节，可以参考上述方法实施例。
请参考图5，其示出了本申请一个实施例提供的纠删码数据的磁盘分配装置500的结构方框图。该装置包括：获取模块510、确定模块520和分配模块530。
该获取模块510，用于获取待存储的目标数据。
该确定模块520，用于确定目标数据对应的第一纠删码，第一纠删码用于对目标数据进行编码，得到包括数据块和校验块的纠删码数据，不同类型的纠删码对应不同的参数值，每种类型的纠删码对应的参数值用于指示按照相应纠删码编码得到的数据块的个数和校验块的个数。
该分配模块530，用于根据第一纠删码在存储空间中分配第一磁盘组，第一磁盘组为一个或者多个，用于存储纠删码数据。
可选地，该分配模块530用于：
若存储空间中不存在已分配的磁盘组，则根据第一纠删码在存储空间中创建第一磁盘组；
若存储空间中存在已分配的第二磁盘组，第二磁盘组用于存储根据第二纠删码编码得到的纠删码数据，则根据第一纠删码及第二纠删码在存储空间中分配第一磁盘组，其中，第二纠删码的类型不同于第一纠删码。
可选地，该分配模块530用于：
若第一纠删码对应的校验块的个数等于第二纠删码对应的校验块的个数，且第一纠删码对应的数据块的个数等于第二纠删码对应的数据块的个数，则将第二磁盘组作为第一磁盘组；
若第一纠删码对应的校验块的个数等于第二纠删码对应的校验块的个数，且第一纠删码对应的数据块的个数不等于第二纠删码对应的数据块的个数，则根据第一纠删码在存储空间中创建第一磁盘组；
若第一纠删码对应的校验块的个数大于第二纠删码对应的校验块的个数，则在第二磁盘组中分配第一磁盘组；
若第一纠删码对应的校验块的个数小于第二纠删码对应的校验块的个数，则备份第二纠删码对应的磁盘分配信息，重新分配第一磁盘组和第二磁盘组。
可选地，该分配模块530用于：
对于任一第一磁盘组，依次选择存储空间中的磁盘作为第一磁盘组中的一个磁盘，第一磁盘组中的磁盘的个数等于第一纠删码对应的数据块的个数和校验块的个数之和，多个第一磁盘组的起始磁盘之间至少间隔有一个磁盘。
可选地，该分配模块530用于：
从第二磁盘组中选择目标磁盘组，目标磁盘组为一个或多个；从所选择的目标磁盘组中分别选择至少一个磁盘，作为一个第一磁盘组中的磁盘。
可选地，该分配模块530，用于：
跳过重叠的第二磁盘组，从剩余的第二磁盘组中依次选择目标磁盘组，该重叠的第二磁盘组是包括在先选择的目标磁盘组中所包括的任一磁盘的第二磁盘组。
可选地，该分配模块530用于：
在存储空间中分配第一磁盘组；
若分配第一磁盘组失败，则根据备份的第二纠删码对应的磁盘分配信息分配第二磁盘组；
若分配第一磁盘组成功，则基于第一磁盘组和第二纠删码，在存储空间中分配第二磁盘组；
若分配第二磁盘组失败，则根据备份的第二纠删码对应的磁盘分配信息分配第二磁盘组；
若分配第二磁盘组成功，则删除备份的第二类型纠删码对应的磁盘分配信息。
获取待存储的目标数据后，根据该目标数据对应的第一纠删码在存储空间中分配第一磁盘组，以通过第一磁盘组包括的磁盘来存储根据第一纠删码对目标数据进行编码得到的纠删码数据。由于不同类型的纠删码对应不同的参数值，每种类型的纠删码对应的参数值用于指示按照相应纠删码编码得到的数据块的个数和校验块的个数，由此可见，本申请实施例提供的技术方案可根据纠删码类型的不同，来动态调整分配的磁盘，提高了磁盘分配的灵活性。
此外，通过根据纠删码来分配磁盘组，可使得纠删码数据对应的数据块和校验块能够相对均匀地存储于该存储空间的磁盘中，提高了存储空间的利用率。
参见图6，其示出了本申请实施例提供的一种磁盘分配装置的结构示意图。该装置可能是服务器或者终端，具体来讲：
计算系统600包括中央处理单元(CPU)601、包括随机存取存储器(RAM)602和只读存储器(ROM)603的系统存储器604，以及连接系统存储器604和中央处理单元601的系统总线605。计算系统600还包括帮助计算机内的各个器件之间传输信息的基本输入/输出系统(I/O系统)606，和用于存储操作系统613、应用程序614和其他程序模块615的大容量存储设备607。
基本输入/输出系统606包括有用于显示信息的显示器608和用于用户输入 信息的诸如鼠标、键盘之类的输入设备609。其中显示器608和输入设备609都通过连接到系统总线605的输入输出控制器610连接到中央处理单元601。基本输入/输出系统606还可以包括输入输出控制器610以用于接收和处理来自键盘、鼠标、或电子触控笔等多个其他设备的输入。类似地，输入输出控制器610还提供输出到显示屏、打印机或其他类型的输出设备。
大容量存储设备607通过连接到系统总线605的大容量存储控制器(未示出)连接到中央处理单元601。大容量存储设备607及其相关联的计算机可读介质为计算系统600提供非易失性存储。也就是说，大容量存储设备607可以包括诸如硬盘或者CD-ROM驱动器之类的计算机可读介质(未示出)。
不失一般性，计算机可读介质可以包括计算机存储介质和通信介质。计算机存储介质包括以用于存储诸如计算机可读指令、数据结构、程序模块或其他数据等信息的任何方法或技术实现的易失性和非易失性、可移动和不可移动介质。计算机存储介质包括RAM、ROM、EPROM、EEPROM、闪存或其他固态存储其技术，CD-ROM、DVD或其他光学存储、磁带盒、磁带、磁盘存储或其他磁性存储设备。当然，本领域技术人员可知计算机存储介质不局限于上述几种。上述的系统存储器604和大容量存储设备607可以统称为存储器。
根据本申请的各种实施例，计算系统600还可以通过诸如因特网等网络连接到网络上的远程计算机运行。也即计算系统600可以通过连接在系统总线605上的网络接口单元611连接到网络612，或者说，也可以使用网络接口单元611来连接到其他类型的网络或远程计算机系统(未示出)。
上述存储器还包括一个或者一个以上的程序，一个或者一个以上程序存储于存储器中，被配置由CPU执行。所述一个或者一个以上程序包含用于进行图3和图4其中任一所提供的纠删码数据的磁盘分配方法的指令。
本申请实施例还提供了一种非临时性计算机可读存储介质，当所述存储介质中的指令由计算系统的处理器执行时，使得计算系统能够执行图3和图4其中任一提供的纠删码数据的磁盘分配方法。
一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行进行图3和图4其中任一提供的纠删码数据的磁盘分配方法的指令。
应当理解的是，在本文中提及的“多个”是指两个或两个以上。“和/或”，描述关联对象的关联关系，表示可以存在三种关系。字符“/”一般表示前后关联对 象是一种“或”的关系。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
以上所述仅为本申请的示例性实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (16)

  1. 一种磁盘分配方法，其特征在于，所述方法包括：
    获取待存储的目标数据；
    确定所述目标数据对应的第一纠删码，所述第一纠删码用于对所述目标数据进行编码，得到包括数据块和校验块的纠删码数据，不同类型的纠删码对应不同的参数值，每种类型的纠删码对应的参数值用于指示按照相应纠删码编码得到的数据块的个数和校验块的个数；
    根据所述第一纠删码在存储空间中分配第一磁盘组，所述第一磁盘组为一个或者多个，用于存储所述纠删码数据。
  2. 根据权利要求1所述的方法，其特征在于，所述根据所述第一纠删码在存储空间中分配第一磁盘组，包括：
    若所述存储空间中不存在已分配的磁盘组，则根据所述第一纠删码在所述存储空间中创建所述第一磁盘组；
    若所述存储空间中存在已分配的第二磁盘组，所述第二磁盘组用于存储根据第二纠删码编码得到的纠删码数据，则根据所述第一纠删码及所述第二纠删码在所述存储空间中分配所述第一磁盘组，其中，所述第二纠删码的类型不同于所述第一纠删码。
  3. 根据权利要求2所述的方法，其特征在于，所述根据所述第一纠删码及所述第二纠删码在所述存储空间中分配所述第一磁盘组，包括：
    若所述第一纠删码对应的校验块的个数等于所述第二纠删码对应的校验块的个数，且所述第一纠删码对应的数据块的个数等于所述第二纠删码对应的数据块的个数，则将所述第二磁盘组作为所述第一磁盘组；
    若所述第一纠删码对应的校验块的个数等于所述第二纠删码对应的校验块的个数，且所述第一纠删码对应的数据块的个数不等于所述第二纠删码对应的数据块的个数，则根据所述第一纠删码在所述存储空间中创建所述第一磁盘组；
    若所述第一纠删码对应的校验块的个数大于所述第二纠删码对应的校验块的个数，则基于所述第二磁盘组分配所述第一磁盘组；
    若所述第一纠删码对应的校验块的个数小于所述第二纠删码对应的校验块 的个数，则备份所述第二纠删码对应的磁盘分配信息，重新分配所述第一磁盘组和所述第二磁盘组。
  4. 根据权利要求2或3所述的方法，其特征在于，所述根据所述第一纠删码在所述存储空间中创建所述第一磁盘组，包括：
    对于任一第一磁盘组，依次选择存储空间中的磁盘作为所述第一磁盘组中的一个磁盘，所述第一磁盘组中的磁盘的个数等于所述第一纠删码对应的数据块的个数和校验块的个数之和，多个所述第一磁盘组的起始磁盘之间至少间隔有一个磁盘。
  5. 根据权利要求3所述的方法，其特征在于，所述基于所述第二磁盘组分配所述第一磁盘组，包括：
    从所述第二磁盘组中选择目标磁盘组，所述目标磁盘组为一个或多个；
    从所选择的目标磁盘组中分别选择至少一个磁盘，作为一个第一磁盘组中的磁盘。
  6. 根据权利要求5所述的方法，其特征在于，所述从所述第二磁盘组中选择目标磁盘组，包括：
    跳过重叠的第二磁盘组，从剩余的第二磁盘组中依次选择所述目标磁盘组，所述重叠的第二磁盘组是包括在先选择的目标磁盘组中所包括的任一磁盘的第二磁盘组。
  7. 根据权利要求3所述的方法，其特征在于，所述重新分配所述第一磁盘组和所述第二磁盘组，包括：
    在所述存储空间中分配所述第一磁盘组；
    若分配所述第一磁盘组失败，则根据备份的所述第二纠删码对应的磁盘分配信息分配第二磁盘组；
    若分配所述第一磁盘组成功，则基于所述第一磁盘组和所述第二纠删码，在所述存储空间中分配所述第二磁盘组；
    若分配所述第二磁盘组失败，则根据备份的所述第二纠删码对应的磁盘分 配信息分配第二磁盘组；
    若分配所述第二磁盘组成功，则删除备份的所述第二类型纠删码对应的磁盘分配信息。
  8. 一种磁盘分配装置，其特征在于，所述装置包括：
    获取模块，用于获取待存储的目标数据；
    确定模块，用于确定所述目标数据对应的第一纠删码，所述第一纠删码用于对所述目标数据进行编码，得到包括数据块和校验块的纠删码数据，不同类型的纠删码对应不同的参数值，每种类型的纠删码对应的参数值用于指示按照相应纠删码编码得到的数据块的个数和校验块的个数；
    分配模块，用于根据所述第一纠删码在存储空间中分配第一磁盘组，所述第一磁盘组为一个或者多个，用于存储所述纠删码数据。
  9. 根据权利要求8所述的装置，其特征在于，所述分配模块，用于：
    若所述存储空间中不存在已分配的磁盘组，则根据所述第一纠删码在所述存储空间中创建所述第一磁盘组；
    若所述存储空间中存在已分配的第二磁盘组，所述第二磁盘组用于存储根据第二纠删码编码得到的纠删码数据，则根据所述第一纠删码及所述第二纠删码在所述存储空间中分配所述第一磁盘组，其中，所述第二纠删码的类型不同于所述第一纠删码。
  10. 根据权利要求9所述的装置，其特征在于，所述分配模块，用于：
    若所述第一纠删码对应的校验块的个数等于所述第二纠删码对应的校验块的个数，且所述第一纠删码对应的数据块的个数等于所述第二纠删码对应的数据块的个数，则将所述第二磁盘组作为所述第一磁盘组；
    若所述第一纠删码对应的校验块的个数等于所述第二纠删码对应的校验块的个数，且所述第一纠删码对应的数据块的个数不等于所述第二纠删码对应的数据块的个数，则根据所述第一纠删码在所述存储空间中创建所述第一磁盘组；
    若所述第一纠删码对应的校验块的个数大于所述第二纠删码对应的校验块的个数，则在所述第二磁盘组中分配所述第一磁盘组；
    若所述第一纠删码对应的校验块的个数小于所述第二纠删码对应的校验块的个数，则备份所述第二纠删码对应的磁盘分配信息，重新分配所述第一磁盘组和所述第二磁盘组。
  11. 根据权利要求9或10所述的装置，其特征在于，所述分配模块，用于对于任一第一磁盘组，依次选择存储空间中的磁盘作为所述第一磁盘组中的一个磁盘，所述第一磁盘组中的磁盘的个数等于所述第一纠删码对应的数据块的个数和校验块的个数之和，多个所述第一磁盘组的起始磁盘之间至少间隔有一个磁盘。
  12. 根据权利要求10所述的装置，其特征在于，所述分配模块，用于从所述第二磁盘组中选择目标磁盘组，所述目标磁盘组为一个或多个；从所选择的目标磁盘组中分别选择至少一个磁盘，作为一个第一磁盘组中的磁盘。
  13. 根据权利要求12所述的装置，其特征在于，所述分配模块，用于跳过重叠的第二磁盘组，从剩余的第二磁盘组中依次选择所述目标磁盘组，所述重叠的第二磁盘组是包括在先选择的目标磁盘组中所包括的任一磁盘的第二磁盘组。
  14. 根据权利要求10所述的装置，其特征在于，所述分配模块，用于：
    在所述存储空间中分配所述第一磁盘组；
    若分配所述第一磁盘组失败，则根据备份的所述第二纠删码对应的磁盘分配信息分配第二磁盘组；
    若分配所述第一磁盘组成功，则基于所述第一磁盘组和所述第二纠删码，在所述存储空间中分配所述第二磁盘组；
    若分配所述第二磁盘组失败，则根据备份的所述第二纠删码对应的磁盘分配信息分配第二磁盘组；
    若分配所述第二磁盘组成功，则删除备份的所述第二类型纠删码对应的磁盘分配信息。
  15. 一种磁盘分配装置，其特征在于，所述装置包括处理器和存储器，所述存储器中存储有至少一条指令，所述指令由所述处理器加载并执行以实现如权利要求1至7任一项所述的磁盘分配方法。
  16. 一种计算机可读存储介质，其特征在于，所述存储介质中存储有至少一条指令，所述指令由处理器加载并执行以实现如权利要求1至7任一项所述的磁盘分配方法。
PCT/CN2019/116783 2018-11-09 2019-11-08 磁盘分配方法、装置和可读存储介质 WO2020094134A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811332967.4A CN111176880B (zh) 2018-11-09 2018-11-09 磁盘分配方法、装置和可读存储介质
CN201811332967.4 2018-11-09

Publications (1)

Publication Number Publication Date
WO2020094134A1 true WO2020094134A1 (zh) 2020-05-14

Family

ID=70611685

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/116783 WO2020094134A1 (zh) 2018-11-09 2019-11-08 磁盘分配方法、装置和可读存储介质

Country Status (2)

Country Link
CN (1) CN111176880B (zh)
WO (1) WO2020094134A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113918378A (zh) * 2020-07-10 2022-01-11 华为技术有限公司 数据存储方法、存储系统、存储设备及存储介质
CN116010160A (zh) * 2023-03-09 2023-04-25 安超云软件有限公司 分布式文件系统的纠删码数据分布的方法及应用

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140040702A1 (en) * 2012-07-31 2014-02-06 International Business Machines Corporation Managing a storage array
CN104598162A (zh) * 2013-10-31 2015-05-06 国际商业机器公司 用于跨存储设备写入数据的方法和系统
CN106708651A (zh) * 2016-11-16 2017-05-24 北京三快在线科技有限公司 基于纠删码的部分写入方法及装置、存储介质、设备
CN107943421A (zh) * 2017-11-30 2018-04-20 成都华为技术有限公司 一种基于分布式存储系统的分区划分方法及装置
CN108334419A (zh) * 2017-12-25 2018-07-27 华为技术有限公司 一种数据恢复的方法和装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104503706B (zh) * 2014-12-23 2017-10-10 中国科学院计算技术研究所 一种基于磁盘阵列的数据存储及读取方法
WO2016058289A1 (zh) * 2015-01-20 2016-04-21 北京大学深圳研究生院 一种能修复多个节点失效的mds纠删码

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140040702A1 (en) * 2012-07-31 2014-02-06 International Business Machines Corporation Managing a storage array
CN104598162A (zh) * 2013-10-31 2015-05-06 国际商业机器公司 用于跨存储设备写入数据的方法和系统
CN106708651A (zh) * 2016-11-16 2017-05-24 北京三快在线科技有限公司 基于纠删码的部分写入方法及装置、存储介质、设备
CN107943421A (zh) * 2017-11-30 2018-04-20 成都华为技术有限公司 一种基于分布式存储系统的分区划分方法及装置
CN108334419A (zh) * 2017-12-25 2018-07-27 华为技术有限公司 一种数据恢复的方法和装置

Also Published As

Publication number Publication date
CN111176880B (zh) 2021-08-13
CN111176880A (zh) 2020-05-19

Similar Documents

Publication Publication Date Title
US11048416B2 (en) Method, apparatus and computer program product for managing storage system
US10146624B1 (en) Disk extent rebalancing in mapped RAID storage arrays
EP3208714B1 (en) Data reconstruction method, apparatus and system in distributed storage system
CN107807794B (zh) 一种数据存储方法和装置
US10977124B2 (en) Distributed storage system, data storage method, and software program
JP6346565B2 (ja) ディスク記憶に消去符号化データを割り当てるための方法および装置
US10146447B1 (en) Mapped RAID (redundant array of independent disks) in a data storage system with drive extents allocated to individual RAID extents from individual sub-groups of storage made up of ranges of logical block addresses defined across a group of hard disk drives
US20180011766A1 (en) Disaster recovery systems and methods
US9678680B1 (en) Forming a protection domain in a storage architecture
WO2020000817A1 (zh) 一种归置组所属硬盘分配方法、系统、装置及存储介质
US10664392B2 (en) Method and device for managing storage system
US20210117296A1 (en) Method, device, and computer program product for managing storage system
CN110096218B (zh) 用于为存储对象提供raid数据保护的方法、系统和介质
US11074130B2 (en) Reducing rebuild time in a computing storage environment
US10761761B2 (en) Method and device for managing storage system
WO2020094134A1 (zh) 磁盘分配方法、装置和可读存储介质
US20190347165A1 (en) Apparatus and method for recovering distributed file system
CN112732168B (zh) 用于管理存储系统的方法、设备和计算机程序产品
CN114385409A (zh) 基于纠删码的编码方法、分布式系统、设备及存储介质
CN109213429B (zh) 存储管理方法和设备
US10168937B2 (en) Storage space allocation
EP3846017A1 (en) Resource allocation method and device for hard disks in distributed storage system
CN111857535A (zh) 用于存储管理的方法、电子设备和计算机可读存储介质
US10423506B1 (en) Fast rebuild using layered RAID
CN108932176B (zh) 数据降级存储方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19881572

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19881572

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 19881572

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 03.02.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 19881572

Country of ref document: EP

Kind code of ref document: A1