CN112667156A - Method and device for realizing virtualization raid - Google Patents

Method and device for realizing virtualization raid Download PDF

Info

Publication number
CN112667156A
CN112667156A CN202011563169.XA CN202011563169A CN112667156A CN 112667156 A CN112667156 A CN 112667156A CN 202011563169 A CN202011563169 A CN 202011563169A CN 112667156 A CN112667156 A CN 112667156A
Authority
CN
China
Prior art keywords
physical hard
disk
disks
storage pool
pgs
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011563169.XA
Other languages
Chinese (zh)
Inventor
董文祥
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Innovation Technology Co ltd
Original Assignee
Shenzhen Innovation Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Innovation Technology Co ltd filed Critical Shenzhen Innovation Technology Co ltd
Priority to CN202011563169.XA priority Critical patent/CN112667156A/en
Publication of CN112667156A publication Critical patent/CN112667156A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a method and a device for realizing virtualization raid, wherein the method comprises the following steps: dividing a physical hard disk into a first preset number of virtual disks, and numbering the virtual disks; sequentially numbering the physical hard disks of a second preset number, and selecting the physical hard disks of not less than a third preset number to form a storage pool; selecting a fourth preset number of physical hard disks from the storage pool to form a pg, and finishing the initialization of the pg; and taking out a preset logical volume lv from the storage pool, selecting the maximum pg in the idle sub-pgs from the pgs, returning the sub-pgs, degrading the fifth preset number of pgs when the physical hard disk is damaged, reconstructing the degraded pgs, and reconstructing data to the residual space of each physical hard disk in the storage pool. The invention can obviously improve the reconstruction performance and shorten the reconstruction time.

Description

Method and device for realizing virtualization raid
Technical Field
The invention relates to the technical field of hard disk data, in particular to a method and a device for realizing a virtualized raid.
Background
The capacity of a single disk of a current magnetic disk is larger and is up to 20TB, the traditional Raid5 and Raid6 need to be rebuilt when the disk is lost and damaged, the rebuilding time is very long, for example, Raid5 is used, a 20TB hard disk forms Raid5, 1 damaged disk needs to be rebuilt, the rebuilding speed is assumed to be 100MB/s, and the rebuilding time is 58.98 hours.
The detailed calculation method is as follows, 20 ^ 1000/1024 ^4 ^ 1024 ^ 1024/100/3600 ^ 58.98h
During the Raid group reconstruction, the Raid group is in a degraded state, which will cause the Raid group to be corrupted if a bad disk occurs again. The reconstruction time should be as short as possible. Simply increasing the disk reconstruction writing speed will affect the external service performance of the disk set, which is not preferable.
Disclosure of Invention
The embodiment of the specification provides a method and a device for realizing virtualization raid.
In one aspect, an implementation method for virtualization raid provided in an embodiment of the present specification includes: dividing a physical hard disk into a first preset number of virtual disks, and numbering the virtual disks; sequentially numbering the physical hard disks of a second preset number, and selecting the physical hard disks of not less than a third preset number to form a storage pool; selecting a fourth preset number of physical hard disks from the storage pool to form a pg, and finishing the initialization of the pg; taking out a preset logical volume lv from the storage pool, selecting a maximum free sub-pg from the pg by adopting a simplified configuration mode, and returning the sub-pg, wherein a single sub-pg is a basic raid storage unit, and 7 virtual disks in the sub-pg are respectively from 7 different physical hard disks; when the physical hard disks are damaged, degrading a fifth preset number of pg, reconstructing the degraded pg, and reconstructing data to the residual space of each physical hard disk in the storage pool; after reconstruction is completed, the hot standby disk is added to the storage pool, data in part of pg is copied by taking sd as a unit, switching of pg data units is completed after copying is completed, and reserved spaces of all physical hard disks are recovered.
On the other hand, an implementation apparatus for virtualizing raid provided in an embodiment of the present specification includes: the physical hard disk space dividing module is used for dividing physical hard disks into a first preset number of virtual disks and numbering the virtual disks; the storage pool generating module is used for sequentially numbering the physical hard disks with the second preset number and selecting the physical hard disks with no less than a third preset number from the physical hard disks to form a storage pool; a pg generating and initializing module, configured to select a fourth predetermined number of physical hard disks from the storage pool to form a pg, and complete initialization of the pg; a sub-pg returning module, configured to take out a predetermined logical volume lv from the storage pool, select a maximum pg of idle sub-pgs from the pgs in a simplified configuration manner, and return the sub-pgs, where a single sub-pg is a basic raid storage unit, and 7 virtual disks in the sub-pgs are from 7 different physical hard disks respectively; the rebuilding module is used for degrading a fifth preset number of pg when the physical hard disk is damaged, rebuilding the degraded pg, and rebuilding data to the residual space of each physical hard disk in the storage pool; after reconstruction is completed, the hot standby disk is added to the storage pool, data in part of pg is copied by taking the virtual disk as a unit, switching of pg data units is completed after copying is completed, and reserved spaces of all the physical hard disks are recovered.
The invention can obviously improve the reconstruction performance and shorten the reconstruction time.
Drawings
FIG. 1 is a flow diagram of a method for implementing a virtualized raid according to some embodiments of the present description.
Fig. 2 is a schematic diagram of space division from a physical hard disk to a virtual disk according to some embodiments of the present disclosure.
FIG. 3 is a schematic illustration of a storage pool in accordance with some embodiments of the present description.
FIG. 4 is a diagram illustrating possible sub-pg data distributions in some embodiments of the present disclosure.
FIG. 5 is a schematic illustration of a combination of lv data maps according to some embodiments of the present disclosure.
Fig. 6 is a block diagram of a virtualized raid implementation apparatus according to some embodiments of the present specification.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only a part of the embodiments of the present specification, and not all of the embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments in the present specification without any inventive step should fall within the scope of protection of the present specification.
As shown in fig. 1, some embodiments of the present specification provide a method for implementing virtualization raid, including dividing a physical hard disk into a first predetermined number of virtual disks, and numbering the virtual disks; sequentially numbering the physical hard disks of a second preset number, and selecting the physical hard disks of not less than a third preset number to form a storage pool; selecting a fourth preset number of physical hard disks from the storage pool to form a pg, and finishing the initialization of the pg; taking out a preset logical volume lv from the storage pool, selecting the maximum pg in idle sub-pgs from the pgs by adopting a simplified configuration mode, and returning the sub-pgs, wherein a single sub-pg is a basic raid storage unit, and 7 virtual disks in the sub-pgs come from 7 different physical hard disks; when the physical hard disks are damaged, degrading a fifth preset number of pg, reconstructing the degraded pg, and reconstructing data to the residual space of each physical hard disk in the storage pool; after reconstruction is completed, the hot standby disk is added to the storage pool, data in part of pg is copied by taking the virtual disk as a unit, switching of pg data units is completed after copying is completed, and reserved spaces of all the physical hard disks are recovered. In connection with the embodiment in fig. 2, a physical hard disk may be abbreviated as OSD and has a capacity of CMB, and a physical hard disk may be divided into a plurality of virtual disks, for example, the physical hard disk is divided into a plurality of virtual disks in units of 128M, and the virtual disks check sd and are numbered, for example, sd0 and sd1 … sdx. As shown in fig. 3, in some embodiments of the present description, multiple OSDs may be sequentially numbered, such as 0, 1, 2 … n-1; selecting not less than m (m)>9) One OSD constitutes a memory pool, and 10 OSDs are selected in fig. 3 to constitute a memory pool. When 7 OSD components pg are selected from among them, the number of combinations that can be selected is
Figure BDA0002860957330000031
When m equals 9, 36 combinations can be formed. As shown in table 1, when m is equal to 5, 3 OSD compositions pg are selected therefrom, the following 10 combinations can be formed.
Pg0 (0,1,2)
Pg1 (0,1,3)
Pg2 (0,1,4)
Pg3 (0,2,3)
Pg4 (0,2,4)
Pg5 (0,3,4)
Pg6 (1,2,3)
Pg7 (1,2,4)
Pg8 (1,3,4)
Pg9 (2,3,4)
TABLE 1
In some embodiments of the present description, selecting a fourth predetermined number of physical hard disk drives from the storage pool to form a pg list, numbering the pgs from 0 to n, and forming a pg list of pg0 through pgn; and for each pg, selecting a smallest unused virtual disk from the first physical hard disk of the pg, sequentially traversing other physical hard disks, and selecting a smallest unused virtual disk from each other physical hard disk, thereby completing initialization of the data segments of pg0-0 to pgn-0.
Further, in some embodiments of the present disclosure, after completing initialization of the data segments from pg0-0 to pgn-0, initialization of other data segments continues until completing initialization of the data segments from pg0-t to pgn-t, where t is calculated by: (disk number-1) single disk capacity)/(single sd capacity 7), reserving at least a predetermined capacity at each end of the physical hard disk, the predetermined capacity being calculated by: (single disc capacity/(disc number-1)). times (single sd capacity).
For each pg initialization, pg0 selects an sd from the first OSD, and sd is maintained by the corresponding OSD. The selection method comprises the following steps: and the appointed OSD returns a minimum unused sd, then an sd is selected from the second OSD, all OSDs are traversed in sequence, and the initialization of a pg0-0 data segment is completed, so that the initialization from pg0-0 to pg35-0 can be completed in sequence, and the rest is repeated until the initialization from pg0-t to pg35-t is completed. the algorithm method of the t value is as follows: (disk number-1) single disk capacity)/(single sd capacity 7), omit decimal, round. According to this algorithm, at least capacity (single disk capacity/(disk number-1)) × (single sd capacity) is reserved per OSD termination.
In some embodiments of the present specification, of the 7 virtual disks in the sub-pg, 6 virtual disks are data disks, and 1 virtual disk is a check disk.
As shown in fig. 4 and fig. 5, fig. 4 is a schematic diagram of possible sub-pg data distribution in some embodiments of the present description, and fig. 5 is a schematic diagram of lv data mapping combination in some embodiments of the present description. In some embodiments of the present specification, the available logical volume lv is taken out from the storage pool, a reduction configuration mode is adopted, a maximum pg in free sub-pgs is selected from the pg pool, and a sub-pg is returned by the pg, for example, pg0-0 gives lv, so as to form a first data segment of a first lv. And lv is used subsequently, if the data segment is not mapped to pg, selecting one sub pg to be mapped to lv through an algorithm.
In some embodiments of the present description, not less than m (m) is selected>9) When 7 OSD components pg are selected from the OSD component storage pool, if 1 physical hard disk is damaged, the storage pool will be provided with the OSD components pg
Figure BDA0002860957330000042
The individual pgs are degraded, which requires reconstruction. In the reconstruction process, a new physical hard disk is not used as a hot standby disk for reconstruction, but data is reconstructed into the residual space of each hard disk in the storage pool. The reconstruction speed is (single disk reconstruction speed (M-1)) and is (M-1) times faster than the conventional raid 5. Assuming that M is 20 in practical use, the single-disk reconstruction speed is adjusted to 50MB/s, and the reconstruction speed can reach 3.2 TB/h.
As shown in Table 2, Table 2 is a pg remap after a disk crash.
Figure BDA0002860957330000041
TABLE 2
With reference to fig. 6, an embodiment of the present invention further provides a virtualization raid implementation apparatus, including a physical hard disk space dividing module, configured to divide a physical hard disk into a first predetermined number of virtual disks, and number the virtual disks; the storage pool generating module is used for sequentially numbering the physical hard disks with the second preset number and selecting the physical hard disks with no less than a third preset number from the physical hard disks to form a storage pool; a pg generating and initializing module, configured to select a fourth predetermined number of physical hard disks from the storage pool to form a pg, and complete initialization of the pg; a sub-pg returning module, configured to take out a predetermined logical volume lv from the storage pool, select a maximum pg of idle sub-pgs from the pgs in a simplified configuration manner, and return the sub-pgs, where a single sub-pg is a basic raid storage unit, and 7 virtual disks in the sub-pgs are from 7 different physical hard disks respectively; the rebuilding module is used for degrading a fifth preset number of pg when the physical hard disk is damaged, rebuilding the degraded pg, and rebuilding data to the residual space of each physical hard disk in the storage pool; after reconstruction is completed, the hot standby disk is added to the storage pool, data in part of pg is copied by taking the virtual disk as a unit, switching of pg data units is completed after copying is completed, and reserved spaces of all the physical hard disks are recovered.
There is also provided in some embodiments of the present specification an electronic device and a computer-readable storage medium, the electronic device comprising a memory for storing a computer software program; and the processor is used for realizing the steps of the virtualization raid realization method when the computer software program is run. The computer readable storage medium stores a computer software program that when executed implements the steps of a virtualized raid implementation method.
In conclusion, the embodiment of the invention can obviously improve the reconstruction performance and shorten the reconstruction time.
While the process flows described above include operations that occur in a particular order, it should be appreciated that the processes may include more or less operations that are performed sequentially or in parallel (e.g., using parallel processors or a multi-threaded environment). The present invention is described with reference to flowchart illustrations and/or block diagrams of methods according to embodiments of the invention.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method or device comprising the element.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the method embodiment, since it is substantially similar to the apparatus embodiment, the description is simple, and the relevant points can be referred to the partial description of the apparatus embodiment. The above description is only an example of the present specification, and is not intended to limit the present specification. Various modifications and alterations to this description will become apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification should be included in the scope of the claims of the present specification.

Claims (7)

1. A method for realizing virtualization raid is characterized in that the method comprises the following steps:
dividing a physical hard disk into a first preset number of virtual disks, and numbering the virtual disks;
sequentially numbering the physical hard disks of a second preset number, and selecting the physical hard disks of not less than a third preset number to form a storage pool;
selecting a fourth preset number of physical hard disks from the storage pool to form a pg, and finishing the initialization of the pg;
taking out a preset logical volume lv from the storage pool, selecting the maximum pg in idle sub-pgs from the pgs by adopting a simplified configuration mode, and returning the sub-pgs, wherein a single sub-pg is a basic raid storage unit, and 7 virtual disks in the sub-pgs are respectively from 7 different physical hard disks;
when the physical hard disks are damaged, degrading a fifth preset number of pg, reconstructing the degraded pg, and reconstructing data to the residual space of each physical hard disk in the storage pool;
after reconstruction is completed, the hot standby disk is added to the storage pool, data in part of pg is copied by taking the virtual disk as a unit, switching of pg data units is completed after copying is completed, and reserved spaces of all the physical hard disks are recovered.
2. The virtualization raid implementation method of claim 1,
selecting a fourth predetermined number of physical hard disk components pg from the storage pool, numbering the pgs from 0 to n, and forming a pg list consisting of pg0 to pgn;
and for each pg, selecting a smallest unused virtual disk from the first physical hard disk of the pg, sequentially traversing other physical hard disks, and selecting a smallest unused virtual disk from each other physical hard disk, thereby completing initialization of the data segments of pg0-0 to pgn-0.
3. The virtualization raid implementation method of claim 2,
after the initialization of the data segments from pg0-0 to pgn-0 is completed, the initialization of other data segments is continuously completed until the initialization of the data segments from pg0-t to pgn-t is completed, wherein the calculation method of t is as follows: (disk number-1) single disk capacity)/(single sd capacity 7), reserving at least a predetermined capacity at each end of the physical hard disk, the predetermined capacity being calculated by: (single disc capacity/(disc number-1)). times (single sd capacity).
4. The virtualization raid implementation method of claim 1,
and 6 virtual disks in the 7 virtual disks in the sub pg are data disks, and 1 virtual disk is a check disk.
5. A virtualization raid implementation apparatus, comprising:
the physical hard disk space dividing module is used for dividing physical hard disks into a first preset number of virtual disks and numbering the virtual disks;
the storage pool generating module is used for sequentially numbering the physical hard disks with the second preset number and selecting the physical hard disks with no less than a third preset number from the physical hard disks to form a storage pool;
a pg generating and initializing module, configured to select a fourth predetermined number of physical hard disks from the storage pool to form a pg, and complete initialization of the pg;
a sub-pg returning module, configured to take out a predetermined logical volume lv from the storage pool, select a maximum pg of idle sub-pgs from the pgs in a simplified configuration manner, and return the sub-pgs, where a single sub-pg is a basic raid storage unit, and 7 virtual disks in the sub-pgs are from 7 different physical hard disks respectively;
the rebuilding module is used for degrading a fifth preset number of pg when the physical hard disk is damaged, rebuilding the degraded pg, and rebuilding data to the residual space of each physical hard disk in the storage pool; after reconstruction is completed, the hot standby disk is added to the storage pool, data in part of pg is copied by taking the virtual disk as a unit, switching of pg data units is completed after copying is completed, and reserved spaces of all the physical hard disks are recovered.
6. An electronic device, comprising
A memory for storing a computer software program;
a processor for implementing the steps of the virtualized raid implementation method according to any one of claims 1 to 4 when running said computer software program.
7. A computer-readable storage medium, characterized in that,
the computer readable storage medium having stored thereon a computer software program which when executed performs the steps of a virtualized raid implementation method according to any one of claims 1 to 4.
CN202011563169.XA 2020-12-25 2020-12-25 Method and device for realizing virtualization raid Pending CN112667156A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011563169.XA CN112667156A (en) 2020-12-25 2020-12-25 Method and device for realizing virtualization raid

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011563169.XA CN112667156A (en) 2020-12-25 2020-12-25 Method and device for realizing virtualization raid

Publications (1)

Publication Number Publication Date
CN112667156A true CN112667156A (en) 2021-04-16

Family

ID=75409186

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011563169.XA Pending CN112667156A (en) 2020-12-25 2020-12-25 Method and device for realizing virtualization raid

Country Status (1)

Country Link
CN (1) CN112667156A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113515233A (en) * 2021-05-24 2021-10-19 锐捷网络股份有限公司 Block group building method and device and electronic equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101135955A (en) * 2007-10-19 2008-03-05 杭州华三通信技术有限公司 Method for realizing magnetic disc redundant array rebuilding and magnetic disc redundant array
CN101980137A (en) * 2010-10-19 2011-02-23 成都市华为赛门铁克科技有限公司 Method, device and system for reconstructing redundant array of inexpensive disks
CN103617006A (en) * 2013-11-28 2014-03-05 曙光信息产业股份有限公司 Storage resource management method and device
CN104375784A (en) * 2014-11-17 2015-02-25 浙江宇视科技有限公司 Method and device for lowering complexity of virtual disk management
CN105893188A (en) * 2014-09-30 2016-08-24 伊姆西公司 Method and device for speeding up data reconstruction of disk array
US20170371782A1 (en) * 2015-01-21 2017-12-28 Hewlett Packard Enterprise Development Lp Virtual storage

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101135955A (en) * 2007-10-19 2008-03-05 杭州华三通信技术有限公司 Method for realizing magnetic disc redundant array rebuilding and magnetic disc redundant array
CN101980137A (en) * 2010-10-19 2011-02-23 成都市华为赛门铁克科技有限公司 Method, device and system for reconstructing redundant array of inexpensive disks
CN103617006A (en) * 2013-11-28 2014-03-05 曙光信息产业股份有限公司 Storage resource management method and device
CN105893188A (en) * 2014-09-30 2016-08-24 伊姆西公司 Method and device for speeding up data reconstruction of disk array
CN104375784A (en) * 2014-11-17 2015-02-25 浙江宇视科技有限公司 Method and device for lowering complexity of virtual disk management
US20170371782A1 (en) * 2015-01-21 2017-12-28 Hewlett Packard Enterprise Development Lp Virtual storage

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113515233A (en) * 2021-05-24 2021-10-19 锐捷网络股份有限公司 Block group building method and device and electronic equipment

Similar Documents

Publication Publication Date Title
US10210045B1 (en) Reducing concurrency bottlenecks while rebuilding a failed drive in a data storage system
JP6346565B2 (en) Method and apparatus for assigning erasure coded data to disk storage
US10126988B1 (en) Assigning RAID extents and changing drive extent allocations within RAID extents when splitting a group of storage drives into partnership groups in a data storage system
US10146447B1 (en) Mapped RAID (redundant array of independent disks) in a data storage system with drive extents allocated to individual RAID extents from individual sub-groups of storage made up of ranges of logical block addresses defined across a group of hard disk drives
US10089026B1 (en) Virtual drive based RAID groups
US10140041B1 (en) Mapped RAID (redundant array of independent disks) in a data storage system with RAID extent sub-groups that are used to perform drive extent allocation and data striping for sequential data accesses to a storage object
US10289336B1 (en) Relocating data from an end of life storage drive based on storage drive loads in a data storage system using mapped RAID (redundant array of independent disks) technology
US20190129614A1 (en) Load Balancing of I/O by Moving Logical Unit (LUN) Slices Between Non-Volatile Storage Represented by Different Rotation Groups of RAID (Redundant Array of Independent Disks) Extent Entries in a RAID Extent Table of a Mapped RAID Data Storage System
CN110096219B (en) Effective capacity of a pool of drive zones generated from a group of drives
US10229022B1 (en) Providing Raid-10 with a configurable Raid width using a mapped raid group
US8612678B2 (en) Creating logical disk drives for raid subsystems
CN110806839B (en) Storage control method, memory device, memory controller and storage server
US20160085467A1 (en) Intelligent data placement
JP2007516524A (en) Quasi-static distributed technology
CN111095217B (en) RAID mechanism-based data storage system with global sharing of resources
US10678643B1 (en) Splitting a group of physical data storage drives into partnership groups to limit the risk of data loss during drive rebuilds in a mapped RAID (redundant array of independent disks) data storage system
CN110096218B (en) Method, system, and medium for providing RAID data protection for storage objects
US10592111B1 (en) Assignment of newly added data storage drives to an original data storage drive partnership group and a new data storage drive partnership group in a mapped RAID (redundant array of independent disks) system
WO2017175285A1 (en) Computing machine system, control method for physical storage device, and recording medium
US8656131B2 (en) Method and apparatus for expanding a virtual storage device
CN112667156A (en) Method and device for realizing virtualization raid
US10977130B2 (en) Method, apparatus and computer program product for managing raid storage in data storage systems
US11010247B2 (en) Dynamic memory management system
US11860746B2 (en) Resilient data storage system with efficient space management
US11868637B2 (en) Flexible raid sparing using disk splits

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination