CN111880748A - Wear balancing method for solid state disk of distributed storage system - Google Patents
Wear balancing method for solid state disk of distributed storage system Download PDFInfo
- Publication number
- CN111880748A CN111880748A CN202010764608.7A CN202010764608A CN111880748A CN 111880748 A CN111880748 A CN 111880748A CN 202010764608 A CN202010764608 A CN 202010764608A CN 111880748 A CN111880748 A CN 111880748A
- Authority
- CN
- China
- Prior art keywords
- storage
- weight
- wear
- storage media
- storage system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 239000007787 solid Substances 0.000 title claims abstract description 27
- 238000005299 abrasion Methods 0.000 claims description 19
- 230000005012 migration Effects 0.000 claims description 6
- 238000013508 migration Methods 0.000 claims description 6
- 238000012544 monitoring process Methods 0.000 claims description 5
- 230000036541 health Effects 0.000 description 11
- 230000007246 mechanism Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0616—Improving the reliability of storage systems in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0653—Monitoring storage devices or systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
- Automatic Disk Changers (AREA)
Abstract
The invention relates to a wear leveling method for a solid state disk of a distributed storage system, which comprises the following steps: the method comprises the following steps: collecting storage media, recording the weight information of the current storage cluster as an initial weight, and grouping the media according to fault domains to ensure that the storage media under different fault domains are distributed in different groups; step two: recording the weight information of the current storage medium, generating the wear degree e of the storage medium according to the writing amount m, the number b of bad blocks, the read error retry rate r and the official writing life L of the current storage medium in the SMART information of the storage medium, and then judging measures taken by the storage cluster according to the distribution condition of the wear degree e between 0 and 1. The invention ensures the performance of the storage cluster when the storage cluster is not idle, and also ensures the stability and controllability of the storage cluster.
Description
Technical Field
The invention relates to the field of computing and storage, in particular to a method for balancing abrasion of a solid state disk of a distributed storage system.
Background
With the development of technologies such as a host, a disk, a network and the like, a data storage mode and a data storage architecture are continuously changed, and a local storage is developed to a network storage, a single-machine storage is developed to a distributed storage, so that problems are brought. The read and write amount of distributed storage of an enterprise is very large, and naturally, extremely high requirements are put on the reliability and consistency of physical storage equipment above the enterprise. With the improvement of network performance and the improvement of the requirement of a user on data throughput, more and more cloud storage systems use the solid state disk to store files, the reading and writing speed of the files is improved, and the response time is shortened. Under the guarantee of the wear mechanism of the existing distributed storage system, after each solid state disk enters a storage cluster for a period of time, the wear degree of each solid state disk and other equipment in the storage cluster converge, and after the service life of the whole solid state disk reaches the end, people cannot know which solid state disks have problems, and the uncontrollable way causes higher maintenance cost.
Wear leveling is one of the necessary characteristics of cloud storage, and the purpose of the wear leveling is to make the relative lifetime of each storage device reach a level of leveling, which is achieved by specific allocation of management write data streams, but the reverse wear leveling is crucial to the stability of the health and performance of the storage cluster.
In order to realize wear balance and reverse wear balance among storage clusters, data of each hard disk of the storage clusters are monitored, health degree data of each hard disk is calculated, so that a health degree ranking is obtained, expected data write-in distribution is calculated by means of the ranking, and then write-in weight of the hard disks is changed, so that the health degree of the hard disks is changed on a data write-in layer. For reverse wear leveling, devices needing to be worn preferentially are screened out according to the overall health degree, and then data is written preferentially, so that the storage medium of the storage cluster is worn in a planned way.
Disclosure of Invention
The present invention is directed to a method for wear leveling of solid state disks in a distributed storage system, which is used to solve the above-mentioned problems of the prior art.
The invention discloses a solid state disk wear leveling method for a distributed storage system, which comprises the following steps: the method comprises the following steps: collecting storage media, recording the weight information of the current storage cluster as an initial weight, and grouping the media according to fault domains to ensure that the storage media under different fault domains are distributed in different groups; step two: recording the weight information of the current storage medium, generating the wear degree e of the storage medium according to the writing amount m, the number b of bad blocks, the read error retry rate r and the official writing life L of the current storage medium in the SMART information of the storage medium, and then judging measures taken by the storage cluster according to the distribution condition of the wear degree e between 0 and 1.
An embodiment of a method for wear leveling of a solid state disk for a distributed storage system according to the present invention includes: the SMART information comprises the writing quantity, the number of bad blocks and the retry rate of reading errors.
An embodiment of a method for wear leveling of a solid state disk for a distributed storage system according to the present invention includes: SMART information also captures the official write lifetime of the storage medium.
An embodiment of a method for wear leveling of a solid state disk for a distributed storage system according to the present invention includes: and continuously monitoring the operation condition of the storage cluster.
In an embodiment of the method for wear leveling of a solid state disk for a distributed storage system according to the present invention, the wear level e of the storage medium is generated by e ═ 0.9(L/m) +0.05 × (b/10000) +0.05 × r.
According to an embodiment of the method for wear leveling of the solid state disk for the distributed storage system, for a wear degree e: if the mean square error of the abrasion degrees of the storage media of the storage clusters is less than 0.025, the storage media in the interval of two standard deviations of the weight average is more than 95%, the storage clusters are considered to be healthy, the weight information of the storage clusters at the moment is recorded, and if the storage clusters are not the initial weight stored previously, the storage clusters are restored to the initial weight through the weight adjusting command of the storage clusters; if the mean square error of the wear degrees of the storage media of the storage clusters is larger than 0.025, the storage media in an interval of two standard deviations of the mean of the wear degrees are smaller than 95%, and at the moment, the data writing of the storage media of the storage clusters is considered to be unbalanced, and starting wear is carried out; and if the abrasion degrees of the storage media of the storage clusters are all above 0.75, performing reverse abrasion balance.
According to an embodiment of the method for wear leveling of the solid state disk for the distributed storage system, the wear leveling includes: recording the storage media with the wearing degree larger than 95% smaller than the average wearing degree in the storage media, sorting the storage media from large to small, reducing the weight value of the first 50% of the storage media to seventy percent of the original weight value, calculating a brand new weight value, submitting the brand new weight value to a storage cluster, and when the storage system operates normally and has no data migration, considering that the storage system is healthy, and writing the weight back to the storage cluster by using the storage system through a self command.
According to an embodiment of the method for wear leveling of the solid state disk for the distributed storage system, the method for wear leveling of the solid state disk comprises the following steps: sorting all storage media in a storage cluster from high to low according to the abrasion degree, increasing the weight of the first 20 percent of the storage media to 200 percent of the original weight, calculating a brand new weight value, submitting the brand new weight value to the storage cluster, considering that the storage system is healthy when the storage system operates normally and has no data migration, writing a self command of the weight using storage system back to the storage cluster, paying attention to the abrasion degree of the selected storage media by a program at any time, recording logs of the weight at the moment when the abrasion degree of the selected storage media reaches 0.98, and adjusting the weight value of the selected storage media to 0 by using the self command of the storage system.
The invention establishes a set of monitoring system for the storage medium while the storage cluster works normally, and forms an automatic weight balancing mechanism under the condition of not influencing the performance of the storage cluster, thereby achieving the automation of wear balancing.
Drawings
FIG. 1 is a flow chart illustrating wear leveling;
fig. 2 is a flow chart of reverse wear equalization.
Detailed Description
In order to make the objects, contents, and advantages of the present invention clearer, the following detailed description of the embodiments of the present invention will be made in conjunction with the accompanying drawings and examples.
FIG. 1 is a flow chart of a wear leveling die; fig. 2 is a flowchart illustrating reverse wear leveling, and as shown in fig. 1 and fig. 2, the present invention provides a method for wear leveling of a solid state disk for a distributed storage system, including the following steps:
the method comprises the following steps: collecting detailed information of the storage medium, and calculating to obtain the specific health degree of the storage medium through a model; in the SMART information, average failure time of the hard disk, total write-in quantity, error times and total capacity are required to be acquired; additionally, the official write lifetime of the storage medium is to be obtained.
After obtaining the data, the health degree of each hard disk is calculated, and the health degrees of the hard disks are classified and arranged according to fault domains.
Step two: recording the weight information of the current storage medium, generating the wear degree e of the storage medium according to the writing amount m, the number b of bad blocks, the read error retry rate r and the official writing life L of the current storage medium in the SMART information of the storage medium, and then judging measures taken by the storage cluster according to the distribution condition of the wear degree e between 0 and 1.
Step three: monitoring the state of the storage cluster, starting a temporary wear leveling mechanism after the overall wear state deviates, correspondingly changing the weight of the storage medium in the deviating state, continuously monitoring the state of the storage cluster, and adjusting the weight to recover to the normal state after the storage cluster recovers to the normal state.
For the degree of wear e: if the mean square error of the abrasion degrees of the storage media of the storage clusters is less than 0.025, the storage media in the interval of two standard deviations of the weight average is more than 95%, the storage clusters are considered to be healthy, the weight information of the storage clusters at the moment is recorded, and if the storage clusters are not the initial weight stored previously, the storage clusters are restored to the initial weight through the weight adjusting command of the storage clusters; if the mean square error of the wear degrees of the storage media of the storage clusters is larger than 0.025, the storage media in an interval of two standard deviations of the mean of the wear degrees are smaller than 95%, and at the moment, the data writing of the storage media of the storage clusters is considered to be unbalanced, and starting wear is carried out; and if the abrasion degrees of the storage media of the storage clusters are all above 0.75, performing reverse abrasion balance.
The wear leveling includes: recording the storage media with the wearing degree larger than 95% smaller than the average wearing degree in the storage media, sorting the storage media from large to small, reducing the weight value of the first 50% of the storage media to seventy percent of the original weight value, calculating a brand new weight value, submitting the brand new weight value to a storage cluster, and when the storage system operates normally and has no data migration, considering that the storage system is healthy, and writing the weight back to the storage cluster by using the storage system through a self command.
The reverse wear leveling includes: sorting all storage media in a storage cluster from high to low according to the abrasion degree, increasing the weight of the first 20 percent of the storage media to 200 percent of the original weight, calculating a brand new weight value, submitting the brand new weight value to the storage cluster, considering that the storage system is healthy when the storage system operates normally and has no data migration, writing a self command of the weight using storage system back to the storage cluster, paying attention to the abrasion degree of the selected storage media by a program at any time, recording logs of the weight at the moment when the abrasion degree of the selected storage media reaches 0.98, and adjusting the weight value of the selected storage media to 0 by using the self command of the storage system.
And modifying the storage cluster in the step two by the wear leveling and the reverse wear leveling, wherein the step three is a temporary modification measure, and the data volume suddenly increased in a short time or the imbalance between the newly added storage medium and the existing storage cluster is modified by the step three.
The invention provides a method for correcting the health degree of a storage cluster, which corrects the weight of a storage medium of the storage cluster by using the idle time of the storage cluster while ensuring the health of the storage cluster, monitors in real time, ensures the performance of the storage cluster when the storage cluster is not idle, and also ensures the stability and controllability of the storage cluster; for the wear-leveling device, when the administrator changes the storage structure, the health degree of the whole storage cluster does not have overlarge deviation, and the method can automatically adjust for a new storage architecture.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.
Claims (8)
1. A method for wear leveling of a solid state disk for a distributed storage system, comprising:
the method comprises the following steps: collecting storage media, recording the weight information of the current storage cluster as an initial weight, and grouping the media according to fault domains to ensure that the storage media under different fault domains are distributed in different groups;
step two: recording the weight information of the current storage medium, generating the wear degree e of the storage medium according to the writing amount m, the number b of bad blocks, the read error retry rate r and the official writing life L of the current storage medium in the SMART information of the storage medium, and then judging measures taken by the storage cluster according to the distribution condition of the wear degree e between 0 and 1.
2. The method for wear leveling of the solid state disks of the distributed storage system according to claim 1, comprising: the SMART information comprises the writing quantity, the number of bad blocks and the retry rate of reading errors.
3. The method for wear leveling of the solid state disk for the distributed storage system according to claim 2, comprising: SMART information also captures the official write lifetime of the storage medium.
4. The method for wear leveling of the solid state disks of the distributed storage system according to claim 1, comprising: and continuously monitoring the operation condition of the storage cluster.
5. The method of claim 1, wherein the wear leveling of the storage medium is performed by e ═ 0.9(L/m) +0.05 × (b/10000) +0.05 × r.
6. The method for wear leveling of solid state disks for distributed storage system according to claim 1, wherein for a degree of wear e:
if the mean square error of the abrasion degrees of the storage media of the storage clusters is less than 0.025, the storage media in the interval of two standard deviations of the weight average is more than 95%, the storage clusters are considered to be healthy, the weight information of the storage clusters at the moment is recorded, and if the storage clusters are not the initial weight stored previously, the storage clusters are restored to the initial weight through the weight adjusting command of the storage clusters;
if the mean square error of the wear degrees of the storage media of the storage clusters is larger than 0.025, the storage media in an interval of two standard deviations of the mean of the wear degrees are smaller than 95%, and at the moment, the data writing of the storage media of the storage clusters is considered to be unbalanced, and starting wear is carried out;
and if the abrasion degrees of the storage media of the storage clusters are all above 0.75, performing reverse abrasion balance.
7. The method for wear leveling of the solid state disks of the distributed storage system according to claim 6, wherein the wear leveling comprises: recording the storage media with the wearing degree larger than 95% smaller than the average wearing degree in the storage media, sorting the storage media from large to small, reducing the weight value of the first 50% of the storage media to seventy percent of the original weight value, calculating a brand new weight value, submitting the brand new weight value to a storage cluster, and when the storage system operates normally and has no data migration, considering that the storage system is healthy, and writing the weight back to the storage cluster by using the storage system through a self command.
8. The method for wear leveling of the solid state disks of the distributed storage system according to claim 6, wherein the reverse wear leveling comprises: sorting all storage media in a storage cluster from high to low according to the abrasion degree, increasing the weight of the first 20 percent of the storage media to 200 percent of the original weight, calculating a brand new weight value, submitting the brand new weight value to the storage cluster, considering that the storage system is healthy when the storage system operates normally and has no data migration, writing a self command of the weight using storage system back to the storage cluster, paying attention to the abrasion degree of the selected storage media by a program at any time, recording logs of the weight at the moment when the abrasion degree of the selected storage media reaches 0.98, and adjusting the weight value of the selected storage media to 0 by using the self command of the storage system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010764608.7A CN111880748B (en) | 2020-07-30 | 2020-07-30 | Solid state disk wear balancing method for distributed storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010764608.7A CN111880748B (en) | 2020-07-30 | 2020-07-30 | Solid state disk wear balancing method for distributed storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111880748A true CN111880748A (en) | 2020-11-03 |
| CN111880748B CN111880748B (en) | 2023-10-31 |
Family
ID=73205998
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010764608.7A Active CN111880748B (en) | 2020-07-30 | 2020-07-30 | Solid state disk wear balancing method for distributed storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111880748B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112506725A (en) * | 2020-12-04 | 2021-03-16 | 苏州浪潮智能科技有限公司 | Method, device and equipment for judging grade of repaired solid state disk and readable medium |
| CN115934007A (en) * | 2023-03-15 | 2023-04-07 | 浪潮电子信息产业股份有限公司 | Data storage method, system, equipment and storage medium of distributed storage system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101419842A (en) * | 2008-11-07 | 2009-04-29 | 成都市华为赛门铁克科技有限公司 | Loss equalizing method, apparatus and system for hard disc |
| CN110737924A (en) * | 2018-07-20 | 2020-01-31 | 中移(苏州)软件技术有限公司 | method and equipment for data protection |
-
2020
- 2020-07-30 CN CN202010764608.7A patent/CN111880748B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101419842A (en) * | 2008-11-07 | 2009-04-29 | 成都市华为赛门铁克科技有限公司 | Loss equalizing method, apparatus and system for hard disc |
| CN110737924A (en) * | 2018-07-20 | 2020-01-31 | 中移(苏州)软件技术有限公司 | method and equipment for data protection |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112506725A (en) * | 2020-12-04 | 2021-03-16 | 苏州浪潮智能科技有限公司 | Method, device and equipment for judging grade of repaired solid state disk and readable medium |
| CN112506725B (en) * | 2020-12-04 | 2023-01-06 | 苏州浪潮智能科技有限公司 | Method, device and equipment for judging grade of repaired solid state disk and readable medium |
| CN115934007A (en) * | 2023-03-15 | 2023-04-07 | 浪潮电子信息产业股份有限公司 | Data storage method, system, equipment and storage medium of distributed storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111880748B (en) | 2023-10-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6892276B2 (en) | Increased data availability in raid arrays using smart drives | |
| CN102024044B (en) | Distributed file system | |
| US8244671B2 (en) | Replica placement and repair strategies in multinode storage systems | |
| US7580956B1 (en) | System and method for rating reliability of storage devices | |
| CN105335250B (en) | A kind of data reconstruction method and device based on distributed file system | |
| US10572356B2 (en) | Storing data in multi-region storage devices | |
| EP2672387A1 (en) | A distributed object storage system | |
| CN111880748B (en) | Solid state disk wear balancing method for distributed storage system | |
| US20140297926A1 (en) | Information processing device and method for controlling replacement of semiconductor storage device | |
| CN103763383A (en) | Integrated cloud storage system and storage method thereof | |
| CN110750213A (en) | Hard disk management method and device | |
| CN111324304A (en) | Data protection method and device based on SSD hard disk life prediction | |
| CN111858240B (en) | Monitoring method, system, equipment and medium of distributed storage system | |
| US11199968B2 (en) | Using recurring write quotas to optimize utilization of solid state storage in a hybrid storage array | |
| US20230136274A1 (en) | Ceph Media Failure and Remediation | |
| CN106980468A (en) | Trigger the method and device that RAID array is rebuild | |
| CN106990918A (en) | Trigger the method and device that RAID array is rebuild | |
| CN106528349B (en) | A kind of date storage method and device | |
| US20150205531A1 (en) | Adding Storage Capacity to an Object Storage System | |
| US9858147B2 (en) | Storage apparatus and method of controlling storage apparatus | |
| CN106951338B (en) | A kind of fault-tolerant distributed storage method of height based on GlusterFS | |
| CN113051428B (en) | Method and device for back-up storage at front end of camera | |
| JPWO2013190649A1 (en) | Information processing method and apparatus for virtual disk migration | |
| CN103631535A (en) | Method and device for achieving space distribution in storage pool | |
| CN114489515B (en) | Data writing method based on stream storage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |