CN113849123B - Method, system, equipment and medium for processing data of slow disk - Google Patents
Method, system, equipment and medium for processing data of slow disk Download PDFInfo
- Publication number
- CN113849123B CN113849123B CN202110933570.6A CN202110933570A CN113849123B CN 113849123 B CN113849123 B CN 113849123B CN 202110933570 A CN202110933570 A CN 202110933570A CN 113849123 B CN113849123 B CN 113849123B
- Authority
- CN
- China
- Prior art keywords
- disk
- data blocks
- slow
- data
- slow disk
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 22
- 238000012545 processing Methods 0.000 title claims description 12
- 230000004044 response Effects 0.000 claims abstract description 100
- 238000005192 partition Methods 0.000 claims abstract description 35
- 238000011084 recovery Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 238000001514 detection method Methods 0.000 claims 1
- 238000012795 verification Methods 0.000 abstract description 8
- 238000003672 processing method Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013524 data verification Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0653—Monitoring storage devices or systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0689—Disk arrays, e.g. RAID, JBOD
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Quality & Reliability (AREA)
- Signal Processing For Digital Recording And Reproducing (AREA)
Abstract
The invention discloses a data processing method of a slow disk, which comprises the following steps: acquiring the I/O response time of a disk in a disk array, and determining the disk with the I/O response time larger than a threshold value as a slow disk; responding to a received writing request, and determining a stripe corresponding to data to be written according to a stripe number carried in the writing request; determining a first partition corresponding to the slow disk and a plurality of second partitions corresponding to other disks on the strip; dividing data to be written into a plurality of data blocks and calculating parity information of the plurality of data blocks; and storing a plurality of data blocks and parity information in the plurality of data blocks onto a plurality of second partitions, respectively. The invention also discloses a system, computer equipment and a readable storage medium. According to the scheme provided by the embodiment of the invention, after the slow disk is detected, the data of the slow disk is recorded in other disks in a verification mode, so that the overall RAID reliability can be improved under the condition that the existing hot standby disk is not occupied.
Description
Technical Field
The present invention relates to the field of storage, and in particular, to a method, a system, an apparatus, and a storage medium for processing data of a slow disk.
Background
RAID is a virtualization technology that virtualizes a plurality of physical disks into one large-capacity logical disk for an external host. In order to ensure high performance and reliability of the logical disk, the RAID adopts key technologies such as mirror redundancy, data stripes, data verification and the like in the development process. And RAID slow disk management is one way to maintain RAID reliability.
Most of the conventional common slow disk processing modes are to reconstruct the marked slow disk directly into a hot standby disk, and copy back after the slow disk is normal. However, this would occupy the existing hot standby disk, thereby reducing the subsequent stability of the RAID array due to insufficient hot standby.
Disclosure of Invention
In view of this, in order to overcome at least one aspect of the above-mentioned problems, an embodiment of the present invention provides a data processing method for a slow disk, including the following steps:
acquiring I/O (Input/Output) response time of a disk in a disk array, and determining the disk with the I/O response time larger than a threshold value as a slow disk;
responding to a received writing request, and determining a stripe corresponding to data to be written according to a stripe number carried in the writing request;
determining a first partition corresponding to the slow disk and a plurality of second partitions corresponding to other disks on the stripe;
dividing the data to be written into a plurality of data blocks and calculating parity information of the plurality of data blocks;
and respectively storing a plurality of data blocks and the parity information in the plurality of data blocks on the plurality of second blocks.
In some embodiments, further comprising:
creating a bitmap;
recording the strip number carried in each received write request into a gram;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
In some embodiments, further comprising:
traversing the bitmap to determine a gram recorded with a stripe number in response to the I/O response time of the slow disk being less than a threshold;
and determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks except the data blocks in the data blocks by utilizing the data blocks stored in the second blocks and the parity check information, and writing the recovered data blocks into the first blocks.
In some embodiments, obtaining an I/O response time of a disk in a disk array and determining a disk with an I/O response time greater than a threshold as a slow disk further comprises:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
responding to the I/O response time of the disk being larger than the slow disk event judgment threshold value, and updating the occurrence times of the slow disk event of the corresponding disk;
and determining the disk with the occurrence of the slow disk event times larger than the preset times as the slow disk in response to the occurrence of the slow disk event times within the preset time period.
Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a data processing system of a slow disk, including:
the judging module is configured to acquire the I/O response time of the disks in the disk array and determine the disk with the I/O response time larger than a threshold value as a slow disk;
the first determining module is configured to respond to a received writing request and determine a stripe corresponding to data to be written according to a stripe number carried in the writing request;
the second determining module is configured to determine a first partition corresponding to the slow disk and a plurality of second partitions corresponding to other disks on the stripe;
a block module configured to divide the data to be written into a plurality of data blocks and calculate parity information of the plurality of data blocks;
and a storage module configured to store a plurality of data blocks of the plurality of data blocks and the parity information onto the plurality of second partitions, respectively.
In some embodiments, the system further comprises a recording module configured to:
creating a bitmap;
recording the strip number carried in each received write request into a gram;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
In some embodiments, the system further comprises a recovery module configured to:
traversing the bitmap to determine a gram recorded with a stripe number in response to the I/O response time of the slow disk being less than a threshold;
and determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks except the data blocks in the data blocks by utilizing the data blocks stored in the second blocks and the parity check information, and writing the recovered data blocks into the first blocks.
In some embodiments, the determination module is further configured to:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
responding to the I/O response time of the disk being larger than the slow disk event judgment threshold value, and updating the occurrence times of the slow disk event of the corresponding disk;
and determining the disk with the occurrence of the slow disk event times larger than the preset times as the slow disk in response to the occurrence of the slow disk event times within the preset time period.
Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer apparatus, including:
at least one processor; and
a memory storing a computer program executable on the processor, wherein the processor executes the program to perform the steps of:
acquiring the I/O response time of a disk in a disk array, and determining the disk with the I/O response time larger than a threshold value as a slow disk;
responding to a received writing request, and determining a stripe corresponding to data to be written according to a stripe number carried in the writing request;
determining a first partition corresponding to the slow disk and a plurality of second partitions corresponding to other disks on the stripe;
dividing the data to be written into a plurality of data blocks and calculating parity information of the plurality of data blocks;
and respectively storing a plurality of data blocks and the parity information in the plurality of data blocks on the plurality of second blocks.
In some embodiments, further comprising:
creating a bitmap;
recording the strip number carried in each received write request into a gram;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
In some embodiments, further comprising:
traversing the bitmap to determine a gram recorded with a stripe number in response to the I/O response time of the slow disk being less than a threshold;
and determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks except the data blocks in the data blocks by utilizing the data blocks stored in the second blocks and the parity check information, and writing the recovered data blocks into the first blocks.
In some embodiments, obtaining an I/O response time of a disk in a disk array and determining a disk with an I/O response time greater than a threshold as a slow disk further comprises:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
responding to the I/O response time of the disk being larger than the slow disk event judgment threshold value, and updating the occurrence times of the slow disk event of the corresponding disk;
and determining the disk with the occurrence of the slow disk event times larger than the preset times as the slow disk in response to the occurrence of the slow disk event times within the preset time period.
Based on the same inventive concept, according to another aspect of the present invention, there is also provided a computer-readable storage medium storing a computer program which, when executed by a processor, performs the steps of:
acquiring the I/O response time of a disk in a disk array, and determining the disk with the I/O response time larger than a threshold value as a slow disk;
responding to a received writing request, and determining a stripe corresponding to data to be written according to a stripe number carried in the writing request;
determining a first partition corresponding to the slow disk and a plurality of second partitions corresponding to other disks on the stripe;
dividing the data to be written into a plurality of data blocks and calculating parity information of the plurality of data blocks;
and respectively storing a plurality of data blocks and the parity information in the plurality of data blocks on the plurality of second blocks.
In some embodiments, further comprising:
creating a bitmap;
recording the strip number carried in each received write request into a gram;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
In some embodiments, further comprising:
traversing the bitmap to determine a gram recorded with a stripe number in response to the I/O response time of the slow disk being less than a threshold;
and determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks except the data blocks in the data blocks by utilizing the data blocks stored in the second blocks and the parity check information, and writing the recovered data blocks into the first blocks.
In some embodiments, obtaining an I/O response time of a disk in a disk array and determining a disk with an I/O response time greater than a threshold as a slow disk further comprises:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
responding to the I/O response time of the disk being larger than the slow disk event judgment threshold value, and updating the occurrence times of the slow disk event of the corresponding disk;
and determining the disk with the occurrence of the slow disk event times larger than the preset times as the slow disk in response to the occurrence of the slow disk event times within the preset time period.
The invention has one of the following beneficial technical effects: according to the scheme provided by the embodiment of the invention, after the slow disk is detected, the data of the slow disk is recorded in other disks in a verification mode, so that the overall RAID reliability can be improved under the condition that the existing hot spare disk is not occupied on the premise of not affecting the existing performance. And when data recovery is carried out, the repairing granularity mainly based on the stripes is adopted, the repairing process is more accurate, and the repairing accuracy is ensured.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are necessary for the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention and that other embodiments may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for processing data of a slow disk according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a data processing system of a slow disk according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a computer device according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention will be described in further detail with reference to the accompanying drawings.
It should be noted that, in the embodiments of the present invention, all the expressions "first" and "second" are used to distinguish two entities with the same name but different entities or different parameters, and it is noted that the "first" and "second" are only used for convenience of expression, and should not be construed as limiting the embodiments of the present invention, and the following embodiments are not described one by one.
In an embodiment of the present invention, a stripe is a method of dividing continuous data into data blocks of the same size, and writing each segment of data onto a different disk in the array, i.e., the blocks on the different disk form the stripe. At least one of the plurality of blocks constituting the stripe stores parity information, and any two blocks within the stripe are distributed on different disks. The stripe size is equal to the number of disks used to store data multiplied by the segment size of the stripe on each disk, that is, one stripe cuts the aligned disks into a stripe, the hard disk corresponds to a vertical stripe, and the small grid formed after the stripe and the vertical stripe are cut is the segment, which is also called stripe depth.
According to an aspect of the present invention, an embodiment of the present invention proposes a data processing method of a slow disc, as shown in fig. 1, which may include the steps of:
s1, acquiring the I/O response time of a disk in a disk array, and determining the disk with the I/O response time larger than a threshold value as a slow disk;
s2, responding to a received writing request, and determining a stripe corresponding to data to be written according to a stripe number carried in the writing request;
s3, determining a first partition corresponding to the slow disk and a plurality of second partitions corresponding to other magnetic disks on the strip;
s4, dividing the data to be written into a plurality of data blocks and calculating parity information of the plurality of data blocks;
s5, storing a plurality of data blocks and the parity information in the plurality of data blocks on the plurality of second blocks respectively.
According to the scheme provided by the embodiment of the invention, after the slow disk is detected, the data of the slow disk is recorded in other disks in a verification mode, so that the overall RAID reliability can be improved under the condition that the existing hot spare disk is not occupied on the premise of not affecting the existing performance.
In some embodiments, the disk array generally includes a plurality of disks for storing data, a disk for storing check data, and a disk for hot standby, and when a slow disk is detected, the disk may be considered to enter a CID mode (Component in Doubt, which is suspicious for the currently problematic disk), that is, the disk is temporarily unavailable to RAID, until the disk is normal, the I/O data for this period of time is not written in the disk in the CID mode, nor in the hot standby disk, but other data and parity data corresponding to the other data are stored on the other disk.
When the slow disk is detected to appear, and the I/O of the write request is received, firstly determining a stripe to be written by data to be written in the write request, wherein each partition in the stripe is from a different disk, dividing the data to be stored into different data blocks when the data storage is normally carried out, and then calculating parity information of a plurality of data blocks.
For example, the stripe to be written includes 4 blocks, the 4 blocks being provided by different disks, respectively. When one of the 4 disks is determined to be a slow disk, only 3 blocks of the stripe can be written with data, at this time, the data to be written can be blocked according to normal processing logic, then the parity value of the data block is calculated, and finally a part of the data block and the parity value are stored in the remaining 3 blocks, namely, the data block of the slow disk corresponding block which is supposed to be stored on the stripe is temporarily discarded.
In some embodiments, further comprising:
creating a bitmap;
recording the strip number carried in each received write request into a gram;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
Specifically, when a slow disc appears, a gain (one unit of measure) may be added to record the band number. A bitmap can then be created, how many grains create how long. Each grain corresponds to one position in the bitmap, and after the stripe number is recorded in the grain, the flag bit of the corresponding position of the bitmap can be modified to a first preset value (for example, 1).
In some embodiments, further comprising:
traversing the bitmap to determine a gram recorded with a stripe number in response to the I/O response time of the slow disk being less than a threshold;
and determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks except the data blocks in the data blocks by utilizing the data blocks stored in the second blocks and the parity check information, and writing the recovered data blocks into the first blocks.
Specifically, when the I/O response time of the slow disk is smaller than the threshold, the slow disk may be considered to be in a normal mode, and at this time, unrecorded data of the slow disk during the CID period of the slow disk is recovered by data of other disks and the check data. When the recovery is carried out, the corresponding stripe can be determined according to the stripe number recorded in the grain, and then the data of the corresponding block of the disk is recovered by utilizing the data recorded by other blocks in the stripe and the verification data.
For example, a stripe includes 4 partitions, with the 4 partitions being provided by different disks, respectively. When one of the 4 disks is determined to be a slow disk, only 3 blocks of the stripe can be used for writing data, at this time, the data to be written can be blocked according to normal processing logic, then the parity value of the data block is calculated, and finally a part of the data block and the parity value are stored in the remaining 3 blocks. When the disk is determined not to be a slow disk, data that should be stored to a block corresponding to the slow disk may be restored using data and parity values in the remaining 3 blocks and then stored to a block corresponding to the slow disk.
In some embodiments, obtaining an I/O response time of a disk in a disk array and determining a disk with an I/O response time greater than a threshold as a slow disk further comprises:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
responding to the I/O response time of the disk being larger than the slow disk event judgment threshold value, and updating the occurrence times of the slow disk event of the corresponding disk;
and determining the disk with the occurrence of the slow disk event times larger than the preset times as the slow disk in response to the occurrence of the slow disk event times within the preset time period.
Specifically, the number of times that the I/O response time of each disk is greater than the threshold may be recorded, and when the number of times is greater than the preset number of times, the disk may be considered as a slow disk.
In the scheme provided by the embodiment of the invention, after the slow disk is detected, the data of the slow disk is recorded in other disks in a verification mode, so that the overall RAID reliability can be improved under the condition of not occupying the existing hot spare disk on the premise of not influencing the existing performance. And when data recovery is carried out, the repairing granularity mainly based on the stripes is adopted, the repairing process is more accurate, and the repairing accuracy is ensured.
Based on the same inventive concept, according to another aspect of the present invention, there is also provided a data processing system 400 of a slow disk, as shown in fig. 2, including:
the judging module 401 is configured to obtain the I/O response time of the disks in the disk array, and determine the disk with the I/O response time greater than the threshold as a slow disk;
a first determining module 402, configured to determine, in response to receiving a write request, a stripe corresponding to data to be written according to a stripe number carried in the write request;
a second determining module 403 configured to determine a first partition corresponding to the slow disk and a plurality of second partitions corresponding to other disks on the stripe;
a partitioning module 404 configured to partition the data to be written into a plurality of data blocks and calculate parity information of the plurality of data blocks;
a storage module 405 configured to store a number of the plurality of data blocks and the parity information onto the plurality of second partitions, respectively.
In some embodiments, the system further comprises a recording module configured to:
creating a bitmap;
recording the strip number carried in each received write request into a gram;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
In some embodiments, the system further comprises a recovery module configured to:
traversing the bitmap to determine a gram recorded with a stripe number in response to the I/O response time of the slow disk being less than a threshold;
and determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks except the data blocks in the data blocks by utilizing the data blocks stored in the second blocks and the parity check information, and writing the recovered data blocks into the first blocks.
In some embodiments, the determination module is further configured to:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
responding to the I/O response time of the disk being larger than the slow disk event judgment threshold value, and updating the occurrence times of the slow disk event of the corresponding disk;
and determining the disk with the occurrence of the slow disk event times larger than the preset times as the slow disk in response to the occurrence of the slow disk event times within the preset time period.
According to the scheme provided by the embodiment of the invention, after the slow disk is detected, the data of the slow disk is recorded in other disks in a verification mode, so that the overall RAID reliability can be improved under the condition that the existing hot spare disk is not occupied on the premise of not affecting the existing performance. And when data recovery is carried out, the repairing granularity mainly based on the stripes is adopted, the repairing process is more accurate, and the repairing accuracy is ensured.
Based on the same inventive concept, according to another aspect of the present invention, as shown in fig. 3, an embodiment of the present invention further provides a computer apparatus 501, including:
at least one processor 520; and
the memory 510, the memory 510 stores a computer program 511 executable on a processor, and the processor 520 executes the program to perform the steps of:
s1, acquiring the I/O response time of a disk in a disk array, and determining the disk with the I/O response time larger than a threshold value as a slow disk;
s2, responding to a received writing request, and determining a stripe corresponding to data to be written according to a stripe number carried in the writing request;
s3, determining a first partition corresponding to the slow disk and a plurality of second partitions corresponding to other magnetic disks on the strip;
s4, dividing the data to be written into a plurality of data blocks and calculating parity information of the plurality of data blocks;
s5, storing a plurality of data blocks and the parity information in the plurality of data blocks on the plurality of second blocks respectively.
In some embodiments, further comprising:
creating a bitmap;
recording the strip number carried in each received write request into a gram;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
In some embodiments, further comprising:
traversing the bitmap to determine a gram recorded with a stripe number in response to the I/O response time of the slow disk being less than a threshold;
and determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks except the data blocks in the data blocks by utilizing the data blocks stored in the second blocks and the parity check information, and writing the recovered data blocks into the first blocks.
In some embodiments, obtaining an I/O response time of a disk in a disk array and determining a disk with an I/O response time greater than a threshold as a slow disk further comprises:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
responding to the I/O response time of the disk being larger than the slow disk event judgment threshold value, and updating the occurrence times of the slow disk event of the corresponding disk;
and determining the disk with the occurrence of the slow disk event times larger than the preset times as the slow disk in response to the occurrence of the slow disk event times within the preset time period.
According to the scheme provided by the embodiment of the invention, after the slow disk is detected, the data of the slow disk is recorded in other disks in a verification mode, so that the overall RAID reliability can be improved under the condition that the existing hot spare disk is not occupied on the premise of not affecting the existing performance. And when data recovery is carried out, the repairing granularity mainly based on the stripes is adopted, the repairing process is more accurate, and the repairing accuracy is ensured.
Based on the same inventive concept, according to another aspect of the present invention, as shown in fig. 4, an embodiment of the present invention further provides a computer-readable storage medium 601, the computer-readable storage medium 601 storing computer program instructions 610, the computer program instructions 610 when executed by a processor performing the steps of:
s1, acquiring the I/O response time of a disk in a disk array, and determining the disk with the I/O response time larger than a threshold value as a slow disk;
s2, responding to a received writing request, and determining a stripe corresponding to data to be written according to a stripe number carried in the writing request;
s3, determining a first partition corresponding to the slow disk and a plurality of second partitions corresponding to other magnetic disks on the strip;
s4, dividing the data to be written into a plurality of data blocks and calculating parity information of the plurality of data blocks;
s5, storing a plurality of data blocks and the parity information in the plurality of data blocks on the plurality of second blocks respectively.
In some embodiments, further comprising:
creating a bitmap;
recording the strip number carried in each received write request into a gram;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
In some embodiments, further comprising:
traversing the bitmap to determine a gram recorded with a stripe number in response to the I/O response time of the slow disk being less than a threshold;
and determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks except the data blocks in the data blocks by utilizing the data blocks stored in the second blocks and the parity check information, and writing the recovered data blocks into the first blocks.
In some embodiments, obtaining an I/O response time of a disk in a disk array and determining a disk with an I/O response time greater than a threshold as a slow disk further comprises:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
responding to the I/O response time of the disk being larger than the slow disk event judgment threshold value, and updating the occurrence times of the slow disk event of the corresponding disk;
and determining the disk with the occurrence of the slow disk event times larger than the preset times as the slow disk in response to the occurrence of the slow disk event times within the preset time period.
According to the scheme provided by the embodiment of the invention, after the slow disk is detected, the data of the slow disk is recorded in other disks in a verification mode, so that the overall RAID reliability can be improved under the condition that the existing hot spare disk is not occupied on the premise of not affecting the existing performance. And when data recovery is carried out, the repairing granularity mainly based on the stripes is adopted, the repairing process is more accurate, and the repairing accuracy is ensured.
Finally, it should be noted that, as will be appreciated by those skilled in the art, all or part of the procedures in implementing the methods of the embodiments described above may be implemented by a computer program for instructing relevant hardware, and the program may be stored in a computer readable storage medium, and the program may include the procedures of the embodiments of the methods described above when executed.
Further, it should be appreciated that the computer-readable storage medium (e.g., memory) herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that as used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The foregoing embodiment of the present invention has been disclosed with reference to the number of embodiments for the purpose of description only, and does not represent the advantages or disadvantages of the embodiments.
It will be appreciated by those of ordinary skill in the art that all or part of the steps of implementing the above embodiments may be implemented by hardware, or may be implemented by a program to instruct related hardware, and the program may be stored in a computer readable storage medium, where the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Those of ordinary skill in the art will appreciate that: the above discussion of any embodiment is merely exemplary and is not intended to imply that the scope of the disclosure of embodiments of the invention, including the claims, is limited to such examples; combinations of features of the above embodiments or in different embodiments are also possible within the idea of an embodiment of the invention, and many other variations of the different aspects of the embodiments of the invention as described above exist, which are not provided in detail for the sake of brevity. Therefore, any omission, modification, equivalent replacement, improvement, etc. of the embodiments should be included in the protection scope of the embodiments of the present invention.
Claims (10)
1. A method of processing data for a slow disk, comprising the steps of:
acquiring the I/O response time of a disk in a disk array, and determining the disk with the I/O response time larger than a threshold value as a slow disk;
responding to a received writing request, and determining a stripe corresponding to data to be written according to a stripe number carried in the writing request;
determining a first partition corresponding to the slow disk and a plurality of second partitions corresponding to other disks on the stripe;
dividing the data to be written into a plurality of data blocks and calculating parity information of the plurality of data blocks;
storing a plurality of data blocks of the plurality of data blocks and the parity information on the plurality of second partitions, respectively, and discarding data blocks of the plurality of data blocks except the plurality of data blocks;
and in response to detecting that the slow disk is recovered to be normal, recovering data blocks except the data blocks in the plurality of data blocks according to the data blocks and the parity information stored in the plurality of second partitions, and writing the recovered data blocks into the first partition.
2. The method as recited in claim 1, further comprising:
creating a bitmap;
recording the strip number carried in each received write request into a gram;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
3. The method as recited in claim 2, further comprising:
traversing the bitmap to determine a gram recorded with a stripe number in response to the I/O response time of the slow disk being less than a threshold;
and determining a corresponding stripe according to the stripe number recorded in each grain, and recovering the data blocks except the data blocks in the data blocks by utilizing the data blocks stored in the second blocks and the parity check information.
4. The method of claim 1, wherein obtaining the I/O response times for the disks in the disk array and determining the disk having an I/O response time greater than a threshold as a slow disk, further comprises:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
responding to the I/O response time of the disk being larger than the slow disk event judgment threshold value, and updating the occurrence times of the slow disk event of the corresponding disk;
and determining the disk with the occurrence of the slow disk event times larger than the preset times as the slow disk in response to the occurrence of the slow disk event times within the preset time period.
5. A data processing system for a slow disk comprising:
the judging module is configured to acquire the I/O response time of the disks in the disk array and determine the disk with the I/O response time larger than a threshold value as a slow disk;
the first determining module is configured to respond to a received writing request and determine a stripe corresponding to data to be written according to a stripe number carried in the writing request;
the second determining module is configured to determine a first partition corresponding to the slow disk and a plurality of second partitions corresponding to other disks on the stripe;
a block module configured to divide the data to be written into a plurality of data blocks and calculate parity information of the plurality of data blocks;
a storage module configured to store a plurality of data blocks of the plurality of data blocks and the parity information onto the plurality of second partitions, respectively, and discard data blocks of the plurality of data blocks other than the plurality of data blocks;
and the recovery module is configured to respond to detection that the slow disk is recovered to be normal, recover the data blocks except the data blocks in the data blocks according to the data blocks stored in the second partitions and the parity information, and write the recovered data blocks into the first partition.
6. The system of claim 5, further comprising a recording module configured to:
creating a bitmap;
recording the strip number carried in each received write request into a gram;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
7. The system of claim 6, wherein the recovery module is further configured to:
traversing the bitmap to determine a gram recorded with a stripe number in response to the I/O response time of the slow disk being less than a threshold;
and determining a corresponding stripe according to the stripe number recorded in each grain, and recovering the data blocks except the data blocks in the data blocks by utilizing the data blocks stored in the second blocks and the parity check information.
8. The system of claim 5, wherein the determination module is further configured to:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
responding to the I/O response time of the disk being larger than the slow disk event judgment threshold value, and updating the occurrence times of the slow disk event of the corresponding disk;
and determining the disk with the occurrence of the slow disk event times larger than the preset times as the slow disk in response to the occurrence of the slow disk event times within the preset time period.
9. A computer device, comprising:
at least one processor; and
a memory storing a computer program executable on the processor, wherein the processor performs the steps of the method of any of claims 1-4 when the program is executed.
10. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor performs the steps of the method according to any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110933570.6A CN113849123B (en) | 2021-08-14 | 2021-08-14 | Method, system, equipment and medium for processing data of slow disk |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110933570.6A CN113849123B (en) | 2021-08-14 | 2021-08-14 | Method, system, equipment and medium for processing data of slow disk |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113849123A CN113849123A (en) | 2021-12-28 |
| CN113849123B true CN113849123B (en) | 2023-08-25 |
Family
ID=78975376
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110933570.6A Active CN113849123B (en) | 2021-08-14 | 2021-08-14 | Method, system, equipment and medium for processing data of slow disk |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113849123B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115934003B (en) * | 2023-03-09 | 2023-05-16 | 浪潮电子信息产业股份有限公司 | Method, device and equipment for identifying slow disk in disk array and readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102981778A (en) * | 2012-11-15 | 2013-03-20 | 浙江宇视科技有限公司 | Redundant array of independent disks (RAID) array reconstruction method and device thereof |
| CN106354590A (en) * | 2015-07-17 | 2017-01-25 | 中兴通讯股份有限公司 | Method and apparatus for disk detection |
| CN109634523A (en) * | 2018-12-10 | 2019-04-16 | 深圳市网心科技有限公司 | A kind of disk management method, system and electronic equipment and storage medium |
| CN109684140A (en) * | 2018-12-11 | 2019-04-26 | 广东浪潮大数据研究有限公司 | A kind of slow disk detection method, device, equipment and computer readable storage medium |
| CN111857540A (en) * | 2019-04-25 | 2020-10-30 | 伊姆西Ip控股有限责任公司 | Data access method, device and computer program product |
| CN112148218A (en) * | 2020-09-11 | 2020-12-29 | 北京浪潮数据技术有限公司 | Method, device and equipment for storing check data of disk array and storage medium |
-
2021
- 2021-08-14 CN CN202110933570.6A patent/CN113849123B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102981778A (en) * | 2012-11-15 | 2013-03-20 | 浙江宇视科技有限公司 | Redundant array of independent disks (RAID) array reconstruction method and device thereof |
| CN106354590A (en) * | 2015-07-17 | 2017-01-25 | 中兴通讯股份有限公司 | Method and apparatus for disk detection |
| CN109634523A (en) * | 2018-12-10 | 2019-04-16 | 深圳市网心科技有限公司 | A kind of disk management method, system and electronic equipment and storage medium |
| CN109684140A (en) * | 2018-12-11 | 2019-04-26 | 广东浪潮大数据研究有限公司 | A kind of slow disk detection method, device, equipment and computer readable storage medium |
| CN111857540A (en) * | 2019-04-25 | 2020-10-30 | 伊姆西Ip控股有限责任公司 | Data access method, device and computer program product |
| CN112148218A (en) * | 2020-09-11 | 2020-12-29 | 北京浪潮数据技术有限公司 | Method, device and equipment for storing check data of disk array and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113849123A (en) | 2021-12-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9424141B2 (en) | Hard disk data recovery method, apparatus, and system | |
| JP4988214B2 (en) | Detection and recovery of lost writes in storage | |
| US7681071B2 (en) | Storage apparatus, control method therefor and program | |
| CN109684140B (en) | Slow disk detection method, device and equipment and computer readable storage medium | |
| US7640452B2 (en) | Method for reconstructing data in case of two disk drives of RAID failure and system therefor | |
| CN104166601B (en) | The backup method and device of a kind of data storage | |
| US10572335B2 (en) | Metadata recovery method and apparatus | |
| CN108255414B (en) | Solid state disk access method and device | |
| CN113849123B (en) | Method, system, equipment and medium for processing data of slow disk | |
| CN108170375B (en) | Overrun protection method and device in distributed storage system | |
| WO2021027271A1 (en) | Bad block information protection method and apparatus, computer device and storage medium | |
| CN111816239A (en) | Disk detection method and device, electronic equipment and machine-readable storage medium | |
| CN112068770B (en) | Stripe write optimization method based on RAID | |
| CN115599589B (en) | Data recovery method and related device | |
| Tsai et al. | A study of soft error consequences in hard disk drives | |
| CN110688242A (en) | Method for recovering block information after abnormal power failure of SSD and related components | |
| CN113190179B (en) | Method for prolonging service life of mechanical hard disk, storage device and system | |
| US20220374310A1 (en) | Write request completion notification in response to partial hardening of write data | |
| US9798473B2 (en) | Storage volume device and method for increasing write speed for data streams while providing data protection | |
| WO2021170048A1 (en) | Data storage method and apparatus, and storage medium | |
| CN111026332B (en) | SSD bad block information protection method, SSD bad block information protection device, computer equipment and storage medium | |
| CN109344011B (en) | Data backup method and device | |
| CN114930299A (en) | Method, device and storage medium for managing data storage array | |
| CN112562774B (en) | Storage device mounting method and device, computer device and storage medium | |
| US20240176489A1 (en) | Priority based raid (redundant array of independent disks) rebuild for a data storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |