CN113849123A - Data processing method, system, equipment and medium for slow disk - Google Patents
Data processing method, system, equipment and medium for slow disk Download PDFInfo
- Publication number
- CN113849123A CN113849123A CN202110933570.6A CN202110933570A CN113849123A CN 113849123 A CN113849123 A CN 113849123A CN 202110933570 A CN202110933570 A CN 202110933570A CN 113849123 A CN113849123 A CN 113849123A
- Authority
- CN
- China
- Prior art keywords
- disk
- slow
- disks
- data blocks
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 8
- 230000004044 response Effects 0.000 claims abstract description 104
- 238000011084 recovery Methods 0.000 claims description 24
- 238000005192 partition Methods 0.000 claims description 16
- 238000000034 method Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013524 data verification Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0653—Monitoring storage devices or systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0689—Disk arrays, e.g. RAID, JBOD
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Quality & Reliability (AREA)
- Signal Processing For Digital Recording And Reproducing (AREA)
Abstract
The invention discloses a data processing method of a slow disc, which comprises the following steps: acquiring the I/O response time of the disks in the disk array, and determining the disks with the I/O response time larger than a threshold value as slow disks; in response to receiving the write request, determining a stripe corresponding to data to be written according to a stripe number carried in the write request; determining a first block corresponding to a slow disk and a plurality of second blocks corresponding to other disks on a strip; dividing data to be written into a plurality of data blocks and calculating parity check information of the data blocks; and respectively storing a plurality of data blocks in the plurality of data blocks and the parity check information on the plurality of second blocks. The invention also discloses a system, a computer device and a readable storage medium. According to the scheme provided by the embodiment of the invention, after the slow disk is detected, the data of the slow disk is recorded in other disks in a checking mode, so that the reliability of the whole RAID can be improved under the condition of not occupying the existing hot spare disk.
Description
Technical Field
The invention relates to the field of storage, in particular to a data processing method, a system, equipment and a storage medium of a slow disk.
Background
RAID is a virtualization technology that virtualizes a plurality of physical disks into a large-capacity logical disk to an external host. In order to ensure the high performance and reliability of the logical disk, the RAID adopts key technologies such as mirror redundancy, data striping, and data verification in the development process. Whereas RAID slow disk management is one way to maintain RAID reliability.
Most of the conventional common slow disk processing modes are that a marked slow disk is directly reconstructed into a hot standby disk, and the slow disk is copied back after being normal. However, the condition of the existing hot spare disk is occupied, and the subsequent stability of the RAID array is reduced due to insufficient hot spare.
Disclosure of Invention
In view of the above, in order to overcome at least one aspect of the above problems, an embodiment of the present invention provides a data processing method for a slow disc, including the following steps:
acquiring I/O (Input/Output) response time of a disk in a disk array, and determining the disk with the I/O response time larger than a threshold value as a slow disk;
in response to receiving a write request, determining a stripe corresponding to data to be written according to a stripe number carried in the write request;
determining a first block corresponding to the slow disk and a plurality of second blocks corresponding to other disks on the strip;
dividing the data to be written into a plurality of data blocks and calculating parity check information of the data blocks;
storing a number of the plurality of data blocks and the parity information on the plurality of second partitions, respectively.
In some embodiments, further comprising:
creating a bitmap;
recording the strip number carried in each received write request into a grain;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
In some embodiments, further comprising:
traversing the bitmap to determine a grain with a stripe number recorded in response to the I/O response time of the slow disk being less than a threshold;
determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks, except the data blocks, in the plurality of data blocks by using the data blocks and the parity information stored in the second blocks on the corresponding stripe, and writing the recovered data blocks into the first blocks.
In some embodiments, obtaining the I/O response time of the disks in the disk array, and determining the disk whose I/O response time is greater than the threshold as a slow disk, further includes:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
updating the number of occurrences of the slow disk events of the corresponding disk in response to the fact that the I/O response time of the disk is larger than the slow disk event judgment threshold;
and responding to the fact that the magnetic disk with the occurrence frequency of the slow disk events larger than the preset frequency is determined as the slow disk within the preset time period.
Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a data processing system of a slow disc, including:
the judging module is configured to acquire the I/O response time of the disks in the disk array and determine the disks with the I/O response time larger than a threshold value as slow disks;
the first determining module is configured to respond to a received write request, and determine a stripe corresponding to data to be written according to a stripe number carried in the write request;
a second determination module configured to determine a first partition corresponding to the slow disk and a plurality of second partitions corresponding to other disks on the stripe;
a block dividing module configured to divide the data to be written into a plurality of data blocks and calculate parity information of the plurality of data blocks;
a storage module configured to store a number of the plurality of data blocks and the parity information on the plurality of second partitions, respectively.
In some embodiments, the system further comprises a recording module configured to:
creating a bitmap;
recording the strip number carried in each received write request into a grain;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
In some embodiments, further comprising a recovery module configured to:
traversing the bitmap to determine a grain with a stripe number recorded in response to the I/O response time of the slow disk being less than a threshold;
determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks, except the data blocks, in the plurality of data blocks by using the data blocks and the parity information stored in the second blocks on the corresponding stripe, and writing the recovered data blocks into the first blocks.
In some embodiments, the determination module is further configured to:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
updating the number of occurrences of the slow disk events of the corresponding disk in response to the fact that the I/O response time of the disk is larger than the slow disk event judgment threshold;
and responding to the fact that the magnetic disk with the occurrence frequency of the slow disk events larger than the preset frequency is determined as the slow disk within the preset time period.
Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer apparatus, including:
at least one processor; and
a memory storing a computer program operable on the processor, wherein the processor executes the program to perform the steps of:
acquiring the I/O response time of the disks in the disk array, and determining the disks with the I/O response time larger than a threshold value as slow disks;
in response to receiving a write request, determining a stripe corresponding to data to be written according to a stripe number carried in the write request;
determining a first block corresponding to the slow disk and a plurality of second blocks corresponding to other disks on the strip;
dividing the data to be written into a plurality of data blocks and calculating parity check information of the data blocks;
storing a number of the plurality of data blocks and the parity information on the plurality of second partitions, respectively.
In some embodiments, further comprising:
creating a bitmap;
recording the strip number carried in each received write request into a grain;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
In some embodiments, further comprising:
traversing the bitmap to determine a grain with a stripe number recorded in response to the I/O response time of the slow disk being less than a threshold;
determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks, except the data blocks, in the plurality of data blocks by using the data blocks and the parity information stored in the second blocks on the corresponding stripe, and writing the recovered data blocks into the first blocks.
In some embodiments, obtaining the I/O response time of the disks in the disk array, and determining the disk whose I/O response time is greater than the threshold as a slow disk, further includes:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
updating the number of occurrences of the slow disk events of the corresponding disk in response to the fact that the I/O response time of the disk is larger than the slow disk event judgment threshold;
and responding to the fact that the magnetic disk with the occurrence frequency of the slow disk events larger than the preset frequency is determined as the slow disk within the preset time period.
Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer-readable storage medium storing a computer program which, when executed by a processor, performs the steps of:
acquiring the I/O response time of the disks in the disk array, and determining the disks with the I/O response time larger than a threshold value as slow disks;
in response to receiving a write request, determining a stripe corresponding to data to be written according to a stripe number carried in the write request;
determining a first block corresponding to the slow disk and a plurality of second blocks corresponding to other disks on the strip;
dividing the data to be written into a plurality of data blocks and calculating parity check information of the data blocks;
storing a number of the plurality of data blocks and the parity information on the plurality of second partitions, respectively.
In some embodiments, further comprising:
creating a bitmap;
recording the strip number carried in each received write request into a grain;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
In some embodiments, further comprising:
traversing the bitmap to determine a grain with a stripe number recorded in response to the I/O response time of the slow disk being less than a threshold;
determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks, except the data blocks, in the plurality of data blocks by using the data blocks and the parity information stored in the second blocks on the corresponding stripe, and writing the recovered data blocks into the first blocks.
In some embodiments, obtaining the I/O response time of the disks in the disk array, and determining the disk whose I/O response time is greater than the threshold as a slow disk, further includes:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
updating the number of occurrences of the slow disk events of the corresponding disk in response to the fact that the I/O response time of the disk is larger than the slow disk event judgment threshold;
and responding to the fact that the magnetic disk with the occurrence frequency of the slow disk events larger than the preset frequency is determined as the slow disk within the preset time period.
The invention has one of the following beneficial technical effects: according to the scheme provided by the embodiment of the invention, after the slow disk is detected, the data of the slow disk is recorded in other disks in a checking mode, so that the whole RAID reliability can be improved under the condition of not occupying the existing hot spare disk on the premise of not influencing the existing performance. And when data recovery is carried out, the recovery granularity which mainly comprises the strip is adopted, so that the recovery process is more accurate, and the recovery accuracy is ensured.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a data processing method of a slow disc according to an embodiment of the present invention;
FIG. 2 is a block diagram of a data processing system for a slow disk according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a computer device provided in an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two entities with the same name but different names or different parameters, and it should be noted that "first" and "second" are merely for convenience of description and should not be construed as limitations of the embodiments of the present invention, and they are not described in any more detail in the following embodiments.
In the embodiment of the invention, the stripe is a method for dividing continuous data into data blocks with the same size and writing each data block into different disks in the array respectively, namely, the blocks on different disks form the stripe. At least one of the blocks that make up the stripe stores parity information, and any two blocks within the stripe are distributed across different disks. The size of the stripe is equal to the number of disks for storing data multiplied by the size of the segment constituting the stripe on each disk, that is, one stripe cuts a plurality of disks arranged into a stripe in the transverse direction, the hard disk is equivalent to a vertical bar, and the small lattice formed after the transverse bar and the vertical bar are cut is the segment, also called the stripe depth.
According to an aspect of the present invention, an embodiment of the present invention provides a data processing method for a slow disc, as shown in fig. 1, which may include the steps of:
s1, obtaining the I/O response time of the disks in the disk array, and determining the disks with the I/O response time larger than a threshold value as slow disks;
s2, responding to the received write request, and determining the stripe corresponding to the data to be written according to the stripe number carried in the write request;
s3, determining a first block corresponding to the slow disk and a plurality of second blocks corresponding to other disks on the strip;
s4, dividing the data to be written into a plurality of data blocks and calculating the parity check information of the data blocks;
s5, storing the plurality of data blocks and the parity information in the plurality of data blocks to the plurality of second partitions respectively.
According to the scheme provided by the embodiment of the invention, after the slow disk is detected, the data of the slow disk is recorded in other disks in a checking mode, so that the whole RAID reliability can be improved under the condition of not occupying the existing hot spare disk on the premise of not influencing the existing performance.
In some embodiments, the disk array generally includes a plurality of disks for storing data, a disk for storing check data, and a disk for hot standby, and when a slow disk is detected, the disk may be considered to enter a CID mode (which is suspicious for a currently problematic disk), that is, the disk is in a state of being temporarily unavailable for RAID, and until the disk is normal, the I/O data of this period of time is not written in the disk in the CID mode and is not written in the hot standby disk, but other data and parity data corresponding to the other data are stored in other disks.
Therefore, when a slow disk is detected to appear, and a write request I/O is received at the moment, a stripe to be written with data to be written in the write request is determined at first, each block in the stripe is from different magnetic disks, the data to be stored is divided into different data blocks when data storage is normally carried out, then parity check information of a plurality of data blocks is calculated, and other data blocks and check data in the plurality of data blocks are written into blocks, corresponding to other magnetic disks, of the stripe because the slow disk is in an unavailable state at the moment, namely the slow disk cannot provide storage service.
For example, the stripe to be written includes 4 blocks, and the 4 blocks are provided by different disks, respectively. When one disk of the 4 disks is determined as a slow disk, only 3 blocks of the stripe can write data, at this time, the data to be written can be blocked according to normal processing logic, then the parity check value of the data block is calculated, and finally a part of the data block and the parity check value are stored in the remaining 3 blocks, that is, the data block of the block corresponding to the slow disk which should be stored in the stripe is temporarily discarded.
In some embodiments, further comprising:
creating a bitmap;
recording the strip number carried in each received write request into a grain;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
Specifically, when a slow disc occurs, a grain (one unit of measure) may be added to record the number of the band. A bitmap can then be created, with how long the bitmap created is, how many grains. Each of the grams corresponds to a position in the bitmap, and after the stripe number is recorded in the gram, the flag bit of the position corresponding to the bitmap can be modified to a first preset value (e.g., 1).
In some embodiments, further comprising:
traversing the bitmap to determine a grain with a stripe number recorded in response to the I/O response time of the slow disk being less than a threshold;
determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks, except the data blocks, in the plurality of data blocks by using the data blocks and the parity information stored in the second blocks on the corresponding stripe, and writing the recovered data blocks into the first blocks.
Specifically, when the I/O response time of the slow disk is less than the threshold, the slow disk may be considered to be in the normal mode, and at this time, the data that is not recorded during CID of the disk is recovered through the data of other disks and the check data. When recovery is performed, a corresponding stripe can be determined according to the stripe number recorded in the grain, and then the data of the corresponding block of the disk is recovered by using the data and the check data recorded in other blocks in the stripe.
For example, a stripe includes 4 blocks, and the 4 blocks are provided by different disks, respectively. When one disk of the 4 disks is determined to be a slow disk, only 3 blocks of the stripe can be written with data, at this time, the data to be written can be blocked according to normal processing logic, then the parity check value of the data block is calculated, and finally a part of the data block and the parity check value are stored in the remaining 3 blocks. When the disk is determined not to be the slow disk, the data that should be stored in the block corresponding to the slow disk can be recovered by using the data and parity values in the remaining 3 blocks, and then stored in the block corresponding to the slow disk.
In some embodiments, obtaining the I/O response time of the disks in the disk array, and determining the disk whose I/O response time is greater than the threshold as a slow disk, further includes:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
updating the number of occurrences of the slow disk events of the corresponding disk in response to the fact that the I/O response time of the disk is larger than the slow disk event judgment threshold;
and responding to the fact that the magnetic disk with the occurrence frequency of the slow disk events larger than the preset frequency is determined as the slow disk within the preset time period.
Specifically, the number of times that the I/O response time of each disk is greater than the threshold may be recorded, and when the number of times is greater than the preset number, the disk may be considered as a slow disk.
In the scheme provided by the embodiment of the invention, after the slow disk is detected, the data of the slow disk is recorded in other disks in a checking mode, so that the whole RAID reliability can be improved without occupying the existing hot spare disk on the premise of not influencing the existing performance. And when data recovery is carried out, the recovery granularity which mainly comprises the strip is adopted, so that the recovery process is more accurate, and the recovery accuracy is ensured.
Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a data processing system 400 of a slow disc, as shown in fig. 2, including:
the determining module 401 is configured to obtain I/O response time of a disk in a disk array, and determine the disk with the I/O response time greater than a threshold as a slow disk;
a first determining module 402, configured to determine, in response to receiving a write request, a stripe corresponding to data to be written according to a stripe number carried in the write request;
a second determining module 403 configured to determine a first partition corresponding to the slow disk and a plurality of second partitions corresponding to other disks on the stripe;
a block dividing module 404 configured to divide the data to be written into a plurality of data blocks and calculate parity information of the plurality of data blocks;
a storage module 405 configured to store the plurality of data blocks and the parity information on the plurality of second partitions, respectively.
In some embodiments, the system further comprises a recording module configured to:
creating a bitmap;
recording the strip number carried in each received write request into a grain;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
In some embodiments, further comprising a recovery module configured to:
traversing the bitmap to determine a grain with a stripe number recorded in response to the I/O response time of the slow disk being less than a threshold;
determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks, except the data blocks, in the plurality of data blocks by using the data blocks and the parity information stored in the second blocks on the corresponding stripe, and writing the recovered data blocks into the first blocks.
In some embodiments, the determination module is further configured to:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
updating the number of occurrences of the slow disk events of the corresponding disk in response to the fact that the I/O response time of the disk is larger than the slow disk event judgment threshold;
and responding to the fact that the magnetic disk with the occurrence frequency of the slow disk events larger than the preset frequency is determined as the slow disk within the preset time period.
According to the scheme provided by the embodiment of the invention, after the slow disk is detected, the data of the slow disk is recorded in other disks in a checking mode, so that the whole RAID reliability can be improved under the condition of not occupying the existing hot spare disk on the premise of not influencing the existing performance. And when data recovery is carried out, the recovery granularity which mainly comprises the strip is adopted, so that the recovery process is more accurate, and the recovery accuracy is ensured.
Based on the same inventive concept, according to another aspect of the present invention, as shown in fig. 3, an embodiment of the present invention further provides a computer apparatus 501, comprising:
at least one processor 520; and
a memory 510, the memory 510 storing a computer program 511 executable on the processor, the processor 520 executing the program to perform the steps of:
s1, obtaining the I/O response time of the disks in the disk array, and determining the disks with the I/O response time larger than a threshold value as slow disks;
s2, responding to the received write request, and determining the stripe corresponding to the data to be written according to the stripe number carried in the write request;
s3, determining a first block corresponding to the slow disk and a plurality of second blocks corresponding to other disks on the strip;
s4, dividing the data to be written into a plurality of data blocks and calculating the parity check information of the data blocks;
s5, storing the plurality of data blocks and the parity information in the plurality of data blocks to the plurality of second partitions respectively.
In some embodiments, further comprising:
creating a bitmap;
recording the strip number carried in each received write request into a grain;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
In some embodiments, further comprising:
traversing the bitmap to determine a grain with a stripe number recorded in response to the I/O response time of the slow disk being less than a threshold;
determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks, except the data blocks, in the plurality of data blocks by using the data blocks and the parity information stored in the second blocks on the corresponding stripe, and writing the recovered data blocks into the first blocks.
In some embodiments, obtaining the I/O response time of the disks in the disk array, and determining the disk whose I/O response time is greater than the threshold as a slow disk, further includes:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
updating the number of occurrences of the slow disk events of the corresponding disk in response to the fact that the I/O response time of the disk is larger than the slow disk event judgment threshold;
and responding to the fact that the magnetic disk with the occurrence frequency of the slow disk events larger than the preset frequency is determined as the slow disk within the preset time period.
According to the scheme provided by the embodiment of the invention, after the slow disk is detected, the data of the slow disk is recorded in other disks in a checking mode, so that the whole RAID reliability can be improved under the condition of not occupying the existing hot spare disk on the premise of not influencing the existing performance. And when data recovery is carried out, the recovery granularity which mainly comprises the strip is adopted, so that the recovery process is more accurate, and the recovery accuracy is ensured.
Based on the same inventive concept, according to another aspect of the present invention, as shown in fig. 4, an embodiment of the present invention further provides a computer-readable storage medium 601, where the computer-readable storage medium 601 stores computer program instructions 610, and the computer program instructions 610, when executed by a processor, perform the following steps:
s1, obtaining the I/O response time of the disks in the disk array, and determining the disks with the I/O response time larger than a threshold value as slow disks;
s2, responding to the received write request, and determining the stripe corresponding to the data to be written according to the stripe number carried in the write request;
s3, determining a first block corresponding to the slow disk and a plurality of second blocks corresponding to other disks on the strip;
s4, dividing the data to be written into a plurality of data blocks and calculating the parity check information of the data blocks;
s5, storing the plurality of data blocks and the parity information in the plurality of data blocks to the plurality of second partitions respectively.
In some embodiments, further comprising:
creating a bitmap;
recording the strip number carried in each received write request into a grain;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
In some embodiments, further comprising:
traversing the bitmap to determine a grain with a stripe number recorded in response to the I/O response time of the slow disk being less than a threshold;
determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks, except the data blocks, in the plurality of data blocks by using the data blocks and the parity information stored in the second blocks on the corresponding stripe, and writing the recovered data blocks into the first blocks.
In some embodiments, obtaining the I/O response time of the disks in the disk array, and determining the disk whose I/O response time is greater than the threshold as a slow disk, further includes:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
updating the number of occurrences of the slow disk events of the corresponding disk in response to the fact that the I/O response time of the disk is larger than the slow disk event judgment threshold;
and responding to the fact that the magnetic disk with the occurrence frequency of the slow disk events larger than the preset frequency is determined as the slow disk within the preset time period.
According to the scheme provided by the embodiment of the invention, after the slow disk is detected, the data of the slow disk is recorded in other disks in a checking mode, so that the whole RAID reliability can be improved under the condition of not occupying the existing hot spare disk on the premise of not influencing the existing performance. And when data recovery is carried out, the recovery granularity which mainly comprises the strip is adopted, so that the recovery process is more accurate, and the recovery accuracy is ensured.
Finally, it should be noted that, as will be understood by those skilled in the art, all or part of the processes of the methods of the above embodiments may be implemented by a computer program, which may be stored in a computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above.
Further, it should be appreciated that the computer-readable storage media (e.g., memory) herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps of implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.
Claims (10)
1. A data processing method for a slow disc, comprising the steps of:
acquiring the I/O response time of the disks in the disk array, and determining the disks with the I/O response time larger than a threshold value as slow disks;
in response to receiving a write request, determining a stripe corresponding to data to be written according to a stripe number carried in the write request;
determining a first block corresponding to the slow disk and a plurality of second blocks corresponding to other disks on the strip;
dividing the data to be written into a plurality of data blocks and calculating parity check information of the data blocks;
storing a number of the plurality of data blocks and the parity information on the plurality of second partitions, respectively.
2. The method of claim 1, further comprising:
creating a bitmap;
recording the strip number carried in each received write request into a grain;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
3. The method of claim 2, further comprising:
traversing the bitmap to determine a grain with a stripe number recorded in response to the I/O response time of the slow disk being less than a threshold;
determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks, except the data blocks, in the plurality of data blocks by using the data blocks and the parity information stored in the second blocks on the corresponding stripe, and writing the recovered data blocks into the first blocks.
4. The method of claim 1, wherein obtaining I/O response times of disks in a disk array, and determining a disk with the I/O response time greater than a threshold as a slow disk, further comprises:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
updating the number of occurrences of the slow disk events of the corresponding disk in response to the fact that the I/O response time of the disk is larger than the slow disk event judgment threshold;
and responding to the fact that the magnetic disk with the occurrence frequency of the slow disk events larger than the preset frequency is determined as the slow disk within the preset time period.
5. A slow disk data processing system, comprising:
the judging module is configured to acquire the I/O response time of the disks in the disk array and determine the disks with the I/O response time larger than a threshold value as slow disks;
the first determining module is configured to respond to a received write request, and determine a stripe corresponding to data to be written according to a stripe number carried in the write request;
a second determination module configured to determine a first partition corresponding to the slow disk and a plurality of second partitions corresponding to other disks on the stripe;
a block dividing module configured to divide the data to be written into a plurality of data blocks and calculate parity information of the plurality of data blocks;
a storage module configured to store a number of the plurality of data blocks and the parity information on the plurality of second partitions, respectively.
6. The system of claim 5, further comprising a logging module configured to:
creating a bitmap;
recording the strip number carried in each received write request into a grain;
and modifying the zone bit of the position corresponding to the grain recorded with the strip number in the bitmap into a first preset value.
7. The system of claim 6, further comprising a recovery module configured to:
traversing the bitmap to determine a grain with a stripe number recorded in response to the I/O response time of the slow disk being less than a threshold;
determining a corresponding stripe according to the stripe number recorded in each grain, recovering data blocks, except the data blocks, in the plurality of data blocks by using the data blocks and the parity information stored in the second blocks on the corresponding stripe, and writing the recovered data blocks into the first blocks.
8. The system of claim 5, wherein the determination module is further configured to:
acquiring the I/O response time of the disks in the disk array, and comparing the I/O response time of each disk with a slow disk event judgment threshold;
updating the number of occurrences of the slow disk events of the corresponding disk in response to the fact that the I/O response time of the disk is larger than the slow disk event judgment threshold;
and responding to the fact that the magnetic disk with the occurrence frequency of the slow disk events larger than the preset frequency is determined as the slow disk within the preset time period.
9. A computer device, comprising:
at least one processor; and
memory storing a computer program operable on the processor, characterized in that the processor executes the program to perform the steps of the method according to any of claims 1-4.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, is adapted to carry out the steps of the method according to any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110933570.6A CN113849123B (en) | 2021-08-14 | 2021-08-14 | Method, system, equipment and medium for processing data of slow disk |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110933570.6A CN113849123B (en) | 2021-08-14 | 2021-08-14 | Method, system, equipment and medium for processing data of slow disk |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113849123A true CN113849123A (en) | 2021-12-28 |
| CN113849123B CN113849123B (en) | 2023-08-25 |
Family
ID=78975376
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110933570.6A Active CN113849123B (en) | 2021-08-14 | 2021-08-14 | Method, system, equipment and medium for processing data of slow disk |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113849123B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115934003A (en) * | 2023-03-09 | 2023-04-07 | 浪潮电子信息产业股份有限公司 | Slow disk identification method, device and equipment in disk array and readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102981778A (en) * | 2012-11-15 | 2013-03-20 | 浙江宇视科技有限公司 | Redundant array of independent disks (RAID) array reconstruction method and device thereof |
| CN106354590A (en) * | 2015-07-17 | 2017-01-25 | 中兴通讯股份有限公司 | Method and apparatus for disk detection |
| CN109634523A (en) * | 2018-12-10 | 2019-04-16 | 深圳市网心科技有限公司 | A kind of disk management method, system and electronic equipment and storage medium |
| CN109684140A (en) * | 2018-12-11 | 2019-04-26 | 广东浪潮大数据研究有限公司 | A kind of slow disk detection method, device, equipment and computer readable storage medium |
| CN111857540A (en) * | 2019-04-25 | 2020-10-30 | 伊姆西Ip控股有限责任公司 | Data access method, device and computer program product |
| CN112148218A (en) * | 2020-09-11 | 2020-12-29 | 北京浪潮数据技术有限公司 | Method, device and equipment for storing check data of disk array and storage medium |
-
2021
- 2021-08-14 CN CN202110933570.6A patent/CN113849123B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102981778A (en) * | 2012-11-15 | 2013-03-20 | 浙江宇视科技有限公司 | Redundant array of independent disks (RAID) array reconstruction method and device thereof |
| CN106354590A (en) * | 2015-07-17 | 2017-01-25 | 中兴通讯股份有限公司 | Method and apparatus for disk detection |
| CN109634523A (en) * | 2018-12-10 | 2019-04-16 | 深圳市网心科技有限公司 | A kind of disk management method, system and electronic equipment and storage medium |
| CN109684140A (en) * | 2018-12-11 | 2019-04-26 | 广东浪潮大数据研究有限公司 | A kind of slow disk detection method, device, equipment and computer readable storage medium |
| CN111857540A (en) * | 2019-04-25 | 2020-10-30 | 伊姆西Ip控股有限责任公司 | Data access method, device and computer program product |
| CN112148218A (en) * | 2020-09-11 | 2020-12-29 | 北京浪潮数据技术有限公司 | Method, device and equipment for storing check data of disk array and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115934003A (en) * | 2023-03-09 | 2023-04-07 | 浪潮电子信息产业股份有限公司 | Slow disk identification method, device and equipment in disk array and readable storage medium |
| CN115934003B (en) * | 2023-03-09 | 2023-05-16 | 浪潮电子信息产业股份有限公司 | Method, device and equipment for identifying slow disk in disk array and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113849123B (en) | 2023-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10705932B2 (en) | Method, device and computer program product for managing a storage system | |
| US7681071B2 (en) | Storage apparatus, control method therefor and program | |
| US7640452B2 (en) | Method for reconstructing data in case of two disk drives of RAID failure and system therefor | |
| US7818524B2 (en) | Data migration systems and methods for independent storage device expansion and adaptation | |
| US7995300B2 (en) | Detection of defective tape drive by aggregating read error statistics | |
| CN111857592A (en) | Data storage method and device based on object storage system and electronic equipment | |
| US11947413B2 (en) | Dynamic system log preprocessing | |
| US20180210784A1 (en) | Metadata Recovery Method and Apparatus | |
| US20140317443A1 (en) | Method and apparatus for testing a storage system | |
| US8843781B1 (en) | Managing drive error information in data storage systems | |
| CN113849123B (en) | Method, system, equipment and medium for processing data of slow disk | |
| US20060215456A1 (en) | Disk array data protective system and method | |
| CN102609223B (en) | Redundant array of independent disk system and initializing method thereof | |
| CN108170375B (en) | Overrun protection method and device in distributed storage system | |
| WO2021027271A1 (en) | Bad block information protection method and apparatus, computer device and storage medium | |
| US20060075287A1 (en) | Detecting data integrity | |
| CN110348245B (en) | Data integrity protection method, system and device based on NVM and storage medium | |
| CN107545022B (en) | Disk management method and device | |
| WO2021170048A1 (en) | Data storage method and apparatus, and storage medium | |
| CN112068770B (en) | Stripe write optimization method based on RAID | |
| CN111026332B (en) | SSD bad block information protection method, SSD bad block information protection device, computer equipment and storage medium | |
| US20130198563A1 (en) | Disk storage system with rebuild sequence and method of operation thereof | |
| CN109542687B (en) | RAID level conversion method and device | |
| US10599343B2 (en) | Proactively resilvering a striped disk array in advance of a predicted disk drive failure | |
| CN115599589B (en) | Data recovery method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |