CN110658994B - Data processing method and device based on HDD (hard disk drive) and SSD (solid State disk) hybrid disk array - Google Patents
Data processing method and device based on HDD (hard disk drive) and SSD (solid State disk) hybrid disk array Download PDFInfo
- Publication number
- CN110658994B CN110658994B CN201910828441.3A CN201910828441A CN110658994B CN 110658994 B CN110658994 B CN 110658994B CN 201910828441 A CN201910828441 A CN 201910828441A CN 110658994 B CN110658994 B CN 110658994B
- Authority
- CN
- China
- Prior art keywords
- data
- disk
- check
- ssd
- disk array
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000007787 solid Substances 0.000 title claims abstract description 16
- 238000003672 processing method Methods 0.000 title claims abstract description 15
- 238000012545 processing Methods 0.000 claims abstract description 30
- 238000000034 method Methods 0.000 claims abstract description 26
- 238000004364 calculation method Methods 0.000 claims description 49
- 238000012795 verification Methods 0.000 claims description 36
- 238000013500 data storage Methods 0.000 claims description 5
- 238000007906 compression Methods 0.000 claims description 4
- 230000006835 compression Effects 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 abstract description 8
- 238000013144 data compression Methods 0.000 description 2
- 230000006837 decompression Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000013403 standard screening design Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1004—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's to protect a block of data words, e.g. CRC or checksum
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/062—Securing storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0685—Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
Abstract
The embodiment of the application discloses a data processing method and a data processing device based on a HDD (hard disk drive) and SSD (solid state disk) hybrid disk array, which not only improve the data processing rate of the disk array, but also improve the data processing performance of a network transmission and storage system. The method comprises the following steps: the method comprises the steps of firstly receiving a target data writing request, then reading out storage data of a corresponding data disk HDD in a disk array according to the target data writing request, then calculating to obtain second check data by utilizing the target data, the storage data and first check data stored by a check disk SSD in the disk array, and further writing the target data into the corresponding data disk HDD in the disk array and writing the second check data into the check disk SSD in the disk array.
Description
Technical Field
The application relates to the technical field of data processing, in particular to a data processing method and device based on a HDD and SSD hybrid disk array.
Background
With the rapid development of computer technology, the network data processing amount is also increased with storm wind, and nowadays, in order to improve the read-write performance in the data processing process and ensure the security of data, data is stored in a form of a disk array, such as a commonly used RAID-3 disk array, and the data throughput of a storage system is greatly improved by storing and reading data on a plurality of disks in the disk array at the same time.
In general, a RAID-3 Disk array includes a check Disk and a plurality of data disks, and the Disk types of the check Disk and the data disks are the same, and are all Hard Disk drives (Hard Disk drives for short), but in this storage manner, in a scenario of writing data randomly, if it is assumed that data needs to be written into Disk arrays of N different stripes and data is written into only one different data Disk in the Disk array of each stripe, data needs to be read and written once on each data Disk, and data needs to be read and written N times on the check Disk. Therefore, in a scene of randomly writing data with a large load, because the read-write speed of the check disk HDD is low, the check disk HDD is likely to become a performance bottleneck of the disk array, and even when the storage system is busy, the check disk HDD becomes a performance bottleneck of the whole storage system. This will seriously affect the data processing performance of the network transmission and storage system.
Therefore, how to increase the data processing rate of the RAID-3 disk array so as to further improve the data processing performance of the network transmission and storage system becomes a problem to be solved.
Disclosure of Invention
In order to solve the above problems, the present application provides a data processing method and apparatus based on a HDD and SSD (Solid State Disk, SSD for short) hybrid Disk array, and the specific technical solution is as follows:
in a first aspect, the present application provides a data processing method based on a HDD and SSD hybrid disk array, where the method is applied to a RAID-3 disk array, a data disk in the disk array is a mechanical hard disk HDD, and a check disk in the disk array is a solid state hard disk SSD, and the method includes:
receiving a target data write request;
reading the storage data of the corresponding data disk HDD in the disk array according to the target data writing request;
calculating to obtain second check data by using the target data, the storage data and the first check data stored by the check disk SSD of the disk array;
and writing the target data into a corresponding data disk HDD in a disk array, and writing the second check data into a check disk SSD of the disk array.
In an optional implementation manner, a capacity difference between the data disk HDD and the check disk SSD in the disk array is less than or equal to a preset threshold.
In an optional implementation manner, the calculating, by using the target data, the storage data, and first parity data stored in a parity disk SSD of the disk array, to obtain second parity data includes:
Performing exclusive or calculation on the stored data and the target data to obtain third check data;
and performing exclusive OR calculation on the third check data and the first check data stored in the SSD in the disk array to obtain second check data.
In an optional implementation manner, when a capacity difference between a data disk HDD and a verification disk SSD in the disk array is greater than a preset threshold and the capacity of the verification disk SSD is smaller than the capacity of the data disk HDD, the verification disk SSD further includes an index table, and the index table stores a corresponding relationship between the disk array stripe number and a compressed data storage location of the first verification data.
In an optional implementation manner, the calculating, by using the target data, the storage data, and first parity data stored in a parity disk SSD of the disk array, to obtain second parity data includes:
performing exclusive-or calculation on the stored data and the target data to obtain third check data;
acquiring first check data stored in a check disk SSD of the disk array according to the index table;
performing exclusive-or calculation on the third check data and the first check data to obtain fourth check data;
And compressing the fourth check data to obtain second check data.
In a second aspect, the present application provides a data processing apparatus based on a hybrid disk array of an HDD and an SSD, where the apparatus is applied to a RAID-3 disk array, a data disk in the disk array is a mechanical hard disk HDD, and a check disk in the disk array is a solid state hard disk SSD, and the apparatus includes:
a receiving unit configured to receive a target data write request;
a reading unit, configured to read, according to the target data write request, storage data of a corresponding data disk HDD in the disk array;
the calculation unit is used for calculating to obtain second check data by utilizing the target data, the storage data and the first check data stored in the check disk SSD of the disk array;
and the writing unit is used for writing the target data into a corresponding data disk HDD in a disk array and writing the second check data into a check disk SSD of the disk array.
In an optional implementation manner, a capacity difference between the data disk HDD and the check disk SSD in the disk array is less than or equal to a preset threshold.
In an alternative implementation, the computing unit includes:
The first calculation subunit is used for performing exclusive-or calculation on the storage data and the target data to obtain third check data;
and the second calculating subunit is configured to perform exclusive or calculation on the third check data and the first check data stored in the check disk SSD in the disk array to obtain second check data.
In an optional implementation manner, when a capacity difference between a data disk HDD and a verification disk SSD in the disk array is greater than a preset threshold and the capacity of the verification disk SSD is smaller than the capacity of the data disk HDD, the verification disk SSD further includes an index table, and the index table stores a corresponding relationship between the disk array stripe number and a compressed data storage location of the first verification data.
In an alternative implementation, the computing unit includes:
the third calculation subunit is used for performing exclusive-or calculation on the storage data and the target data to obtain third check data;
the obtaining subunit is configured to obtain, according to the index table, first verification data stored in a verification disk SSD of the disk array;
the fourth calculating subunit is configured to perform xor calculation on the third check data and the first check data to obtain fourth check data;
And the compression subunit is used for compressing the fourth check data to obtain second check data.
According to the data processing method based on the HDD and SSD hybrid disk array, a target data write-in request is received, then the storage data of the corresponding data disk HDD in the disk array is read according to the target data write-in request, then the target data, the storage data and the first check data stored by the check disk SSD in the disk array are utilized, the second check data are obtained through calculation, further, the target data can be written into the corresponding data disk HDD in the disk array, and the second check data are written into the check disk SSD in the disk array. Therefore, in the embodiment of the application, the solid state disk SSD is used as the check disk of the disk array, and the read-write performance of the solid state disk SSD is greatly improved compared with the HDD disk, so that the check disk does not become a performance bottleneck of the disk array in a scenario of randomly writing data with a large load, and thus the data processing rate of the disk array can be improved, and the data processing performance of the network transmission and storage system can be further improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.
Fig. 1 is a flowchart of a data processing method based on a HDD and SSD hybrid disk array according to an embodiment of the present application;
fig. 2 is one of the flow diagrams of data writing based on the HDD and SSD hybrid disk array according to the embodiment of the present application;
fig. 3 is a second schematic flowchart of data writing based on a HDD and SSD hybrid disk array according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a data processing apparatus based on a HDD and SSD hybrid disk array according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In order to facilitate understanding of the technical solutions provided in the present application, the following briefly describes the research background of the technical solutions in the present application.
As is well known, with the rapid development of network technology, the processing amount of network data is increasing, and in order to improve the read-write performance during the processing of network data and ensure the security of data, data is stored in the form of a disk array, for example, in a commonly used RAID-3 disk array, but since a check disk and a plurality of data disks included in the disk array are HDDs, in this way, in a scenario where data is randomly written in with a large load, since the read-write speed of the check disk HDD is low, the performance bottleneck of the disk array will be likely to be caused, and even when the storage system is busy, the check disk HDD may become the bottleneck of the performance of the entire storage system, which will seriously affect the data processing performance of the network transmission and storage system.
Based on this, in order to solve the above problems, the present application provides a data processing method and apparatus based on a HDD and SSD hybrid disk array, so as to improve the data processing rate of the disk array, and further improve the data processing performance of the network transmission and storage system.
The data processing method based on the HDD and SSD hybrid disk array according to the embodiment of the present application will be described in detail below with reference to the drawings. Referring to fig. 1, which shows a flowchart of a data processing method based on a HDD and SSD hybrid disk array according to an embodiment of the present application, the embodiment may include the following steps:
s101: a target data write request is received.
In practical application, the data processing method based on the HDD and SSD hybrid disk array provided in this embodiment may be applied to a RAID-3 disk array, and in the disk array, the mechanical hard disk HDD is used as a data disk, and the solid state hard disk SSD is used as a check disk, so that the hybrid disk array based on the HDD and the SSD is configured to perform data processing.
In this embodiment, any data written or read by using this embodiment is defined as target data. Also, the present embodiment does not limit the data type of the target data, for example, the target data may be binary data, hexadecimal data, or the like. Further, the HDD and SSD based hybrid disk array may continue to perform the subsequent step S102 after receiving the target data write request.
The target data write request indicates a write address of the target data, that is, an identifier (such as an address of a corresponding data disk) indicating that the target data needs to be written into the corresponding data disk of the HDD and SSD based hybrid disk array.
S102: and reading the storage data of the corresponding data disk HDD in the disk array according to the target data writing request.
In this embodiment, after receiving the target data write request in step S101, the data stored in the target data write request may be further read from the corresponding data disk HDD in the disk array according to the write address of the target data indicated by the write request (i.e. the identifier of the corresponding data disk in the disk array).
Specifically, referring to fig. 2, assuming that the hybrid disk array based on HDDs and SSDs is composed of 4-block data disks HDDs and one check disk SSD, and the target data write request indicates that the target data B needs to be written in the leftmost data disk HDD of the disk array, the storage data a needs to be read out from the leftmost data disk HDD first for the subsequent step S103.
S103: and calculating to obtain second check data by using the target data, the storage data and the first check data stored in the check disk SSD of the disk array.
In this embodiment, after the storage data corresponding to the data disk HDD in the disk array is read in step S102, the verification data (defined as the first verification data here) stored in the verification disk SSD of the disk array needs to be read, and on this basis, the storage data read from the data disk HDD, the target data, and the first verification data read from the verification disk SSD may be used to calculate the second verification data for performing the subsequent step S104.
In an optional implementation manner, a capacity difference between the data disk HDD and the check disk SSD in the disk array is less than or equal to a preset threshold.
In this implementation, the preset threshold refers to a critical value used for distinguishing whether the capacities of the data disk HDD and the verification disk SSD are the same, and if the capacity difference between the data disk HDD and the verification disk SSD is smaller than or equal to the critical value, it may be determined that the capacities between the data disk HDD and the verification disk SSD are the same, and further, the following steps a1-a2 may be continuously performed; otherwise, it can be determined that the capacities between the data disk HDD and the check disk SSD are different. It should be noted that the specific value of the preset threshold may be set according to an actual situation, which is not limited in this embodiment of the application.
Based on this, the specific implementation process of this step S103 may include the following steps a1-a 2:
step A1: and carrying out XOR calculation on the stored data and the target data to obtain third check data.
In this implementation manner, after the storage data of the corresponding data disk HDD in the disk array and the first check data stored in the check disk SSD are read, the storage data and the first check data may be further subjected to xor calculation to obtain third check data.
Specifically, referring to fig. 2, assuming that the HDD and SSD based hybrid disk array is composed of 4 blocks of data disks HDD and one check disk SSD, and the target data write request indicates that the target data B needs to be written into the leftmost data disk HDD of the disk array, after the storage data a is read from the leftmost data disk HDD, a and B may be subjected to an exclusive or calculation to obtain a calculation result C, which is defined as third check data, for performing the subsequent step a 2.
For example, the following steps are carried out: assuming that the target data B is 1111 and the storage data a is 0010 read from the data disk HDD at the leftmost side in fig. 2, the calculation result C is 1101 obtained by performing an exclusive or calculation on the target data B and the storage data a, which may be further used as third check data to perform the subsequent step a 2.
Step A2: and performing exclusive-or calculation on the third check data and the first check data stored in the SSD in the disk array to obtain second check data.
In this implementation manner, after the third parity data is calculated in step a1, the third parity data may be subjected to xor calculation with the first parity data stored in the read parity disk SSD, so as to obtain the second parity data.
Specifically, referring to fig. 2, assuming that the hybrid disk array based on HDDs and SSDs is composed of 4 data disks HDD and one parity disk SSD, and the target data write request indicates that the target data B needs to be written in the leftmost data disk HDD of the disk array, and at the same time, the first parity data D is read from the parity disk SSD, and the third parity data C is calculated through step a1, then C and D may be further subjected to xor calculation to obtain a calculation result E, which is defined as the second parity data, for performing the subsequent step S104.
For example, the following steps are carried out: assuming that the target data B is 1111 and the storage data a is 0010 read from the leftmost data disk HDD in fig. 2, the xor calculation between the target data B and the storage data a can obtain a calculation result C of 1101, and if the first check data D read from the check disk SSD is 1111, the xor calculation between C and D can be further performed, that is, the xor calculation between 1111 and 1101 can obtain a calculation result E of 0010, which is used as the second check data for performing the subsequent step S104.
In another possible implementation manner of the embodiment of the present application, when a capacity difference between the data disk HDD and the check disk SSD in the disk array is greater than a preset threshold, and the capacity of the check disk SSD is smaller than the capacity of the data disk HDD, the check disk SSD further includes an index table, where the index table stores a corresponding relationship between a disk array stripe number and a storage location of compressed data of the first check data.
In this implementation manner, the capacity difference between the data disk HDD and the check disk SSD in the disk array is greater than the preset threshold, and the capacity of the check disk SSD is smaller than the capacity of the data disk HDD, so as to ensure the smoothness of data processing, the check data may be compressed first, and then the compressed check data is written into the check disk SSD, so as to ensure that the data processing amount of the check disk is consistent with the data processing amount of the multiple data disk HDDs, so as to ensure that the data processing rate of the entire disk array is not reduced.
Specifically, an index table is pre-established in the check disk SSD to store a corresponding relationship between the number of the stripe of the disk array and the storage location of the check data after the first check data is compressed, so that the corresponding compressed data is read from the designated location according to the number of the stripe of the disk array by querying the corresponding relationship stored in the index table, and then the compressed data is decompressed to obtain the corresponding first check data, so as to perform the subsequent steps.
Based on this, the specific implementation process of the step S103 may include the following steps B1-B3:
step B1: and carrying out XOR calculation on the stored data and the target data to obtain third check data.
It should be noted that step B1 is identical to step a1, and please refer to the description of step a1 for further description.
Step B2: and acquiring first check data stored in a check disk SSD of the disk array according to the index table.
In this implementation manner, since the capacity of the check disk SSD is smaller than the capacity of the data disk HDD, the corresponding relationship between the stripe number of the current disk array and the storage location of the compressed data of the first check data needs to be queried according to the index table pre-established in the check disk SSD, so as to query the compressed data of the first check data corresponding to the current disk array, and further, the check data needs to be decompressed to obtain the first check data corresponding to the current disk array, so as to execute the subsequent step B3.
Step B3: and carrying out XOR calculation on the third check data and the first check data to obtain fourth check data.
The third check data is obtained through the step B1, and after the first check data is obtained through the step B2, the first check data and the second check data may be further subjected to xor calculation to obtain fourth check data.
Specifically, referring to fig. 3, it is assumed that the HDD and SSD-based hybrid disk array is composed of 4 data disks HDD and a check disk SSD, and the target data write request indicates that the target data B needs to be written into the leftmost data disk HDD of the disk array, and at the same time, the compressed data is read from the check disk SSD as D according to the index table, and after decompressing it, the first check data E is obtained, and the third check data C is calculated through step B1, the calculation result F can be obtained by xoring C with E, and defined as the fourth check data, so as to execute the subsequent step B4.
Step B4: and compressing the fourth check data to obtain second check data.
After the fourth check data is obtained in step B3, in order to reduce the storage space occupied by the fourth check data, it needs to be compressed, and the compressed result is used as the second check data to execute the subsequent step S104. For example, referring to fig. 3, after the xor calculation is performed on the first check data E and the third check data C to obtain the fourth check data F, the fourth check data F may be further compressed to obtain a compression result G, which is used as the second check data to perform the subsequent step S104.
It should be noted that, the volume data compression method and the corresponding decompression method in the above steps may both adopt existing or future data compression and decompression methods, and the method may be selected according to actual situations, which is not limited in the embodiment of the present application.
S104: and writing the target data into a corresponding data disk HDD in the disk array, and writing the second check data into a check disk SSD of the disk array.
In this embodiment, after the second parity data is obtained in step S103, the second parity data may be further written into the parity disk SSD of the current disk array to update the parity data in the parity disk. Meanwhile, the target data can be written into the data disk indicated by the target data writing request, and the writing process of the target data is completed.
In summary, in the data processing method based on the HDD and SSD hybrid disk array provided in this embodiment, a target data write request is received first, then the storage data of the corresponding data disk HDD in the disk array is read according to the target data write request, and then the second check data is calculated by using the target data, the storage data, and the first check data stored by the check disk SSD in the disk array, so that the target data can be written into the corresponding data disk HDD in the disk array, and the second check data can be written into the check disk SSD in the disk array. Therefore, in the embodiment, the solid state disk SSD is used as the check disk of the disk array, and the read-write performance of the solid state disk SSD is greatly improved compared with the HDD disk, so that the check disk does not become a performance bottleneck of the disk array in a scenario of randomly writing data with a large load, thereby improving the data processing rate of the disk array and further improving the data processing performance of the network transmission and storage system.
Based on the above data processing method based on the HDD and SSD hybrid disk array, the present application also provides a data processing apparatus based on the HDD and SSD hybrid disk array, where the apparatus is applied to a RAID-3 disk array, a data disk in the disk array is a mechanical hard disk HDD, and a check disk is a solid state hard disk SSD, and as shown in fig. 4, the apparatus includes:
a receiving unit 401 configured to receive a target data write request;
a reading unit 402, configured to read, according to the target data write request, storage data of a corresponding data disk HDD in the disk array;
a calculating unit 403, configured to calculate to obtain second check data by using the target data, the storage data, and first check data stored in a check disk SSD of the disk array;
a writing unit 404, configured to write the target data into a corresponding data disk HDD in a disk array, and write the second parity data into a parity disk SSD of the disk array.
In an implementation manner of this embodiment, a capacity difference between the data disk HDD and the verification disk SSD in the disk array is less than or equal to a preset threshold.
In an implementation manner of this embodiment, the calculating unit 404 includes:
The first calculation subunit is used for performing exclusive-or calculation on the storage data and the target data to obtain third check data;
and the second calculating subunit is configured to perform exclusive or calculation on the third check data and the first check data stored in the check disk SSD in the disk array to obtain second check data.
In an implementation manner of this embodiment, when a capacity difference between a data disk HDD and a verification disk SSD in the disk array is greater than a preset threshold, and the capacity of the verification disk SSD is smaller than the capacity of the data disk HDD, the verification disk SSD further includes an index table, where the index table stores a corresponding relationship between a disk array stripe number and a compressed data storage location of the first verification data.
In an implementation manner of this embodiment, the calculating unit 404 includes:
the third calculation subunit is used for performing exclusive-or calculation on the storage data and the target data to obtain third check data;
the obtaining subunit is configured to obtain, according to the index table, first verification data stored in a verification disk SSD of the disk array;
the fourth calculating subunit is configured to perform xor calculation on the third check data and the first check data to obtain fourth check data;
And the compression subunit is used for compressing the fourth check data to obtain second check data.
In summary, in the data processing apparatus based on a hybrid disk array of an HDD and an SSD provided in this embodiment, a target data write request is first received, then, according to the target data write request, stored data of a corresponding data disk HDD in the disk array is read, and then, second parity data is calculated by using the target data, the stored data, and first parity data stored by a parity disk SSD in the disk array, so that the target data can be written into the corresponding data disk HDD in the disk array, and the second parity data can be written into the parity disk SSD in the disk array. Therefore, in the embodiment, the solid state disk SSD is used as the check disk of the disk array, and the read-write performance of the solid state disk SSD is greatly improved compared with the HDD disk, so that the check disk does not become a performance bottleneck of the disk array in a scenario of randomly writing data with a large load, thereby improving the data processing rate of the disk array and further improving the data processing performance of the network transmission and storage system.
It should be noted that, in the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. For the system or the device disclosed by the embodiment, the description is simple because the system or the device corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (8)
1. A data processing method based on a mixed disk array of an HDD and an SSD is characterized in that the method is applied to a RAID-3 disk array, a data disk in the disk array is a mechanical hard disk (HDD), a check disk in the disk array is a solid state hard disk (SSD), and the method comprises the following steps:
receiving a target data write request;
reading the storage data of the corresponding data disk HDD in the disk array according to the target data writing request;
calculating to obtain second check data by using the target data, the storage data and the first check data stored by the check disk SSD of the disk array;
writing the target data into a corresponding data disk HDD in a disk array, and writing the second check data into a check disk SSD of the disk array;
When the capacity difference value between the data disk HDD and the verification disk SSD in the disk array is larger than a preset threshold value, and the capacity of the verification disk SSD is smaller than the capacity of the data disk HDD, the verification disk SSD also comprises an index table, and the index table stores the corresponding relation between the disk array strip number and the compressed data storage position of the first verification data.
2. The method according to claim 1, wherein the capacity difference between the data disks HDD and the check disks SSD in the disk array is less than or equal to a preset threshold.
3. The method of claim 2, wherein the calculating second parity data using the target data, the storage data, and first parity data stored by a parity disk (SSD) of the disk array comprises:
performing exclusive-or calculation on the stored data and the target data to obtain third check data;
and performing exclusive OR calculation on the third check data and the first check data stored in the SSD in the disk array to obtain second check data.
4. The method of claim 1, wherein the calculating second parity data using the target data, the storage data, and first parity data stored in a parity disk (SSD) of the disk array comprises:
Performing exclusive or calculation on the stored data and the target data to obtain third check data;
acquiring first check data stored in a check disk SSD of the disk array according to the index table;
performing exclusive-or calculation on the third check data and the first check data to obtain fourth check data;
and compressing the fourth check data to obtain second check data.
5. A data processing device based on a mixed disk array of an HDD and an SSD is characterized in that the device is applied to a RAID-3 disk array, a data disk in the disk array is a mechanical hard disk (HDD), a check disk in the disk array is a solid state hard disk (SSD), and the device comprises:
a receiving unit configured to receive a target data write request;
a reading unit, configured to read, according to the target data write request, storage data of a corresponding data disk HDD in the disk array;
the calculation unit is used for calculating to obtain second check data by utilizing the target data, the storage data and the first check data stored in the check disk SSD of the disk array;
a write unit, configured to write the target data into a corresponding data disk HDD in a disk array, and write the second parity data into a parity disk SSD of the disk array;
When the capacity difference value between the data disk HDD and the verification disk SSD in the disk array is larger than a preset threshold value, and the capacity of the verification disk SSD is smaller than the capacity of the data disk HDD, the verification disk SSD also comprises an index table, and the index table stores the corresponding relation between the disk array strip number and the compressed data storage position of the first verification data.
6. The apparatus of claim 5, wherein a difference between capacities of the data disks HDD and the check disks SSD in the disk array is less than or equal to a preset threshold.
7. The apparatus of claim 6, wherein the computing unit comprises:
the first calculation subunit is used for performing exclusive-or calculation on the storage data and the target data to obtain third check data;
and the second calculating subunit is configured to perform exclusive or calculation on the third check data and the first check data stored in the check disk SSD in the disk array to obtain second check data.
8. The apparatus of claim 5, wherein the computing unit comprises:
the third calculation subunit is used for performing exclusive-or calculation on the storage data and the target data to obtain third check data;
The obtaining subunit is configured to obtain, according to the index table, first verification data stored in a verification disk SSD of the disk array;
the fourth calculating subunit is configured to perform xor calculation on the third check data and the first check data to obtain fourth check data;
and the compression subunit is used for compressing the fourth check data to obtain second check data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910828441.3A CN110658994B (en) | 2019-09-03 | 2019-09-03 | Data processing method and device based on HDD (hard disk drive) and SSD (solid State disk) hybrid disk array |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910828441.3A CN110658994B (en) | 2019-09-03 | 2019-09-03 | Data processing method and device based on HDD (hard disk drive) and SSD (solid State disk) hybrid disk array |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110658994A CN110658994A (en) | 2020-01-07 |
| CN110658994B true CN110658994B (en) | 2022-08-12 |
Family
ID=69037820
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910828441.3A Active CN110658994B (en) | 2019-09-03 | 2019-09-03 | Data processing method and device based on HDD (hard disk drive) and SSD (solid State disk) hybrid disk array |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110658994B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114579057A (en) * | 2022-03-11 | 2022-06-03 | 长江存储科技有限责任公司 | Storage method, storage device and storage system for check data of cache equipment |
| CN117806568A (en) * | 2024-02-29 | 2024-04-02 | 山东云海国创云计算装备产业创新中心有限公司 | Data updating method, device, equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102122235A (en) * | 2011-01-24 | 2011-07-13 | 武汉固捷联讯科技有限公司 | RAID4 (redundant array of independent disks) system and data reading and writing method thereof |
| CN103019882A (en) * | 2012-11-12 | 2013-04-03 | 记忆科技(深圳)有限公司 | RAID4 (redundant array of independent disks 4) system of solid-state disc |
-
2019
- 2019-09-03 CN CN201910828441.3A patent/CN110658994B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102122235A (en) * | 2011-01-24 | 2011-07-13 | 武汉固捷联讯科技有限公司 | RAID4 (redundant array of independent disks) system and data reading and writing method thereof |
| CN103019882A (en) * | 2012-11-12 | 2013-04-03 | 记忆科技(深圳)有限公司 | RAID4 (redundant array of independent disks 4) system of solid-state disc |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110658994A (en) | 2020-01-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9798620B2 (en) | Systems and methods for non-blocking solid-state memory | |
| US8799571B1 (en) | System and method for configuring a device array upon detecting addition of a storage device | |
| CN105573681B (en) | Method and system for establishing RAID in SSD | |
| US8456972B2 (en) | Efficient access to storage devices with usage bitmaps | |
| US10852966B1 (en) | System and method for creating mapped RAID group during expansion of extent pool | |
| US10733051B2 (en) | Redistributing data across drives of a storage array based on drive health metrics | |
| CN107463342B (en) | CDN edge node file storage method and device | |
| CN110658994B (en) | Data processing method and device based on HDD (hard disk drive) and SSD (solid State disk) hybrid disk array | |
| CN112684975B (en) | Data storage method and device | |
| US11385823B2 (en) | Method, electronic device and computer program product for rebuilding disk array | |
| CN110413454B (en) | Data reconstruction method and device based on storage array and storage medium | |
| CN103473266A (en) | Solid state disk and method for deleting repeating data thereof | |
| US11656942B2 (en) | Methods for data writing and for data recovery, electronic devices, and program products | |
| CN103399823A (en) | Method, equipment and system for storing service data | |
| CN111400083A (en) | Data storage method and system and storage medium | |
| CN107729536A (en) | A kind of date storage method and device | |
| CN107678894B (en) | Memory test method, device and system | |
| CN113326005A (en) | Read-write method and device for RAID storage system | |
| US11922019B2 (en) | Storage device read-disturb-based block read temperature utilization system | |
| CN110504002B (en) | Hard disk data consistency test method and device | |
| KR20170114438A (en) | Data storage device and operating method thereof | |
| KR20170095570A (en) | Network traffic recording device and method thereof | |
| CN108958660B (en) | Distributed storage system and data processing method and device thereof | |
| CN114442950A (en) | Data recovery method, system, device and computer readable storage medium | |
| CN106980471B (en) | Method and device for improving hard disk writing performance of intelligent equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |