CN108958656B - Dynamic stripe system design method based on RAID5 solid state disk array - Google Patents

Dynamic stripe system design method based on RAID5 solid state disk array Download PDF

Info

Publication number
CN108958656B
CN108958656B CN201810680397.1A CN201810680397A CN108958656B CN 108958656 B CN108958656 B CN 108958656B CN 201810680397 A CN201810680397 A CN 201810680397A CN 108958656 B CN108958656 B CN 108958656B
Authority
CN
China
Prior art keywords
stripe
data
dynamic
blocks
stripes
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810680397.1A
Other languages
Chinese (zh)
Other versions
CN108958656A (en
Inventor
殷双飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Institute of Computer Technology and Application
Original Assignee
Beijing Institute of Computer Technology and Application
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Institute of Computer Technology and Application filed Critical Beijing Institute of Computer Technology and Application
Priority to CN201810680397.1A priority Critical patent/CN108958656B/en
Publication of CN108958656A publication Critical patent/CN108958656A/en
Application granted granted Critical
Publication of CN108958656B publication Critical patent/CN108958656B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from or digital output to record carriers, e.g. RAID, emulated record carriers, networked record carriers
    • G06F3/0601Dedicated interfaces to storage systems
    • G06F3/0602Dedicated interfaces to storage systems specifically adapted to achieve a particular effect
    • G06F3/0614Improving the reliability of storage systems
    • G06F3/0616Improving the reliability of storage systems in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from or digital output to record carriers, e.g. RAID, emulated record carriers, networked record carriers
    • G06F3/0601Dedicated interfaces to storage systems
    • G06F3/0602Dedicated interfaces to storage systems specifically adapted to achieve a particular effect
    • G06F3/061Improving I/O performance
    • G06F3/0611Improving I/O performance in relation to response time
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from or digital output to record carriers, e.g. RAID, emulated record carriers, networked record carriers
    • G06F3/0601Dedicated interfaces to storage systems
    • G06F3/0628Dedicated interfaces to storage systems making use of a particular technique
    • G06F3/0629Configuration or reconfiguration of storage systems
    • G06F3/0631Configuration or reconfiguration of storage systems by allocating resources to storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from or digital output to record carriers, e.g. RAID, emulated record carriers, networked record carriers
    • G06F3/0601Dedicated interfaces to storage systems
    • G06F3/0668Dedicated interfaces to storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0683Plurality of storage devices
    • G06F3/0689Disk arrays, e.g. RAID, JBOD

Abstract

The invention relates to a dynamic stripe system design method and a dynamic stripe strategy implementation method based on a RAID5 solid state disk array, and relates to the technical field of data storage. The invention replaces the data updating mode of read modification write and read reconstruction write in the RAID5 solid state disk array by using the mode of constructing the dynamic stripe, wherein the dynamic stripe design system comprises a stripe reorganization module and a metadata module. The stripe recombination module reconstructs the data blocks of the local stripes to form dynamic stripes; and the metadata module records and manages the mapping state and the storage state of the data. The combination of both with a RAID5 array changes the request processing pattern of a conventional RAID5 array. The system separates the writing of the full stripe and the writing of the local stripe, so that the data blocks in the local stripe are recombined to construct a dynamic stripe, and the dynamic and static stripes are mapped and managed by metadata, thereby reducing the writing times of the solid state disk, prolonging the service life of the array, reducing the pre-reading delay before writing and improving the writing performance of the array.

Description

Dynamic stripe system design method based on RAID5 solid state disk array
Technical Field
The invention relates to the technical field of data storage, in particular to a dynamic stripe system design method and a dynamic stripe strategy implementation method based on a RAID5 solid state disk array.
Background
RAID is a mechanism for providing data redundancy protection, and abstracts a plurality of physical hard disks upward as a logical disk, and forwards a file read-write request issued by an upper system downward, thereby implementing a function of redirecting and forwarding a data stream. In the process of forwarding data streams, RAID implements various forms of data redundancy, including data mirroring and data checking, to provide data protection capabilities. Among RAID levels, RAID5 has better parallel performance and higher utilization rate of hard disk space, so the RAID is widely used.
The stripe technology is a technology commonly used in a RAID array system, and organizes an array space into stripes, data redundancy is performed in units of stripes, and read and write operations are performed in units of stripes. In the RAID5 array, the common data and the check data are located in the same stripe, and the update of the common data needs to be synchronous to the update of the check data.
Data updates for a stripe can be divided into two types, one being updates to a full stripe and the other being updates to a non-full stripe (i.e., a partial stripe). The read-write operation required by updating the full strip is minimum, and the efficiency is highest; the updating of the non-full strip can be divided into two specific cases, namely reading and modifying, and reading and reconstructing, and no matter which case is used for data updating, the old data needs to be read in advance, and the new data is written after the verification calculation.
The full flash memory array is generally a RAID array system constructed by solid state disks, the storage medium of the solid state disks is nand flash, the number of times of erasing is limited, and therefore the number of times of erasing of the array as a whole is also limited. The updating method of the stripe is optimized, the updating times of the verification are reduced, on one hand, the writing times of the array can be reduced, the service life of the array is prolonged, on the other hand, the writing delay of the data can be reduced, and the performance of the array is improved.
Disclosure of Invention
Technical problem to be solved
The technical problem to be solved by the invention is as follows: how to change the data updating mode of the read modification write and the read reconstruction write in the RAID5 solid state disk array to reduce the frequent check updating brought by the local stripe updating in the array.
(II) technical scheme
In order to solve the technical problem, the invention provides a method for designing a dynamic stripe system based on a RAID5 solid state disk array, wherein a stripe reorganization module and a metadata module are designed in the dynamic stripe system;
the strip recombination module is designed as follows: the data storage part is used for temporarily storing local stripe data blocks, the local stripe data blocks are regarded as a buffer pool, and when the data blocks in the pool reach an upper limit or a set brushing time interval, the stripe reorganization part acquires the data blocks in the buffer pool and reorganizes the data blocks into a new stripe, which is called a dynamic stripe;
the buffer pool is composed of a plurality of buffers, the number of the buffers is the same as that of data blocks in a logic strip, each buffer is a bidirectional linked list with LRU characteristics, the data blocks with frequent writing times are arranged at the head of the linked list, the data blocks with less writing times are arranged at the tail of the linked list, if a writing request is divided into a plurality of data blocks, a static strip and a local strip are separated after the strip is identified, the static strip is written into an SSD array after verification calculation and metadata recording, the local strip data blocks are put into the corresponding buffers according to the offset of the strip, the data in the buffers are arranged from bottom to top in a time sequence, the strips are distributed and calculated and verified after being recombined into a dynamic strip, and the writing operation is executed and flushed back into the solid state hard disk array after being recorded by a metadata module;
when the number of data blocks in any buffer reaches an upper limit, activating the stripe reorganization module to extract data from the bottom of each buffer, along with the reduction of the capacity of each buffer, if the capacity of the buffer reaches the upper limit, at the moment, the stripe reorganization module extracts the data blocks at the bottom of all the buffers to form dynamic stripes according to the original offset sequence, if each buffer has a data block, the reorganized stripes are full dynamic stripes, otherwise, the reorganized stripes are not full dynamic stripes;
designing a global striping table and a dynamic striping table in the metadata module; the metadata of the dynamic stripe system records mapping information and state information of stripes, and is used for data block mapping and stripe management, the metadata has two table structures, namely a global stripe mapping table and a dynamic stripe mapping table, the global stripe mapping table records the mapping information and the state information of all the stripes in the array, and the dynamic stripe mapping table records the state information of the mapped stripes;
and the local stripe data block is directly put into a data cache, is recombined into a dynamic stripe, is distributed with a stripe and is calculated and checked, and is also recorded by the metadata module and then is written into the solid state disk array and is flushed back to the solid state disk array.
Preferably, the global slice mapping table is referred to as a global slice table for short, and in a data structure of the global slice mapping table, a structure body represents record information of one slice in the global slice table, the whole global slice table has N records, where N is the number of slices in the array, each record is referred to as an entry, and each entry records state information and mapping information of a slice and a slice unit block; in a data structure of a global stripe table, LSN represents an offset serial number of a logic stripe in the global stripe table, the offset serial number is obtained by calculating a logic address of the stripe, the value range of the LSN is 0-N-1, an entry is divided into two parts, one part is a type value for recording the use state of the stripe, the other part is a set of a stripe unit block structure body, the value of a type field represents the use state of the stripe, 4 values are totally taken, 0 represents that the current stripe is in an initial state, and no data is stored in a unit block; 1 represents that the current stripe is used by a static stripe, and static data is stored in a unit block; 2, the current stripe is converted from stripe type 1, and the data in at least one unit block in the stripe is remapped to other stripes; 3 represents that the current stripe is occupied by the mapped stripe;
the number of unit blocks in the structure is the same as the number of unit blocks in the logical stripe and starts from 0 to end with 2 in the order of offset, and 5 pieces of information are recorded per unit block data structure, which means as follows: the remap field indicates whether the data in the current unit block is mapped and stored, 0 indicates none, and 1 indicates mapped and stored; the mapLSN field represents a stripe address mapped by a unit block and is distributed by RADS; the state field indicates whether the current cell block address stores data, 0 indicates stored data, and 1 indicates that no data is stored; lsn field indicates the starting logical sector address of the stored data within the current cell block; the size field indicates the size of the stored data block;
the dynamic stripe mapping table is called as a dynamic stripe table for short, in a data structure of the dynamic stripe mapping table, a structure body represents the recording information of one stripe in the dynamic stripe table, the dynamic stripe table also has N entries, only the entry which records the mapping information is effective, the stripes recorded in the dynamic stripe table correspond to the stripes with the type value of 3 in the global stripe table one by one, and the global stripe table entry with the type value of 3 has one dynamic stripe table entry;
in a data structure of a dynamic stripe table, LSN represents the offset serial number of a stripe in the dynamic stripe table, the value range of the LSN is 0 to N-1, each entry in the dynamic stripe table records two parts of information, one part records the use state of the stripe, and the other part is the set of a stripe unit block structure body; the value of the used _ state field indicates the use state of the strip in the dynamic strip table, 2 values are total, when the value is 0, the entry is not used, and the value is 1, the entry records strip mapping information; the vector field indicates the valid state of each data block in the mapping stripe, and indicates the state of all cell blocks in the dynamic stripe, each bit in the vector is used for marking, the value of 0 indicates invalid, and the value of 1 indicates valid;
the number of unit blocks in the structure is the same as the number of unit blocks in the logical stripe and starts from 0 to end with 2 in the order of offset, and 4 pieces of information are recorded per unit block data structure, which means as follows: the valid field represents the valid state of the data in the current unit block, the value range is 0 to 2, 0 represents invalid, 1 represents invalid data mapping but can be used for data reconstruction, and 2 represents valid; lsn field indicates the starting logical sector address of the stored data within the current cell block; the size field indicates the size of the stored data block and the LSN indicates the original logical slice number of the data within the unit block.
Preferably, the current stripe mapped stripes include dynamic stripes and remapped static stripes.
Preferably, in a dynamic band with a cell block of 3, a vector value of 101 indicates that cell blocks with offsets of 0 and 2 are valid, and that cell blocks with offset of 1 are invalid.
Preferably, the system is further designed with a data reconstruction module for reconstructing failure data and performing read-write processing when the array is degraded.
Preferably, the system is further designed with a recovery module for recovering the outdated data.
Preferably, the system is implemented within a RAID controller.
The invention also provides a method for realizing the dynamic stripe strategy by using the system designed by the method, which comprises the following steps when the write request is realized:
when a write request enters a RAID controller, the write request is divided and calculated according to the encoding mode of RAID5, continuous request addresses are divided into a plurality of address blocks with fixed sizes according to the preset size of a stripe unit, partial continuous address blocks in the address blocks form a full stripe, namely a static stripe, partial continuous address blocks form a local stripe, identification of the static stripe and the local stripe is judged according to the number of data blocks in the stripe, the static stripe and the local stripe are processed in different modes, the static stripe data blocks are recorded by a metadata module after calculation and verification, and the request write operation is executed and flushed back to a solid state disk array; the local stripe data block is directly placed into a buffer pool by a data cache part, a stripe is distributed and calculated after being recombined into a dynamic stripe by a stripe recombination part, and the dynamic stripe is also written back to the solid state disk array after being recorded by a metadata module, and the static stripe with the stripe address occupied can be subjected to check calculation after the stripe number is distributed, and can be written into the bottom layer array after the metadata is recorded;
the method comprises the following steps when the read request is realized:
storing the divided request blocks into a read request queue indiscriminately no matter the divided request blocks are positioned in static stripes or local stripes, retrieving data from a solid state disk array when a read operation is executed, inquiring a global stripe table, returning different values according to the mapping condition if the type of the stripe to which the request belongs is 0, returning mapping address data when the address of the stripe is mapped, and otherwise returning to null; if the stripe type is 1, returning request address data; the stripe type is 2, and a corresponding value is returned according to the mapping condition of the address; and if the stripe type is 3, the data of the mapping address is directly returned.
Preferably, the two types of stripes, the modification of the metadata is determined according to the stripe type of the corresponding write address in the global stripe table, and the method includes the following steps when implementing static stripe metadata processing:
static stripe metadata processing flow:
when a static stripe is written, by inquiring a global stripe mapping table, for a stripe with the type of 0, if a data block in the stripe has mapping information, firstly clearing the original mapping information, then recording storage address information, and juxtaposing the stripe with the type of 1; for the stripe with the type 1, directly recording the storage address information; the processing mode of the strip with the type 2 is the strip with the same type 0; for the stripe of type 3, after the stripe number is allocated, the processing is performed in a dynamic stripe manner.
Preferably, the two types of stripes, the modification of the metadata is determined according to the stripe type of the corresponding write address in the global stripe table, and the method includes the following steps when implementing dynamic stripe metadata processing:
the data in the dynamic stripe unit block is formed by combining data blocks at different offsets in different stripes, before the dynamic stripe is written, a stripe number is firstly distributed, current mapping information and state information are added, and the original mapping information of the unit block is deleted;
correspondingly processing metadata according to the type of a logical stripe to which each data block belongs in a stripe in a global stripe table, and clearing mapping information if a block address is mapped for a data block with an original stripe type of 0, and then recording current mapping information; for the data block with the original stripe type of 1, after recording the current mapping information, setting the original stripe type of 2; and recording the information of the storage address of the data block with the original stripe type of 2 or 3 in the same processing mode as the data block with the original stripe type of 0 after the data blocks at all the offsets in the dynamic stripe are processed.
(III) advantageous effects
The invention replaces the data updating mode of read modification write and read reconstruction write in the RAID5 solid state disk array by using the mode of constructing the dynamic stripe, wherein the dynamic stripe design system comprises a stripe reorganization module and a metadata module. The stripe recombination module reconstructs the data blocks of the local stripes to form dynamic stripes; and the metadata module records and manages the mapping state and the storage state of the data. The combination of both with a RAID5 array changes the request processing pattern of a conventional RAID5 array. The system separates the writing of the full stripe and the writing of the local stripe, so that the data blocks in the local stripe are recombined to construct a dynamic stripe, and the dynamic and static stripes are mapped and managed by metadata, thereby reducing the writing times of the solid state disk, prolonging the service life of the array, reducing the pre-reading delay before writing and improving the writing performance of the array.
Drawings
FIG. 1 is a diagram of the overall architecture design of the RADS of the present invention;
FIG. 2 is a schematic diagram of data caching in the system design of the present invention;
FIG. 3 is a schematic representation of the recombination of bands in the system design of the present invention;
FIG. 4 is a schematic diagram of a global stripe map designed in the system of the present invention;
FIG. 5 is a schematic diagram of a dynamic stripe map designed in the system of the present invention;
FIG. 6 is a flow chart of a request write in a dynamic stripe policy implementation of the present invention;
FIG. 7 is a flow chart of a request read in the dynamic stripe policy implementation of the present invention;
FIG. 8 is a flow chart of static stripe metadata processing in the dynamic stripe policy implementation of the present invention;
FIG. 9 is a flow chart of static stripe metadata processing in the dynamic stripe policy implementation method of the present invention.
Detailed Description
In order to make the objects, contents, and advantages of the present invention clearer, the following detailed description of the embodiments of the present invention will be made in conjunction with the accompanying drawings and examples.
The invention particularly relates to a Dynamic stripe system design of a RAID5 solid state disk Array system, which is named as RADS (redundant Array Dynamic striping). The design aims to replace a data updating mode of read modification, write and read reconstruction and write in a RAID5 solid state disk array by using a mode of constructing a dynamic stripe, and reduce frequent check updating caused by local stripe updating in the array.
RADS is a dynamic stripe strategy of RAID level, and is implemented in a RAID controller, the overall architecture design is shown in FIG. 1, and the system is divided into two parts: the system comprises a RAID controller part for executing RADS strategy at the upper layer and a solid state disk array part for storing data at the lower layer. It comprises two main modules: one is a stripe reorganization module and the other is a metadata module.
(1) Stripe reorganization module design
Stripe reorganization is for local stripe data blocks that form dynamic stripes by stripe reorganization. The band recombination module in the RADS strategy comprises two parts of data caching and band recombination. The data buffer is used for temporarily storing the local stripe data block, and is regarded as a buffer pool, and when the data block in the pool reaches an upper limit or a set brushing time interval arrives, the part in charge of stripe reorganization in the stripe reorganization module acquires the data block in the buffer pool and reorganizes the data block into a new stripe, which is called a dynamic stripe.
The buffer pool is composed of a number of buffers, the number of which is the same as the number of data blocks in the logical stripe, in this example the number of buffers is 3. Each buffer is a doubly linked list with LRU characteristics, and as shown in fig. 2(a), the data block with frequent writing times is placed at the head of the list, and the data block with less writing times is placed at the tail of the list. In fig. 2(b), a write request is divided into ten data blocks D0-D9, a static stripe and a local stripe are separated after stripe recognition, the static stripe is written into the SSD array after parity computation and metadata recording, the local stripe data blocks are placed into the corresponding buffers 0-2 according to the offset of the stripe, see (c), (D), and (e) in fig. 2, the data blocks D0 and D1 of the first local stripe are respectively placed into the corresponding buffers 1 and 2 at the offset, and the data blocks D8 and D9 of the second stripe are respectively placed into the buffers 0 and 1. The data in the buffer is placed from bottom to top in chronological order.
When the number of data blocks in any buffer reaches an upper limit, the stripe reorganization module is activated to extract data from the bottom of each buffer. As shown in fig. 3, as the capacity of each buffer decreases, the capacity of buffer 1 in (a) of fig. 3 reaches an upper limit, at which time, the stripe reorganization module extracts the data blocks at the bottom of buffers 0 to 2 to compose dynamic stripes in the original offset order, if each buffer has a data block, the reorganized stripe is a full dynamic stripe, such as the stripe composed of D8, D0 and D1 in (b) of fig. 3, otherwise, the stripe is a non-full dynamic stripe.
(2) Metadata module design
1. Global stripe map
The metadata of the RADS records mapping information and state information of the stripes in the array system for data block mapping and stripe management. The metadata has two table structures, namely a global stripe mapping table and a dynamic stripe mapping table, wherein the global stripe mapping table records mapping information and state information of all stripes in the array, and the dynamic stripe mapping table specially records state information of the mapping stripes.
The data structure of the global slice mapping table (referred to as global slice table for short) is shown in fig. 4, where the structure in the figure represents record information of one slice in the global slice table, N records are total in the entire global slice table, where N is the number of slices in the array, each record is referred to as an entry, and each entry records state information and mapping information of a slice and a slice unit block.
In the figure, LSN (logical Stripe number) represents an offset number of a logical Stripe in the global Stripe table, which is calculated from a logical address of the Stripe, and a value range of the LSN is 0 to N-1. One entry is divided into two parts, one part is a type value that records a slice use state, and the other part is a set of slice unit block structures. The value of the type field represents the use state of the stripe, 4 values (0,1,2 and 3) are provided in total, 0 represents that the current stripe is in an initial state, and no data is stored in the unit block; 1 represents that the current stripe is used by a static stripe, and static data is stored in a unit block; 2, the current stripe is converted from stripe type 1, and the data in at least one unit block in the stripe is remapped to other stripes; 3 denotes that the current stripe is occupied by a mapped stripe (including dynamic stripes and remapped static stripes).
The number of unit blocks in the structure is the same as the number of unit blocks in the logical stripe, and starts from 0 to ends at 2 in the order of offset (e.g., D0, D1, D2 in fig. 4), and each unit block data structure records 5 pieces of information, which are as follows as shown in fig. 4: the remap field indicates whether the data in the current unit block is mapped and stored, 0 indicates none, and 1 indicates mapped and stored; the mapLSN field represents a stripe address mapped by a unit block and is distributed by RADS; the state field indicates whether the current cell block address stores data, 0 indicates stored data, and 1 indicates that no data is stored; lsn field indicates the starting logical sector address of the stored data within the current cell block; the size field indicates the size of the stored data block.
2. Dynamic stripe map
The data structure of the dynamic stripe mapping table (referred to as the dynamic stripe table for short) is shown in fig. 5, where the structure in the figure represents the record information of one stripe in the dynamic stripe table, and like the global stripe table, the dynamic stripe table also has N entries, and only the entry recorded with the mapping information is valid. The stripes recorded in the dynamic stripe table are in one-to-one correspondence with the stripes with the type value of 3 in the global stripe table, and one dynamic stripe table entry is provided when one global stripe table entry with the type value of 3 is provided.
In fig. 5, LSN represents the offset number of the slice in the dynamic slice table, and the value range of LSN is 0 to N-1. Each entry in the dynamic band table also records two parts of information, one part records the use state of the band, and the other part is the set of the structure body of the band unit block.
The value of the used _ state field indicates the use state of a stripe in the dynamic stripe table, and 2 values (0 and 1) are provided in total, when the value is 0, the entry is not used, and when the value is 1, the entry records stripe mapping information; the vector field indicates the valid state of each data block in the mapped stripe, for example, when a non-full dynamic stripe is written, a part of cell blocks in the stripe have no data, the state of the cell block is invalid, and to indicate the state of all cell blocks in the dynamic stripe, each bit in the vector is used for marking, the value is 0 to indicate invalid, the value is 1 to indicate valid, in a dynamic stripe with a cell block of 3, the value of 101 to 101 indicates that the cell blocks with offsets of 0 and 2 are valid, and the cell block with offset of 1 is invalid.
The number of unit blocks in the structure is the same as the number of unit blocks in the logical stripe, and starts from 0 to ends at 2 in the order of offset (D0, D1, D2), and each unit block data structure records 4 pieces of information, which have the following meanings: the valid field represents the valid state of the data in the current unit block, the value range is 0 to 2, 0 represents invalid, 1 represents invalid data mapping but can be used for data reconstruction, and 2 represents valid; lsn field indicates the starting logical sector address of the stored data within the current cell block; the size field indicates the size of the stored data block and the LSN indicates the original logical slice number of the data within the unit block.
The RADS dynamic stripe strategy is realized through modularization, wherein a core function module is a stripe reorganization module and a metadata module, the stripe reorganization module is responsible for reorganization of dynamic stripes and comprises two parts of data caching and stripe reorganization, and the metadata module is responsible for recording and managing metadata. In addition, two modules closely related to the stability of the array are a data reconstruction module and a recovery module, the data reconstruction module is responsible for reconstruction of failure data and read-write processing when the array is degraded, and the recovery module is responsible for recovery of outdated data. Other parts in the RADS dynamic stripe strategy are responsible for preprocessing of requests and data reading and writing, such as request segmentation, stripe identification, request data uploading and issuing and the like.
When a write request enters the RAID controller, it is partitioned and computed according to the encoding mode of RAID 5. According to the preset size of the stripe unit, continuous request addresses are divided into a plurality of address blocks with fixed sizes. Of these address blocks, partially contiguous address blocks constitute full stripes, i.e., static stripes, and partially contiguous address blocks constitute partial stripes. The identification of the static stripe and the local stripe is judged according to the number of data blocks in the stripe, the static stripe and the local stripe are processed in different modes, the static stripe data blocks are recorded by a metadata module after calculation and verification, and the execution request writing operation is flushed back to the solid state disk array; and the local stripe data block is directly put into a data cache, is recombined into a dynamic stripe, is distributed with a stripe and is calculated and checked, and is also recorded by the metadata module and then is written into the solid state disk array and is flushed back to the solid state disk array. For static stripes with occupied stripe addresses, check calculation can be carried out after the stripe numbers are distributed, and metadata can be written into the bottom layer array after being recorded. The entire write request process is shown in fig. 6.
For read requests, whether the split request block is located in a static stripe or a local stripe, the split request block is stored in a read request queue without distinction, and data can be retrieved from the solid state disk array by performing read operations. If the stripe type of the request is 0, returning different values according to the mapping condition, returning mapping address data if the stripe address is mapped, and otherwise returning null; if the stripe type is 1, returning request address data; the stripe type is 2, and a corresponding value is returned according to the mapping condition of the address; and if the stripe type is 3, the data of the mapping address is directly returned. The entire read request process is shown in fig. 7.
The modification of the metadata of the two types of stripes is determined according to the stripe type of the corresponding write address in the global stripe table, and the processing mode is as follows:
(1) static stripe metadata processing flow
When a static stripe is written, by inquiring a global stripe mapping table, for a stripe with the type of 0, if a data block in the stripe has mapping information, firstly clearing the original mapping information, then recording storage address information, and juxtaposing the stripe with the type of 1; for the stripe with the type 1, directly recording the storage address information; the processing mode of the strip with the type 2 is the strip with the same type 0; for the stripe of type 3, after the stripe number is allocated, the processing is performed in a dynamic stripe manner. The process flow is shown in fig. 8.
(2) Dynamic stripe metadata processing flow
The data in the unit block of dynamic band is composed of data blocks at different offsets in different bands, before writing in the dynamic band, the number of the band is allocated first, and the current mapping information and state information are added, and the original mapping information of the unit block is deleted, as shown in fig. 9.
And correspondingly processing the metadata according to the type of the logic stripe to which each data block in the stripe belongs in the global stripe table. For a data block with the original stripe type of 0, if the block address is mapped, firstly clearing mapping information, and then recording current mapping information; for the data block with the original stripe type of 1, after recording the current mapping information, setting the original stripe type of 2; and the processing mode of the data block with the original stripe type of 2 or 3 is the same as that of the data block with the original stripe type of 0. And after all the data blocks at the offset positions in the dynamic band are processed, recording the information of the storage addresses of the data blocks.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.

Claims (9)

1. A design method of a dynamic stripe system based on a RAID5 solid state disk array is characterized in that a stripe reorganization module and a metadata module are designed in the dynamic stripe system;
the strip recombination module is designed as follows: the data storage part is used for temporarily storing local stripe data blocks, the local stripe data blocks are regarded as a buffer pool, and when the data blocks in the pool reach an upper limit or a set brushing time interval, the stripe reorganization part acquires the data blocks in the buffer pool and reorganizes the data blocks into a new stripe, which is called a dynamic stripe;
the buffer pool is composed of a plurality of buffers, the number of the buffers is the same as that of data blocks in a logic strip, each buffer is a bidirectional linked list with LRU characteristics, the data blocks with frequent writing times are arranged at the head of the linked list, the data blocks with less writing times are arranged at the tail of the linked list, if a writing request is divided into a plurality of data blocks, a static strip and a local strip are separated after the strip is identified, the static strip is written into an SSD array after verification calculation and metadata recording, the local strip data blocks are put into the corresponding buffers according to the offset of the strip, the data in the buffers are arranged from bottom to top in a time sequence, the strips are distributed and calculated and verified after being recombined into a dynamic strip, and the writing operation is executed and flushed back into the solid state hard disk array after being recorded by a metadata module;
when the number of data blocks in any buffer reaches an upper limit, activating the stripe reorganization module to extract data from the bottom of each buffer, along with the reduction of the capacity of each buffer, if the capacity of the buffer reaches the upper limit, at the moment, the stripe reorganization module extracts the data blocks at the bottom of all the buffers to form dynamic stripes according to the original offset sequence, if each buffer has a data block, the reorganized stripes are full dynamic stripes, otherwise, the reorganized stripes are not full dynamic stripes;
designing a global striping table and a dynamic striping table in the metadata module; the metadata of the dynamic stripe system records mapping information and state information of stripes, and is used for data block mapping and stripe management, the metadata has two table structures, namely a global stripe mapping table and a dynamic stripe mapping table, the global stripe mapping table records the mapping information and the state information of all the stripes in the array, and the dynamic stripe mapping table records the state information of the mapped stripes;
and the local stripe data block is directly put into a data cache, is recombined into a dynamic stripe, is distributed with a stripe and is calculated and checked, and is also recorded by the metadata module and then is written into the solid state disk array and is flushed back to the solid state disk array.
2. The method of claim 1, wherein the global slice mapping table is referred to as a global slice table for short, and in a data structure of the global slice table, a structure body represents record information of one slice in the global slice table, N records are totally included in the entire global slice table, where N is the number of slices in the array, each record is referred to as an entry, and each entry records state information and mapping information of the slice and slice unit blocks; in a data structure of a global stripe table, LSN represents an offset serial number of a logic stripe in the global stripe table, the LSN is obtained by calculating a logic address of the stripe, the value range of the LSN is 0-N-1, an entry is divided into two parts, one part is a type value for recording the use state of the stripe, the other part is a set of stripe unit block structures, the value of a type field represents the use state of the stripe, 4 values are totally obtained, 0 represents that the current stripe is in an initial state, and no data is stored in a unit block; 1 represents that the current stripe is used by a static stripe, and static data is stored in a unit block; 2, the current stripe is converted from stripe type 1, and the data in at least one unit block in the stripe is remapped to other stripes; 3 represents that the current stripe is occupied by the mapped stripe;
the number of unit blocks in the structure is the same as the number of unit blocks in the logical stripe, and starts from 0 to ends with 2 in the order of offset, and each unit block data structure records 5 pieces of information, the meaning of the 5 pieces of information is as follows: the remap field indicates whether the data in the current unit block is mapped and stored, 0 indicates none, and 1 indicates mapped and stored; the mapLSN field represents a stripe address mapped by a unit block and is distributed by RADS; the state field indicates whether the current cell block address stores data, 0 indicates stored data, and 1 indicates that no data is stored; lsn field indicates the starting logical sector address of the stored data within the current cell block; the size field indicates the size of the stored data block;
the dynamic stripe mapping table is called as the dynamic stripe table for short, in a data structure of the dynamic stripe table, a structure body represents the recording information of one stripe in the dynamic stripe table, the dynamic stripe table also has N entries, only the entry which records the mapping information is effective, the stripes recorded in the dynamic stripe table correspond to the stripes with the type value of 3 in the global stripe table one by one, and one dynamic stripe table entry exists when one global stripe table entry with the type value of 3 exists;
in a data structure of a dynamic stripe table, LSN represents the offset serial number of a stripe in the dynamic stripe table, the value range of the LSN is 0 to N-1, each entry in the dynamic stripe table records two parts of information, one part records the use state of the stripe, and the other part is the set of a stripe unit block structure body; the value of the used _ state field indicates the use state of the strip in the dynamic strip table, 2 values are total, when the value is 0, the entry is not used, and the value is 1, the entry records strip mapping information; the vector field indicates the valid state of each data block in the mapping stripe, and indicates the state of all cell blocks in the dynamic stripe, each bit in the vector is used for marking, the value of 0 indicates invalid, and the value of 1 indicates valid;
the number of unit blocks in the structure is the same as the number of unit blocks in the logical stripe, and starts from 0 to ends with 2 in the order of offset, and each unit block data structure records 4 pieces of information, the meaning of the 4 pieces of information is as follows: the valid field represents the valid state of the data in the current unit block, the value range is 0 to 2, 0 represents invalid, 1 represents invalid data mapping but can be used for data reconstruction, and 2 represents valid; lsn field indicates the starting logical sector address of the stored data within the current cell block; the size field indicates the size of the stored data block and the LSN indicates the original logical slice number of the data within the unit block.
3. The method of claim 1, wherein in the dynamic band having the cell block of 3, the vector value of 101 indicates that the cell blocks of offsets 0 and 2 are valid, and the cell block of offset 1 is invalid.
4. The method of claim 1, wherein the system is further designed with a data reconstruction module for reconstructing failure data and performing read-write processing when the array is degraded.
5. The method of claim 1, wherein a reclamation module is further configured in the system for reclamation of obsolete data.
6. The method of claim 2, wherein the system is implemented within a RAID controller.
7. A method for implementing dynamic stripe policy implemented by a system designed according to the method of claim 6, wherein the method comprises the following steps when implementing a write request:
when a write request enters a RAID controller, the write request is divided and calculated according to the encoding mode of RAID5, continuous request addresses are divided into a plurality of address blocks with fixed sizes according to the preset size of a stripe unit, partial continuous address blocks in the address blocks form a full stripe, namely a static stripe, partial continuous address blocks form a local stripe, identification of the static stripe and the local stripe is judged according to the number of data blocks in the stripe, the static stripe and the local stripe are processed in different modes, the static stripe data blocks are recorded by a metadata module after calculation and verification, and the request write operation is executed and flushed back to a solid state disk array; local stripe data blocks are directly placed into a buffer pool by the data cache part, the stripe reconstruction part reconstructs the local stripe data blocks into dynamic stripes, the stripes are distributed and checked, the dynamic stripe data blocks are also recorded by the metadata module and then written into the solid state hard disk array, the writing operation is executed and the writing operation is brushed back into the solid state hard disk array, for static stripes with stripe addresses occupied, the checking calculation can be carried out after the stripe numbers are distributed, and the static stripes can be written into the bottom layer array after the metadata is recorded;
the method comprises the following steps when the read request is realized:
storing the divided request blocks into a read request queue indiscriminately no matter the divided request blocks are positioned in static stripes or local stripes, retrieving data from a solid state disk array when a read operation is executed, inquiring a global stripe table, returning different values according to the mapping condition if the type of the stripe to which the request belongs is 0, returning mapping address data when the address of the stripe is mapped, and otherwise returning to null; if the stripe type is 1, returning request address data; the stripe type is 2, and a corresponding value is returned according to the mapping condition of the address; and if the stripe type is 3, the data of the mapping address is directly returned.
8. The method of claim 7, wherein the two types of stripes, and the modification of the metadata is determined according to the stripe type of the corresponding write address in the global stripe table, the method comprises the following steps when implementing static stripe metadata processing:
static stripe metadata processing flow:
when a static stripe is written, by inquiring a global stripe mapping table, for a stripe with the type of 0, if a data block in the stripe has mapping information, firstly clearing the original mapping information, then recording storage address information, and juxtaposing the stripe with the type of 1; for the stripe with the type 1, directly recording the storage address information; the processing mode of the strip with the type 2 is the strip with the same type 0; for the stripe of type 3, after the stripe number is allocated, the processing is performed in a dynamic stripe manner.
9. The method of claim 7, wherein the two types of stripes and the modification of the metadata are determined according to the stripe type of the corresponding write address in the global stripe table, and the method comprises the following steps when implementing dynamic stripe metadata processing:
the data in the dynamic stripe unit block is formed by combining data blocks at different offsets in different stripes, before the dynamic stripe is written, a stripe number is firstly distributed, current mapping information and state information are added, and the original mapping information of the unit block is deleted;
correspondingly processing metadata according to the type of a logical stripe to which each data block belongs in a stripe in a global stripe table, and clearing mapping information if a block address is mapped for a data block with an original stripe type of 0, and then recording current mapping information; for the data block with the original stripe type of 1, after recording the current mapping information, setting the original stripe type of 2; and recording the information of the storage address of the data block with the original stripe type of 2 or 3 in the same processing mode as the data block with the original stripe type of 0 after the data blocks at all the offsets in the dynamic stripe are processed.
CN201810680397.1A 2018-06-27 2018-06-27 Dynamic stripe system design method based on RAID5 solid state disk array Active CN108958656B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810680397.1A CN108958656B (en) 2018-06-27 2018-06-27 Dynamic stripe system design method based on RAID5 solid state disk array

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810680397.1A CN108958656B (en) 2018-06-27 2018-06-27 Dynamic stripe system design method based on RAID5 solid state disk array

Publications (2)

Publication Number Publication Date
CN108958656A CN108958656A (en) 2018-12-07
CN108958656B true CN108958656B (en) 2021-01-08

Family

ID=64487319

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810680397.1A Active CN108958656B (en) 2018-06-27 2018-06-27 Dynamic stripe system design method based on RAID5 solid state disk array

Country Status (1)

Country Link
CN (1) CN108958656B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111949434A (en) * 2019-05-17 2020-11-17 华为技术有限公司 RAID management method, RAID controller and system
CN110297601B (en) * 2019-06-06 2020-06-23 清华大学 Solid state disk array construction method, electronic device and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2645225A1 (en) * 2012-03-30 2013-10-02 Fujitsu Limited Storage control apparatus, storage system, and storage control method
CN103699343A (en) * 2013-12-25 2014-04-02 华为技术有限公司 RAID (redundant arrays of independent disks) determination method and device
CN104794070A (en) * 2015-04-23 2015-07-22 南京道熵信息技术有限公司 Solid-state flash memory write cache system and method based on dynamic non-covering RAID technology
CN106933490A (en) * 2015-12-29 2017-07-07 伊姆西公司 The method and apparatus that control is written and read operation to disk array
CN108205423A (en) * 2016-12-20 2018-06-26 华为技术有限公司 A kind of physical hard disk abrasion equilibrium method, apparatus and system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9940256B2 (en) * 2015-11-09 2018-04-10 International Business Machines Corporation Implementing hardware accelerator for storage write cache management for managing cache line updates for writes, reads, and destages in storage write cache

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2645225A1 (en) * 2012-03-30 2013-10-02 Fujitsu Limited Storage control apparatus, storage system, and storage control method
CN103699343A (en) * 2013-12-25 2014-04-02 华为技术有限公司 RAID (redundant arrays of independent disks) determination method and device
CN104794070A (en) * 2015-04-23 2015-07-22 南京道熵信息技术有限公司 Solid-state flash memory write cache system and method based on dynamic non-covering RAID technology
CN106933490A (en) * 2015-12-29 2017-07-07 伊姆西公司 The method and apparatus that control is written and read operation to disk array
CN108205423A (en) * 2016-12-20 2018-06-26 华为技术有限公司 A kind of physical hard disk abrasion equilibrium method, apparatus and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
S-RAID5:一种适用于顺序数据访问的节能磁盘阵列;李元章,等.;《计算机学报》;20130630;第36卷(第6期);第1290-1302页 *

Also Published As

Publication number Publication date
CN108958656A (en) 2018-12-07

Similar Documents

Publication Publication Date Title
CN103049222B (en) A kind of RAID5 writes IO optimized treatment method
KR101678868B1 (en) Apparatus for flash address translation apparatus and method thereof
US5537588A (en) Partitioned log-structured file system and methods for operating the same
CN102436353B (en) Semiconductor memory controlling device
KR100827677B1 (en) A method for improving I/O performance of RAID system using a matrix stripe cache
US5309451A (en) Data and parity prefetching for redundant arrays of disk drives
US5315602A (en) Optimized stripe detection for redundant arrays of disk drives
US7958331B2 (en) Storage device with opportunistic address space
US7761655B2 (en) Storage system and method of preventing deterioration of write performance in storage system
US7281089B2 (en) System and method for reorganizing data in a raid storage system
JP2004213647A (en) Writing cache of log structure for data storage device and system
JP2008204041A (en) Storage device and data arrangement control method
CN101916173B (en) RAID (Redundant Array of Independent Disks) based data reading and writing method and system thereof
US5634109A (en) Method and system for enhanced data management efficiency in memory subsystems utilizing redundant arrays of disk memory devices and a nonvolatile cache
CN104503706B (en) A kind of data storage and read method based on disk array
CN108958656B (en) Dynamic stripe system design method based on RAID5 solid state disk array
US20020118582A1 (en) Log-structure array
US9009396B2 (en) Physically addressed solid state disk employing magnetic random access memory (MRAM)
US20090265578A1 (en) Full Stripe Processing for a Redundant Array of Disk Drives
CN102779096B (en) Page, block and face-based three-dimensional flash memory address mapping method
CN101236482A (en) Method for processing data under degrading state and independent redundancy magnetic disc array system
CN107221351B (en) Optimization processing method of error correcting code in solid-state disk system and application thereof
CN106293543A (en) A kind of based on incidentally reclaim watt record a RAID write ordered method and device
US6678787B2 (en) DASD-free non-volatile updates
CN107924291B (en) Storage system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant