CN117234436A

CN117234436A - Method, device, storage system and product for expanding capacity of disk array

Info

Publication number: CN117234436A
Application number: CN202311515609.8A
Authority: CN
Inventors: 夏方健; 朱红玉
Original assignee: Suzhou Metabrain Intelligent Technology Co Ltd
Current assignee: Suzhou Metabrain Intelligent Technology Co Ltd
Priority date: 2023-11-14
Filing date: 2023-11-14
Publication date: 2023-12-15
Anticipated expiration: 2043-11-14
Also published as: CN117234436B

Abstract

The application relates to the technical field of data storage, and discloses a capacity expansion method, a capacity expansion device, a storage system and a storage product of a disk array. The method comprises the following steps: dividing the space of the newly-added hard disk into a plurality of newly-added data blocks according to the size of the array blocks, and adding the plurality of newly-added data blocks to the disk array; marking all newly added data blocks as unexpanded, and keeping the space distribution of original data blocks and check blocks in the disk array unchanged; when data is written into a data block which is not expanded for the first time, generating full-block data based on the data to be written and writing the full-block data into the data block which is not expanded, and updating the check data of the stripe to which the data block belongs; and marking the data blocks as the capacity expansion is completed after the data blocks which are not expanded are written into the full-block data for the first time. By adopting the method, a large amount of data migration generated during the capacity expansion of the disk can be avoided, the capacity expansion process is quickened, and the processing performance of the storage system on IO requests in the capacity expansion process is improved.

Description

Method, device, storage system and product for expanding capacity of disk array

Technical Field

The present application relates to the field of data storage technologies, and in particular, to a method and apparatus for expanding a disk array, a storage system, and a product.

Background

RAID (Redundant Array of Independent Disk, redundant array of independent disks) is a virtualization technique that combines multiple independent physical disks into a virtual disk group in different ways. Thus solving the defects of small capacity, low performance and poor reliability of a single magnetic disk. The RAID uses a stripe technology to divide data requested by a host into a plurality of sub-IOs (Input/Output data reading and writing requests) according to a certain topology and divide the sub-IOs onto different disks, and when the data is accessed, related disks in an array act together, so that the access speed of the data and the utilization rate of a disk operating space are greatly improved.

When a user uses the storage device, a volume needs to be created on the RAID array as block equipment, the block equipment is provided for a front-end host to perform read-write operation, when the storage space of the volume is insufficient and more data cannot be stored, the volume needs to be expanded, and if the RAID array where the expanded volume is located has free available space, the available space can be directly allocated from the RAID array to expand the volume. However, when there is no free space available in the RAID array, it is necessary to add a hard disk to expand the RAID before expanding the volume.

When the capacity of the disk array is expanded, the disk array redistributes the storage space according to the current number of hard disks, and the positions of the data blocks and the check blocks in the disk array change in the process, so that the data stored in the disk needs to be migrated, namely the data is read from the original data blocks into the memory, and then the data is written into the new data blocks from the memory. The large amount of data migration operation can seriously occupy the system bandwidth, reduce the processing performance of the storage system for the foreground host service IO (read/write request), and influence the normal operation of the user service.

Disclosure of Invention

In view of this, the present application is directed to a method, an apparatus, a storage system and a product for expanding a disk array, so as to solve the problem that the performance of the storage system in handling the service IO of a foreground host is severely reduced due to a large amount of data migration generated during the expansion of the disk array.

In order to achieve the above purpose, the technical scheme of the application is as follows:

an embodiment of the present application provides a method for expanding a disk array, where the method includes:

dividing the space of the newly-added hard disk into a plurality of newly-added data blocks according to the size of the array blocks, and adding the plurality of newly-added data blocks to the disk array; the array block size is the data block size appointed when the disk array is created;

Marking all newly added data blocks as unexpanded, and keeping the space distribution of original data blocks and check blocks in the disk array unchanged;

when data is written into a data block which is not expanded for the first time, generating full-block data based on the data to be written and writing the full-block data into the data block which is not expanded, and updating the check data of the stripe to which the data block belongs; the size of the space occupied by the full-block data is consistent with the size of the array blocks;

and marking the data blocks as the capacity expansion is completed after the data blocks which are not expanded are written into the full-block data for the first time.

Optionally, the capacity expansion method of the disk array further includes:

initializing a capacity expansion bitmap according to the newly added data blocks in the disk array; the capacity expansion bitmap is used for recording whether the disk array is expanded;

and updating the capacity expansion bitmap according to the position of the data block in the disk array after the unexpanded data block is written with full block data for the first time.

Optionally, adding the plurality of newly added data chunks to a disk array includes:

and respectively adding each newly added data block to the tail of each stripe in the disk array by taking the hard disk as a unit, so that each stripe contains one data block of each hard disk.

Optionally, generating the full block data based on the data to be written includes:

acquiring the space size occupied by data to be written and the array block size; and complementing the data to be written by using all-zero data to generate full block data.

creating an all-zero memory area in the memory of the disk array according to the array block size, and storing all-zero data;

and according to the size of the array block, taking out all-zero data from the all-zero memory area to complement the data to be written so that the size of the space occupied by the data to be written is consistent with the size of the array block.

Optionally, generating full block data based on the data to be written, further includes:

when the size of the data to be written is equal to the array block size, taking the data to be written as the full block data;

when the size of the space occupied by the data to be written is larger than the size of the array block, dividing the data to be written according to the size of the array block;

and generating full block data for each part of data obtained by dividing, and writing different unexpanded data blocks.

Optionally, initializing a capacity expansion bitmap according to the newly added data block in the disk array, including:

creating a capacity expansion bitmap, and recording strip numbers and strip identifications according to the newly added data blocks, wherein the strip numbers correspond to the strip identifications one by one, the strip numbers indicate the positions of the strips, and the strip identifications consist of the block identifications; the block identifier comprises an unexpanded state and a completed expanded state and is used for indicating whether the newly added data blocks in the strip are expanded or not;

and when initializing the capacity expansion bitmap, setting all the block identifiers to be in an unexpanded state.

Optionally, updating the capacity expansion bitmap according to the position of the data block in the disk array includes:

acquiring the band number of the band where the data block is located;

determining a location of the data chunk in the stripe;

and determining a corresponding strip identifier from the capacity expansion bitmap according to the strip number, and setting a corresponding position character as a capacity expansion completion state according to the position of the data block in the strip.

Optionally, the capacity expansion method of the disk array further includes:

when a data writing request is received, judging whether the area of the data writing request operation is a newly added data block;

If the region of the data writing request operation is not the newly added data block, the data is directly written into the region;

if the area of the data writing request operation is a newly added data block, inquiring the capacity expansion bitmap to judge whether the data block is expanded; and if the data block is expanded, writing the data into the area directly.

Optionally, the capacity expansion method of the disk array further includes:

when a data reading request is received, judging whether an area operated by the data reading request is a newly added data block or not;

if the region of the read data request operation is not the newly added data block, directly reading the data of the region;

if the area of the read data request operation is a newly added data block, inquiring the capacity expansion bitmap to judge whether the data block is expanded; if the capacity expansion of the data blocks is finished, directly reading the data of the data blocks; if the data block is not expanded, the data block is not read and all-zero data is directly returned.

Optionally, storing the initialized capacity expansion bitmap in a nonvolatile memory;

and deleting the capacity expansion bitmap and releasing the memory when all the block identifiers in the capacity expansion bitmap are in a capacity expansion completion state.

Optionally, the disk array is a disk array with check blocks.

According to a second aspect of an embodiment of the present application, there is provided a capacity expansion device of a disk array, for implementing a method according to the first aspect of the embodiment of the present application, where the device includes:

the adding module is configured to divide the newly-added hard disk space into a plurality of newly-added data blocks according to the size of the array blocks, and add the plurality of newly-added data blocks to the disk array; the array block size is the data block size appointed when the disk array is created;

the recording module is configured to mark all newly added data blocks as unexpanded, and keep the space distribution of original data blocks and check blocks in the disk array unchanged;

the data access module is configured to generate full-block data based on data to be written and write the full-block data into the unexpanded data block when the data is written into the unexpanded data block for the first time, and calculate and update the verification data of the stripe to which the data block belongs according to the full-block data; the size of the space occupied by the full-block data is consistent with the size of the array blocks;

the recording module is further configured to mark the data blocks as being expanded after the data blocks which are not expanded are written with full block data for the first time.

According to a third aspect of embodiments of the present application, there is provided a storage system, the system comprising:

the upper layer unit is used for receiving an access request or a capacity expansion instruction issued by the host and sending the access request or the capacity expansion instruction to the array management module; the access request comprises a read data request and a write data request;

an array management unit for managing a disk array, including: controlling the hard disk and the drive unit to access the data of the disk array according to the access request; according to the capacity expansion instruction, executing the method according to the first aspect of the embodiment of the application, and expanding the capacity of the disk array;

the hard disk and drive unit comprises a disk array composed of a plurality of hard disks and is used for executing data access to the disk array according to the access request.

According to a fourth aspect of embodiments of the present application, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method according to the first aspect of embodiments of the present application.

According to a fifth aspect of the embodiments of the present application, there is provided an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps in the method according to the first aspect of the embodiments of the present application when the computer program is executed.

According to the capacity expansion method of the disk array, the non-expanded data blocks are marked, so that the reassignment of the disk storage space is omitted, a large amount of data migration is not needed, and the occupation of the system bandwidth is reduced. When data is written into the unexpanded data blocks for the first time, the data to be written is converted into full-block data, the unexpanded data blocks are covered, the operation of emptying the unexpanded data blocks is saved, the access times to the hard disk are reduced, and the performance of the storage system for processing IO requests in the expansion period is further improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments of the present application will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a method for expanding a disk array according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a capacity expansion device of a disk array according to an embodiment of the present application;

FIG. 3 is a flow chart of expanding a disk array using an expansion bitmap according to an embodiment of the present application;

FIG. 4 is a schematic diagram illustrating a spatial distribution of disk array expansion according to an embodiment of the present application;

FIG. 5 is a flow chart of processing IO requests by a disk array in accordance with one embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

In various embodiments of the present application, it should be understood that the sequence numbers of the following processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the accompanying claims.

It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other.

Disk arrays with parity information, such as RAID5, RAID6, etc., are more secure and reliable than disk arrays without parity data blocks, and therefore are widely used. When a hard disk fails in RAID, the data damaged by the failure can be recovered by checking the data and the data in the rest hard disks. Taking RAID5 as an example, the flow of disk array capacity expansion is as follows:

(1) And starting capacity expansion operation, starting silence and suspending host business processing in the disk array, and reallocating space distribution. The check blocks of each stripe in the disk array are arranged according to a certain rule, and after the hard disk space is newly added, the disk array can readjust the space distribution of the data blocks and the check blocks.

(2) And finishing the spatial distribution, finishing silence, recovering service processing, performing data migration according to the new spatial distribution, recalculating the check data and writing the check data into the check block. After readjusting the spatial distribution, the locations of the data blocks and the check blocks change, wherein part of the stored data needs to be migrated to a new location, thus involving a large number of hard disk read-write operations.

(3) In the data migration process, a small amount of newly added data blocks which do not need to be migrated into data are emptied by writing all-zero data into the newly added data blocks.

(4) When all data migration in the disk array is completed, the disk array completes capacity expansion. The data migration process needs to migrate the migrated data from the original hard disk to the newly allocated hard disk area, specifically, the data is firstly read from the original hard disk and put into the memory, then the data is written into another hard disk from the memory, and then the check data of the stripe is recalculated, namely, the exclusive OR operation is performed on the data of all the data blocks in the stripe to obtain the new check data.

The capacity expansion process brings a large number of read-write and exclusive-OR operations, occupies a large amount of system bandwidth resources, and causes the reduction of the processing performance of the system for the foreground host business IO during the capacity expansion. The application records the data blocks without capacity expansion by not carrying out data migration, and adopts the mode of writing in the form of full block data when writing data for the first time, thereby saving a large number of times of reading and writing to the hard disk, saving the system bandwidth and improving the capacity expansion efficiency.

The application will be described in detail below with reference to the drawings in connection with embodiments.

FIG. 1 is a flow chart of a method for expanding a disk array according to an embodiment of the present application. As shown in fig. 1, the method includes:

s1: dividing the space of the newly-added hard disk into a plurality of newly-added data blocks according to the size of the array blocks, and adding the plurality of newly-added data blocks to the disk array; the array block size is a data block size specified when the disk array is created.

As one embodiment of the present application, adding the plurality of newly added data blocks to the disk array includes:

S2: and marking all the newly added data blocks as unexpanded, and keeping the spatial distribution of the original data blocks and the verification blocks in the disk array unchanged.

When the capacity of the disk is expanded, the number of the newly-added hard disks is required, taking RAID5 as an example, each stripe in RAID5 with the stripe width of N is provided with a check block, so that the block of the actually stored data in the stripe is N-1, and when the capacity is expanded, the number M of the newly-added hard disks is smaller than N-1.

FIG. 4 is a schematic diagram illustrating a spatial distribution of disk array expansion according to an embodiment of the present application. Fig. 4 is a schematic expansion diagram of a disk array, where the disk array is generated based on 4 hard disks, and each stripe includes three data blocks D1, D2, and D3 for storing user data, and a check block P for storing check data. When a hard disk (hard disk 5) is newly added for capacity expansion, the hard disk 5 is divided into newly added data blocks and is programmed into the tail of the disk array (as shown by a D4 block in fig. 4), and the spatial distribution of the original data blocks and the check blocks in the disk array is kept unchanged, namely, the positions of the original data blocks and the check blocks in the disk array are kept unchanged. By not changing the spatial distribution of the disk array, a large number of data migration operations caused by the change of the spatial distribution are avoided, the bandwidth resources of the system are saved, and the capacity expansion efficiency of the disk array is improved.

When more than one hard disk is newly added, the data blocks are orderly coded at the tail of the disk array by taking the hard disk as a unit, so that each strip of the disk array contains one data block of all the hard disks. As shown in fig. 4, when two newly added hard disks are added, the newly added D5 data blocks are added to the D4 blocks in sequence, so that each stripe includes: d1 data block, D2 data block, D3 data block, D4 data block, D5 data block and verification block P.

In this embodiment, the newly added hard disk space is added according to the array block size specified when the disk array is created. According to the size of the designated array block, adding the newly added hard disk space into the disk array in the form of data blocks, and then marking all the newly added data blocks as unexpanded data blocks, so that the spatial distribution of the disk array before expansion is maintained, and multiple accesses to the hard disk caused by a large amount of data migration in the disk array are avoided, thereby saving the system bandwidth.

When the hard disk is added to the disk array, the residual data belongs to garbage data to be cleared, and if new data is directly written into the data blocks, data errors are caused, so that the data of the newly added data blocks need to be cleared before the data is written. In order to further reduce the number of times of reading and writing to the hard disk, the application does not directly write all zero data into the newly added data blocks for data clearing, but marks all the newly added data blocks as unexpanded. Before data is written to the unexpanded data blocks for the first time, the unexpanded data blocks are defaulted to store all-zero data.

S3: when data is written into a data block which is not expanded for the first time, generating full-block data based on the data to be written and writing the full-block data into the data block which is not expanded, and updating the check data of the stripe to which the data block belongs; the size of the space occupied by the full-block data is consistent with the size of the array blocks.

S4: and marking the data blocks as the capacity expansion is completed after the data blocks which are not expanded are written into the full-block data for the first time.

In this embodiment, when data is written into a data block without capacity expansion for the first time, the writing is performed in the form of full-block data, and the space occupied by the full-block data is the same as the size of the data block. For example, when the array block size set at the time of creating the disk array is 256k, when the disk is expanded, the newly added data block size is added according to the array block size, and the full block data is 256k data. Since full block data can completely cover the whole data block, garbage data possibly existing in the data block can be cleared while the full block data is written. In this embodiment, by writing full block data when writing data for the first time, the number of accesses to the hard disk is further reduced, so that the system bandwidth is saved, and the performance of the system for processing IO requests is improved.

After the full-block data is written for the first time, the non-expanded data is subjected to block expansion, and then normal read-write operation can be performed on the non-expanded data. In this embodiment, the capacity expansion process of the disk array is monitored by marking the data blocks with the capacity expanded.

When the disk array expands, the check data in each stripe needs to be updated because the data blocks of the stripe change. The check data is obtained by exclusive-or of the data of all the data blocks in the stripes, and because the newly added data blocks are defaulted to store all the zero data, namely, after the all the zero data is exclusive-or with the original check data, the check data is unchanged, so that when the capacity of each stripe is expanded by adopting the method of the application, the check data of each stripe does not need to be recalculated, and the system bandwidth is further saved.

As an embodiment of the present application, the method further comprises:

initializing a capacity expansion bitmap according to the newly added data blocks in the disk array; the capacity expansion bitmap is used for recording whether the disk array is expanded.

In one embodiment, whether the disk array is fully expanded is recorded by an expansion bitmap. The capacity expansion bitmap is used for recording whether the newly added data blocks in the disk array are expanded or not, and when all the newly added data blocks are expanded, the disk array is expanded.

As one embodiment of the present application, initializing a capacity expansion bitmap includes:

In this embodiment, a capacity expansion bitmap is created, and the capacity expansion state of the newly added data block in the disk array is recorded in units of stripes. The capacity expansion bitmap stores the number of each stripe and the corresponding stripe identification. The stripe identifier consists of one or more block identifiers. The block identifier indicates the capacity expansion state of the newly added data block at the corresponding position. The identification is used for distinguishing the unexpanded state from the expanded state, and can be set according to the needs in practical application. For example, the unexpanded state is set to "1", and the completed expanded state is set to "0". Initializing a capacity expansion bitmap when a new hard disk space is added to the disk array, and setting block identifiers in all strip identifiers to be 1, which means that all current new data blocks are unexpanded data blocks.

The stripe mark in the capacity expansion bitmap indicates the capacity expansion condition of the current disk array, when the data blocks which are not expanded in the disk array are written with full block data for the first time, the data blocks are expanded, and the block mark in the capacity expansion bitmap is correspondingly modified to be in the expansion state.

As one embodiment of the present application, updating the capacity expansion bitmap according to the location of the data block in the disk array includes:

acquiring the band number of the band where the data block is located;

determining a location of the data chunk in the stripe;

And obtaining a corresponding stripe identifier from the capacity expansion bitmap through the stripe number, and determining a block identifier corresponding to the data block according to the position of the data block in the stripe. And after the data partitioning is completed to expand the capacity, modifying the corresponding partitioning identification into a completed capacity expansion state. For example, the chunk identifier of the corresponding position of the data chunk is set to "0".

FIG. 3 is a flow chart of expanding a disk array using an expansion bitmap according to an embodiment of the present application. As shown in fig. 3, the steps for expanding the capacity of the disk array are as follows:

(1) When creating a disk array, the array block size of the disk array is specified by a command line. The array block size may be set according to actual needs, for example, the array block size may be set to 256k.

(2) When the disk array detects that the capacity expansion task is triggered, initializing a capacity expansion bitmap in the disk array according to the strip information of the current disk array. The capacity expansion bitmap is used to record the stripe label and the stripe identification in the current disk array. The stripe mark comprises block marks of data blocks newly added to the disk array, for example, when the number of newly added hard disks is 2 during capacity expansion, the stripe mark consists of 2 block marks, and each block mark corresponds to the capacity expansion state of one data block respectively. The capacity expansion bitmap is used for recording the number of the stripes in the array and whether the stripes are expanded. And when the capacity expansion bitmap is initialized, setting all the stripe marks to be all 1, namely, all the newly added data blocks are not expanded.

As shown in fig. 3, in the present application, after the space of the hard disk is newly added, the data block D4 area of the newly added hard disk 5 is defaulted to be an all-zero area, that is, the D4 area is regarded as storing all-zero data, but the all-zero data is not written therein, and the D4 area is recorded as an unexpanded state by the capacity expansion bitmap mark. When an IO request is received, whether a capacity expansion bitmap exists or not is firstly judged, and the current capacity expansion state of the D4 area is judged according to the capacity expansion bitmap so as to determine the subsequent IO processing flow.

After the newly added hard disk space is written into the disk array, the check data of the stripe needs to be updated. The calculation mode of the check data is that exclusive or calculation is carried out on the data of all the data blocks in the stripe, namely:

p= (D1 block data) xor (D2 block data) xor (D3 block data).

In this embodiment, since the data migration is not performed on other data blocks after the D4 area is newly added, the manner of calculating the new verification data P' is as follows:

p' = (data of D1 block) xor (data of D2 block) xor (data of D3 block) xor (data of D4 block). The D4 block stores all zero data by default, so that new check data P' =p, i.e. check data does not need to be recalculated and written, thereby further saving system bandwidth resources.

(3) When new data is written into the disk array, firstly judging whether the written data block is an unexpanded data block, if so, writing the data to be written into the unexpanded data block, and if so, writing the data to be written into the unexpanded data block in the form of full block data.

After the data blocks which are not expanded are written into full-block data for the first time, the data blocks are expanded, and the block marks at the corresponding positions in the expansion bitmap are set to be 0.

(4) And (3) until all the unexpanded data blocks are covered and written once, namely after all the unexpanded data blocks are written with full block data once, all the stripe marks in the expansion bitmap are set to 0, and at the moment, the expansion of the disk array is completed, and the bitmap is deleted and the occupied space is released.

As one embodiment of the present application, generating full block data based on data to be written includes:

In one embodiment, full chunk data is generated by supplementing all zero data when writing data to a chunk of unexpanded data. Firstly, determining the size of the space occupied by data to be written, and then supplementing the data to be written by using all-zero data (namely, the value of all bytes in the data is 0) according to the size of the array block, namely, the size of the data block which is not expanded, so as to generate full-block data.

For example, when the array block size is 256k and the data to be written is 156k, the data to be written needs to be complemented with all zero data of 100k, and full block data of 256k is generated and is overwritten.

In this embodiment, full-block data is used to write the data blocks which are not expanded for the first time, so that the operation of data clearing for the newly added data blocks during expansion is saved, the expansion efficiency is improved, and the system bandwidth resources are saved.

In one embodiment, an all-zero memory area is newly built in advance in the disk array, and when the full block data needs to be generated, all-zero data with a required size is taken out from the all-zero memory area to complement the full block data.

Specifically, at disk array creation time, the array block size specified by the create RAID array command line is obtained, such as 256k. And adding a block of all-zero memory area with the size of 256k in the disk array according to the acquired array block size. When data is written into the unexpanded data block for the first time, the all-zero data is taken out from the preset all-zero memory area, and the data to be written is complemented.

In this embodiment, an all-zero data area with the same size as the array block is generated in advance in the disk array, and when the full block data needs to be generated, the part of the all-zero data with the required size, which is missing, can be quickly fetched out to complement the data to be written, so that the efficiency of processing the IO request during the capacity expansion of the disk array is increased.

As one embodiment of the present application, generating full block data based on data to be written, further includes:

In one embodiment, when data is first written to a block of unexpanded data, full block data needs to be written to the block of data. The method comprises the following steps:

judging the size of the data to be written, if the size of the data to be written is consistent with the size of the array blocks (for example, the size of the data to be written is 256 k), the data to be written is full-block data, and the data is directly written into the data blocks without capacity expansion;

If the size of the data to be written is smaller than the array block size, adopting all-zero data to complement the array block size to be written, and writing;

if the size of the data to be written is larger than the array block size, the data to be written is segmented, the part meeting the full block data size is directly used as full block data to be written into the unexpanded data block, and the rest part of data smaller than the array block size is complemented by adopting all-zero data to generate a new full block data to be written into the other unexpanded data block.

As an embodiment of the present application, the method further comprises:

In this embodiment, when the RAID receives an IO request, it first determines whether a capacity expansion bitmap exists in the disk array. If the capacity expansion bitmap does not exist, the new data block does not exist in the disk array, namely, the data block which is not expanded exists in the disk array. At this time, the IO request is normally processed: if the IO request is a data writing request, recalculating the verification data of the stripe according to the data to be written, and writing the data to be written and the new verification data into the data block and the verification block respectively; if the IO request is a read data request, the data blocks are directly read.

FIG. 5 is a flow chart of processing IO requests by a disk array in accordance with one embodiment of the present application. As shown in fig. 5, when the IO request is a write data request, the system processes the IO as follows:

(1) And judging whether a capacity expansion bitmap exists in the disk array or not. And if the capacity expansion bitmap does not exist, directly performing data writing operation on the data blocks.

(2) When the capacity expansion bitmap exists, judging whether the data writing request carries out data writing to the newly added data block. If the data writing request is not to write the data into the newly added data block, the data to be written can be directly written into the data block; if the data is written into the newly added data block, the newly added data block needs to be further queried from the capacity expansion bitmap whether the capacity expansion of the newly added data block is completed. Specifically, if the block identifier of the corresponding position in the bitmap is 1, it is indicated that the data block is not expanded, and if the block identifier of the corresponding position in the bitmap is 0, it is indicated that the data block is expanded.

(3) And writing data into the data blocks with the capacity expanded according to a normal data writing mode and recalculating check data. And for the data blocks which are not expanded, adopting a mode of writing data to the data blocks which are not expanded for the first time, namely, supplementing the data to be written with all zero data to generate full block data, writing the full block data into the data blocks, and updating the check data.

(4) After full block data is written into the non-expanded data blocks, the block mark at the corresponding position in the expansion bitmap is set to be in the expansion completion state, namely, 0 is set.

As an embodiment of the present application, the method further comprises:

In the above embodiment, as shown in fig. 5, when the IO request is a read data request, the system processes the IO as follows:

(1) And judging whether a capacity expansion bitmap exists in the disk array or not. And if the capacity expansion bitmap does not exist, directly performing data reading operation on the data blocks.

(2) When the capacity expansion bitmap exists, whether the data writing request reads data from the newly added data block or not is judged. If the data writing request does not read the data from the newly added data block, the data block can be directly read; if the data is read from the newly added data block, the capacity expansion bitmap is further required to inquire whether the newly added data block is expanded. Specifically, if the block identifier of the corresponding position in the bitmap is 1, it is indicated that the data block is not expanded, and if the block identifier of the corresponding position in the bitmap is 0, it is indicated that the data block is expanded.

(3) And partitioning the data subjected to capacity expansion, and directly reading the data according to a normal data reading mode. For the data block which is not expanded, the data block is not directly read, but the data block is defaulted to store all-zero data, so that the all-zero data is directly returned as a reading result.

In this embodiment, the non-expanded data block is regarded as storing all zero data, and when the data reading request is performed on the non-expanded data block, the hard disk is not required to be actually read, but all zero data is directly returned as a reading result, so that the number of times of accessing the hard disk is reduced, and meanwhile, the data error caused by the fact that the garbage data possibly remained in the non-expanded data block is read is avoided.

As an embodiment of the present application, the method further comprises:

storing the initialized capacity expansion bitmap in a nonvolatile memory;

In one embodiment, to prevent loss of the expanded state of a data block in the disk array due to a system power failure, the expanded bitmap may be stored in nonvolatile memory. When all newly added data blocks in the disk array are written into the full-block data, all the stripe marks in the corresponding capacity expansion bitmap are 0, and at the moment, the capacity expansion of the disk array is completed, the capacity expansion bitmap can be deleted, and the occupied memory is released.

As one embodiment of the present application, updating the check data of the stripe to which the data block belongs includes:

exclusive or is carried out on the original check data in the stripe to which the data block belongs and the data in the data block to obtain new check data;

and writing the new check data into the check blocks in the strip to replace the original check data.

In this embodiment, each time a write operation is performed on the hard disk, the stripe's parity data needs to be updated. The check data is calculated by exclusive-or operation of the data stored in all the data blocks in the stripe. In the process, the data blocks in the hard disk are required to be read, if the unexpanded data blocks exist in the stripe, the unexpanded data blocks are not read, but the unexpanded data blocks are defaulted to store all-zero data, 0 is returned, and exclusive OR operation is carried out on the data blocks and the data in other data blocks.

The access times to the hard disk are reduced in a mode of not reading the unexpanded data blocks, and as the unexpanded data blocks possibly remain garbage data, the unexpanded data blocks are defaulted to store all-zero data, and the exclusive-or operation is carried out on the all-zero data and the data of other data blocks, the influence of the garbage data on the calculation result of check data can be avoided, and data errors are prevented.

Based on the same inventive concept, an embodiment of the present application provides a capacity expansion device of a disk array. Referring to fig. 2, fig. 2 is a schematic diagram of a capacity expansion device 100 of a disk array according to an embodiment of the application. As shown in fig. 2, the apparatus includes:

an adding module 101 configured to divide the newly added hard disk space into a plurality of newly added data blocks according to the array block size, and add the plurality of newly added data blocks to the disk array; the array block size is the data block size appointed when the disk array is created;

the recording module 102 is configured to mark all newly added data blocks as unexpanded, and keep the space distribution of original data blocks and check blocks in the disk array unchanged;

a data access module 103 configured to generate full-block data based on data to be written and write the full-block data into the unexpanded data block when the data is written into the unexpanded data block for the first time, and calculate and update check data of a stripe to which the data block belongs according to the full-block data; the size of the space occupied by the full-block data is consistent with the size of the array blocks;

the recording module 102 is further configured to mark the data blocks as being expanded after the data blocks that are not expanded are first written with full block data.

As an embodiment of the present application, the recording module 102 includes:

the bitmap management submodule is configured to initialize a capacity expansion bitmap according to the newly added data blocks in the disk array; the capacity expansion bitmap is used for recording whether the disk array is expanded;

and the bitmap updating sub-module is configured to update the capacity-expansion bitmap according to the position of the data block in the disk array after the non-capacity-expanded data block is written with full-block data for the first time.

As an embodiment of the present application, the adding module 101 is configured to perform the following steps:

As an embodiment of the present application, the data access module 103 is configured to perform the following steps:

As an embodiment of the present application, the data access module 103 includes:

The all-zero data management module is configured to establish an all-zero memory area in a memory of the disk array according to the array block size and store all-zero data;

and the data access module 103 takes out all-zero data from the all-zero memory area according to the array block size to complement the data to be written so that the space size occupied by the data to be written is consistent with the array block size.

As an embodiment of the present application, the data access module 103 is further configured to perform the following steps:

As an embodiment of the present application, the bitmap management submodule is configured to perform the following steps:

As an embodiment of the present application, the bitmap update sub-module is configured to perform the following steps:

acquiring the band number of the band where the data block is located;

determining a location of the data chunk in the stripe;

As an embodiment of the present application, the bitmap management submodule is further configured to perform the following steps:

storing the initialized capacity expansion bitmap in a nonvolatile memory;

Based on the same inventive concept, an embodiment of the present application provides a storage system including:

an array management unit for managing a disk array, including: controlling the hard disk and the drive unit to access the data of the disk array according to the access request; according to the capacity expansion instruction, executing the capacity expansion method of the disk array, which is proposed in the embodiment, and expanding the capacity of the disk array;

Based on the same inventive concept, an embodiment of the present application provides a readable storage medium having stored thereon a computer program, which when executed by a processor, implements the steps of the method for expanding a disk array according to any of the above embodiments of the present application.

Based on the same inventive concept, an embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor executes the steps in the method for expanding a disk array according to any of the foregoing embodiments of the present application.

The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.

The foregoing description of the preferred embodiments of the application is not intended to be limiting, but rather is intended to cover all modifications, equivalents, alternatives, and improvements that fall within the spirit and scope of the application.

For the purposes of simplicity of explanation, the methodologies are shown as a series of acts, but one of ordinary skill in the art will recognize that the present application is not limited by the order of acts described, as some acts may, in accordance with the present application, occur in other orders and concurrently. Further, those skilled in the art will recognize that the embodiments described in the specification are all of the preferred embodiments, and that the acts and components referred to are not necessarily required by the present application.

It will be apparent to those skilled in the art that embodiments of the present application may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the application may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the application.

Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal device comprising the element.

The capacity expansion method, device, storage system and product of the disk array provided by the application are described in detail, and specific examples are applied to the description of the principle and implementation mode of the application, and the description of the above examples is only used for helping to understand the method and core idea of the application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.

Claims

1. A method for expanding a disk array, comprising:

2. The method of expanding a disk array according to claim 1, further comprising:

3. The method of claim 1, wherein adding the plurality of newly added data blocks to the disk array comprises:

4. The capacity expansion method of a disk array according to claim 1, wherein generating full block data based on data to be written comprises:

5. The capacity expansion method of a disk array according to claim 1, wherein generating full block data based on data to be written comprises:

6. The capacity expansion method of a disk array according to claim 4 or 5, wherein generating full block data based on data to be written, further comprises:

7. The capacity expansion method of claim 2, wherein initializing the capacity expansion bitmap according to the newly added data block in the disk array comprises:

8. The capacity expansion method of claim 7, wherein updating the capacity expansion bitmap according to the location of the data block in the disk array comprises:

acquiring the band number of the band where the data block is located;

determining a location of the data chunk in the stripe;

9. The method of expanding a disk array according to claim 2, further comprising:

10. The method of expanding a disk array according to claim 2, further comprising:

11. The method of expanding a disk array according to claim 7, further comprising:

storing the initialized capacity expansion bitmap in a nonvolatile memory;

12. The capacity expansion method of claim 1, wherein updating the check data of the stripe to which the data block belongs comprises:

13. A capacity expansion device of a disk array, for implementing the method according to any one of claims 1 to 12, comprising:

14. A storage system, comprising:

an array management unit for managing a disk array, including: controlling the hard disk and the drive unit to access the data of the disk array according to the access request; according to the capacity expansion instruction, executing the capacity expansion method of the disk array according to any one of claims 1-12, and expanding the capacity of the disk array;

15. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any of claims 1-12.

16. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the steps of the method according to any of claims 1-12.