CN113419685A

CN113419685A - RAID creation based on SMR, data writing aiming at RAID and RAID recovery method

Info

Publication number: CN113419685A
Application number: CN202110784803.0A
Authority: CN
Inventors: 赵修伟
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2021-07-12
Filing date: 2021-07-12
Publication date: 2021-09-21
Anticipated expiration: 2041-07-12
Also published as: CN113419685B; WO2023284753A1

Abstract

The embodiment of the invention provides a RAID (redundant array of independent disks) creation method based on SMR (SMR), relating to the technical field of data storage. The SMR-based RAID creation method is applied to a preset RAID drive, and comprises the following steps: determining a plurality of SMRs for creating RAID; configuring SMR areas of the plurality of SMRs as a first array of a target RAID to be created; wherein the first array comprises a plurality of stripes, each stripe comprises a writing area with the same offset address in an SMR area of the plurality of SMRs and is used for storing data files written in an aligned sequential writing mode; configuring the CMR region of the plurality of SMRs as a second array of the target RAID. Compared with the prior art, by applying the method for creating the RAID, the RAID can be created by using the SMR without using other hard disks as the cache.

Description

RAID creation based on SMR, data writing aiming at RAID and RAID recovery method

Technical Field

The invention relates to the technical field of data storage, in particular to RAID creation based on SMR, data writing aiming at RAID and a RAID recovery method.

Background

SMR (shielded Magnetic Recording) improves the track density of the SMR (Shingled Magnetic Recording) by overlapping tracks, thereby increasing the capacity of the SMR. To take full advantage of SMRs, there is a need to create RAID (Redundant Arrays of Independent Disks) with SMRs.

However, considering that each write area in an SMR (Shingled Magnetic Recording) area of an SMR supports only sequential writing, but does not support out-of-order writing, in the related art, when a RAID created by the SMR is used, data storage needs to be additionally realized by using an SSD (Solid State Disk) as a cache.

Based on this, how to create RAID by using SMR without using other hard disks as cache becomes a technical problem to be solved at present.

Disclosure of Invention

The embodiment of the invention aims to provide a RAID creation method based on SMR, which is used for creating RAID by using SMR without using other hard disks as cache. In addition, the embodiment of the invention also provides a data writing method and a RAID recovery method aiming at the RAID, so as to realize the effective data writing and recovery aiming at the RAID created by the embodiment of the invention. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides a RAID creation method based on SMR, which is applied to a preset RAID driver, and the method includes:

determining a plurality of SMRs for creating RAID;

configuring SMR areas of the plurality of SMRs as a first array of a target RAID to be created; wherein the first array comprises a plurality of stripes, each stripe comprises a write area with the same offset address in an SMR area of the plurality of SMRs, and each stripe is used for storing data files written in an aligned sequential writing manner;

configuring the CMR region of the plurality of SMRs as a second array of the target RAID.

In a second aspect, an embodiment of the present invention provides a data writing method for RAID, which is applied to a preset RAID driver, and the method includes:

when detecting that the data volume of the data to be stored is not less than a first data volume, reading the data to be stored, wherein the data volume is the first data volume and is not written into a target RAID (redundant array of independent disks), and taking the data as a data file to be written; wherein the target RAID is created based on the method of any one of claims 1 to 5; the first data volume is the data volume of each data file stored by the target RAID;

writing the data file to be written into a first cache preset in a second array of the target RAID;

and for each write-in area included in a designated stripe in the first array of the target RAID, allocating the data content of the data file to be written cached in the first cache, so that the SMR to which each write-in area belongs writes the data content allocated to the write-in area into the write-in area.

In a third aspect, an embodiment of the present invention provides a RAID recovery method, which is applied to a preset RAID driver, and the method includes:

when the target RAID is restarted after abnormal power failure, determining a minimum write pointer and a maximum write pointer from write pointers in SMR areas of a plurality of SMRs in the target RAID; wherein the target RAID is created based on the method of any one of claims 1 to 5; the minimum write pointer is: the write pointer with the minimum corresponding storage address in write pointers in SMR areas of a plurality of SMRs in the target RAID; the maximum write pointer is: the write pointer with the largest corresponding storage address in write pointers in SMR areas of a plurality of SMRs in the target RAID;

determining data stored in each stripe in the first array of the target RAID before the stripe where the minimum write pointer is located as valid data in the first array of the target RAID;

moving a write pointer in an SMR area of the plurality of SMRs to a start position of a write area belonging to a target stripe; wherein the target stripe is: a stripe in the first array of the target RAID that is located after the stripe where the largest write pointer is located.

In a fourth aspect, embodiments of the present invention provide a RAID driver, where the RAID driver is configured to perform any one of the steps of the method described above.

In a fifth aspect, an embodiment of the present invention provides an SMR-based RAID creation apparatus, applied to a preset RAID driver, where the apparatus includes:

an SMR determination module to determine a plurality of SMRs for creating RAID;

a first array configuration module to configure SMR areas of the plurality of SMRs as a first array of a target RAID to be created; wherein the first array comprises a plurality of stripes, each stripe comprises a write area with the same offset address in an SMR area of the plurality of SMRs, and each stripe is used for storing data files written in an aligned sequential writing manner;

a second array configuration module to configure the CMR region of the plurality of SMRs as a second array of the target RAID; wherein the second array is at least for caching data content of each of the data files and storing metadata of each of the data files.

In a sixth aspect, an embodiment of the present invention provides a data writing apparatus for RAID, which is applied to a preset RAID driver, and the apparatus includes:

the data reading module is used for reading the data to be stored when detecting that the data volume of the data to be stored is not less than a first data volume, wherein the data volume is the first data volume and is not written into the target RAID, and the data is used as a data file to be written; wherein the target RAID is created based on the method of any one of claims 1 to 5; the first data volume is the data volume of each data file stored by the target RAID;

the data caching module is used for writing the data file to be written into a first cache preset in a second array of the target RAID;

and the data distribution module is used for distributing the data content of the data file to be written cached in the first cache for each writing area included by the specified stripe in the first array of the target RAID, so that the data content distributed to the writing area is written into the writing area by the SMR to which each writing area belongs.

In a seventh aspect, an embodiment of the present invention provides a RAID recovery apparatus, which is applied to a preset RAID driver, and the apparatus includes:

the pointer determining module is used for determining a minimum write pointer and a maximum write pointer from write pointers in SMR areas of a plurality of SMRs in a target RAID when the target RAID is restarted due to abnormal power failure; wherein the target RAID is created based on the creation method; the minimum write pointer is: the write pointer with the minimum corresponding storage address in write pointers in SMR areas of a plurality of SMRs in the target RAID; the maximum write pointer is: the write pointer with the largest corresponding storage address in write pointers in SMR areas of a plurality of SMRs in the target RAID;

a valid data determining module, configured to determine, as valid data in the first array of the target RAID, data stored in each stripe located before the stripe where the minimum write pointer is located in the first array of the target RAID;

a pointer moving module for moving a write pointer in an SMR area of the plurality of SMRs to a start position of a write area belonging to a target stripe; wherein the target stripe is: a stripe in the first array of the target RAID that is located after the stripe where the largest write pointer is located.

In an eighth aspect, an embodiment of the present invention provides an electronic device, which is provided with a RAID driver, and includes a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface complete communication between the memory and the processor through the communication bus;

a memory for storing a computer program;

and a processor for implementing the steps of any of the above methods when executing the program stored in the memory.

In a ninth aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of any one of the above methods.

In a tenth aspect, embodiments of the present invention also provide a computer program product containing instructions, which when run on a computer, cause the computer to perform any of the methods described above.

The embodiment of the invention has the following beneficial effects:

as can be seen from the above, when creating RAID using SMRs, a preset RAID driver may first determine a plurality of SMRs used for creating RAID, and may further configure SMR areas of the plurality of SMRs as a first array of a target RAID to be created, where the first array includes a plurality of stripes, each stripe includes a write area with the same offset address in the SMR areas of the plurality of SMRs, and each stripe is used for storing a data file written in an aligned sequential writing manner. In addition, the CMR area of the plurality of SMRs may be configured as a second array of the target RAID to be created.

In this way, the created target RAID may be considered as one large SMR integrated by a plurality of SMRs, wherein the second array of the target RAID corresponds to the CMR area of the large SMR, the first array of the target RAID corresponds to the SMR area of the large SMR, and further, each stripe in the first array of the target RAID corresponds to each write area in the SMR area of the large SMR. Writing a data file to the target RAID is equivalent to writing a data file to a single large SMR. When the data file is written into the single SMR, other hard disks do not need to be additionally used as cache, so that when the data file is written into the created target RAID, other hard disks do not need to be additionally used as cache.

Based on the above, by applying the method for creating the RAID based on the SMR provided in the embodiment of the present invention, RAID creation using the SMR can be realized without using another hard disk as a cache.

In addition, the data writing method for the RAID provided by the embodiment of the present invention can implement data writing for the RAID created by the present scheme; the RAID recovery method provided by the embodiment of the invention can realize the data recovery of the RAID created by the scheme.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by referring to these drawings.

FIG. 1 is a schematic diagram of the structure of an SMR;

fig. 2 is a schematic flowchart of a RAID creation method based on SMR according to an embodiment of the present invention;

fig. 3 is a schematic structural diagram of a target RAID created by applying a RAID creation method based on SMR according to an embodiment of the present invention;

FIG. 4 is a schematic flowchart of another SMR-based RAID creation method according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of a target RAID created by applying a method for creating a RAID based on an SMR according to an embodiment of the present invention;

fig. 6 is a flowchart illustrating a data writing method for RAID according to an embodiment of the present invention;

fig. 7 is a flowchart illustrating another data writing method for RAID according to an embodiment of the present invention;

fig. 8 is a schematic flowchart of a RAID recovery method according to an embodiment of the present invention;

FIG. 9 is a diagram illustrating write pointers in SMR regions of SMRs included during a target RAID abnormal power outage reboot;

fig. 10 is a schematic diagram of write pointers in SMR areas of SMRs included in the target RAID after the target RAID shown in fig. 9 is recovered by using a RAID recovery method according to an embodiment of the present invention;

fig. 11 is a schematic structural diagram of a RAID creation apparatus based on SMR according to an embodiment of the present invention;

fig. 12 is a schematic structural diagram of a data writing apparatus for RAID according to an embodiment of the present invention;

fig. 13 is a schematic structural diagram of a RAID recovery apparatus according to an embodiment of the present invention;

fig. 14 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived from the embodiments given herein by one of ordinary skill in the art, are within the scope of the invention.

Considering that each write area in the SMR area of the SMR only supports sequential write, but not out-of-order write, when RAID created by the SMR is used, data storage can be achieved by additionally using the SSD as a cache. Based on this, how to create RAID by using SMR without using other hard disks as cache becomes a technical problem to be solved at present.

In order to solve the above technical problem, an embodiment of the present invention provides a RAID creation method based on SMR.

The creating method is applicable to any application scenario that needs to create RAID using SMR, for example, a video monitoring scenario, but is not limited to this.

Also, the creation method is applied to RAID drives. In a specific application, the RAID driver for executing the creating method may be a RAID driver preset in various electronic devices such as a notebook computer and a desktop computer, and certainly, the RAID driver for executing the creating method may also be a RAID driver preset in a device cluster.

The method for creating a RAID based on SMR according to an embodiment of the present invention may include the following steps:

determining a plurality of SMRs for creating RAID;

configuring an SMR region of the plurality of SMRs as a first array of a target RAID to be created (RAID-S); wherein the first array includes a plurality of stripes, each stripe (stripe) includes a write area of the same offset address in an SMR area of the plurality of SMRs, and each stripe (stripe) is to store a data file written in an aligned sequential write manner;

configuring the CMR region of the plurality of SMRs as a second array of the target RAID (RAID-C).

Before specifically describing a method for creating a RAID based on an SMR according to an embodiment of the present invention, first, the SMR is briefly described.

SMRs adjust the way data is stored in the SMRs by overlapping tracks, and further, the track density of the SMRs is increased, thereby increasing the storage capacity of a single SMR. Among them, SMR provides a magnetic disk data storage technology, by which data tracks on a disk can be partially overlapped in a manner similar to the stacking of tiles on a roof, and thus, SMR can be called a shingled magnetic recording technology, and thus, the storage density of a magnetic disk can be greatly improved by applying SMR.

However, due to the special storage structure, the SMR does not support random writing and in-place updating of data, and thus, the data storage operability of the SMR is poor and the data storage efficiency is low.

The SMR divides the track into bands, i.e., into continuously writable areas each of which is formed of a continuous track, and each of the continuously writable areas forms a basic unit to be sequentially written. Band is a physical concept of an SMR, and its corresponding logical concept is "zone (zone)", which is a sequential write area in an SMR that can be seen by upper layer software that can operate the SMR.

Typically, the spatial size of one zone in an SMR is on the order of 256 MB. Among them, since the SMR needs to ensure that data can be read from the non-overlapped portion of the track, data reading in each zone can be performed at random. Also, the SMR is area-divided at the time of production and manufacture, so that the start position of each zone can be identified by leaving a large space between each zone. In addition, a hard disk manufacturer can set a small block area on the SMR and use a non-overlapping track organization manner of a traditional hard disk in the small block area to trade the random writing capability in the small block area at the expense of partial storage density. The small block area may be referred to as a "legacy Zone" so that a foothold may be provided for random writing operations of an upper layer application capable of operating the SMR.

FIG. 1 is a schematic diagram of the structure of an SMR, wherein, as shown in FIG. 1, the following characteristics of the SMR can be obtained:

SMR takes a zone as a management unit, and the space size of each zone is generally 256 MB; each zone in the SMR region of the SMR supports out-of-order write and read operations, but only sequential write operations, i.e., data can only be written sequentially, from front to back, in order; SMRs include an SMR region and a CMR (Conventional Magnetic Recording disk) region, the CMR region has a small spatial size and supports out-of-order write operations; reading and writing can be carried out only after each zone in the SMR is opened, and the number of opened zones is limited; the SMR supports ZAC/ZBC commands to acquire, read, write and clear zone states.

The following describes a method for creating a RAID based on SMR according to an embodiment of the present invention with reference to the accompanying drawings.

Fig. 2 is a schematic flowchart of a method for creating a RAID based on SMR according to an embodiment of the present invention, and as shown in fig. 2, the creating method may include the following steps S201 to S203:

s201: determining a plurality of SMRs for creating RAID;

when creating RAID using SMRs, a plurality of SMRs may be associated with an electronic device in which a RAID drive is located, so that the RAID drive can detect the SMRs that are currently present, and further, the RAID drive can determine the plurality of SMRs used to create RAID among the detected SMRs.

Optionally, in a specific implementation manner, a user may send hard disk identifiers of multiple selected SMRs for creating RAID to the RAID driver by inputting an identifier, clicking an identifier, and the like, so that the RAID driver may determine, as an SMR for creating RAID, an SMR having the received hard disk identifier in the detected SMRs.

Alternatively, in a specific implementation, the RAID driver may determine all of the detected SMRs as SMRs used to create the RAID.

Alternatively, in another specific implementation manner, the RAID driver may obtain configuration information about creating the RAID, and thus, according to the configuration information, the SMR used to create the RAID is randomly determined from the detected SMRs.

The specific implementation manner of step S101 is given as an example only, and should not be construed as a limitation to the embodiments of the present invention, and any manner capable of implementing step S101 is within the scope of the present invention.

S202: configuring SMR areas of a plurality of SMRs as a first array of a target RAID to be created;

wherein the first array comprises a plurality of stripes (stripes), each stripe (stripe) comprises a writing area (zone) with the same offset address in an SMR area of a plurality of SMRs, and each stripe (stripe) is used for storing data files written in an aligned sequential writing manner;

after determining the SMRs for creating the RAID, the RAID drive may configure SMR areas of the SMRs as a first array of a target RAID to be created (RAID-S).

The data content corresponding to each data file written into the target RAID is uniformly stored in a writing area (zone) in SMR areas of the plurality of SMRs, each writing area (zone) does not support disorder writing, and each writing area (zone) does not support simultaneous existence of two writing operations, so that each writing area (zone) can store the data content corresponding to one data file, and therefore, in order to enable write pointers in the SMR areas of the plurality of SMRs to be consistent, when a first array (RAID-S) of the target RAID is created, the writing areas (zones) with the same offset address in the SMR areas of the plurality of SMRs are used for storing the data content corresponding to the same data file.

Based on this, in the process of configuring the first array, the RAID driver may determine that, of the SMR areas of the SMRs for creating the RAID, a write area (zone) having the same offset address is located in the SMR areas of different SMRs, and the write area having the same offset address is configured as one stripe (stripe) in the first array of the target RAID. In this way, after all the write areas in the SMR areas of the SMRs for creating the RAID are traversed and all the stripes are configured, the configuration of the first array (RAID-S) of the target RAID can be completed.

In this way, the first array (RAID-S) of the configured target RAID may include a plurality of stripes (stripes), each stripe (stripe) including a write area (zone) having the same offset address in the SMR area of the plurality of SMRs used to create the RAID.

Furthermore, since the SMR area of the SMR does not support the out-of-order write operation, when writing a data file to each write area (zone) in the SMR area, the write pointer must be sequentially moved from the start position of the write area (zone) to the end position of the write area (zone), and therefore, the first array (RAID-S) of the target RAID configured similarly does not support the out-of-order write operation. In this way, when writing a data file to each stripe (stripe) in the first array (RAID-S) of the configured target RAID, the writing method employed is: aligned sequential writing.

That is, when a data file is written to each stripe (stripe) in the first array (RAID-S) of the configured target RAID, the write pointers in the respective write areas (zones) included in the stripe are identical, and in each write area (zone) included in the stripe (stripe), the write pointers must be sequentially moved from the start position of the write area (zone) to the end position of the write area (zone).

Based on this, each stripe (stripe) in the configured first array (RAID-S) of the target RAID may be used to store data files written in an aligned sequential writing manner.

S203: configuring a CMR region of the plurality of SMRs as a second array of the target RAID;

after determining the SMRs for creating the RAID, the RAID driver may also configure the CMR area of the SMRs as a second array of the target RAID to be created (RAID-C).

Since the CMR area of the SMR supports out-of-order write operations, the configured second array of the target RAID (RAID-C) can also support out-of-order write operations.

Optionally, a configured second array of target RAIDs (RAID-C) may store metadata for each data file.

Since the storage area for storing the metadata of the data file is usually operated by out-of-order write operation when the metadata of the data file stored in the first array of the configured target RAID is changed or added, the first array (RAID-S) of the configured target RAID is used for storing the data file, and further, the second array (RAID-C) of the configured target RAID can be used for storing the metadata of the data file stored in the first array (RAID-S).

Optionally, a configured second array of target RAIDs (RAID-C) may be used to cache the data content of each data file.

When writing a data file to the configured first array (RAID-S) of the target RAID, the data file needs to be written into the cache associated with the target RAID first, and when writing a data file to the cache, the cache may be generally operated by out-of-order write operation, so when the configured second array (RAID-C) of the target RAID supports out-of-order write operation, the configured second array (RAID-C) of the target RAID may be used to cache the data content of the data file written to the configured first array (RAID-S) of the target RAID.

The execution sequence of steps S202 and S203 is not limited in the embodiment of the present invention.

For example, fig. 3 is a schematic structural diagram of a target RAID created by applying a method for creating a RAID based on SMR according to an embodiment of the present invention.

As shown in fig. 3, the columns corresponding to SMR1-SMR3 are SMRs for creating a target RAID, the RAID-C area is the second array of the target RAID, the RAID-S area is the first array of the target RAID, and the column corresponding to RAID is the target RAID created using SMR1-SMR 3.

Wherein, Date _ File-1 to Date _ File-N are respectively stripes (stripes) in the first array of the target RAID. Note that Date _ File-1 is composed of three write areas Zone-1 having the same offset address and located in SMRs 1-SMR3, Date _ File-2 is composed of three write areas Zone-2 having the same offset address and located in SMRs 1-SMR3, and so on, and Date _ File-N is composed of three write areas Zone-N having the same offset address and located in SMRs 1-SMR 3.

According to the schematic structural diagram of the target RAID shown in fig. 3, the target RAID created by applying the SMR-based RAID creation method according to the embodiment of the present invention may be regarded as a large SMR integrated by a plurality of SMRs, where the second array of the target RAID (RAID-C) corresponds to a CMR area of the large SMR, the first array of the target RAID (RAID-S) corresponds to an SMR area of the large SMR, and each stripe (stripe) in the first array of the target RAID corresponds to each write area (zone) in the SMR area of the large SMR.

That is, when building a target RAID with multiple SMRs, each SMR may be divided into two block devices, where one block device is an SMR area of the SMR and the other block device is a CMR area of the SMR. In this way, a first array (RAID-S) and a second array (RAID-C) of the target RAID may be constructed using the partitioned block devices in each SMR.

In this way, when writing a data file into a single SMR, it is not necessary to additionally use another hard disk as a cache, and therefore, after the target RAID is created by applying the method for creating a RAID based on an SMR according to the embodiment of the present invention, writing a data file into the target RAID is equivalent to writing a data file into a single large SMR, and therefore, it is also not necessary to additionally use another hard disk as a cache.

When storing a data file using each stripe (stripe) in a first array (RAID-S) of a target RAID, in order to improve data security so that data recovery can be performed more quickly and efficiently in the event of a loss of part of data in the data file, a check value of the data file may be stored generally at a location corresponding to a storage location of the data file.

That is, for each stripe (stripe) in the first array (RAID-S) of the target RAID, one write area (zone) for storing the check value may be selected from among the respective write areas (zones) included in the stripe (stripe), so that, when a data file is written to the stripe (stripe), the data file may be written to the respective write areas (zones) included in the stripe (stripe) except for the write area (zone) for storing the check value, and the calculated check value of the data file may be written to the selected write area (zone) for storing the check value.

Based on this, optionally, in a specific implementation manner, as shown in fig. 4, the method for creating a RAID based on an SMR according to an embodiment of the present invention may further include the following step S200.

S200: and setting a plurality of SMRs as a first type of hard disk and at least one second type of hard disk.

Accordingly, each strip (stripe) comprises: the hard disk comprises a first writing area (zone) belonging to a first type of hard disk and at least one second writing area (zone) respectively belonging to each second type of hard disk, wherein the first writing area (zone) is used for storing a check value of a data file, and each second writing area (zone) is used for storing partial data content of the data file.

In this specific implementation manner, after determining the SMRs for creating RAID, the RAID driver may further set the SMRs to one first type hard disk and at least one second type hard disk.

Alternatively, the RAID driver may set any one of the plurality of SMRs as a first type of hard disk at random, and set the other ones of the plurality of SMRs as a second type of hard disk.

Optionally, the RAID driver may set, according to a preset rule, an SMR in the plurality of SMRs that meets the preset rule as a first type of hard disk, and set another SMR in the plurality of SMRs as a second type of hard disk.

For example, the preset rule may be: setting the SMR with the smallest hard disk identification in the plurality of SMRs as a first type of hard disk; the following steps can be also included: and setting the SMR detected firstly in the plurality of SMRs as the first type of hard disk. This is all reasonable.

The above example of the preset rule is merely used to exemplify a manner of setting the first type hard disk and the second type hard disk for the RAID drive, and is not limited, and any manner capable of implementing the step S100 is within the protection scope of the present invention.

In this way, since each stripe (stripe) in the first array (RAID-S) of the target RAID includes a write area (zone) having the same offset address in the SMR area of the SMRs used to create the target RAID, each stripe (stripe) includes one first write area (zone) belonging to the first type of hard disk and at least one second write area (zone) respectively belonging to the respective second type of hard disk. Wherein, when writing a data file to each stripe (stripe) in a first array (RAID-S) of a target RAID, the data file is written to each second writing area (zone) included in the stripe (stripe), and a check value of the data file is written to a first writing area (zone) included in the stripe (stripe).

Optionally, the check value of the data file is calculated according to an exclusive or operation of data contents belonging to the data file written in each second writing area (zone) included in the stripe (stripe). The embodiment of the invention does not limit the specific calculation mode of the check value.

In the specific implementation manner, the RAID can be created by using the SMR without using other hard disks as the cache; moreover, by setting the plurality of SMRs as a first type hard disk and at least one second type hard disk, each stripe (stripe) can include a first write area (zone) belonging to the first type hard disk and at least one second write area (zone) respectively belonging to each second type hard disk, thereby realizing storage of both data content of the data file and a check value of the data file, and ensuring data security.

Since the second array of target RAIDs (RAID-C) may be used to store metadata for each data file stored by the first array of target RAIDs (RAID-S), a target area for storing metadata for a data file may be determined in the second array of target RAIDs (RAID-C).

Based on this, optionally, in a specific implementation manner, on the basis of the specific implementation manners shown in fig. 2 and fig. 4, the method for creating a RAID based on an SMR according to an embodiment of the present invention may further include the following steps 11 to 12.

Step 11: determining a first number of all data files which can be stored by the target RAID based on the space size of the first array and the first data volume of each data file;

step 12: determining a target region in the second array based on the first number;

wherein the target area is for each metadata of the stored data file.

Since the data files are stored in the first array (RAID-S) of the target RAID, the spatial size of the first array (RAID-S) of the target RAID can be understood as the spatial size of the storage area for storing the data files, so that after the first data amount of each data file is obtained, the first number of all the data files that can be stored by the target RAID can be determined based on the spatial size of the first array (RAID-S) of the target RAID and the first data amount.

The determining method of the first data size of each data file may be various, and for example, the first data size of each data file may be obtained through configuration information; the first data amount of each data file may also be determined according to the second number of the SMRs used to create the target RAID and a space size of each write area (zone) in the SMR areas of the SMRs, and a specific determination manner will be described later for clarity of the scheme and clarity of the layout.

Optionally, the first number may be: a ratio of a spatial size of a first array (RAID-S) of the target RAID to the first amount of data.

Since the second array (RAID-C) of the target RAID may be used to store the metadata of the data file, for each data file stored in the first array (RAID-S) of the target RAID, the metadata of the data file may be stored in the second array (RAID-C) of the target RAID, so that the number of metadata of the data file stored in the second array (RAID-C) of the target RAID is the same as the number of data files stored in the first array (RAID-S) of the target RAID.

Further, since the maximum number of data files that can be stored in the first array (RAID-S) of the target RAID is the determined first number, the maximum number of data files stored in the first array (RAID-S) of the target RAID is also the determined first number. Based on this, the target area for storing metadata of the data files determined in the second array (RAID-C) of the target RAID needs to be able to store the metadata of the first number of data files determined as described above.

In this way, a target area for storing metadata for the data file may be determined in a second array of target RAIDs (RAID-C) based on the determined first number.

For example, fig. 5 is a schematic structural diagram of a target RAID created by applying a method for creating a RAID based on SMR according to an embodiment of the present invention.

As shown in fig. 5, the file index area included in RAID-C is a target area determined in the second array of the target RAID for storing metadata of data files, and the user metadata area included in RAID-C may be used for storing other data that may be written out of order or modified out of order.

Optionally, as shown in fig. 5, the user metadata area included in RAID-C may also be used to cache the data content of the data file written to the first array (RAID-S) of the target RAID.

Optionally, in a specific implementation manner, the step 12 of determining the target area in the second array based on the first number may include the following steps 121 and 122.

Step 121: determining a product of the first amount and a preset second data amount of each metadata;

step 122: a region of the second array is determined as the target region, the region having a spatial size that is a product of the spatial sizes.

In this embodiment, since the metadata of the data files is preset in a fixed format, the data size of the metadata of each data file is the same, and therefore, the size of the space of the target area capable of storing the metadata of the first number of data files determined as above is: the product of the determined first amount and a preset second amount of metadata for each data file.

In this way, the product of the determined first amount and the preset second amount of data of the metadata of each data file may be calculated first, so that an area having a space size of the above product, which is a target area for storing the metadata of the data file, may be determined in the second array (RAID-C) of the target RAID.

Since the write operations supported by the respective stripes (stripes) in the first array (RAID-S) of the target RAID are aligned sequential write operations, it is necessary to prevent multiple write operations to respective write areas in the same stripe (stripe) from occurring at the same time. Based on this, since one write operation for each stripe (stripe) in the first array (RAID-S) of the target RAID is used to instruct to store one data file, the written data content should occupy the storage space of one stripe (stripe) each time one write operation is performed to complete the writing of one data file.

Based on this, optionally, in a specific implementation, since the number of write areas (zones) included in each stripe (stripe) in the first array (RAID-S) of the target RAID is the same as the second number of the plurality of SMRs used to create the target RAID, the size of the data file written to each stripe (stripe) in the first array (RAID-S) of the target RAID is: the second number is multiplied by the spatial size of each write area (zone) in the SMR area.

Optionally, in another specific implementation manner, when a plurality of SMRs used for creating the target RAID are set as one first type of hard disk and at least one second type of hard disk, and when a data file is written to each stripe (stripe) in the first array (RAID-S) of the target RAID, the number of write areas (zones) used for storing data content of the data file in the stripe (stripe) is a difference obtained by subtracting 1 from the second number of the plurality of SMRs used for creating the target RAID, so that the size of the data file written to each stripe (stripe) in the first array (RAID-S) of the target RAID is: the difference is multiplied by the space size of each writing area (zone) in the SMR area, and in order to ensure that when writing of one data file is completed, the written data content should occupy the storage space of one stripe (stripe), the check value of the data file stored in the first writing area (zone) in each stripe (stripe) should occupy the entire storage space of the first writing area (zone), and therefore, the data amount of the check value of the data file stored in the first writing area (zone) in each stripe (stripe) is the space size of each writing area (zone) in the SMR area.

Based on this, in the present specific implementation, the determination manner of the first data amount of each data file may include the following steps.

Calculating a first data volume of each data file according to a preset formula; in fact, the preset formula is:

D＝S*(N-1)

where D is the first amount of data, S is a space size of each write area (zone) in an SMR area of the SMR, and N is a second number of SMRs to create the target RAID.

Accordingly, in the present embodiment, among all the write areas (zones) of the first array (RAID-S) of the target RAID, only the write area (zone) belonging to the second type of hard disk is used for storing the data file, and the write area (zone) belonging to the first type of hard disk is used for storing only the check value of the data file, so that when the first number of all the data files that can be stored by the target RAID is determined, the determination cannot be performed directly using the space size of the first array (RAID-S) of the target RAID.

Based on this, in this specific implementation manner, the step 11 of determining, based on the spatial size of the first array and the first data amount of each data file, the first number of all data files that can be stored by the target RAID may include the following steps:

determining a first number of total data files that can be stored by a target RAID based on a specified space size in a first array (RAID-S) and a first amount of data for each data file;

wherein the specified space size is a space size of a writing area (zone) belonging to the second type of hard disk.

In this embodiment, since the data file is stored in the write area belonging to the second type of hard disk in the first array (RAID-S) of the target RAID, the space size of the write area (zone) belonging to the second type of hard disk in the first array (RAID-S) of the target RAID can be understood as the space size of the write area (zone) for storing the data file, so that the first number of all data files that can be stored by the target RAID can be determined based on the space size, that is, the designated space size, of the write area (zone) belonging to the second type of hard disk in the first array (RAID-S) of the target RAID and the first data amount.

Optionally, the first number may be: the ratio of the space size of a write area (zone) belonging to the second type of hard disk in the first array (RAID-S) of the target RAID to the first data amount.

Corresponding to the method for creating a RAID based on an SMR provided in the embodiment of the present invention, the embodiment of the present invention further includes a RAID created based on any one of the methods for creating a RAID based on an SMR provided in the embodiment of the present invention.

That is to say, an embodiment of the present invention further provides a RAID, and the RAID is created based on any one of the SMR-based RAID creation methods provided in the above embodiments of the present invention.

The RAID comprises a first array (RAID-S) and a second array (RAID-C), wherein the first array (RAID-S) is configured by SMR areas in SMRs in the RAID, the first array comprises a plurality of stripes (stripes), each stripe (stripe) comprises a writing area (zone) with the same offset address in the SMR areas of the SMRs, and each stripe (stripe) is used for storing data files written in an aligned sequence writing mode; the second array (RAID-C) is configured from the CMR region in each SMR in the RAID.

In addition, the RAID supports any writing method for RAID and any RAID recovery method provided by the following embodiments of the present invention.

The specific contents of a RAID provided in the embodiment of the present invention are the same as the specific contents of a RAID created in the method for creating a RAID based on an SMR provided in the embodiment of the present invention, and are not described herein again.

Corresponding to the above specific implementation manners, an embodiment of the present invention further provides a data writing method for RAID.

The data writing method is applicable to any application scenario that requires data files to be stored in a target RAID, for example, a video monitoring scenario, but is not limited to this. And, the target RAID is created based on any one of the SMR-based RAID creation methods provided in the embodiments of the present invention described above.

In addition, the data writing method is applied to RAID driving. In a specific application, the RAID driver for executing the data writing method may be a RAID driver preset in various electronic devices such as a notebook computer and a desktop computer, and certainly, the RAID driver for executing the data writing method may also be a RAID driver preset in an equipment cluster.

The execution subject of the data writing method may be the same as or different from the execution subject of the SMR-based RAID creation method according to the embodiment of the present invention.

Fig. 6 is a flowchart illustrating a data writing method for RAID according to an embodiment of the present invention, and as shown in fig. 6, the data writing method may include the following steps S601 to S603.

S601: when detecting that the data volume of the data to be stored is not less than the first data volume, reading the data which is not written into the target RAID and has the first data volume in the data to be stored as a data file to be written;

the target RAID is created based on any SMR-based RAID creation method provided by the embodiment of the invention; the first data volume is the data volume of each data file stored by the target RAID;

according to the above description of the SMR-based RAID creation method according to the embodiment of the present invention, a write operation for each stripe (stripe) in the first array (RAID-S) of the target RAID is used to instruct to store one data file, so that each time a write operation is performed to complete writing of one data file, the written data content should occupy the storage space of one stripe (stripe).

Based on this, when data is written to the target RAID, the size of the data file to be written, which is read each time, is the data volume of each data file stored by the target RAID, which is determined when the target RAID is created, that is, the first data volume.

Therefore, when the data volume of the data to be stored is detected to be not less than the first data volume, the data which is not written into the target RAID and is the first data volume in the data to be stored can be read, and the data file to be written is obtained. The data to be stored is data to be written into the target RAID.

S602: writing a data file to be written into a first cache preset in a second array (RAID-C) of a target RAID;

since the second array (RAID-C) of the target RAID can cache the data file written to each stripe (stripe) in the first queue of the target RAID, when the data file to be written is obtained, the data file to be written can be written to the first cache preset in the second array (RAID-C) of the target RAID.

S603: and allocating the data content of the data file to be written cached in the first cache for each writing area (zone) included by a designated stripe (stripe) in a first array (RAID-S) of the target RAID, so that the SMR of each writing area (zone) is enabled to write the data content allocated to the writing area (zone) into the writing area (zone).

The data file to be written is a data file to be written into a stripe (stripe) in a first array (RAID-S) of a target RAID, and after the data file to be written is written into a first cache preset in a second array (RAID-C) of the target RAID, a specific stripe (stripe) for storing the data file to be written may be determined in each stripe (stripe) in the first array (RAID-S) of the target RAID.

The designated stripe (stripe) is a stripe (stripe) to which a write area (zone) in which a write pointer in an SMR area of each SMR in the target RAID belongs. The write operation supported by each stripe (stripe) in the first array (RAID-S) of the target RAID is aligned sequence write, so that write pointers in SMR areas of SMRs in the target RAID are consistent, and write areas (zones) where the write pointers in the SMR areas of the SMRs in the target RAID belong to the same stripe (stripe) and are located at the starting position of the write area (zone).

After the designated stripe (stripe) for storing the data file to be written is determined, the data content of the data file to be written cached in the first cache may be allocated to each writing area (zone) included in the designated stripe (stripe). The capacity of each writing area (zone) included in the designated stripe (stripe) is the same, and data is written into each writing area (zone) included in the designated stripe (stripe) in an aligned writing mode, so that the data amount of the data content of the file to be data cached in the first cache allocated to each writing area (zone) included in the designated stripe (stripe) is the same.

In this way, for each write area (zone) included in the designated stripe (stripe), after allocating the data content of the file to be data cached in the first cache to the write area (zone), the SMR to which the write area (zone) belongs may write the data content allocated to the write area (zone) into the write area (zone). After the data content allocated to the write area (zone) is completely written in the write area (zone), the write pointer in the SMR area of the SMR to which the write area (zone) belongs is moved to the end position of the write area (zone).

Further, when a new data file to be written is acquired again, the next stripe (stripe) of the designated stripe (stripe) is a new designated stripe (stripe) for storing the new data file to be written.

Based on this, by applying the scheme provided by the embodiment of the present invention, it is possible to write data to each stripe (stripe) in the first array (RAID-S) of the target RAID created by any one of the SMR-based RAID creation methods provided by the above embodiments of the present invention, thereby implementing data storage to the RAID created by the SMR without additionally using another hard disk as a cache. Moreover, since the data volume of the data file to be written stored each time is the data volume of each data file stored by the target RAID determined when the target RAID is created, the stored data content can occupy the storage space of the specified stripe (stripe) after the data writing of each time is completed, and therefore, it is possible to avoid the simultaneous existence of a plurality of write operations for each write area (zone) in the same stripe (stripe).

Optionally, in a specific implementation, each stripe (stripe) of the first array (RAID-S) includes: a first writing area (zone) and at least one second writing area (zone), wherein the first writing area (zone) is used for storing the check value of the data file, and each second writing area (zone) is used for storing partial data content of the data file;

accordingly, in this specific implementation manner, in the step S603, allocating, for each write area (zone) included in the designated stripe (stripe) in the first array (RAID-S) of the target RAID, the data content of the to-be-written data file cached in the first cache may include the following steps 21 and 22.

Step 21: distributing the data content of the data file to be written cached in the first cache for each second writing area (zone) included by a designated stripe (stripe) in a first array (RAID-S) of a target RAID;

step 22: distributing a check value to be written of a data file to be written for a first writing area (zone) included by a designated stripe (stripe);

wherein the check value to be written is calculated on the basis of the data content allocated for the respective second writing area (zone).

In this particular implementation, the plurality of SMRs in the target RAID may be set to one first type of hard disk and at least one second type of hard disk, such that each stripe (stripe) in the first array (RAID-S) of the target RAID includes one first write area (zone) belonging to the first type of hard disk and at least one second write area (zone) belonging to the second type of hard disk.

Wherein the respective second writing area (zone) included in each stripe (stripe) in the first array (RAID-S) of the target RAID is used for writing the data file, and the first writing area (zone) included in each stripe (stripe) in the first array (RAID-S) of the target RAID is used for writing the check value of the data file written by the respective second writing area (zone) in the stripe (stripe).

Based on this, after the data file to be written is written into a preset first cache in a second array (RAID-C) of a target RAID and a specified stripe (stripe) for storing the data file to be written is determined, the data content of the data file to be written cached in the first cache may be allocated to each second writing area (zone) included in the stripe (stripe). Furthermore, a check value of the data file to be written may be determined based on the data content assigned to each second writing area (zone), and the check value may be assigned to the first writing area (zone) in the designated stripe (stripe).

Alternatively, the check value of the data file to be written may be determined by the exclusive or calculation of the data content allocated to each second writing area (zone).

The second array (RAID-C) of the target RAID is configured to store, in addition to the data file written to each stripe (stripe) in the first array (RAID-S) of the target RAID, at least metadata of the data file written to each stripe (stripe) in the first array (RAID-S) of the target RAID, so that after the data file to be written is completely written, the metadata of the data file to be written may be further added to the second array (RAID-C) of the target RAID.

Based on this, optionally, in a specific implementation manner, on the basis of the above specific implementation manners, the data writing method for RAID provided in the embodiment of the present invention may further include the following steps:

in a target area included in a second array (RAID-C) of the target RAID, metadata of a data file to be written is added.

In this specific implementation manner, after the data file to be written is written, the metadata of the data file to be written may be further added to a target area included in a second array (RAID-C) of the target RAID. Wherein the target area is an area determined in a second array of target RAID (RAID-C) for storing metadata of the data file.

It is understood that in the monitoring technology field, the monitoring video data collected by the monitoring camera can be written into each stripe (stripe) in the second array (RAID-C) of the target RAID for storage. However, since the video data collected by the surveillance camera is continuous and has a large data volume, and the data volume of the data file stored in each stripe (stripe) in the second array (RAID-C) of the target RAID is fixed, there may be a case where the video surveillance data collected by the surveillance camera is not as much as being written into each stripe (stripe) in the second array (RAID-C) of the target RAID and needs to be cached.

Based on this, in a specific implementation manner, as shown in fig. 7, the data writing method for RAID according to the embodiment of the present invention may further include the following step S600.

S600: acquiring target data and writing the target data into a second cache in the memory;

the space size of the second cache is as follows: an integer multiple of the first amount of data;

accordingly, in this specific implementation manner, in the step S601, when it is detected that the data amount of the data to be stored is not less than the first data amount, reading the data, which is not written in the target RAID and has the first data amount, in the data to be stored, as the data file to be written, the following step S601A may be included.

S601A: and when the data volume of the target data in the second cache is not less than the first data volume, reading the data which is not written into the target RAID and is in the first data volume in the second cache as a data file to be written.

In this specific implementation manner, a second cache whose space size is an integral multiple of the first data size may be applied in the memory in advance.

The memory for applying for the second cache is: the memory of the electronic device where the RAID driver is used to execute the data writing method for RAID provided by the embodiment of the present invention.

In this way, the target data to be written into each stripe (stripe) in the first array (RAID-S) of the target RAID may be acquired, and the acquired target data may be written into the second cache as applied above.

The target data may be data acquired from another device, may also be data acquired by an electronic device in which a RAID driver for executing the data writing method for RAID according to the embodiment of the present invention is located, and may also be pre-stored data acquired by an electronic device in which a RAID driver for executing the data writing method for RAID according to the embodiment of the present invention is located from a storage device of the electronic device. This is reasonable, and the embodiments of the present invention are not particularly limited.

After the target data is written into the second cache, the data amount of the target data written into the second cache may be detected in real time, so that when the data amount of the target data written into the second cache is not less than the first data amount, the target data written into the second cache may be read from the second cache, where the data amount is the first data amount and is not written into the target RAID, and the read data is a data file to be written.

Alternatively, the target data written in the second cache may be read from the second cache in an order from early to late of the time when the data is written in the second cache, where the data amount is the first data amount and is not written in the target RAID, and the data is used as the data to be written.

In this way, the storage order of the target data in each stripe (stripe) in the first array (RAID-S) of the target RAID can be ensured to be consistent with the acquisition order thereof.

Optionally, in a specific implementation manner, on the basis of the specific implementation manner shown in fig. 7, before the step S600, the data writing method for RAID provided in the embodiment of the present invention may further include the following steps 31 to 32:

step 31: determining the data volume of each data file stored by the target RAID as a first data volume;

step 32: and applying for a second cache in the memory according to the first data volume.

In this specific implementation manner, before data writing is performed on the target RAID, the data volume of each data file stored by the target RAID may be determined as the first data volume, so that according to the obtained first data volume, a second cache may be applied in the memory of the electronic device where the RAID driver for executing the data writing method for RAID provided in the embodiment of the present invention is located.

And the space size of the applied second cache is integral multiple of the obtained first data volume.

Alternatively, the first data amount of each data file stored by the target RAID may be directly obtained.

For example, the first data amount input by the user may be directly acquired, the first data amount may be directly requested from another device, or information related to the target RAID may be recorded in advance, and the first data amount may be read from the related information.

Alternatively, the second number of SMRs in the target RAID and the space size of each write area (zone) in the SMR area of the SMRs may be first determined, and the first data amount may be calculated using the determined second number and space size.

Optionally, the first data amount may be: the product of the determined second number and the size of the space.

Optionally, when multiple SMRs in the target RAID are configured as one first type of hard disk and at least one second hard disk, the manner of determining the first data amount may be: first, a difference obtained by subtracting 1 from the determined second amount is calculated, and then, a product of the difference and the determined space size is calculated, so that the calculated product is the first data amount.

Corresponding to the above specific implementation manners, an embodiment of the present invention further provides a RAID recovery method.

The recovery method is applicable to any application scenario in which a target RAID with abnormal power failure needs to be recovered, for example, a video monitoring scenario, but is not limited to this. And, the target RAID is created based on any one of the SMR-based RAID creation methods provided in the embodiments of the present invention described above.

In addition, the recovery method is applied to RAID driving. In a specific application, the RAID driver for executing the recovery method may be a RAID driver preset in various electronic devices such as a notebook computer and a desktop computer, and certainly, the RAID driver for executing the recovery method may also be a RAID driver preset in an equipment cluster.

It is reasonable that the execution subject of the recovery method, the execution subject of the SMR-based RAID creation method provided in the embodiment of the present invention, and the execution subject of the RAID-based data writing method provided in the embodiment of the present invention may be the same execution subject, may be completely different execution subjects, or may be partially the same execution subject.

Fig. 8 is a flowchart illustrating a RAID recovery method according to an embodiment of the present invention, and as shown in fig. 7, the RAID recovery method may include the following steps S801 to S804.

S801: when the target RAID is restarted after abnormal power failure, determining a minimum write pointer and a maximum write pointer from write pointers in SMR areas of a plurality of SMRs in the target RAID;

the target RAID is created based on any SMR-based RAID creation method provided by the embodiment of the invention; the minimum write pointer is: the write pointer with the minimum corresponding storage address in write pointers in SMR areas of a plurality of SMRs in the target RAID; the maximum write pointer is: the write pointer with the largest corresponding storage address in write pointers in SMR areas of a plurality of SMRs in the target RAID;

since each SMR in the target RAID independently performs an IO (Input/Output) operation, write pointers in SMR areas of the SMRs in the target RAID may be inconsistent after the target RAID is abnormally powered down and restarted.

For example, as shown in FIG. 9, the columns corresponding to ZONE-SMR1 through ZONE-SMR4 are the SMRs in the target RAID, respectively, and the write pointer in the SMR region of each SMR, i.e., the WP pointer in FIG. 9, is inconsistent.

In this way, the write mode supported by each stripe (stripe) in the first array (RAID-S) of the target RAID is: aligned sequential writes, therefore, it is necessary to ensure that the write pointers in the SMR areas of multiple SMRs in the target RAID are consistent. Therefore, when the target RAID is restarted after abnormal power failure, the target RAID needs to be restored such that the write pointers in the SMR areas of the SMRs in the restored target RAID are identical.

After the abnormal power failure restart of the target RAID, because the SMR area terminal write pointers of the SMRs in the target RAID may be inconsistent, the minimum write pointer and the maximum write pointer may be determined from the write pointers in the SMR areas of the SMRs in the target RAID.

Each write pointer corresponds to a storage address, each write pointer indicates that when data writing is carried out again, writing is started from the storage address corresponding to the write pointer, and based on the fact that the determined minimum write pointer is the write pointer with the minimum storage address in the write pointers in the SMR areas of the SMRs in the target RAID; correspondingly, the determined maximum write pointer is the write pointer with the largest corresponding storage address in the write pointers in the SMR areas of the SMRs in the target RAID.

The storage addresses are information for representing storage locations of data in the storage space, and generally, the storage addresses increase sequentially in an order from a start location of the storage space to an end location of the storage space. And for the write operation to the storage space, sequentially writing to a sequence pointing to the end position of the storage space from the storage address corresponding to the write pointer, and as the write operation progresses, continuously moving the write pointer to the sequence pointing to the end position of the storage space, that is, as the write operation progresses, continuously increasing the storage address corresponding to the write pointer until the write operation is completed, and moving the write pointer to a new position, that is, completing updating of the write pointer. And the storage address corresponding to the updated write pointer is larger than the storage address corresponding to the write pointer before updating.

In this way, by specifying the minimum write pointer and the maximum write pointer from among the write pointers in the SMR areas of the plurality of SMRs in the target RAID, it is possible to specify by determining the size of the storage location corresponding to the write pointer in the SMR area of the plurality of SMRs in the target RAID. Wherein, according to the above description of the write pointer and the storage address, the determined minimum write pointer is: among write pointers in an SMR area of a plurality of SMRs in the target RAID, the write pointer with the smallest distance from the starting position of the SMR area is located, and the determined largest write pointer is as follows: among write pointers in an SMR area of a plurality of SMRs in a target RAID, the write pointer having the largest distance from the starting position of the SMR area is located.

For example, as shown in FIG. 9, the minimum write pointer is the WP pointer in ZONE-SMR2 and the maximum write pointer is the WP pointer in ZONE-SMR 1.

S802: determining the data stored in each stripe (stripe) positioned before the stripe (stripe) with the minimum write pointer in the first array (RAID-S) of the target RAID as the valid data in the first array (RAID-S) of the target RAID;

after the minimum write pointer and the maximum write pointer are determined, the data stored in each stripe (stripe) in the first array (RAID-S) of the target RAID, which is located before the stripe (stripe) where the minimum write pointer is located, can be determined as valid data in the first array (RAID-S) of the target RAID.

The storage address corresponding to the starting position of the writing area (zone) included in each strip (stripe) before the strip (stripe) where the minimum writing pointer is located is smaller than the storage address corresponding to the starting position of the writing area (zone) where the minimum writing pointer is located. The above manner of determining valid data in the first array of the target RAID (RAID-S) may be referred to as: stripe (stripe) alignment based on the smallest write pointer rounds down.

For example, as shown in fig. 9, if the stripe (stripe) where the smallest write pointer is located is a stripe (stripe) M, the data stored in the stripe (stripe) 1-stripe (stripe) M-1 may be determined as valid data in the first array (RAID-S) of the target RAID.

S803: moving a write pointer in an SMR area of the SMRs to a start position of a write area (zone) belonging to a target stripe (stripe);

wherein, the target strip (stripe) is: a stripe (stripe) located after the stripe (stripe) where the largest write pointer is located in a first array (RAID-S) of the target RAID;

after determining the minimum write pointer and the maximum write pointer, determining a target stripe (stripe) in each stripe (stripe) in a first array (RAID-S) of a target RAID according to the maximum write pointer, where the target stripe (stripe) is: in the first array (RAID-S) of the target RAID, the stripe located after the stripe (stripe) where the maximum write pointer is located can be moved to the start position of the write area (zone) belonging to the target stripe (stripe) in the SMR areas of the SMRs of the target RAID.

Optionally, the target strip (stripe) may be: in the first array (RAID-S) of the target RAID, the stripe (stripe) located after the stripe (stripe) where the maximum write pointer is located is adjacent to the stripe (stripe) where the maximum write pointer is located.

In this way, when a data file is written again to each stripe (stripe) in the first array (RAID-S) of the target RAID, the written data file is written to the target stripe (stripe), that is, the position where the write pointer in the SMR area of each of the SMRs of the target RAID is moved is the data file write start position.

The storage address corresponding to the starting position of the writing area (zone) included in the stripe (stripe) located after the stripe (stripe) where the maximum writing pointer is located is larger than the storage address corresponding to the starting position of the writing area (zone) where the maximum writing pointer is located. When the target strip (stripe) is: in the first array (RAID-S) of the target RAID, when the stripe (stripe) located after the stripe (stripe) where the maximum write pointer is located and adjacent to the stripe (stripe) where the maximum write pointer is located, moving the write pointer in the SMR area of the plurality of SMRs to the start position of the write area (zone) belonging to the target stripe (stripe) may be referred to as: the stripe (stripe) alignment based on the largest write pointer is rounded up.

For example, as shown in fig. 9 to 10, if the stripe (stripe) where the maximum write pointer is located is the stripe (stripe) M +1, the stripe (stripe) M +2 may be determined as the target stripe (stripe), and thus, the write pointers in the SMR areas of the SMRs of the target RAID may be moved to the start positions of the respective write areas (zones) included in the stripe (stripe) M + 2.

The embodiment of the present invention does not limit the execution sequence of the steps S702 and S703.

Optionally, in a specific implementation manner, the RAID recovery method provided in the embodiment of the present invention may further include the following steps:

filling a designated area in each SMR with preset null data;

wherein, the designated area is: an area between the write pointer before the movement and the write pointer after the movement.

After the effective data determination and the write pointer movement are realized, an area where data is not stored in an area between the position where the write pointer is located before the movement and the position where the write pointer is located after the movement can be found, so that the area between the write pointer before the movement and the write pointer after the movement can be used as a designated area in each SMR in the target RAID, and the designated area in each SMR can be filled with preset null data.

For example, as shown in fig. 9-10, fig. 9 is a schematic diagram of write pointers in SMR areas of each SMR included when a target RAID is restarted under abnormal power outage; fig. 10 is a schematic diagram of write pointers in SMR areas of SMRs included in the target RAID after the target RAID shown in fig. 9 is restored by using a RAID restoration method according to an embodiment of the present invention.

As shown in fig. 10, at the time of restart of the target RAID abnormal power failure, the data stored in the stripe (stripe) M and the stripe (stripe) M +1 are determined as invalid data, and at the time of restart of the target RAID abnormal power failure, the area in the stripe (stripe) M and the stripe (stripe) M +1 in which data is not stored is filled with preset null data.

and recovering the data stored in the designated area in each SMR before the abnormal power failure of the target RAID.

In this specific implementation, when determining valid data in the first array (RAID-S) of the target RAID, the data already stored in the designated area of each SMR of the target RAID is determined to be invalid data, and thus, this portion of data is lost.

Optionally, before the abnormal power failure of the target RAID, the manner of recovering the data stored in the designated area in each SMR is the same as the manner of recovering the data in a single SMR.

Based on this, by applying the scheme provided by the embodiment of the invention, when the target RAID is restarted due to abnormal power failure and the write pointers in the SMR areas of the SMRs in the target RAID are inconsistent, the target RAID can be recovered to determine valid data in the target RAID and a data file write start position, so that it is ensured that the target RAID can continue to store data after the target RAID is restarted due to abnormal power failure.

In some cases, at the time of abnormal power outage reboot of the target RAID, the write pointers in the SMR areas of the SMRs in the target RAID may be consistent, and at this time, the target RAID may not need to be restored.

Based on this, optionally, in a specific implementation manner, when the target RAID is restarted due to abnormal power failure, before the step of determining the minimum write pointer and the maximum write pointer from the write pointers in the SMR areas of the SMRs in the target RAID is performed in step S801, the RAID recovery method provided in the embodiment of the present invention may further include the following steps:

determining whether write pointers in SMR areas of a plurality of SMRs in a target RAID are consistent; if not, the step of determining the minimum write pointer and the maximum write pointer from the write pointers in the SMR areas of the SMRs in the target RAID in the step S801 is executed.

In this specific implementation, when the target RAID is restarted after abnormal power failure, it may be determined whether write pointers in SMR areas of the plurality of SMRs in the target RAID are consistent, so that when it is determined that the write pointers in the SMR areas of the plurality of SMRs in the target RAID are not consistent, a minimum write pointer and a maximum write pointer may be further determined from the write pointers in the SMR areas of the plurality of SMRs in the target RAID.

In this way, when the target RAID is restarted after abnormal power failure, the target RAID can be recovered only when the write pointers in the SMR areas of the SMRs in the target RAID are inconsistent, so that the target RAID is prevented from being meaninglessly recovered, equipment resources are saved, and the equipment load is reduced.

Corresponding to the above specific implementation manners, an embodiment of the present invention provides a RAID driver, where the RAID driver may be configured to implement the above specific implementation manners, that is, at least one specific implementation manner of the specific implementation manners of the SMR-based RAID creating method, the RAID-specific data writing method, and the RAID recovery method provided in the embodiment of the present invention may be completed by operating the RAID driver.

The embodiment of the present invention does not limit the specific writing manner and the writing language of the RAID driver, and as long as the RAID driver that can implement at least one specific implementation manner of the specific implementation manners of the SMR-based RAID creating method, the RAID-specific data writing method, and the RAID recovery method provided in the embodiment of the present invention is within the protection scope of the present invention.

Corresponding to the above specific implementation manners, an embodiment of the present invention provides an SMR-based RAID creation apparatus, which is applied to a preset RAID driver.

Fig. 11 is a schematic structural diagram of a RAID creation apparatus based on SMR according to an embodiment of the present invention, and as shown in fig. 11, the creation apparatus may include the following modules:

an SMR determination module 1110 to determine SMRs for creating RAID;

a first array configuration module 1120 configured to configure SMR areas of the plurality of SMRs as a first array of a target RAID to be created; wherein the first array comprises a plurality of stripes, each stripe comprises a write area with the same offset address in an SMR area of the plurality of SMRs, and each stripe is used for storing data files written in an aligned sequential writing manner;

a second array configuration module 1130 to configure the CMR area of the SMRs as a second array of the target RAID.

Based on this, by applying the scheme provided by the embodiment of the present invention, the data content of the data file written to the target RAID can be cached by using the second array cache of the RAID formed by the CMR areas of the plurality of SMRs, so that it is not necessary to use another hard disk as the cache. Therefore, the RAID can be created by using the SMR without using other hard disks as a cache.

Optionally, in a specific implementation manner, the apparatus further includes:

a hard disk setting module, configured to set the SMRs as a first type of hard disk and at least one second type of hard disk before configuring SMR areas of the SMRs as a first array of a target RAID to be created;

wherein each of the strips comprises: the first writing area is used for storing the check value of the data file, and each second writing area is used for storing partial data content of the data file.

a first quantity determination module, configured to determine a first quantity of all the data files that can be stored by the target RAID based on a spatial size of the first array and a first data quantity of each of the data files;

a target area determination module to determine a target area in the second array based on the first number; wherein the target area is used for storing the metadata of each data file

Optionally, in a specific implementation manner, in a case that the plurality of SMRs are set as one first type hard disk and at least one second type hard disk, the apparatus further includes: a first data volume determination module for determining a first data volume for each of the data files;

the first data volume determining module is used for calculating the first data volume of each data file according to a preset formula; in fact, the preset formula is:

D＝S*(N-1)

wherein D is the first amount of data, S is a space size of each write area in an SMR area of the SMR, and N is a second number of the SMRs;

the first quantity determination module is specifically configured to: determining a first number of all the data files that can be stored by the target RAID based on a specified space size in the first array and a first amount of data for each of the data files; and the specified space size is the space size of a writing area of the second type of hard disk.

Optionally, in a specific implementation manner, the target area determining module is specifically configured to:

determining a product of the first amount and a preset second data amount of each metadata;

and determining a region with the spatial size of the product in the second array as a target region.

Corresponding to the above specific implementation manners, embodiments of the present invention provide a data writing device for RAID, which is applied to a preset RAID driver.

Fig. 12 is a schematic structural diagram of a data writing apparatus for RAID according to an embodiment of the present invention, and as shown in fig. 12, the data writing apparatus may include the following modules:

the data reading module 1210 is configured to, when it is detected that a data amount of data to be stored is not less than a first data amount, read data, which is not written into a target RAID and is the first data amount, in the data to be stored as a data file to be written; the target RAID is created based on any SMR-based RAID creation method provided by the embodiment of the invention; the first data volume is the data volume of each data file stored by the target RAID;

the data caching module 1220 is configured to write the data file to be written into a first cache preset in the second array of the target RAID;

a data allocating module 1230, configured to allocate, for each write area included in a designated stripe in the first array of the target RAID, the data content of the to-be-written data file cached in the first cache, so that the SMR to which each write area belongs writes the data content allocated to the write area into the write area.

Based on this, by applying the scheme provided by the embodiment of the present invention, it is possible to write data to each stripe in the first array of the target RAID created by any one of the SMR-based RAID creation methods provided by the embodiments of the present invention, so that data storage of the RAID created by the SMR is achieved without additionally using another hard disk as a cache. Moreover, the data volume of the data file to be written stored each time is determined when the target RAID is created, and the data volume of each data file stored by the target RAID is determined when the target RAID is created, so that the stored data content can occupy the storage space of the designated stripe after the data file to be written is written, and therefore, it is possible to avoid simultaneous existence of a plurality of write operations for each write area in the same stripe.

Optionally, in a specific implementation manner, each stripe of the first array includes: a first writing area and at least one second writing area, wherein the first writing area is used for storing the check value of the data file, and each second writing area is used for storing partial data content of the data file; the data allocating module 1230 is specifically configured to:

distributing the data content of the data file to be written cached in the first cache for each second writing area included in a specified stripe in the first array of the target RAID;

distributing a check value to be written of the data file to be written to the first writing area included by the specified stripe; wherein the check value to be written is calculated based on the data content allocated to each of the second writing areas.

and the metadata adding module is used for adding the metadata of the data file to be written in a target area included by the second array of the target RAID.

the data acquisition module is used for reading the data to be stored when the data volume of the data to be stored is detected to be not less than a first data volume, wherein the data volume is the first data volume and is not written into the target RAID, acquiring target data before the data is used as a data file to be written, and writing the target data into a second cache in the memory; wherein the space size of the second cache is: an integer multiple of the first amount of data;

the data reading module 1210 is specifically configured to: and when the data volume of the target data in the second cache is not less than the first data volume, reading the data which is not written into the target RAID and has the data volume of the first data volume in the second cache as a data file to be written.

a cache application module, configured to determine, before the target data is obtained, a data volume of each data file stored in the target RAID as a first data volume; and applying for the second cache in the memory according to the first data volume.

Corresponding to the above specific implementation manners, an embodiment of the present invention provides a RAID recovery apparatus, which is applied to a preset RAID driver.

Fig. 13 is a schematic structural diagram of a RAID recovery apparatus according to an embodiment of the present invention, and as shown in fig. 13, the RAID recovery apparatus may include the following modules:

a pointer determining module 1310, configured to determine a minimum write pointer and a maximum write pointer from write pointers in SMR areas of SMRs in a target RAID when the target RAID is restarted due to abnormal power failure; the target RAID is created based on any SMR-based RAID creation method provided by the embodiment of the invention; the minimum write pointer is: the write pointer with the minimum corresponding storage address in write pointers in SMR areas of a plurality of SMRs in the target RAID; the maximum write pointer is: the write pointer with the largest corresponding storage address in write pointers in SMR areas of a plurality of SMRs in the target RAID;

a valid data determining module 1320, configured to determine, as valid data in the first array of the target RAID, data stored in each stripe located before the stripe where the minimum write pointer is located in the first array of the target RAID;

a pointer moving module 1330 for moving the write pointer in the SMR areas of the plurality of SMRs to a start position of a write area belonging to the target stripe; wherein the target stripe is: a stripe in the first array of the target RAID that is located after the stripe where the largest write pointer is located.

Optionally, in a specific implementation manner, the apparatus may further include:

the region filling module is used for filling a designated region in each SMR by using preset null data; wherein the designated area is: an area between the write pointer before the movement and the write pointer after the movement.

Optionally, in a specific implementation manner, the target band is: and in the first array of the target RAID, a stripe which is positioned after the stripe where the maximum write pointer is positioned and is adjacent to the stripe where the maximum write pointer is positioned.

a pointer judging module, configured to determine whether write pointers in SMR areas of the plurality of SMRs in the target RAID are consistent before determining a minimum write pointer and a maximum write pointer of the write pointers in the SMR areas of the plurality of SMRs in the target RAID; if not, the pointer determination module 1310 is triggered.

and the data recovery module is used for recovering the data stored in the specified area in each SMR before the abnormal power failure of the target RAID.

Corresponding to the above-mentioned specific implementation manners, the embodiment of the present invention further provides an electronic device, which is provided with a RAID driver, and as shown in fig. 14, the electronic device includes a processor 1401, a communication interface 1402, a memory 1403, and a communication bus 1404, where the processor 1401, the communication interface 1402, and the memory 1403 complete communication with each other through the communication bus 1404,

a memory 1403 for storing a computer program;

the processor 1401 is configured to implement the steps of any one of the above specific implementations when executing the program stored in the memory 1403.

The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the electronic equipment and other equipment.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.

In yet another embodiment of the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program is executed by a processor to perform the steps of any one of the above specific implementations.

In yet another embodiment of the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the steps of any of the above-described specific implementations.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus embodiments, the electronic device described, the RAID embodiments, the computer-readable storage medium embodiments, and the computer program product embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference may be made to some descriptions of the method embodiments for relevant points.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A RAID creation method based on SMR is characterized in that the method is applied to a preset RAID drive, and the method comprises the following steps:

determining a plurality of SMRs for creating RAID;

2. The method of claim 1, wherein prior to the step of configuring the SMR region of the plurality of SMRs as a first array of a target RAID to be created, the method further comprises:

setting the SMRs as a first type of hard disk and at least one second type of hard disk;

3. The method according to claim 1 or 2, characterized in that the method further comprises:

determining a first number of all the data files that can be stored by the target RAID based on the spatial size of the first array and the first amount of data of each of the data files;

determining a target region in the second array based on the first number; wherein the target area is used for storing metadata of each data file.

4. The method of claim 3, wherein, in the case that the plurality of SMRs are configured as a first type of hard disk and at least one second type of hard disk, determining the first amount of data of each of the data files comprises:

D＝S*(N-1)

the step of determining a first number of all the data files that can be stored by the target RAID based on the spatial size of the first array and the first data amount of each of the data files includes:

determining a first number of all the data files that can be stored by the target RAID based on a specified space size in the first array and a first amount of data for each of the data files; and the specified space size is the space size of a writing area of the second type of hard disk.

5. The method of claim 3, wherein the step of determining a target area in the second array based on the first quantity comprises:

6. A data writing method for RAID (redundant array of independent disks), which is applied to a preset RAID drive, comprises the following steps:

7. The method of claim 6, wherein each strip of the first array comprises: a first writing area and at least one second writing area, wherein the first writing area is used for storing the check value of the data file, and each second writing area is used for storing partial data content of the data file;

the step of allocating, for each write area included in a designated stripe in the first array of the target RAID, data content of the to-be-written data file cached in the first cache includes:

8. The method according to claim 6 or 7, characterized in that the method further comprises:

and adding the metadata of the data file to be written in a target area included by the second array of the target RAID.

9. The method according to claim 6 or 7, wherein before the step of reading the data to be stored, which is the first data amount and is not written into the target RAID, as the data file to be written when it is detected that the data amount of the data to be stored is not less than the first data amount, the method further comprises:

acquiring target data and writing the target data into a second cache in the memory; wherein the space size of the second cache is: an integer multiple of the first amount of data;

when detecting that the data volume of the data to be stored is not less than a first data volume, reading the data to be stored, the data volume being the first data volume and not written into the target RAID, as a data file to be written, including:

and when the data volume of the target data in the second cache is not less than the first data volume, reading the data which is not written into the target RAID and has the data volume of the first data volume in the second cache as a data file to be written.

10. The method of claim 9, wherein prior to the step of obtaining target data, the method further comprises:

determining the data volume of each data file stored by the target RAID as a first data volume;

and applying for the second cache in the memory according to the first data volume.

11. A RAID recovery method is applied to a preset RAID drive, and the method comprises the following steps:

12. The method of claim 11, further comprising:

filling a designated area in each SMR with preset null data; wherein the designated area is: an area between the write pointer before the movement and the write pointer after the movement.

13. The method according to claim 11 or 12, wherein the target band is: and in the first array of the target RAID, a stripe which is positioned after the stripe where the maximum write pointer is positioned and is adjacent to the stripe where the maximum write pointer is positioned.

14. The method of claim 11 or 12, wherein prior to the step of determining a minimum write pointer and a maximum write pointer of the write pointers in the SMR region of the plurality of SMRs in the target RAID, the method further comprises:

determining whether write pointers in SMR regions of a plurality of SMRs in the target RAID are consistent;

and if not, executing the step of determining the minimum write pointer and the maximum write pointer in the write pointers of the SMRs in the target RAID.

15. The method according to claim 11 or 12, characterized in that the method further comprises:

and before the abnormal power failure of the target RAID, restoring the data stored in the designated area in each SMR.

16. A RAID created based on the method steps of any one of claims 1 through 5.

17. A RAID drive configured to perform the method steps of any one of claims 1 through 15.

18. An SMR-based RAID creation apparatus applied to a preset RAID drive, the apparatus comprising:

an SMR determination module to determine a plurality of SMRs for creating RAID;

19. A data writing apparatus for RAID, applied to a predetermined RAID driver, the apparatus comprising:

20. An apparatus for recovering RAID, applied to a predetermined RAID driver, the apparatus comprising:

the pointer determining module is used for determining a minimum write pointer and a maximum write pointer from write pointers in SMR areas of a plurality of SMRs in a target RAID when the target RAID is restarted due to abnormal power failure; wherein the target RAID is created based on the method of any one of claims 1 to 5; the minimum write pointer is: the write pointer with the minimum corresponding storage address in write pointers in SMR areas of a plurality of SMRs in the target RAID; the maximum write pointer is: the write pointer with the largest corresponding storage address in write pointers in SMR areas of a plurality of SMRs in the target RAID;

21. The electronic equipment is characterized by being provided with a RAID drive and comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;

a memory for storing a computer program;

a processor for implementing the method steps of any of claims 1-15 when executing a program stored in the memory.

22. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of the claims 1-15.