CN109558066B

CN109558066B - Method and device for recovering metadata in storage system

Info

Publication number: CN109558066B
Application number: CN201710880549.8A
Authority: CN
Inventors: 方协云; 吴丹; 魏明昌; 朱小兵
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2017-09-26
Filing date: 2017-09-26
Publication date: 2020-10-27
Anticipated expiration: 2037-09-26
Also published as: CN109558066A

Abstract

The application provides a method and a device for recovering metadata in a storage system, wherein the storage system comprises a first storage block group CKG and a second CKG, the first CKG comprises at least one stripe, the second CKG comprises at least one stripe, and the method comprises the following steps: acquiring first data of a first strip in a first CKG and second data of a second strip in a second CKG; if the first data and the second data are the same, selecting data from the first data and the second data according to the number of valid columns of the first stripe and the number of valid columns of the second stripe, and the state of the first CKG and the state of the second CKG, wherein the states include a fully written state and a fully un-written state. The method and the device for recovering the metadata in the storage system can improve the reliability of the data.

Description

Method and device for recovering metadata in storage system

Technical Field

The present application relates to the field of computer storage, and more particularly, to a method and apparatus for recovering metadata in a storage system.

Background

In a storage system, a disk needs to store metadata information for managing user data in addition to user data. In order to accelerate the reconstruction speed of the metadata after power up and power down, the metadata is generally organized and managed by adopting a hierarchical idea. Generally, a relatively small storage space is divided on a disk to store top-level metadata. The reliability of the top-level metadata is the basis for the reliability of the system. The bottom metadata is stored by using a Redundant Array of Independent Disks (RAID).

Redundant array of independent disks RAID is a technology that combines a plurality of independent hard disks (physical hard disks) in different ways to form one hard disk group (logical hard disk), and RAID can provide higher storage performance and data backup performance than a single hard disk. Data in a RAID is distributed in the form of stripes (Stripe), which may be understood as a set of location-dependent blocks on two or more partitions in a hard disk array, which may also be referred to as Stripe Units (SU), i.e., a Stripe is composed of multiple Stripe units. One stripe includes a stripe unit for storing data and a stripe unit for storing parity data of the data.

In the recovery operation of metadata, for two identical copies of data, the prior art randomly selects one copy of data, and thus the reliability of the selected data is low. For example, if a certain data is stored in different CKG stripes at the same time, the number of valid columns of the CKG where one data is located is 3 columns, and the number of valid columns of the CKG where another data is located is only 1 column, data with the number of valid columns of only 1 column may be selected in a random selection manner, so that after system loading is performed, if a disk where the 1 column of data is located fails, data loss may be caused, and reliability is low. Therefore, it is desirable to provide a scheme for improving the reliability of data.

Disclosure of Invention

The application provides a method and a device for recovering metadata in a storage system, which can improve the reliability of data, thereby improving the reliability of the whole system.

In a first aspect, a method for recovering metadata in a storage system is provided, where the storage system includes a first storage block group CKG and a second CKG, the first CKG includes at least one stripe, and the second CKG includes at least one stripe, the method includes:

acquiring first data of a first stripe in the first CKG and second data of a second stripe in the second CKG;

and if the first data and the second data are the same, selecting data from the first data and the second data according to the number of valid columns of the first stripe, the number of valid columns of the second stripe, the state of the first CKG and the state of the second CKG, wherein the states comprise a full-written state and a non-full-written state.

In the embodiment of the application, when the first data and the second data are the same, the storage device selects data from the first data and the second data according to the number of valid columns of the first stripe, the number of valid columns of the second stripe, the state of the first CKG and the state of the second CKG, and can select data with higher reliability, so that the reliability of the storage system is improved.

In some possible implementations, the state of the first CKG is the same as the state of the second CKG, and the number of valid columns of the first stripe is different from the number of valid columns of the second stripe;

wherein selecting data from the first data and the second data according to the number of valid columns of the first stripe and the number of valid columns of the second stripe, and the state of the first CKG and the state of the second CKG comprises:

and selecting the data with the maximum number of effective columns of the corresponding stripe from the first data and the second data.

Therefore, when the CKG states are the same, data with higher reliability can be selected according to the number of valid columns of the stripe.

In some possible implementations, the state of the first CKG is different from the state of the second CKG, and the number of valid columns of the first stripe is the same as the number of valid columns of the second stripe;

and selecting the data in which the state of the CKG is a fully written state from the first data and the second data.

Therefore, when the number of valid columns of the stripe is the same, data with higher reliability can be selected according to the state of CKG.

In some possible implementations, selecting data from the first data and the second data according to the number of valid columns of the first stripe and the number of valid columns of the second stripe, and the state of the first CKG and the state of the second CKG includes:

calculating a weight corresponding to the first data according to the number of the valid columns of the first strip and the state of the first CKG;

calculating a weight corresponding to the second data according to the number of valid columns of the second stripe and the state of the second CKG;

and comparing the weight of the first data with the weight of the second data, and selecting the data with the maximum weight from the first data and the second data.

Therefore, the weight corresponding to the data can be calculated according to the effective column number of the stripe and the state of the CKG, so that the data with larger weight can be selected, and the data with higher reliability can be obtained.

In some possible implementations, calculating the weight corresponding to the first data or calculating the weight corresponding to the second data includes:

if the state of the CKG where the stripe corresponding to the data is in a full state, calculating a weight according to the following formula:

W＝N*a+b，

if the state of the CKG where the stripe corresponding to the data is located is an unwritten state, calculating the weight according to the following formula:

W＝N*a；

or, if the state of the CKG in which the stripe corresponding to the data is located is a fully written state, calculating the weight according to the following formula:

W＝N*a，

W＝N*a-b，

wherein, W represents a weight corresponding to the data, N represents the number of valid columns of a stripe corresponding to the data, a is a factor of the number of valid columns, b is a state factor, a and b are constants greater than 0, and a > b.

Therefore, the weight value of the data can be calculated by adopting the weight value calculation formula, so that the data with larger weight value can be selected, and the data with higher reliability can be obtained.

In a second aspect, an apparatus for recovering metadata in a storage system is provided, which is configured to perform the method in the first aspect or any possible implementation manner of the first aspect. In particular, the apparatus comprises means for performing the method of the first aspect described above or any possible implementation manner of the first aspect.

In a third aspect, an apparatus for recovering metadata in a storage system is provided and includes a processor, a memory, and a transceiver. The processor is coupled to the memory and the transceiver. The memory is for storing instructions, the processor is for executing the instructions, and the transceiver is for communicating with other network elements under control of the processor. The processor, when executing the instructions stored by the memory, causes the processor to perform the method of the first aspect or any possible implementation of the first aspect.

In a fourth aspect, there is provided a computer-readable storage medium storing a program for causing an apparatus for restoring metadata in a storage system to perform the method for managing network slices of the first aspect described above and any of its various implementations.

In a fifth aspect, there is provided a storage system comprising the apparatus of any possible implementation of the third aspect or the third aspect; alternatively, the storage system comprises the apparatus of the fourth aspect or any possible implementation manner of the fourth aspect.

A sixth aspect provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of the first aspect described above or any possible implementation thereof.

Drawings

FIG. 1 is a schematic diagram of a memory device.

Fig. 2 is a schematic diagram of the layout of the space on the disc.

FIG. 3 is a schematic diagram of one example of a module for metadata recovery according to an embodiment of the present application.

FIG. 4 is a flow chart of an example of a scan wheel according to an embodiment of the present application.

FIG. 5 is a schematic flow chart diagram of a method for recovering metadata in a storage system according to an embodiment of the present application.

FIG. 6 is a schematic diagram of an example according to an embodiment of the present application.

FIG. 7 is a schematic diagram of another example according to an embodiment of the present application.

Fig. 8 is a schematic block diagram of an apparatus for restoring metadata in a storage system according to an embodiment of the present application.

Fig. 9 is a block diagram illustrating an apparatus for restoring metadata in a storage system according to an embodiment of the present application.

Detailed Description

The technical solution in the present application will be described below with reference to the accompanying drawings.

The technical solution of the embodiment of the present application may be applied to a storage device or a storage system using a Stripe (Stripe) as a management manner, for example, a storage array formed by a Solid State Disk (SSD), or a storage array formed by an SSD itself or a Shingled Magnetic Recording (SMR) Disk.

A Stripe is a method for merging multiple hard disks (in a single disk, multiple storage media) into one volume, and may be understood as a set of location-dependent partitions on two or more partitions in a hard disk array (or hard disk), which may also be referred to as a Stripe Unit (SU), i.e., a Stripe is composed of multiple SUs. Stripe is the result of striping management of storage space, as can be seen in particular by those skilled in the art. SU corresponds to a Block (Block) in the hard disk, for example, as shown in fig. 1, Stripe0 includes 3 SUs, and these 3 SUs correspond to a data Block (Block) in a storage Block (Chunk, CK)1 in a row indicated by 102 in fig. 1, a data Block in a storage Block 2, and a check data Block in a hard disk n, respectively; stripe1 comprises 3 SUs, and these 3 SUs correspond to the data block in storage block 1, the check data block in storage block 2 in the row indicated by 104 in fig. 1, respectively. Stripe3 includes 3 SUs, which 3 SUs correspond to the data block in storage block 1, the data block in storage block 2 in the row indicated by 106 in fig. 1, respectively. Stripe3 includes 3 SUs, which 3 SUs correspond to the parity data block in memory block 1, the data block in memory block 2, and the data block in hard disk n, respectively, in the row indicated by 108 in fig. 1. In fig. 1, one memory block Group (CKG) is composed of n CK.

By way of example and not limitation, the storage device shown in fig. 1 is, for example, an SSD storage array, and for example, storage block 1, storage block 2, and storage block … shown in fig. 1 are all SSDs. For convenience of understanding and description, the embodiment of the present application is described below as applied to an SSD-based storage device as an example, but it is clearly understood by those skilled in the art according to the teachings of the embodiment of the present application that the method of the embodiment of the present application can be applied to other storage devices using Stripe as a management manner, and such applications all fall within the scope of the present application.

The SSD is a new type of storage system using flash memory as a storage medium, the flash memory does not have mechanical parts like a magnetic head, and the overhead of the random access mode and the sequential access mode is comparable, so the SSD has a greater performance advantage than the conventional hard disk. For example, when data is to be updated in a flash memory, a new value of the data cannot be written at a physical address where an old value of the data is located until the old value of the data is not erased, that is, the old value of the data needs to be erased first to write the new value of the data at the physical address where the old value of the data is located. However, the flash memory has also characteristics that read and write operations are in units of pages, and erase operations are in units of blocks (including a plurality of pages). In other words, to implement the writing of a new value of data at the physical address where the old value of data is located, it is necessary to erase the data over the entire block (typically 64K or 128K) where the old value of data is located before the new value of data can be written in the block in the erased state. First, however, the cost of erasing a block is very large because although the block in the flash memory can be erased repeatedly, the number of times of erasing each block is limited, and the lifetime of the flash memory is calculated according to the number of times of erasing, for example, the lifetime of a Multi-Level Cell (MLC) flash memory is generally 10000 times of 1000-. Secondly, if the block to be erased also includes valid data, the valid data needs to be read out before erasing the block, and then a new block address is written, this process is called data migration, and the data migration may generate Write Amplification (WA), that is, the actually written data amount will be larger than the data amount that needs to be written into the SSD, and the write amplification is an undesirable phenomenon. In order to make room for a user to write data, Garbage Collection (GC) is required, that is, all valid data existing in the stripe is moved to other free stripes, and then the stripe is recovered. In the moving process, the abnormal power failure condition exists when the old block is not erased, or the hard disk of a certain CK in the CKG of the old block is pulled out before the old block is erased, and the conditions can cause that single data is stored in a plurality of different CKGs. For these situations, the prior art generally selects a data randomly, and the reliability is low.

As shown in fig. 1, the storage array composed of SSDs includes a controller and SSDs connected to the controller (assuming that the hard disks shown in fig. 1 are all SSDs). Wherein the controller is configured to divide the SSD into a plurality of strips (such as strips 0, strips 1, strips 2, and strips 3 shown in fig. 1), and to provide Logical Unit Numbers (LUNs) to the host using the strips. The LUN can be regarded as a logical hard disk that can be accessed by the host, which can also be referred to as a logical space unit, and is also a space where the user can implement reading and writing. As shown in fig. 1, a Stripe includes a plurality of SUs, each SU having a Block (Block) corresponding thereto. It should be appreciated that when a host performs a write operation in a LUN, the corresponding data will be written to the SU, and correspondingly, to the corresponding block. When the Stripe is garbage collected, the corresponding block is also collected, namely the corresponding block is erased, and the storage space of the corresponding block is released.

Specifically, the controller shown in fig. 1 is, for example, a RAID controller, and then Stripe0, Stripe1, Stripe2, and Stripe3 shown in fig. 1 are all stripes built based on RAID technology.

It should be understood that the embodiments of the present application may be applied to a storage system using RAID1 technology, and may also be applied to other scenarios based on RAID technology, such as RAID6, RAID10, RAID1E, RAID-TP, and other technologies.

In a storage system, a hard disk needs to store metadata information for managing user data in addition to user data. The storage of metadata may employ the RAID technology based scenario described above in FIG. 1. Metadata is generally organized and managed by adopting a hierarchical idea, a first-layer metadata area is metadata information of a data area, a second-layer metadata area is metadata information of a first-layer metadata area of the first-layer metadata area, …, and an nth-layer metadata area is metadata information of an n-1 th-layer metadata area, so that the data volume of top-layer (nth-layer) metadata is as small as possible, and therefore, after power is turned on and off, the root of the metadata is quickly reconstructed, and read-write services are provided for users.

In a storage system, a relatively small storage space is usually partitioned on a disk for storing top-level metadata, and the top-level metadata is usually recovered by a disk-scanning manner. Fig. 2 shows a layout of the space on the disc. As shown in fig. 2, each hard disk is divided into a top-level metadata area (meta area) for storing top-level metadata and a data area (data area) for storing data and non-top-level metadata. The process of reconstructing the top-level metadata by power-on scanning is to scan the meta area so as to identify valid data and recover the top-level metadata.

In recovering the metadata, a software module for metadata recovery may be included in the storage device. FIG. 3 shows a schematic diagram of one example of modules for metadata recovery. As shown in fig. 3, in the metadata recovery process, a Pool Management (PMGR) module is used to control the metadata process, a Block Management (BM) module is used to scan a specified storage space, a ZONE (ZONE) module is used to organize scanned CK, a metadata service (MDS) module is used to take charge of organization management and verification of metadata, and a Block Device Management (BDM) module is used to interface device drivers and hardware and take charge of management of hard disks. The above modules can be used in combination or separately to implement the technical solution of the embodiments of the present application. The modules may be deployed in a storage device, or may exist independently, which is not limited to this.

The tray sweeping process related to the embodiment of the present application is described herein with reference to the flowchart in fig. 4. FIG. 4 shows a flow diagram of one example 400 of a scan tray according to an embodiment of the application. For example, as shown in fig. 4, this example 400 includes:

401, it is determined whether the scan of the designated storage space is complete.

Specifically, the BM module of the memory device in fig. 3 may scan the memory block CK in the specified memory space according to the granularity of the memory block CK, and determine whether the memory space is completely scanned. If the designated storage space is judged to be completely scanned, the scanning is stopped, and if the designated storage space is judged to be not completely scanned, the process goes to step 402.

402, the data of the next memory block CK is read.

Specifically, if it is determined that the specified storage space is not completely scanned, the scanning operation may be continued, that is, the data of the next storage block CK in the storage space is read.

And 403, judging whether the data is valid data or not.

Specifically, the MDS module of the memory device in fig. 3 can check whether the data in the memory block CK is valid data. If the data in the memory block CK is not valid data, go to 401 to continue scanning, and if the data in the memory block CK is all valid data, go to 404.

404 and other scanned memory blocks CK form a memory block group CKG.

Specifically, the ZONE module of the memory device in fig. 3 may group the scanned valid memory blocks CK into a memory block group CKG. The state of the block group CKG may be written full or not. For convenience of description, a memory block set in a fully written state is referred to as a full CKG, and a memory block set in an incompletely written state is referred to as an under CKG.

Alternatively, the set of memory blocks CKG may be a full CKG or a less than full CKG.

The scan flow in the metadata recovery flow is described herein with reference to fig. 4. The purpose of scanning is to obtain valid CKGs, and to discard grain data (the minimum storage access unit of metadata) in the CKGs to the metadata management module according to the states of the CKGs, so that a subsequent metadata management module can select data with higher reliability according to the states of the CKGs and the number of columns of valid data.

The embodiment of the application is applied to a metadata recovery scene. Generally, metadata recovery is based on top-level metadata recovery, i.e., the metadata of other layers can be recovered layer by layer only after the top-level metadata is completely recovered. The embodiment of the application provides a method for recovering metadata in a storage system, and when the metadata is recovered, data with higher reliability can be selected, so that the reliability of the whole system is improved.

FIG. 5 illustrates a method 500 for restoring metadata in a storage system according to an embodiment of the present application. The memory system includes a first bank of memory blocks CKG including at least one stripe and a second CKG including at least one stripe. The method 500 may be applied to the scenario described above with respect to fig. 1. Alternatively, the method 500 may be implemented by a storage device or a metadata management module in the storage device, and the metadata management module may be deployed in the RAID controller in fig. 1, or the metadata management module may be provided separately, which is not limited thereto. For convenience of description, the following will describe a storage device as an example. As shown in fig. 5, the method 500 includes:

s510, obtain first data of a first stripe in the first CKG and second data of a second stripe in the second CKG.

Optionally, the storage device may obtain the first data when the data verification passes by performing validity verification on the data in the first stripe. Wherein the first data is for a first stripe stored in the first CKG in RAID. For example, the first CKG may organize management data using RAID1 backup.

Optionally, the storage device may obtain the second data when the data check passes by performing validity check on the data in the second stripe. Wherein the second data is for a second stripe stored in the second CKG in RAID. For example, the second CKG may organize management data using RAID1 backup.

It should be understood that the first strip or the second strip described above refers to a single strip. Data in a single stripe may be stored in a multi-backup manner, wherein the smallest storage access unit of the data may be grain data.

Alternatively, the first data or the second data may be stored in a stripe in a full CKG or may be stored in a stripe in a less-full CKG. For example, the data is stored in the full CKG or the less-than-full CKG stripe obtained in the above-mentioned tray-sweeping process of fig. 4.

Optionally, the first stripe or the second stripe has a corresponding number of valid columns. For example, in fig. 1, for stripe1, in n memory blocks, if the data stored in memory block 1 and memory block 2 is valid after being checked, memory block 1 and memory block 2 can be understood as the valid column number of stripe 1.

Alternatively, the storage device may compare the data content of the first data with the data content of the second data, and if the data content of the first data is the same as the data content of the second data, a subsequent selection step may be performed to select data from the first data and the second data. If the data content of the first data and the data content of the second data are different, data information of the first data and data information of the second data are recorded. That is, if the storage device may also acquire data that has not been previously presented, the storage device may first save the information of the data that has not been previously presented, so as to facilitate subsequent use.

It should be understood that the first data and the second data are used as examples for illustration and are not intended to limit the scope of the embodiments of the present application.

It should be further understood that the first CKG and the second CKG in the memory system are described as an example, and there may be more CKGs in practice, which is not limited thereto.

S520, if the first data is the same as the second data, selecting data from the first data and the second data according to the number of valid columns of the first stripe, the number of valid columns of the second stripe, and the states of the first CKG and the second CKG, where the states include a full-written state and a non-full-written state.

Alternatively, when the storage device determines that the data content of the first data is the same as the content of the second data, the following factors may be considered together: (1) the number of active columns of stripes; (2) and selecting the data with the highest reliability from the first data and the second data according to the state of the CKG where the stripe is located. Further, after the data is selected, the metadata management module may store or record information of the selected data, and place the selected data into metadata organization management.

For example, in the process of reconstructing the top-level metadata, the metadata management module may perform a data recovery operation according to the recorded information of the selected data.

It should be understood that, in the embodiment of the present application, the numbers "first" and "second" … are introduced only for distinguishing different objects, for example, distinguishing different "CKGs", or distinguishing different "strips", and do not limit the scope of the embodiment of the present application.

It should be noted that the above description is only given by taking an example that the same data may appear in the first stripe and the second stripe, and does not limit the scope of the embodiments of the present application. In a specific implementation, the same data may appear in N stripes (N is an integer greater than or equal to 2), and in this case, the technical solution of the embodiment of the present application is still applicable. That is, a person skilled in the art may select data with the largest weight value from the N stripes according to a scheme of determining the weight value based on the number of valid columns of the stripes and the state of the CKG in the embodiment of the present application.

Embodiments for selecting data to be processed based on the above two factors are described in detail below, respectively.

Optionally, as an embodiment, the state of the first CKG is the same as the state of the second CKG, and the number of valid columns of the first stripe is different from the number of valid columns of the second stripe;

wherein, S520 includes:

That is, if the state of the first CKG and the state of the second CKG are the same, for example, both are fully written state CKG or both are not fully written state CKG, the memory device only needs to consider the number of valid columns of the stripe. Then, the metadata management module selects the data with the most effective columns in the corresponding stripe from the first data and the second data. The greater the number of active columns, the greater the reliability of the data stored in the stripe. In this way, under the condition that the CKG states are the same, the storage device can select data with better reliability according to the corresponding number of valid columns in each stripe.

Optionally, as an embodiment, the state of the first CKG is different from the state of the second CKG, and the number of valid columns of the first stripe is the same as the number of valid columns of the second stripe;

wherein, S520 includes:

That is, if the number of valid columns of the first stripe and the number of valid columns of the second stripe are the same, the memory device only needs to consider the state of the block group CKG. The memory device selects the data in which the memory block group CKG is in the write-full state from the first data and the second data. Since the data of the memory block group CKG in the fully written state is more complete, the reliability of the data stored in the memory block group CKG in the fully written state is higher. For example, if the first CKG is in a fully written state and the second CKG is in an un-fully written state, then the memory device selects the first data. Thus, when the number of valid columns of the stripe is the same, the data with better reliability can be selected according to the state of the memory block group CKG in which each stripe is located.

Optionally, S520 includes:

That is, the weight of the data may be calculated based on the number of valid columns of the stripe and the state of CKG, so as to select the data with the largest weight. Wherein, the weight value can be understood as a weight value for describing the reliability of the data.

Further, the embodiment of the present application provides the following formula for calculating the weight of the data, where the following formula is used for calculating the weight of the data based on the number of valid columns of the stripe and the state of the CKG. Here, the first data and the second data may each calculate a corresponding weight value by using the following formula.

For example, if the state of the CKG in which the stripe corresponding to the data is located is a fully written state, the weight may be calculated by the following formula:

W＝N*a+b (1.1)，

correspondingly, if the state of the CKG in which the stripe corresponding to the data is located is an unwritten state, the weight may be calculated by using the following formula:

W＝N*a (1.2)；

or, if the state of the CKG where the stripe corresponding to the data is in the full-written state, the weight may be calculated by using the following formula:

W＝N*a (1.3)，

W＝N*a-b (1.4)；

wherein, W represents a weight corresponding to the data, N represents the number of valid columns of a stripe corresponding to the data, a is a factor of the number of valid columns, b is a state factor of the storage block group CKG, a and b are constants greater than 0, and a > b.

Here, when the number of valid columns is the same, or the state of CKG is the same, the weight of data can be calculated according to the above equations 1.1-1.4, which is not described herein again.

Therefore, in the embodiment of the application, a calculation weight calculation formula is introduced, the weight corresponding to the data is calculated based on the calculation weight formula, and the data with the larger weight is selected by comparing the weights of the data, so that the reliability of the data is improved.

To facilitate understanding by those skilled in the art, the following description is provided in conjunction with the flow of the specific implementation in fig. 6 and 7. In the examples of fig. 6 and 7 below, CKGs store data in multiple copies written by RAID 1.

For example, fig. 6 shows a process flow diagram for a full CKG. As shown in fig. 6, the process 600 includes:

601, read the first column CK data in the full CKG.

It should be understood that the data read from the first column CK is used as an example and is not limited thereto.

For a full CKG, since the current CKG organizes management data in a backup manner of RAID1, data of each column CK is the same, and thus data of one column CK can be read randomly, for example, a BM module of a storage device can read data of a first column CK in the full CKG.

At 602, it is determined whether the first column CK data is processed completely. If all the processing is finished, ending the process; if not, go to 603.

Here, the ZONE module of the storage device needs to traverse each data in the first column of CK data until all data in the first column of CK data is processed.

603, judging whether the single data in the first column of CK data passes the check. If the verification is passed, the ZONE module of the storage device sends the data to a metadata management module of the storage device, and the operation is switched to 604; if the verification fails, the flow goes to the processing flow of the not-full CKG (described below).

Here, validity checking needs to be performed on the single data. If the verification fails, processing according to the flow of the not-full CKG; if the check is successful, the following 604 can continue.

604, it is determined whether the same data is present. If the same data is present, a jump is made to 605.

For example, the metadata management module in the storage device determines whether data identical to the data that passes the check occurs, and if data identical to the data that passes the check occurs, the process jumps to 605 to compare the weights corresponding to the identical data. The calculation method of the weight may refer to the above-mentioned formula of full CKG, which is not described herein again. If the data passing the verification does not appear before, go to 606, that is, the information of the data not appearing can be retained first, so as to facilitate comparison when the same data appears subsequently.

605 select the data with larger weight and go to 606.

For example, if Data1 exists in both stripe1 in CKG1 and stripe2 in CKG2, the foregoing weight formula can be used to calculate the weight of Data1 based on the effective column number of stripe1 and the state of CKG1, and calculate the weight of Data1 based on the effective column number of stripe2 and the state of CKG2, and select the Data1 with larger weight, and record the information of the Data for management. In the flow of fig. 7, the calculation of the weight of the data needs to be calculated according to the formula of the full CKG described above.

And 606, placing metadata organization management.

Specifically, the metadata management module in the storage device may store information of the data with the largest weight, so as to be used in the subsequent metadata recovery. The metadata management module in the storage device can also record the information that the same data does not appear, so as to be convenient for judging and using when the same data appears in the following. For example, if the metadata management module in the storage device determines that the data contents of the first data and the second data are different, the information of the first data and the information of the second data are recorded.

For another example, fig. 7 shows a process flow diagram for a less than full CKG. As shown in fig. 7, the process 700 includes:

701, judging whether the data in all the strips in the CKG are processed or not. If the operation is finished, ending; if not, then proceed to 702.

Here, the ZONE module of the storage device needs to sequentially traverse the data of each stripe in the less-full CKG until the data processing of each stripe is completed.

For each stripe in CKG, the corresponding offset data in each column CK is read 702.

That is, for each stripe, multiple copies of data stored in the stripe need to be read.

703, in the same stripe, judging whether the data contents of the multiple copies of data passing the check are the same.

Since the current CKG organizes the management data in a manner that is a backup of RAID1, the multiple copies of data in a stripe are the same. Here, each piece of data in the same stripe needs to be read in sequence, and if the data content of the plurality of pieces of data is the same, the step goes to 704, and the data can be put into metadata organization management; if the data contents of the multiple copies of data are different, go to 707. For example, for the Data1, n copies, that is, n copies of Data1 may be stored in the same stripe, and when the Data content of n copies of Data1 is judged to be the same, Data1 is put into metadata organization management.

And 704, judging whether the same data appears. If the same data exists, go to 705; if the same data does not exist, then proceed to 706.

For example, after obtaining Data1, the metadata management module in the storage device determines whether the same Data as Data1 has appeared before. If yes, go to 705 to compare the weight of the data; if the same Data as Data1 has not appeared, Data1 is put into metadata organizational management.

705, select data with large weights, and go to 706.

For example, after the metadata management module in the storage device obtains Data1, it determines that the same Data as Data1 appears before, and then calculates the weight corresponding to Data1, and selects the Data with larger weight. For example, if Data1 exists in both stripe1 in CKG1 and stripe2 in CKG2, then the Data1 can be weighted based on the valid column number of stripe1 and the state of CKG1, and Data1 can be weighted based on the valid column number of stripe2 and the state of CKG2, and the Data1 with the greater weight can be selected using the weight formula described above. In the flow of fig. 7, the calculation of the weight of the data needs to be calculated according to the formula of the above-mentioned not-full CKG.

And 706, putting metadata organization management.

Specifically, the metadata management module in the storage device may store information of data with a larger weight value, so as to facilitate subsequent management. The metadata management module in the storage device can also record the information that the same data does not appear, so as to be convenient for judging and using when the same data appears in the following. For example, in 704, if it is determined that the same Data as Data1 does not exist, information of Data1 can be recorded.

707, skipping the data.

When reading each piece of data in the same stripe, if data with different data contents appears, the data needs to be skipped and not processed.

It should be understood that the specific implementation procedures according to the embodiments of the present application are described above with reference to fig. 6 and fig. 7, and do not limit the scope of the embodiments of the present application.

It should be understood that the examples of fig. 6 and 7 are merely for facilitating understanding of the embodiments of the present application by those skilled in the art, and are not intended to limit the embodiments of the present application to the specific scenarios illustrated. It will be apparent to those skilled in the art that various equivalent modifications or variations are possible in light of the examples shown in fig. 6 and 7, and such modifications or variations are intended to be included within the scope of the embodiments of the present application.

Having described the method for restoring metadata in a storage system according to an embodiment of the present application, an apparatus for restoring metadata in a storage system according to an embodiment of the present application will be described below.

Fig. 8 shows a schematic block diagram of an apparatus 800 for restoring metadata in a storage system according to an embodiment of the present application. Optionally, the memory system comprises a first bank of memory blocks CKG and a second CKG, the first CKG comprising at least one stripe and the second CKG comprising at least one stripe, the apparatus 800 is configured to perform the method or steps of the foregoing embodiments. Alternatively, the modules in the apparatus 800 may be implemented by software. Alternatively, the apparatus 800 may be deployed in a general-purpose computer device. As shown in fig. 8, the apparatus 800 includes:

an obtaining module 810, configured to obtain first data of a first stripe in a first CKG and second data of a second stripe in a second CKG;

a selecting module 820, configured to select data from the first data and the second data according to the valid column number of the first stripe, the valid column number of the second stripe, and the states of the first CKG and the second CKG if the first data and the second data are the same, where the states include a full-written state and a non-full-written state.

Optionally, the state of the first CKG is the same as the state of the second CKG, and the number of valid columns of the first stripe is different from the number of valid columns of the second stripe;

wherein the selecting module 820 is specifically configured to:

Optionally, the state of the first CKG is different from the state of the second CKG, and the number of valid columns of the first stripe is the same as the number of valid columns of the second stripe;

wherein the selecting module 820 is specifically configured to:

Optionally, the selecting module 820 comprises:

the calculating unit is used for calculating a weight corresponding to the first data according to the number of the valid columns of the first strip and the state of the first CKG; the second data processing module is further configured to calculate a weight corresponding to the second data according to the number of valid columns of the second stripe and the state of the second CKG;

and the comparison unit is used for comparing the weight of the first data with the weight of the second data and selecting the data with the maximum weight from the first data and the second data.

Optionally, the computing unit is specifically configured to:

W＝N*a+b，

W＝N*a；

W＝N*a，

if the state of the CKG where the stripe corresponding to the data is located is an unwritten state, calculating a weight according to the following formula:

W＝N*a-b，

It should be understood that the apparatus 800 according to the embodiment of the present application may be configured to perform the steps in the method for recovering metadata in the foregoing storage system, and the foregoing and other management operations and/or functions of the modules and/or units in the apparatus 800 are respectively for implementing the corresponding steps in the foregoing methods, so that the beneficial effects in the foregoing method embodiments may also be implemented, and for brevity, no further description is provided here.

Fig. 9 is a block diagram illustrating an apparatus 900 for restoring metadata in a storage system according to another embodiment of the present application. The apparatus 900 is, for example, a general purpose hardware computer device, or a storage device. The apparatus 900 for restoring metadata in the storage system shown in fig. 9 includes: a processor 901, a memory 902, and a transceiver 903.

The processor 901, memory 902 and transceiver 903 communicate with each other, passing control and/or data signals, over the internal connection paths. In one possible design, the processor 901, the memory 902, and the transceiver 903 may be implemented by chips. The memory 902 may store program codes, and the processor 901 calls the program codes stored in the memory 902 to implement the corresponding functions of the apparatus for restoring metadata in the storage system.

The processor 901 is configured to perform the following steps:

acquiring first data of a first strip in a first CKG and second data of a second strip in a second CKG; and if the first data and the second data are the same, selecting data from the first data and the second data according to the number of valid columns of the first stripe, the number of valid columns of the second stripe, the state of the first CKG and the state of the second CKG, wherein the states comprise a full-written state and a non-full-written state.

It will be appreciated that although not shown, the device 900 may also include other devices, such as an input device, an output device, a battery, etc.

In one possible design, in some embodiments, memory 902 may store instructions for performing the methods performed by the storage devices in the methods described above. The processor 901 may execute the instructions stored in the memory 902 to complete the steps executed by the storage device in the foregoing method in combination with other hardware (e.g., the transceiver 903), and specific working procedures and advantages may be referred to the description in the foregoing method embodiments.

An embodiment of the present application further provides a storage system, which includes the apparatus 900 in the foregoing embodiment.

The method disclosed in the embodiments of the present application may be applied to a processor, or may be implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The processor may be a general-purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, a system on chip (SoC), a Central Processing Unit (CPU), a Network Processor (NP), a Digital Signal Processor (DSP), a Microcontroller (MCU), a programmable logic controller (PLD), or other integrated chip. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in a Random Access Memory (RAM), a flash memory, a read-only memory (ROM), a programmable ROM, an electrically erasable programmable memory, a register, or other storage media that are well known in the art. The storage medium is located in a memory, and a processor reads instructions in the memory and combines hardware thereof to complete the steps of the method.

It should also be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

It should also be understood that the term "and/or" herein is merely one type of association that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for recovering metadata in a storage system, wherein the storage system comprises a first storage block group CKG and a second storage block group CKG, the first storage block group CKG comprises at least one stripe, the second storage block group CKG comprises at least one stripe, the method comprising:

acquiring first data of a first stripe in the first memory block group CKG and second data of a second stripe in the second memory block group CKG;

if the first data is the same as the second data, selecting data from the first data and the second data according to the number of valid columns of the first stripe, the number of valid columns of the second stripe, the state of the first block group CKG and the state of the second block group CKG, wherein the states include a full write state and a non-full write state, and the number of valid columns refers to: the stored data is verified to be valid storage blocks;

the state of the first memory block group CKG is the same as the state of the second memory block group CKG, and the number of valid columns of the first stripe is different from the number of valid columns of the second stripe; wherein selecting data from the first data and the second data according to the valid column number of the first stripe and the valid column number of the second stripe, and the state of the first block group CKG and the state of the second block group CKG includes: selecting data with the largest number of valid columns of the corresponding stripe from the first data and the second data;

or the state of the first block group CKG is different from the state of the second block group CKG, and the number of valid columns of the first stripe is the same as the number of valid columns of the second stripe; wherein selecting data from the first data and the second data according to the valid column number of the first stripe and the valid column number of the second stripe, and the state of the first block group CKG and the state of the second block group CKG includes: selecting data in which the state of the memory block group CKG is a write-full state, from the first data and the second data;

selecting data from the first data and the second data according to the valid column number of the first stripe and the valid column number of the second stripe, and the state of the first memory block group CKG and the state of the second memory block group CKG, and further comprising: calculating a weight corresponding to the first data according to the number of the effective columns of the first stripe and the state of the first memory block group CKG; calculating a weight corresponding to the second data according to the number of the valid columns of the second stripe and the state of the second memory block group CKG; and comparing the weight of the first data with the weight of the second data, and selecting the data with the maximum weight from the first data and the second data.

2. The method according to claim 1, wherein the calculating the weight corresponding to the first data or the calculating the weight corresponding to the second data comprises:

W＝N*a+b，

W＝N*a；

W＝N*a，

W＝N*a-b，

3. An apparatus for recovering metadata in a storage system, wherein the storage system comprises a first block group CKG and a second block group CKG, the first block group CKG comprises at least one stripe, the second block group CKG comprises at least one stripe, the apparatus comprising:

an obtaining module, configured to obtain first data of a first stripe in the first memory block group CKG and second data of a second stripe in the second memory block group CKG;

a selecting module, configured to select data from the first data and the second data according to the number of valid columns of the first stripe and the number of valid columns of the second stripe, and the state of the first block group CKG and the state of the second block group CKG if the first data and the second data are the same, where the states include a full write state and a non-full write state, and the number of valid columns refers to: the stored data is verified to be valid storage blocks;

the state of the first memory block group CKG is the same as the state of the second memory block group CKG, and the number of valid columns of the first stripe is different from the number of valid columns of the second stripe; the selecting module is configured to select data from the first data and the second data according to the valid column number of the first stripe, the valid column number of the second stripe, the state of the first block group CKG, and the state of the second block group CKG, and includes: selecting data with the largest number of valid columns of the corresponding stripe from the first data and the second data;

or the state of the first block group CKG is different from the state of the second block group CKG, and the number of valid columns of the first stripe is the same as the number of valid columns of the second stripe; the selecting module is configured to select data from the first data and the second data according to the valid column number of the first stripe, the valid column number of the second stripe, the state of the first block group CKG, and the state of the second block group CKG, and includes: selecting the state of the CKG in the first data and the second data as the data in the full writing state;

the selecting module is configured to select data from the first data and the second data according to the valid column number of the first stripe, the valid column number of the second stripe, the state of the first block group CKG, and the state of the second block group CKG, and further includes: the calculating unit is used for calculating a weight corresponding to the first data according to the number of the valid columns of the first stripe and the state of the first memory block group CKG; the second data processing module is further configured to calculate a weight corresponding to the second data according to the number of valid columns of the second stripe and the state of the second memory block group CKG; and the comparison unit is used for comparing the weight of the first data with the weight of the second data and selecting the data with the maximum weight from the first data and the second data.

4. The apparatus according to claim 3, wherein the computing unit is specifically configured to:

W＝N*a+b，

W＝N*a；

W＝N*a，

W＝N*a-b，