CN117806558A

CN117806558A - Data protection method, storage array controller and storage device

Info

Publication number: CN117806558A
Application number: CN202311861661.9A
Authority: CN
Inventors: 黄运新; 方浩俊
Original assignee: Shenzhen Dapu Microelectronics Co Ltd
Current assignee: Shenzhen Dapu Microelectronics Co Ltd
Priority date: 2023-12-28
Filing date: 2023-12-28
Publication date: 2024-04-02

Abstract

The embodiment of the application relates to the technical field of storage arrays and discloses a data protection method, a storage array controller and storage equipment.

Description

Data protection method, storage array controller and storage device

Technical Field

The present disclosure relates to the field of storage arrays, and in particular, to a data protection method, a storage array controller, and a storage device.

Background

Disk array (Redundant Arrays of Independent Disks, RAID) storage technology utilizes multiple storage devices grouped together to form a storage array, so that the multiple storage devices can operate simultaneously to improve data storage performance and provide data redundancy protection.

At present, a common RAID storage technology is to cut Data into a plurality of Data blocks after calculating check Data (Parity Data) of the Data by a RAID controller (RAIDcontroller), and write the cut plurality of Data blocks and the check Data into a memory of a storage array, and when one of the Data blocks has an error, calculate the check Data and other Data blocks without damage by reading to recover the error Data block, however, the problem exists in this way: when errors occur in the cut data blocks and the check data at the same time, the complexity of data recovery is increased, and the protection of the data is reduced.

Disclosure of Invention

The embodiment of the application provides a data protection method, a storage array controller and storage equipment, which solve the technical problem of low data protection force and increase the data protection force so as to increase the reliability of the data.

In order to solve the technical problems, the embodiment of the application provides the following technical scheme:

in a first aspect, an embodiment of the present application provides a data protection method applied to a storage array, where the storage array includes at least two memories, the method includes:

acquiring user data, wherein the user data is divided into at least two first data blocks;

calculating the check data corresponding to at least two first data blocks to generate first check blocks;

compressing the first data blocks to obtain second data blocks corresponding to the first data blocks one by one, and compressing the first check blocks to obtain second check blocks corresponding to the first check blocks one by one;

adding protection data to the second data block to generate a third data block, and adding protection data to the second check block to generate a third check block;

and storing the third data block to a corresponding memory, and storing the third check block to the corresponding memory.

In some embodiments, the method further comprises:

acquiring the data length of the second data block and acquiring the data length of the second check block;

adding protection data to the second data block to generate a third data block, comprising:

determining a data protection type corresponding to the second data block according to the data length of the second data block;

adding protection data to the second data block according to the data protection type corresponding to the second data block to generate a third data block;

adding protection data to the second check block to generate a third check block, including:

determining a data protection type corresponding to the second check block according to the data length of the second check block;

and adding protection data to the second check block according to the data protection type corresponding to the second check block to generate a third check block.

In some embodiments, the data protection types include a first data protection type, a second data protection type;

according to the data length of the second data block, determining the data protection type corresponding to the second data block includes:

when the data length of the second data block is greater than a preset length threshold, determining that the data protection type corresponding to the second data block is the first data protection type;

When the data length of the second data block is smaller than or equal to a preset length threshold value, determining that the data protection type corresponding to the second data block is the second data protection type;

according to the data length of the second check block, determining the data protection type corresponding to the second check block comprises the following steps:

when the data length of the second check block is greater than a preset length threshold, determining that the data protection type corresponding to the second check block is the first data protection type;

and when the data length of the second check block is smaller than or equal to a preset length threshold value, determining that the data protection type corresponding to the second check block is the second data protection type.

In some embodiments, the data length of the first protection data corresponding to the first data protection type is less than the data length of the second protection data corresponding to the second data protection type;

according to the data protection type corresponding to the second data block, adding protection data to the second data block to generate a third data block, including:

adding first protection data or second protection data to the second data block according to the first data protection type or the second data protection type to generate a third data block;

according to the data protection type corresponding to the second check block, adding protection data to the second check block to generate a third check block, including:

And adding the first protection data or the second protection data to the second check block according to the first data protection type or the second data protection type to generate a third check block.

In some embodiments of the present invention, in some embodiments,

the first data protection type corresponds to a first preset algorithm, and the method further comprises the steps of:

calculating first protection data according to a first preset algorithm and the data length of the second data block;

the second data protection type corresponds to a second preset algorithm, and the method further comprises:

and calculating second protection data according to a second preset algorithm and the data length of the second check block.

In some embodiments, the method further comprises:

reading user data from a memory;

decoding the user data according to the data protection type corresponding to the user data to obtain decoded user data;

if the decoded user data has error codes, judging whether the data protection type corresponding to the user data is a second data protection type;

if yes, correcting the error of the decoded user data to obtain corrected user data, and forwarding the corrected user data to a host;

if not, the decoded user data is forwarded to the host.

In some embodiments, after error correction of the decoded user data, the method further comprises:

Judging whether error correction is successful or not;

if the error correction is successful, forwarding the user data after the error correction to a host, and writing the user data after the error correction back into the memory;

if the error correction fails, the RAID calculation is carried out by reading other data blocks of the stripe corresponding to the user data through the storage array so as to recover the user data.

In a second aspect, embodiments of the present application provide a memory array controller, including: an algorithm engine, a compression engine and a data protection engine;

the algorithm engine is connected with the compression engine and is used for acquiring at least two first data blocks corresponding to the user data, calculating check data corresponding to the at least two first data blocks to generate a first check block, and sending the at least two first data blocks and the first check block to the compression engine;

the compression engine is connected with the algorithm engine and the data protection engine and is used for compressing the first data blocks to obtain second data blocks corresponding to the first data blocks one by one, compressing the first check blocks to obtain second check blocks corresponding to the first check blocks one by one, and sending the second data blocks and the second check blocks to the data protection engine;

the data protection engine is connected with the compression engine and is used for adding protection data to the second data block to generate a third data block, and adding protection data to the second check block to generate a third check block.

In some embodiments, the data protection engine comprises a data preprocessing engine;

the data preprocessing engine is used for acquiring the data length of the second data block, acquiring the data length of the second check block, determining the protection engine corresponding to the second data block according to the data length of the second data block, and determining the protection engine corresponding to the second check block according to the data length of the second check block.

In some embodiments, the data protection engine comprises a first protection engine and a second protection engine, wherein the first protection engine corresponds to a first preset algorithm, and the second protection engine corresponds to a second preset algorithm;

the data preprocessing engine is further used for determining that the protection engine corresponding to the second data block is a first protection engine when the data length of the second data block is greater than a preset length threshold value, and determining that the protection engine corresponding to the second data block is a second protection engine when the data length of the second data block is less than or equal to the preset length threshold value;

the data preprocessing engine is specifically further configured to determine that the protection engine corresponding to the second check block is the first protection engine when the data length of the second check block is greater than a preset length threshold, and determine that the protection engine corresponding to the second check block is the second protection engine when the data length of the second check block is less than or equal to the preset length threshold;

The data protection engine is further used for adding the first protection data or the second protection data to the second data block according to the first protection engine or the second protection engine to generate a third data block, and adding the first protection data or the second protection data to the second check block according to the first protection engine or the second protection engine to generate a third check block.

In some embodiments, the algorithm engine comprises a RAID algorithm engine;

the first protection engine comprises a DIF/DIX engine;

the second protection engine includes a DPECC engine.

In some embodiments of the present invention, in some embodiments,

the data protection engine is also used for reading the user data from the memory, decoding the user data according to the data protection type corresponding to the user data to obtain decoded user data, correcting errors of the decoded user data by the data protection engine if the error occurs in the decoded user data and the data protection type corresponding to the user data is the second data protection type, and forwarding the corrected user data to the compression engine;

the compression engine is also used for decompressing the user data after error correction and forwarding the decompressed user data to the algorithm engine so that the algorithm engine forwards the decompressed user data to the host interface.

In some embodiments of the present invention, in some embodiments,

the algorithm engine is further used for reading other data blocks of the stripe corresponding to the user data through the storage array to perform RAID calculation so as to recover the user data when the data protection type corresponding to the user data is the first data protection type;

the algorithm engine is further used for performing RAID calculation by reading other data blocks of the stripe corresponding to the user data through the storage array after the error correction failure of the decoded user data by the data protection engine so as to recover the user data.

In a third aspect, embodiments of the present application provide a storage device, including:

the storage array controller of the second aspect, wherein the storage array controller comprises a host interface and a storage device interface, the host interface for connecting to a host;

the memory array includes at least two memories, each of which is communicatively coupled to the memory array controller via a memory device interface.

The beneficial effects of the embodiment of the application are that: in a case different from the prior art, the data protection method provided in the embodiment of the present application is applied to a storage array, and the method includes: acquiring user data, wherein the user data is divided into at least two first data blocks; calculating the check data corresponding to at least two first data blocks to generate first check blocks; compressing the first data blocks to obtain second data blocks corresponding to the first data blocks one by one, and compressing the first check blocks to obtain second check blocks corresponding to the first check blocks one by one; adding protection data to the second data block to generate a third data block, and adding protection data to the second check block to generate a third check block; and storing the third data block to a corresponding memory, and storing the third check block to the corresponding memory.

Through compressing at least two data blocks and check blocks corresponding to user data, and respectively adding protection data to the compressed data blocks and check blocks, so that the data blocks and check blocks after the protection data are added are stored into corresponding memories.

Drawings

One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which the figures of the drawings are not to be taken in a limiting sense, unless otherwise indicated.

FIG. 1 is an exemplary schematic diagram of a stripe store provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of a connection relationship between a storage device and a host according to an embodiment of the present disclosure;

fig. 3 is a schematic structural diagram of a storage device according to an embodiment of the present application;

FIG. 4A is a schematic diagram of a memory array controller according to an embodiment of the present disclosure;

FIG. 4B is a schematic diagram of another memory array controller according to an embodiment of the present disclosure;

Fig. 5 is a schematic flow chart of a data protection method according to an embodiment of the present application;

fig. 6 is an exemplary schematic diagram of a second data block obtained after compressing a first data block according to an embodiment of the present application;

fig. 7 is a schematic diagram of a refinement flow of step S504 of fig. 5;

fig. 8 is a schematic diagram of the refinement flow of step S5041 of fig. 7;

fig. 9 is a schematic diagram of the refinement flow of step S5042 of fig. 7;

fig. 10 is a schematic diagram of a refinement flow of step S504 of fig. 5;

fig. 11 is a schematic diagram of the refinement flow of step S5043 of fig. 10;

fig. 12 is a schematic diagram of the refinement flow of step S5044 of fig. 10;

FIG. 13 is a flowchart of calculating first protection data according to an embodiment of the present application;

FIG. 14 is a flowchart of calculating second protection data according to an embodiment of the present application;

fig. 15 is an exemplary schematic diagram of compression and decompression of a data block and a check block according to an embodiment of the present application.

FIG. 16 is an exemplary schematic diagram of a third data block and a third parity block provided by an embodiment of the present application stored to a storage array;

FIG. 17 is a schematic diagram of a memory array controller according to an embodiment of the present disclosure;

FIG. 18 is a schematic diagram of a data protection engine according to an embodiment of the present disclosure;

FIG. 19 is a schematic diagram of the overall structure of a memory device according to an embodiment of the present application;

FIG. 20 is a schematic diagram of reading user data in a memory according to an embodiment of the present application;

FIG. 21 is a schematic diagram of another embodiment of a method for reading user data in a memory;

FIG. 22 is a schematic diagram of still another embodiment of a method for reading user data in a memory;

FIG. 23 is a schematic flow chart of reading user data in a memory according to an embodiment of the present application;

fig. 24 is a schematic flow chart of another method for reading user data in a memory according to an embodiment of the present application.

Reference numerals illustrate:

100	storage device	1232	First protection engine
				110	Memory array	1233	Second protection engine
120	Memory array controller	122	NVMe controller
				121	Data path	123	Storage device interface
1201	Algorithm engine	200	Host machine
				1202	Compression engine	201	Processor system
1203	Data protection engine	202	Dynamic random access memory
				1231	Data preprocessing engine	203	Storage interface
124	Host interface

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

It should be noted that, if not conflicting, the various features in the embodiments of the present application may be combined with each other, which is within the protection scope of the present application. In addition, while functional block division is performed in a device diagram and logical order is shown in a flowchart, in some cases, the steps shown or described may be performed in a different order than the block division in the device, or in the flowchart. Moreover, the words "first," "second," "third," and the like as used herein do not limit the data and order of execution, but merely distinguish between identical or similar items that have substantially the same function and effect.

In addition, technical features described below in the various embodiments of the present application may be combined with each other as long as they do not conflict with each other.

Prior to describing the embodiments of the present application, RAID storage techniques known to the inventors of the present application will be briefly described, so that the embodiments of the present application will be conveniently understood later.

RAID is a common storage technology that utilizes multiple storage devices grouped together into a storage array that operates simultaneously to improve performance or to provide data redundancy protection.

Referring to fig. 1, fig. 1 is an exemplary schematic diagram of a stripe storage according to an embodiment of the present application;

as shown in FIG. 1, A1-A3 form a stripe, ap is the parity data of A1-A3, B1-B3 form a stripe, bp is the parity data of B1-B3, C1-C3 form a stripe, cp is the parity data of C1-C3, D1-D3 form a stripe, dp is the parity data of D1-D3, the storage array comprises a plurality of memories, and each data block and parity data in each stripe is stored on a different memory.

The RAID storage technology mainly realizes the improvement of data security and storage performance through three operations, namely data striping, data verification and mirroring, and is specific:

data Striping (Data Striping): the continuous data are stored on different hard disks after being divided, namely, the data are distributed on the different hard disks in the form of stripes, so that the throughput of the system is improved, wherein the size of each stripe depends on the number and the type of the hard disks in the storage array.

Data verification (Data Parity): and calculating the Data in the storage array to calculate check Data (Parity Data) corresponding to the Data, when the Data is read, calculating whether the Data is correct according to the check Data, and when the Data on one hard disk is wrong, recovering the Data through the check Data and the Data on other hard disks, thereby improving the reliability of the Data.

Mirror (Mirroring): and the data is written into a plurality of hard disks at the same time so as to realize the complete backup of the data, thereby improving the reliability of the data and increasing the redundancy of storage.

The following specifically describes the technical scheme of the present application with reference to the drawings of the specification:

fig. 2 is a schematic diagram of a connection relationship between a storage device and a host according to an embodiment of the present application;

as shown in fig. 2, the storage device 100 includes a storage array 110 (storage array), a storage array controller 120, where the storage device 100 is communicatively connected to a host 200 through a wired interface for data interaction, where the wired interface includes a PCIe bus interface.

The storage array 110 is communicatively connected to the storage array controller 120, and the storage array 110 is used for storing host data and check data sent by the storage array controller 120, and the check data is used for detecting errors and correcting errors when errors occur in the host data. The storage array 110 includes a plurality of memories, and the memories include serial hard disks (SATA), and the plurality of SATA hard disks form a disk array (Redundant Arrays of Independent Disks, RAID), and the storage array 110 may be formed by a disk array. It is understood that the memory also includes a mechanical Hard Disk (HDD), a SCSI interface (Small Computer System Interface) -based Hard Disk, or the like for storing data.

The storage array controller 120 is communicatively connected to the storage array 110 and the host 200, and the storage array controller 120 is configured to process data, add protection measures to the host data and store the data to the storage array 110, provide redundancy protection to the host data to improve the security of the data of the storage array, and convert the data in the storage array 110 into a data format corresponding to the host 200 for sending to the host 200. Referring to fig. 3, fig. 3 is a schematic structural diagram of a memory device according to an embodiment of the present application;

as shown in fig. 3, the memory device 100 includes a memory array 110, a memory array controller 120.

The storage array 110 is communicatively connected with the storage array controller 120, and the storage array 110 is used for storing data, wherein the storage array 110 comprises a plurality of memories, and the memories comprise memories such as magnetic tapes and solid state disks for storing data. The storage array controller 120 includes a host interface for interfacing with a host.

The storage array controller 120 is connected to the storage array 110, and is configured to process data to generate check data corresponding to the data, where the check data is used to detect whether the data generates an error and data recovery (data recovery), and the storage array controller includes a RAID controller, which may be a hardware device, and the RAID controller includes a chip, for example: intel, AMD, NVIDIA, and the like, and a RAID controller is provided in a slot of a host, for example: PCI slots, PCIe slots, through the host's bus, for example: system PCIe Bus (PCIe Bus of System) for transferring host data to RAID controller for managing data storage and redundancy by RAID controller, RAID controller further comprising multiple storage device interfaces, when storage device is SATA hard disk, RAID controller communicates with multiple SATA hard disk connections through SATA interfaces for storing data onto SATA hard disk; the RAID controller may also be a software program, and the software program corresponding to the RAID controller is run on an operating system of the host, and communicates with an external storage array through a specific hardware interface, for example: USB interfaces, PCIe interfaces, and the like.

Host 200 includes a processor system 201, a dynamic random access memory 202, and a storage interface 203.

The processor system 201 is communicatively connected with the dynamic random access memory 202 and the storage interface 203, and the processor system 201 is used for data inside the host 200 and controlling and coordinating operations among components inside the host 200.

The dynamic random access memory 202 is communicatively connected to the processor system 201 and the storage interface 203, where the dynamic random access memory 202 is configured to store data of the host 200, and can increase the access speed of the data to improve the performance of the host 200, and the dynamic random access memory 202 may include DDR memory, for example: double data rate synchronous dynamic random access memory (Double Data Rate synchronous dynamic random-access memory, DDR SDRAM), fourth generation double data rate synchronous dynamic random access memory (DDR 4 SDRAM), low power fourth generation double data rate synchronous dynamic random access memory (Low Power Double Data Rate Version SDRAM, LPDDR4 SDRAM), graphics double data rate dynamic random access memory (Graphics Double Data Rate SDRAM, GDDR SDRAM), low power dynamic random access memory (Low Power Double Data Rate SDRAM, LPDDR SDRAM), rambus Dynamic Random Access Memory (RDRAM), and the like. The DDR memory comprises DDR particles, wherein the DDR particles are provided with DDR space for caching host data.

The storage interface 203 is communicatively connected to the storage array controller 120, the processor system 201, and the dynamic random access memory 202, and is used for implementing data transmission between the storage device 100 and the host 200.

Referring to fig. 4A, fig. 4A is a schematic structural diagram of a memory array controller according to an embodiment of the present disclosure;

as shown in fig. 4A, the storage array controller 120 includes a data path 121, the data path 121 including an algorithm engine 1201, the algorithm engine 1201 being configured to generate check data for data of the host 200, for example: the algorithm engine 1201 receives data of the host 200, generates check data corresponding to the data based on the data of the host 200, adds the check data to the data of the host 200, and stores the check data in the storage array 110 to improve the security of the data of the host 200, and when the host 200 acquires the data from the storage array, transmits the data in the storage array 110 to the algorithm engine 1201 to check whether the data acquired from the storage array 110 generates an error, if no error is generated, the data is transmitted to the host 200, and if the data generates an error, the data generating an error is recovered.

Referring to fig. 4B again, fig. 4B is a schematic structural diagram of another memory array controller according to an embodiment of the present disclosure;

In this embodiment of the present application, since the data of the host 200 is stored in the storage array 110 in the stripe form, in fig. 4A, the check data is only generated for the data of the host 200, and no protection measures are added for each data block and the check data in the stripe corresponding to the data of the host 200, when the data block is in error, the data is restored by using the check data only, which cannot play a good role in protecting the data, and further corresponding protection measures need to be added for each data block to reduce the risk of data error, and meanwhile, corresponding protection measures are added for the check data to avoid the risk of leakage of the check data.

As shown in fig. 4B, the improvement of fig. 4B is based on fig. 4A, where the storage array controller 120 includes a data path 121, and the data path 121 includes an algorithm engine 1201, a compression engine 1202, and a data protection engine 1203.

The algorithm engine 1201 is connected to the compression engine 1202, and is configured to obtain user data divided into at least two first data blocks, calculate check data corresponding to the at least two first data blocks, generate a first check block, and send the at least two first data blocks and the first check block to the compression engine 1202.

The compression engine 1202 is connected to the algorithm engine 1201 and the data protection engine 1203, and is configured to add protection data to each of the first data block and the first check block, compress the protection data to generate a second data block corresponding to the first data block one to one, generate a second check block corresponding to the first check block one to one, and send the second data block and the second check block to the data protection engine 1203.

The data protection engine 1203 is connected to the algorithm engine 1201, and is configured to add protection data to the second data block and the second check block, generate a third data block and a third check block, and store the third data block and the third check block in the storage array 110.

In the embodiment of the application, protection measures are added to the user data and the verification data through the algorithm engine 1201, the compression engine 1202 and the data protection engine 1203, so that the protection strength of the data of the storage array can be improved.

In the embodiment of the application, the algorithm engine includes a RAID algorithm engine, where the RAID algorithm engine may configure different RAID levels (such as RAID0, RAID1, and RAID 5) according to requirements, and when the RAID algorithm engine configures RAID0 or RAID1, check data is not required to be generated on user data, data blocks in stripes corresponding to the user data are directly compressed by the compression engine 1202, and protection measures are further added to the data blocks by the data protection engine 1203.

Note that RAID0 is a method of distributing data over a plurality of memories, and RAID0 includes only striping operations; RAID1 is a method of fully copying data onto multiple memories, including mirroring and striping operations; RAID5 is a method of storing data and parity data across multiple memories in a decentralized manner. It will be appreciated that RAID0 and RAID1 do not need to generate parity data, and only need to compress user data and add protection measures when user data is processed through RAID0 or RAID 1.

Referring to fig. 5, fig. 5 is a flow chart of a data protection method according to an embodiment of the present application;

the data protection method is applied to a storage array, and the storage array comprises at least two memories.

As shown in fig. 5, the data protection method includes:

step S501: acquiring user data, wherein the user data is divided into at least two first data blocks;

specifically, the user data sent by the host is received to obtain the user data, where the user data may also originate from other systems or applications, for example: the central processing unit (Central Processing Unit, CPU) system may also be derived from a dynamic random access memory, from which the user data cached in the dynamic random access memory is retrieved.

Specifically, the acquired user data is striped to divide the user data into at least two first data blocks, the at least two first data blocks constituting one stripe.

In the embodiment of the application, the user data is divided into the plurality of data blocks, so that the plurality of data blocks are stored in different memories, redundant backup of the data can be realized, and the reliability and the availability of the data are ensured.

Step S502: calculating the check data corresponding to at least two first data blocks to generate first check blocks;

specifically, the check data of the user data divided into at least two first data blocks is calculated using a check Algorithm to generate a first check block, wherein the check Algorithm includes, but is not limited to, an exclusive or check Algorithm (ExclusiveOR, XOR), a cyclic redundancy check (Cyclic Redundancy Check, CRC), a Message-Digest Algorithm (md5), a hash Algorithm (including SM2, SM3, SM4, SM 9), a secure hash Algorithm (Secure Hash Algorithm, SHA), and the like. It should be noted that the above-mentioned verification algorithm is the prior art, and detailed description of calculating the verification data is not needed here, and reference may be made to the prior art specifically.

In an embodiment of the present application, the first check block and at least two first data blocks are stored in different memories at the same time.

In this embodiment of the present application, the first check block is used to check whether the user data is in error, that is, the first data block is modified or lost, and when the user data is divided into one of the at least two data blocks, the error occurs, the first data block can be recovered by calculating the other lossless first data block and the first check block.

Step S503: compressing the first data blocks to obtain second data blocks corresponding to the first data blocks one by one, and compressing the first check blocks to obtain second check blocks corresponding to the first check blocks one by one;

specifically, the first data block is compressed through a compression algorithm to obtain second data blocks corresponding to the first data blocks one by one, and the first check block is compressed through the compression algorithm to obtain second check blocks corresponding to the first check blocks one by one, wherein the storage space occupied by the second data blocks is smaller than that occupied by the first data blocks, and the storage space occupied by the second check blocks is smaller than that occupied by the first check blocks. It should be noted that, the compression algorithm is an existing compression algorithm, for example: transparent compression algorithms such as: zlib, gzip, bzip2, deflate, lz4, lzo, snpey, etc.

In this embodiment of the present application, a first data block and a first check block are compressed by a compression algorithm, where the first data block and the first check block are compressed by taking granularity as a unit, and the granularity is the minimum data unit, the smaller the granularity is, that is, the smaller the unit of granularity is, the smaller the space occupied by unit data is, where the type of granularity is the unit of granularity used by the compression algorithm, for example: the granularity is of the type byte, and the first data block of 4K size is compressed to 64B by a byte-based compression algorithm, i.e., the first data block of 4K size is subdivided to 64B.

It should be noted that the types of granularity corresponding to different compression algorithms are different, and the specific granularity type is adjusted according to the specific algorithm.

In the embodiment of the application, the first data block and the first check block are compressed to obtain the reserved space corresponding to the second data block and the reserved space corresponding to the second check block, wherein the reserved space corresponding to the second data block is a reserved space obtained by subtracting the storage space occupied by the second data block from the storage space occupied by the first data block, the reserved space corresponding to the second check block is a reserved space obtained by subtracting the storage space occupied by the second check block from the storage space occupied by the first check block, the reserved space is set to be unavailable for storing user data, and the reserved space is used for storing and protecting data for the second data block and the second check block so as to further protect the data security of each second data block and each second check block and improve the data security.

Referring to fig. 6 again, fig. 6 is an exemplary schematic diagram of a second data block obtained after compressing a first data block according to an embodiment of the present application;

as shown in fig. 6, after the first data block is compressed, a second data block is obtained, and a reserved space corresponding to the second data block is obtained, where the reserved space corresponding to the second data block is obtained by subtracting a storage space occupied by the second data block from a storage space occupied by the first data block.

Step S504: adding protection data to the second data block to generate a third data block, and adding protection data to the second check block to generate a third check block;

in this embodiment of the present application, protection data is added to the second data block, and before protection data is added to the second parity block, the data lengths of the second data block and the second parity block need to be acquired first.

In this embodiment of the present application, when compressing the first data block and the first check block to obtain the second data block and the second check block, compression information (Compress Information, CI) is generated, where the compression information includes a type of granularity and a number of granularity, and the number of granularity is a number of units corresponding to the type of granularity, for example: the first data block with the size of 4K is compressed 64B, the granularity is of the type of bytes, the granularity number is 64, and further, the length corresponding to the granularity type is multiplied by the granularity number, so that the data length of the second data block and the second check block can be obtained.

Specifically, compressed information corresponding to the second data block is obtained, the compressed information corresponding to the second data block comprises the type of granularity corresponding to the second data block and the number of granularity corresponding to the second data block, the length corresponding to the type of granularity corresponding to the second data block is multiplied by the number of granularity corresponding to the second data block to obtain the data length corresponding to the second data block, protection data is added to the second data block according to the data length corresponding to the second data block, and a third data block is generated, wherein the data length of the third data block is equal to the data length of the first data block.

Specifically, compression information corresponding to the second check block is obtained, the compression information corresponding to the second check block comprises the type of granularity corresponding to the second check block and the number of granularity corresponding to the second check block, the length corresponding to the type of granularity corresponding to the second check block is multiplied by the number of granularity corresponding to the second check block to obtain the data length corresponding to the second check block, protection data are added to the second check block according to the data length corresponding to the second check block, and a third check block is generated, wherein the data length of the third check block is equal to the data length of the first check block.

In the embodiment of the application, the protection data is used for ensuring the integrity and the accuracy of the second data block and the second check block in the transmission process.

Referring to fig. 7 again, fig. 7 is a schematic diagram of the refinement flow of step S504 in fig. 5;

as shown in fig. 7, the step S504 includes:

step S5041: determining a data protection type corresponding to the second data block according to the data length of the second data block;

in the embodiment of the application, the data protection types include a first data protection type and a second data protection type, and the first data protection type and the second data protection type are used for adding protection data to the second data block and the second check block so as to add protection measures to the second data block and the second check block, so that the second data block and the second check block are prevented from being tampered in the transmission process. The first data protection type corresponds to a first preset algorithm, the second data protection type corresponds to a second preset algorithm, the first preset algorithm and the second preset algorithm can generate check codes, the check codes are used for detecting errors generated in the data transmission process, the data length of first protection data corresponding to the first data protection type is smaller than the data length of second protection data corresponding to the second data protection type, namely the length of check codes generated by the first preset algorithm is smaller than the length of check codes generated by the second preset algorithm, namely the check codes generated by the second preset algorithm need more reserved space to store.

Specifically, according to the data length of the second data block, the size of the reserved space remained after the first data block is compressed is determined, so as to determine the data protection type corresponding to the second data block.

Referring to fig. 8 again, fig. 8 is a schematic diagram of the refinement process of step S5041 in fig. 7;

as shown in fig. 8, this step S5041 includes:

step S5411: judging whether the data length of the second data block is larger than a preset length threshold value or not;

specifically, it is determined whether the data length of the second data block is greater than the preset length threshold, if the data length of the second data block is greater than the preset length threshold, the step is skipped to step S5412, and if the data length of the second data block is less than or equal to the preset length threshold, the step is skipped to step S5412.

Step S5412: determining the data protection type corresponding to the second data block as a first data protection type;

specifically, if the data length of the second data block is greater than the preset length threshold, that is, the reserved space corresponding to the second data block is smaller, that is, the reserved space can only store the check code with a shorter length, it is determined that the data protection type corresponding to the second data block is the first data protection type, so that the check code corresponding to the second data block is generated through the second data protection type.

Step S5413: determining the data protection type corresponding to the second data block as a second data protection type;

specifically, if the data length of the second data block is smaller than or equal to the preset length threshold, that is, the reserved space corresponding to the second data block is larger, that is, the reserved space can store the check code with a longer length, the data protection type corresponding to the second data block is determined to be the second data protection type, so that the check code corresponding to the second data block is generated through the second data protection type.

Step S5042: adding protection data to the second data block according to the data protection type corresponding to the second data block to generate a third data block;

specifically, according to the data protection type corresponding to the second data block, adding protection data to the second data block, namely if the data length of the second data block is determined to be greater than a preset length threshold, determining that the data protection type corresponding to the second data block is a first data protection type, generating protection data through the first data protection type, and after adding the protection data to the second data block, generating a third data block; if the data length of the second data block is smaller than or equal to the preset length threshold, determining that the data protection type corresponding to the second data block is the second data protection type, generating protection data through the second data protection type, and generating a third data block after adding the protection data to the second data block. Wherein the protection data comprises METAData (METAData) comprising a check code, an identification (Flag), which Flag comprises integrity protection information, key information, algorithms, etc.

In the embodiment of the application, the data protection force of each data block can be increased by adding the protection data to each data block, the protection effectiveness of the data is improved, and the risk of generating errors of the data is reduced.

Referring to fig. 9 again, fig. 9 is a schematic diagram of the refinement process of step S5042 in fig. 7;

as shown in fig. 9, this step S5042 includes:

step S5421: adding first protection data or second protection data to the second data block according to the first data protection type or the second data protection type to generate a third data block;

specifically, if the data protection type corresponding to the second data block is the first data protection type, according to the first data protection typeCalculating first protection data of the second data block according to a corresponding first preset algorithm, adding the first protection data into the second data block, and generating a third data block; if the data protection type corresponding to the second data block is the second data protection type, calculating second protection data of the second data block according to a second preset algorithm corresponding to the second data protection type, adding the second protection data to the second data block, and generating a third data block, wherein the data length of the first protection data is smaller than that of the second protection data. The first preset algorithm comprises the following steps: DIF/DIX algorithm, second preset algorithm comprises Differential Private Exponential Constant Compounding Code (DPECC), DPECC comprises ErrorCorrectionCode (ECC) algorithm based on Reed-Solomon (RS), BCH algorithm (BCH) _。

Referring to fig. 10 again, fig. 10 is a schematic diagram of a refinement flow of step S504 in fig. 5;

as shown in fig. 10, this step S504 includes:

step S5043: determining a data protection type corresponding to the second check block according to the data length of the second check block;

specifically, according to the data length of the second check block, the size of the reserved space reserved after the first check block is compressed is determined, so as to determine the data protection type corresponding to the second check block.

Referring to fig. 11 again, fig. 11 is a schematic diagram of the refinement flow of step S5043 in fig. 10;

as shown in fig. 11, this step S5043 includes:

step S5431: judging whether the data length of the second check block is larger than a preset length threshold value or not;

specifically, it is determined whether the data length of the second check block is greater than a preset length threshold, if the data length of the second check block is greater than the preset length threshold, the step is skipped to step S5432, and if the data length of the second check block is less than or equal to the preset length threshold, the step is skipped to step S5432.

Step S5432: determining the data protection type corresponding to the second check block as the first data protection type;

specifically, if the data length of the second check block is greater than the preset length threshold, that is, the reserved space corresponding to the second check block is smaller, that is, the reserved space can only store check codes with shorter length, the data protection type corresponding to the second check block is determined to be the first data protection type, so that the check code corresponding to the second check block is generated through the second data protection type.

Step S5433: determining the data protection type corresponding to the second check block as a second data protection type;

specifically, if the data length of the second check block is smaller than or equal to the preset length threshold, that is, the reserved space corresponding to the second check block is larger, that is, the reserved space can store check codes with longer length, the data protection type corresponding to the second check block is determined to be the second data protection type, so that the check code corresponding to the second check block is generated through the second data protection type.

Step S5044: adding protection data to the second check block according to the data protection type corresponding to the second check block to generate a third check block;

specifically, adding protection data to the second check block according to the data protection type corresponding to the second check block, namely if the data length of the second check block is determined to be greater than a preset length threshold value, determining the data protection type corresponding to the second check block as a first data protection type, generating protection data through the first data protection type, and generating a third check block after adding the protection data to the second check block; if the data length of the second check block is smaller than or equal to the preset length threshold, determining that the data protection type corresponding to the second check block is the second data protection type, generating protection data through the second data protection type, and generating a third check block after adding the protection data to the second check block.

In this embodiment of the present application, the third data block includes a second data block and protection data, when the sum of the data length of the second data block and the data length of the protection data is smaller than the data length of the first data block, that is, the protection data in the third data block does not fully occupy the reserved space corresponding to the second data block, virtual data is added in the reserved space corresponding to the second data block to fill the reserved space corresponding to the second data block, so that the data length of the third data block is equal to the length of the first data block, where the data length of the third data block=the data length of the second data block+the data length of the protection data+the data length of the virtual data.

In the embodiment of the application, the virtual data is used for processing the missing value, the abnormal value or the data which cannot be directly observed, so that the integrity and the accuracy of the data can be improved.

In the embodiment of the application, the protection data is added to the second check block so as to prevent the second check block from being tampered, so that the safety of the data can be further improved.

Referring to fig. 12 again, fig. 12 is a schematic diagram of the refinement process of step S5044 in fig. 10;

as shown in fig. 12, this step S5044 includes:

Step S5441: adding first protection data or second protection data to the second check block according to the first data protection type or the second data protection type to generate a third check block;

specifically, if the data protection type corresponding to the second check block is the first data protection type, calculating and encoding to generate first protection data of the second check block according to a first preset algorithm corresponding to the first data protection type, and adding the first protection data to the second check block to generate a third check block; if the data protection type corresponding to the second data block is the second data protection type, calculating second protection data of the second check block generated by coding according to a second preset algorithm corresponding to the second data protection type, and adding the second protection data to the second check block to generate a third check block, wherein the data length of the first protection data is smaller than that of the second protection data.

Referring to fig. 13 again, fig. 13 is a schematic flow chart of calculating first protection data according to an embodiment of the present application;

as shown in fig. 13, calculating the first protection data includes:

step S1301: calculating first protection data according to a first preset algorithm and the data length of the second data block;

In the embodiment of the application, a DIF/DIX algorithm is taken as an example, where the DIF/DIX algorithm adopts an end-to-end protection protocol, and the end-to-end protection protocol is a protocol for ensuring that data can be sufficiently protected and encrypted in a transmission process, and the end-to-end protection protocol includes SSL/TLS protocols (Secure Sockets Layer, SSL; transport Layer Security, TLS), IPsec protocols, zero Touch Encryption (ZTE).

Specifically, according to the data length of the second data block, calculating a difference value between the data length of the first data block and the data length of the second data block, presetting a check code length corresponding to a first preset algorithm according to the difference value, taking the preset check code length corresponding to the first preset algorithm as a parameter of a DIF/DIX algorithm, and generating first protection data through encoding of the DIF/DIX algorithm, wherein the length of the first protection data is identical to the preset check code length corresponding to the first preset algorithm.

For example: the data length of the first data block is 64B, the data length of the second data block is 32B, and if the difference between the data length of the first data block and the data length of the second data block is 32B, the check code length may be preset to be 32B.

In the embodiment of the present application, when the first protection data is generated for the second check block by using the first data protection type, the principle thereof is the same as that of calculating the first protection data of the second data block, and will not be described in detail herein.

In this embodiment of the present application, given that the preset check code length is known, when the data length of the first data block is not an integer multiple of the data length of the second data block, the first protection data is added to the second data block at this time, and the data length of the generated third data block is not equal to the data length of the first data block, then the difference between the data length of the first data block and the data length of the second data block is subtracted by the preset check code length to obtain a virtual data length, the virtual data length is complemented by using the virtual data, and then the first protection data is added to the virtual data, for example: the data length of the first data block is 64B, the data length of the second data block is 28B, the difference between the data length of the first data block and the data length of the second data block is 36B, the preset check code length is 32B, the virtual data length is 36B-32 b=4b, and the virtual data length is filled by virtual data filling. The virtual data is simulated or tested data, is used for processing missing values, abnormal values or data which cannot be directly observed, and can improve the integrity and accuracy of the data, and the virtual data comprises Dummy data.

Referring to fig. 14 again, fig. 14 is a flowchart illustrating a process of calculating second protection data according to an embodiment of the present application;

as shown in fig. 14, calculating the second protection data includes:

step S1401: calculating second protection data according to a second preset algorithm and the data length of a second check block;

in the embodiment of the present application, taking a Reed-Solomon (RS) -based ErrorCorrectionCode (ECC) algorithm as an example, the second protection data is calculated.

Specifically, according to the data length of the second data block, calculating a difference value between the data length of the first check block and the data length of the second check block, according to the difference value, presetting a check code length corresponding to a second preset algorithm, taking the preset check code length corresponding to the second preset algorithm as a parameter of a ErrorCorrectionCode (ECC) algorithm based on a Reed-Solomon (RS), and generating second protection data through ErrorCorrectionCode (ECC) algorithm coding based on the Reed-Solomon (RS), wherein the length of the second protection data is the same as the preset check code length corresponding to the second preset algorithm.

In the embodiment of the present application, when the second protection type is used to generate the first protection data for the second data block, the principle is the same as that of calculating the second protection data of the second check block, and will not be described in detail herein.

In this embodiment of the present application, when the data length of the first check block is not an integer multiple of the data length of the second check block, virtual data padding is used, so that the data length of the third check block is equal to the data length of the first check block, where the third check block is composed of the second data block, the virtual data, and the second protection data.

In the embodiment of the application, the data security and reliability of each data block and each check block can be improved by adding the protection data to each second data block and each second check block.

Referring to fig. 15 again, fig. 15 is a schematic diagram illustrating an example of compression and decompression of a data block and a check block according to an embodiment of the present application.

As shown in fig. 15, the algorithm engine corresponds to a check algorithm, divides user data into three first data blocks, calculates the three first data blocks to obtain first check blocks corresponding to the three first data blocks through the check algorithm, sends the three first data blocks and the first check blocks to the compression engine, compresses each first data block and each first check block through the compression algorithm corresponding to the compression engine to obtain second data blocks and second check blocks, sends a plurality of second data blocks and the second check blocks to the data protection engine, adds protection data to the second data blocks and the second check blocks through the protection algorithm corresponding to the data protection engine, and supplements the second data blocks and the second check blocks with virtual data to obtain third data blocks and third check blocks, wherein the third data blocks are composed of the second data blocks, the virtual data and the first protection data/the second protection data, and the third check blocks are composed of the first check blocks, the virtual data and the first protection data/the second protection data.

Step S505: storing the third data block to a corresponding memory, and storing the third check block to a corresponding memory;

specifically, the third data block to which the protection data is added is stored in the memory, and the third check block to which the protection data is added is stored in the memory.

Referring to fig. 16 again, fig. 16 is an exemplary schematic diagram of storing a third data block and a third parity block in a storage array according to an embodiment of the present application;

as shown in fig. 16, A1, A2, A3, B1, B2, B3, C1, C2, C3, D1, D2, D3 are all first data blocks, A1-A3, B1-B3, C1-C3, D1-D3 respectively form a stripe, ap is a second data block compressed by the first check block corresponding to A1-A3, bp is a second check block compressed by the first check block corresponding to B1-B3, cp is a second check block compressed by the first check block corresponding to C1-C3, dp is a second check block compressed by the first check block corresponding to D1-D3, dp includes dummy data and protection data, a11-a31 is a second data block, B11-B31 is a second data block, C11-C31 is a second data block, D11-D31 is a second data block, cp is a second check block and a third check block, dp is a second data block compressed by the first check block corresponding to C1-C3, dp is a third data block, and a third data array is formed in the memory array, wherein the memory array is formed by storing multiple data in the memory array, and the memory array is stored in the memory array, and the memory is 4 and the memory is stored in the array.

It should be noted that, in fig. 16, only dividing each user data into 3 data blocks and adding protection data to each data block is taken as an example, the specific division of the user data into a plurality of data blocks may be determined according to specific requirements, and the amount of memory included in the memory array 110 may be determined according to specific requirements, which is not limited herein.

In the embodiment of the application, the protection data is added to each data block in the stripe corresponding to each user data, so that errors of the user data in the transmission process can be avoided, and the protection degree of the data is improved.

In this embodiment of the present application, when user data is acquired from a storage array, decoding the data in the storage array is required, which specifically includes: decoding the third data block specifically includes:

obtaining a third data block corresponding to user data from a storage array, decoding the third data block through an algorithm corresponding to protection data corresponding to the third data block, judging whether the third data block has an error code, deleting the protection data in the third data block to obtain a second data block if the third data block has no error code, decompressing the second data block by adopting a compression algorithm, and recovering the third data block with the error code if the third data block has the error code, which indicates that the third data block generates an error in the transmission process, wherein the method specifically comprises the following steps:

Judging the data protection type of the third data block, if the data protection type is the second data protection type, directly correcting the error of the third data block, and if the error correction is successful, completing the recovery of the third data block. And stripping and recovering the protection data of the third data block into a second data block, decompressing the second data block to obtain a first data block, and thus finishing decoding of the user data.

If the data protection type of the third data block is the first data protection type or the error correction fails when the second data protection type performs error correction, entering a RAID recovery flow, specifically including: and reading other data blocks of the stripe corresponding to the user data through the storage array, and performing RAID calculation to recover the user data, wherein the other data blocks comprise check blocks.

And acquiring other lossless third data blocks and third check blocks in the same stripe from the storage array, and recovering the third data blocks with the current error code.

The algorithm corresponding to the protection data corresponding to the third data block comprises a first preset algorithm and a second preset algorithm, and the first preset algorithm and the second preset algorithm are verification algorithms.

Referring to fig. 20 again, fig. 20 is a schematic diagram of reading user data in a memory according to an embodiment of the present application;

As shown in fig. 20, the user data a11 in the memory 1 is read, the data protection engine decodes the user data a11, and determines the correctness of the user data a11, if the user data a11 is normal, i.e. the user data a11 has no error code, the data A1 is sent to the compression engine, so that the compression engine decompresses the data A1 to obtain decompressed data A1, then the compression engine forwards the decompressed data A1 to the algorithm engine, and the algorithm engine transparently transmits the decompressed data A1 to the host interface for transmission to the host.

The above mentioned situation is that the user data a11 has no error code, and if the user data a11 has an error code, the user data a11 needs to be corrected by the data protection engine at this time.

Referring to fig. 21 again, fig. 21 is a schematic diagram of another embodiment of reading user data in a memory;

as shown in fig. 21, when the data protection engine decodes the user data a11, if the user data a11 has an error, the data protection engine corrects the error of the user data a11 to obtain corrected user data, and sends the corrected user data to the compression engine. After receiving the corrected user data A1, the compression engine decompresses the user data A1 after error correction to obtain decompressed user data A1, and sends the decompressed user data A1 to the algorithm engine, and the algorithm engine transparently transmits the decompressed data A1 to the host interface for transmission to the host.

It will be appreciated that the above-mentioned case is that the data protection engine is successful in error correction of the user data, and that the data protection engine is transmitting to the compression engine the user data that is successful in error correction. If the data protection engine is not corrected successfully, the algorithm engine is required to recover the data by utilizing the data blocks in other memories.

Specifically, referring to fig. 22 again, fig. 22 is a schematic diagram of still another embodiment of reading user data in a memory;

as shown in fig. 22, when an error occurs in the user data a11 in the memory 1, and the data protection engine cannot successfully correct the error of the user data a11, for example: the data protection algorithm is a non-ECC protection algorithm, or, if ECC recovery fails, the algorithm engine needs to use the data blocks of other memories to perform recovery operations, for example: by performing data recovery from the user data a21 in the memory 2, the user data a31 in the memory 3, and the parity data Ap in the memory 4, it can be understood that the user data a11, a21, a31 constitute one stripe, and the parity data Ap is the parity data of the stripe.

Referring to fig. 23 again, fig. 23 is a schematic flow chart of reading user data in a memory according to an embodiment of the present application;

As shown in fig. 23, a process of reading user data in a memory includes:

step S2301: user data of the storage array is read through a storage device interface and sent to a data protection engine;

specifically, the storage array controller reads user data of the storage array through the storage device interface, the user data including a third data block, such that the data protection engine receives the third data block.

Step S2302: the data protection engine receives user data and decodes the user data according to the data protection type;

specifically, the data protection engine receives user data, that is, the third data block, and decodes the third data block according to the data protection type corresponding to the third data block, for example: if the data protection type corresponding to the third data block is the first data protection type, decoding the third data block through a first preset algorithm to obtain a decoded third data block; and if the data protection type corresponding to the third data block is the second data protection type, decoding the third data block through a second preset algorithm to obtain a decoded third data block.

Step S2303: whether there is an error code;

specifically, judging whether the decoded third data block has an error code, namely whether the decoded user data has an error or not;

If the decoded user data has an error code, the step S2307 is performed;

if the decoded user data has no error, the process proceeds to step S2304.

Step S2304: the data protection engine forwards the data to the compression engine;

specifically, if the decoded user data does not have error codes, the data protection engine deletes the protection data in the third data block to obtain a second data block corresponding to the third data block, and forwards the second data block to the compression engine.

Step S2305: the compression engine receives the data, decompresses the data and forwards the data to the algorithm engine;

specifically, the compression engine receives a second data block sent by the data protection engine, decompresses the second data block to obtain a first data block corresponding to the second data block, and forwards the first data block to the algorithm engine.

Step S2306: the algorithm engine receives the data and directly forwards the data to the host interface;

specifically, the algorithm engine receives the first data block sent by the compression engine and forwards the first data block to the host through the host interface, so that the host receives the first data block, namely decompressed user data.

Step S2307: whether it is of the second data protection type;

specifically, it is determined whether the third data block corresponds to a second data protection type, where the second data protection type corresponds to an error-correctable algorithm, for example: ECC algorithm, BCH algorithm, at this time, the data protection engine can correct errors of the data through error correction algorithm.

If the third data block corresponds to the second data protection type, step S2308 is performed;

if the third data block does not correspond to the second data protection type, for example: the third data block corresponds to the first data protection type, and the process proceeds to step S2310.

Step S2308: the data protection engine performs error correction;

specifically, if the third data block corresponds to the second data protection type, the data protection engine performs data error correction on the third data block, and specifically, the data protection engine performs data error correction on the third data block through an error correction algorithm, for example: and performing data error correction through an ECC algorithm and a BCH algorithm.

Step S2309: whether error correction is successful;

specifically, it is determined whether the data protection engine successfully corrects the error of the third data block, and if so, the process proceeds to step S2304 and step S2311.

Specifically, if the data protection engine performs error correction on the third data block to obtain user data after error correction, the data protection engine forwards the user data after error correction to the compression engine, the compression engine receives the user data after error correction, decompresses the data, forwards the user data after decompression to the algorithm engine, and the algorithm engine returns the user data after decompression to the host through the host interface.

Step S2310: entering a RAID recovery flow;

specifically, if the data protection engine fails to correct the error of the third data block, entering a RAID recovery flow, where the RAID recovery flow specifically includes: and reading other data blocks of the stripe corresponding to the user data through the storage array to perform RAID calculation so as to recover the user data, wherein the other data blocks comprise check blocks, for example: referring to fig. 22, the specific flow of recovering the second data block A1 by the data blocks a21, a31 in the memory 2, the memory 3, and the check block Ap in the memory 4, respectively, includes:

(1) Acquiring a third data block A21 in a memory 2, a third data block A31 in a memory 3 and a third check block Ap in a memory 4, wherein a stripe is formed by a third data block A11 in the memory 1, the third data block A21 in the memory 2 and the third data block A31 in the memory 3, and the third check block Ap in the memory 4 is a check block corresponding to the third data block A11 in the memory 1, the third data block A21 in the memory 2 and the third data block A31 in the memory 3;

(2) The data protection engine decodes the data of the third data block a21 in the memory 2, the third data block a31 in the memory 3 and the third check block Ap in the memory 4 to judge whether the error code occurs, and if the error code does not occur, the data protection engine deletes the protection data DP in the third data block a21 in the memory 2, the third data block a31 in the memory 3 and the third check block Ap in the memory 4 respectively to obtain a second data block A2 corresponding to the third data block a21, a second data block A3 corresponding to the third data block a31 and a second check block Ap corresponding to the third check block Ap.

It will be appreciated that if any of the third data block a21 in the memory 2, the third data block a31 in the memory 3, and the third parity block Ap in the memory 4 has an error code, the RAID recovery fails, and the algorithm engine returns a data error to the host.

(3) The compression engine decompresses the second data block A2 corresponding to the third data block A21, the second data block A3 corresponding to the third data block A31 and the second check block Ap corresponding to the third check block Ap respectively to obtain a first data block A '2 corresponding to the second data block A2, a first data block A '3 corresponding to the second data block A3 and a first check block A ' p corresponding to the second check block Ap.

(4) And the algorithm engine recovers the first data block A '1 corresponding to the third data block A11 according to the first data block A'2, the first data block A '3 and the first check block A' p.

(5) The algorithm engine sends the first data block a'1 to the host.

Specifically, the algorithm engine sends the first data block a '1 to the host interface through the NVMe controller, so that the host obtains the first data block a'1.

Step S2311: directly writing the correct data back to the original storage array;

specifically, if the data protection engine performs error correction on the third data block successfully, the data protection engine writes back the user data after error correction to the memory corresponding to the storage array, so that the memory stores correct user data. It is understood that the write-back process belongs to overwrite, and is consistent with the write-in process, and will not be described in detail herein.

Referring to fig. 24 again, fig. 24 is a schematic flow chart of another method for reading user data in a memory according to an embodiment of the present application;

as shown in fig. 24, a process of reading user data in a memory includes:

step S2401: user data of the storage array is read through the storage interface and forwarded to the data protection engine;

step S2402: the data protection engine receives user data and decodes the user data according to the data protection type;

step S2403: whether there is an error code;

step S2404: the data protection engine forwards the data to the compression engine;

step S2405: the compression engine receives the data, decompresses the data and forwards the data to the algorithm engine;

step S2406: the algorithm engine receives data;

step S2407: whether all other stripe data is received;

specifically, the algorithm engine determines whether all other stripe data is received, and if not, returns to step S2401 to continue to acquire other stripe data until all other stripe data is acquired. It will be appreciated that all other stripe data, including a third data block and a third parity block on the same stripe, need to be acquired to recover the erroneous third data block.

Step S2408: the algorithm engine performs RAID calculation, recovers target data and forwards the target data to the host interface;

the target data refers to uncompressed data corresponding to original user data, namely, third data block with error, namely, first data block in the storage array.

Step S2409: whether it is of the second data protection type;

step S2410: the data protection engine performs error correction;

step S2411: whether error correction is successful;

step S2412: and reporting the RAID recovery flow failure to a host data error.

Specifically, if the RAID recovery flow fails, or the third data block does not correspond to the second data protection type, data recovery cannot be performed, for example: if a read error is encountered, if the DPECC is unable to correct errors or is of the DIX type, the entire recovery process fails, at which point a data error message is sent to the host by the algorithm engine.

The data with error codes is recovered through the data protection engine and/or RAID recovery flow, and the data reading can be better realized.

In the embodiment of the application, user data is obtained, wherein the user data is divided into at least two first data blocks; calculating the check data corresponding to at least two first data blocks to generate first check blocks; compressing the first data blocks to obtain second data blocks corresponding to the first data blocks one by one, and compressing the first check blocks to obtain second check blocks corresponding to the first check blocks one by one; adding protection data to the second data block to generate a third data block, and adding protection data to the second check block to generate a third check block; and storing the third data block to a corresponding memory, and storing the third check block to the corresponding memory.

Through compressing at least two data blocks and check blocks that correspond to user data respectively to add protection data to the data block and check block after compressing respectively, with data block and check block storage to corresponding memory after adding protection data, this application can increase the protection dynamics of storage array's data, in order to improve storage array's data security.

Referring to fig. 17, fig. 17 is a schematic structural diagram of a memory array controller according to an embodiment of the present disclosure;

as shown in fig. 17, the memory array controller 120 includes: an algorithm engine 1201, a compression engine 1202, and a data protection engine 1203.

The algorithm engine 1201 is connected to the compression engine 1202, and is configured to obtain at least two first data blocks corresponding to the user data, calculate check data corresponding to the at least two first data blocks, and send the at least two first data blocks and the check data to the compression engine 1202. The algorithm engine 1201 includes a RAID algorithm engine, which may configure a verification algorithm corresponding to different RAID levels according to requirements, for example: RAID levels include RAID0, RAID1, RAID5, and the like.

The compression engine 1202 is connected with the algorithm engine 1201 and the data protection engine 1203, and is configured to compress the first data block to obtain a second data block corresponding to the first data block one by one, compress the first check block to obtain a second check block corresponding to the first check block one by one, and send the second data block and the second check block to the data protection engine 1203.

The data protection engine 1203 is connected to the compression engine 1202, and is configured to add protection data to the second data block to generate a third data block, and add protection data to the second check block to generate a third check block, and store the third data block to the storage array 110 through the storage device interface 123.

Referring to fig. 18, fig. 18 is a schematic structural diagram of a data protection engine according to an embodiment of the present application;

as shown in fig. 18, the data protection engine 1203 includes a data preprocessing engine 1231, a first protection engine 1232, and a second protection engine 1233.

The data preprocessing engine 1231 is configured to obtain a data length of the second data block, obtain a data length of the second check block, further determine a protection engine corresponding to the second data block according to the data length of the second data block, and determine the protection engine corresponding to the second check block according to the data length of the second check block.

The data preprocessing engine 1231 is further configured to determine that the protection engine corresponding to the second data block is the first protection engine when the data length of the second data block is greater than a preset length threshold, and determine that the protection engine corresponding to the second data block is the second protection engine when the data length of the second data block is less than or equal to the preset length threshold.

The data preprocessing engine 1231 is specifically further configured to determine that the protection engine corresponding to the second check block is the first protection engine when the data length of the second check block is greater than a preset length threshold, and determine that the protection engine corresponding to the second check block is the second protection engine when the data length of the second check block is less than or equal to the preset length threshold.

The first protection engine 1232 is configured to calculate first protection data according to a first preset algorithm and a data length of the second data block. The first protection engine comprises a DIF/DX engine, the DIF/DIX engine corresponds to a DIF/DIX mechanism, and the DIF/DIX engine is used for generating first protection data for the second data block, and the first protection data can detect whether the second data block is in error or not.

The second protection engine 1233 is configured to calculate second protection data according to a second preset algorithm and a data length of the second check block. The second protection engine 1233 includes a DPECC engine, where the DPECC engine corresponds to a second preset algorithm, and the second preset algorithm includes a plurality of verification algorithms, for example: the ECC algorithm, the DPECC engine, is used to generate second protection data for the second check block, the second protection data being able to detect if the second data block is erroneous and correct corrupted data.

In the embodiment of the present application, before user data is sent to the data path 121, the algorithm engine 1201 needs to be initialized, for example: determining the number of each user data divided into a plurality of data blocks, for example, determining to divide each user data into 3 data blocks, so as to calculate 3 data blocks corresponding to each user data to generate a check block, and initializing a Buffer (Buffer) to 0 to ensure that the Buffer is not used by other programs, wherein the Buffer is used for buffering the user data.

Further, the compression engine 1202 is initialized to determine the size of granularity, i.e., to determine the smallest compressed data unit, such that the user data is compressed based on the smallest compressed data unit.

Further, the data protection engine 1203 is initialized, and a protection engine corresponding to the protection data is set, that is, an algorithm corresponding to different data lengths is determined.

In the embodiment of the present application, by initializing the data path 121, the processing efficiency of the user data can be improved.

In the embodiment of the application, at least two data blocks and check blocks corresponding to user data are respectively compressed, and protection data are respectively added to the compressed data blocks and check blocks, so that the data blocks and check blocks after the protection data are added are stored in the corresponding memories.

Referring to fig. 19, fig. 19 is a schematic view of the overall structure of a memory device according to an embodiment of the present application;

as shown in fig. 19, the storage device 100 includes a storage array 110, a storage array controller 120, the storage array controller 120 includes a data path 121, an NVMe controller 122, a storage device interface 123, a host interface 124, and the host 200 includes a processor system 201, a dynamic random access memory 202, and a storage interface 203.

The data path 121 is connected to the NVMe controller 122 and the storage device interface 123, and is used for processing data of the host 200, adding protection data to the data, and storing the data added with the protection data into the storage array 110. The data path 121 includes an algorithm engine 1201, a compression engine 1202, and a data protection engine 1203, among others.

The algorithm engine 1201 is connected to the compression engine 1202, and is configured to obtain at least two first data blocks corresponding to user data, where the user data originates from the host 200, calculate check data corresponding to the at least two first data blocks, and send the at least two first data blocks and the check data to the compression engine 1202.

Specifically, when the data protection type corresponding to the user data is the first data protection type, the algorithm engine 1201 is further configured to read, through the storage array, other data blocks of the stripe corresponding to the user data, and perform RAID calculation to recover the user data;

The algorithm engine 1201 is further configured to, after the error correction failure of the decoded user data by the data protection engine, read, through the storage array, other data blocks of the stripe corresponding to the user data, and perform RAID calculation to recover the user data.

The compression engine 1202 is connected to the RAID algorithm engine 1201 and the data protection engine 1203, and is configured to compress a first data block to obtain a second data block corresponding to the first data block one to one, compress a first check block to obtain a second check block corresponding to the first check block one to one, and send the second data block and the second check block to the data protection engine 1203.

Specifically, the compression engine 1202 is further configured to decompress the user data after error correction, and forward the decompressed user data to the algorithm engine 1201, so that the algorithm engine 1201 forwards the decompressed user data to the host interface.

Specifically, the data protection engine 1203 is further configured to read the user data from the memory, decode the user data according to the data protection type corresponding to the user data, obtain decoded user data, and if the decoded user data has an error code, and the data protection type corresponding to the user data is the second data protection type, correct the error of the decoded user data, obtain corrected user data, and forward the corrected user data to the compression engine.

The NVMe controller 122 is communicatively connected to the storage interface 203 and the data path 121, and is used to implement data exchange and communication between the host 200 and the storage device 100, for example: receiving a data stream transmitted by the host 200 and transmitted by the storage interface 203, where the data stream is user data in the host 200, performing format conversion on the user data and transmitting the user data to the data path 121, so as to add protection data to the user data through the data path 121, and storing the user data to which the protection data is added in the storage array 110, where the NVMe controller 122 adopts an application level communication protocol, for example: hypertext transfer protocol (Hypertext Transfer Protocol, HTTP), file transfer protocol (File Transfer Protocol, FTP), simple mail transfer protocol (Simple Mail Transfer Protocol, SMTP), instant messaging protocol (Message Queuing Telemetry Transport, MQTT).

Storage device interface 123, communicatively coupled to storage array 110, data path 121, storage device interface 123 is configured to enable data exchange between storage device controller 120 and storage array 110, for example: receiving a data stream sent from the storage array 110, the data stream is data in the storage array 110, and format converting the data in the storage array 110, where when the storage array 110 includes multiple SATA hard disks, the storage device interface 123 may employ a SATA interface protocol, and communicate with the multiple SATA hard disks through the SATA interface protocol.

A host interface 124 for interfacing with host 200 to enable data and/or command interaction between storage array controller 120 and host 200, such as: the host interface 124 interfaces with the storage interface 203 of the host 200 to enable data and/or command interaction between the storage array controller 120 and the host 200. In the present embodiment, host interface 124 includes, but is not limited to, a USB interface, a PCIe interface, and the like.

Specifically, host 200 includes a processor system 201, a dynamic random access memory 202, and a storage interface 203.

The processor system 201 is communicatively connected to the dynamic random access memory 202 and the storage interface 203, and the processor system 201 is used for processing data inside the host 200 and controlling and coordinating operations among components inside the host 200.

The dynamic random access memory 202 is connected to the processor system 201 and the storage interface 203, and is used for caching data to increase the speed of processing the data by the host 200.

The storage interface 203 is communicatively connected to the processor system 201 and the dynamic random access memory 202, and is used for implementing data transmission between the storage device 100 and the host 200, where the storage interface 123 includes a PCIe controller and a physical layer (Port Physical Layer, PHY), and the PCIe controller adopts a PCIe interface protocol.

It should be noted that the algorithm engine 1201, the compression engine 1202 and the data protection engine 1203 mentioned above are further used to perform other related steps, and specific content may refer to related content mentioned in the above embodiment, which is not described herein again. In the embodiment of the application, user data is acquired, wherein the user data is divided into at least two first data blocks; calculating the check data corresponding to at least two first data blocks to generate first check blocks; compressing the first data blocks to obtain second data blocks corresponding to the first data blocks one by one, and compressing the first check blocks to obtain second check blocks corresponding to the first check blocks one by one; adding protection data to the second data block to generate a third data block, and adding protection data to the second check block to generate a third check block; and storing the third data block to a corresponding memory, and storing the third check block to the corresponding memory.

The apparatus or device embodiments described above are merely illustrative, in which the unit modules illustrated as separate components may or may not be physically separate, and the components shown as unit modules may or may not be physical units, may be located in one place, or may be distributed over multiple network module units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

From the above description of embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus a general purpose hardware platform, or may be implemented by hardware. Based on such understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the related art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., and include several instructions for up to a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method of each embodiment or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting thereof; the technical features of the above embodiments or in the different embodiments may also be combined under the idea of the present application, the steps may be implemented in any order, and there are many other variations of the different aspects of the present application as above, which are not provided in details for the sake of brevity; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of data protection applied to a storage array, the storage array comprising at least two memories, the method comprising:

2. The method according to claim 1, wherein the method further comprises:

the adding protection data to the second data block to generate a third data block includes:

3. The method of claim 2, wherein the data protection type comprises a first data protection type, a second data protection type;

the determining the data protection type corresponding to the second data block according to the data length of the second data block includes:

when the data length of the second data block is greater than a preset length threshold, determining that the data protection type corresponding to the second data block is a first data protection type;

when the data length of the second data block is smaller than or equal to a preset length threshold value, determining that the data protection type corresponding to the second data block is a second data protection type;

the determining the data protection type corresponding to the second check block according to the data length of the second check block includes:

when the data length of the second check block is greater than a preset length threshold, determining that the data protection type corresponding to the second check block is a first data protection type;

And when the data length of the second check block is smaller than or equal to a preset length threshold value, determining that the data protection type corresponding to the second check block is a second data protection type.

4. A method according to claim 3, wherein the data length of the first protection data corresponding to the first data protection type is smaller than the data length of the second protection data corresponding to the second data protection type;

and adding protection data to the second data block according to the data protection type corresponding to the second data block to generate a third data block, including:

adding the first protection data or the second protection data to the second data block according to the first data protection type or the second data protection type to generate a third data block;

and adding protection data to the second check block according to the data protection type corresponding to the second check block to generate a third check block, wherein the third check block comprises:

5. The method of claim 4, wherein the step of determining the position of the first electrode is performed,

The first data protection type corresponds to a first preset algorithm, and the method further comprises:

calculating the first protection data according to the first preset algorithm and the data length of the second data block;

and calculating the second protection data according to the second preset algorithm and the data length of the second check block.

6. The method according to claim 1, wherein the method further comprises:

reading user data from the memory;

if the decoded user data has error codes, judging whether the data protection type corresponding to the user data is a second data protection type or not;

if not, the decoded user data is forwarded to the host.

7. The method of claim 6, wherein after error correcting the decoded user data, the method further comprises:

Judging whether error correction is successful or not;

if error correction fails, reading other data blocks of the stripe corresponding to the user data through the storage array, and performing RAID calculation to recover the user data.

8. A memory array controller for use with a memory array, the memory array comprising at least two memories, the memory array controller comprising: an algorithm engine, a compression engine and a data protection engine;

the algorithm engine is connected with the compression engine and is used for acquiring at least two first data blocks corresponding to user data, calculating check data corresponding to the at least two first data blocks to generate a first check block, and sending the at least two first data blocks and the first check block to the compression engine;

9. The storage array controller of claim 8, wherein the data protection engine comprises a data preprocessing engine;

the data preprocessing engine is configured to obtain a data length of the second data block, obtain a data length of the second check block, determine a protection engine corresponding to the second data block according to the data length of the second data block, and determine the protection engine corresponding to the second check block according to the data length of the second check block.

10. The storage array controller of claim 9, wherein the data protection engine comprises a first protection engine and a second protection engine, the first protection engine corresponding to a first preset algorithm and the second protection engine corresponding to a second preset algorithm;

the data preprocessing engine is further configured to determine that the protection engine corresponding to the second data block is a first protection engine when the data length of the second data block is greater than a preset length threshold, and determine that the protection engine corresponding to the second data block is a second protection engine when the data length of the second data block is less than or equal to the preset length threshold;

The data preprocessing engine is specifically further configured to determine that the protection engine corresponding to the second check block is a first protection engine when the data length of the second check block is greater than a preset length threshold, and determine that the protection engine corresponding to the second check block is a second protection engine when the data length of the second check block is less than or equal to the preset length threshold;

the first protection engine is configured to calculate the first protection data according to the first preset algorithm and the data length of the second data block;

the second protection engine is configured to calculate the second protection data according to the second preset algorithm and the data length of the second check block.

11. The memory array controller of claim 8, wherein,

the data protection engine is further configured to read user data from the memory, decode the user data according to a data protection type corresponding to the user data, obtain decoded user data, and if the decoded user data has an error code, and the data protection type corresponding to the user data is a second data protection type, correct the decoded user data by the data protection engine to obtain corrected user data, and forward the corrected user data to the compression engine;

The compression engine is further configured to decompress the user data after error correction, and forward the decompressed user data to the algorithm engine, so that the algorithm engine forwards the decompressed user data to a host interface.

12. The memory array controller of claim 11, wherein,

the algorithm engine is further configured to, when the data protection type corresponding to the user data is a first data protection type, read other data blocks of the stripe corresponding to the user data through the storage array, and perform RAID calculation to recover the user data;

the algorithm engine is further configured to, after the error correction failure of the decoded user data by the data protection engine, read other data blocks of the stripe corresponding to the user data through the storage array, and perform RAID calculation to recover the user data.

13. A memory device, comprising:

the storage array controller of any of claims 8-12, wherein the storage array controller comprises a host interface and a storage device interface, the host interface to connect to a host;

and the storage array comprises at least two memories, and each memory is in communication connection with the storage array controller through the storage device interface.