CN113687978A - Data processing method for storage array controller - Google Patents
Data processing method for storage array controller Download PDFInfo
- Publication number
- CN113687978A CN113687978A CN202111044793.3A CN202111044793A CN113687978A CN 113687978 A CN113687978 A CN 113687978A CN 202111044793 A CN202111044793 A CN 202111044793A CN 113687978 A CN113687978 A CN 113687978A
- Authority
- CN
- China
- Prior art keywords
- data
- instruction
- raid
- storage
- host
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/16—Handling requests for interconnection or transfer for access to memory bus
- G06F13/1668—Details of memory controller
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/20—Handling requests for interconnection or transfer for access to input/output bus
- G06F13/28—Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access DMA, cycle steal
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention belongs to the field of data storage, and particularly relates to a data processing method for a storage array controller. The data processing method is implemented based on a data processing apparatus comprising: the system comprises a host, a RAID controller and a storage device; the RAID controller realizes data reading and writing operations on the storage equipment according to a data storage access instruction sent by the host, and completes the functions of redundancy check data calculation and data parallel transmission processing in the access operation process; the method can realize an independent RAID control function, and realize the multi-media parallel access and data redundancy protection functions of the storage array. The host data transmission adopts a special interface unit for processing and automatic data recombination, and the data transmission capability is improved. The functional units are independent from each other and are suitable for hardware implementation. Each processing unit can realize pipeline processing by scheduling of the processor, and the processing efficiency of the system is improved.
Description
Technical Field
The invention belongs to the field of data storage, and particularly relates to a data processing method for a storage array controller.
Background
Redundant Array of Independent Disks (RAID) enables multiple physical hard Disks to be combined into one logical hard disk, providing higher storage performance than a single hard disk by accessing multiple hard Disks in parallel. On the other hand, the storage array can provide data redundancy protection, encode data through a RAID structure, generate redundancy check information, and store in the storage array together with the original data for data recovery after a failure occurs. Different RAID levels are used in the storage array, mainly including RAID0, RAID1, RAID5, RAID6, and the like.
The RAID controller is a device which adopts special design to realize the function of the storage array, and can relieve the resource occupation of the array storage control operation on a CPU and an operating system of a server system. With the continuous increase of the computing and storing requirements of the host and the continuous increase of the read-write speed of the Solid State Disk (SSD), the computing power of the RAID controller directly affects the processing power and the processing efficiency of the entire storage system, and is likely to become a bottleneck that affects the processing efficiency of the storage system. Therefore, how to design and implement a more efficient storage array data processing method becomes a technical problem to be solved at present.
Disclosure of Invention
Technical problem to be solved
The technical problem to be solved by the invention is as follows: a data processing method and apparatus for a storage array controller is provided.
(II) technical scheme
In order to solve the above technical problem, the present invention provides a data processing method for a storage array controller, the data processing method being implemented based on a data processing apparatus, the data processing apparatus comprising: the system comprises a host, a RAID controller and a storage device; the host is equipment with data processing or computing capacity, and the data storage function of the host is connected with the storage equipment through the RAID controller; the RAID controller comprises a processor, a host transmission interface, a storage device interface, a memory and a RAID calculation unit, wherein the memory comprises at least one storage medium;
the RAID controller is used for receiving a data access instruction initiated by the host and carrying out data calculation and processing, and the access instruction comprises a reading instruction and a writing instruction;
if the access instruction is a write instruction, the data processing method performs processing according to the following steps:
step 101: the processor calls the host transmission interface to transmit the written instruction content to the memory, and performs instruction analysis to obtain an analyzed instruction;
step 102: the processor calls a host transmission interface, and transmits host data to be written to the allocated memory space according to the analyzed instruction content;
step 103: the processor completes calculation of the RAID array data space, determines the RAID mode and configures RAID calculation units;
step 104: the RAID computing unit reads host data in the memory according to the current RAID mode configuration;
step 105: the RAID calculation unit calculates redundancy check data of the host data;
step 106: the RAID calculation unit writes the redundancy check data into a memory and combines the redundancy check data into all storage array data in the memory;
step 107: the processor configures a storage device interface to split all storage array data, and the split data are respectively written into each storage medium in the storage device to be stored;
step 108: the processor completes the processing of the current write-in instruction;
if the access instruction is a read instruction, the data processing method performs processing according to the following steps:
step 201: the processor calls the host transmission interface to transmit the read instruction content to the memory, and performs instruction analysis to obtain an analyzed instruction;
step 202: the processor obtains the position of the data to be read in each storage medium of the corresponding storage equipment according to the analyzed instruction content;
step 203: the storage device interface respectively reads out the data in the storage medium into the memory space;
step 204: the processor judges the next processing step according to the state of the RAID array, if the RAID array is in a normal state, the step 208 is carried out; when the RAID array is in a degraded state, entering step 205;
step 205: reading the existing data in the memory by the RAID computing unit;
step 206: the RAID calculation unit calculates according to an RAID algorithm to obtain missing data;
step 207: the RAID computing unit writes the missing data back to the memory and combines the missing data into all data to be read out in the memory;
step 208: the processor calls a host transmission interface to transmit the required data to the corresponding position of the host memory according to the requirement of the reading instruction;
step 209: the processor completes the current instruction processing.
In step 204, the condition that the RAID array is in the degraded state includes: and in RAID5 mode, 1 storage medium can not be read normally.
The functions of all functional units in the RAID controller are mutually independent, and all units are processed in a pipeline mode in a processing flow; when a host sends a plurality of access instructions, the access instructions are divided into an instruction 1, an instruction 2 and an instruction 3 … … according to time sequence;
the host transmission interface receives the instruction of the instruction 1, reports the instruction to the processor to complete instruction analysis, transmits the written data to the storage space of the controller according to the instruction content, and then starts the RAID calculation unit to perform RAID calculation, namely, step 101, step 102, and step 103; in the calculation processes of step 104, step 105, and step 106, the host transmission interface may receive the instruction of instruction 2, and perform the analysis and transmission processing operations of step 101, step 102, and step 103;
after the RAID redundancy check data calculation of instruction 1 is completed, the processor starts the write operation of the storage device, that is, step 107 and step 108; at this time, the write data corresponding to the instruction 2 has already been transmitted to the storage space, so as to start the RAID calculation of the instruction 2, that is, the step 104, the step 105, and the step 106, and at the same time, the host transmission interface may also receive the instruction of the instruction 3, and perform the analysis and transmission processing operations, that is, the step 101, the step 102, and the step 103;
therefore, the RAID controller can simultaneously process a plurality of host instructions, so that the overall processing efficiency of the controller is improved.
In practical applications, the positions of data in the host memory are not necessarily continuous, but may be dispersed in different positions of the memory; wherein, the instruction corresponding to the RAID controller in step 101 stores instruction information in a linked list manner;
the host computer generates a plurality of storage access instructions in the memory according to the sequence of program function execution, taking instruction 1 as an example; the content in the instruction 1 address is the first address of an instruction 1 descriptor, and the instruction 1 descriptor comprises parameter information of the instruction 1, a data block list address and information of the total number m of data blocks; the instruction 1 parameter information comprises an instruction type, a logic address LBA in a logic hard disk and data length information; the data block list contains addresses and lengths of m data blocks, respectively.
After receiving the instruction 1, in step 102, the RAID controller invokes a host transmission interface, first transmits all instruction contents of an instruction list to a storage space of a storage of the RAID controller according to an instruction format, and then completes data transmission between a host memory transmission and the storage of the RAID controller according to the instruction contents;
the host transmission interface reads data by using a direct memory access mode; when the RAID controller sends a write instruction, the host transmission interface judges that data should be transmitted from a host memory to a storage of the RAID controller according to the type of the instruction, and allocates a corresponding storage space in the storage according to the total length of the data; firstly, acquiring the address and the length of a data block 1, and transmitting data at a corresponding position to a memory; after the transmission is finished, automatically reading the address and the length of the data block 2, and continuously transmitting the data at the corresponding position to the memory; after traversing all the data blocks m, completing all data transmission operations; the method can reduce the calculation burden of the RAID controller and improve the transmission efficiency; the read command is in the opposite direction to the above operation, and is not described herein again.
According to the RAID array format, continuous data blocks are divided into a plurality of strips for storage, and each strip is divided into a plurality of data units according to the number of RAID data media; in step 104, reading the content of the data unit by the RAID calculation unit according to a given RAID format, calculating RAID redundant data in step 105, and then writing redundant data 1 back to a set position in the stripe in step 106; the strip 2/3/4 operates in sequence, and finally the generation operation of all redundant data is completed; taking RAID5 format as an example, if 3 data disk spaces and 1 redundant check disk space are contained, the calculation formula is as follows:
redundant data 1 is data unit 1XOR data unit 2XOR data unit 3.
When the write data and the redundant data are prepared, the redundant data need to be dispersed to different storage media according to the RAID format requirement, so that the access amount of a single storage medium can be prevented from being remarkably increased; continuing to take the RAID5 format as an example, in step 107, according to a round robin manner, writing redundant data 1 into the storage medium 4, writing redundant data 2 into the storage medium 3, writing redundant data 3 into the storage medium 2, writing redundant data 4 into the storage medium 1, and placing other data units in the storage media in a normal order; and after the write-in corresponding relation is determined, the processor calls the storage device interface to execute the data write-in operation of the storage medium, and the write-in instruction of the RAID controller is completed.
Each unit included in the RAID controller is specifically implemented by hardware;
the implementation mode of each unit of the RAID controller comprises the following steps: by application specific integrated circuits, or by programmable logic devices;
the implementation mode of the PLD comprises a complex program logic device, a field programmable gate array, general array logic or any combination thereof;
wherein the memory may be a cache or an off-chip random access memory;
the storage medium is a carrier for storing data and comprises a solid state disk or a mechanical hard disk; the hard disk interface types comprise a serial connection small computer system interface hard disk and a serial advanced technology attachment hard disk.
The host and the RAID controller are communicated by adopting a high-speed transmission interface protocol, wherein the high-speed transmission interface protocol comprises a PCI, PCIe, NVMe, SATA, SAS, Ethernet or wireless transmission interface.
(III) advantageous effects
The invention discloses a data processing method and a data processing device for a storage array controller, which can realize the functions of data storage and RAID array management, effectively share the host computer computing pressure of a storage system and improve the overall computing performance of the storage system.
In a first aspect, the present invention provides a data processing method for use in a RAID controller including a storage array processing function. The RAID controller realizes data reading and writing operations on the storage device according to a data storage access instruction sent by the host, and completes the functions of redundancy check data calculation and data parallel transmission processing in the access operation process.
If the data is a data writing instruction, the RAID controller receives and temporarily stores the written data according to the instruction content, then calculates the redundant check data information of the written data, and writes the written data and the check data into the corresponding logical addresses in the storage medium respectively according to the RAID array configuration.
If the command is a data reading command, the RAID controller calculates the logical address of the data to be read in each storage medium according to the command content, recombines the data into complete data after acquiring the data, and sends the complete data to the host to finish data reading.
By implementing the invention, the RAID controller can generate redundancy check data, and the data integrity is protected under the condition of abnormal storage equipment or data. In addition, the data access instruction of the host is divided into a plurality of data access sub-instructions according to the determined rule, and different data access sub-instructions are simultaneously sent to a plurality of storage media for processing.
Each stage of the RAID controller processing flow may implement pipelining processing, taking a write instruction as an example, when the RAID controller calculates redundancy check data information of a first write instruction, a data receiving operation of a second write instruction may be started at the same time, and data is transmitted to an independent memory space; when the data of the first write command starts the write operation of the storage medium, the calculation operation of the redundancy check data information of the second write command can be started. By adopting the method, the overall RAID processing performance can be effectively improved, and the effect of multi-operation parallel processing can be achieved macroscopically.
The RAID controller can call the DMA to automatically process data according to a linked list form, and processor intervention is not needed according to the RAID format requirement.
In a second aspect, the present invention provides a data processing apparatus comprising functional modules or units for performing the method as described in the first aspect above or in any possible implementation of the first aspect.
In a third aspect, the present invention provides a computer program for a RAID control process, which when run on a RAID controller causes the controller to perform the method of the above aspects.
Compared with the prior art, the invention has the beneficial effects that:
(1) the independent RAID control function can be realized, and the multi-media parallel access and data redundancy protection functions of the storage array can be realized.
(2) The host data transmission adopts a special interface unit for processing and automatic data recombination, and the data transmission capability is improved.
(3) The functional units are mutually independent and are suitable for hardware implementation.
(4) Each processing unit can realize pipeline processing by scheduling of the processor, and the processing efficiency of the system is improved.
Drawings
FIG. 1 is a schematic diagram of a RAID storage system according to the present invention.
FIG. 2 is a schematic view of a RAID instruction processing flow according to the present invention.
FIG. 3 is a diagram illustrating the instruction content of the present invention.
FIG. 4 is a diagram illustrating RAID calculation in the memory of the present invention.
FIG. 5 is a schematic diagram of RAID storage according to the present invention.
Detailed Description
In order to make the objects, contents, and advantages of the present invention clearer, the following detailed description of the embodiments of the present invention will be made in conjunction with the accompanying drawings and examples.
In order to solve the above technical problem, the present invention provides a data processing method for a storage array controller, the data processing method being implemented based on a data processing apparatus, the data processing apparatus comprising: the system comprises a host, a RAID controller and a storage device; the host is equipment with data processing or computing capacity, and the data storage function of the host is connected with the storage equipment through the RAID controller; the RAID controller comprises a processor, a host transmission interface, a storage device interface, a memory and a RAID calculation unit, as shown in FIG. 1; the memory includes at least one storage medium;
the RAID controller is used for receiving a data access instruction initiated by the host and carrying out data calculation and processing, and the access instruction comprises a reading instruction and a writing instruction;
if the access instruction is a write instruction, the data processing method performs processing according to the following steps:
step 101: the processor calls the host transmission interface to transmit the written instruction content to the memory, and performs instruction analysis to obtain an analyzed instruction;
step 102: the processor calls a host transmission interface, and transmits host data to be written to the allocated memory space according to the analyzed instruction content;
step 103: the processor completes calculation of the RAID array data space, determines the RAID mode and configures RAID calculation units;
step 104: the RAID computing unit reads host data in the memory according to the current RAID mode configuration;
step 105: the RAID calculation unit calculates redundancy check data of the host data;
step 106: the RAID calculation unit writes the redundancy check data into a memory and combines the redundancy check data into all storage array data in the memory;
step 107: the processor configures a storage device interface to split all storage array data, and the split data are respectively written into each storage medium in the storage device to be stored;
step 108: the processor completes the processing of the current write-in instruction;
if the access instruction is a read instruction, the data processing method performs processing according to the following steps:
step 201: the processor calls the host transmission interface to transmit the read instruction content to the memory, and performs instruction analysis to obtain an analyzed instruction;
step 202: the processor obtains the position of the data to be read in each storage medium of the corresponding storage equipment according to the analyzed instruction content;
step 203: the storage device interface respectively reads out the data in the storage medium into the memory space;
step 204: the processor judges the next processing step according to the state of the RAID array, if the RAID array is in a normal state, the step 208 is carried out; when the RAID array is in a degraded state, entering step 205;
step 205: reading the existing data in the memory by the RAID computing unit;
step 206: the RAID calculation unit calculates according to an RAID algorithm to obtain missing data;
step 207: the RAID computing unit writes the missing data back to the memory and combines the missing data into all data to be read out in the memory;
step 208: the processor calls a host transmission interface to transmit the required data to the corresponding position of the host memory according to the requirement of the reading instruction;
step 209: the processor completes the current instruction processing.
In step 204, the condition that the RAID array is in the degraded state includes: and in RAID5 mode, 1 storage medium can not be read normally.
Wherein, the functions of each functional unit in the RAID controller are independent from each other, and each unit is processed in a pipeline manner in a processing flow, as shown in fig. 2; when a host sends a plurality of access instructions, the access instructions are divided into an instruction 1, an instruction 2 and an instruction 3 … … according to time sequence;
the host transmission interface receives the instruction of the instruction 1, reports the instruction to the processor to complete instruction analysis, transmits the written data to the storage space of the controller according to the instruction content, and then starts the RAID calculation unit to perform RAID calculation, namely, step 101, step 102, and step 103; in the calculation processes of step 104, step 105, and step 106, the host transmission interface may receive the instruction of instruction 2, and perform the analysis and transmission processing operations of step 101, step 102, and step 103;
after the RAID redundancy check data calculation of instruction 1 is completed, the processor starts the write operation of the storage device, that is, step 107 and step 108; at this time, the write data corresponding to the instruction 2 has already been transmitted to the storage space, so as to start the RAID calculation of the instruction 2, that is, the step 104, the step 105, and the step 106, and at the same time, the host transmission interface may also receive the instruction of the instruction 3, and perform the analysis and transmission processing operations, that is, the step 101, the step 102, and the step 103;
therefore, the RAID controller can simultaneously process a plurality of host instructions, so that the overall processing efficiency of the controller is improved.
In practical applications, the positions of data in the host memory are not necessarily continuous, but may be dispersed in different positions of the memory; one possible case is that the instruction corresponding to the RAID controller in step 101 stores instruction information in a linked list manner, as shown in fig. 3;
the host computer generates a plurality of storage access instructions in the memory according to the sequence of program function execution, taking instruction 1 as an example; the content in the instruction 1 address is the first address of an instruction 1 descriptor, and the instruction 1 descriptor comprises parameter information of the instruction 1, a data block list address and information of the total number m of data blocks; the instruction 1 parameter information comprises an instruction type, a logic address LBA in a logic hard disk and data length information; the data block list contains addresses and lengths of m data blocks, respectively.
After receiving the instruction 1, in step 102, the RAID controller invokes a host transmission interface, first transmits all instruction contents of an instruction list to a storage space of a storage of the RAID controller according to an instruction format, and then completes data transmission between a host memory transmission and the storage of the RAID controller according to the instruction contents;
the host transmission interface reads data in a Direct Memory Access (DMA) mode; for example, when the RAID controller sends a write instruction, the host transmission interface determines, according to the type of the instruction, that data should be transmitted from the host memory to the storage of the RAID controller, and allocates a corresponding storage space in the storage according to the total length of the data; firstly, acquiring the address and the length of a data block 1, and transmitting data at a corresponding position to a memory; after the transmission is finished, automatically reading the address and the length of the data block 2, and continuously transmitting the data at the corresponding position to the memory; after traversing all the data blocks m, completing all data transmission operations; the method can reduce the calculation burden of the RAID controller and improve the transmission efficiency; the read command is in the opposite direction to the above operation, and is not described herein again.
According to the RAID array format, continuous data blocks are divided into a plurality of stripes for storage, and each stripe is divided into a plurality of data units according to the number of RAID data media, as shown in fig. 4; in step 104, the RAID calculation unit reads the content of the data unit according to a given RAID format, in step 105, RAID redundant data is calculated, and then in step 106, redundant data 1 is written back to a set position in a strip in the figure; the strip 2/3/4 operates in sequence, and finally the generation operation of all redundant data is completed; taking RAID5 format as an example, if 3 data disk spaces and 1 redundant check disk space are contained, the calculation formula is as follows:
redundant data 1 is data unit 1XOR data unit 2XOR data unit 3.
When the write data and the redundant data are prepared, the redundant data need to be dispersed to different storage media according to the RAID format requirement, so that the access amount of a single storage medium can be prevented from being remarkably increased; continuing with the RAID5 format as an example, in step 107, according to a round-robin manner, redundant data 1 is written in the storage medium 4, redundant data 2 is written in the storage medium 3, redundant data 3 is written in the storage medium 2, redundant data 4 is written in the storage medium 1, and other data units are placed in the storage media in the normal order, as shown in fig. 5 below; and after the write-in corresponding relation is determined, the processor calls the storage device interface to execute the data write-in operation of the storage medium, and the write-in instruction of the RAID controller is completed.
In practical application, each unit included in the RAID controller may be specifically implemented by hardware or software code;
for example, the illustrated implementation of each unit of the RAID controller includes: implemented by an Application Specific Integrated Circuit (ASIC), or by a Programmable Logic Device (PLD);
the implementation manner of the PLD includes a Complex Programmable Logic Device (CPLD), a field-programmable gate array (FPGA), a General Array Logic (GAL), or any combination thereof;
meanwhile, the related functions of the RAID controller can also be realized by software, and the RAID controller and each unit thereof can also be a software unit.
Wherein the memory may be a cache or an off-chip random access memory;
the storage medium is a carrier for storing data and comprises a solid state disk or a mechanical hard disk; the hard disk interface types include Serial Attached Small computer system interface (SAS) hard disks, Serial Advanced Technology Attachment (SATA) hard disks.
The host and the RAID controller are communicated by adopting a high-speed transmission interface protocol, wherein the high-speed transmission interface protocol comprises a PCI, PCIe, NVMe, SATA, SAS, Ethernet or wireless transmission interface.
Example 1
In this embodiment, the host is a device with data processing or computing capability, and the data storage function of the host is connected to the storage device through the RAID controller. The RAID controller comprises a processor, a host transport interface, a storage device interface, a memory, and a RAID computation unit, as shown in figure 1. The memory may be a cache or an off-chip random access memory.
The storage device includes at least one storage medium. Storage medium refers to a carrier for storing data, which may include, but is not limited to, a solid state disk or a mechanical hard disk. The hard disk interface types include Serial Attached Small computer system interface (SAS) hard disks, Serial Advanced Technology Attachment (SATA) hard disks, and the like.
The host and the RAID controller usually adopt a high-speed transmission interface protocol to implement communication, including PCI, PCIe, NVMe, and the like, and are not limited to SATA, SAS, ethernet, or wireless transmission interface.
The RAID controller can receive data access instructions initiated by a host computer and perform data calculation and processing, wherein the access instructions comprise reading and writing instructions.
If the command is a write command, processing according to the following steps:
step 101: the processor calls the host transmission interface to transmit the written instruction content to the memory, and performs instruction analysis to obtain an analyzed instruction;
step 102: the processor calls a host transmission interface, and transmits host data to be written to the allocated memory space according to the analyzed instruction content;
step 103: the processor completes calculation of the RAID array data space, determines the RAID mode and configures RAID calculation units;
step 104: the RAID computing unit reads host data in the memory according to the current RAID mode configuration;
step 105: the RAID calculation unit calculates redundancy check data of the host data;
step 106: the RAID calculation unit writes the redundancy check data into a memory and combines the redundancy check data into all storage array data in the memory;
step 107: the processor configures a storage device interface to split all storage array data, and the split data are respectively written into each storage medium in the storage device to be stored;
step 108: the processor completes processing of the current write instruction.
If the command is a read command, processing according to the following steps:
step 201: the processor calls the host transmission interface to transmit the read instruction content to the memory, and performs instruction analysis to obtain an analyzed instruction;
step 202: the processor obtains the position of the data to be read in each storage medium of the corresponding storage equipment according to the analyzed instruction content;
step 203: the storage device interface respectively reads out the data in the storage medium into the memory space;
step 204: the processor judges the next processing step according to the state of the RAID array, if the RAID array is in a normal state, the step 208 is carried out; when the RAID array is in a degraded state, for example, 1 storage medium cannot be read normally in the RAID5 mode, step 205 is entered;
step 205: reading the existing data in the memory by the RAID computing unit;
step 206: the RAID calculation unit calculates according to an RAID algorithm to obtain missing data;
step 207: the RAID computing unit writes the missing data back to the memory and combines the missing data into all data to be read out in the memory;
step 208: the processor calls a host transmission interface to transmit the required data to the corresponding position of the host memory according to the requirement of the reading instruction;
step 209: the processor completes the current instruction processing.
The functions of the functional units in the RAID controller are independent of each other, and the units are processed in a pipeline manner in the processing flow, as shown in fig. 2. The host transmission interface receives the instruction of the instruction 1, reports the instruction to the processor to complete the instruction analysis, transmits the written data to the storage space of the controller according to the instruction content, and then starts the RAID calculation unit to perform RAID calculation, namely step 101, step 102 and step 103; in the calculation processes of step 104, step 105 and step 106, the host transmission interface can receive the instruction of instruction 2, and perform the analysis and transmission processing operations of step 101, step 102 and step 103;
after the RAID redundancy check data calculation of instruction 1 is completed, the processor starts the write operation of the storage device, that is, step 107 and step 108; at this time, the write data corresponding to the instruction 2 has already been transmitted to the storage space, so as to start the RAID calculation of the instruction 2, that is, step 104, step 105, and step 106, and the host transmission interface may also receive the instruction of the instruction 3, and perform the analysis and transmission processing operations, that is, step 101, step 102, and step 103; the RAID controller can realize simultaneous processing of a plurality of host instructions so as to improve the overall processing efficiency of the controller;
in practical applications, the location of data in the host memory is not necessarily continuous, but may be dispersed in different locations of the memory; one possible case is that the instruction corresponding to the RAID controller in step 101 stores instruction information in a linked list manner, as shown in fig. 3;
the host computer generates a plurality of storage access instructions in the memory according to the sequence of program function execution, taking instruction 1 as an example; the content in the instruction 1 address is the first address of an instruction 1 descriptor, and the instruction 1 descriptor comprises parameter information of the instruction 1, a data block list address and information of the total number m of data blocks; the instruction 1 parameter information comprises an instruction type, a logic address LBA in a logic hard disk and data length information; the data block list respectively comprises addresses and lengths of m data blocks;
after the RAID controller receives the instruction 1, in step 102, the processor calls a host transmission interface, first transmits all instruction contents of the instruction list to a storage space of a storage of the RAID controller according to an instruction format, and then completes data transmission between a host memory transmission and the storage of the RAID controller according to the instruction contents;
the host transport interface reads data using Direct Memory Access (DMA). For example, when the RAID controller sends a write command, the host transfer interface determines, according to the type of the command, that data should be transferred from the host memory to the storage of the RAID controller, and allocates a corresponding storage space in the storage according to the total length of the data. Firstly, the address and the length of the data block 1 are acquired, and the data at the corresponding position is transmitted to the memory. After the transmission is finished, the address and the length of the data block 2 are automatically read, and the data at the corresponding position are continuously transmitted to the memory. And after traversing all the data blocks m, completing all the data transmission operations. The method can reduce the calculation burden of the RAID controller and improve the transmission efficiency. The read command is in the opposite direction to the above operation, and is not described herein again.
According to the RAID array format, a contiguous data block is partitioned into multiple stripes for storage, with each stripe being partitioned into multiple data units according to the number of RAID data media, as shown in fig. 4. In step 104, the RAID calculation unit reads the content of the data unit according to the predetermined RAID format, in step 105, calculates RAID redundant data, and then, in step 106, writes the redundant data 1 back to the position in the drawing. The stripe 2/3/4 operates in sequence to eventually complete the generation of all redundant data. Taking RAID5 format as an example, if 3 data disk spaces and 1 redundant check disk space are contained, the calculation formula is as follows:
redundant data 1-data unit 1XOR data unit 2XOR data unit 3
When the write data and the redundant data are ready, the redundant data need to be dispersed to different storage media according to the RAID format requirement, so that the access amount of a single storage medium can be prevented from being increased remarkably. Continuing with the above example, in step 107, in a round-robin manner, redundant data 1 is written to storage medium 4, redundant data 2 is written to storage medium 3, redundant data 3 is written to storage medium 2, redundant data 4 is written to storage medium 1, and other data units are placed in the respective storage media in the normal order, as shown in fig. 5. And after the write-in corresponding relation is determined, the processor calls the storage device interface to execute the data write-in operation of the storage medium, and the write-in instruction of the RAID controller is completed.
In practical applications, each unit included in the RAID controller may be specifically implemented by hardware or software codes. For example, the elements of the RAID controller shown in the drawings may be implemented by an Application Specific Integrated Circuit (ASIC), or a Programmable Logic Device (PLD), which may be a Complex Programmable Logic Device (CPLD), a field-programmable gate array (FPGA), a General Array Logic (GAL), or any combination thereof. The related functions of the RAID controller may also be implemented by software, and the RAID controller and its respective units may also be software units.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A data processing method for a storage array controller, the data processing method being implemented based on a data processing apparatus comprising: the system comprises a host, a RAID controller and a storage device; the host is equipment with data processing or computing capacity, and the data storage function of the host is connected with the storage equipment through the RAID controller; the RAID controller comprises a processor, a host transmission interface, a storage device interface, a memory and a RAID calculation unit; the memory includes at least one storage medium;
the RAID controller is used for receiving a data access instruction initiated by the host and carrying out data calculation and processing, and the access instruction comprises a reading instruction and a writing instruction;
if the access instruction is a write instruction, the data processing method performs processing according to the following steps:
step 101: the processor calls the host transmission interface to transmit the written instruction content to the memory, and performs instruction analysis to obtain an analyzed instruction;
step 102: the processor calls a host transmission interface, and transmits host data to be written to the allocated memory space according to the analyzed instruction content;
step 103: the processor completes calculation of the RAID array data space, determines the RAID mode and configures RAID calculation units;
step 104: the RAID computing unit reads host data in the memory according to the current RAID mode configuration;
step 105: the RAID calculation unit calculates redundancy check data of the host data;
step 106: the RAID calculation unit writes the redundancy check data into a memory and combines the redundancy check data into all storage array data in the memory;
step 107: the processor configures a storage device interface to split all storage array data, and the split data are respectively written into each storage medium in the storage device to be stored;
step 108: the processor completes the processing of the current write-in instruction;
if the access instruction is a read instruction, the data processing method performs processing according to the following steps:
step 201: the processor calls the host transmission interface to transmit the read instruction content to the memory, and performs instruction analysis to obtain an analyzed instruction;
step 202: the processor obtains the position of the data to be read in each storage medium of the corresponding storage equipment according to the analyzed instruction content;
step 203: the storage device interface respectively reads out the data in the storage medium into the memory space;
step 204: the processor judges the next processing step according to the state of the RAID array, if the RAID array is in a normal state, the step 208 is carried out; when the RAID array is in a degraded state, entering step 205;
step 205: reading the existing data in the memory by the RAID computing unit;
step 206: the RAID calculation unit calculates according to an RAID algorithm to obtain missing data;
step 207: the RAID computing unit writes the missing data back to the memory and combines the missing data into all data to be read out in the memory;
step 208: the processor calls a host transmission interface to transmit the required data to the corresponding position of the host memory according to the requirement of the reading instruction;
step 209: the processor completes the current instruction processing.
2. The data processing method for a storage array controller of claim 1, wherein the step 204, wherein the condition that the RAID array is in a degraded state comprises: and in RAID5 mode, 1 storage medium can not be read normally.
3. The data processing method for a storage array controller of claim 1, wherein the functions of each functional unit in the RAID controller are independent of each other, and each unit is processed in a pipeline manner in a processing flow; when a host sends a plurality of access instructions, the access instructions are divided into an instruction 1, an instruction 2 and an instruction 3 … … according to time sequence;
the host transmission interface receives the instruction of the instruction 1, reports the instruction to the processor to complete instruction analysis, transmits the written data to the storage space of the controller according to the instruction content, and then starts the RAID calculation unit to perform RAID calculation, namely, step 101, step 102, and step 103; in the calculation processes of step 104, step 105, and step 106, the host transmission interface may receive the instruction of instruction 2, and perform the analysis and transmission processing operations of step 101, step 102, and step 103;
after the RAID redundancy check data calculation of instruction 1 is completed, the processor starts the write operation of the storage device, that is, step 107 and step 108; at this time, the write data corresponding to the instruction 2 has already been transmitted to the storage space, so as to start the RAID calculation of the instruction 2, that is, the step 104, the step 105, and the step 106, and at the same time, the host transmission interface may also receive the instruction of the instruction 3, and perform the analysis and transmission processing operations, that is, the step 101, the step 102, and the step 103;
therefore, the RAID controller can simultaneously process a plurality of host instructions, so that the overall processing efficiency of the controller is improved.
4. The data processing method for the storage array controller of claim 1, wherein in practical applications, the locations of the data in the host memory are not necessarily continuous, but may be dispersed in different locations of the memory; wherein, the instruction corresponding to the RAID controller in step 101 stores instruction information in a linked list manner;
the host computer generates a plurality of storage access instructions in the memory according to the sequence of program function execution, taking instruction 1 as an example; the content in the instruction 1 address is the first address of an instruction 1 descriptor, and the instruction 1 descriptor comprises parameter information of the instruction 1, a data block list address and information of the total number m of data blocks; the instruction 1 parameter information comprises an instruction type, a logic address LBA in a logic hard disk and data length information; the data block list contains addresses and lengths of m data blocks, respectively.
5. The data processing method for the storage array controller according to claim 1, wherein after the RAID controller receives the instruction 1, in step 102, the processor invokes the host transfer interface, first transfers all instruction contents of the instruction list to the storage space of the storage of the RAID controller according to the instruction format, and then completes data transfer between the host memory transfer and the storage of the RAID controller according to the instruction contents;
the host transmission interface reads data by using a direct memory access mode; when the RAID controller sends a write instruction, the host transmission interface judges that data should be transmitted from a host memory to a storage of the RAID controller according to the type of the instruction, and allocates a corresponding storage space in the storage according to the total length of the data; firstly, acquiring the address and the length of a data block 1, and transmitting data at a corresponding position to a memory; after the transmission is finished, automatically reading the address and the length of the data block 2, and continuously transmitting the data at the corresponding position to the memory; after traversing all the data blocks m, completing all data transmission operations; the method can reduce the calculation burden of the RAID controller and improve the transmission efficiency; the read command is in the opposite direction to the above operation, and is not described herein again.
6. The data processing method for a storage array controller of claim 5, wherein a contiguous data block is partitioned into a plurality of stripes for storage as specified by a RAID array format, each stripe being partitioned into a plurality of data units according to the number of RAID data media; in step 104, reading the content of the data unit by the RAID calculation unit according to a given RAID format, calculating RAID redundant data in step 105, and then writing redundant data 1 back to a set position in the stripe in step 106; the strip 2/3/4 operates in sequence, and finally the generation operation of all redundant data is completed; taking RAID5 format as an example, if 3 data disk spaces and 1 redundant check disk space are contained, the calculation formula is as follows:
redundant data 1 is data unit 1XOR data unit 2XOR data unit 3.
7. The data processing method for the storage array controller of claim 6, wherein when the write data and the redundant data are ready, the redundant data need to be distributed to different storage media according to RAID format requirements, such that a significant increase in the access volume of a single storage medium can be avoided; continuing to take the RAID5 format as an example, in step 107, according to a round robin manner, writing redundant data 1 into the storage medium 4, writing redundant data 2 into the storage medium 3, writing redundant data 3 into the storage medium 2, writing redundant data 4 into the storage medium 1, and placing other data units in the storage media in a normal order; and after the write-in corresponding relation is determined, the processor calls the storage device interface to execute the data write-in operation of the storage medium, and the write-in instruction of the RAID controller is completed.
8. The data processing method for a storage array controller of claim 1, wherein each unit comprised by the RAID controller is specifically implemented by hardware;
the implementation mode of each unit of the RAID controller comprises the following steps: by application specific integrated circuits, or by programmable logic devices;
the implementation manner of the PLD includes a complex program logic device, a field programmable gate array, a general array logic, or any combination thereof.
9. The data processing method for a storage array controller of claim 1, wherein the memory may be a cache or an off-chip random access memory;
the storage medium is a carrier for storing data and comprises a solid state disk or a mechanical hard disk; the hard disk interface types comprise a serial connection small computer system interface hard disk and a serial advanced technology attachment hard disk.
10. The data processing method for the storage array controller of claim 1, wherein the host and the RAID controller implement communication using a high speed transport interface protocol, including PCI, PCIe, NVMe, SATA, SAS, ethernet, or wireless transport interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111044793.3A CN113687978B (en) | 2021-09-07 | 2021-09-07 | Data processing method for memory array controller |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111044793.3A CN113687978B (en) | 2021-09-07 | 2021-09-07 | Data processing method for memory array controller |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113687978A true CN113687978A (en) | 2021-11-23 |
| CN113687978B CN113687978B (en) | 2024-05-28 |
Family
ID=78585754
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111044793.3A Active CN113687978B (en) | 2021-09-07 | 2021-09-07 | Data processing method for memory array controller |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113687978B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115794522A (en) * | 2022-11-25 | 2023-03-14 | 天津津航计算技术研究所 | RAID array storage test system |
| WO2024098635A1 (en) * | 2022-11-11 | 2024-05-16 | 苏州元脑智能科技有限公司 | Raid management method and system, and electronic device and non-volatile readable storage medium |
| US11995316B2 (en) | 2022-06-15 | 2024-05-28 | Samsung Electronics Co., Ltd. | Systems and methods for a redundant array of independent disks (RAID) using a decoder in cache coherent interconnect storage devices |
| US12298907B2 (en) | 2022-06-15 | 2025-05-13 | Samsung Electronics Co., Ltd. | Systems and methods for a redundant array of independent disks (RAID) using a RAID circuit in cache coherent interconnect storage devices |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030120836A1 (en) * | 2001-12-21 | 2003-06-26 | Gordon David Stuart | Memory system |
| CN102693096A (en) * | 2012-05-17 | 2012-09-26 | 山西达鑫核科技有限公司 | Bit-based serial transmission cloud storage method and device |
| CN103970487A (en) * | 2014-05-23 | 2014-08-06 | 浙江宇视科技有限公司 | Redundant array of independent disks and implementation method |
| CN107250975A (en) * | 2014-12-09 | 2017-10-13 | 清华大学 | Data-storage system and date storage method |
| CN109800187A (en) * | 2017-11-17 | 2019-05-24 | 三星电子株式会社 | It is configured as updating the storage equipment and its operating method of field programmable gate array |
| US10318474B1 (en) * | 2015-06-30 | 2019-06-11 | EMC IP Holding Company LLC | Data storage system with heterogenous parallel processors |
| WO2020173092A1 (en) * | 2019-02-28 | 2020-09-03 | 华为技术有限公司 | Data processing method, redundant array of independent disks controller, and data storage system |
-
2021
- 2021-09-07 CN CN202111044793.3A patent/CN113687978B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030120836A1 (en) * | 2001-12-21 | 2003-06-26 | Gordon David Stuart | Memory system |
| CN102693096A (en) * | 2012-05-17 | 2012-09-26 | 山西达鑫核科技有限公司 | Bit-based serial transmission cloud storage method and device |
| CN103970487A (en) * | 2014-05-23 | 2014-08-06 | 浙江宇视科技有限公司 | Redundant array of independent disks and implementation method |
| CN107250975A (en) * | 2014-12-09 | 2017-10-13 | 清华大学 | Data-storage system and date storage method |
| US10318474B1 (en) * | 2015-06-30 | 2019-06-11 | EMC IP Holding Company LLC | Data storage system with heterogenous parallel processors |
| CN109800187A (en) * | 2017-11-17 | 2019-05-24 | 三星电子株式会社 | It is configured as updating the storage equipment and its operating method of field programmable gate array |
| WO2020173092A1 (en) * | 2019-02-28 | 2020-09-03 | 华为技术有限公司 | Data processing method, redundant array of independent disks controller, and data storage system |
| CN111625181A (en) * | 2019-02-28 | 2020-09-04 | 华为技术有限公司 | Data processing method, independent hard disk redundant array controller and data storage system |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11995316B2 (en) | 2022-06-15 | 2024-05-28 | Samsung Electronics Co., Ltd. | Systems and methods for a redundant array of independent disks (RAID) using a decoder in cache coherent interconnect storage devices |
| US12298907B2 (en) | 2022-06-15 | 2025-05-13 | Samsung Electronics Co., Ltd. | Systems and methods for a redundant array of independent disks (RAID) using a RAID circuit in cache coherent interconnect storage devices |
| WO2024098635A1 (en) * | 2022-11-11 | 2024-05-16 | 苏州元脑智能科技有限公司 | Raid management method and system, and electronic device and non-volatile readable storage medium |
| CN115794522A (en) * | 2022-11-25 | 2023-03-14 | 天津津航计算技术研究所 | RAID array storage test system |
| CN115794522B (en) * | 2022-11-25 | 2025-09-02 | 天津津航计算技术研究所 | A RAID array storage test system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113687978B (en) | 2024-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113687977B (en) | Data processing device for improving computing performance based on RAID controller | |
| CN113687978B (en) | Data processing method for memory array controller | |
| US7730257B2 (en) | Method and computer program product to increase I/O write performance in a redundant array | |
| US10725859B2 (en) | Parity generation offload using peer-to-peer data transfers in data storage system | |
| US10761929B2 (en) | Data storage drive rebuild with parity generation offload using peer-to-peer data transfers | |
| US7206899B2 (en) | Method, system, and program for managing data transfer and construction | |
| US10635529B2 (en) | Parity offload for multiple data storage devices | |
| US7321955B2 (en) | Control device, control method and storage medium recording a control program for controlling write-back schedule of data from cache memory to a plurality of storage devices | |
| CN111104056B (en) | Data recovery method, system and device in storage system | |
| US20130290613A1 (en) | Storage system and storage apparatus | |
| CN108877862B (en) | Data organization of page stripes and method and apparatus for writing data to page stripes | |
| CN109358809B (en) | RAID data storage system and method | |
| CN109375868B (en) | Data storage method, scheduling device, system, equipment and storage medium | |
| US9092152B1 (en) | Data storage system employing a distributed compute engine memory controller with embedded logic and arithmetic functionality and method for data migration between high-performance computing architectures and data storage devices using the same | |
| CN117193672B (en) | Data processing methods and devices for storage devices, storage media and electronic equipment | |
| US10802958B2 (en) | Storage device, its controlling method, and storage system having the storage device | |
| CN118331782A (en) | System and method for distributed parity calculation load shifting | |
| CN106919339B (en) | A kind of hard disk array and hard disk array processing operation request method | |
| WO2023020136A1 (en) | Data storage method and apparatus in storage system | |
| EP4715559A1 (en) | Data storage method and related apparatus | |
| CN212341857U (en) | Intelligent storage device, system and hard disk cartridge | |
| CN114415985A (en) | Stored data processing unit based on numerical control separation architecture | |
| CN115686345A (en) | Data processing method and device | |
| US7757130B2 (en) | Computer system having raid control function and raid control method | |
| CN119200987A (en) | Write command processing method, terminal and storage medium based on cache optimization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |