CN111007984A - RAID5 storage system based on FPGA and implementation method - Google Patents
RAID5 storage system based on FPGA and implementation method Download PDFInfo
- Publication number
- CN111007984A CN111007984A CN201911008181.1A CN201911008181A CN111007984A CN 111007984 A CN111007984 A CN 111007984A CN 201911008181 A CN201911008181 A CN 201911008181A CN 111007984 A CN111007984 A CN 111007984A
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- unit
- channel
- channel number
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0689—Disk arrays, e.g. RAID, JBOD
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a RAID5 storage system based on FPGA and an implementation method thereof, wherein, data flow enters a data distribution control unit, the data distribution control unit respectively inputs related data to each buffer area of a RAID data generation unit according to a register parameter group, the data enters a data channel remapping unit, the register parameter group is a storage channel number output by a storage channel number calculation unit, the storage channel number calculation unit inputs the storage channel number to the data channel remapping unit, the data channel remapping unit distributes the data to a corresponding storage interface unit according to the storage channel number, and the storage interface unit writes the data into a corresponding storage disk. The invention can effectively improve the parallelism of data transmission and distribution and improve the data bandwidth; the verification calculation and the data distribution are synchronously carried out, and the verification calculation is completed when the data transmission of the (n-1) th packet is completed, so that the verification generation delay introduced by other methods is eliminated, and the efficiency is improved.
Description
Technical Field
The invention relates to the field of computer communication, in particular to a logic control system for RAID5 storage and an implementation method thereof.
Background
RAID5 is a control method of data storage, which divides the path data into multiple paths of parallel storage disks, calculates the check value of all data, writes the check value into the check position of the disk array, and can effectively improve the bandwidth and the safety of data storage.
At present, the conventional RAID5 storage scheme is implemented by using a CPU and software, and since data needs to enter and exit a buffer area for multiple times, and can be sent to a storage channel after data distribution and verification calculation are all completed, execution efficiency is low, and delay is large.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a RAID5 storage system based on an FPGA and an implementation method thereof. The invention is realized by adopting FPGA programmable logic, and the data distribution and the check computation are executed in parallel, so that the distributed data can be immediately transmitted to the storage channel, thereby effectively improving the efficiency of each link of data transmission and eliminating the extra time overhead brought by the check computation.
The technical scheme adopted by the invention for solving the technical problems is as follows:
an original data stream enters a data distribution control unit, the data distribution control unit inputs related data into each buffer area of a RAID data generation unit respectively according to parameter requirements input by a register parameter group, data of each buffer enters a data channel remapping unit, the register parameter group is a storage channel number output by a storage channel number calculation unit, the storage channel number calculation unit inputs the storage channel number into the data channel remapping unit, the data channel remapping unit distributes the data to a corresponding storage interface unit according to the storage channel number, and the storage interface unit writes the data of n channels into a corresponding storage disk.
The implementation method of the RAID5 storage system based on the FPGA comprises the following specific steps:
1) setting a register parameter group in the FPGA, and controlling the number of the storage disks and the data volume of one-time disk storage, wherein the number n of the storage disks is more than 2, and the data volume of one-time disk storage is more than 1 KB;
2) the data distribution control unit receives an original data stream to be stored, divides the original data into n-1 parts of data according to the data quantity determined by the parameters determined by the register parameter group and the number of the storage disks, buffers the data into n-1 paths of buffer areas of the RAID data generation unit, inputs the n-1 parts of data into the check data calculation unit and stores the data into the nth path of buffer areas of the RAID data generation unit;
when the n-1 paths of data are sequentially input to the check calculation unit, the check calculation reads the intermediate calculation result in the current nth path of buffer area, and writes the intermediate calculation result and the current channel data into the nth path of buffer area again after the check calculation unit performs check calculation on the previous n-1 paths of data until the check calculation unit finishes all the calculation on the previous n-1 paths of data and writes the previous n-1 paths of data into the nth path of buffer area;
inputting n paths of data of the RAID data generation unit into a data channel remapping unit;
3) the storage channel number calculation unit calculates to obtain n-1 storage channel numbers corresponding to the current n-1 buffer channels, and inputs the n-1 storage channel numbers into the data channel remapping unit;
4) the storage channel number calculation unit calculates to obtain an nth storage channel number corresponding to the nth buffer channel, and the data channel remapping unit outputs the nth buffer channel data to the corresponding storage interface unit;
5) the storage interface unit writes the data of the n channels into a storage disk;
6) returning to step 1), preparing to receive new data.
The invention has the beneficial effects that:
1) the FPGA programmable logic is adopted for realizing, so that the parallelism of data transmission and distribution can be effectively improved, and the data bandwidth is improved;
2) the verification calculation and the data distribution are synchronously carried out, and the verification calculation is completed when the data transmission of the (n-1) th packet is completed, so that the verification generation delay introduced by other methods is eliminated, and the efficiency is improved.
Drawings
FIG. 1 is a schematic diagram of an FPGA-based RAID5 storage system of the present invention.
Detailed Description
The invention is further illustrated with reference to the following figures and examples.
An original data stream enters a data distribution control unit, the data distribution control unit inputs related data into each buffer area of a RAID data generation unit respectively according to parameter requirements input by a register parameter group, data of each buffer enters a data channel remapping unit, the register parameter group is a storage channel number output by a storage channel number calculation unit, the storage channel number calculation unit inputs the storage channel number into the data channel remapping unit, the data channel remapping unit distributes the data to a corresponding storage interface unit according to the storage channel number, and the storage interface unit writes the data of n channels into a corresponding storage disk.
The RAID5 storage design architecture based on the FPGA and the implementation method thereof comprise the following specific steps:
1) setting a register parameter group in the FPGA, and controlling the number of the storage disks and the data volume of one-time disk storage, wherein the number n of the storage disks is more than 2, and the data volume of one-time disk storage is more than 1 KB;
2) the data distribution control unit receives an original data stream to be stored, divides the original data into n-1 parts of data according to the data quantity determined by the parameters determined by the register parameter group and the number of the storage disks, buffers the data into n-1 paths of buffer areas of the RAID data generation unit, inputs the n-1 parts of data into the check data calculation unit and stores the data into the nth path of buffer areas of the RAID data generation unit;
when the n-1 paths of data are sequentially input to the check calculation unit, the check calculation reads the intermediate calculation result in the current nth path of buffer area, and writes the intermediate calculation result and the current channel data into the nth path of buffer area again after the check calculation unit performs check calculation on the previous n-1 paths of data until the check calculation unit finishes all the calculation on the previous n-1 paths of data and writes the previous n-1 paths of data into the nth path of buffer area;
inputting n paths of data of the RAID data generation unit into a data channel remapping unit;
3) the storage channel number calculation unit calculates to obtain n-1 storage channel numbers corresponding to the current n-1 buffer channels, and inputs the n-1 storage channel numbers into the data channel remapping unit;
4) the storage channel number calculation unit calculates to obtain an nth storage channel number corresponding to the nth buffer channel, and the data channel remapping unit outputs the nth buffer channel data to the corresponding storage interface unit;
5) the storage interface unit writes the data of the n channels into a storage disk;
6) returning to step 1), preparing to receive new data.
Claims (2)
1. An FPGA-based RAID5 storage system, characterized in that:
according to the RAID5 storage system based on the FPGA, an original data stream enters a data distribution control unit, the data distribution control unit inputs related data into each buffer area of a RAID data generation unit respectively according to parameter requirements input by a register parameter group, data of each buffer enters a data channel remapping unit, the register parameter group is a storage channel number output by a storage channel number calculation unit, the storage channel number calculation unit inputs the storage channel number into the data channel remapping unit, the data channel remapping unit distributes the data to a corresponding storage interface unit according to the storage channel number, and the storage interface unit writes the data of n channels into a corresponding storage disk.
2. A method for implementing the FPGA-based RAID5 storage system of claim 1, comprising the steps of:
1) setting a register parameter group in the FPGA, and controlling the number of the storage disks and the data volume of one-time disk storage, wherein the number n of the storage disks is more than 2, and the data volume of one-time disk storage is more than 1 KB;
2) the data distribution control unit receives an original data stream to be stored, divides the original data into n-1 parts of data according to the data quantity determined by the parameters determined by the register parameter group and the number of the storage disks, buffers the data into n-1 paths of buffer areas of the RAID data generation unit, inputs the n-1 parts of data into the check data calculation unit and stores the data into the nth path of buffer areas of the RAID data generation unit;
when the n-1 paths of data are sequentially input to the check calculation unit, the check calculation reads the intermediate calculation result in the current nth path of buffer area, and writes the intermediate calculation result and the current channel data into the nth path of buffer area again after the check calculation unit performs check calculation on the previous n-1 paths of data until the check calculation unit finishes all the calculation on the previous n-1 paths of data and writes the previous n-1 paths of data into the nth path of buffer area;
inputting n paths of data of the RAID data generation unit into a data channel remapping unit;
3) the storage channel number calculation unit calculates to obtain n-1 storage channel numbers corresponding to the current n-1 buffer channels, and inputs the n-1 storage channel numbers into the data channel remapping unit;
4) the storage channel number calculation unit calculates to obtain an nth storage channel number corresponding to the nth buffer channel, and the data channel remapping unit outputs the nth buffer channel data to the corresponding storage interface unit;
5) the storage interface unit writes the data of the n channels into a storage disk;
6) returning to step 1), preparing to receive new data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911008181.1A CN111007984A (en) | 2019-10-22 | 2019-10-22 | RAID5 storage system based on FPGA and implementation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911008181.1A CN111007984A (en) | 2019-10-22 | 2019-10-22 | RAID5 storage system based on FPGA and implementation method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111007984A true CN111007984A (en) | 2020-04-14 |
Family
ID=70111508
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911008181.1A Pending CN111007984A (en) | 2019-10-22 | 2019-10-22 | RAID5 storage system based on FPGA and implementation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111007984A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003280826A (en) * | 2002-03-27 | 2003-10-02 | Hitachi Ltd | Storage sub-system |
| CN1825292A (en) * | 2005-02-23 | 2006-08-30 | 华为技术有限公司 | Access device for direct memory access and method for implementing single channel bidirectional data interaction |
| US20070055905A1 (en) * | 2005-09-02 | 2007-03-08 | Teh-Chern Chou | Parity engine for use in storage virtualization controller and method of generating data by parity engine |
| US20090313430A1 (en) * | 2007-04-27 | 2009-12-17 | Siemens Medical Solutions Usa, Inc. | Positron Emission Tomography Event Stream Buffering |
| CN105556480A (en) * | 2013-09-16 | 2016-05-04 | 上海宝存信息科技有限公司 | Method and system for constructing raid in storage system on the basis of flash memory |
| US9823968B1 (en) * | 2015-08-21 | 2017-11-21 | Datadirect Networks, Inc. | Data storage system employing a variable redundancy distributed RAID controller with embedded RAID logic and method for data migration between high-performance computing architectures and data storage devices using the same |
| CN109358809A (en) * | 2018-09-28 | 2019-02-19 | 方信息科技（上海）有限公司 | A kind of RAID data storage system and method |
-
2019
- 2019-10-22 CN CN201911008181.1A patent/CN111007984A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003280826A (en) * | 2002-03-27 | 2003-10-02 | Hitachi Ltd | Storage sub-system |
| CN1825292A (en) * | 2005-02-23 | 2006-08-30 | 华为技术有限公司 | Access device for direct memory access and method for implementing single channel bidirectional data interaction |
| US20070055905A1 (en) * | 2005-09-02 | 2007-03-08 | Teh-Chern Chou | Parity engine for use in storage virtualization controller and method of generating data by parity engine |
| US20090313430A1 (en) * | 2007-04-27 | 2009-12-17 | Siemens Medical Solutions Usa, Inc. | Positron Emission Tomography Event Stream Buffering |
| CN105556480A (en) * | 2013-09-16 | 2016-05-04 | 上海宝存信息科技有限公司 | Method and system for constructing raid in storage system on the basis of flash memory |
| US9823968B1 (en) * | 2015-08-21 | 2017-11-21 | Datadirect Networks, Inc. | Data storage system employing a variable redundancy distributed RAID controller with embedded RAID logic and method for data migration between high-performance computing architectures and data storage devices using the same |
| CN109358809A (en) * | 2018-09-28 | 2019-02-19 | 方信息科技（上海）有限公司 | A kind of RAID data storage system and method |
Non-Patent Citations (4)
| Title |
|---|
| YU WANG: "CR5M: A mirroring-powered channel-RAID5 architecture for an SSD", 《2014 30TH SYMPOSIUM ON MASS STORAGE SYSTEMS AND TECHNOLOGIES (MSST)》 * |
| 孙志卓等: "连续数据存储中面向RAID5的写操作优化设计", 《计算机研究与发展》 * |
| 李红艳: "针对固态盘的I/O优化技术研究", 《信息科技》 * |
| 王振升等: "基于FPGA的可堆叠存储阵列设计与优化", 《微电子学与计算机》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101176159B (en) | Method to handle write mask in DRAM memory | |
| CN105103454A (en) | Modular and scalable cyclic redundancy check computation circuit | |
| CN105912501A (en) | SM4-128 encryption algorithm implementation method and system based on large-scale coarseness reconfigurable processor | |
| CN102004711A (en) | Single-interrupt real-time data transmission method based on FPGA (Field Programmable Gate Array) | |
| CN113489570A (en) | Data transmission method, device and equipment for PCIe link | |
| CN111007984A (en) | RAID5 storage system based on FPGA and implementation method | |
| CN102355232A (en) | FPGA (field-programmable gate array)-based high-speed FIR (finite impulse response) digital filter | |
| Kwon et al. | Scalable multi-FPGA acceleration for large RNNs with full parallelism levels | |
| US20230274130A1 (en) | Hardware-assisted gradient optimization using streamed gradients | |
| CN111986070B (en) | VDIF format data heterogeneous parallel framing method based on GPU | |
| CN105138735B (en) | A kind of clock tree synthesis method of more macroelement multi-clock chips | |
| CN106325259A (en) | Multi-LRU parallel safety testing method | |
| CN115146485A (en) | Radio frequency link simulation method based on GPU acceleration | |
| CN104778015A (en) | Disk array performance optimization method and system | |
| Hennessy et al. | Readout firmware of the Vertex Locator for LHCb Run 3 and beyond | |
| Du et al. | A solution to overcome some limitations of SDF based models | |
| CN101355401A (en) | Method and apparatus for decoding turbo code | |
| CA2817467C (en) | Method and apparatus for decoding low-density parity-check codes | |
| Martín-Álvarez et al. | Malleability implementation in a mpi iterative method | |
| CN105203821B (en) | A kind of method and device analyzed reference clock phase jitter | |
| Potkonjak et al. | High performance embedded system optimization using algebraic and generalized retiming techniques | |
| Note et al. | Rapid prototyping of DSP systems: requirements and solutions | |
| CN113162629B (en) | Multi-core polarization code encoder supporting multi-code length and multi-code rate common mode | |
| CN112800629B (en) | Multi-data stream driven signal processing system determination time sequence calculation method | |
| KR102636314B1 (en) | Method and system for weight memory mapping for streaming operation of giant generative artificial intelligence hardware |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200414 |