CN111159074A - Super-large-scale data hash operation accelerator card based on FPGA - Google Patents
Super-large-scale data hash operation accelerator card based on FPGA Download PDFInfo
- Publication number
- CN111159074A CN111159074A CN201911413494.5A CN201911413494A CN111159074A CN 111159074 A CN111159074 A CN 111159074A CN 201911413494 A CN201911413494 A CN 201911413494A CN 111159074 A CN111159074 A CN 111159074A
- Authority
- CN
- China
- Prior art keywords
- module
- data
- calculation
- cache
- crc
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
- G06F13/20—Handling requests for interconnection or transfer for access to input/output bus
- G06F13/28—Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access DMA, cycle steal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/38—Information transfer, e.g. on bus
- G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation
- G06F13/4282—Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F2213/0026—PCI express
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The invention provides a super-large scale data Hash operation accelerator card based on an FPGA (field programmable gate array), which belongs to the technical field of circuit design and consists of a PCIE (peripheral component interface express) interface, a DMA (direct memory access) module, a selector module, a receiving cache module, a CRC _ Hash calculation module and a sending cache module. The board card has the advantages of ingenious design, advanced technology and convenient and simple use.
Description
Technical Field
The invention relates to a circuit design technology, in particular to a super-large scale data hash operation accelerator card based on an FPGA.
Background
With the rapid development of social economy and scientific technology, the development of information technology and internet technology is rapid, and data is gradually pushed to the big data era. Information data also faces greater challenges in the development of the big data era, brings great impact to information processing, effectively deals with the development of the big data era, improves the technical level of computer information processing, and is a focus of attention of many technicians in the development of the big data era.
The development of the big data era is extremely rapid, and certain requirements are provided for computer information processing, but in actual computer processing, due to a plurality of defects in the technology, the computer information processing technology cannot meet the development requirements of the big data era.
In recent years, with the increasing popularity of informatization, the processing of large data is important. The processing problem of a large data table represented by a database is more and more prominent, wherein the Hash calculation of the large data needs tens of seconds if the CPU is used for calculation alone, and the calculation of the Hash of 1 hundred million table entries greatly loses the calculation capability of the CPU.
Disclosure of Invention
In order to solve the technical problems, the invention provides a super-large scale data hash operation acceleration card based on an FPGA (field programmable gate array), which realizes the acceleration of the hash operation of super-large scale data.
The technical scheme of the invention is as follows:
an ultra-large scale data hash operation accelerator card based on FPGA,
the system comprises a PCIE interface, a DMA module, a selector module, a receiving cache module, a CRC _ Hash calculation module and a sending cache module; the PCIE interface is connected to the DMA module,
the receiving cache module is respectively connected with the CRC _ Hash calculation module and the selector module; and the sending cache module is respectively connected with the CRC _ Hash calculation module and the selector module.
Further, in the above-mentioned case,
the selector module is divided into a receive path selector and a transmit path selector.
In a still further aspect of the present invention,
the receiving cache module is respectively connected with the output end of the receiving path selector module and the input end of the CRC _ Hash calculation module;
and the sending cache module is respectively connected with the output end of the CRC _ Hash calculation module and the input end of the sending path selector module.
In a still further aspect of the present invention,
a PCIE interface is used as a data external interface and is provided with a DMA module to acquire data automatically.
In a still further aspect of the present invention,
data are acquired through PCIE and DMA, and a receiving buffer module is added in the FPGA through a selector module to form table tennis operation.
In a still further aspect of the present invention,
the receiving buffer module and the sending buffer module are composed of on-chip RAMs and are provided with selectors for performing ping-pong operation.
In a still further aspect of the present invention,
calculating a Hash value through a CRC _ Hash module; and after the calculation is finished, storing the calculation value into a sending buffer module, and returning the calculation value by the ping-pong operation again.
Still further, the working process is as follows:
1) slicing the oversized data, configuring PCIE DMA, and starting the PCIE DMA to acquire the data;
2) caching data to a receiving cache 1 through a selector;
3) starting a CRC _ Hash calculation module 1;
4) when the calculation is started, the data are transported to the receiving cache 2 again, so that the caches 1 and 2 form ping-pong operation;
5) the calculation module 1 writes data into the sending cache 1, and the calculation module 2 writes data into the sending cache 2;
6) and (4) finishing the caching of a piece of data by sending the cache at will, namely starting DMA to move the data back to the CPU, and finishing one calculation.
7) And repeating the slicing calculation until all data slices are calculated.
The invention has the advantages that
The data external interface adopts a PCIE interface and is provided with DMA (direct memory access) to acquire data by self without participation of a CPU (central processing unit), so that the CPU resource is saved.
The receiving buffer and the sending buffer are composed of on-chip RAMs, and can be used for ping-pong operation by being provided with a selector.
And a hardware CRC _ Hash module is adopted, and CRC can be calculated by hardware only in one clock cycle, so that the efficiency is greatly improved.
Drawings
Fig. 1 is a working block diagram of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention, and based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts belong to the scope of the present invention.
Aiming at the Hash calculation of the super-large scale data, the super-large scale data Hash operation acceleration card is arranged based on the FPGA, the KU115 series FPGA is used as a core processing chip, a PCIE high-speed bus interface is externally adopted, and a storage medium completely uses the RAM resource on the FPGA chip, so that the Hash operation acceleration of the super-large scale data is realized, the calculation bandwidth is more than 2 hundred million/s, and the calculation performance is greatly improved.
The invention is composed of a PCIE interface, a DMA module, a selector module, a receiving cache module, a CRC _ Hash calculation module and a sending cache module.
The PCIE interface is connected to the DMA module,
the selector module is divided into a receive path selector and a transmit path selector.
The receiving cache module 1 is respectively connected with the output end of the receiving path selector module and the input end of the CRC _ Hash calculation module 1;
the sending cache module 1 is respectively connected with the output end of the CRC _ Hash calculation module 1 and the input end of the sending path selector module.
The receiving cache module 2 is respectively connected with the output end of the receiving path selector module and the input end of the CRC _ Hash calculation module 2;
the sending cache module 2 is respectively connected with the output end of the CRC _ Hash calculation module 2 and the input end of the sending path selector module.
The working process of the invention is as follows:
1) slicing the oversized data, configuring PCIE DMA, and starting the PCIE DMA to acquire the data;
2) caching data to a receiving cache 1 through a selector;
3) starting a CRC _ Hash calculation module 1;
4) when the calculation is started, the data are transported to the receiving cache 2 again, so that the caches 1 and 2 form ping-pong operation;
5) the calculation module 1 writes data into the sending cache 1, and the calculation module 2 writes data into the sending cache 2;
6) and if any one of the sending caches is full, starting DMA to move the data back to the CPU, and finishing one calculation.
7) And repeating the slicing calculation until all data slices are calculated.
During practical application, the PCIE interface of the accelerator card is inserted into a PICE slot of a computer mainboard, after the driver is installed, the register of the accelerator card is configured, the accelerator card is started, data can be automatically acquired, hash calculation of super-large-scale data is completed, a calculated value is returned, and huge performance improvement is obtained.
The above description is only a preferred embodiment of the present invention, and is only used to illustrate the technical solutions of the present invention, and not to limit the protection scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (8)
1. An ultra-large scale data hash operation accelerator card based on FPGA is characterized in that,
the system comprises a PCIE interface, a DMA module, a selector module, a receiving cache module, a CRC _ Hash calculation module and a sending cache module; the PCIE interface is connected to the DMA module,
the receiving cache module is respectively connected with the CRC _ Hash calculation module and the selector module; and the sending cache module is respectively connected with the CRC _ Hash calculation module and the selector module.
2. The accelerator card of claim 1,
the selector module is divided into a receive path selector and a transmit path selector.
3. The accelerator card of claim 2,
the receiving cache module is respectively connected with the output end of the receiving path selector module and the input end of the CRC _ Hash calculation module;
and the sending cache module is respectively connected with the output end of the CRC _ Hash calculation module and the input end of the sending path selector module.
4. The accelerator card of claim 3,
a PCIE interface is used as a data external interface and is provided with a DMA module to acquire data automatically.
5. The accelerator card of claim 4,
data are acquired through PCIE and DMA, and a receiving buffer module is added in the FPGA through a selector module to form table tennis operation.
6. The accelerator card of claim 5,
the receiving buffer module and the sending buffer module are composed of on-chip RAMs and are provided with selectors for performing ping-pong operation.
7. Accelerator card according to claim 5 or 6,
calculating a Hash value through a CRC _ Hash module; and after the calculation is finished, storing the calculation value into a sending buffer module, and returning the calculation value by the ping-pong operation again.
8. The accelerator card of claim 7,
the working process is as follows:
1) slicing the oversized data, configuring PCIE DMA, and starting the PCIE DMA to acquire the data;
2) caching data to a receiving cache 1 through a selector;
3) starting a CRC _ Hash calculation module 1;
4) when the calculation is started, the data are transported to the receiving cache 2 again, so that the caches 1 and 2 form ping-pong operation;
5) the calculation module 1 writes data into the sending cache 1, and the calculation module 2 writes data into the sending cache 2;
6) the random sending cache completes the cache of one data, namely starting DMA to move the data back to the CPU, and completing one calculation;
7) and repeating the slicing calculation until all data slices are calculated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911413494.5A CN111159074A (en) | 2019-12-31 | 2019-12-31 | Super-large-scale data hash operation accelerator card based on FPGA |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911413494.5A CN111159074A (en) | 2019-12-31 | 2019-12-31 | Super-large-scale data hash operation accelerator card based on FPGA |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111159074A true CN111159074A (en) | 2020-05-15 |
Family
ID=70560150
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911413494.5A Pending CN111159074A (en) | 2019-12-31 | 2019-12-31 | Super-large-scale data hash operation accelerator card based on FPGA |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111159074A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111813713A (en) * | 2020-09-08 | 2020-10-23 | 苏州浪潮智能科技有限公司 | Data acceleration operation processing method and device and computer readable storage medium |
| CN113296705A (en) * | 2021-05-27 | 2021-08-24 | 浙江萤火虫区块链科技有限公司 | Architecture system for parallel computing Poseidon Hash in Filecin |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102143083A (en) * | 2011-04-02 | 2011-08-03 | 南京邮电大学 | Method for designing and realizing double buffer in Ares protocol analysis system |
| CN102970043A (en) * | 2012-11-14 | 2013-03-13 | 无锡芯响电子科技有限公司 | GZIP (GNUzip)-based hardware compressing system and accelerating method thereof |
| CN104202054A (en) * | 2014-09-16 | 2014-12-10 | 东南大学 | Hardware LZMA (Lempel-Ziv-Markov chain-Algorithm) compression system and method |
| CN105224482A (en) * | 2015-10-16 | 2016-01-06 | 浪潮(北京)电子信息产业有限公司 | A kind of FPGA accelerator card high-speed memory system |
| CN107835437A (en) * | 2017-10-20 | 2018-03-23 | 广东省南方数字电视无线传播有限公司 | Dispatching method and device based on more caching servers |
| CN109977116A (en) * | 2019-03-14 | 2019-07-05 | 山东超越数控电子股份有限公司 | Hash connection operator accelerated method and system based on FPGA-DDR |
-
2019
- 2019-12-31 CN CN201911413494.5A patent/CN111159074A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102143083A (en) * | 2011-04-02 | 2011-08-03 | 南京邮电大学 | Method for designing and realizing double buffer in Ares protocol analysis system |
| CN102970043A (en) * | 2012-11-14 | 2013-03-13 | 无锡芯响电子科技有限公司 | GZIP (GNUzip)-based hardware compressing system and accelerating method thereof |
| CN104202054A (en) * | 2014-09-16 | 2014-12-10 | 东南大学 | Hardware LZMA (Lempel-Ziv-Markov chain-Algorithm) compression system and method |
| CN105224482A (en) * | 2015-10-16 | 2016-01-06 | 浪潮(北京)电子信息产业有限公司 | A kind of FPGA accelerator card high-speed memory system |
| CN107835437A (en) * | 2017-10-20 | 2018-03-23 | 广东省南方数字电视无线传播有限公司 | Dispatching method and device based on more caching servers |
| CN109977116A (en) * | 2019-03-14 | 2019-07-05 | 山东超越数控电子股份有限公司 | Hash connection operator accelerated method and system based on FPGA-DDR |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111813713A (en) * | 2020-09-08 | 2020-10-23 | 苏州浪潮智能科技有限公司 | Data acceleration operation processing method and device and computer readable storage medium |
| CN111813713B (en) * | 2020-09-08 | 2021-02-12 | 苏州浪潮智能科技有限公司 | Data acceleration operation processing method and device and computer readable storage medium |
| CN113296705A (en) * | 2021-05-27 | 2021-08-24 | 浙江萤火虫区块链科技有限公司 | Architecture system for parallel computing Poseidon Hash in Filecin |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11307769B2 (en) | Data storage method, apparatus and storage medium | |
| CN108805272A (en) | A kind of general convolutional neural networks accelerator based on FPGA | |
| CN110909025A (en) | Database query method, query device and terminal | |
| US20150127691A1 (en) | Efficient implementations for mapreduce systems | |
| WO2017041570A1 (en) | Method and apparatus for writing data to cache | |
| CN103761988A (en) | SSD (solid state disk) and data movement method | |
| WO2023029254A1 (en) | Data query method and apparatus, and device and medium | |
| CN111159074A (en) | Super-large-scale data hash operation accelerator card based on FPGA | |
| CN110058816B (en) | DDR-based high-speed multi-user queue manager and method | |
| US20230128085A1 (en) | Data aggregation processing apparatus and method, and storage medium | |
| WO2022199027A1 (en) | Random write method, electronic device and storage medium | |
| CN112256623A (en) | Heterogeneous system-based processing performance optimization method and device | |
| CN111858612B (en) | Data accelerated access method and device based on graph database and storage medium | |
| CN111813709A (en) | High-speed parallel storage method based on FPGA (field programmable Gate array) storage and calculation integrated framework | |
| CN104636284A (en) | Method and device for achieving flash memory storage array | |
| CN105187162B (en) | A kind of parallel dissociation rate matching method of more granularities and device | |
| CN111126619A (en) | Machine learning method and device | |
| CN110750210A (en) | Storage system | |
| CN210804421U (en) | Server system | |
| CN115934583A (en) | Hierarchical caching method, device and system | |
| CN111858665B (en) | Method, system, terminal and storage medium for improving soft copy reading performance | |
| CN202217274U (en) | Multi-core processor storage system device | |
| WO2020238750A1 (en) | Data processing method and apparatus, electronic device, and computer storage medium | |
| CN114116711A (en) | Data processing method, data processing device, database, storage medium and program product | |
| CN111338884B (en) | Cache miss rate monitoring method and device, electronic equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200515 |