CN107766266B

CN107766266B - High-speed data acquisition and storage system based on FPGA and PCIe

Info

Publication number: CN107766266B
Application number: CN201610695492.XA
Authority: CN
Inventors: 夏梦月; 黄海波; 马超; 杨建超; 史小斌; 顾红; 苏卫民; 陆锦辉; 曹鑫泉
Original assignee: Nanjing University of Science and Technology
Current assignee: Nanjing University of Science and Technology
Priority date: 2016-08-21
Filing date: 2016-08-21
Publication date: 2020-06-19
Anticipated expiration: 2036-08-21
Also published as: CN107766266A

Abstract

The invention discloses a high-speed data acquisition and storage system based on FPGA and PCIe, which comprises an AD acquisition module, a signal preprocessing module, a double-port RAM, a PCIe data transmission module, a solid-state disk array, a self-checking module, a power supply module, a serial port control module and a clock module, wherein the signal preprocessing module is connected with the double-port RAM; the AD acquisition module is used for acquiring signal data, and the signal preprocessing module is used for preprocessing the AD acquired data; the PCIe data transmission module is used for storing the data processed by the FPGA to the solid-state disk array; the self-checking module is used for performing fast Fourier transform on data stored in the solid-state disk array and judging whether the system works normally or not. The invention can acquire data at the ultrahigh sampling rate of 3.6GHz at most, and realizes real-time data transmission by preprocessing the data, finishing bit interception, selectively storing and adding data identifiers.

Description

High-speed data acquisition and storage system based on FPGA and PCIe

Technical Field

The invention relates to a data acquisition and storage system, in particular to a high-speed data acquisition and storage system based on an FPGA and PCIe.

Background

With the development of high speed information technology, the throughput of data processing is higher and higher, and the requirement for high speed and real-time performance is higher and higher, so that the technology of high speed data acquisition and real-time storage is more and more important in digital signal processing. The quality of a data acquisition and storage system is evaluated, primarily with regard to the speed and storage capacity of the data being processed. Under the condition of ensuring stability, the data read-write speed and the storage capacity are improved as much as possible, and the requirements of modern industrial and scientific research on the system speed and capacity are met.

The high-speed data acquisition has extremely high requirements on signal integrity, signal interference, high-speed PCB wiring, data processing and high-speed real-time storage, the application environment of the high-speed data acquisition is very complex, and in the current practical application, a data acquisition system which can carry out high-speed data acquisition and large-capacity storage is difficult to find so as to ensure that the long-time high-speed data acquisition can be carried out.

Disclosure of Invention

The invention aims to provide a high-speed data acquisition and storage system based on an FPGA and PCIe.

The technical scheme for realizing the purpose of the invention is as follows: a high-speed data acquisition and storage system based on FPGA and PCIe comprises an AD acquisition module, a signal preprocessing module, a double-port RAM, a PCIe data transmission module, a solid-state disk array, a self-checking module, a power supply module, a serial port control module and a clock module;

the AD acquisition module is used for acquiring input signals;

the signal preprocessing module is used for performing digital down-conversion processing and down-sampling processing on the input signals acquired by the AD, and then storing the preprocessed data into the dual-port RAM;

the PCIe data transmission module is used for transmitting and storing the data preprocessed by the signal preprocessing module into the solid-state disk array;

the solid-state disk array is used for storing data preprocessed by the signal preprocessing module;

the self-checking module is used for performing fast Fourier transform on data stored in the solid-state disk array, judging whether the acquisition and storage system works normally or not according to the peak position after the transform, and stopping the acquisition if the acquisition and storage system works abnormally;

the power supply module supplies power to the high-speed data acquisition and storage system;

the serial port control module is used for real-time communication between the high-speed data acquisition and storage system and an external PC (personal computer) to complete system parameter setting; the system parameters comprise signal storage proportion and interval and signal amplitude;

the clock module is used for providing a working clock for the AD acquisition module, the signal preprocessing module, the PCIe data transmission module and the serial port control module.

Compared with the prior art, the invention has the following remarkable advantages: (1) the invention can realize the ultrahigh-speed acquisition and data storage of 3.6GHz through the high-speed ADC, LVDS transmission, speed reduction processing and the PCIe3.0 interface; (2) the invention can flexibly intercept the collected data according to the setting, and the data correspond to different signal amplitudes; (3) the invention can selectively store the collected data according to the setting so as to save the data transmission bandwidth; (4) the invention can insert data identifiers into the collected data; (5) the invention can carry out self-check on the acquired data so as to judge the working performance of the system.

Drawings

FIG. 1 is a block diagram of the high-speed data acquisition and storage system based on FPGA and PCIe in accordance with the present invention.

Fig. 2 is a block diagram of a clock module in the system of fig. 1.

Fig. 3 is a block diagram of a reset module in the system of fig. 1.

Fig. 4 is a block diagram of an AD acquisition module in the system shown in fig. 1.

Fig. 5 is a block diagram of a signal preprocessing module in the system of fig. 1.

FIG. 6 is a block diagram of a PCIe data transfer module in the system of FIG. 1.

Detailed Description

With reference to fig. 1, the high-speed data acquisition and storage system based on FPGA and PCIe of the present invention includes an AD acquisition module, a signal preprocessing module, a dual-port RAM, a PCIe data transmission module, a solid state disk array, a self-checking module, a power supply module, a serial port control module, and a clock module;

the AD acquisition module is used for acquiring input signals;

the serial port control module is used for real-time communication between the high-speed data acquisition and storage system and the PC to complete system parameter setting; the system parameters comprise signal storage proportion and interval and signal amplitude;

Further, the AD acquisition module includes:

the AD chip is used for collecting input signals;

the AD configuration module is used for selecting the working mode of the AD chip according to the user requirement;

and the data acquisition module is used for carrying out speed reduction processing on the input signals acquired by the AD chip through ISERDES primitives.

Further, the signal preprocessing module comprises:

the signal amplitude control module is used for carrying out data truncation on the preprocessed data and controlling the amplitude of a signal;

the data storage selection module is used for controlling the proportion and the interval of the stored data;

the identifier inserting module is used for inserting an identifier in front of the cached data block;

and the data caching module is used for caching the preprocessed data, controlling the two double-port RAMs and realizing the pipeline operation by using a ping-pong structure.

Further, the PCIe data transmission module is configured to transmit the data preprocessed by the signal preprocessing module to an external PC through a PCIe3.0 interface and store the data in the solid state disk array.

Further, the clock module includes:

the clock chip is used for generating a stable clock;

the clock chip configuration module is used for configuring the working mode of the clock chip;

and the reset module is used for enabling the AD acquisition module, the signal preprocessing module, the PCIe data transmission module and the serial port control module after the clock chip configuration module generates a stable clock.

The invention is further elucidated with reference to the drawings and the specific embodiments.

Examples

With reference to fig. 1, the high-speed data acquisition and storage system based on FPGA and PCIe of this embodiment includes an AD acquisition module, a self-checking module, a signal preprocessing module, a dual-port RAM, a PCIe data transmission module, a solid-state disk array, a power supply module, a serial port control module, and a clock module;

the AD acquisition module is used for acquiring signal data, acquiring the received echo signal at a sampling rate of 8-bit width of the highest 3.6GHz, and transmitting the data to the FPGA for preprocessing;

the signal preprocessing module is used for preprocessing the signals acquired by the AD and caching the preprocessed data into the dual-port RAM;

the PCIe data transmission module is used for transmitting the data of the cache unit to the PC through a PCIe3.0 interface and storing the data in a solid disk array of the PC so as to be used for subsequent signal processing;

the power supply module is used for supplying power to the acquisition and storage system;

the serial port control module is used for real-time communication between the PCIE data transmission module and external equipment;

the clock module is used for providing a working clock for the AD acquisition module, the PCIE data transmission module and the serial port control module.

With reference to fig. 2 and 3, the clock module includes:

the configuration modules of the clock chips SI5344 and ADF4351 generate clocks with the highest 3.6GHz according to the requirements;

and the reset module is used for enabling the AD acquisition module, the signal preprocessing module, the PCIe data transmission module and the serial port control module after the configuration of the clock chip is finished.

The clock module works as follows:

s11, the crystal oscillator of the system board card provides a 14.7456MHz clock signal to be sent to the FPGA global clock pin;

s12, the FPGA accesses the clock to a global clock buffer inside the chip to increase the driving capability of the clock; then the clock is used as a driving clock on the SPI bus and is used for configuring working modes of an SI5344 clock chip and an AD chip;

s13, after the clock chip SI5344 is configured with the operating mode, the external input reference clock signal is used as the input signal, and the two clock signals with different frequencies are respectively output through the frequency multiplier and divider inside the chip: the 200MHz clock signal is output to a chip ADF 4351; the output 60MHz signal is sent to a clock management module of the FPGA, and the FPGA outputs the frequency multiplication or frequency division of the DCM for a working clock of an internal circuit of the FPGA; the reference clock signal is a 100MHz reference signal provided by the microwave unit;

s14, the ADF4351 chip multiplies the frequency of the input 200MHz clock to the maximum 3.6GHz, and sends the input 200MHz clock to the AD chip to be used as a sampling working clock.

The reset module realizes the following specific functions:

and S21, powering on the system board card, providing a 14.7456MHz clock signal by the crystal oscillator to a global clock pin of the FPGA, accessing the clock to the BUFG module in the chip by the FPGA, and enabling all modules on the board card to be in a reset or non-enabled state before the BUFG module outputs no clock.

S22, using the clock signal with enhanced drive ability through BUFG as the drive clock on the SPI bus, starting to configure the working modes of the SI5344 clock chip and the AD chip, and after the SPI configuration of the above 2 chips is completed, setting high SI5344_ config _ done and AD _ config _ done signals respectively to indicate that the chip configuration is successful.

S23, the system detects whether the SI5344_ config _ done signal is set high, if the set high represents that the SI5344 chip starts to output a 200MHz clock signal at the moment, the reset module sets the reset signals of the clock chip ADF4351 and the FPGA to be invalid at the moment.

S24, when the ADF4351 clock chip is configured and starts to output the highest 3.6GHz clock, detecting whether the FPGA and the AD _ config _ done are configured or not; when the conditions are met, the reset module sets the reset signal of the AD chip to be invalid, and the AD chip starts to work normally at the moment.

With reference to fig. 4, the AD acquisition module includes:

the AD configuration module selects a working mode of an AD chip according to user requirements, and the chip used in the embodiment is ADC12D 1800;

and the data acquisition module is used for carrying out speed reduction processing on the high-speed data acquired by the AD in the FPGA.

The sampling rate of the path is 3.6GHz at most, and the bit width is 8 bits. The AD acquisition module works as follows:

s31, the SPI configuration of the ADC12D1800 assigns values to corresponding registers according to the chip manual thereof, and collection is carried out by selecting a DES _1:4DEMUX working mode.

And S32, after the configuration is finished, the FPGA receives and splices the data acquired by the AD. The data processing of the ADC12D1800 is to concatenate four 12-bit data I, Id, Q, Qd of two channels. And finally splicing the data rate of each path into 4 paths by using 24 serial-parallel conversion modules, wherein the data rate is reduced to 225MHz and becomes I1, I2, I3, I4, Id1, Id2, Id3, Id4, Q1, Q2, Q3, Q4, Qd1, Qd2, Qd3 and Qd4 according to the acquisition sequence Qd4, Id4, Q4, I4, Qd3, Id3, Q3, I3, Qd2, Id2, Q2, I2, Qd1, Id1, Q1 and I1.

Referring to fig. 5, the signal preprocessing module includes:

and the data caching module is used for caching the preprocessed data in the dual-port RAM.

The signal preprocessing module realizes the following specific functions:

and S41, the data acquired by the AD is 12-bit, and 8-bit is intercepted by combining the system precision requirement and the PCIe transmission data characteristic. Specifically intercepting the low 8 bits or the high 8 bits, which are set by an upper computer program, can intercept signals from the low 8 bits to the high 8 bits and correspond to different signal amplitudes.

And S42, setting a storage enabling signal for selectively storing the data acquired by the AD. Only when the signal is enabled, the signal is stored. The triggering and resetting of the signal is controlled by the upper computer program. By setting different parameters and controlling the proportion and the interval of the stored data, the bandwidth of data transmission is fully utilized, and the subsequent signal processing is conveniently carried out by an upper computer.

At S43, the stored data is identified by inserting a data identifier because of the discontinuity of the stored data caused by the storage enable. The data is not stored uninterruptedly, but is stored in segments, and the corresponding buffer space is an individual data block. An identifier is inserted in front of each data block of the cache, thereby distinguishing different data blocks. When the upper computer processes data, each data block is distinguished by retrieving the identifier.

And S44, storing the preprocessed data into the double-port RAM, and using a ping-pong structure to conveniently take out the data from the RAM and send the data to PCIe.

With reference to fig. 6, the PCIe data transfer module includes:

and the clock module is used for providing a clock for the DMA control module. The mainboard provides a 100MHz clock to the FPGA, and outputs a 250MHz clock for the DMA control module after the PLL of the clock management module.

And the configuration module is used for setting the parameters of the DMA control module. After the system is powered on, an initialization operation is performed, and in this process, parameters of the DMA control module are set, including the data volume that can be loaded by a single TLP packet and the number of TLPs for a single data transfer.

And the DMA control module transmits data to the solid-state disk array of the PC through the PCIe bus in a DMA working mode. The specific working mode of the PCIe data transmission module is as follows:

s51, PCIe transfers data through TLP packets, the maximum number of TLP packets that can be loaded by a single TLP is 1024 double words, considering the performance of actual hardware and the stability of the system, in this embodiment, each TLP packet loads 128B data, and 32768 TLP packets are transferred at a time when transfer is started each time, that is, 4MB data is transferred at a time.

S52, the preprocessed data is buffered in the buffer unit, and when the data amount in the buffer unit reaches 4MB, that is, the data amount of one data transmission is reached, the enable signal is triggered, and when the trigger signal is detected, the DMA control module starts one data transmission. At the moment, the DMA control module reads data from the cache unit under a 250MHz clock, reads 64-bit per beat and transmits the data to the PC through the PCIe bus.

And S53, after the 4MB data is transmitted, ending the transmission operation, writing the transmitted data into a file by the upper computer program, resetting each control signal and state signal by the DMA control program, and waiting for the triggering of the next enabling signal.

The solid state disk array and the RAID array card form a data storage module of the system; the RAID array card selects RAID9271CV-8IRAID array card of LSI company, is connected with PC through PCIe3.0 interface, and is connected with solid disk array through special SATA line to realize the requirement of high-speed large-capacity storage; the solid-state disk array is formed by 8 Samsung 850PRO256GB solid-state disks, the read-write speed of a single solid-state disk is 500MB/s, the read-write speed of the formed solid-state disk array reaches 4GB/s, and the speed requirement of system data transmission is met.

Claims

1. A high-speed data acquisition and storage system based on FPGA and PCIe is characterized by comprising an AD acquisition module, a signal preprocessing module, a double-port RAM, a PCIe data transmission module, a solid-state disk array, a self-checking module, a power supply module, a serial port control module and a clock module;

the AD acquisition module is used for acquiring input signals;

2. The FPGA and PCIe based high speed data acquisition and storage system of claim 1, wherein the AD acquisition module comprises:

the AD chip is used for collecting input signals;

3. The FPGA and PCIe based high speed data acquisition and storage system of claim 1, wherein the signal preprocessing module comprises:

and the data caching module is used for caching the preprocessed data.

4. The FPGA and PCIe based high-speed data acquisition and storage system according to claim 1, wherein the PCIe data transmission module is configured to transmit the data preprocessed by the signal preprocessing module to an external PC through a PCIe3.0 interface and store the data in the solid-state disk array.

5. The FPGA and PCIe based high speed data collection and storage system of claim 1 wherein said clock module comprises:

the clock chip is used for generating a stable clock;