CN114064547B

CN114064547B - PCIe-based communication protocol stack hardware acceleration architecture construction method

Info

Publication number: CN114064547B
Application number: CN202111369382.1A
Authority: CN
Inventors: 王程; 王羽琪; 赵晓燕; 童建飞; 王卫东
Original assignee: Beijing University of Posts and Telecommunications
Current assignee: Beijing University of Posts and Telecommunications
Priority date: 2021-11-15
Filing date: 2021-11-15
Publication date: 2023-06-13
Anticipated expiration: 2041-11-15
Also published as: CN114064547A

Abstract

The invention relates to a communication protocol stack hardware acceleration architecture construction method based on PCIe, belonging to the technical field of communication. The invention adopts a PCIe universal interface to realize the data interaction between the communication protocol stack and the FPGA, the communication protocol stack simulation platform is built on the PC, and the FPGA uses an AXI bus architecture to carry out data transmission; the software interface of the PCIe universal interface is integrated in the communication protocol stack simulation platform, and the hardware interface is provided with an XDMA IP core; MIG, AXIDMA, DDR and FIFO IP cores are set in the FPGA. The invention realizes the movement of data from an AXI4 bus to an AXI4-Stream bus through AXIDMA, accelerates the internal data transmission speed of the FPGA, and ensures the time sequence synchronization in the circuit through FIFO; the upper computer is interrupted to clear the setting, so that data can be sent to the FPGA without limitation and is not interrupted; the PCIe universal interface has strong applicability.

Description

PCIe-based communication protocol stack hardware acceleration architecture construction method

Technical Field

The invention belongs to the technical field of communication, relates to a semi-physical simulation technology of signal transmission and signal processing in a communication protocol stack, and in particular relates to a hardware acceleration architecture construction method of the communication protocol stack based on PCIe (high-speed serial computer expansion bus standard).

Background

In an environment where the amount of 5G information is rapidly increasing, high-speed data transmission has been focused on. The main characteristics of high-speed data transmission are large data volume and high data transmission speed, and video images always occupy a large proportion in transmission, so how to transmit video images at high speed and accurately in a communication system is always the key point of expert research.

In data transmission systems, the data transmission bus protocol initially complies with the ISA (industry standard architecture) bus, and has been eliminated from the field of data transmission, which is developed at a high speed, because it can only support 16-bit I/O (input/output) devices, and the maximum transmission speed is only 16 MB/s. The second generation transmission protocol is PCI (peripheral component interconnect standard) bus, adopts parallel transmission, has stronger universality and development, but has the highest speed of only 133MB/s, and can not meet the existing high-speed data transmission system. The PCIe bus overcomes the disadvantages of the PCI bus, so that the performance of data transmission is greatly improved, and the PCIe bus adopts end-to-end, full duplex and differential signals to perform signal transmission, avoids interference between signals, can meet the requirement of the device for exclusive channel bandwidth, improves the quality of data transmission, and is widely used in high-speed data transmission systems, for example: small embedded systems, large server systems, etc.

Compared with a general purpose computer, an FPGA (field programmable gate array) has faster calculation speed, the FPGA can run in parallel during signal processing, the processing speed is high, but the operation with complex precision is not as good as that of a PC (personal computer) computer, the PC computer has quite strong data processing capability, the high-precision complex calculation is usually carried out by the computer, and the calculation with low complexity and large operation amount is carried out by the FPGA. The above situation relates to the problem of large-scale data transmission between the PC and the FPGA, so that the construction of a high-speed data transmission system of the PC and the FPGA becomes a current research trend. Semi-physical simulation of signal transmission and signal processing in a communication protocol stack is helpful for studying the above problems. But the audio and video in the communication system occupy a larger proportion, and the physical simulation difficulty for the communication protocol stack is larger.

Disclosure of Invention

The invention aims to provide a communication protocol stack hardware acceleration architecture construction method based on PCIe, which is suitable for video and image transmission scenes, and uses a PCIe interface to complete interaction of a PC protocol stack and an FPGA and acceleration of signal processing, thereby providing an effective solution for software and hardware data interaction. The invention transmits the video image in the real scene into the simulation protocol stack to construct a complete video image transmission protocol stack semi-physical simulation platform, and simultaneously provides a thinking for communication protocol stack semi-physical simulation.

The PCIe-based communication protocol stack hardware acceleration architecture construction method realizes data interaction between the communication protocol stack and the FPGA by adopting the PCIe universal interface, the communication protocol stack simulation platform is built on a PC, and the FPGA uses an AXI bus architecture to transmit data. The PCIe universal interface includes a PCIe universal interface software interface and a PCIe universal interface hardware interface. The PCIe universal interface software interface is integrated in the communication protocol stack emulation platform. An XDMA IP core is arranged in the PCIe universal interface hardware interface. The FPGA is provided with MIG IP core, AXIDMA IP core, DDR (double rate synchronous dynamic random access memory) and FIFO (first in first out) memory.

In the hardware interface of the PCIe universal interface, a pc ie-mgt interface of the XDMA is arranged to receive data transmitted by the upper computer, and then the M-AXI interface outputs the data and stores the data in the DDR of the FPGA through the MIG, and the data is on an AXI4 bus at the moment; the MM2S channel of AXIDMA obtains the data in the DDR and converts the data into data on an AXI4-Stream bus; the FPGA internally reads data by using an AXI4-Stream bus; after the data processing module of the FPGA finishes data processing, firstly transmitting the data into an M_AXIS interface of the FIFO, synchronizing clock signals in a circuit, and then streaming the data from an S_AXIS of the FIFO back to the DDR; the M_AXI interface of the XDMA reads the data in the DDR into the memory of the XDMA through the MIG, and the data is returned to the upper computer through the pcie_ mgt interface;

in the hardware interface of the PCIe universal interface, a usr_irq_req interface of the XDMA is set to receive an interrupt signal generated by AXIDMA and transmit the interrupt signal to an upper computer; control commands that the PC transmits to the FPGA interior are transmitted by the AXI4-Lite bus.

The upper PC judges the transmission condition of data in the FPGA through an interrupt signal generated by AXIDMA in the FPGA, and receives the interrupt information of AXIDMA through a usr_irq_req interface of the XDMA and transmits the interrupt information to the PC; when the interrupt signal is changed from 1 to 0, the AXIDMA works normally, and data can be transmitted; when the interrupt signal changes from 0 to 1, which indicates that the data transmission has ended, the interrupt needs to be cleared to continue transmitting data.

Compared with the prior art, the invention has the advantages and positive effects that: (1) The hardware circuit of the PCIe universal interface designed by the invention realizes the movement of data from an AXI4 bus to an AXI4-Stream bus through AXIDMA, accelerates the speed of data transmission in the FPGA, and ensures the time sequence synchronization in the circuit through FIFO. (2) The invention also realizes the setting of interrupt clearing of the upper computer, and ensures that the PC can transmit data to the FPGA without limitation and is not interrupted. (3) The PCIe universal interface realizes the function of moving data into the FPGA for data processing, and decouples the PCIe interface from a specific program, so that any C program can be called, and the application applicability is strong.

Drawings

FIG. 1 is a schematic diagram of a PCIe-based communications protocol stack hardware acceleration architecture of the present invention;

FIG. 2 is a schematic diagram of the main structural design of the PCIe universal interface hardware circuit and FPGA of the present invention;

FIG. 3 is a flow chart of the upper computer transmitting data and controlling in the architecture of the present invention;

FIG. 4 is a schematic diagram of an ILA debug location in an embodiment of the present invention;

FIG. 5 is a functional schematic of an image acquisition and reduction module according to the present invention;

FIG. 6 is a schematic diagram of ILA data acquisition results in an embodiment of the present invention; (a) XDMA to DDR data; (b) DDR to AXIDMA data; (c) data for conversion of the AXI4 bus to an AXI4-Stream bus; (d) DDR data returned to the XDMA;

fig. 7 is a gray scale image of a display of a receiving side image at different SNRs (signal to noise ratios) in an embodiment of the present invention.

Detailed Description

The invention will be described in further detail with reference to the drawings and examples.

In the embodiment of the invention, in a video transmission scene, as shown in fig. 1, a Linux system is installed on a PC, a communication protocol stack simulation platform is built under the Linux system, data acquired by a video image acquisition module is used as user data of the protocol stack simulation platform, and the user data is input into the simulation platform through a protocol stack data stream input interface; and the PCIe universal interface is used as a data transmission channel to realize the data interaction between the communication protocol stack and the FPGA, and the data processing function of the communication protocol stack is transferred to the FPGA to perform the function of hardware simulation acceleration. The PCIe universal interface is implemented by both software and hardware. The PCIe universal interface software interface is arranged on the PC. The communication protocol stack simulation platform accesses the PCIe universal interface hardware interface through the PCIe universal interface software interface.

In terms of hardware, the present invention proposes to use an AXI bus architecture for communication in an FPGA, using XDMA as the PCIe bus IP core in the hardware circuit. AXI buses include AXI4, AXI4-Stream, and AXI4-Lite buses. The XDMA accesses the internal data of the FPGA through the AXI4 (Advanced eXtensible Interface) bus, and the PC must transmit the data through the access address when accessing the AXI4 bus, and the AXI4-Stream bus can directly read and write the data without the access address relative to the AXI4 bus, so that the speed of data transmission can be increased, and therefore, the invention proposes to use the AXI4-Stream bus to read the data in the FPGA. And simultaneously completing an AXI4 bus to AXI4-Stream bus data moving function by using DDR and AXIDMA.

Hardware circuitry of the PCIe universal interface as shown in fig. 2, an XDMA IP core is used. An IP core such as MIG, AXIDMA, DDR and FIFO memories is provided in the FPGA. DDR is the data memory in the FPGA; the FIFO is mainly responsible for the clock synchronization problem in the regulating circuit. XDMA is mainly responsible for data transmission of DDR in PC and FPGA, and is a controller for PCIe data transmission; MIG is mainly responsible for data communication between an AXI4 bus and DDR in FPGA; AXIDMA has a data transfer function, and can transfer data on an AXI4 bus to an AXI4-Stream and an AXI4-Lite bus. The invention improves the interaction logic between the signals of time, data, control, etc. between the IP cores, as described in detail below.

First, the PC incoming data is received by the PC ie mgt interface of the XDMA, and the M AXI interface outputs and saves the data in the DDR, where the data is on the AXI4 bus. As shown in fig. 2, the chip xdma_0 receives PC data from the pcie_ mgt interface, and controls DDR output to the FPGA via the slave axi_interface_0 and axi_interface_1 input modules mig_7services_0. After the data transmission from the PC to the FPGA is completed, the FPGA processes the received data.

Second, the MM2S channel of AXIDMA takes the data in DDR and converts it to data on the AXI4-Stream bus. After finishing the data processing, the FPGA firstly transmits the data into the M_AXIS interface of the FIFO to synchronize the clock signals in the circuit, then the data flow flows out from the S_AXIS of the FIFO and returns to the DDR, the M_AXI interface of the XDMA reads the data in the DDR into the memory thereof through the MIG, and then the data is returned to the PC through the pcie_ mgt interface, thus finishing the data transmission from the PC to the FPGA.

In the framework, a PC judges the transmission condition of data in an FPGA through an interrupt signal generated by an AXIDMA in the FPGA, and receives interrupt information of the AXIDMA through a usr_irq_req interface of the XDMA and transmits the interrupt information to the PC; in the embodiment of the invention, the interrupt signal is changed from 1 to 0, which represents that AXIDMA works normally and can transmit data; when the interrupt signal changes from 0 to 1, which indicates that the data transmission has ended, the interrupt needs to be cleared to continue transmitting data. The control command of the FPGA is responsible for the AXI4-Lite bus, and the method separates the data bus and the command bus, so that the convenience of the system is improved. The m_axi_lite interface of the XDMA in fig. 2 transfers the control signal of the PC to the s_axi_lite interface of the AXIDMA to control the opening and closing of the AXIDMA. As the XDMA, DDR and AXIDMA all use the AXI bus, the invention uses the AXI Interconnect to realize the interconnection of the AXI bus, improves the interconnection time sequence and improves the system performance. When the FPGA processes data, time sequence is asynchronous, and the time sequences of all modules can be synchronized through the FIFO.

After the hardware circuit is successfully built, the PC protocol stack sends transmission data and control information to the FPGA by accessing the PCIe device file. The design concept of the PC software is shown in fig. 3: firstly, the PC stores data to be transmitted in a continuous memory, and accesses PCIe equipment files by memory mapping; the access h2c interface transmits the data to the DDR of the unique address, and the access c2h interface fetches the data from the DDR of the corresponding address. Secondly, accessing an AXIDMA register by referring to an AXIDMA user manual, controlling the switching of AXIDMA and clearing interrupt; the access event interface obtains the AXIDMA interrupt state, and if the interrupt exists, the interrupt is cleared. As shown in fig. 3, specifically, the process of sending transmission data and control information to the FPGA by accessing the PCIe device file by the PC protocol stack includes: (1) Memory mapping to a lite interface, and transmitting command control to AXIDMA; (2) passing the data in memory from PCIe to DDR; (3) Starting an MM2S channel of AXIDMA, moving data from DDR to an AXI4_stream interface of AXIDMA, completing one-time data transmission, generating an interrupt signal by AXIDMA and sending the interrupt signal to a PC protocol stack, and continuing to transmit the data after interrupt clearing; (4) performing hardware acceleration processing on the data in the FPGA; (5) Starting an S2MM channel of AXIDMA to carry out data movement, moving the processed data to DDR, generating an interrupt signal and sending the interrupt signal to a PC protocol stack, and removing the interrupt signal to continuously move the data; (6) transferring the data from DDR back to the protocol stack.

As the simulation difficulty of the XDMA and DDR two IP cores is high, the workload is heavy, and the invention provides the verification of the transmission architecture by adopting the upper computer communication and the FPGA on-line logic analysis. In the on-line logic analysis of the FPGA, the location of the ILA (Integrated Logic Analyzer ) is shown in fig. 4, and ILA IP core grabbing signals are added to the input and output interfaces of the XDMA, the data input and output interface of the DDR and the input and output interface of the AXIDMA, and then whether the signal transmission in the FPGA is correct is observed.

Turning on the hardware manager of VIVADO, a debug interface is displayed. The ILA grabbing signal is realized by setting a triggering condition, the AXI bus follows a handshake protocol, and data can be transmitted only when the device signal and the tvaild signal are at high level at the same time. The ready and tvaild signals are AXI4-Stream interface signals, tvaild signals indicate that the master is driving an active transmission, and the tready signal indicates that the slave is able to receive a transmission in the current period, and when both ready and tvaild signals are high, data can be transmitted. Setting a test or tvaild signal as a trigger condition, transmitting a control signal to the DMA register by the PC to change the test signal, then single-step debugging the program, and observing the data captured by the ILA, thereby completing the verification of the transmission architecture provided by the invention. Note that after each trigger signal is triggered, the trigger condition needs to be reset to perform the next signal capture. The ratio of the data before and after grabbing is determined by changing the grabbing depth, so that the comparison of the data before and after grabbing is convenient.

The OAI (OpenAirInterface) item is a 3GPP protocol stack simulation platform realized based on a C language under a Linux system, and a PCIe universal interface software interface (namely a software driver) and an upper computer program are also written based on the C language, so that the invention integrates the software interface program of the PCIe universal interface into an OAI item, packages a PCIe host program into a function, the function input parameter is transmission data, and the return value is data processed by an FPGA. Defining a PCIe_interface file according to the OAI compiling rule, storing the PCIe_interface function, packaging the file in a tools folder, adding the macro definition and the header file into other files, and adding compiling sentences into the compiling file corresponding to the OAI. Through verification, the PCIe universal interface software interface is successfully integrated into the physical layer of the OAI protocol stack, which interface can be invoked at a specific place in the emulation system.

And an image acquisition and restoration module is also arranged in the communication protocol stack simulation platform of the upper computer. The invention transmits the video image collected in the real scene into the protocol stack simulation platform to simulate the protocol stack hardware acceleration architecture in the video transmission scene. Because the protocol stack simulation platform is built under the Linux system, the device with the UVC function is used for collecting video images, V4L2 is an API interface for realizing the UVC function under the Linux operation system, and the protocol stack can write a video image collecting and restoring module through the interface. Taking OAI downlink emulation link as an example: as shown in fig. 5, the user data randomly generated in the OAI is replaced by the video image collected by the device, the data is backed up, and the front and rear data are compared after the data processing is completed.

Because the data volume of the video image information acquired by the outside is large, the video image can be compressed by adopting a compression coding mode, the number is reduced, the video transmission time is shortened, and the compressed data is input into a protocol stack. In the link-level downlink analog channel, the image acquisition and restoration module of the simulation platform stores the transmitted user data volume in a list, and can determine the data volume transmitted in one frame time by setting different parameters. Because the memory occupied by the video image is not fixed, the data volume transmitted by one frame of the OAI simulation platform cannot complete video transmission, the image acquisition and reduction module divides the video into n frames for transmission, and if the residual data of the last frame is less than the fixed transmission length, the end is complemented with 0, and n is a positive integer. In the data link of each frame, the data stream is input before the transmitting end, the receiving end receives the data stream after decoding and stores the data stream in a blank memory, and then the data of the next frame is stored at the last bit address of the data of the previous frame. After the video transmission is finished, the receiving end receives all data, the invention changes the data in the memory into a visual image, and observes the data transmission condition of the image under different SNR.

In order to verify the correctness of the generic interface, the data that is passed into the FPGA is returned to the PC without any processing. When data in the emulation protocol stack is transferred to the FPGA through the PCIe universal interface, the PCIe universal interface is to be verified according to the debug flow set forth in fig. 4. The results of the data grabbed by the ILA are shown in fig. 6, from which it can be seen that the data from the XDMA to the DDR is the same as the data from the DDR to the AXIDMA, as is the data returned from the AXIDMA to the XDMA, and that the PCIe universal interface verification is successful.

Taking an OAI simulation platform as an example, embedding a PCIe-interface universal software interface into the platform, calling the PCIe universal software interface in a downlink simulation platform, and comparing the data before and after the input interface, wherein the data are identical. Through the above verification, it may be determined that PCIe has been successfully integrated into the OAI emulation platform. Correspondingly, for other simulation platforms based on C language, the function of interacting with the FPGA can be realized by only importing the PCIe_interface file designed by the invention into the corresponding platform.

For the whole image transmission simulation platform, the image acquired by the camera is compressed and input into the OAI according to the scheme of fig. 5, and the data before and after transmission are compared. The effect of different SNR values on the transmitted data is observed. As shown in FIG. 7, the image collected by the camera is processed into a form that can be directly observed by a Linux system, the value of the SNR is from-10 to 19, and as the SNR value increases, the image is gradually and completely displayed from invisible to complete, the influence of the SNR on the image data transmission is completely shown, and a protocol stack simulation platform under a video transmission scene is successfully built. The framework provided by the invention provides a thought for building a protocol stack semi-physical simulation platform and has a certain reference value.

Other than the technical features described in the specification, all are known to those skilled in the art. Descriptions of well-known components and well-known techniques are omitted so as to avoid redundant description. The embodiments described in the above examples are not intended to represent all the embodiments consistent with the present application, and on the basis of the technical solutions of the present invention, various modifications or variations may be made by those skilled in the art without the need for inventive efforts, while remaining within the scope of the present invention.

Claims

1. A communication protocol stack hardware acceleration architecture construction method based on PCIe adopts PCIe universal interface to realize data interaction between communication protocol stack and FPGA, a communication protocol stack simulation platform is built on an upper computer PC, and AXI bus architecture is used for data transmission in FPGA; the PCIe universal interface comprises a PCIe universal interface software interface and a PCIe universal interface hardware interface; the PCIe universal interface software interface is integrated in the communication protocol stack simulation platform; PCIe represents a high-speed serial computer expansion bus standard, and FPGA represents a field programmable gate array;

an XDMA IP core is arranged in the PCIe universal interface hardware interface; the FPGA is provided with an AXIDMA IP core, an MIG IP core, a DDR and a FIFO memory; wherein DDR represents double rate synchronous dynamic random access memory, FIFO represents first in first out;

in the hardware interface of the PCIe universal interface, a pc ie-mgt interface of the XDMA is arranged to receive data transmitted by the upper computer, and then the M-AXI interface outputs the data and stores the data in the DDR of the FPGA through the MIG, and the data is on an AXI4 bus at the moment; the MM2S channel of AXIDMA obtains the data in the DDR and converts the data into data on an AXI4-Stream bus; the FPGA internally reads data by using an AXI4-Stream bus; after finishing data processing, the FPGA firstly transmits data into an M_AXIS interface of the FIFO, synchronizes clock signals in the circuit, and then flows the data from the S_AXIS of the FIFO back to the DDR; the M_AXI interface of the XDMA reads the data in the DDR into the memory of the XDMA through the MIG, and the data is returned to the upper computer through the pcie_ mgt interface;

2. The method for constructing a hardware acceleration architecture of a PCIe-based communication protocol stack according to claim 1, wherein the upper PC determines a data transmission condition in the FPGA by an interrupt signal generated by the AXIDMA in the FPGA, receives interrupt information of the AXIDMA through a usr_irq_req interface of the XDMA, and transmits the interrupt information to the PC; when the interrupt signal is changed from 1 to 0, the AXIDMA works normally, and data can be transmitted; when the interrupt signal changes from 0 to 1, which indicates that the data transmission has ended, the interrupt needs to be cleared to continue transmitting data.

3. The method for constructing the hardware acceleration architecture of the communication protocol stack based on PCIe according to claim 1, wherein when the communication protocol stack simulation platform of the upper PC sends the transmission data and the control information to the FPGA, firstly, the data to be transmitted is stored in a continuous memory, and the memory mapping is adopted to access the equipment file of PCIe; the access h2c interface transmits data to the DDR, and the access c2h interface extracts the data from the DDR corresponding to the address; and secondly, accessing an AXIDMA register, controlling the switch of AXIDMA and clearing interrupt, and acquiring the interrupt state of AXIDMA by a PC access event interface, and clearing interrupt if the interrupt exists.

4. The method for constructing the hardware acceleration architecture of the communication protocol stack based on PCIe according to claim 1, wherein an image acquisition and restoration module is arranged in the communication protocol stack simulation platform and used for reading acquired video images and storing the acquired video images in a list, determining the data quantity transmitted in one frame time, dividing the video into n frames for transmission, wherein n is a positive integer.

5. The method for constructing the hardware acceleration architecture of the communication protocol stack based on PCIe according to claim 1, wherein the architecture adopts a mode of upper computer communication and FPGA on-line logic analysis to verify, adds an integrated logic analyzer ILA IP core grabbing signal into an input and output interface of XDMA, a data input and output interface of DDR and an input and output interface of AXIDMA, and then observes whether signal transmission in the FPGA is correct.

6. The method for constructing a hardware acceleration architecture of a PCIe-based communication protocol stack according to claim 1, wherein the PCIe universal interface software interface is implemented as follows: encapsulating the PCIe host program into a function, wherein the function input parameter is transmission data, and the return value is data processed by the FPGA; defining a PCIe_interface file according to the OAI compiling rule, storing the PCIe_interface function, packaging the file in a tools folder, adding the macro definition and the header file into other files, and adding a compiling statement into a compiling file corresponding to the OAI.