CN114064547B - PCIe-based communication protocol stack hardware acceleration architecture construction method - Google Patents

Abstract

The invention relates to a method for constructing a PCIe-based communication protocol stack hardware acceleration framework, belonging to the technical field of communication. The present invention adopts the PCIe general interface to realize the data interaction between the communication protocol stack and the FPGA, the communication protocol stack simulation platform is built on the PC, and the FPGA uses the AXI bus architecture for data transmission; the software interface of the PCIe general interface is integrated in the communication protocol stack simulation platform, XDMA IP core is set in the hardware interface; MIG, AXIDMA, DDR and FIFO IP core are set in FPGA. The invention realizes the transfer of data from the AXI4 bus to the AXI4-Stream bus through AXIDMA, accelerates the internal data transmission speed of the FPGA, and at the same time ensures the timing synchronization in the circuit through the FIFO; through the host computer interrupt clear setting, it can be sent to the FPGA without limit The data will not be interrupted; the PCIe universal interface has strong applicability.

Description

A PCIe-based communication protocol stack hardware acceleration architecture construction method

technical field

The invention belongs to the technical field of communication, and relates to a hardware-in-the-loop simulation technology of signal transmission and signal processing in a communication protocol stack, in particular to a method for constructing a communication protocol stack hardware acceleration framework based on PCIe (high-speed serial computer expansion bus standard).

Background technique

In an environment where the amount of 5G information is rapidly increasing, high-speed data transmission has been focused on. The main characteristics of high-speed data transmission are large data volume and fast data transmission speed, and video images have always accounted for a large proportion of transmission, so how to transmit video images at high speed and accurately in communication systems has always been the focus of expert research.

In the data transmission system, the data transmission bus protocol initially follows the ISA (Industry Standard Architecture) bus, because it can only support 16-bit I/O (input/output) devices, and the maximum transmission speed is only 16MB/s, which has already Can not keep up with the rapid development of the field of data transmission and be eliminated. The second-generation transmission protocol is the PCI (Peripheral Component Interconnection Standard) bus, which adopts parallel transmission and has stronger versatility and development, but the maximum speed is only 133MB/s, which cannot meet the existing high-speed data transmission system. The PCIe bus overcomes the shortcomings of the PCI bus and greatly improves the performance of data transmission. It uses end-to-end, full-duplex, and differential signals for signal transmission, avoiding interference between signals, and can satisfy the exclusive channel of the device. Bandwidth improves the quality of data transmission and is now widely used in high-speed data transmission systems, such as: small embedded systems, large server systems, etc.

Compared with general-purpose computers, FPGA (Field Programmable Logic Gate Array) has a faster calculation speed. FPGA can run in parallel during signal processing, and the processing speed is fast, but it is not as good as PC (personal computer) computers for complex precision operations. PC computers have considerable Powerful data processing capability, under normal circumstances, high-precision and complex calculations will be handled by the computer, and low-complexity and large-scale calculations will be performed by the FPGA. The above situation involves the problem of large-scale data transmission between PC and FPGA, so building a high-speed data transmission system between PC and FPGA has become a current research trend. The hardware-in-the-loop simulation of signal transmission and signal processing in the communication protocol stack is helpful to the research of the above problems. However, in the communication system, audio and video account for a large proportion, and it is difficult to simulate the communication protocol stack in kind.

Contents of the invention

The purpose of the present invention is to provide a PCIe-based communication protocol stack hardware acceleration architecture construction method, which is suitable for video and image transmission scenarios, and uses the PCIe interface to complete the interaction between the PC protocol stack and the FPGA and the acceleration of signal processing, providing data interaction between software and hardware. An effective solution. The invention transfers the video image in the real scene into the simulation protocol stack, constructs a complete hardware-in-the-loop simulation platform of the video image transmission protocol stack, and provides an idea for the communication protocol stack hardware-in-the-loop simulation.

The PCIe-based communication protocol stack hardware acceleration framework construction method implemented by the present invention adopts the PCIe general interface to realize the data interaction between the communication protocol stack and the FPGA, the communication protocol stack simulation platform is built on the PC, and the FPGA uses the AXI bus architecture for data transmission . The PCIe general interface includes a PCIe general interface software interface and a PCIe general interface hardware interface. The PCIe general interface software interface is integrated in the communication protocol stack simulation platform. The PCIe general interface hardware interface is provided with an XDMA IP core. The FPGA is provided with MIG IP core, AXIDMA IP core, DDR (Double Rate Synchronous Dynamic Random Access Memory) and FIFO (First In First Out) memory.

In the described PCIe general interface hardware interface, the pcie_mgt interface of XDMA is set to receive the data that host computer imports, then by M_AXI interface, data output is saved in the DDR of FPGA by MIG, and this moment data is on AXI4 bus line; MM2S channel of AXIDMA Obtain the data in the DDR and convert it into data on the AXI4-Stream bus; the FPGA internally uses the AXI4-Stream bus to read data; after the data processing module of the FPGA completes the data processing, it first transmits the data into the M_AXIS of the FIFO The interface synchronizes the clock signal in the circuit, and then returns the data from the S_AXIS of the FIFO to the DDR; the M_AXI interface of XDMA reads the data in the DDR to its own memory through the MIG and returns it to the host computer through the pcie_mgt interface;

In the PCIe general interface hardware interface, the usr_irq_req interface of XDMA is set to receive the interrupt signal generated by AXIDMA and transmit it to the host computer; the control command transmitted by PC to FPGA is transmitted by AXI4-Lite bus.

Described upper computer PC judges the situation that data is transmitted in FPGA by the interruption signal that AXIDMA produces in FPGA, receives the interruption information of AXIDMA by the usr_irq_req interface of XDMA and transmits to PC; When interruption signal changes 0 by 1, represents AXIDMA normal work , data can be transmitted; when the interrupt signal changes from 0 to 1, it means that the data transmission has ended, and the interrupt needs to be cleared to continue to transmit data.

Compared with the prior art, the advantages and positive effects of the present invention are: (1) the hardware circuit of the PCIe universal interface designed by the present invention has realized the movement of data from the AXI4 bus to the AXI4-Stream bus by the AXIDMA, which has accelerated the FPGA internal data The speed of transmission, while ensuring the timing synchronization in the circuit through FIFO. (2) The present invention also realizes the setting of interrupt clearing of the upper computer, which ensures that the PC can send data to the FPGA without restriction without being interrupted. (3) The PCIe universal interface in the present invention realizes the function of moving data to the FPGA for data processing, and decouples the PCIe interface from specific programs at the same time, so that any C program can be called, and the application applicability is strong.

Description of drawings

Fig. 1 is the schematic diagram of the PCIe-based communication protocol stack hardware acceleration framework of the present invention;

Fig. 2 is the main structural design schematic diagram of PCIe universal interface hardware circuit of the present invention and FPGA;

Fig. 3 is a schematic flow diagram of upper computer transmitting data and controlling in the framework of the present invention;

Fig. 4 is the schematic diagram of ILA debugging position in the embodiment of the present invention;

Fig. 5 is the functional diagram of the acquisition and restoration module of the image provided by the present invention;

Fig. 6 is the schematic diagram of ILA data collection result in the embodiment of the present invention; (a) XDMA to the data of DDR; (b) DDR to the data of AXIDMA; (c) AXI4 bus line is converted to the data of AXI4-Stream bus line; (d) DDR returns data to XDMA;

FIG. 7 is a grayscale image of images displayed at the receiving end under different SNR (Signal-to-Noise Ratio) in an embodiment of the present invention.

Detailed ways

The present invention will be further described in detail with reference to the accompanying drawings and embodiments.

In the video transmission scenario, the embodiment of the present invention provides a method for constructing a PCIe-based communication protocol stack hardware acceleration architecture. As shown in FIG. The data collected by the image acquisition module is used as the user data of the protocol stack simulation platform, which is input into the simulation platform through the protocol stack data stream input interface; the PCIe general interface is used as the data transmission channel to realize the data interaction between the communication protocol stack and FPGA, and complete the communication protocol stack data. The processing function is transferred to FPGA for hardware simulation acceleration function. The PCIe general interface is implemented by both software and hardware. The PCIe general interface software interface is set on the PC. The communication protocol stack simulation platform accesses the PCIe general interface hardware interface through the PCIe general interface software interface.

In terms of hardware, the implementation of the present invention proposes to use the AXI bus architecture in the FPGA for communication, and use XDMA as the PCIe bus IP core in the hardware circuit. AXI buses include AXI4, AXI4-Stream and AXI4-Lite buses. XDMA accesses FPGA internal data through the AXI4 (Advanced eXtensible Interface) bus, and the PC must pass the data through the access address when accessing the AXI4 bus, while the AXI4-Stream bus can directly read and write data without accessing the address compared to the AXI4 bus. The speed of data transmission is accelerated, so the present invention proposes to use the AXI4-Stream bus to read data inside the FPGA. At the same time, DDR and AXIDMA are used to complete the data transfer function from the AXI4 bus to the AXI4-Stream bus.

The hardware circuit of the PCIe general interface is shown in Figure 2, using the XDMA IP core. IP cores such as MIG, AXIDMA, DDR and FIFO memory are set in the FPGA. DDR is the data memory in FPGA; FIFO is mainly responsible for adjusting the clock synchronization problem in the circuit. XDMA is mainly responsible for the data transmission of DDR inside PC and FPGA, and is the controller of PCIe data transmission; MIG is mainly responsible for the data communication between AXI4 bus and DDR in FPGA; AXIDMA has the function of data transfer, which can move the data on AXI4 bus to AXI4 -Stream and AXI4-Lite bus come up. The present invention improves the interaction logic among signals such as time, data and control among IP cores, which will be described in detail below.

First, the incoming data from the PC is received by the pcie_mgt interface of XDMA, and the M_AXI interface outputs the data and saves it in DDR. At this time, the data is on the AXI4 bus. As shown in Figure 2, the chip XDMA_0 receives PC data from the pcie_mgt interface, and inputs the module MIG_7series_0 through the slave devices AXI_interconnect_0 and AXI_interconnect_1 to control the DDR output to the FPGA. After the data transmission from PC to FPGA is completed, FPGA will process the received data.

Secondly, the MM2S channel of AXIDMA acquires the data in DDR and converts it into data on the AXI4-Stream bus. After the FPGA completes the data processing, it first transfers the data into the M_AXIS interface of the FIFO to synchronize the clock signal in the circuit, then the data stream flows out from the S_AXIS of the FIFO and returns to the DDR, and reads the data in the DDR through the MIG on the M_AXI interface of the XDMA to its own memory and then returned to the PC through the pcie_mgt interface, and the data transmission from the PC to the FPGA is completed.

In this framework, PC judges the situation that data is transmitted in FPGA by the interruption signal that AXIDMA produces in FPGA, receives the interruption information of AXIDMA by the usr_irq_req interface of XDMA and transmits interruption information to PC; In the embodiment of the present invention, when interruption signal is transmitted by 1 changes to 0, which means that AXIDMA is working normally and can transmit data; when the interrupt signal changes from 0 to 1, it means that the data transmission has ended, and the interrupt needs to be cleared to continue to transmit data. The control commands of the FPGA are handled by the AXI4-Lite bus. This method separates the data bus and the command bus, which improves the convenience of the system. In Figure 2, the M_AXI_LITE interface of XDMA transmits the control signal of PC to the S_AXI_LITE interface of AXIDMA to control the opening and closing of AXIDMA. Since XDMA, DDR and AXIDMA all use the AXI bus, the present invention uses the AXI Interconnect to realize the interconnection of the AXI bus, which improves the interconnection timing and system performance. When the FPGA processes data, there will be timing asynchrony, and the timing of each module can be synchronized through the FIFO.

After the hardware circuit is built successfully, the PC protocol stack sends transmission data and control information to the FPGA by accessing the PCIe device file. The design idea of the PC software is shown in Figure 3: first, the PC stores the data to be transmitted in a continuous memory, and uses memory mapping to access the PCIe device file; accesses the h2c interface to transfer the data to the DDR with a unique address, and accesses the The c2h interface fetches the data from the DDR corresponding to the address. Secondly, refer to the AXIDMA user manual to access the AXIDMA register, control the AXIDMA switch and clear the interrupt; access the event interface to obtain the AXIDMA interrupt status, and clear the interrupt if the interrupt exists. As shown in Figure 3, specifically, the process of PC protocol stack sending transmission data and control information to FPGA by accessing PCIe device files includes: (1) memory mapping to lite interface, transmission command control to AXIDMA; (2) memory (3) Start the MM2S channel of AXIDMA, and move the data from DDR to the AXI4_Stream interface of AXIDMA. After a data transmission is completed, AXIDMA generates an interrupt signal and sends it to the PC protocol stack. After the interrupt is cleared, the data can continue to be transmitted data; (4) perform hardware acceleration processing on the data in the FPGA; (5) start the S2MM channel of AXIDMA to move the data, move the processed data to the DDR, and generate an interrupt signal to the PC protocol stack at the same time, clear the interrupt signal can Continue to move the data; (6) transfer the data back to the protocol stack from the DDR.

Since the simulation of the two IP cores of XDMA and DDR is difficult and the workload is heavy, the present invention proposes a transmission architecture verification using upper computer communication and FPGA online logic analysis. In FPGA online logic analysis, the position of ILA (Integrated LogicAnalyzer, integrated logic analyzer) is shown in Figure 4. Add the ILA IP core to the input and output interface of XDMA, the data input and output interface of DDR and the input and output interface of AXIDMA Take the signal, and then observe whether the signal transmission in the FPGA is correct.

Open the hardware manager of VIVADO, and the debugging interface will be displayed. The ILA capture signal is realized by setting the trigger condition. The AXI bus follows the handshake protocol. Only when the tready and tvaild signals are high at the same time, the data can be transmitted. The ready and tvaild signals are AXI4-Stream interface signals. The tvaild signal indicates that the master device is driving an effective transmission. The tready signal indicates that the slave device can receive a transmission in the current cycle. When both the ready and tvaild signals are set to high, the data can be transmitted. . Set the tready or tvaild signal as the trigger condition, transmit the control signal to the DMA register through the PC to change the tready signal, and then debug the program step by step, and observe the data captured by the ILA to complete the verification of the transmission architecture proposed by the present invention. Note that after each trigger signal is triggered, it is necessary to reset the trigger condition to capture the next signal. Change the capture depth to determine the data ratio before and after capture, so as to facilitate the comparison of data transmission before and after.

The OAI (OpenAirInterface) project is a 3GPP protocol stack emulation platform based on the C language under the Linux system, and the PCIe general interface software interface (i.e. software driver) and the host computer program are also written based on the C language, so the present invention uses the PCIe general interface The software interface program is integrated into the OAI project, and the PCIe host program is encapsulated into a function. The input parameter of the function is the transmission data, and the return value is the data processed by the FPGA. Define a PCIe_interface file to store the PCIe_interface function according to the OAI compilation rules, encapsulate this file in the tools folder, add macro definitions and header files to other files, and add compilation statements to the OAI corresponding compilation file. After verification, the PCIe general interface software interface is successfully integrated into the physical layer of the OAI protocol stack, and this interface can be called at a specific place in the simulation system.

An image acquisition and restoration module is also set in the communication protocol stack simulation platform of the upper computer. The invention transfers the video images collected in the real scene into the protocol stack simulation platform, and simulates the hardware acceleration framework of the protocol stack in the video transmission scene. Since the protocol stack simulation platform is built under the Linux system, devices with UVC functions are used to capture video images. V4L2 is an API interface for realizing UVC functions under the Linux operating system. The protocol stack can write video image acquisition and restoration modules through this interface. . Take the OAI downlink simulation link as an example: as shown in Figure 5, replace the randomly generated user data in OAI with video images collected by the device, and back up the data, and compare the before and after data after the data processing is completed.

Due to the large amount of video image information collected from the outside world, compression coding can be used to compress video images to reduce the number and video transmission time, and the compressed data is then input to the protocol stack. In the downlink analog channel at the link level, the image acquisition and restoration module of the simulation platform stores the amount of transmitted user data in a list, and the amount of data transmitted in one frame time can be determined by setting different parameters. Since the memory occupied by the video image is not fixed, the amount of data transmitted by one frame of the OAI simulation platform cannot complete the video transmission, so the image acquisition and restoration module divides the video into n frames for transmission. If the remaining data of the last frame is less than the fixed transmission length, then Add 0 at the end, and n is a positive integer. In the data link of each frame, the data stream is input before the sending end, and the receiving end decodes and receives the data stream and stores it in a blank memory, and then saves the data of the next frame in the last bit address of the previous frame data place. After the video transmission ends, the receiving end receives all the data, and the present invention converts the data in the memory into a visualized image, and observes the data transmission of images under different SNRs.

In order to verify the correctness of the common interface, the data passed into the FPGA is returned to the PC without any processing. When the data in the simulation protocol stack is transmitted to the FPGA through the PCIe general interface, the PCIe general interface should be verified according to the debugging process proposed in Figure 4. The data captured by ILA is shown in Figure 6. From the figure, it can be seen that the data from XDMA to DDR is the same as the data from DDR to AXIDMA, and the data returned from AXIDMA to XDMA is also the same. The PCIe universal interface verification is successful.

Taking the OAI simulation platform as an example, the PCIe_interface general software interface is embedded in the platform, and the PCIe general software interface is invoked in the downlink simulation platform, and the data before and after the input interface are compared, and the data are exactly the same. Through the above verification, it can be determined that PCIe has been successfully integrated into the OAI simulation platform. Correspondingly, for other simulation platforms based on C language, only need to import the PCIe_interface file designed by the present invention into the corresponding platform, the function of interacting with FPGA can be realized.

For the entire image transmission simulation platform, according to the scheme in Figure 5, the images collected by the camera are compressed and input into OAI, and the data before and after transmission are compared. Observe the effect of different SNR values on the transmitted data. As shown in Figure 7, the image collected by the camera is processed into a form that can be directly observed by the Linux system. The SNR value ranges from -10 to 19. With the increase of the SNR value, the image changes from being invisible to gradually complete and complete. The influence of SNR on image data transmission is shown, and the protocol stack simulation platform under the video transmission scenario is successfully built. The framework provided by the invention provides ideas for building a semi-physical simulation platform of the protocol stack, and has certain reference value.

Except for the technical features described in the instructions, all are known technologies by those skilled in the art. Descriptions of well-known components and well-known technologies are omitted in the present invention to avoid redundant description. The implementations described in the above examples do not represent all implementations consistent with the present application. On the basis of the technical solution of the present invention, various modifications can be made by those skilled in the art without creative work Or deformation is still within the protection scope of the present invention.

Claims (6)

1. A PCIe-based communication protocol stack hardware acceleration architecture construction method, using PCIe general interface to realize the data interaction between the communication protocol stack and FPGA, the communication protocol stack simulation platform is built on the upper computer PC, and the AXI bus architecture is used in the FPGA for data transmission; it is characterized in that, described PCIe general interface comprises PCIe general interface software interface and PCIe general interface hardware interface; PCIe general interface software interface is integrated in communication protocol stack emulation platform; Wherein, PCIe represents high-speed serial computer expansion bus standard , FPGA stands for Field Programmable Logic Gate Array; Described PCIe universal interface hardware interface is provided with XDMA IP core; Described FPGA is provided with AXIDMA IP core, MIGIP core, DDR and FIFO memory; Wherein, DDR represents double rate synchronous dynamic random access memory, and FIFO represents first-in-first-out; In the described PCIe general interface hardware interface, the pcie_mgt interface of XDMA is set to receive the data that host computer imports, then by M_AXI interface, data output is saved in the DDR of FPGA by MIG, and this moment data is on AXI4 bus line; MM2S channel of AXIDMA Obtain the data in the DDR and convert it into data on the AXI4-Stream bus; the FPGA internally uses the AXI4-Stream bus to read the data; after the FPGA completes the data processing, it first transmits the data into the M_AXIS interface of the FIFO, and synchronizes the circuit The clock signal in the FIFO, and then return the data from the S_AXIS of the FIFO to the DDR; the M_AXI interface of XDMA reads the data in the DDR to its own memory through the MIG and returns it to the host computer through the pcie_mgt interface; In the PCIe general interface hardware interface, the usr_irq_req interface of XDMA is set to receive the interrupt signal generated by AXIDMA and transmit it to the host computer; the control command transmitted by PC to FPGA is transmitted by AXI4-Lite bus. 2. a kind of communication protocol stack hardware acceleration framework construction method based on PCIe according to claim 1, is characterized in that, described upper computer PC judges the situation that data is transmitted in FPGA by the interruption signal that AXIDMA in FPGA produces, The usr_irq_req interface of XDMA receives the interrupt information of AXIDMA and transmits it to the PC; when the interrupt signal changes from 1 to 0, it means that AXIDMA is working normally and can transmit data; when the interrupt signal changes from 0 to 1, it means that the data transmission has ended, and the interrupt needs to be cleared. Continue to transfer data. 3. a kind of communication protocol stack hardware acceleration framework construction method based on PCIe according to claim 1, is characterized in that, the communication protocol stack emulation platform of described upper computer PC, when sending transmission data and control information to FPGA, first , store the data to be transmitted in a continuous memory, use memory mapping to access the PCIe device file; access the h2c interface to transfer the data to the DDR, access the c2h interface to retrieve the data from the DDR at the corresponding address; secondly, access the AXIDMA register , control the AXIDMA switch and clear the interrupt, the PC accesses the event interface to obtain the AXIDMA interrupt status, and clears the interrupt if the interrupt exists. 4. a kind of communication protocol stack hardware acceleration framework construction method based on PCIe according to claim 1, is characterized in that, is provided with image acquisition and reduction module in described communication protocol stack emulation platform, is used for reading acquisition Video images are stored in a list, determine the amount of data transmitted in one frame time, and divide the video into n frames for transmission, where n is a positive integer. 5. a kind of PCIe-based communication protocol stack hardware acceleration framework construction method according to claim 1, it is characterized in that, described framework adopts the mode of host computer communication to add FPGA online logic analysis to verify, input and in XDMA The output interface, the data input and output interface of DDR and the input and output interface of AXIDMA are added to the integrated logic analyzer ILA IP core to capture the signal, and then observe whether the signal transmission in the FPGA is correct. 6. a kind of PCIe-based communication protocol stack hardware acceleration framework construction method according to claim 1, is characterized in that, the implementation mode of described PCIe general interface software interface is: the PCIe host program is encapsulated into a function, and the function input The parameter is the transmission data, and the return value is the data processed by the FPGA; define a PCIe_interface file to store the PCIe_interface function according to the OAI compilation rules, and encapsulate the file in the tools folder, and add macro definitions and header files to other files, And add compilation statement in the compilation file corresponding to OAI.

Images