CN111143208A

CN111143208A - Verification method for assisting FPGA to realize AI algorithm based on processor technology

Info

Publication number: CN111143208A
Application number: CN201911338986.2A
Authority: CN
Inventors: 曹会扬; 胡志勇; 田宝珠; 孙义兴; 潘红舟; 魏志猛; 陈姗
Original assignee: Shanghai Henglin Photoelectric Technology Co ltd; Hengtong Optic Electric Co Ltd; Jiangsu Hengtong Terahertz Technology Co Ltd
Current assignee: Shanghai Henglin Photoelectric Technology Co ltd; Hengtong Optic Electric Co Ltd; Jiangsu Hengtong Terahertz Technology Co Ltd
Priority date: 2019-12-23
Filing date: 2019-12-23
Publication date: 2020-05-12
Anticipated expiration: 2039-12-23
Also published as: CN111143208B; WO2021128781A1

Abstract

The invention discloses a verification method for assisting an FPGA to realize an AI algorithm based on a processor technology. The invention relates to a verification method for assisting an FPGA to realize an AI algorithm based on a processor technology, which comprises the following steps: transmitting parameters related to the data to be processed and AI algorithm realization to a non-parallel operation processor as the data to be processed of a verification comparison prototype; transmitting parameters related to the data to be processed and AI algorithm realization to an FPGA chip as the data to be processed of the design to be verified; the non-parallel operation processor caches the data to be processed, the process data, the processing result data and the parameters required by AI algorithm realization of the prototype in the data processing process. Has the advantages that: compared with the traditional FPGA software simulation, the method improves the verification efficiency, and can more accurately position and analyze errors compared with the traditional FPGA online debugging.

Description

Verification method for assisting FPGA to realize AI algorithm based on processor technology

Technical Field

The invention relates to the field of FPGA, in particular to a verification method for assisting FPGA to realize AI algorithm based on processor technology.

Background

With the continuous development of various technologies such as artificial intelligence, automatic driving, 5G, cloud computing and the like, various data can be continuously generated, and the total data amount is expected to increase by 10 times compared with the current data amount in the next 5 years. Since the above-mentioned technologies are developed based on the study and analysis of large data, people visually compare the data with petroleum in the future artificial intelligence era.

Referring generally to FPGAs, what was first thought of was a programmable piece of hardware, whether used on embedded devices or in terms of network transport acceleration, did not seem to have much direct contact with software. With the rise of new technologies such as artificial intelligence, automatic driving, 5G, cloud computing and the like, FPGAs are gradually coming into the field of view of the public due to their inherent advantages. Due to the fact that the FPGA has inherent characteristics of super performance and flexibility in hardware and software, when the AI is subjected to complex algorithm and multi-scene application and is difficult to fall to the ground, more and more innovators focus on the flexibly-strained FPGA and derivative products based on the FPGA, and the FPGA meets a huge development opportunity.

The FPGA has the advantages of low power consumption and high operational capability. However, in the above-mentioned large-data and large-operand processing of the FPGA, the highlight of the FPGA is not low power consumption but is accelerated. The CPU has a main frequency of several GHz, and the FPGA has a main frequency of only several hundred MHz. Seeing the main frequency alone, the FPGA seems to have no advantage, but actually the FPGA is used for accelerating the calculation, and the FPGA acceleration of Tencent cloud can realize the performance which is 30 times faster than that of a general CPU type server.

The CPU belongs to a von Neumann structure, and task execution needs to go through processes of fetching, decoding, executing, accessing and writing back and the like. To achieve a sufficiently high versatility, the CPU has a relatively complex instruction stream control logic. Even if the simple neural network is used for optimizing loop control by using C language, an additional instruction cycle is still needed after each multiplication is completed to judge whether to finish the loop and jump to the next calculation. And the FPGA can optimize and allocate hardware resources according to the algorithm, and saves unnecessary control steps and time. Although the CPU has high main frequency, a plurality of calculations still need several clock cycles to be completed, and the FPGA can basically complete multiple operations such as multiply-add, accumulation and the like in a single cycle by virtue of a large amount of parallel processing and a reasonable pipeline structure. In addition, in the calculation process of the CPU, the transmission bandwidth of the access device is often also a performance bottleneck, and the FPGA can fully utilize the distributed storage resources in the chip, and reduce data exchange with the external accessor as much as possible, thereby improving performance. In some cases, the CPU also has an external application specific coprocessor (ASIC) to speed up processing of a particular computation. However, the development period of the special chip is long, the cost is high, the application occasions are limited, and the computing resources cannot be dynamically adjusted as required like an FPGA. Therefore, high-performance computation cannot be accelerated by the FPGA, and especially deep learning related to artificial intelligence is achieved.

Hardware acceleration may have several different options from an implementation point of view: conventional processors, FPGAs, and ASICs. The traditional processor is most flexible and can cover a variety of different applications, but its power (efficiency) is the weakest. ASIC cost, performance and power consumption are best, but AISC development iteration cycles are relatively long. Currently, AI algorithms are infinite, and the long design period and high design cost of the ASIC cannot meet various requirements.

If programmability and efficiency are to be achieved at the same time, a display chip and an FPGA may be used. In terms of power consumption and efficiency, FPGAs are stronger than display chips. Especially in AI reasoning, the performance power consumption of FPGA is 16 times larger than that of display chip for low precision scene. The display chip is preferably used on the server side, while the FPGA is preferably used on the edge side. The FPGA is suitable for reasoning, and the display chip is suitable for training.

In addition to reasoning about direction on the line of AI, FPGAs can also play a role in many other ways. When the FPGA is oriented to compute-intensive tasks such as matrix operation, image processing, machine learning, compression, asymmetric encryption, sequencing of search and the like, the FPGA with pipeline parallelism and data parallelism has high efficiency.

At present, two verification methods for FPGA design exist, one is software simulation, and the other is real-time observation of an operation result through an online debugging tool.

The specific FPGA software simulation is divided into: first, the first simulation performed in most designs for RTL level behavioral simulation (also known as functional simulation, pre-simulation) will be RTL behavioral simulation. This stage of simulation can be used to check for errors in the code and the correctness of the code behavior, without including device-related information. The simulation at this stage can also be made device independent if some of the special underlying elements associated with the device are not instantiated. Second, the second simulation in the design flow is the integrated back gate level functional simulation. Most synthesis tools can output a Verilog or VHDL netlist in addition to a standard netlist file, wherein the standard netlist file is used for transferring design data among the tools and cannot be used for simulation, and the output Verilog or VHDL netlist can be used for simulation, so-called gate-level simulation because the simulation netlist given by the synthesis tool is already corresponding to the bottom component model of the device of the manufacturer. Thirdly, the last simulation of the time sequence simulation (also called post simulation) in the design flow is the time sequence simulation. After the layout and routing are completed, a timing simulation model can be provided, which also includes some information of the device, and an SDF timing annotation file (Standard Delay format timing) can be provided. xlinx uses SDF as the extension of the time series annotation file, and Altera uses SDO as the extension of the time series annotation file. While the functionality of the design has been defined in the initial stages of the design, it is only after the layout of the design is routed into one device that accurate delay information is available, at which stage the behavior of the circuit can be simulated relatively close to the actual circuit.

At present, a large artificial intelligence algorithm is hundreds of layers deep and thousands of neuron nodes, so that the detection of errors in design by software simulation even RTL-level behavior simulation is impossible in the process of accelerating the realization of the AI algorithm by using the FPGA, not to mention gate-level simulation and post-simulation, because the complicated AI algorithm software simulation is difficult to calculate and finish in a short time. The FPGA is used as a server to complete software simulation of a round of complex AI algorithm, which requires several weeks or even months, not to mention that the FPGA is used for simulating software to complete multi-round AI algorithm implementation (the FPGA is used for completing the one-round AI algorithm, namely completing AI algorithm processing once, for example, the AI algorithm identified by video is taken as an example, namely processing a frame in the video by using the AI algorithm, and the FPGA is used for completing the multi-round AI algorithm implementation, namely processing multiframes in the video by using the AI algorithm, namely processing continuous video streams by using the AI algorithm).

The on-line debugging tool of the FPGA combines the logic codes of the on-line debugging tool and the system design logic codes and sends the combination to FPGA development software for compiling, synthesizing, laying out and wiring, generating a programming file (the logic codes of the on-line debugging tool are contained in the programming file), and configuring the programming file into the FPGA. Once the FPGA meets the triggering condition of the signal to be tested during running, the logic of the line debugging tool is started immediately, the data of the signal to be tested is captured according to the frequency of the sampling clock and temporarily stored in the RAM in the FPGA chip, the memory in the chip is continuously refreshed by the sampling data, and finally the captured signal is transmitted to the FPGA development software from the RAM in the chip through the JTAG port to be displayed in real time. The FPGA online debugging principle flow is shown in FIG. 1.

The traditional technology has the following technical problems:

the on-line debugging method of the FPGA has the problems that on-chip resources (particularly on-chip RAM) of the FPGA are limited, so that only a small part of numbers in the running process of the FPGA can be stored and transmitted back for error detection in the design process of the FPGA, a large-scale artificial intelligent algorithm is often hundreds of layers deep at present, thousands of neuron nodes exist, and a large amount of operation results and process data are inevitably generated in the operation process of realizing the algorithms by the FPGA, so that the on-line debugging method of the FPGA cannot accurately position errors occurring in the operation process of realizing a complex AI algorithm by the FPGA at all, and the errors occurring in the operation process of accurately positioning the FPGA in the operation process of operating a plurality of rounds of AI algorithms are needless to say.

Disclosure of Invention

The invention aims to provide a verification method for assisting an FPGA to realize an AI algorithm based on a processor technology.

In order to solve the technical problem, the invention provides a verification method for assisting an FPGA to realize an AI algorithm based on a processor technology, which comprises the following steps:

transmitting parameters related to the data to be processed and AI algorithm realization to a non-parallel operation processor as the data to be processed of a verification comparison prototype;

transmitting parameters related to the data to be processed and AI algorithm realization to an FPGA chip as the data to be processed of the design to be verified;

the non-parallel operation processor caches the data to be processed, the process data, the processing result data and parameters required by AI algorithm realization of a prototype in the data processing process;

the FPGA chip caches parameters required for realizing data to be processed, process data, processing result data and AI algorithm of the design to be verified in the data processing process;

and comparing the process data and the operation result of the prototype with the process data and the operation result of the verification design, thereby analyzing and verifying whether the operation result of the FPGA in the process of realizing the algorithm and after the algorithm is finished is correct or not.

The invention has the beneficial effects that:

compared with the traditional FPGA software simulation, the verification efficiency is improved (because the modern AI algorithm is very complex, the complete AI algorithm can not be simulated by software simulation in the algorithm realization process of the FPGA, not to mention the operation process of simulating multi-round AI algorithms), and compared with the traditional FPGA online debugging, the method can more accurately position and analyze errors (aiming at the FPGA online debugging, because the on-chip resources, especially the storage resources, of the FPGA are limited, a large amount of operation results and process data can not be stored, so the FPGA online debugging has higher simulation efficiency compared with the FPGA software, but can not play the roles of accurately positioning and analyzing errors due to the overlarge data amount generated in the AI algorithm realization process).

In one embodiment, the method further comprises the following steps: and displaying the process data and the operation result of the comparison prototype through a display screen.

In one embodiment, the method further comprises the following steps: and displaying the process data and the operation result of the verification design through a display screen.

In one embodiment, the method further comprises the following steps: and displaying the comparison result of the process data and the operation result of the comparison prototype and the process data and the operation result of the verification design through a display screen.

In one embodiment, the method further comprises the following steps: acquiring an instruction of a user, wherein the instruction is used for setting a specific verification mode;

and analyzing and verifying whether the operation result of the FPGA in the algorithm implementation process and after the algorithm implementation is correct or not according to the set specific verification mode.

In one embodiment, the method further comprises the following steps: and transmitting the process data and the operation result of the comparison prototype to an external system.

In one embodiment, the method further comprises the following steps: and transmitting the process data and the operation result of the verification design to an external system.

In one embodiment, the method further comprises the following steps: and transmitting the comparison result of the process data and the operation result of the comparison prototype and the process data and the operation result of the verification design to an external system.

In one embodiment, the non-parallel operation processor is an ARM processor or a DSP processor.

In one embodiment, the non-parallel operation processor processes data by using C, C + + or Python language.

Drawings

Fig. 1 is a schematic diagram of an FPGA online debugging process in the background art in the verification method for assisting an FPGA to implement an AI algorithm based on a processor technology according to the present invention.

Fig. 2 is a functional block diagram of processor technology-assisted verification in the verification method for assisting the FPGA to implement the AI algorithm based on the processor technology.

Fig. 3 is a schematic block diagram of processor technology-assisted verification in the verification method for assisting the FPGA to implement the AI algorithm based on the processor technology.

Fig. 4 is a structural block diagram illustrating an example of processor technology-assisted verification in the verification method for assisting an FPGA to implement an AI algorithm based on the processor technology.

FIG. 5 is a flowchart of a verification method for assisting an FPGA to implement an AI algorithm based on processor technology according to the present invention.

Detailed Description

The present invention is further described below in conjunction with the following figures and specific examples so that those skilled in the art may better understand the present invention and practice it, but the examples are not intended to limit the present invention.

Referring to fig. 1, a verification method for assisting an FPGA to implement an AI algorithm based on a processor technology includes:

s110, transmitting parameters related to the data to be processed and AI algorithm realization to a non-parallel operation processor as the data to be processed of a verification comparison prototype;

s120, transmitting parameters related to the data to be processed and AI algorithm realization to an FPGA chip as the data to be processed of the design to be verified;

s130, caching parameters required by the prototype to-be-processed data, the process data, the processing result data and the AI algorithm in the data processing process by the non-parallel operation processor;

s140, caching parameters required by data to be processed, process data, processing result data and AI algorithm realization of the to-be-verified design in the data processing process by the FPGA chip;

s150, comparing and comparing the process data and the operation result of the prototype with the process data and the operation result of the verification design, and analyzing and verifying whether the operation result of the FPGA in the process of realizing the algorithm and after the algorithm is finished is correct or not.

The invention has the beneficial effects that:

It is understood that the specific verification mode may specifically set a certain layer of the AI algorithm for the verification FPGA and the processor, how many rounds of the AI algorithm are performed by the verification FPGA and the processor, and the like, for example.

As shown in the functional block diagram for auxiliary verification of processor technology in FIG. 2, the core part of the invention is processor (the processor is distinguished from FPGA and ASIC, which are technologies for completing operations in a concurrent processing manner, such as CPU, which is a technology for completing operations in a sequential processing manner), AI algorithm implementation 1 and FPGA AI algorithm implementation 2, which complete the same AI algorithm, and the difference is that the algorithm implementation completed by the processor is generally completed by high-level languages (such as C, C + +, Python, etc.), the algorithm implementation model is more mature and stable than the FPGA AI algorithm implementation, the method has the functions of auxiliary verification of comparison prototypes in the FPGAAI algorithm implementation design process, while the FPGA implementation 2 is generally completed by hardware description languages (such as Verilog, VHDL, etc.), and the algorithm implementation model is more mature than the processor AI algorithm implementation, Unstable modules. In the method, the algorithm implementation completed by the processor can be regarded as a correct algorithm implementation prototype (comparison prototype), while the algorithm implementation completed by the FPGA is the algorithm implementation in the design process to be verified and improved, so-called auxiliary verification is a verification method provided for assisting the FPGA to implement AI algorithm design. The storage module 4 is used for storing data to be processed by the AI algorithm, control parameters, results of the AI algorithm processing, cached data in the AI algorithm processing process and comparison data in the two AI algorithm implementation processes. The control module 3 mainly performs control functions of system control and verification processes (e.g., which layer of operation is compared with the AI algorithm, which round of operation is compared with the AI algorithm, and which layer of operation is compared with the AI algorithm, such as pooling and activation). The communication module 6 mainly completes the communication function between the verification system and the outside, specifically, for example, the data to be processed and the AI algorithm parameters are transmitted into the system, the processor algorithm implementation 1 and the FPGA AI algorithm implementation 2 are processed to obtain a result transmission system, and the comparison data, the comparison result and the processor algorithm implementation 1 and the FPGA AI algorithm implementation 2 are processed to obtain a process data transmission system. The display module 7 can be used to display control information, comparison results, analysis results, process data, AI algorithm to-be-processed data, AI algorithm processing results, and the like. The comparison and analysis module 5 has the functions of completing the comparison of operation results, the comparison and analysis of process data and the like so as to determine whether errors exist in the design process realized by the FPGA AI algorithm and further perform positioning analysis on the error places.

The workflow of the processor technology assisted validation functional block diagram corresponding to fig. 2 is introduced below:

referring to fig. 3, the communication module 7 transmits the parameters related to the realization of the data to be processed and the AI algorithm to the processor AI algorithm realization module 1 through the path 1 as the data to be processed of the verification comparison prototype, and transmits the parameters related to the realization of the data to be processed and the AI algorithm to the FPGA AI algorithm realization module 2 through the path 2 as the data to be processed of the design to be verified. The two sets of data are identical. The processor AI algorithm realizing module 1 caches the data to be processed of the prototype, the process data, the processing result data and the parameters required for realizing the AI algorithm in the storage module 3 through the passages 3 and 5 in the process of processing the data, the FPGA AI algorithm realizing module 2 caches the data to be processed of the to-be-verified design, the process data, the processing result data and the parameters required for realizing the AI algorithm in the storage module 4 through the passages 4 and 6 in the process of processing the data, the processor AI algorithm realizing module 1 transmits the comparison prototype operation result and the process data to the comparison and analysis module 6 through the passage 7 under the control of the control module 5, the FPGA AI algorithm realizing module 2 transmits the operation result and the process data of the to-be-verified design to the comparison and analysis module 6 through the passage 8 under the control of the control module 5, after the process data and the operation result of the prototype are compared with the process data and the operation result of the design to be verified respectively and completely reach the comparison analysis module 6 under the control of the control module 5, the comparison analysis module 6 respectively compares the process data and the operation result of the prototype and the design to be verified under the control of the control module 5 to compare, analyze and verify whether the operation result of the FPGA in the process of realizing the algorithm and after the algorithm is finished is correct or not so as to verify the correctness of the AI algorithm realized by the FPGA in the design process, and if errors are found, the errors are conveniently positioned. The operation result, the operation process data and the comparison process data of the processor AI algorithm realization module 1 and the FPGA AI algorithm realization module 2 can be transmitted to the display module 8 through the passage 10 to be displayed, so that the further positioning and analysis of the error in the FPGA AI algorithm realization design process can be conveniently carried out. Meanwhile, the operation result and the operation process data of the processor AI algorithm realization module 1 and the FPGA AI algorithm realization module 2 can also be transmitted to the communication module 7 through the channel 9 and transmitted to the outside of the system to further position and analyze the error in the FPGA AI algorithm realization design process. The control module 5 may set a specific verification mode through the

paths

7, 11, 12 (e.g., a specific verification FPGA and processor may be set to run to a certain layer of the AI algorithm, a number of rounds of operation of the verification FPGA and processor to the AI algorithm may be set, etc.).

The innovation of the invention is that: the processor technology is introduced to assist in verifying the design in the FPGA AI algorithm implementation process, and design errors of the FPGA in the AI algorithm design implementation process can be rapidly and accurately positioned.

The following describes a specific application scenario of the present invention:

referring to fig. 4, the AI algorithm implementation module 1, the control module 2, the comparison and analysis module 3 and other functions are implemented by the ARM processor, and the ARM processor implements data communication between the PCIE function and the computer through the PCIE interface, the PCIE function implemented by the ARM processor is equivalent to the communication module 7 in fig. 3, the computer transmits data to be processed and parameters to the corresponding module in the system through the PCEI interface, and the computer receives result data, process data and compared process data processed by the corresponding module in the system through the PCEI interface for further deep comparison and analysis. The ARM processor also realizes the HDMI 5 function and is connected with the display 15 through the HDMIPHY 7 and the HDMI data line 3, wherein the combination of the HDMI 5 and the HDMI PHY 7 realized in the ARM processor and the display 15 is equivalent to the display module 8 in fig. 3. The FPGA AI algorithm is completed in the FPGA to realize 8 and MIG9 functions, wherein the MIG9 is responsible for managing the data communication between the FPGA and the DDR 411. DDR410 and DDR 11 correspond to memory module 3 and memory module 4 in fig. 3. In addition, the FPGA and the ARM processor interact through the AXI data bus 3, and since the communication module 7, the control module 5, the comparison and analysis module 6, and the display module 8 in fig. 3 are all implemented by the ARM processor, the to-be-processed data, the parameters, the processed process data, and the processed result data implemented by the FPGA AI algorithm need to interact with the ARM processor through the AXI data bus 3. In addition, a power supply module 12 and a clock module 13 are added in the system to respectively provide clock and power for the DDR410, the DDR411, the ARM processor 16, the FPGA 17 and the HDMI PHY 7.

The above-mentioned embodiments are merely preferred embodiments for fully illustrating the present invention, and the scope of the present invention is not limited thereto. The equivalent substitution or change made by the technical personnel in the technical field on the basis of the invention is all within the protection scope of the invention. The protection scope of the invention is subject to the claims.

Claims

1. A verification method for assisting an FPGA to realize an AI algorithm based on a processor technology is characterized by comprising the following steps:

and transmitting the data to be processed and the parameters related to the AI algorithm realization to a non-parallel operation processor to be used as the data to be processed of the verification comparison prototype.

2. The processor-technology-based verification method for assisting an FPGA in implementing an AI algorithm of claim 1, further comprising: and displaying the process data and the operation result of the comparison prototype through a display screen.

3. The processor-technology-based verification method for assisting an FPGA in implementing an AI algorithm of claim 1, further comprising: and displaying the process data and the operation result of the verification design through a display screen.

4. The processor-technology-based verification method for assisting an FPGA in implementing an AI algorithm of claim 1, further comprising: and displaying the comparison result of the process data and the operation result of the comparison prototype and the process data and the operation result of the verification design through a display screen.

5. The processor-technology-based verification method for assisting an FPGA in implementing an AI algorithm of claim 1, further comprising: acquiring an instruction of a user, wherein the instruction is used for setting a specific verification mode;

6. The processor-technology-based verification method for assisting an FPGA in implementing an AI algorithm of claim 1, further comprising: and transmitting the process data and the operation result of the comparison prototype to an external system.

7. The processor-technology-based verification method for assisting an FPGA in implementing an AI algorithm of claim 1, further comprising: and transmitting the process data and the operation result of the verification design to an external system.

8. The processor-technology-based verification method for assisting an FPGA in implementing an AI algorithm of claim 1, further comprising: and transmitting the comparison result of the process data and the operation result of the comparison prototype and the process data and the operation result of the verification design to an external system.

9. The processor-based technology assisted FPGA AI algorithm validation method of claim 1 wherein said non-parallel arithmetic processor is an ARM processor or a DSP processor.

10. The AI algorithm verification method of claim 1, wherein the non-parallel processor utilizes C, C + + or Python language to process data.