CN219162697U - Patrol integrated intelligent information processing device based on FPGA and AI core - Google Patents

Abstract

The utility model discloses an integrated intelligent information processing device of an aircraft patrol based on an FPGA and an AI core, which comprises an FPGA, wherein the FPGA is connected with an AI processor. The FPGA is used for preprocessing an input image; the AI processor is used for completing autonomous detection, identification and threat assessment of targets, selecting pre-tracked targets, and realizing real-time tracking of single targets by combining the AI processor with the FPGA. The heterogeneous processing platform architecture adopting the FPGA and the AI chip not only can ensure the high-performance processing capability of the system, but also can ensure the real-time performance and good expansibility of the system, and realizes perfect unification of the performance and the power consumption. By adopting the AI chip of Haisi as a processing center and combining hardware acceleration and a neural network algorithm based on deep learning, the fast autonomous recognition and robust stable tracking of the complex target of the battlefield can be realized.

Description

Patrol integrated intelligent information processing device based on FPGA and AI core

Technical Field

The utility model belongs to the technical field of aircrafts, relates to the design and simulation of image information processing hardware of an aircraft, and particularly relates to an integrated intelligent information processing device of an aircraft patrol based on an FPGA and an AI core.

Background

With the rapid development of big data, unmanned systems, machine learning, artificial intelligence and microelectronic technologies, intelligent unmanned systems represented by patrol aircrafts show higher and higher application values in various fields such as military and civil use. The environment for using the patrol device in the future has the characteristics of more information, more vigorous countermeasure, more complex system, more scattered elements and the like, and has higher requirements on the mission and the task of the patrol device. The traditional single reconnaissance or attack type patrol aircraft cannot be used under the future countermeasure condition, and the intelligent patrol aircraft with the multifunctional load is an important point of future development.

As the patrol device is provided with the multifunctional load and carries out intelligent processing on the multi-source heterogeneous data, the information processing device on the patrol device has higher requirements. The disadvantage of insufficient operand in the intelligent processing process of the multi-load image information is more and more obvious in the traditional information processing device with the DSP+FPGA architecture.

Disclosure of Invention

Aiming at the defects of the prior art, the utility model aims to provide an integrated intelligent information processing device of an aircraft patrol based on an FPGA and an AI core, which solves the technical problem of insufficient operation amount of the aircraft patrol in the intelligent processing process of multi-load image information in the prior art.

In order to solve the technical problems, the utility model adopts the following technical scheme:

an integrated intelligent information processing device of an aircraft patrol based on an FPGA and an AI core comprises an FPGA, wherein the FPGA is connected with an AI processor.

The FPGA comprises a PCIe bus interface module, an SDI input interface module, a tracking algorithm module, an input image preprocessing module, a serial port control module and an SPI interface control module, wherein the SDI input interface module and the tracking algorithm module are respectively connected with the PCIe bus interface module.

The AI processor is connected with the FPGA through a PCIe bus, an SPI and a UART respectively.

The utility model also has the following technical characteristics:

the FPGA is used for preprocessing an input image; the AI processor is used for completing autonomous detection, identification and threat assessment of targets, selecting pre-tracked targets, and realizing real-time tracking of single targets by combining the AI processor with the FPGA.

The FPGA is also connected with a DDR3 memory and a GS2971A decoding chip; the GS2971A decoding chip is used for decoding task load video image data; the task load comprises a visible light load and an infrared load.

The GS2971A decoding chip adopts an SDI interface as a video input interface to read task load video image data.

The AI processor is also connected with an LPDDR4 memory, an EMMC memory, an ADM2587 bus chip and an RTL8211 gigabit Ethernet chip; the ADM2587 bus chip and the RTL8211 gigabit Ethernet chip are used for flight control and data chain transmission of video.

The intelligent information processing device also comprises a secondary power supply, and the secondary power supply is connected with an external power supply to supply power for the integrated intelligent information processing device of the patrol device.

Compared with the prior art, the utility model has the following technical effects:

the heterogeneous processing platform architecture adopting the FPGA and the AI chip can ensure the processing capacity of the system with high performance, the real-time performance and good expansibility of the system, and perfect unification of the performance and the power consumption is realized. By adopting the AI chip of Haisi as a processing center and combining hardware acceleration and a neural network algorithm based on deep learning, the fast autonomous recognition and robust stable tracking of the complex target of the battlefield can be realized.

Based on the current state of the art described in the background art, the utility model designs an integrated intelligent information comprehensive processing device mounted on an aircraft, so as to realize real-time detection, tracking and identification of a target. According to the intelligent information processing device, the requirements of multi-load data input, processing, transmission, storage and the like are combined, a heterogeneous processing platform framework of an FPGA+AI chip is adopted, an FPGA platform is used as a real-time control and data fusion and management center, an AI chip is used as an AI intelligent processing center of an image, and the FPGA and the AI intelligent processing center are organically combined to realize the intelligent information processing device for various different load applications.

Drawings

Fig. 1 is a schematic diagram of the overall connection of an integrated intelligent information processing device of an aircraft patrol based on an FPGA+AI core.

Fig. 2 is a block diagram of a ZYNQ7000 FPGA data processing.

Fig. 3 is a minimum system diagram of an AI processor.

FIG. 4 is a schematic illustration of a PCIe connection.

Fig. 5 is a schematic diagram of SDI input interface connection.

Fig. 6 is a schematic diagram of an ethernet interface configuration.

The following examples illustrate the utility model in further detail.

Detailed Description

It should be noted that all devices, modules and circuits in the present utility model are known in the art, unless otherwise specified.

FPGA, field-Programmable Gate Array, refers to a Field programmable gate array.

AI, artificial Intelligence, refers to artificial intelligence.

AI core, i.e. AI chip.

PCIe, also known as PCIe, peripheral Component Interconnect Express, refers to peripheral component interconnect express, and is a high speed serial computer expansion bus standard.

SPI, serial Peripheral Interface, refers to a serial peripheral interface.

UART, universal Asynchronous Receiver/Transmitter, refers to a universal asynchronous receiver/Transmitter.

DDR3, i.e. 3 ^th Double Data Rate SDRAM, referred to as third generation double data rate SDRAM.

LPDDR4, i.e. 4 ^th Low Power Double Data Rate SDRAM, referred to as fourth generation low power dual data rate SDRAM.

EMMC, embedded Multi Media Card, refers to embedded multimedia cards,

SDI, serial Digital Interface, refers to a digital component serial interface.

The following specific embodiments of the present utility model are provided, and it should be noted that the present utility model is not limited to the following specific embodiments, and all equivalent changes made on the basis of the technical solutions of the present application fall within the protection scope of the present utility model.

Examples:

the embodiment provides an integrated intelligent information processing device of an aircraft patrol based on an FPGA and an AI core, which is shown in figure 1 and comprises an FPGA, wherein the FPGA is connected with an AI processor.

The FPGA is used to preprocess the input image.

The FPGA of this embodiment is designed to select ZYNQ series programmable SOC manufactured by XINLINX company, which has 2 Cortex inside ^TM The A9 core and the FPGA core are integrated together, so that perfect unification of the performance and the power consumption is realized. As a specific scheme of the embodiment, the ZYNQ7000 FPGA data processing module designed by the utility model is shown in fig. 2.

As shown in fig. 2, the FPGA includes a PCIe bus interface module, an SDI input interface module, a tracking algorithm module, an input image preprocessing module, a serial port control module, and an SPI interface control module, which are respectively connected to the PCIe bus interface module.

The AI processor is used for completing autonomous detection, identification and threat assessment of the targets, selecting pre-tracked targets, and realizing real-time tracking of single targets by combining the AI processor with the FPGA.

Considering low power consumption and volume of the patrol aircraft, the AI processor of this embodiment selects the strong edge computing chip of Hua Hei Si to promote 310, which integrates 8-core Cortex ^TM -a55 processor, dual-core AI processor, DVPP digital visual pre-processing subsystem, multi-level system-on-chip Cache (Cache) and Buffer.

The rising 310 breaks through the constraint of the traditional design in the aspects of power consumption, computing capacity and the like, so that the energy efficiency ratio is greatly improved, and the computing requirement of a detection and recognition algorithm based on deep learning under the edge condition can be met. The minimum system of the rising 310 constructed by the utility model is composed of a rising 310 chip, a DDR4 circuit, a memory circuit, a PCIe circuit, a serial communication circuit and the like, and the system is composed as shown in figure 3.

As shown in FIG. 1, the AI processor and the FPGA are respectively connected through a PCIe bus, an SPI and a UART.

In this embodiment, the PCIe circuit is preferably designed as follows: and the FPGA and the AI processor are communicated by adopting a PCIEX4 mode. The AI processor is RC equipment, the FPGA is EP equipment, the PCIe clock adopts a non-homologous clock, the ARM clock is generated by using a crystal oscillator, and the FPGA is provided by an on-board clock chip. A PCIe connection block diagram is shown in fig. 4.

As shown in FIG. 1, the FPGA is also connected with a DDR3 memory and a GS2971A decoding chip. The GS2971A decoding chip is used for decoding task load video image data; the task load includes a visible light load and an infrared load.

As a preferred scheme of the embodiment, the GS2971A decoding chip uses an SDI interface as a video input interface to read task load video image data.

In this embodiment, the video input interface is designed as follows: the utility model selects an SDI interface as a video input interface by considering that the load platform is connected with the front end and is respectively provided with an infrared light load and a visible light load, and the SDI input interface is realized by expanding an SDI decoder chip through a parallel data interface by an FPGA. The connection block diagram is shown in fig. 5.

In this embodiment, the SDI input signal is collected by the decoding chip GS 2971A. The GS2971A decodes the SDI signal into parallel signals and interfaces with the IO interface of the FPGA.

As shown in fig. 1, the AI processor is also connected with an LPDDR4 memory, an EMMC, an ADM2587 bus chip and an RTL8211 gigabit ethernet chip; ADM2587 bus chip and RTL8211 gigabit Ethernet chip are used for flight control and data link transmission of video.

In this embodiment, the DDR4 circuit is preferably designed as follows: the DDR-C interface of the AI processor supports LPDDR4. The main chip has 4 DDR-C interfaces, each DDR-C interface has 16bit address lines and 32bit data lines, and can support 4 LPDDR4 which are connected in a butt joint mode and 32bit wide. The system is externally connected with 4 LPDDR4 with 32 bits, each chip has a capacity of 512MB, and a total of 128-bit 2GB buffer circuits are formed.

As a preferred scheme of the embodiment, the ADM2587 bus chip adopts an RS422 interface as a communication interface; the RTL8211 gigabit ethernet chip employs an ethernet interface as the video output interface.

The video output interface is designed as follows: considering the requirement that video needs to be transmitted through a data chain of a patrol aircraft, the video output interface is realized by adopting a 100M Ethernet interface, and the video output adopts an H264/H265 image compression algorithm. The AI chip selected by the utility model is provided with a hardware H264/H265 compression module and a 10M/100M1000M Ethernet MAC interface, and the 10M/100M/1000M high-speed Ethernet interface can be realized only by adding a corresponding PHY (Physical, port Physical layer) chip outside, and the Ethernet communication interface is shown in figure 6.

As shown in FIG. 1, the intelligent information processing device further comprises a secondary power supply, and the secondary power supply is connected with an external power supply to supply power for the intelligent information processing device integrated with the aircraft.

The integrated intelligent information processing device of the patrol aircraft based on the FPGA and the AI core comprises the following working processes:

step 1, reading task load video image data through an SDI interface, and then sending the task load video image data into an FPGA.

And step 2, preprocessing an input image by the FPGA, and sending the preprocessed image to the AI processor through the PCIe bus.

And 3, completing autonomous detection, identification and threat degree evaluation of the targets by an AI processor, selecting a pre-tracked target, and realizing real-time tracking of a single target by combining the AI processor with an FPGA.

And step 4, the tracking result is sent to the flight controller and the cooperative controller through the serial port, and the tracking image is compressed and then sent to the data chain through the network interface.

Claims (6)

1. The integrated intelligent information processing device for the patrol aircraft based on the FPGA and the AI core is characterized by comprising an FPGA which is connected with an AI processor;

the FPGA comprises a PCIe bus interface module, an SDI input interface module, a tracking algorithm module, an input image preprocessing module, a serial port control module and an SPI interface control module, wherein the SDI input interface module and the tracking algorithm module are respectively connected with the PCIe bus interface module;

the AI processor is connected with the FPGA through a PCIe bus, an SPI and a UART respectively.

2. The integrated intelligent information processing device of the patrol aircraft based on the FPGA and the AI core as claimed in claim 1, wherein the FPGA is used for preprocessing an input image; the AI processor is used for completing autonomous detection, identification and threat assessment of targets, selecting pre-tracked targets, and realizing real-time tracking of single targets by combining the AI processor with the FPGA.

3. The integrated intelligent information processing device of the patrol aircraft based on the FPGA and the AI core as claimed in claim 1, wherein the FPGA is further connected with a DDR3 memory and a GS2971A decoding chip; the GS2971A decoding chip is used for decoding task load video image data; the task load comprises a visible light load and an infrared load.

4. The integrated intelligent information processing device of the patrol aircraft based on the FPGA+AI core as claimed in claim 3, wherein the GS2971A decoding chip adopts an SDI interface as a video input interface to read task load video image data.

5. The integrated intelligent information processing device of the patrol aircraft based on the FPGA+AI core as claimed in claim 1, wherein the AI processor is further connected with an LPDDR4 memory, an EMMC memory, an ADM2587 bus chip and an RTL8211 gigabit Ethernet chip; the ADM2587 bus chip and the RTL8211 gigabit Ethernet chip are used for flight control and data chain transmission of video.

6. The intelligent information processing device integrated with the patrol aircraft based on the FPGA and the AI core as claimed in claim 1, further comprising a secondary power supply, wherein the secondary power supply is connected with an external power supply to supply power for the intelligent information processing device integrated with the patrol aircraft.

Images