CN215581327U

CN215581327U - Multi-core image processing system based on DSP

Info

Publication number: CN215581327U
Application number: CN202121842613.1U
Authority: CN
Inventors: 涂胜; 张恩华; 李彬; 贺也洹; 李慧剑; 韩晓霞; 陈翔
Original assignee: South West Institute of Technical Physics
Current assignee: South West Institute of Technical Physics
Priority date: 2021-08-09
Filing date: 2021-08-09
Publication date: 2022-01-18
Anticipated expiration: 2031-08-09

Abstract

The utility model belongs to the field of image processing, and discloses a multi-core image processing system based on a DSP (digital signal processor), wherein an FPGA (field programmable gate array) module comprises an image preprocessing unit FPGA1 and an image management unit FPGA 2; the image preprocessing unit FPGA1 is connected with a digital video image input device through a CameraLink interface; the FPGA1 and the FPGA2 are respectively connected with the multi-core DSP module in a bidirectional way through an SRIO interface, an EMIF interface and a GPIO interface; the FPGA2 is connected with the laser controller, the beam combiner comprehensive control board and the beam combiner control board through RS422 interfaces; the multi-core DSP module comprises 2 DSPs 1 and 2 which are in parallel isomorphism and are in bidirectional connection. The DSP1, the DSP2 or the DSP1 and the DSP2 simultaneously carry out 1-core accurate measurement on the spot area image preprocessed by the FPGA module and a spot area is processed, the information flow structure can be adaptively adjusted according to the practical application environment, and the problem of unsmooth information exchange of the structure of the multi-board processor is effectively overcome.

Description

Multi-core image processing system based on DSP

Technical Field

The utility model belongs to the technical field of digital image processing and recognition, and relates to a multi-core image processing system based on a DSP.

Background

In recent years, unmanned aerial vehicle technology is continuously developed, and the flight speed and the reaction capability of the unmanned aerial vehicle are continuously improved. Unmanned aerial vehicle discovery and discernment are anti unmanned aerial vehicle's first step also follow-up key to unmanned aerial vehicle tracking, and traditional unmanned aerial vehicle identification technology adopts unmanned aerial vehicle static image to establish fixed unmanned aerial vehicle template database, discerns unmanned aerial vehicle through comparing surveillance image and unmanned aerial vehicle template database, and then effectively tracks to unmanned aerial vehicle. However, for a target unmanned aerial vehicle with a complex high-speed flying attitude change situation, the position change, the shape change and the scale change of the target unmanned aerial vehicle in a monitored image are obvious, effective identification is difficult to achieve by a traditional unmanned aerial vehicle identification scheme, the identification rate is low, and further the failure of unmanned aerial vehicle tracking is easily caused. The method is used for preventing the unmanned aerial vehicle from being counterfeited according to needs, and along with the improvement of the flight speed of the unmanned aerial vehicle, the traditional active physical attack mode, especially the attack is carried out by laser emission, and the success rate is obviously reduced because the target cannot be directly damaged or made invalid by the laser beam which can not be continuously, accurately and directionally emitted.

On the other hand, unmanned aerial vehicle threatens to take place in the complicated situation of weather condition or night period more, and the unmanned aerial vehicle discernment monitoring means based on conventional visual image has the problem that the discernment precision is low that the image acquisition quality is low to lead to, and is difficult to satisfy current anti-unmanned aerial vehicle's demand to the unmanned aerial vehicle dynamic image tracking's in the motion real-time.

CN203397395U discloses moving target detection device under mobile platform based on DSP + FPGA, including power source interface hybrid board, FPGA preliminary treatment board and the DSP image processing board that is sandwich structure, image sensor inserts FPGA preliminary treatment board, and FPGA preliminary treatment board and DSP image processing board are respectively through video interface access display, and are respectively through network interface access gateway. The method comprises the following steps: the FPGA preprocessing board acquires image data acquired by the image sensor, performs preprocessing such as corner detection on the image data and inputs the preprocessed image data into the DSP image processing board; the DSP image processing board establishes a coordinate system by taking the optical center of the camera as a center, and converts the detection of the moving target under the mobile platform into the global motion of the scene and the independent motion of the moving target under the static platform; and determining the FEE dense membership degree through FEE fast epipolar line estimation, finally completing moving target detection and sending the result to a display. However, the moving target detection device under the mobile platform based on the DSP + FPGA has the defects of unsmooth exchange of multi-board processor structure information and influences the real-time performance of image processing.

SUMMERY OF THE UTILITY MODEL

Objects of the utility model

The purpose of the utility model is: the defects of the prior art are overcome, and the DSP-based multi-core image processing system with real-time performance and high resolution is provided.

(II) technical scheme

In order to solve the technical problem, the utility model provides a multi-core image processing system based on DSP, which comprises an FPGA module, a multi-core DSP module, a CameraLink interface and a DVI output interface;

the FPGA module comprises an image preprocessing unit FPGA1 and an image management unit FPGA2 which are bidirectionally connected through an SRIO interface, and the FPGA module is respectively bidirectionally connected with the DDR3 memory and the FLASH memory;

the image preprocessing unit FPGA1 is connected with the digital video image input device through a CameraLink interface, and is used for roughly measuring the positions of multiple paths of light spots in an input digital video image transmitted by the digital video image input device and cutting the light spots into light spot area images; the image preprocessing unit FPGA1 is bidirectionally connected with the multi-core DSP module through SRIO and EMIF interfaces; the image preprocessing unit FPGA1 interfaces with a fiber optic interface for outputting raw video images.

The image management unit FPGA2 is connected with the laser controller, the beam combiner comprehensive control panel and the beam combiner control panel through RS422 interfaces and is used for receiving control instructions of the laser controller and outputting calculation results and states to the beam combiner comprehensive control panel and the beam combiner control panel, and the image management unit FPGA2 is bidirectionally connected with the multi-core DSP module through GPIO interfaces and is in communication connection with an upper computer through a CAN bus; the image management unit FPGA2 is provided with a DVI interface for outputting composite video images;

the multi-core DSP module comprises 2 DSP1 chips and a DSP2 chip which are parallel and isomorphic and are connected in a bidirectional mode through PCIE and Hyperlink interfaces, and is used for receiving light spot area image data sent by the image preprocessing unit FPGA1 through SRIO and calculating the gravity center of a light spot; the multi-core DSP module is respectively bidirectionally connected with the DDR3 memory and the FLASH memory.

Preferably, the image preprocessing unit FPGA1 is externally connected with a 2-chip DDR3 memory, and is used for performing ping-pong operation on image data; the external 2-piece NAND FLASH is mainly used for storing CameraLink interface input picture data; and the external connection of a FLASH and 8-path IO test port.

Preferably, the image management unit FPGA2 is externally connected with 1 DDR3, and is mainly used for caching image data and outputting image display; an external EEPROM (electrically erasable programmable read-only memory); an external NVSRAM (nonvolatile SRAM) which is mainly used for storing a control configuration file of the camera; the external 1 FLASH is mainly used for storing the configuration program of the FPGA.

Preferably, the DSP1 chip and the DSP2 chip are externally connected with 2-4 pieces of DDR3 and used as buffer areas of image data; the DSP1 chip and the DSP2 chip are both connected with an external FLASH through an EMIF interface.

Preferably, the multi-core image processing system further comprises a power supply module and a clock module.

Preferably, a plurality of LED indicator lamps, a reset button and a JTAG interface are arranged on a front panel of the multi-core image processing system, heat dissipation holes are formed in the positions, corresponding to the FPGA module and the multi-core DSP module, of a back panel of the multi-core image processing system, the heat dissipation holes are cellular, and breathable waterproof films cover the heat dissipation holes, so that the waterproof and heat dissipation performance of the multi-core image processing system is improved.

Preferably, the LED indicator lamps comprise a Cameralink video input indicator lamp, a power supply voltage indicator lamp and an image processing working state indicator lamp.

Preferably, the number of the RS422 interfaces is 6, wherein 1 path of the beam combiner comprehensive control board, 1 path of the beam combiner control board, 1 path of the debugging port read-write EEPROM, 1 path of the debugging port is responsible for externally connecting a PC, parameter control communication can be performed on the camera, a result of the center of gravity of the light spot measured by the DSP is output, 1 path of the debugging port is responsible for externally connecting an energy meter, 1 path of the backup is performed, and the RS422 interfaces adopt an isolation type RS422 interface chip.

Preferably, the FPGS chip of the FPGA module adopts XC7K325T of XILINX, and the DSP chip of the multi-core DSP module adopts TMS320C6678 of TI.

Preferably, the image preprocessing unit FPGA1 cuts the input image into four spot area images.

The working principle of the multi-core image processing system based on the DSP comprises the following steps:

the image preprocessing unit FPGA1 receives data of an input digital video image through a CameraLink interface, sequentially performs image data conversion, image preprocessing operation, image cutting, light spot rough measurement, cutting into light spot area images and other processing on the image data, respectively stores 4 pieces of cut light spot area image data into FIFOs 1-4, and notifies a multi-core DSP module to read after single-frame data storage is completed, wherein the size (the maximum data volume can reach 300KB) and the position of a cut light spot area can be modified in the whole program flow;

each core of the DSP1 chip and the DSP2 chip of the multi-core DSP module correspondingly reads image data of a light spot area sent by the FPGA1, then, by operating a light spot measurement algorithm, the accurate measurement of the image of the light spot area is realized, and the position, the size, the gray level threshold of the detected image, the number threshold of the detected effective pixels, the integral time (the duty ratio of the waveform of an external trigger signal) of a camera, a power lookup table and other parameters of the light spot area are determined; the image management unit FPGA2 can frame and extract digital video images input by the Cameralink interface, superimpose characters and a Portal cross line in the DVI image to form composite video image data according to the light spot gravity center result sent by the multi-core DSP module, and output the composite video image to a display terminal (such as a computer) through the DVI interface.

The DDR3 connected with the image preprocessing unit FPGA1 is mainly used for performing ping-pong operation on image data; NAND FLASH connected to the image preprocessing unit FPGA1, for storing CameraLink input picture data; DDR3 connected to the DSP1 chip and the DSP2 chip serves as a buffer for image data. The DDR3 connected with the image management unit FPGA2 is mainly used for caching image data and outputting image display, and the data in the DDR3 is forwarded out through calculation into a standard format when needed, and can be used in cooperation with DVI interface video output; the NVSRAM connected with the image management unit FPGA2 is mainly used for storing a control configuration file of the camera; the FLASH connected with the image management unit FPGA2 is mainly used for storing a configuration program of the FPGA; and the CAN bus connected with the image management unit FPGA2 controls the operation of the multi-core image processing system through CAN communication, and sends the data of the multi-core image processing system to the upper computer for interpretation through the CAN.

(III) advantageous effects

The multi-core image processing system based on DSP provided by the technical scheme has the following beneficial effects:

the FPGA module completes the bridging function of information in the multi-core image processing system, format sorting of input digital image and video, preprocessing of the image and video (image cutting and light spot rough measurement), communication interface management and self-checking reset management; DSP1 chip in the multicore DSP module, DSP2 chip or DSP1 chip and DSP2 chip carry out 1 nuclear to the regional image of facula after FPGA module preliminary treatment simultaneously and handle the accurate measurement of a facula region, confirm the size in facula region, position isoparametric, the accuracy of the discernment of image facula has been showing and has been improved, and then realize carrying out the real time to the image of unmanned aerial vehicle in the motion of gathering, the purpose of accurate output, satisfy the demand to the real-time tracking of unmanned aerial vehicle motion trail and attack, laser beam emission for laser weapon provides accurate location, and then improve its attack success rate.

The DSP-based multi-core image processing system adopts a single-board centralized information processing structure and a large-scale programmable logic device, has flexible and dynamically reconfigurable structure, namely, the size and position measurement algorithm of light spots running on a multi-core DSP module can conveniently acquire information in a front-end FPGA module according to needs, and can adaptively adjust an information flow structure according to actual application environment so as to achieve the optimal light spot image segmentation and measurement performance, thereby effectively overcoming the defect of unsmooth information exchange of the structure of a multi-board processor.

Drawings

Fig. 1 is a schematic diagram of a hardware structure of a DSP-based multi-core image processing system according to an embodiment of the present invention.

Detailed Description

In order to make the objects, contents and advantages of the present invention clearer, the following detailed description of the embodiments of the present invention will be made in conjunction with the accompanying drawings and examples.

Referring to fig. 1, the DSP-based multi-core image processing system of the present invention uses an XC7K325T-2FFG900I FPGS chip of a kintex7 series from XILINX corporation as a co-processing chip, which is mainly used for interface processing of a front-end high-definition camera and a back-end storage unit; a multi-core DSP module is formed by adopting 2 high-performance DSP processors TMS320C6678 of TI company, is used as a data processing core and is mainly used for processing image video data input by FPGA; the system comprises an FPGA module, a multi-core DSP module, a CameraLink interface and a DVI output interface;

the FPGA module comprises an image preprocessing unit FPGA1 and an image management unit FPGA2 which are bidirectionally connected through SRIO, and the FPGA module is respectively bidirectionally connected with the DDR3 memory and the FLASH memory;

the multi-core DSP module comprises 2 parallel and isomorphic DSP1 chips and DSP2 chips, wherein the DSP1 chips and the DSP2 chips are used for receiving light spot area image data sent by an FPGA through SRIO and calculating the center of gravity of a light spot and are connected in a bidirectional mode through PCIE and Hyperlink interfaces, and the multi-core DSP module is respectively connected with a DDR3 memory and a FLASH memory in a bidirectional mode.

The image preprocessing unit FPGA1 is externally connected with a 2-piece DDR3 memory and is used for performing ping-pong operation on image data; the external 2-piece NAND FLASH is mainly used for storing CameraLink interface input picture data; and the external connection of a FLASH and 8-path IO test port.

The image management unit FPGA2 is externally connected with 1 DDR3 and is mainly used for caching image data and outputting image display; an external EEPROM (electrically erasable programmable read-only memory); an external NVSRAM (nonvolatile SRAM) which is mainly used for storing a control configuration file of the camera; the external 1 FLASH is mainly used for storing the configuration program of the FPGA.

The DSP1 chip and the DSP2 chip are both externally connected with 4 DDR3 chips and used as buffer areas of image data; the DSP1 chip and the DSP2 chip are both connected with an external FLASH through an EMIF interface.

The multi-core image processing system further comprises a power supply module and a clock module.

The front panel of the multi-core image processing system is provided with a plurality of LED indicator lamps, a reset button and a JTAG interface, the back panel of the multi-core image processing system is provided with heat dissipation holes corresponding to the positions of the FPGA module and the multi-core DSP module, and the heat dissipation holes are covered with breathable waterproof films, so that the waterproof heat dissipation performance of the multi-core image processing system is improved.

The LED indicator lamps comprise a Cameralink video input indicator lamp, a power supply voltage indicator lamp and an image processing working state indicator lamp.

The number of the RS422 interfaces is 6, wherein 1 path of the beam combiner comprehensive control board, 1 path of the beam combiner control board, 1 path of the debugging port read-write EEPROM, 1 path of the debugging port is in charge of being externally connected with a PC (personal computer), parameter control communication can be carried out on the camera, meanwhile, a light spot gravity center result measured by the DSP is output, 1 path of the debugging port is in charge of being externally connected with an energy meter, 1 path of backup is carried out, and the RS422 interfaces adopt an isolation type RS422 interface chip.

The image preprocessing unit FPGA1 cuts the input image into four spot area images.

The working principle of the multi-core image processing system based on the DSP is as follows:

each core of the DSP1 chip and the DSP2 chip of the multi-core DSP module correspondingly reads one image data of the spot area sent by the FPGA1, and then by running a spot measurement algorithm, it realizes the precise measurement of the image of the spot area, determines the position, size, gray level threshold of the detected image, threshold of the number of effective pixels, integration time of the camera (duty ratio of the waveform of the external trigger signal) and parameters such as the power lookup table, and the FPGA2 can frame and extract the digital video image input by the Cameralink interface, and according to the result of the gravity center of the spot sent by the multi-core DSP module, it can superimpose characters and cross lines of the gates in the DVI image to form composite video image data, and then output the composite video image to a display terminal (such as a computer) through the DVI interface.

The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.

Claims

1. A DSP-based multi-core image processing system, comprising: the system comprises an FPGA module, a multi-core DSP module, a CameraLink interface and a DVI output interface;

the FPGA module comprises an image preprocessing unit FPGA1 and an image management unit FPGA2 which are connected in a bidirectional mode through an SRIO interface; the image preprocessing unit FPGA1 is connected with a digital video image input device through a CameraLink interface, is bidirectionally connected with the multi-core DSP module through an SRIO interface and an EMIF interface, and is connected with an optical fiber interface for outputting an original video image; the image management unit FPGA2 is connected with the laser controller, the beam combiner comprehensive control board and the beam combiner control board through RS422 interfaces, is bidirectionally connected with the multi-core DSP module through GPIO interfaces, and is in communication connection with an upper computer through a CAN bus; the image management unit FPGA2 is provided with a DVI output interface for outputting composite video images;

the multi-core DSP module comprises 2 parallel isomorphic and bidirectionally connected DSP1 chips and a DSP2 chip, wherein the DSP1 chip is bidirectionally connected with the image preprocessing unit FPGA1 through a corresponding SRIO interface and an EMIF interface and is connected with the image management unit FPGA2 through a corresponding GPIO interface; the DSP2 chip is bidirectionally connected with the image preprocessing unit FPGA1 through a corresponding SRIO interface and an EMIF interface, and is connected with the image management unit FPGA2 through a corresponding GPIO interface.

2. The DSP-based multi-core image processing system according to claim 1, wherein the image preprocessing unit FPGA1 is externally connected with 2 DDR3 memories for performing ping-pong operation on image data, 2 NANDFLASH memories for storing CameraLink interface input image data, one FLASH and 8 IO test ports.

3. The DSP-based multi-core image processing system of claim 2, wherein the image management unit FPGA2 is externally connected with 1 DDR3, 1 EEPROM, 1 NVSRAM, and 1 FLASH.

4. The DSP-based multi-core image processing system according to claim 3, wherein the DSP1 chip and the DSP2 chip are each externally connected with 2 DDR 3; the DSP1 chip and the DSP2 chip are both connected with the corresponding external FLASH through an EMIF interface.

5. The DSP-based multi-core image processing system according to any of claims 1 to 4, further comprising a power module for powering the FPGA module, the multi-core DSP module, and a clock module for providing a clock signal to the FPGA module.

6. The DSP-based multi-core image processing system of claim 5, further comprising: the front panel is provided with a plurality of LED indicator lamps, a reset button and a JTAG interface; and heat dissipation holes are formed in the positions, corresponding to the FPGA module and the multi-core DSP module, of the back plate, and a breathable waterproof film covers the heat dissipation holes.

7. The DSP-based multi-core image processing system of claim 6 wherein the LED indicator lights comprise a Cameralink video input indicator light, a power supply voltage indicator light, an image processing operational status indicator light.

8. The DSP-based multi-core image processing system according to claim 7, wherein the number of the RS422 interfaces is 6, wherein 1 RS422 interface is connected with the beam combiner comprehensive control board, 1 RS422 interface is connected with the beam control board, 1 RS422 interface reads and writes the EEPROM, 1 RS422 interface is externally connected with the PC, 1 RS422 interface is externally connected with the energy meter, 1 RS422 interface is used for data backup, and the RS422 interface adopts an isolation type RS422 interface chip.

9. The DSP-based multi-core image processing system of claim 1 wherein the DSP1 chip and DSP2 chip are bi-directionally connected via PCIE, HYPERLINK interfaces.

10. The DSP-based multi-core image processing system according to claim 1, wherein the image preprocessing unit FPGA1 and the image management unit FPGA2 of the FPGA module adopt XC7K325T chips of XILINX, and the DSP1 chip and the DSP2 chip of the multi-core DSP module adopt TMS320C6678 of TI.