WO2015043363A1 - 飞行器地面运动目标红外图像识别装置 - Google Patents

飞行器地面运动目标红外图像识别装置 Download PDF

Info

Publication number
WO2015043363A1
WO2015043363A1 PCT/CN2014/085709 CN2014085709W WO2015043363A1 WO 2015043363 A1 WO2015043363 A1 WO 2015043363A1 CN 2014085709 W CN2014085709 W CN 2014085709W WO 2015043363 A1 WO2015043363 A1 WO 2015043363A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
module
asic
target
processor
Prior art date
Application number
PCT/CN2014/085709
Other languages
English (en)
French (fr)
Inventor
张天序
高士英
王岳环
钟胜
颜露新
俞鹏先
鲁斌
李�浩
Original Assignee
华中科技大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华中科技大学 filed Critical 华中科技大学
Publication of WO2015043363A1 publication Critical patent/WO2015043363A1/zh
Priority to US14/693,847 priority Critical patent/US9286651B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/60—Rotation of whole images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/94—Hardware or software architectures specially adapted for image or video understanding
    • G06V10/955—Hardware or software architectures specially adapted for image or video understanding using specific electronic processors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/20—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from infrared radiation only
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/30—Transforming light or analogous information into electric information
    • H04N5/33—Transforming infrared radiation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Definitions

  • the invention belongs to the technical field of infrared image processing and target recognition, and particularly relates to an infrared image recognition device for an aircraft ground moving target.
  • the aircraft belongs to the moving platform.
  • the movement of the platform will cause the apparent change of the background of the image.
  • the target to be detected is also moving.
  • the foreground motion of the whole image and the background changes caused by the platform motion are mixed together. Separation of motion enables accurate detection of moving targets and tracking tasks. Therefore, compared with the static platform moving target detection and tracking algorithm, the moving platform moving target detection and tracking algorithm is much more complicated.
  • the SIFT (Scale-invariant feature transform) algorithm proposed by DGLowe is a classical and effective image registration algorithm that can be used to distinguish the motion of foreground and background. See the literature: David G. Lowe, Distinctive image features from scale invariant keypoints .International Journal of Computer Vision, 2004.
  • the algorithm has a large amount of computation, and it is difficult to implement SIFT algorithm on a single DSP (digital signal processor) processor for real-time registration.
  • the detection and tracking of ground targets has problems such as complex background and vulnerability to interference (such as occlusion).
  • the automatic target recognition of the infrared image of the aircraft is usually the process of the imaging platform from the far and near target.
  • the target is more represented as the point source target, and the amount of information is small.
  • the target is mostly a spotted target, and the image can be utilized.
  • the characteristics of multi-level and multi-scale are reflected from the target feature model and the target recognition algorithm.
  • the feature extraction map and the feature space of the expression target should be hierarchical, so as to fully exploit the target information of each stage.
  • the general recognition algorithm can not cope with such a search, detection and recognition process, and requires the development of far, medium and close range imaging.
  • the polymorphic recognition process as shown in Figure 1, enables the processing system to correctly detect, track, and identify targets, so that the system is more burdensome.
  • the aircraft acquires a scene at a farther height or distance in order to obtain a wider field of view at the beginning of the target recognition.
  • the target has no shape information and appears as a weak point source target.
  • Algorithms such as matched filtering and multi-stage filtering can be used to suppress background and noise interference in two-dimensional space or time-space three-dimensional space, and to highlight the target, so as to achieve target capture.
  • the aircraft After capturing the target, the aircraft will gradually approach the target to be identified and enter the tracking phase. In the tracking phase, in order to reduce the amount of system calculation, the gate can be set reasonably. At this point, the target has certain shape information and appears as a patchy target.
  • the SIFT operator In order to utilize the motion that distinguishes the background from the foreground, the SIFT operator is used to extract features to achieve image registration. Then, a multi-stage filter is used to highlight the target information to suppress background clutter.
  • Infrared image processing systems for aircraft generally have constraints on volume, weight and power consumption, and the complexity of processing algorithms is high. Therefore, it is necessary to design a processor with high computational power and high flexibility parallel structure to ensure real-time calculation. . This also means that the processor has the following requirements:
  • the target recognition of the aircraft is mostly to guide the aircraft to detect and track the target, while the aircraft generally has a higher speed of movement, so only real-time target recognition can ensure that the platform can track and locate the target more accurately during the movement.
  • the traditional infrared image processing system mostly adopts "DSP+FPGA (Field Programmable Gate Array)" or “multi-DSP+FPGA” structure.
  • This homogeneous structure processing system has the disadvantages of large power consumption and low efficiency, and is also processed by DSP.
  • the versatility of the device makes it suitable for image processing and target detection There are bottlenecks in method optimization.
  • the present invention provides an infrared image recognition device for an aircraft ground moving target, which aims to solve the problems in the prior art in terms of power consumption and real-time performance when identifying a ground moving target under the condition of moving platform imaging.
  • the present invention adopts the following technical solutions:
  • An infrared image recognition device for an aircraft ground moving target comprising an infrared non-uniformity correcting SoC chip, an image rotating ASIC chip, a multi-stage filtering ASIC chip, a connected domain marking and contour tracking ASIC chip, a main DSP0 processor, a slave DSP1 processor, The main FPGA0 processor and the slave FPGA1 processor, where
  • the main DSP0 processor is configured to control the entire target detection and recognition algorithm flow, complete target detection and feature recognition, and communicate with an external interface of the device, receive imaging parameter information of the aircraft, and simultaneously output detection, tracking and recognition result information;
  • the slave DSP1 processor is configured to perform a SIFT feature extraction and image registration function together with the slave FPGA1 processor, wherein the slave DSP1 processor performs key point description and image registration calculation in the image registration step, and Transmitting a keypoint description vector (ie, the obtained SIFT feature) to the main DSP0 as a target feature for target recognition processing;
  • the main FPGA0 processor is configured to form a data transmission channel of each ASIC/SoC chip, a main DSP0 processor, and a slave FPGA1 processor, and complete image preprocessing including perspective transformation and gate setting to assist the main DSP0 processor to complete Control of each ASIC/SoC;
  • the slave FPGA1 processor is configured to perform SIFT feature extraction and image registration functions together with the slave DSP1 processor, and complete scale space extreme value detection, key point location and direction determination in the image registration step from the FPGA1;
  • the infrared non-uniformity correction SoC chip comprises an embedded microprocessor CPU and a correction ASIC core, wherein the embedded microprocessor CPU completes the calibration process of the calibration process and the gain correction parameter, and corrects the ASIC core to complete the real-time correction;
  • the image rotation ASIC chip is used to decompose a two-dimensional rotation transformation into a cubic one-dimensional translation operation, and combines a cubic convolution interpolation (ie, a bicubic interpolation) algorithm to realize an image rotation operation;
  • the multi-stage filtering ASIC chip is configured to construct a band pass filter to suppress background and noise according to analysis of a weak target, a background, and a noise spectrum, wherein the multi-stage filtering algorithm is utilized for the case where multiple sizes and targets coexist. Cascading of the same filtering module implements adjustment of filter bandwidth to extract targets of different sizes;
  • the connected domain labeling and contour tracking ASIC chip is configured to assign a consistent and unique label to the connected pixels having the same gray value in the input multi-value segmentation image according to the eight-neighbor rule; output the marked image, and the label is connected.
  • the fields are assigned from the left to the right in the image, from top to bottom, with natural numbers.
  • the beneficial technical effects of the present invention are: using image processing and target recognition dedicated ASIC/SoC chip, general-purpose DSP processor and FPGA processor to complete different levels of image processing and target recognition algorithms, improve system parallelism, and realize the aircraft in real time. Infrared image recognition algorithm for ground moving targets. At the same time, the low power consumption of the image processing and target recognition dedicated ASIC/SoC chip enables the power consumption of the processing system to meet system power consumption and thermal design requirements.
  • Figure 1 depicts the general flow of automatic recognition processing of aircraft ground moving targets
  • FIG. 2 is a block diagram showing the function realization of an infrared image recognition device for an aircraft ground moving target
  • Figure 3 depicts a hardware implementation structure of an infrared image recognition device for an aircraft ground moving target
  • FIG. 4 depicts the operational flow of the main DSP0 processing
  • FIG. 5 depicts the operational flow of processing from DSP1;
  • Figure 6 depicts an operational flow of an infrared image non-uniformity correcting SoC chip
  • Figure 7 depicts the operational flow of an image rotation ASIC chip
  • Figure 8 depicts a block diagram of an implementation of the SIFT feature extraction and image registration module
  • Figure 9 depicts the operational flow of a multi-stage filter ASIC chip
  • Figure 10 depicts the operational flow of the Connected Domain Tag ASIC chip.
  • the infrared image recognition device of the aircraft ground moving target can be divided into an infrared image non-uniformity correction module, an image rotation module, an image registration module, a multi-stage filtering module, a connected domain marking module, and a target.
  • Detection and feature recognition module, flow control module, and interconnection module implemented by FPGA.
  • the infrared image non-uniformity correction module receives the image information of the infrared imaging focal plane input, and adopts an adaptive correction algorithm based on motion detection to realize real-time correction of the infrared imaging focal plane non-uniformity problem, and an adaptive correction algorithm based on motion detection guidance. It includes four steps: correction preprocessing, real-time correction, iterative step size adjustment and gain correction coefficient update.
  • correction pre-processing In order to detect invalid pixels, the position of invalid pixels must be correctly determined. For fixed invalid pixels, the specific location can be determined by testing in the laboratory; During the imaging process, due to the charge readout and channel obstacles, the relevant unit signal is attenuated or enhanced to appear as invalid pixels.
  • the invalid pixels of this type have randomness and drift, and cannot be pre-detected in the laboratory to determine their position, because it is It changes with time, so it is necessary to introduce a scene-based dynamic detection technology.
  • the scene adaptive calibration algorithm is a dynamic monitoring method that dynamically updates the background frame and the bad element template for real-time correction.
  • Iterative step size adjustment link the motion information guides the iterative process of the gain correction coefficient, and uses the motion variance of the scene as the proportional information of the iteration step of the gain correction coefficient.
  • the iterative step size is increased.
  • the motion is slow, reduce the iteration step size, thereby adaptively controlling the correction coefficient more New speed.
  • Gain correction coefficient update link update the gain correction coefficient in combination with the motion information and the steepest descent method, thereby realizing the real-time update of the gain correction coefficient.
  • the image rotation module receives the non-uniformity corrected image, and uses the three-step translation image rotation algorithm to realize the rotation transformation of the image according to the flight parameters of the aircraft.
  • the so-called three-step translation image rotation algorithm converts the two-dimensional image rotation transformation into three-dimensional one-dimensional image translation. Operation. In general, the rotated pixel cannot be located exactly at the entire pixel of the original image, and the image rotation module uses the bicubic interpolation algorithm to calculate the rotated pixel value.
  • the image registration module uses the SIFT feature to realize the image registration operation, which eliminates the background motion caused by the motion of the imager during the imaging of the moving platform, and the extracted features can be used for the target recognition algorithm.
  • Image registration mainly includes five steps: 1 scale space extremum detection: detecting potential pairs of scales and selecting invariant points of interest through Gaussian differential functions in scale space; 2 key point positioning: determining key points at points of interest Position and scale of the point; 3 direction determination: assign direction to each key point based on the local gradient direction of the image; 4 key point description: measure the local gradient of the image in the neighborhood of each key point, and finally express it with a feature vector; 5 Image registration: According to the minimum false positive probability criterion, the correspondence between the key descriptors (ie, feature vectors) in the two frames is determined, and the registration of the images is completed under the guidance of the key points.
  • the multi-stage filtering module detects the point source target under long-distance imaging conditions and detects the spot-like target under medium-range imaging conditions.
  • the multi-stage filtering algorithm exhibits characteristics according to the weak target, background and noise in the frequency domain: the background energy is mainly concentrated in the low In the frequency band, the target energy is mainly concentrated in the middle frequency band, and the noise is mainly concentrated in the high frequency band, and a band pass filter is constructed to suppress the background and noise, highlight the target, and achieve the goal of improving the signal to noise ratio.
  • the adjustment of the filter bandwidth is realized by cascading the same filtering module.
  • Connected domain labeling module uses the multi-value image connected domain labeling algorithm to perform connected domain labeling on image data obtained by multi-value segmentation under close-range imaging conditions, and calculates connected domain area, connected domain pixel row coordinates, and connected domain pixel column coordinates. And, detect the coordinates of the contour starting point of the connected domain.
  • the connected domain pixel row coordinates and column coordinates and the center of gravity of the connected domain can be used to calculate the contour of the connected domain contour starting point coordinate
  • the starting point of the tracking algorithm execution is used to speed up the execution of the contour tracking algorithm.
  • the multi-value image connected domain labeling algorithm is divided into three steps, namely: 1 image preliminary marking: assigning a temporary mark to each pixel, and recording the equivalence relation of the temporary mark in the equivalent table; 2 sorting the equivalent table : All the temporary tags with equivalence relations are equivalent to the minimum value, and then the connected regions are renumbered in natural number order. The number is used as the final mark, and the equivalence between the temporary mark and the final mark is saved in the equivalent table. Relationship; 3 image substitution: the image is replaced pixel by pixel, and the temporary mark is replaced by the final mark.
  • Target detection and feature recognition module Under long-distance imaging conditions, the module performs binary separation on the image obtained by the multi-stage filter module to obtain the target position, and utilizes the detection results of the multi-frame image before and after to perform the reliability of the target position information. Judging, thereby enhancing the credibility of the target information, and eliminating the "false alarm" caused by noise interference; under the medium distance imaging condition, the module performs binary separation on the image obtained by the multi-stage filtering module to obtain the target position and each target The size information of the region is used to constrain the target tracking process by using the target position and size information of the multi-frame images before and after, and the corresponding relationship between the targets in the multi-frame image is obtained to achieve the target tracking.
  • the module utilizes the connected domain marking module.
  • the connected domain labeling result obtained and the aspect ratio of the connected domain area calculation target and the contour of the target are compared with the aspect ratio and the contour information of the target template to be identified, and the target is determined as the target to be identified, and then the image is calculated.
  • the SIFT feature obtained in the quasi-module and the SIFT feature of the target module Similarity, SIFT features corresponding to the contour of the target before and with enhanced reliability of the recognition result, the final recognition result is output.
  • the flow control module controls the operation sequence and data exchange of each module and controls the transformation of the algorithm flow under different imaging distance conditions, so that each module can complete the target detection, tracking and recognition tasks in a coordinated and orderly manner.
  • the FPGA module provides data channels between modules to effectively solve interconnection problems caused by different data widths, different data rates, and differences between different interfaces.
  • the state machine is used to assist the control module to realize the control of each module, so that each module can complete the image processing task in an orderly manner.
  • the aircraft ground moving target infrared image recognition device adopts "ASICs/SoCs+FPGAs+DSPs" architecture, wherein the processing chip includes an infrared non-uniformity correcting SoC chip, an image rotating ASIC chip, and more Stage Filter ASIC Chip, Connected Domain Marker and Profile Tracking ASIC The chip, the main DSP0 processor, the slave DSP1 processor, the main FPGA0 processor, and the slave FPGA1 processor.
  • Each chip performs the following functions:
  • the main DSP0 processor: 1 is responsible for the whole target detection and recognition algorithm flow control, so that each processing module completes the flow control task in an orderly manner; 2 realizes the target detection and tracking and recognition tasks, and simultaneously performs fusion judgment on the multi-frame results. That is, the target detection and feature recognition are completed; 3, communication with the external interface of the device is realized, the imaging parameter information of the aircraft is received, and the detection, tracking and recognition result information is output at the same time.
  • From DSP1 processor From SI1 and FPGA1, SIFT feature extraction and image registration function are completed.
  • image registration mainly includes five steps: 1 scale space extreme value detection; 2 key point positioning ; 3 direction determination; 4 key point description; 5 image registration.
  • the key point description and image registration calculation are completed from the DSP 1, and the key point description vector (i.e., the obtained SIFT feature) is transmitted from the DSP 1 to the main DSP 0 as the target feature for the target recognition process.
  • the main FPGA0 processor 1 constitutes the ASIC/SoC chip, the main DSP0 processor and the data transmission channel from the FPGA1 processor; 2 completes simple image preprocessing, such as perspective transformation and gate setting; 3 assists the main DSP0 Complete control of each ASIC/SoC. It mainly consists of 8 modules, which are image correction control module, image receiving module, image rotation control module, perspective transformation module, image "cutting" module (ie, wave gate setting), dynamic interconnection module and EMIFA for main DSP0 ( External memory interface A) Address decoding module.
  • the image correction control module completes the configuration of the infrared non-uniformity correction SoC chip working configuration parameter, the bad element template, the background frame and the pre-processing program through the asynchronous serial port, and simultaneously analyzes the control instruction of the main DSP0 into the SoC chip by controlling the state machine.
  • the required control signal thereby controlling the SoC chip to enter the corresponding processing process
  • the image receiving module receives the infrared image data outputted by the infrared non-uniformity correction SoC chip
  • the image rotation control module controls the workflow of the image rotation ASIC, and is powered on
  • the DSP0 control signal is parsed into the reset and start signal of the ASIC chip.
  • the image rotation ASIC After the image rotation ASIC is started normally, the image data and the parameter input are controlled, and the image rotation processing is terminated to start the subsequent perspective transformation module; the perspective transformation module completes the perspective transformation.
  • Algorithm in which the calculation of trigonometric and inverse trigonometric functions is obtained by look-up table; the image "cutting" module is used in close and medium distance imaging strips According to the size of the gate provided by the main DSP0 and the target position information of the previous frame, the dynamic gate module dynamically allocates the internal FIFO interface according to the imaging distance provided by the main DSP0 to complete the input and output under different imaging distance conditions. Switching of interfaces.
  • the perspective-transformed image is output to FPGA1 for registration operation, and the image data obtained after registration is output from FPGA1 to the multi-stage filter ASIC chip, and the cut image is output under medium-range imaging conditions.
  • the registration operation is performed on FPGA1, and the image data obtained after registration is output from FPGA1 to multi-stage filtering and small target tracking ASIC chip.
  • the cut image is output to FPGA1 for registration operation, and the image data obtained after registration is output from FPGA1 to connected domain mark and contour tracking ASIC chip and main DSP0; EMIFA address decoding module assists the main DSP0 completes address allocation for data read and write and parameter configuration.
  • the image registration mainly includes five steps: 1 scale space extreme value detection; 2 key point positioning; 3 direction determination; 4 Key point description; 5 image registration.
  • the scale space extreme value detection, key point positioning and direction determination are completed from FPGA1.
  • Infrared non-uniformity correction SoC chip including an embedded microprocessor CPU and a correction ASIC core, wherein the embedded microprocessor CPU completes the calibration process of the calibration process and the gain correction parameter, and corrects the ASIC core to complete the real-time correction;
  • Image rotation ASIC chip Decomposes the two-dimensional rotation transformation into three-dimensional one-dimensional translation operation, and combines the cubic convolution interpolation (ie bicubic interpolation) algorithm to realize the image rotation operation.
  • Multi-stage filtering ASIC chip According to the analysis of weak target, background and noise spectrum, a band-pass filter is built to suppress background and noise, and the target is enhanced. In the case of coexistence of multiple sizes and targets, multi-stage filtering is based on The algorithm uses the cascade of the same filtering module to implement the adjustment of the filter bandwidth to extract targets of different sizes.
  • Connected domain marking and contour tracking ASIC chip According to the eight neighborhood rule, the connected pixels with the same gray value in the input multi-valued segmentation image are given consistent and unique marks; the outputted marked image is labeled as connected The fields are assigned from the left to the right in the image, from top to bottom, with natural numbers.
  • the main DSP0 processor 1 power-on reset, load processing program from external FLASH0; 2 configure DSP0 internal register and external interface control register, open external interrupt; 3 pairs of ASIC chips and SoC chips Perform configuration; 4 enter the main processing flow, obtain the flight parameters of the aircraft to determine the imaging distance, perform the detection, tracking, and identification procedures; 5 in the main flow, respond to various interrupts in real time, execute the interrupt processing program; 6 output detection, tracking and recognition results .
  • the infrared non-uniformity correction SoC chip 1 power-on reset, the correction SoC executes the BOOTLOADER program in the on-chip ROM, configures the chip communication interface control register, and reads the processing program from the external FLASH; 2 The controller completes the configuration of the infrared non-uniformity correction SoC chip working configuration parameters, the bad element template, the background frame and the pre-processing program through the asynchronous serial port; 3 completes the adaptive calibration or real-time correction, so that the image correction is adapted to the scene change. Invalid element change.
  • the image rotation ASIC chip 1 chip reset when power-on; 2 after power-on reset, each register returns to the default state, start ASIC to rotate an image; 3 write to the on-chip FIFO Rotate the rotation angle, number of rows, number of columns, and pixel values of the rotated image; 4FPGA0 detects rotation completion Whether the flag pin is valid, if valid, indicates that the image rotation ASIC chip processes an image end, and the result is stored in the image rotation ASIC storage DPRAM1, and FPGA0 can read the rotation result in DPRAM1.
  • the multi-stage filter ASIC chip 1 after power-on initialization, wait for DSP0 to write programming parameters through the asynchronous communication module inside FPGA0, including the length and width of the input image, and configure the external output data storage SRAM address segment; 2 after configuration is completed, enter multi-stage filtering working state; 3 receives image data, and performs multi-stage filtering processing, and sequentially sends multi-stage filtered image data to external DPRAM2 according to the set address; 4DSP0 detects whether the multi-level filtering completion flag pin is valid through FPGA0. If it is valid, it indicates that the ASIC processes an image and the controller can read the processing result in DPRAM2.
  • mark the ASIC chip 1 chip is reset after power-on; 2DSP0 configures the ASIC internal register through FPGA0, configures the row and column parameters and control parameters of the image to be marked; 3 writes the corresponding start command to the register to start The ASIC marks an image; 4 writes the pixel value of the image to be marked into the ASIC on-chip input FIFO, and if the image to be marked is M rows and N columns, then M ⁇ N data needs to be written to the input FIFO. 5 When the marking is completed, the number of connected regions after the marking is acquired and the image marking result and the connected region feature value are read from the ASIC on-chip output FIFO.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本发明公开了一种飞行器地面运动目标红外图像识别装置，所述装置包括红外图像非均匀性校正模块、图像旋转模块、图像配准模块、多级滤波模块、连通域标记模块、目标检测与特征识别模块、流程控制模块以及FPGA实现的互联模块。本发明采用图像处理和目标识别专用ASIC/SoC芯片、通用DSP处理器和FPGA处理器，完成不同层次的图像处理和目标识别算法，提高系统并行度、实时性，实时地实现了飞行器地面运动目标红外图像识别算法，同时，有效地降低了装置的功耗。

Description

飞行器地面运动目标红外图像识别装置 【技术领域】
本发明属于红外图像处理与目标识别技术领域，具体涉及一种飞行器地面运动目标红外图像识别装置。
【背景技术】
飞行器属于动平台，平台的运动会导致图像背景的表观变化，同时待检测的目标也在运动，整幅图像的前景运动和平台运动产生的背景变化混合在一起，只有很好地把这两种运动分开，才能够准确的检测出运动目标并实现跟踪任务。因此，与静平台运动目标检测与跟踪算法相比，动平台运动目标检测与跟踪算法要复杂得多。由D.G.Lowe提出的SIFT(Scale-invariant feature transform)算法是一种经典有效的图像配准算法，可以用于区分前景与背景的运动，见文献：David G.Lowe,Distinctive image features from scale invariant keypoints.International Journal of Computer Vision，2004。但是，该算法运算量大，在单一的DSP(digital signal processor)处理器上实现SIFT算法实现实时配准存在困难。另外，地面目标的检测与跟踪又存在背景复杂、易受干扰(如遮挡)等问题。
飞行器红外图像自动目标识别通常是成像平台相对目标由远及近的过程，远距离时，目标多表现为点源目标，信息量少；中等距离时，目标多表现为斑状目标，可以利用其像面大小、简单形状和图像灰度分布信息；最后在较近的距离上，可以获得非常细致的特征信息，包括丰富的形状和纹理特征，并可用于识别分类，此时目标表现为面目标。相应地，从目标特征模型以及目标识别算法上都体现出多层次多尺度的特点。因此，特征提取映射及表达目标的特征空间应该是分级的，从而充分挖掘各个阶段的目标信息，一般的识别算法不能应付这样一个搜索、探测、识别过程，要求开发远、中、近距离成像下的多态识别流程，如图1所示，使得处理系统能够正确的检测、跟踪和识别目标，这样系统负担更大。
(1)远距离成像。一般情况下，飞行器在目标识别开始阶段为了获得更广阔的视野，多在较远的高度或者距离上获取场景。这时目标没有形状信息，表现为弱小的点源目标。采用匹配滤波、多级滤波等算法可以在二维空间或者时间-空间三维空间内抑制背景和噪声干扰，突出目标，从而实现目标的捕获。
(2)中距离成像。飞行器在捕获到目标之后，会逐步靠近待识别目标，进入跟踪阶段。在跟踪阶段，为了减少系统计算量，可以合理设置波门。此时目标具有一定的形状信息，表现为斑状目标。为了利用区分背景与前景的运动，采用SIFT算子提取特征，从而实现图像配准。然后，使用多级滤波器突出目标信息，抑制背景杂波。
(3)近距离成像。随着跟踪阶段飞行器不断靠近目标，目标表现出更多的轮廓、纹理等特征信息，此时目标表现为面目标。此时可以使用连通域标记与轮廓跟踪算法实现目标的跟踪，利用SIFT算子提取特征，在完成实现图像配准的同时，实现目标纹理的匹配与识别，最终达到识别任务。
飞行器红外图像处理系统，一般都存在体积、重量和功耗等方面的约束，同时处理算法复杂性较高，因此必须要设计具有高计算能力和高灵活性并行结构的处理机来保证计算实时性。这同时也意味着对处理机有以下几个方面的要求：
(1)实时性。飞行器目标识别多是为了指引飞行器检测并跟踪目标，而飞行器一般具有较高的运动速度，因此只有做到实时的目标识别才能保证平台在运动中对目标更精准的跟踪与定位。
(2)小型化。飞行器的小型化趋势，要求处理系统实现同样或更多功能时，系统物理尺寸更小。
(3)低功耗。飞行器的小型化将引起系统散热方面的问题。只有设计低功耗的处理系统才能保证系统热设计满足要求，从而保证系统工作的可靠性。
传统的红外图像处理系统多采用“DSP+FPGA(Field Programmable Gate Array)”或者“多DSP+FPGA”结构，这种同构结构的处理系统存在功耗大、效率低等缺点，同时由于DSP处理器的通用性使得它在图像处理与目标检测识别算 法优化方面存在瓶颈。
【发明内容】
针对现有技术的缺陷，本发明提供一种飞行器地面运动目标红外图像识别装置，旨在解决现有技术在动平台成像条件下，识别地面运动目标时功耗和实时性方面存在的问题。
为实现上述目的，本发明采用以下技术方案：
一种飞行器地面运动目标红外图像识别装置，包括红外非均匀性校正SoC芯片、图像旋转ASIC芯片、多级滤波ASIC芯片、连通域标记与轮廓跟踪ASIC芯片、主DSP0处理器、从DSP1处理器、主FPGA0处理器和从FPGA1处理器，其中，
所述主DSP0处理器用于控制整个目标检测识别算法流程，完成目标检测与特征识别，以及与装置外部接口实现通信，接收飞行器的成像参数信息，同时输出检测、跟踪和识别结果信息；
所述从DSP1处理器用于与所述从FPGA1处理器共同完成SIFT特征提取与图像配准功能，其中，所述从DSP1处理器完成图像配准步骤中的关键点描述和图像配准计算,并且将关键点描述向量(即得到的SIFT特征)传输给主DSP0作为目标特征用于目标识别处理；
所述主FPGA0处理器用于构成各个ASIC/SoC芯片、主DSP0处理器和从FPGA1处理器的数据传输通道，并完成包括透视变换和波门设置的图像预处理，协助所述主DSP0处理器完成对各个ASIC/SoC的控制；
所述从FPGA1处理器用于与所述从DSP1处理器共同完成SIFT特征提取与图像配准功能，从FPGA1完成图像配准步骤中的尺度空间极值检测、关键点定位和方向确定；
所述红外非均匀性校正SoC芯片包括一个内嵌微处理器CPU和校正ASIC核，其中内嵌微处理器CPU完成定标过程和增益校正参数的更新过程，校正ASIC核完成实时校正；
所述图像旋转ASIC芯片用于将二维旋转变换分解为三次一维平移运算，同时结合立方卷积插值(即双三次插值)算法，实现图像的旋转操作；
所述多级滤波ASIC芯片用于根据对于弱小目标、背景和噪声频谱的分析，构建带通滤波器来抑制背景和噪声，其中，针对多种大小目标并存的情况，基于多级滤波算法，利用同一滤波模块的级联实现滤波器带宽的调整以提取不同大小的目标；
所述连通域标记与轮廓跟踪ASIC芯片用于按照八邻域规则，对输入的多值分割图像中具有相同灰度值的连通像素赋予一致且唯一的标记；输出标记后的图像，标号按照连通域在图像中由左到右，由上到下出现的先后顺序，以自然数进行赋值。
本发明的有益技术效果为：采用图像处理和目标识别专用ASIC/SoC芯片、通用DSP处理器和FPGA处理器，完成不同层次的图像处理和目标识别算法，提高系统并行度，实时地实现了飞行器地面运动目标红外图像识别算法。同时，图像处理和目标识别专用ASIC/SoC芯片的低功耗特性使得处理系统的功耗能够满足系统功耗和热设计要求。
【附图说明】
图1描述了飞行器地面运动目标自动识别处理的一般流程；
图2描述了飞行器地面运动目标红外图像识别装置的功能实现框图；
图3描述了飞行器地面运动目标红外图像识别装置的硬件实现结构；
图4描述了主DSP0处理的操作流程；
图5描述了从DSP1处理的操作流程；
图6描述了红外图像非均匀校正SoC芯片的操作流程；
图7描述了图像旋转ASIC芯片的操作流程；
图8描述了SIFT特征提取与图像配准模块的实现框图；
图9描述了多级滤波ASIC芯片的操作流程；
图10描述了连通域标记ASIC芯片的操作流程。
【具体实施方式】
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用于解释本发明，并不用于限定本发明。
如图2所示，在功能实现上，飞行器地面运动目标红外图像识别装置可划分为红外图像非均匀性校正模块、图像旋转模块、图像配准模块、多级滤波模块、连通域标记模块、目标检测与特征识别模块、流程控制模块以及FPGA实现的互联模块等。
红外图像非均匀性校正模块接收红外成像焦平面输入的图像信息，采用基于运动检测的自适应校正算法，实现对红外成像焦平面非均匀性问题的实时校正，基于运动检测指导的自适应校正算法包括校正预处理、实时校正、迭代步长调整和增益校正系数更新四个环节。
(1)校正预处理环节：要进行无效像元的检测，就必须正确确定无效像元的位置，对于固定的无效像元，可以采用在实验室进行检测的办法确定其具体的位置；而在成像过程中，由于电荷读出及通道障碍使相关单元信号衰减或增强从而表现为无效像元，该型无效像元具有随机性和漂移性，无法在实验室预先检测确定其位置，由于它是随时间的变化而变化的,因此必须引入基于场景的动态检测技术，场景自适应标定算法就是一种动态监测方法，可以为实时校正动态地更新背景帧和坏元模板。
(2)实时校正环节：在坏元模板的指导下，分别对有效像元和无效像元进行处理。对于无效像元，一般都采用相邻有效像元输出的空间插值来代替，然后结合背景帧以及利用上一帧图像计算得到的增益校正系数进行实时校正得到校正后的图像。
(3)迭代步长调整环节：运动信息来指导增益校正系数的迭代过程，把场景的运动方差作为增益校正系数迭代步长的正比信息，在场景运动充分时，增大迭代步长，在场景运动缓慢时，减小迭代步长，从而自适应地控制了校正系数更 新速度。
(4)增益校正系数更新环节：结合运动信息和最陡下降法更新增益校正系数，从而实现增益校正系数的实时更新。
图像旋转模块接收非均匀性校正后的图像，根据飞行器飞行参数，利用三步平移图像旋转算法实现图像的旋转变换，所谓三步平移图像旋转算法即将二维图像旋转变换转换为三次一维图像平移运算。一般情况下，旋转后的像素点不可能正好位于原始图像的整像素点上，图像旋转模块使用双三次插值算法计算旋转后的像素值。
图像配准模块利用SIFT特征实现图像配准运算，消除动平台成像时由于成像器的运动带来的背景运动，同时提取的特征可以用于目标识别算法。图像配准主要包括五个步骤：①尺度空间极值检测：在尺度空间通过高斯微分函数来检测潜在的对尺度和选择不变的兴趣点；②关键点定位：在兴趣点位置上，确定关键点的位置和尺度；③方向确定：基于图像局部梯度方向，给每个关键点分配方向；④关键点描述：在每个关键点的邻域内测量图像局部梯度，最终用一个特征向量来表达；⑤图像配准：根据最小误判概率准则，确定两帧图像中关键点描述子(即特征向量)的对应关系，在关键点对应关系的指导下，完成图像的配准。
多级滤波模块在远距离成像条件下检测点源目标，在中距离成像条件下检测斑状目标，多级滤波算法根据弱小目标、背景以及噪声在频率域呈现出来的特性：背景能量主要集中在低频段，目标能量主要集中在中频段，噪声主要集中在高频段，构建带通滤波器，抑制背景与噪声，突出目标，达到提高信噪比的目标。同时为了达到对不同大小的弱小目标的检测，利用同一滤波模块的级联实现滤波器带宽的调整。
连通域标记模块在近距离成像条件下，利用多值图像连通域标记算法对多值分割得到的图像数据进行连通域标记，并且计算连通域面积、连通域像素行坐标和、连通域像素列坐标和、检测连通域的轮廓起点坐标。其中，连通域像素行坐标和、列坐标和可以用于计算连通域的重心，连通域轮廓起点坐标指明了轮廓 跟踪算法执行的起点，用于加速轮廓跟踪算法的执行过程。其中多值图像连通域标记算法分为三个步骤，分别是：①图像初步标记：为每个像素赋予临时标记，并且将临时标记的等价关系记录在等价表中；②整理等价表：将具有等价关系的临时标记全部等价为其中的最小值，然后对连通区域以自然数顺序重新编号，该编号作为最终标记，在等价表中保存临时标记与最终标记之间的等价关系；③图像代换：对图像进行逐个像素的代换，把临时标记代换成最终标记。
目标检测与特征识别模块：在远距离成像条件下，该模块对多级滤波模块得到的图像进行二值分割获取目标位置，同时利用前后多帧图像的检测结果对目标位置信息的可信度进行判断，从而可以增强目标信息的可信度，同时消除噪声干扰引起的“虚警”；在中距离成像条件下，该模块对多级滤波模块得到的图像进行二值分割获取目标位置和各目标区域的大小信息，同时利用前后多帧图像的目标位置与大小信息约束目标跟踪过程，得到多帧图像中目标的对应关系，实现目标跟踪；在近距离成像条件下，该模块利用连通域标记模块中得到的连通域标记结果以及连通域面积计算目标的长宽比以及目标的轮廓，与待识别目标模板的长宽比和轮廓信息进行对比，初步判断目标是否为待识别目标，然后计算图像配准模块中得到的SIFT特征与目标模块的SIFT特征之间的相似度，并利用前后对应目标的轮廓与SIFT特征增强识别结果的可信度，最终输出识别结果。
流程控制模块：流程控制模块控制各个模块的运算顺序和数据的交换并在不同的成像距离条件下，控制算法流程的转换，使得各模块能够协同有序地完成目标检测、跟踪和识别任务。
互联模块：FPGA模块提供各模块之间的数据通道，有效地解决不同数据宽度、不同数据速率、不同接口之间的差异引起的互联问题。同时利用状态机协助控制模块实现对各模块的控制，使得各模块能够协同有序地完成图像处理任务。
如图3所示，在硬件实现上，飞行器地面运动目标红外图像识别装置采用“ASICs/SoCs+FPGAs+DSPs”架构，其中的处理芯片包括红外非均匀性校正SoC芯片、图像旋转ASIC芯片、多级滤波ASIC芯片、连通域标记与轮廓跟踪ASIC 芯片、主DSP0处理器、从DSP1处理器、主FPGA0处理器和从FPGA1处理器。
各个芯片完成如下功能：
(1)主DSP0处理器：①负责整个目标检测识别算法流程控制，使得各个处理模块协调有序地完成流程控制任务；②实现目标的检测与跟踪以及识别任务，同时对多帧结果进行融合判断，即完成目标检测与特征识别；③与装置外部接口实现通信，接收飞行器的成像参数信息，同时输出检测、跟踪和识别结果信息。
(2)从DSP1处理器：从DSP1与从FPGA1共同完成SIFT特征提取与图像配准功能，如图8所示，图像配准主要包括五个步骤：①尺度空间极值检测；②关键点定位；③方向确定；④关键点描述；⑤图像配准。从DSP1完成其中关键点描述和图像配准计算,并且从DSP1将关键点描述向量(即得到的SIFT特征)传输给主DSP0作为目标特征用于目标识别处理。
(3)主FPGA0处理器：①构成各个ASIC/SoC芯片、主DSP0处理器和从FPGA1处理器的数据传输通道；②完成简单的图像预处理，如透视变换和波门设置；③协助主DSP0完成对各个ASIC/SoC的控制。主要包含8个模块，分别是图像校正控制模块、图像接收模块、图像旋转控制模块、透视变换模块、图像“剪切”模块(即波门设置)、动态互联模块和用于主DSP0的EMIFA(external memory interface A)地址译码模块。其中，图像校正控制模块通过异步串口完成红外非均匀性校正SoC芯片工作配置参数、坏元模板、背景帧和预处理程序的配置，同时通过控制状态机，将主DSP0的控制指令解析成SoC芯片所需的控制信号，从而控制SoC芯片进入相应的处理过程；图像接收模块接收红外非均匀性校正SoC芯片校正后输出的红外图像数据；图像旋转控制模块控制图像旋转ASIC的工作流程，上电时，将DSP0的控制信号解析为ASIC芯片的复位与启动信号，图像旋转ASIC正常启动后，控制图像数据以及参数的输入，同时等待图像旋转处理结束启动后续的透视变换模块；透视变换模块完成透视变换算法，其中三角函数与反三角函数的计算通过查表法得到；图像“剪切”模块在近距离和中距离成像条 件下根据主DSP0的提供的波门大小及上一帧图像目标位置信息完成波门设置；动态互联模块根据主DSP0提供的成像距离，动态分配内部FIFO接口，完成不同成像距离条件下，输入输出接口的切换。在远距离成像条件下，将透视变换后图像输出到FPGA1进行配准运算，同时配准后得到的图像数据由FPGA1输出到多级滤波ASIC芯片，在中距离成像条件下，将剪切图像输出到FPGA1进行配准运算，同时配准后得到的图像数据由FPGA1输出到多级滤波与小目标跟踪ASIC芯片。在近距离成像条件下，将剪切图像输出到FPGA1进行配准运算，同时配准后得到的图像数据由FPGA1输出到连通域标记与轮廓跟踪ASIC芯片与主DSP0；EMIFA地址译码模块协助主DSP0完成地址分配，以进行数据读写和参数配置。
(4)从FPGA1处理器：从DSP1与从FPGA1共同完成SIFT特征提取与图像配准功能，图像配准主要包括五个步骤：①尺度空间极值检测；②关键点定位；③方向确定；④关键点描述；⑤图像配准。从FPGA1完成其中尺度空间极值检测、关键点定位和方向确定。
(5)红外非均匀性校正SoC芯片：包括一个内嵌微处理器CPU和校正ASIC核，其中内嵌微处理器CPU完成定标过程和增益校正参数的更新过程，校正ASIC核完成实时校正；
(6)图像旋转ASIC芯片：将二维旋转变换分解为三次一维平移运算，同时结合立方卷积插值(即双三次插值)算法，实现图像的旋转操作。
(7)多级滤波ASIC芯片：根据对于弱小目标、背景和噪声频谱的分析，构建带通滤波器来抑制背景和噪声，增强目标，其中，针对多种大小目标并存的情况，基于多级滤波算法，利用同一滤波模块的级联实现滤波器带宽的调整以提取不同大小的目标。
(8)连通域标记与轮廓跟踪ASIC芯片：按照八邻域规则，对输入的多值分割图像中具有相同灰度值的连通像素赋予一致且唯一的标记；输出标记后的图像，标号按照连通域在图像中由左到右，由上到下出现的先后顺序，以自然数进行赋值。
下面详细描述各处理芯片的操作流程。
(1)如图4所示，主DSP0处理器：①上电复位，从外部FLASH0中加载处理程序；②配置DSP0内部寄存器和外部接口控制寄存器、打开外部中断；③对各个ASIC芯片和SoC芯片进行配置；④进入主处理流程，获取飞行器飞行参数确定成像距离，执行检测、跟踪、识别程序；⑤在主流程中，实时响应各种中断，执行中断处理程序；⑥输出检测、跟踪和识别结果。
(2)从FPGA1处理器：①上电加载程序后，接收从DSP1处理器传输的波门大小以确定处理图像大小；②接收主FPGA0传输的图像数据，在尺度空间利用高斯微分函数检测潜在的对尺度和选择不变的兴趣点，在这些潜在的兴趣点位置上确定关键点的位置和尺度，并基于图像局部梯度方向，给每个关键点分配方向；③将得到的关键点位置和尺度信息以及方向信息传输到从DSP1作后续处理。
(3)如图5所示，从DSP1处理器：①上电复位后，从外部FLASH1中加载处理程序；②配置DSP1内部寄存器和外部接口控制寄存器、打开外部中断；③进入主处理流程，通过串口接收主DSP0的高度参数以及波门大小和位置参数，生成SIFT特征向量并且进行图像配准，在近距离成像情况下还完成图像多值分割；④在主流程中，实时响应各种中断，执行中断处理程序；⑤将配准后的图像依次经FPGA1和FPGA0传输到后续处理单元，在近距离成像情况下还将SIFT特征向量通过串口传输到主DSP0用于目标识别。
(4)如图6所示，红外非均匀性校正SoC芯片：①上电复位时，校正SoC执行片上ROM中的BOOTLOADER程序，配置芯片通信接口控制寄存器，并且从外部FLASH中读取处理程序；②控制器通过异步串口完成红外非均匀性校正SoC芯片工作配置参数、坏元模板、背景帧和预处理程序的配置；③完成自适应定标或者实时校正，使得图像的校正适应场景变化引起的无效元的变化。
(5)如图8所示，图像旋转ASIC芯片：①上电时芯片复位；②上电复位后，各寄存器恢复到默认状态，启动ASIC旋转一幅图像；③向片上FIFO中依次写入待旋转图像的旋转角度、行数、列数以及像素值；④FPGA0检测旋转完成 标志引脚是否有效，若有效，说明图像旋转ASIC芯片处理一幅图像结束，并已将结果存入图像旋转ASIC存储DPRAM1中，FPGA0可以读取DPRAM1中的旋转结果。
(6)如图9所示，多级滤波ASIC芯片：①上电初始化后，等待DSP0通过FPGA0内部的异步通讯模块写入编程参数，包括输入图像的长度和宽度，并配置输出数据存储的外部SRAM的地址段；②配置完成后，进入多级滤波工作状态；③接收图像数据，并对其进行多级滤波处理，根据设定的地址依次将多级滤波后的图像数据发送到外部DPRAM2；④DSP0通过FPGA0检测多级滤波完成标志引脚是否有效，若有效，说明ASIC处理一幅图像结束，控制器可以读取DPRAM2中的处理结果。
(7)如图10所示，标记ASIC芯片：①芯片上电后复位；②DSP0通过FPGA0配置ASIC内部寄存器，配置待标记图像的行列参数及控制参数；③向寄存器写入相应的启动命令来启动ASIC标记一幅图像；④向ASIC片内输入FIFO写入待标记图像的像素值，假设待标记的图像为M行N列，那么需要向输入FIFO写入M×N个数据。⑤当标记完成后，获取标记后连通区域的个数并从ASIC片内输出FIFO中读取图像标记结果和连通区域特征值。
本领域的技术人员容易理解，以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。

Claims (3)

  1. 一种飞行器地面运动目标红外图像识别装置，包括红外非均匀性校正SoC芯片、图像旋转ASIC芯片、多级滤波ASIC芯片、连通域标记与轮廓跟踪ASIC芯片、主DSP处理器、从DSP处理器、主FPGA处理器和从FPGA处理器，其中，
    所述主DSP处理器用于控制整个目标检测识别算法流程，完成目标检测与特征识别，以及与装置外部接口实现通信，接收飞行器的成像参数信息，同时输出检测、跟踪和识别结果信息；
    所述从DSP处理器用于与所述从FPGA处理器共同完成SIFT特征提取与图像配准功能，其中，所述从DSP处理器完成图像配准步骤中的关键点描述和图像配准计算,并且将关键点描述向量(即得到的SIFT特征)传输给主DSP作为目标特征用于目标识别处理；
    所述主FPGA处理器用于构成各个ASIC/SoC芯片、主DSP处理器和从FPGA处理器的数据传输通道，并完成包括透视变换和波门设置的图像预处理，协助所述主DSP0处理器完成对各个ASIC/SoC的控制；
    所述从FPGA处理器用于与所述从DSP处理器共同完成SIFT特征提取与图像配准功能，从FPGA完成图像配准步骤中的尺度空间极值检测、关键点定位和方向确定；
    所述红外非均匀性校正SoC芯片包括一个内嵌微处理器CPU和校正ASIC核，其中内嵌微处理器CPU完成定标过程和增益校正参数的更新过程，校正ASIC核完成实时校正；
    所述图像旋转ASIC芯片用于将二维旋转变换分解为三次一维平移运算，同时结合立方卷积插值(即双三次插值)算法，实现图像的旋转操作；
    所述多级滤波ASIC芯片用于根据对于弱小目标、背景和噪声频谱的分析，构建带通滤波器来抑制背景和噪声，其中，针对多种大小目标并存的情况，基于多级滤波算法，利用同一滤波模块的级联实现滤波器带宽的调整以提取不同大小的目标；
    所述连通域标记与轮廓跟踪ASIC芯片用于按照八邻域规则，对输入的多值分割图像中具有相同灰度值的连通像素赋予一致且唯一的标记；输出标记后的图像，标号按照连通域在图像中由左到右，由上到下出现的先后顺序，以自然数进行赋值。
  2. 根据权利要求1所述的飞行器地面运动目标红外图像识别装置，其中，所述主FPGA处理器包括图像校正控制模块、图像接收模块、图像旋转控制模块、透视变换模块、图像“剪切”(即波门设置)模块、动态互联模块和用于所述主DSP的EMIFA(external memory interface A)地址译码模块，
    所述图像校正控制模块通过异步串口完成红外非均匀性校正SoC芯片工作配置参数、坏元模板、背景帧和预处理程序的配置，同时通过控制状态机，将所述主DSP的控制指令解析成红外非均匀性校正SoC芯片所需的控制信号，从而控制该SoC芯片进入相应的处理过程；
    所述图像接收模块接收红外非均匀性校正SoC芯片校正后输出的红外图像数据；
    所述图像旋转控制模块控制所述图像旋转ASIC的工作流程，上电时，将所述主DSP的控制信号解析为图像旋转ASIC的复位与启动信号，图像旋转ASIC正常启动后，控制图像数据以及参数的输入，同时等待图像旋转处理结束以启动后续的透视变换模块；
    所述透视变换模块完成透视变换算法，其中三角函数与反三角函数的计算通过查表法得到；
    所述图像“剪切”模块在近距离和中距离成像条件下根据主DSP的提供的波门大小及上一帧图像目标位置信息完成波门设置；
    所述动态互联模块根据主DSP提供的成像距离，动态分配内部FIFO接口，完成不同成像距离条件下输入输出接口的切换；
    所述EMIFA地址译码模块协助主DSP完成地址分配，以进行数据读写和参数配置。
  3. 根据权利要求2所述的飞行器地面运动目标红外图像识别装置，其中，
    在远距离成像条件下，所述动态互联模块将透视变换后图像输出到所述从FPGA进行配准运算，同时配准后得到的图像数据由所述从FPGA输出到所述多级滤波ASIC芯片；
    在中距离成像条件下，所述动态互联模块将剪切图像输出到所述从FPGA进行配准运算，同时配准后得到的图像数据由所述从FPGA输出到所述多级滤波ASIC芯片；
    在近距离成像条件下，所述动态互联模块将剪切图像输出到所述从FPGA进行配准运算，同时配准后得到的图像数据由所述FPGA输出到所述连通域标记与轮廓跟踪ASIC芯片与主DSP。
PCT/CN2014/085709 2013-09-30 2014-09-02 飞行器地面运动目标红外图像识别装置 WO2015043363A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US14/693,847 US9286651B2 (en) 2013-09-30 2015-04-22 Aircraft-based infrared image recognition device for ground moving target

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201310462581.6 2013-09-30
CN201310462581.6A CN103646232B (zh) 2013-09-30 2013-09-30 飞行器地面运动目标红外图像识别装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US14/693,847 Continuation-In-Part US9286651B2 (en) 2013-09-30 2015-04-22 Aircraft-based infrared image recognition device for ground moving target

Publications (1)

Publication Number Publication Date
WO2015043363A1 true WO2015043363A1 (zh) 2015-04-02

Family

ID=50251444

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2014/085709 WO2015043363A1 (zh) 2013-09-30 2014-09-02 飞行器地面运动目标红外图像识别装置

Country Status (3)

Country Link
US (1) US9286651B2 (zh)
CN (1) CN103646232B (zh)
WO (1) WO2015043363A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109961003A (zh) * 2018-12-26 2019-07-02 安徽继远软件有限公司 一种基于fpga的嵌入式输电线路机载辅助巡视装置
CN114549275A (zh) * 2022-02-10 2022-05-27 中国科学院上海技术物理研究所 一种基于双四核dsp信号处理板的实时数字图像处理方法
CN114565537A (zh) * 2021-11-12 2022-05-31 国科天成科技股份有限公司 一种基于局部信息熵的红外成像装置

Families Citing this family (38)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103646232B (zh) * 2013-09-30 2016-08-17 华中科技大学 飞行器地面运动目标红外图像识别装置
CN104978749A (zh) * 2014-04-08 2015-10-14 南京理工大学 一种基于fpga的sift图像特征提取系统
CN105321164B (zh) * 2014-07-31 2018-04-03 南京理工大学 一种红外小目标预警系统
CN104501959B (zh) * 2014-12-30 2016-08-17 华中科技大学 一种红外图谱关联智能探测方法及装置
CN104635199B (zh) * 2015-02-12 2017-07-04 黑龙江大学 块稀疏重构方法实现的宽带信号快速超分辨测向装置
CN105590105A (zh) * 2016-01-07 2016-05-18 黎海纤 一种快速反应的横向限位和纠偏装置
CN105719242A (zh) * 2016-01-13 2016-06-29 广州市赛普电子科技有限公司 一种基于查表法的图片快速旋转方法及系统
CN106204660B (zh) * 2016-07-26 2019-06-11 华中科技大学 一种基于特征匹配的地面目标跟踪装置
CN108225316B (zh) * 2016-12-22 2023-12-29 成都天府新区光启未来技术研究院 载体姿态信息的获取方法和装置及系统
CN106686278A (zh) * 2016-12-31 2017-05-17 华中科技大学 一种面向实时目标识别的异构处理机并行协调处理方法
CN106874949B (zh) * 2017-02-10 2019-10-11 华中科技大学 基于红外图像的运动成像平台运动目标检测方法及系统
CN107610033B (zh) * 2017-07-28 2020-10-13 湖北航天技术研究院总体设计所 一种星敏感器星点目标快速识别方法
CN108198125B (zh) * 2017-12-29 2021-10-08 深圳云天励飞技术有限公司 一种图像处理方法及装置
CN110275842B (zh) * 2018-07-09 2022-10-21 西北工业大学 基于fpga的高光谱目标跟踪系统及方法
CN110766648B (zh) * 2018-07-27 2022-05-27 深圳百迈技术有限公司 一种特殊非线性滤波图像处理方法
CN109064504B (zh) * 2018-08-24 2022-07-15 深圳市商汤科技有限公司 图像处理方法、装置和计算机存储介质
US10884899B2 (en) * 2018-10-01 2021-01-05 International Business Machines Corporation Optimized trampoline design for fast software tracing
CN111200718A (zh) * 2018-10-31 2020-05-26 迈格恩特(苏州)质量技术服务有限公司 一种双目智能门锁系统
CN109587380B (zh) * 2018-12-06 2020-04-28 天津津航技术物理研究所 小型化高温工作多接口制冷型红外成像器
CN110428466B (zh) * 2019-07-16 2022-02-18 浙江大华技术股份有限公司 一种非均匀性校正的方法和设备
CN110992239B (zh) * 2019-11-14 2023-03-24 中国航空工业集团公司洛阳电光设备研究所 一种基于单片ddr3芯片的图像时域滤波与显示的方法
CN111462060A (zh) * 2020-03-24 2020-07-28 湖南大学 胎儿超声图像中标准切面图像的检测方法和装置
CN111323839B (zh) * 2020-04-07 2023-04-07 中国人民解放军国防科技大学 一种光电探测机构用嵌入式控制系统
CN111461242A (zh) * 2020-04-08 2020-07-28 北京航天新风机械设备有限责任公司 一种用于生产线的多物料快速对比匹配方法
CN111897262B (zh) * 2020-07-30 2023-08-11 电子科技大学 基于多块dsp的并行信号采集处理系统的数据处理方法
CN112419392A (zh) * 2020-11-30 2021-02-26 广州博进信息技术有限公司 基于机器视觉计算运动物体实际大小的方法、设备及介质
CN112581509B (zh) * 2020-12-25 2023-08-15 北京环境特性研究所 一种基于sopc的无人机载地面目标实时跟踪系统及跟踪方法
CN112861877B (zh) * 2021-01-20 2023-06-27 沈阳理工大学 一种基于飞行载体的红外简易图像识别方法
CN113255538B (zh) * 2021-06-01 2024-06-14 大连理工大学 基于fpga的红外弱小目标的检测跟踪装置及方法
CN114419342A (zh) * 2022-01-29 2022-04-29 中国科学院国家空间科学中心 一种基于fpga的高分辨率图像多目标多特征实时提取方法
CN114280571B (zh) * 2022-03-04 2022-07-19 北京海兰信数据科技股份有限公司 一种雨杂波信号的处理方法、装置及设备
CN115170523B (zh) * 2022-07-14 2023-05-05 哈尔滨工业大学 一种基于局部对比度的低复杂度红外弱小目标检测方法
CN114998567B (zh) * 2022-07-18 2022-11-01 中国科学院长春光学精密机械与物理研究所 一种基于多模态特征判别的红外点群目标识别方法
CN115294478B (zh) * 2022-07-28 2024-04-05 北京航空航天大学 一种应用于现代光电平台的空中无人机目标检测方法
CN116091293B (zh) * 2022-09-13 2023-10-31 北京理工大学 一种微小型智能弹载计算机架构
CN115880129B (zh) * 2022-12-12 2024-06-04 华中光电技术研究所(中国船舶集团有限公司第七一七研究所) 基于fpga的高帧率二值图像连通域自动标记方法及系统
CN117830301B (zh) * 2024-03-04 2024-05-14 青岛正大正电力环保设备有限公司 基于红外可见光融合特征的捞渣区域检测方法
CN117953591B (zh) * 2024-03-27 2024-06-14 中国人民解放军空军军医大学 一种智能肢体康复辅助方法及设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101267564A (zh) * 2008-04-16 2008-09-17 中国科学院计算技术研究所 一种多处理器视频编码芯片装置和方法
US8136982B2 (en) * 2009-03-16 2012-03-20 International Business Machines Corporation Thermal profiling to validate electronic device authenticity
CN102801912A (zh) * 2012-07-17 2012-11-28 北京伽略电子系统技术有限公司 一种红外图像处理芯片及处理方法
CN103646232A (zh) * 2013-09-30 2014-03-19 华中科技大学 飞行器地面运动目标红外图像识别装置

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7663502B2 (en) * 1992-05-05 2010-02-16 Intelligent Technologies International, Inc. Asset system control arrangement and method
CN1235037C (zh) * 2003-12-04 2006-01-04 上海交通大学 红外目标检测跟踪与识别系统
US8675995B2 (en) * 2004-07-09 2014-03-18 Terrago Technologies, Inc. Precisely locating features on geospatial imagery
US7660441B2 (en) * 2004-07-09 2010-02-09 Southern California, University System and method for fusing geospatial data
US8031227B2 (en) * 2005-03-07 2011-10-04 The Regents Of The University Of Michigan Position tracking system
JP2007299210A (ja) * 2006-04-28 2007-11-15 Sharp Corp 画像処理装置、画像形成装置、画像読取装置及び画像処理方法
EP2115665A1 (en) * 2007-02-06 2009-11-11 AAI Corporation Utilizing polarization differencing method for detect, sense and avoid systems
CN100498821C (zh) * 2007-07-06 2009-06-10 哈尔滨工程大学 运动背景的红外图像实时目标识别与跟踪器及方法
US8886206B2 (en) * 2009-05-01 2014-11-11 Digimarc Corporation Methods and systems for content processing
US10091439B2 (en) * 2009-06-03 2018-10-02 Flir Systems, Inc. Imager with array of multiple infrared imaging modules
US9501714B2 (en) * 2010-10-29 2016-11-22 Qualcomm Incorporated Systems and methods to improve feature generation in object recognition
CN102314693A (zh) * 2011-07-27 2012-01-11 中国科学院长春光学精密机械与物理研究所 双模视频目标识别跟踪系统
US8717551B2 (en) * 2011-09-30 2014-05-06 Inview Technology Corporation Adaptive search for atypical regions in incident light field and spectral classification of light in the atypical regions
CN202583884U (zh) * 2012-04-18 2012-12-05 北京刘氏尊德科技有限责任公司 无人飞行器有效载荷与自动驾驶仪综合控制计算机系统
US9196041B2 (en) * 2013-03-14 2015-11-24 Lockheed Martin Corporation System, method, and computer program product for indicating hostile fire

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101267564A (zh) * 2008-04-16 2008-09-17 中国科学院计算技术研究所 一种多处理器视频编码芯片装置和方法
US8136982B2 (en) * 2009-03-16 2012-03-20 International Business Machines Corporation Thermal profiling to validate electronic device authenticity
CN102801912A (zh) * 2012-07-17 2012-11-28 北京伽略电子系统技术有限公司 一种红外图像处理芯片及处理方法
CN103646232A (zh) * 2013-09-30 2014-03-19 华中科技大学 飞行器地面运动目标红外图像识别装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
PENG, XIAOJUN ET AL.: "Real-time Infrared Image Processing System Base on DSP+FPGA+ASIC", CONTROL & AUTOMATION, vol. 23, no. 8-2, 31 December 2007 (2007-12-31), pages 199 AND 200 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109961003A (zh) * 2018-12-26 2019-07-02 安徽继远软件有限公司 一种基于fpga的嵌入式输电线路机载辅助巡视装置
CN114565537A (zh) * 2021-11-12 2022-05-31 国科天成科技股份有限公司 一种基于局部信息熵的红外成像装置
CN114549275A (zh) * 2022-02-10 2022-05-27 中国科学院上海技术物理研究所 一种基于双四核dsp信号处理板的实时数字图像处理方法
CN114549275B (zh) * 2022-02-10 2024-03-26 中国科学院上海技术物理研究所 一种基于双四核dsp信号处理板的实时数字图像处理方法

Also Published As

Publication number Publication date
CN103646232A (zh) 2014-03-19
US9286651B2 (en) 2016-03-15
US20150227776A1 (en) 2015-08-13
CN103646232B (zh) 2016-08-17

Similar Documents

Publication Publication Date Title
WO2015043363A1 (zh) 飞行器地面运动目标红外图像识别装置
Cheng et al. FusionNet: Edge aware deep convolutional networks for semantic segmentation of remote sensing harbor images
JP5739522B2 (ja) 全体画像内で注意の焦点に関する画像データを処理するためのシステムと方法
US20190180464A1 (en) Remote determination of containers in geographical region
CN112580416A (zh) 基于深暹罗网络和贝叶斯优化的视频跟踪
US9378431B2 (en) Method of matching image features with reference features and integrated circuit therefor
CN107239740B (zh) 一种多源特征融合的sar图像自动目标识别方法
TW202004560A (zh) 目標檢測系統、自動駕駛車輛以及其目標檢測方法
US20220319011A1 (en) Heterogeneous Image Registration Method and System
US11994593B2 (en) Detection method and device based on laser radar, and computer readable storage medium
WO2018128667A1 (en) Systems and methods for lane-marker detection
US8666170B2 (en) Computer system and method of matching for images and graphs
CN107895375A (zh) 基于视觉多特征的复杂道路线提取方法
US20220262093A1 (en) Object detection method and system, and non-transitory computer-readable medium
CN105631799B (zh) 一种动平台激光红外融合检测识别系统
CN111199198B (zh) 一种图像目标定位方法、图像目标定位装置及移动机器人
CN107240130A (zh) 遥感影像配准方法、装置及系统
Han et al. Adaptive feature representation for visual tracking
CN110176041B (zh) 一种基于双目视觉算法的新型列车辅助装配方法
CN111753858A (zh) 一种点云匹配方法、装置和重定位系统
CN111062341B (zh) 视频图像区域的分类方法、装置、设备及存储介质
CN111062384A (zh) 一种基于深度学习的车窗精确定位方法
WO2021164615A1 (en) Motion blur robust image feature matching
Bodensteiner et al. Multispectral matching using conditional generative appearance modeling
CN112634360B (zh) 一种视觉信息确定方法、装置、设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14847494

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14847494

Country of ref document: EP

Kind code of ref document: A1