CN106204660A - A kind of Ground Target Tracking device of feature based coupling - Google Patents

Abstract

The invention provides a ground target tracking device based on feature matching of image key points, comprising: a programmable gate array FPGA and a digital signal processor DSP; Inter-frame feature matching, sending the successful inter-frame feature matching results to DSP; performing cross-correlation and precise matching according to the inter-frame geometric transformation relationship fed back by DSP; the digital signal processor DSP is used to calculate phase based on the feature matching results output by the FPGA. The transformation relationship between adjacent image frames. The present invention fully implements the complex target tracking method based on feature points on the embedded FPGA, implements the calculation of the inter-frame conversion relationship on the DSP, takes into account the complexity of the algorithm and the low power consumption requirements of the embedded board, and can process a large number of image data, compared with the existing technology, the algorithm processing speed is increased by an order of magnitude.

Description

A Ground Target Tracking Device Based on Feature Matching

technical field

The invention belongs to the field of digital image signal processing, and in particular relates to a ground target tracking device based on feature matching.

Background technique

Target tracking in computer vision has a variety of research applications. In the embedded field, DSP is often used as the core device for tracking methods. However, in embedded DSP, it is difficult to realize complex tracking in DSP due to power consumption and real-time requirements. target tracking algorithm. Image feature points are invariant to rotation, scale, and illumination, making them widely used in image registration, object tracking, and other fields. However, the feature point algorithm itself is relatively complicated, which makes its application in the embedded field have many obstacles. For example, feature extraction algorithms like SIFT and SURF need to establish a scale space, and Gaussian filter images at multiple scales need to be obtained for a single frame image, which requires high power consumption, limited computing resources, and high real-time performance in embedded scenarios. Contradictory requirements. In order to meet the tracking application requirements of embedded scenarios, proprietary hardware such as ASIC, FPGA, etc. can be used to assist in image processing and algorithm acceleration.

Wang, J., et al., An Embedded System-on-Chip Architecture for Real-time Visual Detection and Matching.IEEE Transactions on Circuits&Systems for VideoTechnology, 2014.24(3):p.525-538, proposed a single-chip Real-time visual feature matching system implemented on FPGA. The system designs the system structure of SIFT+BRIEF architecture, implements the whole algorithm on a single FPGA, and completes the registration of front and back frame images in real time, and the speed can reach 60FPS for 720p images. The main feature of the system is that the hardware verification platform only relies on a single FPGA, and can match the front and rear frames of the feature in real time on the chip, and takes up less resources. However, the limitation of this method is that the work is limited to feature extraction acceleration and inter-frame feature matching, but it does not further reasonably coordinate FPGA resources and DSP resources to complete a tracking system.

Contents of the invention

Aiming at the defects and technical requirements of the prior art, the present invention provides a ground target tracking hardware implementation method and device based on feature matching, which fully implements the complex target tracking algorithm based on feature points on the embedded FPGA, and converts the frame The calculation of the conversion relationship is the current DSP, which takes into account the complexity of the algorithm and the low power consumption requirements of the embedded board. The algorithm implementation device with FPGA as the core can process a large amount of image data in real time. Compared with the existing technology, it is an order of magnitude Improve algorithm processing speed.

A ground target tracking device based on image key point feature matching, comprising: a programmable gate array FPGA and a digital signal processor DSP;

The programmable gate array FPGA includes a feature extraction unit, a tracking point coordinate calculation unit and an accurate matching unit; the feature extraction unit is used to extract image features for each frame of an image sequence input from the outside, and complete adjacent frames according to the image features The inter-frame feature matching result, the successful inter-frame feature matching result is sent to the DSP; the tracking point coordinate calculation unit is used to calculate the tracking point coordinates in the current frame according to the inter-frame geometric transformation relationship fed back by the DSP Value; the exact matching unit is used to take the current frame tracking point position coordinates calculated by the DSP as the center, and do cross-correlation matching based on the template in its neighborhood to obtain the precise coordinates of the tracking point coordinates;

The digital signal processor DSP is used for calculating the transformation relationship between adjacent image frames according to the feature matching result output by the FPGA, and feeding it back to the FPGA.

Further, the feature extraction unit includes a feature detection module, a feature description module, a feature storage module, and a feature point registration module of front and rear frame images;

The feature detection module is used to perform multi-scale Gaussian filtering, difference, judgment of extreme points, and elimination of low-responsivity points on image data to obtain the coordinates of detected image feature points;

The feature description module is used to extract image information from the field of the image according to the coordinates of the feature points of the image to obtain a description vector of the feature points;

The feature storage module is used to cache the feature point coordinates and the feature point description information of each frame of image; the feature storage module includes two dual-port random access memories RAMA and RAMB, and adopts a ping-pong operation to cache the previous frame information and the current frame information, i.e. The feature point coordinates and description vectors of the N-1th frame image are stored in RAMA, and the feature point coordinates and description information of the Nth frame image are stored in RAMB;

The image feature point registration module of the front and rear frames is used to complete the feature matching results between adjacent frames according to the feature point coordinates of the image and the feature point description information, and transmit the feature matching results between successful frames to the DSP.

further,

The feature detection module includes a downsampling module and two groups of structurally identical feature point detection modules; the feature point detection module includes a plurality of Gaussian filter units, a plurality of difference calculation units, a plurality of window generation units and a feature point selection unit;

A plurality of Gaussian filtering units of the first group of feature point detection modules are used to perform Gaussian filtering of different scale parameters on each frame image of the image sequence generated by the analog camera device in parallel; the difference calculation unit is used to perform Gaussian filtering on two adjacent scales of Gaussian The difference operation is performed on the filtered two images to obtain a Gaussian difference image; the window generation unit is used to generate a window centered on the pixel point in the Gaussian difference image and its neighborhood as a boundary; the feature point selection unit is used to determine in the generated window Extreme points, using extreme points as candidate feature points, deleting low-contrast points or edge points from candidate feature points, and retaining candidate feature points are the final feature points;

Select an intermediate-scale Gaussian filter unit from multiple Gaussian filter units, and input the Gaussian filter image output by the Gaussian filter unit to the downsampling module. The downsampling module is used to downsample the input image, and the downsampled image output to the second group of feature point detection modules, and the second group of feature point detection modules determine feature points in the same manner as the first group of feature point detection modules.

further,

The feature description module includes a data control module and a description vector calculation module;

The data control module is used to read the coordinates of feature points, and extracts a certain amount of image pixel data based on each feature point and the offset stored in the random number cache;

The description vector calculation module is used to obtain a binary description vector by comparing gray values of two pixels of the extracted image pixel data.

further,

The front and back frame image feature point registration module includes a description vector distance calculator, a read interrupt generator and a matching point pair memory FIFO;

The description vector distance calculator uses the first state machine to read the feature points of the current frame and the previous frame image, uses the second state machine to read the feature point description vector of the current frame and the previous frame image, and uses the current frame and the previous frame image The feature point description vector of the frame image, if the distance is less than a certain threshold, it is determined that the feature points of the two frames are successfully matched;

The matching point pair memory FIFO is used to store the successful matching point pair;

The read interrupt generator is used to give an interrupt signal to the DSP and wait for the DSP to respond when the registration of the feature points of the front and rear frame images ends.

further,

The exact matching unit includes a search area cache module, a relevant matching module, and a template cache and update module;

The search area cache module is used to cache the image input from the external interface, and when a new frame comes, it will overwrite the image of the previous frame;

The correlation matching module is used to create an area to be matched centered on the coordinate value of the tracking point in the current frame, extract the template from the template cache and the update module, and perform gray-scale correlation calculations through window traversal, and the maximum value of the gray-scale correlation calculation results corresponds to The center point of the window is the best matching position, and at the same time, the window corresponding to the maximum value in the gray correlation operation result is updated to the template cache and update module;

The template update module is used to cache templates.

further,

The DSP is used to initiate an enhanced direct memory access after capturing the interrupt signal sent by the FPGA, receive the pair of feature points that are successfully matched in the FPGA, and use random sampling consistency to calculate the geometric transformation relationship between the feature point pairs The transformation matrix; send an interrupt signal to the FPGA, and feed back the transformation matrix to the FPGA after getting a response.

Generally speaking, compared with the prior art, the above technical solutions conceived by the present invention:

The present invention completes a highly parallelized ground target tracking device mainly based on feature matching through the decomposition of the feature point algorithm, and FPGA/DSP and reasonable cooperation. The method utilizes the parallel acceleration of FPGA and the efficient cooperation of FPGA and DSP to realize complex algorithm architecture. Under ground target tracking. Compared with the traditional use of DSP as the main signal processor, more complex algorithms can be realized, that is, feature point algorithms, correlation matching algorithms, and sampling consensus algorithms are organically combined, which can have better real-time effects in ground tracking. Can reach 50 frames per second.

The present invention decomposes and modularizes the feature algorithm, uses FPGA as the core processing device, and completely realizes the complex target tracking algorithm based on feature points and the cross-correlation fine matching algorithm on the embedded FPGA, taking into account the complexity of the algorithm and the embedded The low power consumption requirements of the board, and the large amount of calculation of the algorithm is realized through the parallel design of the FPGA, which can process a large amount of data in real time. Compared with the pure DSP processing algorithm, the algorithm processing speed is increased by an order of magnitude.

The present invention adopts a dynamic cache structure to link different processing components. The use of FIFO and synchronous memory effectively solves the interconnection problems caused by differences in different data widths, different data rates, and different interfaces, reduces resource consumption, and improves system resources. utilization rate.

The present invention can receive and process real-time images synchronously and achieve real-time image tracking. FPGA is used as the core of the system architecture and data processing core. The standardization of the module interface is easy to reconfigure, and it has the characteristics of real-time large data volume throughput and low power consumption. Small in size, it can be effectively used in target tracking, navigation, recognition and other fields.

Description of drawings

Figure 1 is a block diagram of the overall structure of the ground target tracking device based on image key point feature matching in the present invention

Fig. 2 is a detailed structural block diagram of the ground target tracking device based on image key point feature matching in the present invention;

Fig. 3 is a partial block diagram of the ground object tracking device SIFT feature detection based on image key point feature matching in the present invention;

Fig. 4 is the structural block diagram of the BRIEF feature extraction of the ground target tracking device based on image key point feature matching in the present invention;

Fig. 5 is a structural block diagram of feature matching between adjacent frames of the ground target tracking device based on image key point feature matching in the present invention;

6 is a schematic diagram of the communication between FPGA and DSP of the ground target tracking device based on image key point feature matching in the present invention;

Fig. 7 is a detailed structural realization diagram of the gray level cross-correlation matching of the ground target tracking device based on image key point feature matching in the present invention;

Fig. 8 is a working flow chart of the ground object tracking device based on image key point feature matching according to the present invention.

detailed description

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not constitute a conflict with each other.

As shown in FIG. 1 , the ground target tracking device based on image key point feature matching in the present invention includes: a programmable gate array FPGA and a digital signal processor DSP. The interface between FPGA and DSP adopts EMIF interface, the interface between FPGA and analog camera unit adopts cameralink interface, and the interface between FPGA and external host computer is RS422 serial interface.

The programmable gate array FPGA includes a feature extraction unit, a tracking point coordinate calculation unit, and an accurate matching unit; the feature extraction unit is used to extract image features for each frame of image sequence generated by the analog camera device, and complete phase matching according to the image features. The feature matching result between adjacent frames is sent to the DSP; the tracking point coordinate calculation unit is used to calculate the tracking point coordinates in the current frame from the tracking point coordinates of the previous frame according to the geometric transformation relationship between frames fed back by the DSP value; the precise matching unit is used to take the coordinates of the tracking point in the current frame as the center, and perform template-based cross-correlation matching in its neighborhood to obtain the precise coordinates of the tracking point.

The digital signal processor DSP includes a transformation relationship calculation unit, and the transformation relationship calculation unit is used to calculate the transformation relationship between adjacent image frames according to the feature matching result output by the FPGA.

Figure 2 shows a preferred implementation of the programmable gate array FPGA, which includes a feature extraction unit, a tracking point coordinate calculation unit and an exact matching unit.

The feature extraction unit further includes a feature detection module, a feature description module, a feature storage module, and a feature point registration module of front and rear frame images.

The present invention preferably uses the SIFT (Scale Invariant Feature Transform) feature extracted by the feature detection module.

The feature detection module performs multi-scale Gaussian filtering, difference, judgment of extreme points, and elimination of low-responsivity points on the image data to obtain the coordinate information of the detected image feature points. As shown in Figure 3, the feature detection module includes a downsampling module and two sets of feature point detection modules with the same structure; the feature point detection module includes multiple Gaussian filter units, multiple differential calculation units, multiple window generation units and a feature point The selection unit; the multiple Gaussian filtering units of the first group of feature point detection modules are used to perform Gaussian filtering of different scale parameters on each frame of the image sequence generated by the analog camera device in parallel; the difference calculation unit is used to perform two-phase The Gaussian difference image is obtained by performing difference operation on two images after Gaussian filtering in the adjacent scale; the window generation unit is used to generate a window with the pixel point in the Gaussian difference image as the center and its neighborhood as the boundary; the feature point selection unit is used to generate Determine the extreme points in the window, use the extreme points as candidate feature points, delete low-contrast points or edge points from the candidate feature points, and keep the candidate feature points as the final feature points; select the middle point from multiple Gaussian filter units The scaled Gaussian filter unit, the Gaussian filter image output by the Gaussian filter unit is input to the downsampling module, the downsampling module is used to downsample the input image, and output the downsampled image to the second group of feature point detection modules , the second group of feature point detection modules determine feature points in the same manner as the first group of feature point detection modules.

The feature point detection algorithm of the SIFT algorithm needs to perform Gaussian filtering under multiple scale parameters σ on the image and down-sample the image to form a Gaussian scale space. Here, multiple Gaussian filter templates are set inside the FPGA, and the image data is driven by the pixel clock. , enter the template pixel by pixel to perform convolution operation with the image, so multiple Gaussian convolutions can be performed at the same time, and the convolution results will also be driven by the pixel clock to come out pixel by pixel, but there is a certain clock relative to the original image data cycle delay. After all the image data is input, after a certain clock cycle delay, multiple Gaussian filtered images will be generated simultaneously for all the filtering results under the scale parameter σ, thus generating a Gaussian scale space image group. For the generated Gaussian scale space image group, the gray level difference between the images is made between two adjacent layers to generate a Gaussian difference scale space. In the Gaussian difference scale space, except for the top layer and the bottom layer images, for each pixel of each other image layer, 26 adjacent pixels can be found, that is, a 3×3×3 area around the pixel. If each of the pixels described above conforms to the gray extreme value, maximum value or minimum value in the 26 neighborhoods, then it is preliminarily judged as a feature point. The feature point of the preliminary determination will use the Hessian matrix to judge whether the point is an edge response point, and the point that is an edge response point is eliminated. It should be noted that the feature point detection algorithm selected here is the SIFT algorithm, but it is not limited to SIFT. There are also feature point detection methods such as the SURF algorithm, Harris corner point, and Fast corner point that are widely used.

The description module caches the image data, extracts image information from the image field according to the BRIEF (Binary Robust Independent Elementary Feature) algorithm according to the detected coordinate information of the image feature points, and obtains the description information of the feature points. The feature description module includes a data control module and a description vector calculation module; the data control module is used to read the coordinates of feature points, and extract a certain amount of image pixels based on each feature point and the offset stored in the random number cache data; the description vector calculation module is used to obtain a binary description vector by comparing gray values of two pixels of the extracted image pixel data.

The idea of the BRIEF algorithm is to take out an area of the image around the feature point, usually a square centered on the feature point, and store a set of coordinate pairs at the same time, generally 256 pairs are recommended. For each pair of coordinates, extract them from the image block, and compare the gray values of the two extracted pixels. If the former is larger, the comparison result is 1, otherwise it is 0. After comparing 256 pairs, a 256-bit wide binary sequence is obtained, which is the BEIRF description vector of the feature point. The implementation process in the hardware FPGA architecture is shown in Figure 4. The image data of each frame will be cached in the image cache DPRAM, the feature point detection module will be cached in the FIFO, and the random generation module will be generated after the reset. 256 pairs of random coordinate points, the random coordinate points will be stored in a DPRAM, for each feature point coordinates in the FIFO, the point pair comparison module will read the image according to the feature point coordinates and the random number pairs cached in the DPRAM Cache the corresponding gray value of the image in DPRAM, execute the brief algorithm steps, and obtain the brief description information of the current feature point, that is, a 256-bit wide binary vector. It can be pointed out that the extraction of description information here is not limited to the BRIEF algorithm, such as SIFT descriptor extraction, SURF descriptor extraction, etc. can be used.

The feature storage module caches the feature point coordinates and the feature point description information of each frame of image. The feature storage module adopts dual-port random access memory (Random Access Memory). Ping-pong operation is used here, and two dual-port RAMs are used to always cache the previous frame information and current frame information, that is, the two dual-port RAMs are RAMA and RAMB respectively. The feature point coordinates and description information of the N-1 frame image are stored in RAMA, and the feature point coordinates and description information of the Nth frame image are stored in RAMB. In the N+1th frame, the feature point coordinates and description of the image The information is stored in RAMA, and the feature point coordinates and description information of the N-1th frame image originally stored in RAMA are overwritten, and so on.

The front and back frame image feature point registration module includes a BRIEF description vector distance calculator, a read interrupt generator and a matching point pair memory FIFO. As shown in Figure 2, the image feature point registration part, its process is to first read the feature point description vector of a current frame, and then iteratively read the previous frame image feature point description vector cached in another block of RAM one by one, Calculate the feature distance between the two vectors, and if the distance is less than a certain threshold, it is considered that the two points match successfully. For example, the description vector of BRIEF is used in the embodiment, and if the Hamming distance between two description vectors is less than 30, the matching is considered successful. The distance calculation process is shown in Figure 5, using two state machines to complete, the state machine 1 completes the feature point description vector reading of the current frame and the previous frame, and the state machine 2 completes the feature point description of the previous frame image Vector read. All successful matching point pairs are buffered into the matching point pair memory FIFO, and at the end of the matching process, an interrupt signal is given by the FPGA, waiting for the DSP to respond.

The registration information is transmitted to the DSP through the EMIF interface. After the DSP receives the registration information, it can use a random sampling consensus algorithm (RANSAC, RANdom Sample Consensus) to calculate the correspondence between the registration information. transformation relationship. The calculated transformation relationship is a matrix, and the matrix type will be different when using different algorithms. The calculation results are then transmitted back to the FPGA through the EMIF interface and fed back to the precise matching unit of the FPGA.

The exact matching unit calculates the coordinate position of the tracking point of the current frame according to the matrix transformation relationship and the cached tracking coordinate point of the previous frame. The exact matching unit uses the calculated tracking point coordinates as the center point of the search area, performs template matching in a certain surrounding area, such as a 7×7 area, and finds the coordinates that best match the cached template as tracking coordinates. As shown in Figure 2, the exact matching unit includes a search area cache module, a relevant matching module, and a template cache and update module.

Search area cache module: a control module for caching images. The image data is input from the external cameralink interface, and the image is written pixel by pixel into the DPRAM to write a whole frame of image. When a new frame comes over, it will overwrite the image of the previous frame. The dual-port RAM that caches the image, the write end is the image data stream from the previous stage, and the read end is the module that does the template matching later to get the data from here.

Correlation matching: Create an area to be matched centered on the coordinate value of the tracking point in the current frame, extract the template from the template cache and update module, and perform gray-scale correlation operations through window traversal, and the window corresponding to the maximum value in the gray-scale correlation operation results The center point is the best matching position, and at the same time, the window corresponding to the maximum value in the gray correlation operation result is updated to the template cache and update module. For example, please refer to Figure 7, read the XY coordinates of the tracking point in the current frame, and then generate a 15×15 window centered on XY when the external image data is input. When caching this 15×15 area, a 7×7 matching window will be generated at the same time. As the pixels come one by one, the window will move forward pixel by pixel. The data in this window is correlated with the cached template of the previous frame (also a 7×7 template), so that the pipeline calculates backwards, and after the image data in the entire search area is finished, it is delayed by several clock cycles. The results of related operations are also calculated. When the calculation result of each relevant matching is valid, the matching degree of the relevant matching is judged, and the corresponding area center point when the current relevant matching degree is the largest is always cached until the results of all points are calculated. In this way, after the entire traversal matching process ends, the position of the best match can be obtained. When the entire calculation is completed, the cached point coordinates are used as NewXY to the template update module.

The template update module is responsible for updating the corresponding template according to the given NewXY coordinates, and then the updated template information will be used as the template for the next traversal match.

Fig. 5 shows a preferred implementation of the registration of feature points of front and rear frame images in the present invention. The description vector of the feature point is a 256-bit wide binary vector, and the 32-bit data of the coordinates and scale information of the feature point itself are combined into 288-bit wide combined description information. Through ping-pong buffering, the merged description information of the current frame and the merged description information of the previous frame are respectively cached in two DPRAMs. Figure 5 shows the internal data processing flow in the preferred example. The matching process is completed by two finite state machines (Finite State Machine FSM). The first state machine controls to read a description information of the current frame as a cycle of the state machine, and the second state machine controls to find the best matching point in each cycle. In the figure,

State machine 1 is triggered by each new feature point in the current frame by the feature description module, it iterates the feature points of the previous frame continuously, and gives a pair of feature points in each cycle. Each step of the process is as follows:

Others: As an undefined state or a state after reset. Once in this state, immediately jump into the waiting state.

Waiting: Waiting, know that there is a new feature point signal is valid, jump to read the current frame status.

Read the current frame: read a feature point from the feature point DPRAM cache of the current frame, and then jump into the state of reading the previous frame.

Read the previous frame: take out the feature points from the feature point DPRAM cache of the previous frame, and then generate a NewResult signal for state machine 2. In addition, this state is also responsible for judging whether all the feature points in the previous frame have been iterated. If the iteration is over, jump into the write state, otherwise continue to read the next feature point in the previous frame.

Write: Waiting for the state machine 2 to write the matching feature points output by it into the matching point DPRAM, and then jump into the waiting state.

State machine 2 receives the Hamming distance between two feature points, and then selects the feature point pair with the shortest distance. The point pairs that meet the requirements will be stored in the matching point DPRAM. The specific functions of each state in state machine 2 are as follows:

Others: Undefined state or the state after reset, once entering this state, jump into the waiting state immediately.

Wait: Wait until the NewResult signal is valid, then jump into the minimum state, and initialize a minimum distance register (MIN_DIST) as the threshold for judging whether it matches. Only feature point pairs whose distance after comparison is less than this threshold are considered matching points right.

Find the minimum: This state reads a distance result and compares the result with MIN_DIST. If the distance is smaller than MIN_DIST, assign the current distance to MIN_DIST and mark the current feature point pair as a successful matching feature point pair. If the distance is greater than MIN_DIST, nothing will be done. This state has been cyclically iterated until all distances are compared, and then the state machine jumps into the WRITE state.

Write: If the feature point pair found by the FIND_MIN state is a valid feature point pair signal, write the feature point pair into the DPRAM of the feature point pair. Once in this state, the next state is the waiting state.

Figure 6 is a schematic diagram of FPGA and DSP communication circuit design. After each frame of image input completes a certain clock cycle, the FPGA will cache all the matching point pairs of the current frame and the previous frame, and then the FPGA will generate an interrupt signal and pass it to the DSP. After the DSP captures the interrupt signal, it initiates an EDMA transfer according to the settings, and transfers the point pairs cached in the FPGA to the DSP. According to the transmitted point pair data, DSP uses random sampling consistency (RANdomSAmpleConsensus, RANSAC) to calculate the corresponding transformation matrix between point pairs. The obtained matrix DSP also gives the interrupt signal, FPGA captures the interrupt signal, and reads the matrix parameters.

Please refer to Figure 8, the working process of the device of the present invention is as follows: the external analog camera inputs image data in cameralink format, the image data enters the next-level SIFT feature detection module driven by the pixel clock, and the detected feature point information is passed to the BRIEF feature description The extraction module extracts the description vector of each feature point, and the obtained description vector completes the feature point matching between adjacent frames in the feature matching module. After the feature matching is completed, a group of successfully matched matching point pairs is obtained. The matching point pair is stored in FIFO, and the matching point data is taken out by DSP, and the geometric transformation relationship of adjacent frame images in matrix form is calculated by using RANSAC algorithm, and then the matrix data is obtained by FPGA, which combines the previous frame according to the matrix transformation relationship The coordinates of the tracking point are obtained to obtain the coordinate range of the tracking point in the current frame, and the gray-scale correlation precise matching module performs gray-scale correlation precise matching to locate the tracking point in the neighborhood of the coordinates. The output is displayed after the final track point is overlaid with the image.

Those skilled in the art can easily understand that the above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention, All should be included within the protection scope of the present invention.

Claims (7)

1. A ground target tracking device based on image key point feature matching, is characterized in that, comprises: programmable gate array FPGA and digital signal processor DSP; The programmable gate array FPGA includes a feature extraction unit, a tracking point coordinate calculation unit and an accurate matching unit; the feature extraction unit is used to extract image features for each frame of an image sequence input from the outside, and complete adjacent frames according to the image features The inter-frame feature matching result, the successful inter-frame feature matching result is sent to the DSP; the tracking point coordinate calculation unit is used to calculate the tracking point coordinates in the current frame according to the inter-frame geometric transformation relationship fed back by the DSP Value; the exact matching unit is used to take the current frame tracking point position coordinates calculated by the DSP as the center, and do cross-correlation matching based on the template in its neighborhood to obtain the precise coordinates of the tracking point coordinates; The digital signal processor DSP is used for calculating the transformation relationship between adjacent image frames according to the feature matching result output by the FPGA, and feeding it back to the FPGA. 2. The ground target tracking device based on image key point feature matching according to claim 1, wherein the feature extraction unit includes a feature detection module, a feature description module, a feature storage module and front and rear frame image feature point registration module; The feature detection module is used to perform multi-scale Gaussian filtering, difference, judgment of extreme points, and elimination of low-responsivity points on image data to obtain the coordinates of detected image feature points; The feature description module is used to extract image information from the field of the image according to the coordinates of the feature points of the image to obtain a description vector of the feature points; The feature storage module is used to cache the feature point coordinates and the feature point description information of each frame of image; the feature storage module includes two dual-port random access memories RAMA and RAMB, and adopts a ping-pong operation to cache the previous frame information and the current frame information, i.e. The feature point coordinates and description vectors of the N-1th frame image are stored in RAMA, and the feature point coordinates and description information of the Nth frame image are stored in RAMB; The image feature point registration module of the front and rear frames is used to complete the feature matching results between adjacent frames according to the feature point coordinates of the image and the feature point description information, and transmit the feature matching results between successful frames to the DSP. 3. the ground target tracking device based on image key point feature matching according to claim 2, is characterized in that, The feature detection module includes a downsampling module and two groups of structurally identical feature point detection modules; the feature point detection module includes a plurality of Gaussian filter units, a plurality of difference calculation units, a plurality of window generation units and a feature point selection unit; A plurality of Gaussian filtering units of the first group of feature point detection modules are used to perform Gaussian filtering of different scale parameters on each frame image of the image sequence generated by the analog camera device in parallel; the difference calculation unit is used to perform Gaussian filtering on two adjacent scales of Gaussian The difference operation is performed on the filtered two images to obtain a Gaussian difference image; the window generation unit is used to generate a window centered on the pixel point in the Gaussian difference image and its neighborhood as a boundary; the feature point selection unit is used to determine in the generated window Extreme points, using extreme points as candidate feature points, deleting low-contrast points or edge points from candidate feature points, and retaining candidate feature points are the final feature points; Select an intermediate-scale Gaussian filter unit from multiple Gaussian filter units, and input the Gaussian filter image output by the Gaussian filter unit to the downsampling module. The downsampling module is used to downsample the input image, and the downsampled image output to the second group of feature point detection modules, and the second group of feature point detection modules determine feature points in the same manner as the first group of feature point detection modules. 4. The ground target tracking device based on image key point feature matching according to claim 2, wherein the feature description module includes a data control module and a description vector calculation module; The data control module is used to read the coordinates of feature points, and extracts a certain amount of image pixel data based on each feature point and the offset stored in the random number cache; The description vector calculation module is used to obtain a binary description vector by comparing gray values of two pixels of the extracted image pixel data. 5. The ground target tracking device based on image key point feature matching according to claim 2, wherein the front and rear frame image feature point registration module includes a description vector distance calculator, a read interrupt generator and a matching point pair memory FIFO; The description vector distance calculator uses the first state machine to read the feature points of the current frame and the previous frame image, uses the second state machine to read the feature point description vector of the current frame and the previous frame image, and uses the current frame and the previous frame image The feature point description vector of the frame image, if the distance is less than a certain threshold, it is determined that the feature points of the two frames are successfully matched; The matching point pair memory FIFO is used to store the successful matching point pair; The read interrupt generator is used to give an interrupt signal to the DSP and wait for the DSP to respond when the registration of the feature points of the front and rear frame images ends. 6. The ground target tracking device based on image key point feature matching according to claim 1, wherein the precise matching unit includes a search area cache module, a correlation matching module, and a template cache and update module; The search area cache module is used to cache the image input from the external interface, and when a new frame comes, it will overwrite the image of the previous frame; The correlation matching module is used to create an area to be matched centered on the coordinate value of the tracking point in the current frame, extract the template from the template cache and the update module, and perform gray-scale correlation calculations through window traversal, and the maximum value of the gray-scale correlation calculation results corresponds to The center point of the window is the best matching position, and at the same time, the window corresponding to the maximum value in the gray correlation operation result is updated to the template cache and update module; The template update module is used to cache templates. 7. the ground target tracking device based on image key point feature matching according to claim 1, is characterized in that, described DSP is used for after capturing the interruption signal that FPGA sends, initiates an enhanced direct memory access, receives FPGA For the feature point pairs that are successfully matched, the transformation matrix between the feature point pairs that reflects the geometric transformation relationship between frames is calculated using random sampling consistency; an interrupt signal is sent to the FPGA, and the transformation matrix is fed back to the FPGA after receiving a response.