CN100580704C - A Real-Time Adaptive Processing Method for Imaging with Image Motion Elimination - Google Patents

Abstract

The invention discloses a real-time self-adaptive processing method of image motion elimination imaging, which is a real-time adaptive processing method of image motion elimination imaging in a high-speed moving small target imaging system. This method can automatically search, find and point to small targets in high-speed motion. Using FPGA and DSP as a real-time online processing platform, image enhancement and inter-frame correlation recognition algorithms with low algorithm complexity are used to remove the influence of false targets to obtain the target's motion. Features, to drive a high-precision two-dimensional piezoelectric tilting mirror to complete imaging adaptive motion compensation, to achieve clear imaging of small targets with high relative motion speed.

Description

A Real-Time Adaptive Processing Method for Imaging with Image Motion Elimination

technical field

The invention relates to an image acquisition technology, in particular to an adaptive real-time processing method for motion-compensated image-elimination imaging of a high-speed moving small target, which is applied to the acquisition of high-reliability information of a fast-moving small target in the distance.

Background technique

High-speed moving small targets are an important target for infrared image acquisition, and have also been applied in visible light images in recent years. The imaging of this type of target has the advantages of small target size (about less than 1m), long imaging distance (several kilometers), relatively single imaging background and slow change, extremely fast moving speed of the target (up to 1km/s), and high imaging efficiency of the target. It occupies no less than 4×4 pixels and other characteristics. Traditional imaging methods generally have low spatial resolution and are not sensitive to image motion blur, and basically only need to meet the requirements of target monitoring. In order to obtain high-reliability information of the moving target and achieve clear imaging of it, it is necessary to ensure that the target moving image does not move beyond an instantaneous field of view within the integration time, and assuming that the target imaging occupies 4×4 pixels, The target size is 0.5m, the imaging relative motion speed is 1km/s, and the integration time is 5ms, then the target will move 40 pixels within the integration time, so the traditional imaging method has serious image motion blur problem. At the same time, because the moving speed of the target is very fast, according to the moving speed of the above target, assuming that the number of pixels of the detector is 1k×1k, the time for the target to cross the field of view is only 125ms. In order to ensure that the system can capture the target, image motion compensation is performed. Imaging, the real-time processing requirements of the system are particularly important.

There are many traditional imaging methods for eliminating image motion, such as the exposure control adopted by the ordinary DC motion mode, the optical-mechanical method used by the aerial survey camera to drive the camera or detector motion compensation, the electronic motion compensation of the TDI method, and the digital image technology of Beijing Lingyun Vision Technology Co., Ltd. Co., Ltd. applied for the patented aviation full-frame transfer type area array CCD camera image motion compensation method (patent application number 200710117666.5) in 2007, such as electronic image motion compensation, etc., but currently these image motion compensation methods are mainly for the resolution is not In situations where the motion characteristics of the target are very high or the target is completely known, such as aerial survey or synchronous ballistic photography, etc., they generally only need to adjust the motion compensation direction to be consistent with the motion direction of the target image, and set the motion compensation parameters in advance. , while the image data is directly returned, there is no online real-time image processing, and it is not well adaptive to the situation of small targets moving at high speed in the distance and the target trajectory and speed are unknown. At present, the imaging system with real-time online adaptive processing mechanism is mainly used in the task of target monitoring. Most of the objects it faces are point targets or fixed targets with slow moving speed. There are situations where the target easily escapes from the field of view or the image motion in the imaging is completely blurred. To sum up, there is currently no better online real-time adaptive processing solution for clear imaging of high-speed moving small targets with unknown speeds.

Contents of the invention

The purpose of the present invention is to provide a real-time adaptive processing method for image motion elimination imaging of high-speed moving small targets, which eliminates the influence of image motion blur caused by high-speed relative motion of the target by adaptively adjusting the image motion compensation mechanism.

The method of the present invention is implemented on the high-speed moving small target imaging system as shown in FIG. 3 . The entire imaging system is installed on a high-precision two-dimensional tracking and pointing turntable 4. The device first uses the two-dimensional tracking and pointing turntable 4 to run the branch scanning target search mode, and the optical signal enters the large area array focal plane detection through the imaging optical system 1. The image acquisition device 3 realizes image acquisition, and the real-time processing system 5 processes the image acquired by the detector 3 to find the target and obtain the motion compensation amount of image-elimination imaging, and then controls the high-precision two-dimensional image-elimination piezoelectric tilting mirror 2 to perform image motion compensation, At the same time, the orientation of the two-dimensional tracking and pointing turntable 4 is controlled to keep the target in the imaging field of view, thereby realizing clear imaging of small targets moving at high speed.

The flow chart of the inventive method is as shown in Figure 1, and its main steps of real-time processing comprise:

A. Use correlated dual-scale filtering to traverse the acquired image with motion blur to achieve image filtering preprocessing, remove the slowly changing background to obtain the target enhanced image;

B. Use the algorithm of adjacent pixel accumulation to record possible target pixels, extract possible target pixels whose brightness exceeds a certain threshold of background brightness, and suppress the influence of random noise at the same time;

C. Classify the positions of possible target pixels, record the number of candidate targets and the shape, size, center position and brightness of each target;

D. According to steps A, B, and C, continuously acquire 3 frames of images with possible targets, and use the method of inter-frame correlation to remove some false targets extracted from the background with fixed relative positions, and according to the multi-frame motion of the target. Select targets to be tracked based on the principle of responsiveness;

E. For the target selected in step D, calculate the image motion compensation amount for tracking imaging according to the imaging frame rate, detector pixel size, installation position of the image motion compensation mirror and other characteristics, and send it to the controller for image motion compensation imaging.

The above steps are realized by FPGA+DSP hardware. The hardware implementation block diagram of the real-time processing system is shown in Figure 2. After the system is started, the FPGA generates the detector drive sequence for image acquisition, and reads the data output by the detector for image filtering preprocessing and possible target pixel extraction. The extracted The possible target is sent to the DSP for further processing through the EMIF port of the DSP, that is, the above steps A and B; the DSP classifies the received possible target pixels, and runs the inter-frame correlation after continuously acquiring 3 frames of images with the target The false target removal algorithm finds the target and performs motion consistency detection, estimates the position of the target and the amount of motion compensation required for imaging, and sends this information to the two-dimensional pointing turntable and the high-precision two-dimensional pressure through the GPIO port of the DSP. The electrically tilted mirror is controlled to perform image motion compensation imaging, that is, steps C, D, and E above.

In step A: FPGA’s target enhancement filtering process adopts a general correlation double-scale filtering algorithm, that is, a high-pass filter operator with a scale of S×S is used to perform small-scale filtering on the original image to complete the protection of small targets with high-frequency characteristics. The value range of S is an odd number between 1 and 5; at the same time, the original image uses a smoothing filter operator with a scale of L×L to perform large-scale filtering to realize the estimation of the image background, where the selection range of L is between 5 and 9 Odd number, and L>S; the image obtained by double-scale filtering is subtracted, that is, the target enhanced image after background removal is obtained, and random noise is also suppressed.

In step B: the FPGA detects the possible target pixels that exist while actually traversing and filtering the image. It accumulates the 4×4 neighborhood of the image pixel. If the brightness of a certain pixel neighborhood If the accumulated value exceeds a given threshold, it is considered that there may be a target, and its position and brightness value are recorded. The given threshold value is selected between 1.2 and 2 times the estimated value of the background obtained by large-scale filtering, and the selection of the specific value is determined according to the characteristics of the target and the background of the actual application. After the above operations of the FPGA, if no possible target pixel is detected, the image is discarded and image acquisition continues, otherwise the recorded bright spots are sent to the DSP for further processing.

In step C: after the detection by the FPGA, the DSP needs to further process the few detected possible target pixels. DSP first performs circular detection and classification on these possible target pixels, and calculates the number of common possible targets in the image, that is, if some recorded target pixel positions are adjacent, they are classified as the same possible target. After the classification is completed, record the number of possible targets, and count information such as the center position of each possible target class, the average brightness, the number of pixels occupied, and the number of pixels extending in the X and Y directions.

In step D: if the system continuously acquires 3 frames (frame numbers N-1 to N+1) of images containing bright spots of the target, the DSP uses an inter-frame correlation algorithm to remove false targets and detect the consistency of target trajectories. Firstly, the center point position of the N-1 and N-th frames after classification is the smallest target in the X and Y directions as the origin, and the relative position of other target center points to this point is calculated; then compare the two frames, If the relative position errors of most of their possible targets (not less than half of the number of possible targets) in the X and Y directions are within Nf (take 0 to 4) pixels, then these possible targets are considered to be in the background false targets and record their positions; otherwise, select the pixel with the smallest difference from the X position of the original origin as the origin, and continue the above operations until more than half of the false targets can be removed. If all pixels are selected If the pixel is the origin and the false target cannot be removed, discard frame N-1 and continue to read frame N+2. In the same way as above, perform false target query on the Nth frame and the N+1th frame, and remove the false targets recorded twice in the three frames. After this operation, most of the false targets can be removed. If there is still more than one possible target, select the remaining possible targets in the three frames whose brightness difference does not exceed 20% of the mean value, the difference in the number of pixels occupied does not exceed Np (take 0 to 8), and the target is between X and Y The difference in the extension length of the pixels in the direction does not exceed Nxy1 (take 0 to 4) as the candidate target group, and judge its motion consistency in the three frames. If the motion position difference of a certain group of candidate targets is in the X and Y directions If there are no more than Nxy2 (take 0-4) pixels, it is concluded that it is the target of the system search, otherwise, if there is no similar target, the N-1th frame is discarded and the analysis of the N+2th frame is continued until the target is found . The parameters Nf, Np, Nxy1, and Nxy2 are selected according to the degree of strictness of judgment, in order from strict to loose, and each parameter is valued from small to large.

In step E: DSP calculates the average motion speed of the target image movement in the X and Y directions for the searched target, that is, the target position difference between the N+1 frame and the N-1 frame/(2*frame period), The unit is pixel/s. Assuming that the image movement speed is Sx and Sy in the X direction and Y direction respectively, the distance between the detector pixels is A×Aum ² , and the distance between the mirror center of the two-dimensional piezoelectric tilting scanning mirror and the photosensitive unit of the detector is H um, then The motion compensation speed of the two-dimensional piezoelectric tilting scanning mirror is Sx×A/2H rad/s in the X direction, and Sy×A/2H rad/s in the Y direction.

The advantages of this method are:

1. The system adopts an algorithm with low computational complexity, which can meet the requirements of real-time and accuracy, and is suitable for processing high-speed across the imaging field of view. Among them, the correlation double-scale filtering algorithm and the neighborhood accumulation method can better remove the background with little change and determine the possible target pixels, while the multi-frame correlation algorithm can more accurately remove the relative positions in the background. According to the consistency test, it can accurately detect the small target moving at high speed.

2. The real-time processing hardware system adopts the combination of FPGA and DSP, which can not only give full play to the characteristics of fast speed of FPGA hardware processing in image traversal filtering, but also take advantage of the high frequency of DSP instruction operations. They cooperate with each other to complete image processing. Data processing tasks such as ergodic filtering, target search and trajectory estimation can well meet the requirements of real-time processing of the system.

Description of drawings

Figure 1 is a flow chart of the real-time processing system.

Figure 2 is a block diagram of the real-time processing system hardware.

Figure 3 is a high-speed moving small target imaging system;

In the figure, 1-imaging optical system;

2- Two-dimensional image motion compensation piezoelectric tilting mirror;

3 - focal plane detector;

4- Two-dimensional tracking and pointing turntable;

5- Real-time processing and control mechanism.

Specific implementation method

According to the real-time self-adaptive processing method described in the specification, the structural diagram of the platform is shown in Figure 3. The platform consists of an imaging optical system 1, a two-dimensional image motion compensation piezoelectric tilt mirror 2, a focal plane detector 3, a two- Tracking and pointing turntable 4 and real-time processing and control mechanism 5 constitute, wherein:

The imaging optical system 1 adopts an ordinary optical telescope with an aperture of 150mm, a focal length of 1800mm, an F number of 12, and a resolution of <1". The instantaneous field of view of the system is 8urad, and the total field of view is 0.47°;

The two-dimensional image motion compensation piezoelectric tilting mirror 2 adopts a four-point XY axis tilting platform, the closed-loop tilt angle can reach ±2mrad, the resolution can reach 0.05urad, the mirror diameter is 50mm, the closed-loop linearity is 0.2%, and the resonance frequency is 3.3KHz. The distance H between the mirror surface and the focal plane is 50mm;

The large area array focal plane detector 3 adopts a monochromatic area array CMOS device with a global shutter. The area array size is 1k×1k, the response wavelength is 400-1000nm, the pixel size is 14um×14um, the frame frequency is 20fps, and the integration time Can be controlled up to us level;

The two-dimensional tracking and pointing turntable 4 adopts an ordinary two-dimensional turntable with pitch rotation and azimuth rotation, the azimuth rotation angle is 0~360 degrees, the speed is 0.5°/s~4°/s, the accuracy is less than 0.05°, and the pitch rotation angle is 0° ~90°, speed 0.5°/s~4°/s, accuracy <0.1°, load up to 200Kg.

The hardware structure of the real-time processing system is shown in Figure 2. It is composed of FPGA+DSP real-time image processing and recognition module and high-precision two-dimensional controller. The number of FPGA logic units is 12060LEs, with 2 built-in PLLs and 234Kbits RAM. The clock is 40MHz; the selected fixed-point DSP external clock is 50MHz, and the operating frequency is 600MHz. It has GPIO interface and 32-bit EMIF interface, and the program is stored in Flash.

In the real-time processing algorithm, the filter scale L=7 of the large-scale smoothing filter is selected, and the weight value is 1; the scale of the small-scale filter is S=3, the weight value of the center point is 4, and the weight value of the neighborhood is 1; the possible target image The meta-judgment threshold is 1.5 times the estimated background brightness. When the false target is removed, the position error Nf for judging whether it has the characteristics of a false target is taken as 2 pixels, and the brightness difference for judging the similarity of the target is taken as 20%. The pixel difference does not exceed Np=6 pixels, and the difference between the extension of the target pixel in the X and Y directions does not exceed Nxy1=2 pixels. When judging the consistency of the target movement, the difference between the X and Y directions No more than Nxy2=2 pixels in the Y direction.

Claims (4)

1. a real-time adaptive processing method for eliminating image motion, characterized in that: the method may further comprise the steps: A. Use correlated dual-scale filtering to traverse the acquired image with motion blur to achieve image filtering preprocessing, remove the slowly changing background to obtain the target enhanced image; B. Use the algorithm of adjacent pixel accumulation to record possible target pixels, extract possible target pixels whose brightness exceeds a certain threshold of background brightness, and suppress the influence of random noise at the same time; C. Classify the positions of possible target pixels, record the number of candidate targets and the shape, size, center position and brightness of each target; D. According to steps A, B, and C, continuously acquire 3 frames of images with possible targets, and use the method of inter-frame correlation to remove some false targets extracted from the background with fixed relative positions, and according to the multi-frame motion of the target. The target to be tracked is selected based on the principle of security; the specific method of removing the false target is as follows: a. Taking the target with the smallest center point position in the X and Y directions of the possible targets in the N-1 and N-th frames as the origin, calculate the relative positions of other target center points and this point; then compare the two frames , if the relative position errors of most of their possible targets in the X and Y directions are within the set error range Nf, then these possible targets are considered to be false targets in the background and their positions are recorded, otherwise, select The pixel with the smallest difference from the original X position of the origin is the origin. Continue the above operation until more than half of the false objects can be removed. If all the pixels are selected as the origin and the false objects cannot be removed, set Discard the N-1th frame, continue to read the N+2th frame, perform false target query on the Nth frame and the N+1th frame with the above method, and remove the false targets recorded twice in the three frames; b. After the operation of method a, most of the false targets can be removed. If the remaining possible targets are still more than 1, the brightness difference of the remaining possible targets in the three frames selected does not exceed 20% of the average value, and the difference in the number of pixels occupied is the same. More than Np pixels, and the difference between the pixel extension length of the target in the X and Y directions is not more than Nxy1 is the candidate target group, and its motion consistency in the three frames is judged. If the motion difference of a certain group of candidate targets is within If there are no more than Nxy2 pixels in the X and Y directions, it is concluded that it is the target of the system search, otherwise, if there is no similar target, the N-1th frame is discarded and the analysis of the N+2th frame is continued until the target is found; Among them, the parameters Nf, Nxy1, and Nxy2 take values within the range of 0 to 4 pixels, and the parameter Np takes values within the range of 0 to 8 pixels. Parameters take values from small to large; E. For the target selected in step D, calculate the image motion compensation amount for tracking imaging according to the imaging frame rate, detector pixel size, installation position of the image motion compensation mirror and other characteristics, and send it to the controller for image motion compensation imaging. 2. The real-time adaptive processing method of a kind of image motion elimination imaging according to claim 1, characterized in that: said step A adopts a correlation double-scale filtering algorithm, and the S of the original image S×S small-scale filtering The value range is an odd number between 1 and 5; the value range of L for L×L large-scale filtering is an odd number between 5 and 9, and L>S. 3. The real-time adaptive processing method of a kind of image motion elimination imaging according to claim 1, characterized in that: in said step C, the threshold for judging that there may be a target is the background estimated value obtained by large-scale filtering 1.2 to 2 times. 4. The real-time adaptive processing method of a kind of image motion elimination imaging according to claim 1, characterized in that: said real-time adaptive processing method of image motion elimination imaging adopts FPGA+DSP hardware real-time processing to realize.

Images