CN103076003A - Image sequence displacement vector measuring device based on electronic image processor - Google Patents

Abstract

An image sequence displacement vector measuring device based on an electronic image processor relates to the technical field of motion detection in the digital image processing field. The image sequence displacement vector measuring device comprises an area array detector and an image processing circuit, wherein the image processing circuit comprises an image data acquisition interface module, a first image data caching module, a second image data caching module, an image processor reading-writing and testing interface module, an electronic image processor, a D/A (digital/ analog) conversion module and a secondary power module; and the electronic image processor performs a correlation operation on image sequence data acquired by the area array detector to obtain displacement vector information of adjacent image frames so as to achieve precise and rapid detection of a boresight offset angle. The image sequence displacement vector measuring device is small in size, light in weight, low in cost, high in reliability, insensitive to spatial radiation and violent vibration, high in operational precision and high in operational speed, and is very suitable for aerospace.

Description

A kind of image sequence displacement vector measurement mechanism based on the electronic image processor

Technical field

The present invention relates to the motion detection technology in the digital image processing field, a kind of image sequence micrometric displacement device for measuring vector quantity based on the electronic image processor specifically, this device can be applied to Aeronautics and Astronautics long-focus imaging optical system and stablize imaging field.

Background technology

Development along with airborne, spaceborne high-definition remote sensing technology, the focal length of imaging optical system is constantly prolonging, angular resolution is also improving constantly, the satellites such as WorldView-2, GeoEye-1/2, IKONOS such as in recent years emission, focal length all surpasses 10m, and angular resolution is all less than 1 μ rad.The raising of angular resolution is so that system is more responsive for the disturbance in the external world, in fact, because the line of sight shake of satellite platform generation of vibration has become the key factor that affects Remote Sensing Image Quality.

The image quality decrease that causes in order to compensate above-mentioned line of sight shake at first should adopt a kind of suitable means, accurately measures the angular displacement information of line of sight shake within the extremely short time.

C.J.Weaver and J.W.Goodman have proposed " associating correlating transforms device " (C.J.Weaver in 1966, J.W.Goodman. A technique for optically convolving two functions. Appl.Opt., 5:1248-1249,1966), the method is based on Fourier optical principle, can realize by photographic medium or spatial light modulator (Space Light Modulator, SLM).

The ultimate principle of said method is respectively as with reference to image and target image with two adjacent before and after in temporal sequence of images two field pictures, with the two synthetic frame associating input picture, by this image is implemented computing cross-correlation, obtain containing the spectral image of displacement vector information.

If the reference picture of order input is r (x, y), it is centered close to (a, 0); Target image is t (x, y), and it is centered close to (a, 0); The associating input picture that is comprised of reference picture and target image is f (x, y), and then the image of input can be expressed as:

f(x,y)=r(x+a,y)+t(x-a,y)

Then the spectrum distribution through a Fourier transform is:

F(u,v)=exp[j2πau]R(u,v)+exp[-j2πau]T(u,v)

Wherein:

R(u,v)=∫∫r(x,y)exp[-j2π(ux+vy)]dxdy

T(u,v)=∫∫t(x,y)exp[-j2π(ux+vy)]dxdy

F (u, v), R (u, v) and T (u, v) are respectively the Fourier transform of associating input picture, reference picture and target image, then joint power spectrum:

$\begin{array}{c}{\left|F(u,v)\right|}^2={\left|R(u,v)\right|}^2+{\left|T(u,v)\right|}^2\\ +\exp \left[j4\mathrm{\πan}\right]R(u,v)T^{*}(u,v)\\ +\exp [-j4\mathrm{\πan}]T(u,v)R^{*}(u,v)\end{array}$

Above-mentioned joint power spectrum is carried out inverse Fourier transform can be got:

$\begin{array}{c}f(\mathrm{\ξ},\mathrm{\η})=r(\mathrm{\ξ},\mathrm{\η})\⊗r(\mathrm{\ξ},\mathrm{\η})+t(\mathrm{\ξ},\mathrm{\η})\⊗t(\mathrm{\ξ},\mathrm{\η})\\ +r(\mathrm{\ξ},\mathrm{\η})\⊗t(\mathrm{\ξ},\mathrm{\η})*\mathrm{\δ}(\mathrm{\ξ}+2a)\\ +t(\mathrm{\ξ},\mathrm{\η})\⊗r(\mathrm{\ξ},\mathrm{\η})*\mathrm{\δ}(\mathrm{\δ}-2a)\end{array}$

In the formula,

Representative is relevant, and * represents convolution, near front two centers that overlap output plane in the expression formula, is called Zero-order diffractive, is not needed signal.The 3rd and the 4th in the expression formula is called positive and negative first-order diffraction, the two in output image respectively translation-2a and 2a, these two relative shifts that comprised between target image and the reference picture are coherent signals of looking for.By mathematical measures such as centroid algorithms, can search at the image of output the position of relevant peaks, the relative displacement vector before and after this position vector namely represents between two two field pictures.

The hardware implementation mode of said method is as shown in Figure 1: typically unite correlator and be comprised of two identical transducers of structure, the effect of each transducer is that the image of input is implemented a two-dimensional Fourier transform, through twice Fourier transform, can finish related operation.

The image that the associating correlator at first photographs planar array detector 101 is loaded on SLM 105 No. one, then by laser instrument 102 as light source, the light of outgoing is through collimation lens 103 collimations, modulation through a SLM 105, after Fourier transform lens 104 carries out two-dimensional Fourier transform, with spectral imaging on No. two planar array detectors 106.Then, the spectral image of Fourier transform is loaded on SLM 107 No. two for the first time again, and is last, obtains containing the image of simple crosscorrelation information and exporting at No. three planar array detectors 108.

Said method can extract displacement vector effectively, and measuring accuracy is better than 1 pixel.But, if according to above-mentioned implementation method based on SLM, have some problems, as follows:

1) system is too complicated:

This system comprises following device at least: three table top array detectors, two spatial light modulators, two laser instruments, two groups of collimation lenses, two groups of fourier lenses.In addition, corresponding supporting structure, power supply, electronics control panel etc. also to be installed.Increasing weight, the power consumption that above-mentioned device causes system significantly increases, and reliability decrease does not meet basic demand airborne, satellite borne equipment;

2) relatively poor to the adaptive faculty of space environment:

At present, spatial light modulator adopts liquid crystal device more, and well-known, liquid crystal device can not be resisted the damage that space radiation causes.And different from general electronic devices and components, liquid crystal device itself can't implement to carry out radiation hardening.

3) the system responses frequency is lower:

The response frequency of LCD space light modulator is lower, can't adapt to the line of sight shake of real-time detection high frequency.Take the LC-R720 type LCD space light modulator of German Holoeye company as example, its highest response frequency only is 180Hz, sweeps line frequency (being frequency of exposure) far below pushing away of remote sensing camera kHz magnitude.

Summary of the invention

The technical matters that system is too complicated, relatively poor to the adaptive faculty of space environment, the system responses frequency is lower that exists in order to solve existing associating correlator, the invention provides a kind of image sequence displacement vector measurement mechanism based on the electronic image processor, it is complete correlator based on the electronics image processor, have the advantages such as volume is little, lightweight, low in energy consumption, its simple structure, realizability is better, is more applicable for the application under the rugged surroundings such as Aeronautics and Astronautics.

The technical scheme that technical solution problem of the present invention is taked is as follows:

A kind of image sequence displacement vector measurement mechanism based on the electronic image processor comprises: planar array detector and image processing circuit, described image processing circuit comprise image data acquiring interface module, the first view data cache module, the second view data cache module, image processor read-write and test interface module, electronic image processor, D/A modular converter and secondary power supply module;

Described planar array detector is connected with the image data acquiring interface module, the image data acquiring interface module is connected with the first view data cache module, the first view data cache module is connected with the second view data cache module, the electronic image processor is connected with test interface module, D/A modular converter with the first view data cache module, the second view data cache module, image processor read-write respectively, and the secondary power supply module is connected with image data acquiring interface module, the second view data cache module, electronic image processor, D/A modular converter respectively;

The view data of described planar array detector output is gathered to the first view data cache module by the image data acquiring interface module, the first view data cache module store this view data and simultaneously with the image data transmission that receives to the second view data cache module, the second view data cache module is inputted view data the electronic image processor simultaneously with the first view data cache module after receiving the view data that the first view data cache module transmits; The first view data cache module with the image data transmission of inside to the electronic image processor, empty the internal image data or read in again a two field picture and cover original digital image data; After the electronic image processor reads in two frame image datas, the two is spliced into a two field picture, and carry out cross-correlation calculation and obtain the two-dimension displacement vector data, then the two-dimension displacement vector data is transferred to the D/A modular converter, the D/A modular converter is converted into by way of compensation signal output of analog quantity with the two-dimension displacement vector data by digital quantity;

The read-write of described image processor and test interface module are that the electronic image processor downloads or the interface of write-in program, and the secondary power supply module is used for providing power supply to image processing circuit.

Beneficial effect of the present invention is as follows:

1) apparatus structure is significantly simplified, and feasibility significantly improves:

Structure of the present invention only comprises a table top array detector and an electronic image processor, compare with existing embodiment and to have saved the device very responsive to space radiation environment such as laser instrument, lens, SLM, its realizability at space industry is strengthened greatly.

2) device volume dwindles, and installation weight alleviates:

Existing correlator comprises laser instrument, lens, the device that the SLM equal-volume is larger, these optical element general assembly (TW)s account for several kilograms, be not suitable for the aerospace applications to system weight requirement harshness, and the present invention has saved above-mentioned optical element, needed total weight is much smaller than one kilogram, and is approximately tens of to hundreds of grams.

3) measuring accuracy of device and reliability strengthen:

Existing associating correlator needs laser instrument, lens, the triangular optical axis close adjustment of SLM, the displacement measurement precision of the direct confining device of calibration accuracy, when being subject to the high-intensity oscillations such as rocket launching when device, easily make between above-mentioned optical element and produce relative displacement, thereby cause measuring accuracy to descend.The version that the present invention adopts is simple, the impact of extraneous factor such as is vibrated hardly.

4) realize that cost significantly reduces:

If the optical device such as laser instrument adopt space flight level product, its cost is very high, and number of devices of the present invention is less, and cost is lower.

5) arithmetic speed promotes:

The level that reaches according to the product on the market, a lot of CMOS cameras all have higher read-out speed, especially choose little area-of-interest (Region Of Interest, ROI) rear (such as 256 * 128 or 128 * 64 or 64 * 32), its frame frequency can reach more than the 10k fps.And for the electronic image processor, take FPGA as example, fpga chip has the characteristics such as parallel processing, streamline operation, its operation FFT(Fast Fourier Transform (FFT)) time of calculating reaches μ s magnitude, therefore is easy to realize for high speed processing.

Description of drawings

Fig. 1 is the structural representation of existing associating correlator.

Fig. 2 is the schematic diagram that the present invention is based on the image sequence displacement vector measurement mechanism of electronic image processor.

Fig. 3 is the functional block diagram of the image processing circuit among the present invention.

Embodiment

Below in conjunction with drawings and Examples the present invention is done and to describe in further detail.

As shown in Figure 2, the image sequence displacement vector measurement mechanism that the present invention is based on the electronic image processor mainly is comprised of planar array detector 201 and image processing circuit 202.As shown in Figure 3, the image processing circuit 202 in the measurement mechanism of the present invention comprises image data acquiring interface module 301, the first view data cache module 302, the second view data cache module 303, image processor read-write and test interface module 304, electronic image processor 305, D/A modular converter 306 and secondary power supply module 307.

Described planar array detector 201 can adopt area array CCD detector or face battle array cmos detector; The DS90LV032A type interface chip that image data acquiring interface module 301 can adopt National Semiconductor company to produce; The first view data cache module 302 and the second view data cache module 303 all can adopt DRAM(Dynamic Random Access Memory) or SRAM(Static Random Access Memory) or SDRAM(Synchronous Dynamic Random Access Memory, synchronous DRAM) or the FIFO(First Input First Output of FPGA inside), such as the HY57V643220C type SDRAM chip that adopts Hynix company to produce; The XCF16P cake core that image processor read-write and test interface module 304 can adopt XILINX company to produce; Electronic image processor 305 can adopt FPGA or DSP or CPU or GPU(Graphic Processing Unit, graphic process unit) etc. the electronic processors that meets the demands of computing power, such as the XC6VSX475T that selects in the Virtex-6 of the XILINX company series; The AD9777 chip that D/A modular converter 306 adopts AD company to produce; The LT1764A chip that secondary power supply module 307 adopts multi-disc Linear Technology company to produce.

Above-mentioned planar array detector 201 is connected with image data acquiring interface module 301, image data acquiring interface module 301 is connected with the first view data cache module 302, the first view data cache module 302 is connected with the second view data cache module 303, electronic image processor 305 respectively with the first view data cache module 302, the second view data cache module 303, image processor read-write and test interface module 304, D/A modular converter 306 connects, secondary power supply module 307 respectively with image data acquiring interface module 301, the second view data cache module 303, electronic image processor 305, D/A modular converter 306 connects.

The course of work (operation of whole device is the process of a continuous reciprocation cycle) that the present invention is based on the image sequence displacement vector measurement mechanism of electronic image processor is as follows: the view data of planar array detector 201 outputs is gathered to the first view data cache module 302 by image data acquiring interface module 301, the first view data cache module 302 these view data of storage are also given the second view data cache module 303 with the image data transmission of reception simultaneously, the second view data cache module 303 is inputted view data electronic image processor 305 simultaneously with the first view data cache module 302 after receiving the view data that the first view data cache module 302 transmits; The first view data cache module 302 with the image data transmission of inside to electronic image processor 305, empty the internal image data or read in again a two field picture and cover original digital image data; After electronic image processor 305 reads in two frame image datas, the two is spliced into a two field picture, and carry out cross-correlation calculation and obtain the two-dimension displacement vector data, then the two-dimension displacement vector data is transferred to D/A modular converter 306, D/A modular converter 306 is converted into by way of compensation signal output of analog quantity with the two-dimension displacement vector data by digital quantity; Image processor is read and write and test interface module 304 downloads for electronic image processor 305 or the interface of write-in program, the primary power source voltage transitions that secondary power supply module 307 is used for aircraft or satellite platform are provided is the operating voltage of image processing circuit 202 each components and parts, and for whole image processing circuit 202 provides power supply, to guarantee the normal operation of whole device.

Claims (4)

1. image sequence displacement vector measurement mechanism based on the electronic image processor, it is characterized in that, this device comprises planar array detector (201) and image processing circuit (202), and described image processing circuit (202) comprises image data acquiring interface module (301), the first view data cache module (302), the second view data cache module (303), image processor read-write and test interface module (304), electronic image processor (305), D/A modular converter (306) and secondary power supply module (307);

Described planar array detector (201) is connected with image data acquiring interface module (301), image data acquiring interface module (301) is connected with the first view data cache module (302), the first view data cache module (302) is connected with the second view data cache module (303), electronic image processor (305) respectively with the first view data cache module (302), the second view data cache module (303), image processor read-write and test interface module (304), D/A modular converter (306) connects, secondary power supply module (307) respectively with image data acquiring interface module (301), the second view data cache module (303), electronic image processor (305), D/A modular converter (306) connects;

The view data of described planar array detector (201) output is gathered to the first view data cache module (302) by image data acquiring interface module (301), the first view data cache module (302) store this view data and simultaneously with the image data transmission that receives to the second view data cache module (303), the second view data cache module (303) is inputted view data electronic image processor (305) simultaneously with the first view data cache module (302) after receiving the view data that the first view data cache module (302) transmits; The first view data cache module (302) with the image data transmission of inside to electronic image processor (305), empty the internal image data or read in again a two field picture and cover original digital image data; After electronic image processor (305) reads in two frame image datas, the two is spliced into a two field picture, and carry out cross-correlation calculation and obtain the two-dimension displacement vector data, then the two-dimension displacement vector data is transferred to D/A modular converter (306), D/A modular converter (306) is converted into by way of compensation signal output of analog quantity with the two-dimension displacement vector data by digital quantity;

Described image processor is read and write and test interface module (304) downloads for electronic image processor (305) or the interface of write-in program, and secondary power supply module (307) is used for providing power supply to image processing circuit (202).

2. a kind of image sequence displacement vector measurement mechanism based on the electronic image processor as claimed in claim 1 is characterized in that, described planar array detector (201) is area array CCD detector or face battle array cmos detector.

3. a kind of image sequence displacement vector measurement mechanism based on the electronic image processor as claimed in claim 1 is characterized in that, described electronic image processor (305) is FPGA or DSP or CPU or GPU.

4. a kind of image sequence displacement vector measurement mechanism based on the electronic image processor as claimed in claim 1 is characterized in that, described the first view data cache module (302) is the FIFO of DRAM or SRAM or SDRAM or FPGA inside.

Images