Summary of the invention
The present invention be directed to present multi-source image method for registering and have the limitation problem of practical application, a kind of method for registering images based on triangulation has been proposed, used the embedded system of ARM and DSP combination, this system has the mode of the registration strategies called respectively: the unloading phase, user's manual mode and update mode regularly.No matter be same type of sensor registration or foreign peoples's sensor configuration, unified utilize the Delaunay triangulation as a comparison feature carry out registration.Can effectively drive method for registering images utilization in practice, and effectively improve the performance of image registration.
Technical scheme of the present invention is: a kind of method for registering images based on triangulation comprises the steps:
1) obtaining of video image: will be converted into 24 bit digital pixel values of LDVS form from the analog video signal that CCD video camera and thermal infrared imager obtain, after the represented pixel of all simulating signals is changed through A/D, obtain the digital video image signal of a frame one frame, send it to the DSP device and carry out denoising based on multiresolution method;
2) based on the decomposition of multiresolution: adopt method, original image and Hi-pass filter h based on wavelet frame transform
n, low-pass filter g
nCarry out filtering respectively, filtered result is respectively the low-pass coefficients of wavelet frame at the high-pass coefficient and the n layer of n layer, continue if desired to decompose, then needing to proceed zero insertion handles, up to finishing set decomposition goal, for the high pass part of wavelet frame, carried out Filtering Processing, and low pass partly is used to carry out the registration strategies based on Delaunay;
3) Corner Detection: utilize the low-pass coefficients after wavelet frame decomposes, finish by each picture element that scans whole low-pass coefficients image, with each picture element is the center of circle, with certain r that configures in advance is the circle that radius is made a graphics, and on the particular location of image, realize locating, getting the picture element that drops on the graphics circle and the picture element of circle centre position compares, if the picture element on all graphics circle is greater than or less than rank t of gray-scale value arrival that the center of circle goes out picture element, think that then this point is unique point;
4) triangle coupling: construct a super triangle, comprise all unique points, put into the triangle chained list; The unique point that point is concentrated is inserted successively, in the triangle chained list, find out the triangle that its circumscribed circle comprises the insertion point, the leg-of-mutton common edge of deletion influence couples together the insertion point, thereby finishes the insertion of a point in Delaunay triangle chained list with the leg-of-mutton whole summits of influence; According to optimizing criterion the new triangle that forms in part is optimized, the triangle that forms is put into Delaunay triangle chained list; Circulation is carried out unique point and is inserted, and inserts up to all points that loose to finish;
5) registering images: the coupling in the Delaunay triangle chained list is carried out two rigidity characteristic invariants and is calculated, and supposes l
1, l
2, l
3Be three sides of a triangle, m
1, m
2, m
3Be leg-of-mutton three angles:
First rigid transformation invariant:
Second rigid transformation invariant: V
m=[∠ m
1, ∠ m
2, ∠ m
3],
Then the ratio between them is compared, when the summation of the result after the comparison is worth less than one, then can select corresponding anglec of rotation parameter and translation parameters as final registration results.
Described step 2) in based on the method for wavelet frame transform: the employed original small echo of setting of the wavelet frame coefficient in the filtering algorithm program predefine part of wavelet frame is the DB3 small echo, and its high-pass filtering coefficient is:
[0.037828;-0.023849;-0.110624;0.377403;0.852699;0.377404;-0.110624;-0.023849;0.037828]
The low-pass filtering coefficient is:
[-0.064539;0.040689;0.418092;-0.788486;0.418092;0.040689;-0.064539;0;0];
The coefficient that obtains after wavelet coefficient carry out frameworkization is
Its high-pass filtering coefficient is:
The low-pass filtering coefficient is:
The above wavelet frame coefficient when once decomposing only, more high-rise if desired decomposition then needs to proceed zero insertion and handles.
Beneficial effect of the present invention is: the method for registering images that the present invention is based on triangulation, have applied widely, registration accuracy advantages of higher, handle, judge the significant and practical values of work such as decision-making based on the processing or the man-machine interaction of artificial intelligence such as feature level and decision level for the image co-registration that vision signal is follow-up.
Embodiment
As shown in Figure 1 based on the system hardware structure block diagram of the method for registering images of triangulation, two imaging sensors have been adopted, one is AVT industrial CCD camera F-032B/C, another one is that Infrared Solutions company can provide a thermal imaging system based on non-cooling bolometer technology: IR-160 type imager, its exportable 160 * 120 pixel NTSC or PAL video output signals; Sensor kind selector switch and DSP image co-registration chip have all adopted the TMS320C6000 of TI company; A/D multi pass acquisition chip adopts 24 A/D converter CS5381; The ARM940T that the ARM chip has adopted ARM company to produce.
Based on the software architecture diagram of the method for registering images of triangulation, the registration process of arranging in the software architecture realizes calling by three kinds of modes respectively as shown in Figure 2.1) realized the automated graphics registration process of system start-up initial phase, this calls in cold start-up, warm start process and loads automatically; 2) realized regularly calling the function of image registration, this calls is that software mark after overflowing by the clock register that is provided with in the system is automatically realized calling, the initial value of this clock register has determined the interval time that timing diagram calls as registration procedure, generally speaking, this time is set to 300 seconds/time; 3) realized that the calling graph of user's triggering mode is as registration procedure, this calls is to trigger by a button on based on the function interface of the embedded OS of ARM to finish the action that this is compensated when normally the simple fused images after the user finds registration occurs than large deviation.
On the ARM chip, finish the control of startup, on-line operation and the service logic of embedded Control platform program: comprised the display interface of function button such as " being written into ", " image A ", " image B ", " image registration ", " option " and view data in the embedded Control platform program, the operable support platform of this Presentation Function (including but not limited to): OpenGL, DirectX etc.; What " be written into " that button mainly finishes view data is written into procedure for displaying automatically; " image registration " button is realized calling of aforesaid image registration program; What " option " specifically provided is the setting of various image registration parameters and image expanding method for registering, and the present invention has only provided " based on the method for registering images of Delaunay triangulation "; The display interface of view data only is that the api function that has called support platforms such as OpenGL, DirectX has been realized view window, and shows the image behind the registration in this view window; " image A ", " image B " button are respectively the dynamic images that shows two foreign peoples's sensors in view window.
Collection and input, the denoising of video image and the output of video image of video image have been carried out on the DSP module based on the method for registering images of triangulation.Its specific descriptions are as follows:
The collection of image and input: obtaining of video image is by A/D converter, at first will be converted into 24 bit digital pixel values of LDVS form from the analog video signal that CCD video camera (F-032B/C) and thermal infrared imager (IR-160) obtain.After the pixel that all CCD acquisition simulating signals is represented is changed through A/D, obtain the digital video image signal of a frame one frame at last, because the resolution of F-032B/C is 640 * 480, and the resolution of IR-160 is 160 * 120, therefore need be in the DSP side to the ccd signal processing that resamples, make it and 160 * 120 can mating fully that infrared sensor obtains, here need to prove, through manual adjustment, be that the video image that can make two cameras obtain reaches complete registration, according to this angle it carried out mechanical fixation then.After obtaining digital video image, send it to the DSP device and carry out denoising based on multiresolution method.
Each frame of data image signal carries out the denoising based on the data image signal of multiresolution on the DSP device, this processing procedure is divided into the decomposition based on multiresolution, adopts the method based on wavelet frame transform here:
The employed original small echo of setting of the wavelet frame coefficient in the filtering algorithm program predefine part of wavelet frame is the DB3 small echo, and its high-pass filtering coefficient is:
[0.037828;-0.023849;-0.110624;0.377403;0.852699;0.377404;-0.110624;-0.023849;0.037828]
The low-pass filtering coefficient is:
[-0.064539;0.040689;0.418092;-0.788486;0.418092;0.040689;-0.064539;0;0];
The coefficient that obtains after wavelet coefficient carry out frameworkization is
Its high-pass filtering coefficient is:
The low-pass filtering coefficient is:
The above wavelet frame coefficient when once decomposing only, more high-rise if desired decomposition then needs to proceed zero insertion and handles.
Decomposable process is original image (or low-pass pictures of every layer) and Hi-pass filter h
n, low-pass filter g
nCarry out filtering respectively, filtered result is respectively the low-pass coefficients of wavelet frame at the high-pass coefficient and the n layer of n layer.Continue if desired to decompose, then use the low-pass coefficients of n layer to carry out this part computing once more, and the like, up to finishing set decomposition goal.
For the low pass of wavelet frame part and high pass part, follow-uply carried out different processing respectively.For the high pass part, this algorithm has carried out Filtering Processing, and low pass partly is used to carry out the registration strategies based on Delaunay.
Concrete steps based on the filtering algorithm of wavelet frame are: at first the multiresolution that original image is carried out wavelet frame decomposes, and sets up based on mixed Gauss model.
Wherein mixed Gauss model is:
W wherein
iBe hybrid weight, and
g
i() is complex element, i=1 wherein ..., M.The distribution of each element all is a Gaussian distribution, and its average is μ
i, variance is a ∑
i:
Wherein d is the dimension of sampled data.
Used the coefficient of 5 * 5 windows that mixed Gauss model is estimated in the present embodiment, this estimation procedure is the estimation problem of a fragmentary data, and the present invention adopts the method for twice iteration to ask for two optimal values respectively:
First step design conditions probability density:
This step is to calculate in parameter and raw data (image) under the stable condition, the distribute conditional probability density of k item of mixed Gaussian.
The second step undated parameter α
i, α
iValue 1,0, select among the 1} so that the value maximum of (5) formula:
The 3rd step was recomputated conditional probability density distribution g
K, i, l, upgrade real scene S (l):
The 4th step was recomputated conditional probability density distribution g
K, i, l, real scene S (l) and sensor offset amount β
i, the model parameter ω of renewal noise
K, i, σ
K, i 2And β
i:
In the 5th step, use new argument to repeat for first to the 4th step.
In the above-mentioned iterative process, when parameter convergence to some specific scopes, determine that fused images is:
C wherein
iBe the edge feature factor of influence,
Iterate the w that obtains in formula (1) and the formula (2)
i, expectation value μ
iAnd variance ∑
iThe wavelet frame coefficient is by expectation value μ
iReplace.
Cut apart the detected characteristics extractive technique based on carrying out accumulation on the DSP module of the method for registering images of triangulation, its specific descriptions are as follows:
Utilize the low-pass coefficients after wavelet frame decomposes.Detection method is finished by each picture element that scans whole low-pass coefficients image, with each picture element is the center of circle, with certain r that configures in advance is the circle that radius is made a graphics, and on the particular location of image, realize locating, getting the picture element that drops on the graphics circle and the picture element of circle centre position compares, if the picture element on all graphics circle is greater than or less than rank t of gray-scale value arrival that the center of circle goes out picture element, think that then this point is unique point; In this embodiment, the present invention has used 3 as radius, and this means on the graphics circle has 12 pixels.Through a large amount of experimental verification, 9 pixel values should be the best-of-breed technology numbers, that is to say have in these 12 pixels 9 pixels than the gray-scale value of the pixel of circle centre position greater than t, then can think unique point.
Carry out method of addition at random on the DSP module based on the method for registering images of triangulation, its specific descriptions are as follows:
According to the point that at random order insertion point is successively concentrated, in whole process, all to safeguard and upgrade a Delaunay triangulation corresponding with current point set.The present invention adopts shortcut calculation to realize the Delaunay triangulation, and its concrete steps are: 1, initialization: construct a super triangle, comprise all diffusing points, put into the triangle chained list.2, will put concentrated diffusing point inserts successively, in the triangle chained list, find out the triangle (triangle that influences that is called this point) that its circumscribed circle comprises the insertion point, the leg-of-mutton common edge of deletion influence, the insertion point is coupled together with the leg-of-mutton whole summits of influence, thereby finish the insertion of a point in Delaunay triangle chained list.3, according to optimizing criterion the new triangle that forms in part is optimized.The triangle that forms is put into Delaunay triangle chained list.4, above-mentioned the 2nd step is carried out in circulation, inserts up to all points that loose to finish.
Triangle Optimum Matching program has used two indexs that the Delaunay triangle in the iterative process is compared, and when the summation of the result after the comparison is worth less than one, then can select corresponding anglec of rotation parameter and translation parameters as final registration results; The employed optimized Algorithm of matching process has adopted the algorithm of adaptive modeling annealing among the present invention.
These two indexs have been represented two feature invariants respectively, suppose l
1, l
2, l
3Be three sides of a triangle, m
1, m
2, m
3Be leg-of-mutton three angles:
First rigid transformation invariant:
(formula 1)
Second rigid transformation invariant:
V
m=[∠ m
1, ∠ m
2, ∠ m
3] (formula 2)
There are 24 Delaunay triangles among the present invention in the wavelet frame low-pass coefficients of infrared image, in the low-pass coefficients of visible images, have 18 Delaunay triangles.Such one has 24 * 18 (432) individual matching results, and these matching results all carry out two rigidity characteristic invariants and calculate, and then the ratio between them are compared, and chooses that group coupling of ratio maximum.Image after the coupling carries out simple pixel level and merges in the wavelet frame field, the present invention has adopted the fusion method based on traditional small echo, with the effect behind the expression registration.
The input that needs to use two width of cloth video images in the entire image fusion process of the inventive method is ccd image and infrared image as Fig. 3 and Fig. 4, and Fig. 5 is final registration results.