CN103679720A - Fast image registration method based on wavelet decomposition and Harris corner detection - Google Patents

Abstract

The invention discloses a fast image registration method based on wavelet decomposition and Harris corner detection. The size of an image is reduced through the wavelet decomposition, so that the amount of computation is reduced, and the real-time performance of image registration is improved. In addition, the process of the wavelet decomposition is a process of low-pass filtering, and therefore unpitched sound can be processed. Furthermore, a Harris corner detection algorithm is adopted for corner matching. Meanwhile, corners of mismatching points are removed, so that the anti-noise capacity and the anti-jamming capacity are high, and the accuracy of image matching is further improved.

Description

A kind of fast image registration method based on wavelet decomposition and Harris Corner Detection

Technical field

The present invention relates to Digital Image Processing and Model Distinguish correlation technique, be applicable to comprise the image processing and pattern-recognition association area such as navigation, maneuvering target tracking, condition monitoring, be specifically related to a kind of fast image registration method based on wavelet decomposition and Harris Corner Detection.

Background technology

Image registration is the underlying issue that image is processed.Along with the development of computer technology, image registration algorithm has obtained development fast, in fields such as remote sensing, military affairs, medical treatment, navigation, Imaging Guidance, transition detections, is widely used.

At present, method for registering images mainly contains two classes: a class is the method for registering images based on region, refers to that the gray-scale value relation of utilizing between two width image pixels determines the parameter of transformation model, the method has been utilized whole half-tone informations of image, registration accuracy is high, but calculated amount is large, and real-time is poor.At present the common image registration algorithm based on region have ratioing technigue, based on block matching method (claiming again based on template registration Algorithm), mesh fitting method etc., this method is applicable to only have between image the situation of level, vertical translation.

Another kind of is method for registering images based on feature, its basic step is as follows: first extract benchmark image and set of image characteristics subject to registration, then carry out characteristic matching, finally utilize the relation between the feature of registration to estimate geometric transformation model and parametric variable value thereof between benchmark image and image subject to registration.This method utilizes the obvious characteristic in image, rather than utilizes the conversion between information computed image whole in image, and the variation of gradation of image is had to robustness, can be applicable to exist the registration between the image that more complex geometry converts.The common image registration algorithm based on feature has: Harris Corner Detection Algorithm, SUSAN Corner Detection Algorithm, SIFT yardstick invariant features transfer algorithm etc.

At present, in image registration algorithm based on feature, Harris Corner Detection Algorithm is most widely used method for registering images, but this algorithm calculation of complex, make to mate real-time poor, in addition, because the method exists angle point mistake matching problem, make its anti-noise, poor anti jamming capability, thereby cause images match poor accuracy problem.

Summary of the invention

In view of this, the invention provides a kind of fast image registration method based on wavelet decomposition and Harris Corner Detection, thereby make the size reduction of image reduce operand by wavelet decomposition, improved the real-time of image registration, and the process of wavelet decomposition is the process of a low-pass filtering, can process noise, in addition the present invention adopts Harris Corner Detection Algorithm to carry out angle point pairing, also Mismatching point angle point has been carried out rejecting simultaneously and make anti-noise, antijamming capability strong, the accuracy that has further improved images match.

A fast image registration method based on wavelet decomposition and Harris Corner Detection, comprises the following steps:

Step 1, benchmark image and image subject to registration are converted to two-dimentional gray level image from three-dimensional true color image respectively, the gray-scale map of benchmark image and figure subject to registration is designated as respectively to image f ₁with image f ₂;

Step 2, to image f ₁with image f ₂carry out N level wavelet decomposition, the approximate image while obtaining respectively the N time wavelet decomposition, wherein, image f ₁the approximate image carrying out after decomposing for N time is called LLNA, image f ₂the approximate image carrying out after decomposing for N time is called LLNB;

Step 3, the approximate image LLNA and the LLNB that obtain after adopting Harris Corner Detection Algorithm to step 2 wavelet decomposition carry out angle point extraction;

The angle point that the angle point that the approximate image LLNA that step 4, employing NCC matching algorithm obtain step 3 extracts and approximate image LLNB extract carries out the thick registration of angle point, obtains coupling angle point pair;

The angle point that step 5, employing RANSAC algorithm obtain step 4, to verifying, is rejected the angle point pair of mistake coupling, obtains correct coupling angle point pair;

Step 6, utilize the coupling angle point that step 5 obtains to obtain model transferring parameter to carrying out geometric transformation.

Preferably, the N in step 2 equals 2.

Preferably, the geometric transformation in step 6 is affined transformation.

Beneficial effect:

1) first, after N level wavelet decomposition, the size reduction of image is original 1/2 in the present invention ⁿso,, the image based on after wavelet decomposition carries out image registration, has greatly shortened computational complexity, has improved the coupling real-time of image registration.

Secondly, the present invention compares with traditional image registration algorithm based on Harris angle point, introduce wavelet decomposition and rejected two steps of Mismatching point, wavelet decomposition process is the process of a low-pass filtering, to carrying out wavelet decomposition containing noisy image, is that image has been carried out to Denoising disposal in fact, unessential detailed information can be disposed, only retain important essential information, thereby reduced noise, improved images match accuracy.Employing RANSAC algorithm to verifying, is rejected the angle point pair of mistake coupling to angle point, has improved anti-noise and the antijamming capability of method for registering images, has further improved the matching accuracy of image.

2) embodiment of the present invention has been carried out secondary wavelet decomposition to image, does not increase the complexity of calculating in the time of can compressed image, has further guaranteed accuracy and the real-time of images match.

3) the coupling angle point that embodiment of the present invention utilization obtains is to adopting affined transformation to obtain geometric transformation model, this geometric transformation can realize the multiple conversion such as image translation, rotation, again without complicated calculated amount, can further guarantee accuracy and the real-time of images match simultaneously.

Accompanying drawing explanation

Fig. 1 is process flow diagram of the present invention;

Fig. 2 is the approximate image that wavelet decomposition obtains;

Fig. 3 is the angle point extracting on the approximate image of benchmark image and image subject to registration;

Fig. 4 is for adopting NCC matching algorithm to the coupling angle point pair obtaining;

Fig. 5 is for adopting RANSAC algorithm to reject the angle point pair of mistake coupling.

Embodiment

Below in conjunction with the accompanying drawing embodiment that develops simultaneously, describe the present invention.

The invention provides a kind of fast image registration method based on wavelet decomposition and Harris Corner Detection, the method is carried out under computer hardware environment, Windows2000/XP; Any language environment softwares such as matlab or C language or C++ all can be realized, and the present embodiment adopts matlab language environment, and process flow diagram as shown in Figure 1, specifically comprises the following steps:

Step 1, in matlab, input two width image F ₁and F ₂, F wherein ₁for benchmark image, F ₂for image subject to registration, adopt the function rgb2gray () in matlab respectively two width images to be converted to two-dimentional gray level image from three-dimensional true color image respectively, the gray-scale map of benchmark image and figure subject to registration is designated as respectively to image f ₁with image f ₂.

Step 2, to image f ₁with image f ₂carry out N level wavelet decomposition, the approximate image while obtaining respectively the N time wavelet decomposition, wherein, image f ₁the approximate image carrying out after decomposing for N time is called LLNA, image f ₂the approximate image carrying out after decomposing for N time is called LLNB.

For example,, to image f ₁while carrying out one-level wavelet decomposition, first use low-pass filter (L) and Hi-pass filter (H) to image line to carrying out wavelet decomposition, the high fdrequency component obtaining and low frequency component data; Then by the row of the high fdrequency component obtaining and low frequency component data to using respectively low-pass filter (L) and Hi-pass filter (H) to carry out filtering, thereby obtain the low frequency part of image, be called again the first approximation information LL1A of image, the detailed information HL1A of image along continuous straight runs, image detailed information LH1A vertically; Image is along the detailed information HH11 of diagonal.

Image is carried out to secondary wavelet decomposition and the image one-level decomposition approximate information LL1A obtaining is carried out to secondary wavelet decomposition, obtain the two stage approach information LL2A of image, the detailed information HL2A of image along continuous straight runs, image detailed information LH2A is vertically the detailed information HH2A of image along diagonal.

When carrying out N level wavelet decomposition, obtain the N level approximate information of image; The detailed information HLNA of image along continuous straight runs; Image detailed information LHNA vertically; Image is along the detailed information HHNA of diagonal.

This decomposition is a kind of harmless zoom, and approximate component is smooth, thereby wavelet decomposition has compressed image and antimierophonic advantage, the selection of picture breakdown number of times is extremely important, the number of plies of decomposing is more, while carrying out image registration, computation complexity is lower, matching efficiency is higher, yet, decompose the too many words of number of times and can lose a large amount of image informations.Therefore, when image is decomposed, select not only simplified image information of the suitable decomposition number of plies, the important information that has simultaneously retained image, the present embodiment carries out secondary wavelet decomposition to image, in the time of can compressed image, do not increase the complexity of calculating, further guaranteed accuracy and the real-time of images match.

To image f ₁with image f ₂carry out secondary wavelet decomposition, obtain respectively the approximate image after wavelet decomposition, wherein, image f ₁the approximate image carrying out after decomposing for N time is called LL2A, image f ₂the approximate image carrying out after decomposing for N time is called LL2B, and as shown in Figure 2, left figure is LL2A, and right figure is LL2B.

Step 3, employing Harris Corner Detection Algorithm are carried out angle point information extraction to the approximate image LL2A obtaining after step 2 wavelet decomposition and LL2B respectively.

Harris Corner Detection Algorithm (Qu Xiwen. a kind of improved Harris angular-point detection method. mechanical & electrical technology, 2012,40-42) principle is as follows:

First, image window w to be detected is moved to small displacement to any direction, the coordinate of supposing the object pixel in image window w is (x, y), the displacement of moving in x and y direction is respectively u and v, by point (x, y), the grey scale change amount in (u, a v) square window is defined as:

$\begin{array}{c}E(u,v)=\underset{x,y}{\mathrm{\Σ}}w(x,y){[I(x+u,y+v)-I(x,y)]}^2\\ =\underset{x,y}{\mathrm{\Σ}}w(x,y){[I_{x}u+I_{y}v+o(u^2+v^2)]}^2\end{array}$

Wherein: w (x, y) is window function, selects Gauss's window here, to improve antijamming capability; I (x, y), I (x+u, y+v) is the gray scale function of object pixel; o(u ²+ v ²) be displacement dimensionless; I _xand I _yfor the single order shade of gray of object pixel, wherein,

$w(x,y)=\frac{1}{2\mathrm{\π}{\mathrm{\σ}}^2}{e}^{-\frac{x^2+y^2}{2{\mathrm{\σ}}^2}}$

$I_x=\frac{\∂I}{\∂x}=I\⊗(-\mathrm{1,0,1})$

$I_y=\frac{\∂I}{\∂y}=I\⊗(-\mathrm{1,0,1})$

Write as matrix form:

$M=w(x,y)\⊗\left[\begin{array}{cc}{I}_x^2& I_x{I}_y\\ I_x{I}_y& I_y^2\end{array}\right]$

? $E(u,v)=\left[\begin{array}{cc}u& v\end{array}\right]M\left[\begin{array}{c}u\\ v\end{array}\right]$

Wherein: M is the autocorrelation matrix of object pixel; If λ ₁and λ ₂two eigenwerts of M, λ ₁and λ ₂the size of value determined that this object pixel is angle point, edge or flat site:

(1) angle point: λ ₁and λ ₂value all larger;

(2) edge: λ ₁and λ ₂value, one is large, one little;

(3) flat region: λ ₁and λ ₂all very little.

For fear of autocorrelation matrix M is carried out to Eigenvalues Decomposition, definition angle point response function CRF is:

CRF＝det(M)-k*trace ²(M)

Wherein, the determinant that det (M) is matrix M; The mark that trace (M) is matrix M (diagonal of a matrix element and); K is empirical value, conventionally gets 0.04.(Mao Yanming, Lan Meihui, Wang Yunqiong, Feng Qiaosheng. a kind of improved angular-point detection method based on Harris, computer technology and development, 2009.05)

When the present embodiment adopts Harris algorithm to extract angle point to two width images, the criterion of angle point judgement is:

When the value of angle point response function CRF is greater than certain predefined threshold value, be candidate's angle point, otherwise be not angle point, the size of threshold value decides according to the number of required angle point.

Therefore, the number of guaranteeing angle point is abundant, just suitable threshold value need to be set, rule of thumb known, the number of angle point, when 200 left and right, has enough angle points pair, therefore in the time of just guaranteeing image registration, by many experiments, for angle point threshold value setting in the present embodiment image, be 0.000149044.

Two width images in Fig. 2 are carried out to the extraction of Harris angle point, and the angle point of extraction is marked on image, obtain the angle point image shown in Fig. 3.

Harris operator is a kind of effective some feature extraction operator, has following advantage:

1. calculate simple: in Harris operator, only use first order difference and the filtering of gray scale, simple to operate;

2. the some feature of extracting is even and reasonable;

3. stable: in the computing formula of Harris operator, only relate to first order derivative, therefore, insensitive to image rotation, grey scale change, noise effect and viewpoint change, be a kind of more stable feature extraction operator.

The angle point that the angle point that the approximate image LL2A that step 4, employing NCC matching algorithm obtain step 3 extracts and approximate image LL2B extract carries out the thick registration of angle point, obtains coupling angle point pair.

NCC matching algorithm is a kind of matching algorithm of classics, the method be in benchmark image and image subject to registration centered by the angle point extracting, the template of structure M * N, obtain respectively template image (M * N template image that corresponding benchmark image is constructed centered by angle point) and background image (M * N template image that corresponding image subject to registration is constructed centered by angle point), by the cross correlation value of calculation template image and background image, determine the degree of coupling, the maximal value of simple crosscorrelation has determined position and the similarity degree of template image in background image, and then obtain the degree of correlation of angle point on benchmark image and image subject to registration.

In actual match application, the similarity of template image and background image is measured by metric function, and the present embodiment adopts normalizing eliminate indigestion relevant matches metric function C to measure, and it is defined as

$C=\frac{\underset{i=1}{\overset{m}{\mathrm{\Σ}}}\underset{j=1}{\overset{n}{\mathrm{\Σ}}}[f(x_i,y_j)-\stackrel{\‾}{f}][g(x_i^{*},y_j^{*})-\stackrel{\‾}{g}]}{\sqrt{\underset{i=1}{\overset{m}{\mathrm{\Σ}}}\underset{j=1}{\overset{n}{\mathrm{\Σ}}}{[f(x_i,y_j)-\stackrel{\‾}{f}]}^2}\sqrt{\underset{i=1}{\overset{m}{\mathrm{\Σ}}}\underset{j=1}{\overset{n}{\mathrm{\Σ}}}{[g(x_i^{*},y_j^{*})-\stackrel{\‾}{g}]}^2}}---\left(1\right)$

Wherein, f (x _i, y _j) be certain a bit (x on benchmark image _i, y _j) gray-scale value located,

for on image subject to registration, certain is a bit

the gray-scale value at place.

be respectively gray average in the template of M * N on benchmark image and image subject to registration, its value is:

$\stackrel{\‾}{f}=\frac{1}{m\×n}\underset{i,j}{\mathrm{\Σ}}f(x_i,y_j),\stackrel{\‾}{g}=\frac{1}{m\×n}\underset{i,j}{\mathrm{\Σ}}g(x_i^{*},y_j^{*})---\left(2\right)$

Wherein: m and n are respectively the row and column of image.

C is cross-correlation coefficient, when C=1, illustrates that on benchmark image and image subject to registration, corresponding template is complete dependence, when C=0, represents uncorrelated.In actual applications, due to light, the impact of noise on image, make image complete dependence be difficult to, so there is certain error in the correlativity here.In experiment, get C and be greater than at 0.8 o'clock, think that two templates mate, otherwise be unmatched.(Chen soldier. the matching performance comparison [J] of several Measurement of Similarity between Two Images. computer utility, 2010 (1)).

The present embodiment adopts above-mentioned algorithm to carry out the thick registration of angle point to the angle point information of the angle point information of the approximate image LL2A extraction in step 3 and approximate image LL2B extraction, obtains coupling angle point pair, as shown in Figure 4.

The angle point that step 5, employing RANSAC algorithm obtain step 4 is to verifying, the angle point pair of rejecting mistake coupling, as shown in Figure 5, obtains correct coupling angle point pair, improve anti-noise and the antijamming capability of method for registering images, further improved the matching accuracy of image.

As can be seen from Figure 3, the right registration of angle point is inaccurate, and has correct angle point pair, also has the angle point pair of mistake coupling, therefore, in order to improve the precision of angle point to registration, need to mate the right rejecting of angle point by mistake.

RANSAC is the abbreviation of " Random Sample Consensus(stochastic sampling consistance) ", in 1981, by Fischler and Bolles, proposed at first, it is the parameter from one group of data centralization iterative estimate mathematical model that comprises abnormal data, obtains the algorithm of valid data.RANSAC is general adopts fewer point estimation to go out model, recycles remaining point and carrys out verification model, so just alleviates the impact of abnormal data on model parameter estimation while there is gross error point.

The basic assumption of RANSAC algorithm is:

(1) data are comprised of " intra-office point ";

(2) " point not in the know " is the data that can not adapt to this model;

(3) data in addition belong to noise.

Here said intra-office point (inliers) refers to correct data, and abnormity point also referred to as point not in the know (outliers), refers to and departs from normal range, cannot adapt to the data of mathematical model.These abnormity point are normally due to the measuring method of noise, mistake or the false supposition of data produced.

The basic thought of RANSAC algorithm is described below:

1) a required smallest sample of initialization model parameter of consideration is counted n and a sample set P, and the sample number S* of set P is greater than n, then from set P, randomly draws the subset S initialization model M that comprises n sample;

2) by complementary set C _ps=P/S and model M error are less than the sample set of a certain setting threshold t and subset S as the interior point set S* of model M, are called the consistent collection of this model M;

3) if the interior point set number of model M is greater than Q (wherein Q represents that correct model is containing the minimum number of consistent collection), think correct model parameter, and in utilizing, point set S* adopts the methods such as least square to recalculate new model, randomly draw new subset S', and repeat (2) and (3) two steps;

4) after completing certain frequency in sampling, if do not find the algorithm failure of consistent collection, otherwise choose the consistent model M collecting of the maximum obtaining after sampling, judge interior exterior point, algorithm finishes.

Wherein, step 2) in threshold value t be for judgement sample collection whether in model error, generally need to take the mode of manual intervention that suitable threshold value is set, generally get 0.001～0.01;

Randomly drawing sample collection number of times in step 3), is designated as k, and it is determining the precision of model parameter;

$k=\frac{\log (1-p)}{\log (1-e^m)}$

Wherein: m calculates the required smallest match point of restricted model to quantity, and here we select four points to calculating fundamental matrix, so m=4;

The probability that p reaches for our expectation, such as making P=0.98, is also 98% with regard to meaning that we can find the probability of m correct matching double points;

E is correct ratio data, and it is unknown, and it can be along with the operation of program is constantly updated, such as to have chosen for the first time 4 points right, calculate model M, we judge the interior point that meets M, and in these, point accounts for 40% in matching double points set, e just equals 0.4 so, we have chosen again other 4 points for the second time to calculating M, and we judge the interior point that meets M, and in these, point accounts for 70% in matching double points set, e is just updated to 0.7 so, and the rest may be inferred.

Step 6, utilize the coupling angle point that step 5 obtains to obtain model transferring parameter to carrying out geometric transformation.

Image registration is exactly the spatial relation of finding out the same destination object in two images, thereby determines the matching relationship between image, before carrying out image registration, need to select correct transformation model.The registration of image depends on selected geometric transformation model, the method of common acquisition geometric transformation model has rigid body translation, similarity transformation, affined transformation and projective transformation etc., because emulation conversion can realize the multiple conversion such as translation, rotation, yardstick convergent-divergent of image, again without complicated calculated amount, can further guarantee accuracy and the real-time of images match simultaneously.Therefore, in the present embodiment, select affined transformation.

Suppose on image subject to registration coordinate (the u of a bit, v) with benchmark image on any coordinate (x, y) be the angle point pair of a pair of coupling, the representation of being write as homogeneous coordinates is respectively (u', v', w') and (x, y, 1) formula that, image subject to registration transforms to benchmark image through affined transformation is:

$\left[\begin{array}{c}{u}^{\′}\\ v^{\′}\\ w^{\′}\end{array}\right]=\left[\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& 1\end{array}\right]\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]---\left(3\right)$

It in above formula, is the transformation matrix with 8 degree of freedom.

According to corners Matching point set obtained above, adopt stochastic sampling consistance just can estimate this 8 parameters:

Bring these parameters into formula (3), can obtain:

$u=\frac{u^{\′}}{w^{\′}}=\frac{a_{11}x+a_{12}y+a_{13}}{a_{31}x+a_{32}y+1}$

$v=\frac{v^{\′}}{w^{\′}}=\frac{a_{21}x+a_{22}y+a_{23}}{a_{31}x+a_{32}y+1}$

Known according to above formula, have 8 unknown numbers, in order to solve this 8 unknown numbers, at least need the angle point of 4 pairs of registrations.

According to angle point pair obtained above, the transformation model matrix obtaining is:

$M=\left[\begin{array}{ccc}0.0847& -0.3674& -0.0040\\ -0.2635& 0.4805& -0.0023\\ 33.7893& 25.2208& 1.0000\end{array}\right]$

Affine model transformation parameter obtained above, the angle point registration just carrying out for the approximate image after wavelet decomposition obtains, and originally coefficient of dilatation, anglec of rotation during two width image registration equates respectively with coefficient of dilatation, the anglec of rotation that they carry out respectively two width approximate images after wavelet decomposition;

If translational movement during original image registration is (4 Δ t _x, 4 Δ t _y) time, the translational movement that carries out two width approximate images after a wavelet decomposition is (2 Δ t _x, 2 Δ t _y), the translational movement that carries out two width images after secondary wavelet decomposition is (Δ t _x, Δ t _y), therefore, the affine Transform Model matrix of two width images is originally:

Like this, just can obtain the model transferring parameter of two width images.

In sum, these are only preferred embodiment of the present invention, be not intended to limit protection scope of the present invention.Within the spirit and principles in the present invention all, any modification of doing, be equal to replacement, improvement etc., within all should being included in protection scope of the present invention.

Claims (3)

1. the fast image registration method based on wavelet decomposition and Harris Corner Detection, is characterized in that, comprises the following steps:

Step 1, benchmark image and image subject to registration are converted to two-dimentional gray level image from three-dimensional true color image respectively, the gray-scale map of benchmark image and figure subject to registration is designated as respectively to image f ₁with image f ₂;

Step 2, to image f ₁with image f ₂carry out N level wavelet decomposition, obtain respectively an approximate image after the N time wavelet decomposition, wherein, image f ₁the approximate image carrying out after decomposing for N time is called LLNA, image f ₂the approximate image carrying out after decomposing for N time is called LLNB;

Step 3, the approximate image LLNA and the LLNB that obtain after adopting Harris Corner Detection Algorithm to step 2 wavelet decomposition carry out angle point extraction;

The angle point that the angle point that the approximate image LLNA that step 4, employing NCC matching algorithm obtain step 3 extracts and approximate image LLNB extract carries out the thick registration of angle point, obtains coupling angle point pair;

The angle point that step 5, employing RANSAC algorithm obtain step 4, to verifying, is rejected the angle point pair of mistake coupling, obtains correct coupling angle point pair;

Step 6, utilize the coupling angle point that step 5 obtains to obtain model transferring parameter to carrying out geometric transformation.

2. a kind of fast image registration method based on wavelet decomposition and Harris Corner Detection as claimed in claim 1, is characterized in that, the N in described step 2 equals 2.

3. a kind of fast image registration method based on wavelet decomposition and Harris Corner Detection as claimed in claim 1, is characterized in that, the geometric transformation in described step 6 is affined transformation.

Images