CN103679720A - Fast image registration method based on wavelet decomposition and Harris corner detection - Google Patents

Abstract

The invention discloses a fast image registration method based on wavelet decomposition and Harris corner point detection. The size of the image is reduced through wavelet decomposition, thereby reducing the amount of computation and improving the real-time performance of image registration. The process of wavelet decomposition is as follows: A low-pass filtering process can handle noise. In addition, the present invention uses the Harris corner detection algorithm for corner pairing, and at the same time eliminates the corners of mis-matching points, so that the anti-noise and anti-interference capabilities are strong, and the image matching is further improved. the accuracy.

Description

A Fast Image Registration Method Based on Wavelet Decomposition and Harris Corner Detection

technical field

The invention relates to technologies related to digital image processing and model identification, and is suitable for image processing and pattern recognition related fields including navigation, maneuvering target tracking, state monitoring, etc., and specifically relates to a fast image registration method based on wavelet decomposition and Harris corner detection .

Background technique

Image registration is a fundamental problem in image processing. With the development of computer technology, image registration algorithms have developed rapidly, and have been widely used in remote sensing, military, medical, navigation, imaging guidance, transition detection and other fields.

At present, there are two main types of image registration methods: one is the region-based image registration method, which refers to using the gray value relationship between the pixels of two images to determine the parameters of the transformation model. degree information, the registration accuracy is high, but the calculation amount is large and the real-time performance is poor. At present, common region-based image registration algorithms include ratio method, block-based matching method (also known as template-based registration algorithm), grid matching method, etc. This method is suitable for situations where there are only horizontal and vertical translations between images.

The other is a feature-based image registration method, the basic steps of which are as follows: first extract the feature set of the reference image and the image to be registered, then perform feature matching, and finally use the relationship between the registered features to estimate the reference image and The geometric transformation model and its parameter variable values between the images to be registered. This method uses the obvious features in the image instead of using all the information in the image to calculate the transformation between images. It is robust to changes in image grayscale and can be applied to images with more complex geometric transformations. Registration. Common feature-based image registration algorithms include: Harris corner detection algorithm, SUSAN corner detection algorithm, SIFT scale invariant feature transformation algorithm, etc.

At present, the Harris corner detection algorithm is the most widely used image registration method in the feature-based image registration algorithm, but this algorithm is computationally complex, which makes the real-time matching poor. , making its anti-noise and anti-interference ability poor, resulting in poor image matching accuracy.

Contents of the invention

In view of this, the present invention provides a fast image registration method based on wavelet decomposition and Harris corner detection, which reduces the size of the image through wavelet decomposition, thereby reducing the amount of computation and improving the real-time performance of image registration, and the wavelet The process of decomposing is a process of low-pass filtering, which can handle noise. In addition, the present invention uses Harris corner detection algorithm to carry out corner pairing, and also eliminates the corners of wrong matching points so that the anti-noise and anti-interference ability is strong, and further Improved image matching accuracy.

A fast image registration method based on wavelet decomposition and Harris corner detection, comprising the following steps:

Step 1, converting the reference image and the image to be registered from a three-dimensional true color image to a two-dimensional grayscale image respectively, and recording the grayscale images of the reference image and the image to be registered as image f ₁ and image f ₂ respectively;

Step 2: Perform N-level wavelet decomposition on image _f1 and image _f2 , and obtain an approximate image of the Nth wavelet decomposition respectively, wherein the approximate image after N-time decomposition of image _f1 is called LLNA, and image _f2 The approximate image after N times of decomposition is called LLNB;

Step 3, using the Harris corner detection algorithm to extract corners from the approximate images LLNA and LLNB obtained after step 2 wavelet decomposition;

Step 4, using the NCC matching algorithm to roughly register the corner points extracted from the approximate image LLNA obtained in step 3 with the corner points extracted from the approximate image LLNB to obtain a pair of matching corner points;

Step 5. Use the RANSAC algorithm to verify the corner point pairs obtained in step 4, and eliminate the wrongly matched corner point pairs to obtain the correct matching corner point pairs;

Step 6: Using the matching corner point pairs obtained in Step 5 to perform geometric transformation to obtain model transformation parameters.

Preferably, N in step 2 is equal to 2.

Preferably, the geometric transformation in step six is an affine transformation.

Beneficial effect:

1) In the present invention, firstly, after N-level wavelet decomposition, the size of the image is reduced to 1/2 ^N of the original, so image registration is performed based on the image after wavelet decomposition, which greatly reduces the computational complexity and improves image registration real-time matching.

Secondly, compared with the traditional image registration algorithm based on Harris corner points, the present invention introduces two steps of wavelet decomposition and elimination of mismatching points. The wavelet decomposition process is a low-pass filtering process. Decomposition, in fact, is to denoise the image, which can process unimportant details and keep only important basic information, thereby reducing noise interference and improving image matching accuracy. The RANSAC algorithm is used to verify the corner point pairs, and the wrongly matched corner point pairs are eliminated, which improves the anti-noise and anti-interference ability of the image registration method, and further improves the matching accuracy of the image.

2) The embodiment of the present invention performs two-level wavelet decomposition on the image, which can compress the image without increasing the complexity of calculation, further ensuring the accuracy and real-time performance of image matching.

3) The embodiment of the present invention utilizes the obtained matching corner point pairs to obtain a geometric transformation model using affine transformation. This geometric transformation can realize various transformations such as image translation, rotation, etc., and at the same time, there is no complicated calculation amount, which can further ensure the accuracy of image matching. accuracy and timeliness.

Description of drawings

Fig. 1 is a flowchart of the present invention;

Fig. 2 is the approximate image obtained by wavelet decomposition;

Fig. 3 is the corner point extracted on the approximate image of the reference image and the image to be registered;

Figure 4 is a pair of matching corner points obtained by using the NCC matching algorithm pair;

Figure 5 shows the use of the RANSAC algorithm to eliminate incorrectly matched corner pairs.

Detailed ways

The present invention will be described in detail below with reference to the accompanying drawings and examples.

The present invention provides a kind of fast image registration method based on wavelet decomposition and Harris corner point detection, this method is carried out under computer hardware environment, Windows2000/XP; Matlab or C language or C++ etc. any kind of language environment software all can realize , present embodiment adopts matlab language environment, and flow chart as shown in Figure 1, specifically comprises the following steps:

Step 1. Input two images F ₁ and F ₂ in matlab, where F ₁ is the reference image and F ₂ is the image to be registered. Use the function rgb2gray() in matlab to convert the two images from the three-dimensional true color The image is converted into a two-dimensional grayscale image, and the grayscale images of the reference image and the image to be registered are respectively recorded as image f ₁ and image f ₂ .

Step 2: Perform N-level wavelet decomposition on image _f1 and image _f2 , and obtain an approximate image of the Nth wavelet decomposition respectively, wherein the approximate image after N-time decomposition of image _f1 is called LLNA, and image _f2 The approximate image after N times of decomposition is called LLNB.

For example, when performing first-level wavelet decomposition on the image f ₁ , first use the low-pass filter (L) and high-pass filter (H) to perform wavelet decomposition on the row direction of the image to obtain the high-frequency component and low-frequency component data; then the obtained The column direction of high-frequency component and low-frequency component data is filtered by low-pass filter (L) and high-pass filter (H) respectively, so as to obtain the low-frequency part of the image, which is also called the first-level approximation information LL1A of the image, image edge The detail information HL1A in the horizontal direction, the detail information LH1A in the vertical direction of the image, and the detail information HH11 in the diagonal direction of the image.

The second-level wavelet decomposition of the image is to perform the second-level wavelet decomposition on the obtained first-level decomposition approximation information LL1A of the image, and obtain the second-level approximation information LL2A of the image, the detail information HL2A of the image along the horizontal direction, and the detail information LH2A of the image along the vertical direction , is the detail information HH2A of the image along the diagonal direction.

When N-level wavelet decomposition is performed, the N-level approximate information of the image is obtained; the detailed information of the image along the horizontal direction HLNA; the detailed information of the image along the vertical direction LHNA; the detailed information of the image along the diagonal direction HHNA.

This decomposition is a lossless zoom, and the approximate components are smooth, so wavelet decomposition has the advantages of compressing images and anti-noise. The choice of image decomposition times is very important. The more layers of decomposition, the more computational complexity for image registration. The lower the , the higher the matching efficiency. However, if the number of decompositions is too large, a large amount of image information will be lost. Therefore, when decomposing an image, selecting an appropriate number of decomposition layers not only simplifies the image information, but also retains the important information of the image. This embodiment performs two-level wavelet decomposition on the image, which can compress the image without increasing the complexity of calculation. , which further ensures the accuracy and real-time performance of image matching.

Perform two-level wavelet decomposition on image f ₁ and image f ₂ to obtain approximate images after wavelet decomposition respectively. Among them, the approximate image of image f ₁ after N times of decomposition is called LL2A, and the approximate image of image f ₂ after N times of decomposition is called LL2A. The image is called LL2B, as shown in Figure 2, with LL2A on the left and LL2B on the right.

Step 3: using the Harris corner detection algorithm to extract corner information from the approximate images LL2A and LL2B obtained after the wavelet decomposition in step 2, respectively.

The Harris corner detection algorithm (Qu Xiwen. An improved Harris corner detection method. Electromechanical Technology, 2012, 40-42) works as follows:

First, move the image window w to be detected by a small displacement in any direction, assuming that the coordinates of the target pixel in the image window w are (x, y), and the displacements in the x and y directions are u and v respectively, then set The grayscale variation of a point (x, y) in a (u, v) square window is defined as:

$\begin{array}{c}\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>{<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ <span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>{<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; o</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; u</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; v</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}$

Among them: w(x, y) is the window function, the Gaussian window is selected here to improve the anti-interference ability; I(x, y), I(x+u, y+v) is the gray function of the target pixel; o( u ² +v ² ) is the infinitesimal amount of displacement; I _x and I _y are the first-order gray gradient of the target pixel, where,

$\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&pi;</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; e</span>}}^{<span\; class="notranslate">\; -</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}$

${\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; \&PartialD;</span>\mathrm{<span\; class="notranslate">\; I</span>}}{<span\; class="notranslate">\; \&PartialD;</span>\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; \&CircleTimes;</span><span\; class="notranslate">\; (</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1,0,1</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; \&PartialD;</span>\mathrm{<span\; class="notranslate">\; I</span>}}{<span\; class="notranslate">\; \&PartialD;</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; \&CircleTimes;</span><span\; class="notranslate">\; (</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1,0,1</span>}<span\; class="notranslate">\; )</span>$

Written in matrix form:

$\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&CircleTimes;</span>\left[\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}& {\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}\\ {\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}& {\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right]$

but $\mathrm{E.}(u,v)=\left[\begin{array}{cc}u& v\end{array}\right]m\left[\begin{array}{c}u\\ v\end{array}\right]$

Among them: M is the autocorrelation matrix of the target pixel; Let λ ₁ and λ ₂ be two eigenvalues of M, the size of the value of λ ₁ and λ ₂ determines that the target pixel is a corner point, an edge or a flat area:

(1) Corner point: the values of λ ₁ and λ ₂ are relatively large;

(2) Edge: the values of λ ₁ and λ ₂ , one big and one small;

(3) Flat region: Both λ ₁ and λ ₂ are small.

In order to avoid the eigenvalue decomposition of the autocorrelation matrix M, the corner response function CRF is defined as:

CRF＝det(M)-k*trace ² (M)

Among them, det(M) is the determinant of matrix M; trace(M) is the trace of matrix M (the sum of matrix diagonal elements); k is an empirical value, usually 0.04. (Mao Yanming, Lan Meihui, Wang Yunqiong, Feng Qiaosheng. An Improved Harris-Based Corner Detection Method, Computer Technology and Development, 2009.05)

When the present embodiment adopts the Harris algorithm to extract corner points for two images, the criterion for corner point judgment is:

When the value of the corner response function CRF is greater than a preset threshold, it is a candidate corner, otherwise it is not a corner, and the threshold is determined according to the number of required corners.

Therefore, to ensure that the number of corner points is large enough, it is necessary to set an appropriate threshold. According to experience, when the number of corner points is around 200, it can ensure that there are enough corner point pairs for image registration. Therefore, through multiple experiments , the corner threshold in the image of this embodiment is set to 0.000149044.

The Harris corner points are extracted from the two images in Figure 2, and the extracted corner points are marked on the images to obtain the corner point images shown in Figure 3.

The Harris operator is an effective point feature extraction operator, which has the following advantages:

① Simple calculation: only the first-order difference of gray level and filtering are used in the Harris operator, and the operation is simple;

② The extracted point features are uniform and reasonable;

③ Stability: The calculation formula of the Harris operator only involves the first-order derivative. Therefore, it is not sensitive to image rotation, gray level changes, noise effects and viewpoint changes, and is a relatively stable point feature extraction operator.

Step 4: Use the NCC matching algorithm to roughly register the corner points extracted from the approximate image LL2A obtained in step 3 with the corner points extracted from the approximate image LL2B to obtain matching corner point pairs.

The NCC matching algorithm is a classic matching algorithm. The method is to construct an M×N template centered on the extracted corner points in the reference image and the image to be registered, and obtain the template images respectively (corresponding to the reference image with the corner points as The M×N template image constructed at the center) and the background image (corresponding to the M×N template image constructed around the corner of the image to be registered), the degree of matching is determined by calculating the cross-correlation value of the template image and the background image, and the cross-correlation The maximum value of the correlation determines the position and similarity of the template image in the background image, and then obtains the correlation degree of the corner points on the reference image and the image to be registered.

In actual matching applications, the similarity between the template image and the background image is measured by a metric function, and this embodiment uses a normalized product correlation matching metric function C to measure, which is defined as

$\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; ]</span>}{\sqrt{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}\sqrt{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

为待配准图像上某一点

处的灰度值。

分别为基准图像和待配准图像上M×N的模板上灰度均值，其值为：Among them, f( _xi ,y _j ) is the gray value at a certain point ( _xi ,y _j ) on the reference image,

A point on the image to be registered

gray value at .

Respectively, the mean value of the gray level on the M×N template on the reference image and the image to be registered, and its value is:

$\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; no</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; no</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

Among them: m and n are the rows and columns of the image, respectively.

C is the cross-correlation coefficient. When C=1, it means that the reference image and the corresponding template on the image to be registered are completely related. When C=0, it means that they are not related. In practical applications, due to the influence of light and noise on the image, it is difficult to make the image completely correlated, so there is a certain error in the correlation here. In the experiment, when C is greater than 0.8, the two templates are considered to match, otherwise they do not match. (Chen Weibing. Comparison of Matching Performance of Several Image Similarity Measures [J]. Computer Applications, 2010(1)).

In this embodiment, the above algorithm is used to perform rough registration of corner points between the corner point information extracted from the approximate image LL2A in step 3 and the corner point information extracted from the approximate image LL2B to obtain matching corner point pairs, as shown in FIG. 4 .

Step 5. Use the RANSAC algorithm to verify the corner point pairs obtained in step 4, and remove the wrongly matched corner point pairs. As shown in Figure 5, the correct matching corner point pairs are obtained, which improves the anti-noise and anti-noise of the image registration method. The ability to interfere further improves the matching accuracy of images.

It can be seen from Figure 3 that the registration of corner point pairs is not accurate, there are correct corner point pairs, and there are also incorrectly matched corner point pairs. Therefore, in order to improve the accuracy of corner point pair registration, it is necessary to perform mismatch of elimination.

RANSAC is the abbreviation of "Random Sample Consensus (Random Sample Consensus)", which was first proposed by Fischler and Bolles in 1981. It iteratively estimates the parameters of the mathematical model from a set of data sets containing abnormal data to obtain valid data. algorithm. RANSAC generally uses relatively few points to estimate the model, and then uses the remaining points to verify the model, thus reducing the impact of abnormal data on model parameter estimation when there are serious error points.

The basic assumptions of the RANSAC algorithm are:

(1) The data consists of "intra-office points";

(2) "Outlier points" are data that cannot adapt to the model;

(3) Data other than this is noise.

The inliers mentioned here refer to the correct data, and the abnormal points are also called outliers, which refer to the data that deviate from the normal range and cannot adapt to the mathematical model. These outliers are usually due to noise, wrong measurements, or wrong assumptions about the data.

The basic idea of RANSAC algorithm is described as follows:

1) Consider a minimum number of samples n required to initialize the model parameters and a sample set P, the number of samples S* of the set P is greater than n, and then randomly select a subset S containing n samples from the set P to initialize the model M;

2) The sample set and subset S whose error between the residual set C _P s=P/S and the model M is less than a certain threshold t is taken as the internal point set S* of the model M, which is called the consistent set of the model M;

3) If the number of interior point sets of the model M is greater than Q (where Q represents the minimum number of consistent sets contained in the correct model), it is considered to be the correct model parameter, and the least square method is used to use the interior point set S* Recalculate the new model, randomly select a new subset S', and repeat (2) and (3) two steps;

4) After completing a certain number of sampling times, if no consistent set is found, the algorithm fails. Otherwise, the model M of the largest consistent set obtained after sampling is selected to judge the internal and external points, and the algorithm ends.

Among them, the threshold t in step 2) is used to judge whether the sample set is within the model error. Generally, manual intervention is required to set an appropriate threshold, which is generally 0.001 to 0.01;

The number of randomly drawn sample sets in step 3), denoted as k, determines the accuracy of the model parameters;

$\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; e</span>}}^{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; )</span>}$

Among them: m is the minimum number of matching point pairs required to calculate the constraint model, here we choose four point pairs to calculate the basic matrix, so m=4;

p is the probability we expect to achieve, for example, let P=0.98, which means that the probability that we can find m correct matching point pairs is 98%;

e is the correct data ratio, it is unknown, it will be updated continuously with the operation of the program, for example, after selecting 4 point pairs for the first time and calculating the model M, we judge the interior points that satisfy M, these interior points Points account for 40% of the set of matching point pairs, then e is equal to 0.4, and we select another 4 point pairs to calculate M for the second time, and we judge the inliers that satisfy M, and these inliers are in the matching point pair set accounted for 70%, then e is updated to 0.7, and so on.

Step 6: Using the matching corner point pairs obtained in Step 5 to perform geometric transformation to obtain model transformation parameters.

Image registration is to find out the spatial position relationship of the same target object in two images, so as to determine the matching relationship between images. Before image registration, it is necessary to select the correct transformation model. Image registration depends on the selected geometric transformation model. Common methods for obtaining geometric transformation models include rigid body transformation, similarity transformation, affine transformation, and projective transformation. Since the simulation transformation can realize translation, rotation, and scaling of images, etc. A variety of transformations without complicated calculations can further ensure the accuracy and real-time performance of image matching. Therefore, affine transformation is selected in this embodiment.

Assuming that the coordinates (u, v) of a point on the image to be registered and the coordinates (x, y) of a point on the reference image are a pair of matching corner points, the expressions written as homogeneous coordinates are (u', v' ,w') and (x,y,1), the formula for transforming the image to be registered to the reference image through affine transformation is:

$\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; u</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\\ {\mathrm{<span\; class="notranslate">\; v</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\\ {\mathrm{<span\; class="notranslate">\; w</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 13</span>}}\\ {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; one</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; two</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; three</span>}}\\ {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 31</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 32</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

In the above formula is a transformation matrix with 8 degrees of freedom.

According to the set of corner point matching points obtained above, these 8 parameters can be estimated by using random sampling consistency:

Putting these parameters into formula (3), we can get:

$\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; u</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}{{\mathrm{<span\; class="notranslate">\; w</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 13</span>}}}{{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 31</span>}}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 32</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}$

$\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; v</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}{{\mathrm{<span\; class="notranslate">\; w</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; one</span>}}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; two</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; three</span>}}}{{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 31</span>}}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 32</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}$

According to the above formula, there are 8 unknowns in total. In order to solve these 8 unknowns, at least 4 pairs of registered corner points are needed.

According to the corner point pairs obtained above, the obtained transformation model matrix is:

$\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; 0.0847</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.3674</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.0040</span>}\\ <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.2635</span>}& \mathrm{<span\; class="notranslate">\; 0.4805</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.0023</span>}\\ \mathrm{<span\; class="notranslate">\; 33.7893</span>}& \mathrm{<span\; class="notranslate">\; 25.2208</span>}& \mathrm{<span\; class="notranslate">\; 1.0000</span>}\end{array}\right]$

The affine model transformation parameters obtained above are only obtained by the corner registration of the approximate images after wavelet decomposition. The expansion coefficient and rotation angle of the original two images when they are registered are the same as the two approximate images after wavelet decomposition. The expansion coefficient and rotation angle are equal respectively;

If the translation of the original image registration is (4Δt _x ,4Δt _y ), the translation of the two approximate images after the first wavelet decomposition is (2Δt _x ,2Δt _y ), and the two images after the second wavelet decomposition The translation amount of is (Δt _x ,Δt _y ), therefore, the affine transformation model matrix of the original two images is:

In this way, the model transformation parameters of the two images can be obtained.

To sum up, the above are only preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (3)

1. A fast image registration method based on wavelet decomposition and Harris corner detection, is characterized in that, comprises the following steps: Step 1, converting the reference image and the image to be registered from a three-dimensional true color image to a two-dimensional grayscale image respectively, and recording the grayscale images of the reference image and the image to be registered as image f ₁ and image f ₂ respectively; Step 2, perform N-level wavelet decomposition on image _f1 and image _f2 , and obtain an approximate image after Nth wavelet decomposition respectively, wherein, the approximate image after N-time decomposition of image _f1 is called LLNA, and image _f2 The approximate image after N times of decomposition is called LLNB; Step 3, using the Harris corner detection algorithm to extract corners from the approximate images LLNA and LLNB obtained after step 2 wavelet decomposition; Step 4, using the NCC matching algorithm to roughly register the corner points extracted from the approximate image LLNA obtained in step 3 with the corner points extracted from the approximate image LLNB to obtain a pair of matching corner points; Step 5. Use the RANSAC algorithm to verify the corner point pairs obtained in step 4, and eliminate incorrectly matched corner point pairs to obtain correct matching corner point pairs; Step 6: Using the matching corner point pairs obtained in Step 5 to perform geometric transformation to obtain model transformation parameters. 2. A kind of fast image registration method based on wavelet decomposition and Harris corner detection as claimed in claim 1, it is characterized in that, N in the said step 2 is equal to 2. 3. A kind of fast image registration method based on wavelet decomposition and Harris corner detection as claimed in claim 1, is characterized in that, the geometric transformation in the described step 6 is an affine transformation.

Images