CN105869168A - Multi-source remote sensing image shape registering method based on polynomial fitting - Google Patents

Abstract

The invention provides a multi-source remote sensing image shape registering method based on polynomial fitting. According to the method, first of all, edge contour features of remote sensing images are extracted, main contour edges are reserved, feature points are extracted through a feature point extraction algorithm based on polynomial fitting, and principle directions of the feature points are determined; based on this, through improved shape content descriptors, minimizing coupling cost between the feature points so as to complete coarse registering; and finally, removing wrong coupling so as to finish fine registering. The method provided by the invention can realize automatic registering of multi-source remote sensing images of different landforms and has the advantages of high registering precision, fast calculation speed, high robustness, good applicability and the like.

Description

Multi-source remote sensing image shape registration method based on polynomial fitting

Technical Field

The invention relates to a remote sensing image processing technology, in particular to a multi-source remote sensing image shape registration method based on polynomial fitting.

Background

The remote sensing technology is widely applied to the fields of geography and the like through development of more than half a century, opens up a new way for exploring resources, protecting environment, monitoring disasters, analyzing global changes and the like, and becomes an effective means for people to observe and analyze the global environment. Since remote sensing images obtained by different remote sensing satellites have great difference in resolution, visual angle, scale and the like, the registration of the remote sensing images to eliminate the difference as much as possible is the basis of subsequent analysis and application. Image registration can be divided into a grayscale-based registration method and a feature-based registration method according to the difference in image information utilized in the registration process. The registration method based on gray level mainly comprises a mutual information method, a phase correlation method and the like, wherein the mutual information method is commonly used for registration of multi-mode medical images due to high registration precision and is also commonly used for registration of remote sensing images in recent years, but the development and application of the registration method in the field of remote sensing image registration are limited by the calculation speed; the phase correlation method is easy to implement by hardware, but requires strict linear relation between images, and is very sensitive to noise and has large limitation. The feature-based registration method does not depend on the gray characteristic of the image, converts the analysis of the whole image into the analysis of certain feature of the image, greatly reduces the calculated amount, and is the most used remote sensing image registration method at present. However, when the multi-source remote sensing images obtained by different types of sensors are registered, because more feature-based remote sensing image registration algorithms are used at present, all the obvious features in the images need to be extracted, such as Harris, SIFT feature points or linear features, and the like, the images show large gray scale and noise difference due to different imaging mechanisms and pixel representation forms, or the matching accuracy is low, even the matching fails.

Disclosure of Invention

The invention aims to provide a multi-source remote sensing image shape registration method based on polynomial fitting, and solves the problems of low matching precision, time-consuming operation and applicability limitation when a traditional method is used for registering a multi-source remote sensing image. The method comprises the following steps:

step 1, inputting two multi-source remote sensing images as a reference image and an image to be registered;

step 2, extracting edge contour features of the two images, and eliminating useless edge features of a dense area;

step 3, extracting feature points and determining the main direction of the feature points by a feature point extraction algorithm based on polynomial fitting aiming at pixel points on the contours extracted by the two images;

step 4, obtaining an improved shape content descriptor of each feature point;

step 5, traversing feature points in the image to be registered for each feature point in the reference image based on the improved shape content descriptor of the pixel point, and acquiring the feature point in the image to be registered with the minimum matching cost as a matching point pair;

step 6, removing the mismatching and finishing the fine registration;

step 7, estimating a transformation matrix between the two images according to the matching point pairs, and restoring the image to be registered through the transformation matrix to finish registration;

the characteristic point extraction algorithm based on polynomial fitting in step 3 is as follows:

step 3.1, aiming at each pixel point on the extracted remote sensing image contour boundary, carrying out cubic polynomial fitting on the pixel point and the surrounding boundary points to estimate the coefficient of a fitting curve;

step 3.2, calculating the fitting error of the fitting curve and the curvature of the pixel point;

step 3.3, discarding pixel points with large fitting errors, and screening out points with large curvature from the rest pixel points as characteristic points;

step 3.4, adopting the tangent direction of the curve at the pixel point as the main direction of the characteristic point;

wherein the improved shape content descriptor described in step 4 is a circular template and is assigned a direction that is the same as the principal direction of the feature point.

Compared with the prior art, the invention has the following advantages: (1) the method provides a novel characteristic point extraction algorithm based on polynomial fitting, greatly reduces the calculated amount, and enables a classical shape content descriptor to be applied to an actual complex remote sensing image; (2) the method determines the main direction for each characteristic point by utilizing a polynomial fitting method, improves the classical shape content descriptor to ensure that the classical shape content descriptor has rotation invariance, thereby registering multi-source remote sensing images of different terrains and improving the applicability; (3) the method of the invention carries out registration from coarse to fine, removes mismatching by RANSAC algorithm on the basis of coarse registration, and improves registration accuracy.

The present invention is described in further detail below with reference to the attached drawing figures.

Drawings

FIG. 1 is a flow chart of the method of the present invention.

Fig. 2 is a schematic diagram of a shape content descriptor circular template, wherein fig. 2(a) is a conventional shape content descriptor circular template, and fig. 2(b) is a shape content descriptor circular template with rotation invariance improved in the present method.

FIG. 3 is a diagram of a multi-source remote sensing image to be registered and a registration effect obtained by the method of the present invention. In fig. 3, (a), (d), (g) show the satellite images captured by the satellite detector with a resolution of 1024 × 1024, and (b), (e), (h) show the images in the Google map database with a resolution of 696 × 696. (c) (f) (i) is a registration effect graph obtained by the method of the invention.

Fig. 4 is a graph of the registration effect obtained using the conventional SIFT algorithm.

Detailed Description

With reference to fig. 1, the multi-source remote sensing image shape registration method based on polynomial fitting provided by the invention firstly extracts edge contour features of a reference image and an image to be registered respectively, extracts feature points and determines the main directions of the feature points through a feature point extraction algorithm based on polynomial fitting, and completes coarse registration by utilizing an improved shape content descriptor and minimizing the matching cost between the feature points. Finally, removing the mismatching and finishing the fine registration. And on the basis, a homography transformation matrix H between the reference image and the image to be registered is estimated according to the matching point pairs, and the image to be registered is restored through the transformation matrix to complete registration.

The specific implementation steps of the multi-source remote sensing image shape registration method based on polynomial fitting are as follows:

step 1, inputting two multi-source remote sensing images as a reference image and an image to be registered;

step 2, extracting edge contour features of the two images by adopting a Canny operator, eliminating useless edge features of a dense area through morphological transformation, and leaving main contours of the images;

step 3, extracting feature points and determining the main direction of the feature points by a feature point extraction algorithm based on polynomial fitting aiming at pixel points on the contours extracted by the two images;

step 4, obtaining an improved shape content descriptor of each feature point;

step 5, traversing feature points in the image to be registered for each feature point in the reference image based on the improved shape content descriptor of the pixel point, and acquiring the feature point in the image to be registered with the minimum matching cost as a matching point pair;

step 6, removing the mismatching and finishing the fine registration;

and 7, estimating a transformation matrix between the two images according to the matching point pairs, and restoring the image to be registered through the transformation matrix to finish registration.

Specifically, in step 3, main contour features of the reference image and the image to be registered are respectively extracted, and for each pixel point on the extracted contour boundary of the remote sensing image, a cubic polynomial fitting is performed on the pixel point and surrounding boundary points, so that feature points are extracted and the main direction is determined. The operation is divided into the following four steps:

and 3.1, performing polynomial fitting on the pixel points on the outline.

Let p be_iIs a point on the image contour boundary whose coordinates can be expressed as (x)_i,y_i) Extracting a point p on the boundary of the image contour_iThe left and right partial pixel points can be expressed as p_i-m,…,p_i-1,p_i,p_i+1,…,p_i+nThe pixel point correspondence coordinate can be expressed as (x)_i-m,y_i-m),…,(x_i,y_i),…,(x_i+n,y_i+n). Assuming the fitted curve is a cubic polynomial, the fitted curve is expressed as:

y＝ax³+bx²+cx+d (1)

in the formula, a, b, c and d represent coefficients of a fitting curve, and x and y are coordinates of pixel points for fitting.

Since the fitted cubic polynomial curve must be guaranteed to pass through the current point P_i. In the method, a Lagrange multiplier method is adopted to convert the Lagrange multiplier method into a multivariate function extremum solving problem, so that the coefficient of the fitting curve is estimated.

With the coefficients a, b, c, d of the fitted curve as arguments, the problem of estimating the coefficients of the fitted curve can be translated into solving the function z ═ f (a, b, c, d) under additional conditionsThe following extreme value problem. f (a, b, c, d) andthe concrete formula is as follows:

$f(a,b,c,d)=\underset{u=i-m}{\overset{i+n}{\Σ}}{({\mathrm{ax}}_u^3+{\mathrm{bx}}_u^2+{\mathrm{cx}}_u+d-y_u)}^2---\left(2\right)$

lambda is a coefficient, u is from i-m to i + n, namely the Lagrange function formula of the selected pixel point is as follows:

the coefficients a, b, c, d and the parameter λ can be calculated as follows:

[a,b,c,d,λ]＝(X^TX)^-1X^TY (5)

wherein,

step 3.2, further calculating the fitting error of the fitting curve and the point P_iThe curvature of the point P is determined_iWhether or not to be a feature point. Fitting errors D (i) and points P_iThe curvature k (i) is specifically expressed as:

$D\left(i\right)=\frac{1}{m+n+1}\underset{u=i-m}{\overset{i+n}{\Σ}}{({\mathrm{ax}}_u^3+{\mathrm{bx}}_u^2+{\mathrm{cx}}_u+d-y_u)}^2---\left(8\right)$

$k\left(i\right)=\frac{\left|y^{\′\′}\right|}{{\left(1+y^{\′2}\right)}^{\frac{3}{2}}}---\left(9\right)$

step 3.3, the smaller the fitting error D (i), the smaller the curve and the current point P are represented_iThe surrounding real contour boundary has higher consistency. For contoured shape features, the greater the curvature of the contour curve, i.e., the greater the curvature, the more pronounced the feature. Therefore, in order to extract the feature points with more obvious features, the points with large fitting errors D (i) are discarded, and the points with large curvature are screened out from the remaining points with small fitting errors to be used as the feature points. Namely, a first threshold value is set, and if the fitting error is greater than the first threshold value, the pixel point is abandoned. And setting a second threshold, and if the curvature of the pixel point is larger than the second threshold after discarding the point with large fitting error, taking the pixel point as the characteristic point. The first threshold and the second threshold are valued according to the number of the characteristic points to be acquired, the number of the characteristic points is large, the first threshold is large, and the second threshold is small.

And 3.4, calculating the main direction of the characteristic points. The method adopts the tangential direction as the main direction of the characteristic point, and the specific formula is as follows:

$Q=3{\mathrm{ax}}_i^2+2{\mathrm{bx}}_i+c---\left(10\right)$

in step 5, an improved shape content descriptor is calculated for the extracted feature points and a coarse registration is achieved by minimizing the matching cost. In conjunction with fig. 2, fig. 2(a) is a classical shape content descriptor circular template. Suppose there are n points on the image contour boundary, for one of which p_iThe structural relationship between the sampling point and other n-1 sampling points on the boundary can be used as a histogram h_iTo show that:

h_i(k)＝#{q≠p_i；(q-p_i)∈bin(k)} (11)

wherein h is_i(k) I.e. as point p_iIn order to make the descriptor more robust, the shape content descriptor is aligned to the distance reference point p using a log-polar coordinate system as shown in fig. 2(a)_iThe closer the sampling point the more sensitive the change. In the above formula, bin represents segmentation in polar coordinates, k represents the number of segments, and in the method, 5bin is used in the lgr direction and 12bin is used in the theta direction to form a 60-dimensional feature histogram.

The method improves the original circular template of the shape content descriptor, and in the process of calculating the shape content descriptor for the characteristic points, the main direction, namely the tangential direction of the characteristic points is taken as the positive axis of the circular template, so that the circular template is rotated, and the specific formula of the rotation angle alpha is as follows:

α＝arctan Q (12)

in order to determine the corresponding relationship between the boundary characteristic points of two images to be registered, C is adopted_ij≡C(p_i,q_j) The matching cost between the two characteristic points is represented by the following specific formula:

$C_{ij}=\frac{1}{2}\underset{k=1}{\overset{K}{\Σ}}\frac{{\[h_i\left(k\right)-h_j\left(k\right)\]}^2}{h_i\left(k\right)+h_j\left(k\right)}---\left(13\right)$

wherein h is_i(k) And h_j(k) Respectively represent points p_iAnd point q_jK-bin of (a) normalizes the histogram. Calculating the matching cost C of all characteristic point pairs in the two images_ijAnd obtaining the one-to-one corresponding relation of the feature points by minimizing the matching cost.

And 6, removing mismatching by using a RANSAC algorithm for the matching point pair obtained by the coarse registration to finish the fine registration.

The effect of the present invention can be further illustrated by the following simulation results:

fig. 3(a) (d) (g) shows the satellite images from the satellite detector with a resolution of 1024 × 1024, and (b) (e) (h) shows the images in the Google map database with a resolution of 696 × 696. The registration result obtained by the multi-source remote sensing image shape registration method based on polynomial fitting is shown as (c) (f) (i). The locations of the map are from top to bottom, respectively, the Albarya coast, the latitude and longitude of the center point are respectively 40 degrees, 19.60 degrees, north and 19 degrees, 18 degrees, 57.43 degrees; the israzazza lake with a center point latitude and longitude of 32 deg. 51'1.66 "north, 43 deg. 20'15.47" east and indian rhesus (near alaabad), 25 deg. 20'23.27 "north, 82 deg. 14'49.36" east, respectively.

In order to illustrate the advantages of the method in registration accuracy, applicability and calculation speed, the method and the traditional SIFT registration method are used for respectively carrying out registration simulation experiments on the same multi-source remote sensing images. The registration result of the SIFT registration method is shown in fig. 4. In order to further quantitatively evaluate the performance of the remote sensing image registration method, the registration effect is comprehensively evaluated by cross-correlation coefficients, root mean square errors and algorithm operation time. The precision evaluation results are shown in the following table, which shows the registration effect and the operation time comparison of the methods when the method and the SIFT registration method are used for simulation experiments. The cross-correlation coefficient is in the range of (0, 1), and the larger the value is, the larger the correlation between the registered image and the target image is, and the better the registration effect is. The root mean square error measures the distance dispersion between the registered image and the target image, and the smaller the root mean square value is, the smaller the distance between the registered image and the target image is, namely the better the registration effect is. Compared with the SIFT algorithm, the method provided by the invention has the advantages that the adaptability is wider, the obtained registration precision is higher, the calculation speed is increased by 2 times compared with the registration speed of the SIFT algorithm, and the method can be applied to the registration of the multi-source remote sensing image.

Claims (7)

1. A multi-source remote sensing image shape registration method based on polynomial fitting is characterized by comprising the following steps:

step 1, inputting two multi-source remote sensing images as a reference image and an image to be registered,

step 2, extracting edge contour characteristics of the two images, eliminating useless edge characteristics of a dense area,

step 3, extracting characteristic points and determining the main direction of the characteristic points by a characteristic point extraction algorithm based on polynomial fitting aiming at pixel points on the contours extracted by the two images,

step 4, obtaining the improved shape content descriptor of each feature point,

step 5, traversing the feature points in the image to be registered for each feature point in the reference image based on the improved shape content descriptor of the pixel point, acquiring the feature point in the image to be registered with the minimum matching cost as a matching point pair,

step 6, removing the mismatching, finishing the fine registration,

step 7, estimating a transformation matrix between the two images according to the matching point pairs, and restoring the image to be registered through the transformation matrix to finish registration;

the characteristic point extraction algorithm based on polynomial fitting in step 3 is as follows:

step 3.1, aiming at each pixel point on the extracted remote sensing image contour boundary, carrying out cubic polynomial fitting on the pixel point and the surrounding boundary points to estimate the coefficient of a fitting curve;

step 3.2, calculating the fitting error of the fitting curve and the curvature of the pixel point;

step 3.3, discarding pixel points with large fitting errors, and screening out points with large curvature from the rest pixel points as characteristic points;

step 3.4, adopting the tangent direction of the curve at the pixel point as the main direction of the characteristic point;

wherein the improved shape content descriptor described in step 4 is a circular template and is assigned a direction that is the same as the principal direction of the feature point.

2. The method according to claim 1, characterized in that the coefficients of the fitted curve are estimated in step 3.1 using the Lagrangian multiplier method.

3. The method of claim 1 wherein a first threshold is set in step 3.3, and if the fitting error is greater than the first threshold, the pixel is discarded.

4. The method according to claim 3, wherein a second threshold is set in step 3.3, and if the curvature of the pixel point is larger than the second threshold after discarding the point with large fitting error, the pixel point is the feature point.

5. The method according to claim 1, wherein the improved shape content descriptor in step 4 is rotated by an angle α (arctan Q), where Q is the main direction of the feature point.

6. The method according to claim 1, wherein the matching cost between two feature points in step 5 is obtained by using the following formula

$C_{ij}=\frac{1}{2}\underset{k=1}{\overset{K}{\Σ}}\frac{{\[h_i\left(k\right)-h_j\left(k\right)\]}^2}{h_i\left(k\right)+h_j\left(k\right)}$

Wherein i is the index value of the feature point in the reference image, j is the index value of the feature point in the image to be registered, K is the number of bins in the improved shape content descriptor histogram, h_i(k) And h_j(k) Respectively represent characteristic points p_iAnd a feature point q_jK-bin of (a) normalizes the histogram.

7. The method of claim 1, wherein the RANSAC algorithm is used to remove the mismatch in step 6.