CN104167003A - Method for fast registering remote-sensing image - Google Patents

Abstract

The invention relates to a method for fast registering a remote-sensing image. The method comprises the following steps that S1, ORB feature points are extracted from the remote-sensing image to be registered and a reference remote-sensing image; S2, initial matching is carried out on the extracted ORB feature points, and wrongly-matched feature points are removed from the initially-matched feature points; S3, parameter solving is carried out on the remote-sensing image to be registered; S4, resampling is carried out on the remote-sensing image to be registered, and image registration is completed. The method is beneficial to improving the speed and precision of image registration.

Description

A Fast Registration Method for Remote Sensing Images

technical field

The invention relates to the technical field of image processing, in particular to a method for quickly registering two or more remote sensing images at different times, different viewing angles and different sensors.

Background technique

Image registration is an important topic in image processing technology. At present, image registration has been widely used in various fields, such as remote sensing image analysis, medical image analysis, image fusion, machine vision and other fields.

As a basic problem of remote sensing image processing, image registration is the premise and foundation of integrated analysis and application of remote sensing images such as image data fusion and dynamic change monitoring. In order to timely and accurately monitor changes in the measured area, it is necessary to register the obtained remote sensing images in real time. Commonly used image registration methods are usually divided into two categories: registration methods based on grayscale, and registration methods based on features (such as feature points and characteristic curves). The grayscale-based registration method uses the image grayscale value to measure the similarity between images. This type of method is simple to implement, but the speed is slow, and because remote sensing images are acquired by different sensors, different viewing angles, and different weather conditions, etc., There will be changes in scale, rotation, illumination, etc., which will cause this type of algorithm to fail to correctly register images, while methods based on local invariant features are not easily affected. The feature-based registration method determines the registration parameters according to the geometric relationship of the important features between the images to be registered. This type of method can reduce the amount of processed data, and has certain robustness to image distortion and noise. Therefore, the quality of matching performance depends largely on the method of feature description and the quality of feature extraction.

At present, in view of image size scaling, rotation, and translation, SIFT is mainly used to extract feature points as registration features. However, the SIFT feature needs to establish a feature point vector with a high dimension and a large amount of calculation, so it cannot meet the requirements of remote sensing image registration.

Fast corner points are based on the gray value of the image around the feature point, and detect the pixel value of a circle around the candidate feature point. If there are enough pixels in the area around the candidate point and the gray value of the candidate point are large enough, it is considered The candidate point is a feature point, namely:

Among them, I(x) is the gray level of any point on the circumference, I(p) is the gray level of the center of the circle, and ε _d is the threshold value of the gray value difference. If N is greater than the given threshold value, it is generally a quarter of the surrounding circle points Three, it is considered that p is a feature point.

The BRIEF descriptor is to select random point pixel pairs that obey N (N=128, 256, 512) groups of Gaussian distribution in the image smoothed size of 31×31 pixel blocks. The criterion is 0 to form a binary string.

Binary detection is defined as:

Among them, p ( x ₁ ) and p ( x ₂ ) are the pixel gray values at positions x ₁ and x ₂ in the pixel block p, respectively.

Then the BRIEF feature is defined as an n-dimensional binary string vector, namely:

The ORB algorithm has good rotation invariance, and extracts feature points with Fast corner points. Fast corner points are a very fast corner point extraction method. Since Fast does not have rotation invariance, ORB uses the centroid method as the corner points of ORB Adding direction information makes the feature invariant to rotation. In addition, in order to make the features suitable for size scaling, feature extraction can be performed in the case of inconsistent image sizes by establishing an image pyramid. In addition, when the feature point descriptor is established, the Brief descriptor is used for description, and the Brief descriptor is a binary descriptor, which can be matched very quickly. Therefore, image registration based on ORB features is suitable for real-time registration.

Contents of the invention

The purpose of the present invention is to provide a rapid registration method of remote sensing images, which is beneficial to improve the speed and accuracy of image registration.

In order to achieve the above object, the technical solution of the present invention is: a rapid registration method for remote sensing images, comprising the following steps:

Step S1: Extracting ORB feature points from the registration remote sensing image and the reference remote sensing image respectively;

Step S2: Perform initial matching on the extracted ORB feature points, and remove incorrectly matched feature points from the initially matched feature points;

Step S3: solving the parameters of the remote sensing image to be registered;

Step S4: Resampling the remote sensing image to be registered to complete the image registration.

Further, in step S1, image pyramids are respectively established for the remote sensing image to be registered and the reference remote sensing image, and ORB feature points are extracted for each layer of the image pyramid.

Further, in step S1, the method of extracting ORB feature points is: first perform FAST corner detection, perform Harris corner detection, select the first N best points, and then use non-maximum value suppression to verify the corner points, Eliminate false edge points to make the distribution of feature points even.

Further, in step S2, the extracted ORB feature points are initially matched using the Hamming distance feature point matching method based on double thresholds, and the angle between the corner direction is used as a constraint condition for the initially matched feature points, and the incorrectly matched ones are eliminated. Feature points, the specific method is:

The corner direction of the feature point is obtained by the gray-scale centroid method, that is, by calculating the centroid of the pixel gray scale of the corner circular neighborhood, the vector direction formed by the corner point and the centroid represents the corner direction;

Define the circular neighborhood moments of corner points as: , the centroid of the circular neighborhood of the corner point is: , then the vector direction formed by the corner point and the centroid is the corner point direction of the feature point: ;

Among them, m _pq represents the p+q order moment, I(x, y) represents the gray value of the pixel point (x, y) in the corner circular neighborhood, and (x, y) represents the corner circular neighborhood The pixel coordinates of , m ₀₀ represents the zero-order moment, m ₁₀ and m ₀₁ both represent the first-order moment;

Δ θ _i is the corner direction of a feature point in the remote sensing image to be registered, Δ θ _i ' is the corner direction of the corresponding feature point in the reference remote sensing image, and the angle between the corner directions is Δ θ _i =Δ θ _i - Δθ _i ' ;

Then perform initial matching on the extracted ORB feature points according to the following steps, and remove the wrongly matched feature points:

a. The rBRIEF descriptor is established for the corner points extracted from the remote sensing image to be registered and the reference remote sensing image. The set of feature points of the remote sensing image to be registered is {a1, a2, …, an1}, and the set of feature points of the reference remote sensing image is { b1, b2, ..., bn2};

b. Compare the descriptors of the feature point a1 of the remote sensing image to be registered with the descriptors of all feature points b1, b2, ..., bn2 of the reference remote sensing image, and calculate the descriptors of a1 and b1, b2, ..., bn2 The Hamming distance of the descriptor, select the point with the shortest Hamming distance from b1, b2, ..., bn2, and calculate the ratio of the shortest Hamming distance to the next shortest Hamming distance, if the ratio is smaller than the set comparison If the threshold is large, keep the point with the shortest Hamming distance and a1 as the initial pairing point, and calculate the angle Δ θ ₁ between the direction of the corner point, otherwise discard it;

c. According to the above method, calculate the initial pairing points of the feature points a2, ..., an1 of the remote sensing image to be registered in the reference remote sensing image in turn, and calculate the corresponding corner angles Δ θ ₂ , ..., Δ θ _n ;

d. Sort the initial paired points according to the ratio of the shortest Hamming distance to the second shortest Hamming distance from large to small, that is, sort the matching quality of feature points from good to poor, and extract the shortest Hamming distance and the second shortest Hamming distance The initial pairing point whose ratio is smaller than the set smaller threshold;

E, the initial pairing point that step d is extracted obtains optimal corner point direction included angle Δ θ _m by least squares method;

f. Using the angles of the corner direction angles of all the initial pairing points obtained in steps b and c, the optimal corner direction angle Δθ _m is used as a constraint condition, and the initial pairing points that deviate from a certain range of Δθ _m are eliminated.

Further, in step S3, the method for solving the parameters of the remote sensing image to be registered is:

Suppose the pixel point of the reference remote sensing image is f ( x , y ), the pixel point of the image to be registered is g ( x' , y' ), assuming that the coordinates of the point on the reference remote sensing image are ( x _i , y _i ), and The coordinates of the corresponding points on the remote sensing image to be registered are ( xi _' , y _i ' ), then the affine transformation between ( xi _, y _i ) and ( xi _' , y _i ' ) is expressed as :

In the formula, s is the scale factor, θ is the rotation angle, Δ x and Δ y are the translation amounts of the two coordinate axes respectively;

After obtaining m feature points, the optimal transformation matrix is obtained according to the RANSAC algorithm, that is, the registration parameters s , θ , Δ x , Δ y are determined;

According to the registration parameters, the remote sensing image to be registered is resampled, that is, the image registration is completed.

The beneficial effect of the present invention is that in order to reflect the dynamic changes of the monitoring area in time for the situations of different deformation and illumination among the remote sensing images, a rapid registration method of remote sensing images is proposed, which can effectively eliminate mis-matched points and ensure the accuracy of image registration. Accurate precision, strong practicability and broad application prospects.

Description of drawings

Fig. 1 is an implementation flow chart of the embodiment of the present invention.

Fig. 2 is a reference remote sensing image in the embodiment of the present invention.

Fig. 3 is a remote sensing image to be registered in the embodiment of the present invention.

Fig. 4 is an initial feature point matching diagram in an embodiment of the present invention.

FIG. 5 is a matching diagram of feature points after removing mismatched points in an embodiment of the present invention.

Fig. 6 is a diagram of the registration of the remote sensing image to be registered and the fusion of the reference remote sensing image in the embodiment of the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

The rapid registration method of remote sensing image of the present invention, as shown in Figure 1, comprises the following steps:

Step S1: Establish image pyramids for the registration remote sensing image (Fig. 3) and the reference remote sensing image (Fig. 2), and extract ORB feature points for each layer of the image pyramid. The method of extracting ORB feature points is as follows: First, perform FAST corner detection. Since the number of corner points extracted by FAST is too large, and contains edge points and pseudo corner points, the Harris corner detection algorithm is a stable corner detector. For corner point detection, select the top N best points, and then use non-maximum value suppression to verify the corner points and eliminate false edge points to make the distribution of feature points even.

Step S2: In order to obtain as many correctly matched feature points as possible, use the Hamming distance feature point matching method based on double thresholds for initial matching on the extracted ORB feature points, and use the corner angle as a constraint on the initially matched feature points Conditions to remove incorrectly matched feature points. The specific method is:

The corner direction of the feature point is obtained by the gray-scale centroid method, that is, by calculating the centroid of the pixel gray scale of the corner circular neighborhood, the vector direction formed by the corner point and the centroid represents the corner direction;

Define the circular neighborhood moments of corner points as: , the centroid of the circular neighborhood of the corner point is: , then the vector direction formed by the corner point and the centroid is the corner point direction of the feature point: ;

Among them, m _pq represents the p+q order moment, p and q are a coefficient respectively, I(x, y) represents the gray value of the pixel point (x, y) in the circular neighborhood of the corner point, (x, y) Represents the pixel coordinates in the circular neighborhood of the corner point, m ₀₀ represents the zero-order moment, m ₁₀ and m ₀₁ represent the first-order moment;

Δ θ _i is the corner direction of a feature point in the remote sensing image to be registered, Δ θ _i ' is the corner direction of the corresponding feature point in the reference remote sensing image, and the angle between the corner directions is Δ θ _i =Δ θ _i - Δθ _i ' ;

Then perform initial matching on the extracted ORB feature points according to the following steps, and remove the wrongly matched feature points:

a. The rBRIEF descriptor is established for the corner points extracted from the remote sensing image to be registered and the reference remote sensing image. The set of feature points of the remote sensing image to be registered is {a1, a2, …, an1}, and the set of feature points of the reference remote sensing image is { b1, b2, ..., bn2};

b. Using the Brute-Force algorithm, compare the descriptors of the feature point a1 of the remote sensing image to be registered with the descriptors of all feature points b1, b2, ..., bn2 of the reference remote sensing image, and calculate the descriptor of a1 and b1 , b2, …, bn2 descriptors’ Hamming distance, select the point with the shortest Hamming distance from b1, b2, …, bn2, and calculate the ratio of the shortest Hamming distance to the next shortest Hamming distance, and set The maximum threshold is 0.8. If the ratio is smaller than the set maximum threshold, the point with the shortest Hamming distance and a1 will be reserved as the initial pairing point, and the angle Δ θ ₁ between the direction of the corner point will be calculated, otherwise it will be discarded;

c. According to the above method, sequentially calculate the initial pairing points of the feature points a2, ..., an1 of the remote sensing image to be registered in the reference remote sensing image, as shown in Figure 4, and calculate the corresponding corner angle Δ θ ₂ , …, Δ θ _n ;

d. Sort the initial pairing points according to the ratio of the shortest Hamming distance to the second shortest Hamming distance from large to small, that is, sort the matching quality of feature points from good to bad, set the minimum threshold to 0.5, and extract the shortest Hamming The ratio of the Hamming distance to the next shortest Hamming distance is less than the initial pairing point of the set smaller threshold;

E, the initial pairing point that step d is extracted obtains optimal corner point direction included angle Δ θ _m by least squares method;

f. With the corner point direction angles of all the initial pairing points obtained in steps b and c, the optimal corner point direction angle Δθ _m is used as a constraint condition, and the initial pairing points that deviate from a certain range of Δ θ _m are eliminated, such as Figure 5 shows.

Step S3: solving the parameters of the remote sensing image to be registered.

Considering that there are transformations such as rotation and size between the remote sensing image to be registered and the reference remote sensing image, the transformation matrix between the images is determined to be H, and H is expressed as:

The RANSAC algorithm is used to find the optimal transformation parameters, and the process is as follows:

1) Randomly select m samples from the matched feature point pairs, obtain the transformation matrix H from these m samples, and then obtain the same-name points of the feature points in the remote sensing image to be registered in the reference remote sensing image according to the transformation matrix H , and then find the distance between the point with the same name obtained by the transformation matrix H and the matching point obtained by the Hamming distance matching, and use the point whose distance is smaller than the threshold as the interior point;

2) Repeat the above steps k times, and select a point set containing the largest number of interior points;

3) Recalculate the transformation matrix H with the samples in the selected point set, so as to obtain the best transformation model that matches most of the matching points.

Suppose the pixel point of the reference remote sensing image is f ( x , y ), the pixel point of the image to be registered is g ( x' , y' ), assuming that the coordinates of the point on the reference remote sensing image are ( x _i , y _i ), and The coordinates of the corresponding points on the remote sensing image to be registered are ( xi _' , y _i ' ), then the affine transformation between ( xi _, y _i ) and ( xi _' , y _i ' ) is expressed as :

In the formula, s is the scale factor, θ is the rotation angle, Δ x and Δ y are the translation amounts of the two coordinate axes respectively;

After obtaining m (m≥4) feature points, the optimal transformation matrix is obtained according to the RANSAC algorithm, that is, the registration parameters s , θ , Δ x , Δ y are determined.

Step S4: Resampling the remote sensing image to be registered using bilinear interpolation according to the registration parameters to complete the image registration.

The remote sensing image to be registered is scaled and rotated using the calculated transformation matrix H, and resampled by bilinear interpolation. The two images are fused according to the method of 0.5×reference image+0.5×image to be registered, as shown in the figure As shown in 6, the image registration is completed.

The above are the preferred embodiments of the present invention, and all changes made according to the technical solution of the present invention, when the functional effect produced does not exceed the scope of the technical solution of the present invention, all belong to the protection scope of the present invention.

Claims (5)

1. A fast registration method for remote sensing images, comprising the following steps: Step S1: Extracting ORB feature points from the registration remote sensing image and the reference remote sensing image respectively; Step S2: Perform initial matching on the extracted ORB feature points, and remove incorrectly matched feature points from the initially matched feature points; Step S3: solving the parameters of the remote sensing image to be registered; Step S4: Resampling the remote sensing image to be registered to complete the image registration. 2. The rapid registration method of a remote sensing image according to claim 1, wherein in step S1, an image pyramid is established for the remote sensing image to be registered and the reference remote sensing image respectively, and the ORB feature is extracted for each layer of the image pyramid point. 3. A kind of fast registration method of remote sensing image according to claim 1, it is characterized in that, in step S1, the method for extracting ORB feature point is: first carry out FAST corner point detection, carry out Harris corner point detection, select The top N best points, and then use non-maximum suppression to verify the corner points and remove false edge points to make the distribution of feature points uniform. 4. The rapid registration method of a kind of remote sensing image according to claim 1, characterized in that, in step S2, the extracted ORB feature points are initially matched using the Hamming distance feature point matching method based on double thresholds, For the feature points of the initial match, the angle between the direction of the corner point is used as the constraint condition, and the feature points of the wrong match are eliminated. The specific method is as follows: The corner direction of the feature point is obtained by the gray-scale centroid method, that is, by calculating the centroid of the pixel gray scale of the corner circular neighborhood, the vector direction formed by the corner point and the centroid represents the corner direction; Define the circular neighborhood moments of corner points as: , the centroid of the circular neighborhood of the corner point is: , then the vector direction formed by the corner point and the centroid is the corner point direction of the feature point: ; Among them, m _pq represents the p+q order moment, I(x, y) represents the gray value of the pixel point (x, y) in the corner circular neighborhood, and (x, y) represents the corner circular neighborhood The pixel coordinates of , m ₀₀ represents the zero-order moment, m ₁₀ and m ₀₁ both represent the first-order moment; Δ θ _i is the corner direction of a feature point in the remote sensing image to be registered, Δ θ _i ' is the corner direction of the corresponding feature point in the reference remote sensing image, and the angle between the corner directions is Δ θ _i =Δ θ _i - Δθ _i ' ; Then perform initial matching on the extracted ORB feature points according to the following steps, and remove the wrongly matched feature points: a. The rBRIEF descriptor is established for the corner points extracted from the remote sensing image to be registered and the reference remote sensing image. The set of feature points of the remote sensing image to be registered is {a1, a2, …, an1}, and the set of feature points of the reference remote sensing image is { b1, b2, ..., bn2}; b. Compare the descriptors of the feature point a1 of the remote sensing image to be registered with the descriptors of all feature points b1, b2, ..., bn2 of the reference remote sensing image, and calculate the descriptors of a1 and b1, b2, ..., bn2 The Hamming distance of the descriptor, select the point with the shortest Hamming distance from b1, b2, ..., bn2, and calculate the ratio of the shortest Hamming distance to the next shortest Hamming distance, if the ratio is smaller than the set comparison If the threshold is large, keep the point with the shortest Hamming distance and a1 as the initial pairing point, and calculate the angle Δ θ ₁ between the direction of the corner point, otherwise discard it; c. According to the above method, calculate the initial pairing points of the feature points a2, ..., an1 of the remote sensing image to be registered in the reference remote sensing image in turn, and calculate the corresponding corner angles Δ θ ₂ , ..., Δ θ _n ; d. Sort the initial paired points according to the ratio of the shortest Hamming distance to the second shortest Hamming distance from large to small, that is, sort the matching quality of feature points from good to poor, and extract the shortest Hamming distance and the second shortest Hamming distance The initial pairing point whose ratio is smaller than the set smaller threshold; E, the initial pairing point that step d is extracted obtains optimal corner point direction included angle Δ θ _m by least squares method; f. Using the angles of the corner direction angles of all the initial pairing points obtained in steps b and c, the optimal corner direction angle Δθ _m is used as a constraint condition, and the initial pairing points that deviate from a certain range of Δθ _m are eliminated. 5. The rapid registration method of a remote sensing image according to claim 1, characterized in that, in step S3, the method for solving the parameters of the remote sensing image to be registered is: Suppose the pixel point of the reference remote sensing image is f ( x , y ), the pixel point of the image to be registered is g ( x' , y' ), assuming that the coordinates of the point on the reference remote sensing image are ( x _i , y _i ), and The coordinates of the corresponding points on the remote sensing image to be registered are ( xi _' , y _i ' ), then the affine transformation between ( xi _, y _i ) and ( xi _' , y _i ' ) is expressed as : In the formula, s is the scale factor, θ is the rotation angle, Δ x and Δ y are the translation amounts of the two coordinate axes respectively; After obtaining m feature points, the optimal transformation matrix is obtained according to the RANSAC algorithm, that is, the registration parameters s , θ , Δ x , Δ y are determined; According to the registration parameters, the remote sensing image to be registered is resampled, that is, the image registration is completed.