CN117115214A - Multisource remote sensing image registration method based on improved PIIFD feature description - Google Patents
Multisource remote sensing image registration method based on improved PIIFD feature description Download PDFInfo
- Publication number
- CN117115214A CN117115214A CN202310875072.XA CN202310875072A CN117115214A CN 117115214 A CN117115214 A CN 117115214A CN 202310875072 A CN202310875072 A CN 202310875072A CN 117115214 A CN117115214 A CN 117115214A
- Authority
- CN
- China
- Prior art keywords
- matching
- feature
- point
- transformation
- piifd
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 82
- 230000009466 transformation Effects 0.000 claims abstract description 60
- 239000011159 matrix material Substances 0.000 claims abstract description 37
- 230000002457 bidirectional effect Effects 0.000 claims abstract description 5
- 238000001514 detection method Methods 0.000 claims description 13
- 230000004044 response Effects 0.000 claims description 12
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 7
- 230000015556 catabolic process Effects 0.000 claims description 5
- 238000006731 degradation reaction Methods 0.000 claims description 5
- 230000008030 elimination Effects 0.000 claims description 5
- 238000003379 elimination reaction Methods 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims description 4
- 238000011002 quantification Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 3
- 230000000007 visual effect Effects 0.000 abstract description 2
- 238000004422 calculation algorithm Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 230000002146 bilateral effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000009792 diffusion process Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a multisource remote sensing image registration method based on improved PIIFD feature description, and belongs to the field of visual image processing; the method comprises the following steps: firstly, respectively extracting respective characteristic points of a multisource remote sensing image to be registered and a reference image; carrying out feature description on each feature point through an improved PIIFD descriptor to obtain a feature descriptor vector corresponding to each feature point; then, a BBF method is adopted, a feature descriptor vector is utilized to carry out bidirectional initial matching, and a feature main direction and a RANSAC method are utilized to reject false matching, so that a fine matching point pair is obtained; according to different transformation modes, selecting a corresponding number of fine matching point pairs to calculate a transformation matrix, and optimizing parameters of the transformation matrix by using a least square method; and finally, multiplying the transformation matrix with optimized parameters by the image to be registered to obtain a final registration result. The method solves the problem that the multi-scale images are difficult to register, has certain advantages in the matching quantity and the registration accuracy, and has strong practicability.
Description
Technical Field
The invention belongs to the field of image scene generation, and particularly relates to a multisource remote sensing image registration method based on improved PIIFD feature description.
Background
Image registration is the process of geometrically aligning two or more images of the same scene taken at different times, from different viewpoints, and by different sensors.
In recent years, due to the wide application of a multi-sensor vision system, a multi-source remote sensing image acquisition technology is continuously developed, and plays an important role in the fields of pattern recognition, medical imaging, remote sensing and modern military, and registration problems based on multi-mode images are gradually developed.
The existing multi-source remote sensing image registration method still has low registration precision, and can only process remote sensing image registration of two modes generally, but cannot consider multiple modes; and cannot handle the image scale differences brought by different sensor imaging.
Existing registration methods can be broadly divided into two categories: region-based registration methods and feature-based registration methods.
The region-based registration method is mainly: the similarity measurement is established by using the image gray information to perform image registration, but the existing region-based registration method has different degrees of problems on aspects of image mode, intensity transformation, complex space transformation, computational complexity and the like, so that the application of the method is greatly limited.
The feature-based registration method is more robust in coping with problems of image intensity variation, noise and the like, and common features include point features, line features, region features and the like. Because the line features and the area features have the area features, the matching position cannot be accurately determined, and most algorithms mainly extract the point features.
The existing multi-source remote sensing image methods such as PSO-SIFT, RIFT, SURF-RPM and the like cannot process multi-scale multi-source remote sensing image data, so that images cannot be well registered.
Disclosure of Invention
In order to overcome the defects of the multi-source remote sensing image registration method, the invention provides a multi-source remote sensing image registration method based on improved PIIFD feature description, which can adapt to more modes and has multi-scale image processing capability.
The multi-source remote sensing image registration method based on the improved PIIFD feature description comprises the following steps of:
firstly, selecting a KAZE method to extract respective characteristic points of a multi-source remote sensing image and a reference image to be registered;
the method comprises the following specific steps:
firstly, respectively constructing a nonlinear scale space of an image to be registered or a reference image based on nonlinear filtering;
the nonlinear scale space comprises O scale space group numbers and S sub-layer numbers;
then, the process is carried out,for the ith sub-layer (o) i ,s i ) The random pixel points in the image are compared with eight neighboring pixel points around the same layer and nine pixel points corresponding to the same position on the upper layer and the lower layer, and when the pixel point is larger or smaller than all neighboring pixel points, the pixel point is an extreme point;
then, calculating a response value by scale normalization Hessian determinant, wherein an extremum corresponding to the response value is a KAZE characteristic point;
the Hessian formula is:
wherein l xx And L yy Is the second partial derivative of the brightness L in the x or y direction, L xy Is the mixed second partial derivative of brightness L in x and y directions, sigma 2 The number of the scale space groups is Octave;
step two, carrying out feature description on each feature point through an improved PIIFD descriptor to obtain a feature descriptor vector corresponding to each feature point;
the specific process is as follows:
first, for the feature point σ i Calculating the scale factor mu i The following formula:
offset is a constant, typically taking a value of 1.6; lambda (lambda) i For adjusting the variable of the scale range, the sub-pixel approximate coordinates of the feature points are calculated, and the following formula is adopted:
x＝(x,y,λ) T ，
where L (x) is an approximation of the laplace operator,is the space coordinate approximation, byCalculating approximate coordinates by the formula so as to calculate scale factors; lambda is [ -1,1]Is a random variable of (a).
Then, by the scale factor mu i Multiplying the standard quantification k to obtain an improved adaptive multi-scale PIIFD descriptor detection neighborhood size of (kμ) × (kμ);
each feature point corresponds to a different scale factor, so the multi-scale PIIFD descriptor detection neighborhood of each feature is also different.
Finally, in the detection neighborhood of size (kμ) × (kμ), computing improved PIIFD descriptor extraction descriptors;
the method comprises the following steps: extracting 16 small square regions from the square neighborhood of (kμ) × (kμ), each small square region beingCorresponds to a direction histogram.
The feature point σ is obtained by converting the 16 direction histograms covering 0 to 2 pi (0 °,22.5 °,..once, 337.5 °) on average into 8 degradation direction histograms covering 0 to pi (0 °,22.5 °,..once, 157.5 °) on average by calculating the sum of opposite directions, and then normalizing it with a linear descriptor constructed by the degradation direction histogram and the row vector of its 180 ° rotation matrix i The corresponding descriptor vector is 128 in length.
Thirdly, adopting a BBF method, carrying out bidirectional initial matching by utilizing a feature descriptor vector, carrying out mismatching elimination by utilizing a feature main direction, and eliminating wrong matching by adopting a RANSAC method to obtain a fine matching point pair;
the method comprises the following steps:
step 301, sequentially calculating Euclidean distances between the feature point a and all feature points in a reference image aiming at the current feature point a of the image to be registered, selecting the nearest feature point meeting the condition, matching with the feature point a, and adding the nearest feature point to an initial matching result;
the conditions satisfied are as follows:
the nearest feature point and the next nearest feature point in the reference image satisfy the following formula:
wherein d is 1 For the Euclidean distance d between the feature point a and the nearest feature point 2 And eta is a preset threshold value for the Euclidean distance between the characteristic point a and the next-nearest characteristic point.
Step 302, counting initial matching results, and eliminating rotation errors by utilizing a characteristic main direction;
the method comprises the following steps:
randomly selecting two groups of characteristic points corresponding to initial matching, wherein the main directions of the two groups of characteristic points are respectivelyAndn is the number of matched pairs, ">For the principal direction angles of two sets of matching feature points, the feature principal direction angle difference +.>The method comprises the following steps:
after twisting the images to the same direction, rejecting the unsatisfied imagesThe matching pairs which are matched are left, namely the matching result after the rotation error is eliminated.
Step 303, rejecting the error matching by using RANSAC;
the method specifically comprises the following steps: for the current iteration, randomly extracting 4 non-collinear matching point pairs from the matching pairs excluding the rotation error, and taking the 4 non-collinear matching point pairs as sample data to calculate a transformation matrix H, and marking the transformation matrix H as a matrix model M;
then, calculating the cost function error of each matching pair and the matrix model M in sequence, and adding the current matching pair into the inner point set I if the error is smaller than a set threshold value;
the iteration number is increased by 1, 4 non-collinear matching point pairs are randomly extracted again to serve as samples, a matrix model M1 is calculated, the cost function error of each matching point pair and the matrix model M1 is calculated similarly, and matching points with the error smaller than a set threshold value are added into an inner point set I1;
until the set iteration times are reached, selecting an inner point set with the largest number of matching point pairs as a fine matching point pair;
step four, from the fine matching point pairs, selecting a corresponding number of fine matching point pairs according to different transformation modes to calculate a transformation matrix, and optimizing parameters of the transformation matrix by using a least square method;
the transformation mode comprises the following steps: similarity transformation, affine transformation or projective transformation;
the similarity transformation corresponds to two groups of fine matching point pairs, the affine transformation needs three groups of fine matching point pairs, and the projection transformation needs four groups of fine matching point pairs;
and fifthly, multiplying the transformation matrix with optimized parameters with the image to be registered to obtain a final registration result.
The invention has the advantages that:
compared with the prior art, the multi-source remote sensing image registration method based on the improved PIIFD feature description can register image data of multiple modes, solves the problem that multi-scale images are difficult to register through the improved PIIFD descriptor, has certain advantages in the aspects of matching quantity and registration accuracy, and is high in practicability.
Drawings
FIG. 1 is a flow chart of a multi-source remote sensing image registration method based on improved PIIFD feature description of the present invention;
FIG. 2 is a graph showing the inverse addition calculation effect of the present invention;
FIG. 3 is a schematic diagram of a feature point pair for selecting a correct match using a principal direction histogram in accordance with the present invention;
FIG. 4 is a diagram showing the effect of rejecting false matches by using the feature principal direction and the RANSAC method;
FIG. 5 is a schematic diagram of the invention for respectively selecting the main direction consistency and the RANSAC algorithm for mismatching removal;
fig. 6 is a schematic diagram of a picture obtained by transforming a picture according to different transformation methods according to the present invention.
Detailed Description
The present invention is further described in detail below with reference to the drawings and examples for the purpose of facilitating understanding and practicing the present invention by those of ordinary skill in the art. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
The invention provides a multisource remote sensing image registration method with more adaptation modes and multiscale image processing capability based on improved PIIFD feature description, which specifically comprises the following steps: firstly, preprocessing a multisource remote sensing image to be registered to obtain a registration graph, and extracting feature points by a KAZE method; then, characterizing the extracted feature points by the improved PIIFD descriptor; matching the feature description; finally, carrying out parameter estimation and model transformation on the feature point matching result by adopting similarity transformation, affine transformation or projective transformation, and calculating model parameters by using a least square method. The invention can register image data of multiple modes, solves the problem that the multi-scale image is difficult to register through the improved PIIFD descriptor, has certain advantages in the aspects of matching quantity and registration precision, and has strong practicability.
The multi-source remote sensing image registration method based on the improved PIIFD feature description, as shown in figure 1, comprises the following steps:
firstly, selecting a KAZE method to extract respective characteristic points of a multi-source remote sensing image and a reference image to be registered;
the method comprises the steps of preprocessing a multisource image to be registered, namely a group of images into an image to be registered and a reference image, obtaining a reference image, and extracting characteristic points of the reference image and the image to be registered respectively by a KAZE method;
the method comprises the following specific steps:
firstly, respectively constructing a nonlinear scale space of an image to be registered or a reference image based on nonlinear filtering;
the nonlinear scale space comprises O scale space group numbers and S sub-layer numbers; the relationship between the layers is expressed as:
wherein sigma 0 For reference scale level o i Sum s i The number of sets of Octave scale space and the number of sublevels of the sublevels,
o i ∈[0，1，...，O-1],s i ∈[0，1，...，S-1],i∈[0，1，...，N]n is the total number of non-linearly filtered images.
Next, the pixel unit σ is set i Converting into time units, the expression of the scale factor of each sub-layer:
t i for the evolution time, a nonlinear scale space is obtained by using an AOS algorithm according to a set of evolution times.
Then, for the i-th group i-th sublayer (o i ,s i ) The random pixel points in the image are compared with eight neighboring pixel points around the same layer and nine pixel points corresponding to the same position on the upper layer and the lower layer, and when the pixel point is larger or smaller than all neighboring pixel points, the pixel point is an extreme point;
then, in a scale space, calculating a response value of each layer of image at each pixel point through scale normalization Hessian determinant, wherein an extremum corresponding to the response value is a KAZE characteristic point;
for example, the scale space has three layers, and the obtained pixel points, that is, the values of the feature points are the surrounding eight layers and the upper layer and the lower layer 18 layers, and the maximum value of 26 layers is taken as an extremum. The response value is the value of the extremum normalized Hessian determinant, and the Hessian formula is:
wherein L is xx And L yy Is the second partial derivative of the brightness L in the x or y direction, L xy Is the mixed second partial derivative of brightness L in x and y directions, sigma 2 The number of the scale space groups is Octave;
step two, carrying out feature description on each feature point through an improved PIIFD descriptor to obtain a feature descriptor vector corresponding to each feature point;
the feature points are characterized by the improved PIIFD descriptors, mainly the self-adaptive extraction is carried out on the feature description areas, and in the multi-mode images, different mode images generally have different resolutions and different visual areas, so that the scale is changed. PIIFD does not exhibit feature scale variation using a fixed neighborhood size (typically 40 x 40). In the SIFT algorithm, it detects feature points in the scale space, providing each feature point with its scale information. And then determining the size of the neighborhood of the extraction descriptor according to the scale information so as to realize scale invariance, thus the SIFT algorithm thought is used as a reference, and the adaptive neighborhood region is required to realize scale invariance so as to accurately describe the characteristics.
The specific process is as follows:
first, for the i-th feature point σ i (the self-adaptive neighborhood is to describe different feature neighborhood according to the scale information of the feature points, so that different scale factors are needed to be adopted for different feature points), and the scale factor mu is calculated i The following formula:
offset is a constant, typically taking a value of 1.6; lambda (lambda) i For adjusting the variable of the scale range, the sub-pixel approximate coordinates of the feature points are calculated, and the following formula is adopted:
x＝(x,y,λ) T ，
where L (x) is an approximation of the laplace operator,calculating approximate coordinates for the space coordinate approximation value through the above formula, so as to calculate scale factors; lambda is [ -1,1]Is a random variable of (a).
Then, by the scale factor mu i Multiplying the detection neighborhood size with the custom standard quantification k to obtain the improved self-adaptive multi-scale PIIFD descriptor detection neighborhood size which is (kμ),
the standard quantity is customized, and then the scale factor is multiplied by the standard quantity to obtain the detection range of the neighborhood area for each feature to change. Each feature point corresponds to a different scale factor, so the multi-scale PIIFD descriptor detection neighborhood of each feature is also different.
Finally, in the detection neighborhood of size (kμ) × (kμ), computing improved PIIFD descriptor extraction descriptors;
the method comprises the following steps: firstly, calculating the amplitude and direction of an image gradient, and calculating the main direction orientation of a feature point by adopting a continuous average square gradient; extracting 16 small square regions from the square neighborhood of (kμ) × (kμ), each small square region beingCorresponds to a direction histogram.
As shown in fig. 2, the characteristic point σ is obtained by converting the direction histogram of 16 average coverage 0 to 2pi (0 °,22.5 °,..once, 337.5 °) into the degenerate direction histogram of 8 average coverage 0 to pi (0 °,22.5 °,..once, 157.5 °) by calculating the sum of opposite directions, and then normalizing it using the linear descriptor constructed by the degenerate direction histogram and the row vector of its 180 ° rotation matrix i The corresponding descriptor vector is 128 in length.
Thirdly, adopting a BBF method, carrying out bidirectional initial matching by utilizing a feature descriptor vector, carrying out mismatching elimination by utilizing a feature main direction, and eliminating wrong matching by adopting a RANSAC method to obtain a fine matching point pair;
the method comprises the following steps:
step 301, sequentially calculating Euclidean distances between the feature point a and all feature points in a reference image aiming at the current feature point a of the image to be registered, selecting the nearest feature point meeting the condition, matching with the feature point a, and adding the nearest feature point to an initial matching result;
the conditions satisfied are as follows:
the nearest feature point and the next nearest feature point in the reference image satisfy the following formula:
wherein d is 1 For the Euclidean distance d between the feature point a and the nearest feature point 2 And eta is a preset threshold value for the Euclidean distance between the characteristic point a and the next-nearest characteristic point.
Step 302, counting initial matching results, and eliminating rotation errors by utilizing a characteristic main direction;
the method comprises the following steps:
randomly selecting two groups of characteristic points corresponding to initial matching, wherein the main directions of the two groups of characteristic points are respectivelyAndn is the number of matched pairs, ">For the principal direction angles of two sets of matching feature points, the feature principal direction angle difference +.>The method comprises the following steps:
after twisting the images to the same direction, rejecting the unsatisfied imagesThe matching pairs which are matched are left, namely the matching result after the rotation error is eliminated.
For example, if a group of images is matched, the main directions of the feature points ensure the rotation invariance of the feature points, when the images rotate to the same position, the main directions of features between the correct matching pairs of the images are theoretically the same, and the directions of incorrect matching are different, so when the main direction difference value of one matching pair is obviously different from other matching pairs, the matching pairs are considered as one incorrect matching pair, and when the images rotate to the same position, the main direction angle difference value of two matching feature points is smaller than 5 degrees, and the matching pairs are considered as one correct matching pair. As shown in fig. 3, the present embodiment uses a histogram for statistics, taking 5 ° as an interval, the range of the x-axis of the histogram is [0 °,360 °), and the y-axis statistics is contained in the corresponding intervalFinally, taking the feature pairs with the most bins in the histogram as the feature point pairs which are correctly matched in the text.
Step 303, rejecting the error matching by using RANSAC;
the method specifically comprises the following steps: for the current iteration, randomly extracting 4 non-collinear matching point pairs from the matching pairs excluding the rotation error, and taking the 4 non-collinear matching point pairs as sample data to calculate a transformation matrix H, and marking the transformation matrix H as a matrix model M;
then, calculating the cost function error of each matching pair and the matrix model M in sequence, and adding the current matching pair into the inner point set I if the error is smaller than a set threshold value;
the iteration number is increased by 1, 4 non-collinear matching point pairs are randomly extracted again to serve as samples, a matrix model M1 is calculated, the cost function error of each matching point pair and the matrix model M1 is calculated similarly, and matching points with the error smaller than a set threshold value are added into an inner point set I1;
until the set iteration times are reached, selecting an inner point set with the largest number of matching point pairs as a fine matching point pair; at this time, the matrix model is the optimal model, the corresponding cost function is the smallest, and the purpose of RANSAC is to find the optimal parameter matrix model so that the number of data points meeting the matrix model is the largest.
As shown in fig. 4, a schematic diagram of mismatch removal according to the main direction consistency and mismatch rejection according to the RANSAC algorithm is shown. FIG. 4 (a) is a bilateral matching diagram of a set of infrared-visible image extraction AM-PIIFD feature descriptors; FIG. 4 (b) is a schematic illustration of test image with mismatch removed according to primary direction consistency; fig. 4 (c) is a schematic diagram of the mismatch removal of the test image according to the RANSAC algorithm.
Step four, from the fine matching point pairs, selecting a corresponding number of fine matching point pairs according to different transformation modes to calculate a transformation matrix, and optimizing parameters of the transformation matrix by using a least square method;
the transformation mode comprises the following steps: similarity transformation, affine transformation or projective transformation;
the similarity transformation corresponds to two groups of fine matching point pairs, the affine transformation needs three groups of fine matching point pairs, and the projection transformation needs four groups of fine matching point pairs; therefore, when the matching number is less than or equal to a pair, the registration is not completed, and the transformation model cannot be selected for registration; when the matching pairs are two pairs, similar transformation is selected for registration, when the matching pairs are three pairs, affine transformation is selected for registration, and when the number of the matching pairs is more than or equal to four pairs, projection transformation is used for registration. The more matching pairs can be matched, the more complex transformation models can be selected, and the better registration effect is obtained.
And fifthly, multiplying the transformation matrix with optimized parameters by the image to be registered to obtain a final registration result with the reference image.
Examples:
a multi-source remote sensing image registration method based on improved PIIFD feature description, the method comprising the steps of:
step S1, preprocessing an image to be registered to obtain a registration graph, and selecting a KAZE method to extract characteristic points;
the method comprises the following specific steps: firstly, constructing a nonlinear scale space for each image, and calculating a response value of each layer of image at each pixel point in the scale space through scale normalization Hessian determinant, wherein the extremum of the response value is a KAZE characteristic point, and the method specifically comprises the following steps:
the nonlinear scale space of S1.1 is constructed by nonlinear filtering, wherein the nonlinear filtering method can be described by a nonlinear partial differential equation as shown in the following formula:
where L is the brightness of the image, time t is the scale parameter, div andthe gradient and the divergence are respectively represented, c is a conductance function specifically:
wherein:
for a gradient image of the original image after Gaussian smoothing, k is a factor controlling the level of diffusion, so that nonlinear diffusion can be self-adapted to the local features of the image.
Constructing a nonlinear scale space, wherein the scale space comprises O scale space group numbers and S sub-layer numbers, and the relation among the layers is expressed as follows:
then the pixel unit sigma i Converted into time units, the scale factor of each sub-layer is expressed as t i ＝σ i 2 /2，t i For evolutionary time, according to a groupThe time of the transformation, the nonlinear scale space can be obtained by using an AOS algorithm.
S1.2, feature point detection is carried out, and response values of each layer of image at each pixel point are calculated through scale normalization Hessian determinant, wherein the Hessian formula is as follows:
wherein L is xx And L yy Is the second partial derivative of the brightness L in the x or y direction, L xy Is the mixed second partial derivative of luminance L in the x and y directions. And searching response extremum in all the filtered images to obtain characteristic points.
Step S2, carrying out feature description on the feature points extracted in the step S1 through the improved PIIFD descriptor;
the method comprises the following specific steps: firstly, adaptively acquiring a feature description area; then, computing an improved PIIFD descriptor;
s2.1, the invention sets the scale factor of the characteristic point of response to be mu, and the scale factor is represented by the following formula:
offset is 1.6, o i Sum s i The number of the Octave scale space groups and the number of the sublayers of the sublevel lambda of the current feature point i The variable is obtained by calculating the sub-pixel approximate coordinates of the characteristic points, and the following formula is adopted:
x＝(x,y,λ) T ，
where L (x) is an approximation of the laplace operator,is a spatial coordinate approximation. By a scale factor mu i An improved adaptive multi-scale PIIFD descriptor detection neighborhood size (kμ) can be determined, wherek defaults to 6, thus a varying detection range is adopted for each feature.
S2.2 extracting descriptors, firstly calculating the amplitude and the direction of an image gradient, calculating the main direction orientation of characteristic points by adopting continuous average square gradient, and dividing the varied square neighborhood determined in S2.1 into areas for obtaining better precision and calculation efficiency, wherein the extraction area consists of 16 small squares, and the area of each small square isCorresponds to a direction histogram. The 16 average coverage 0-2 pi (0 °,22.5 °,..once, 337.5 °) direction histograms are converted into 8 average coverage 0-pi (0 °,22.5 °,..once, 157.5 °) degradation direction histograms by calculating the sum of opposite directions, which is in order to achieve invariance at the time of gradient inversion, and then normalized to obtain a descriptor vector of length 128 using the obtained direction histogram and a linear descriptor constructed using the row vector of its 180 ° rotation matrix.
Step S3, performing feature matching on the feature description obtained in the step S2;
the specific steps are that after the feature description obtained in the step S2 is subjected to one-time bidirectional matching by utilizing a BBF method, error matching elimination is performed by utilizing a feature main direction, and error matching is performed by utilizing a RANSAC method;
s3.1, performing bilateral matching once by utilizing a BBF method, wherein a matching strategy is Euclidean distance between a feature point corresponding to the registration image and all features in another registration image, and when the nearest feature point and the next nearest feature point meet the following formula:the match is deemed to be correct;
wherein d is 1 D is the Euclidean distance between the characteristic point and the nearest characteristic point 2 And eta is a preset threshold value for the Euclidean distance between the characteristic point and the next-nearest characteristic point.
The S3.2 specific steps of feature main direction consistency are as follows: counting the initial matching obtained in S3.1, wherein the initial matching corresponds toIs the main direction of the two groups of characteristic pointsAnd->N is the number of matched pairs, ">Is the principal direction angle of the feature points of the two sets of multisource images, so the feature principal direction angle difference +.>The method comprises the following steps:
after the rotation error is eliminated, the elimination is not satisfiedAnd (3) the matching pairs are left, then, the RANSAC is adopted to reject the false matching, specifically, the matching point pairs with the number of 4 are randomly extracted from the matching pairs as sample data to carry out estimation parameters, the distances of all the matching point pairs under the estimated transformation parameters are calculated, if the distances are smaller than a preset threshold value, the matching point pairs are regarded as inner points, otherwise, the outer points are regarded as outer points, and the inner points are used as fine matching point pairs through repeated iteration.
And S4, carrying out parameter estimation and model transformation on the feature point matching result obtained in the step S3 by adopting one of similarity transformation, affine transformation and projective transformation, and calculating model parameters by using a least square method.
Claims (4)
1. The multi-source remote sensing image registration method based on the improved PIIFD feature description is characterized by comprising the following specific steps of:
firstly, selecting a KAZE method to extract respective characteristic points of a multi-source remote sensing image and a reference image to be registered;
step two, carrying out feature description on each feature point through an improved PIIFD descriptor to obtain a feature descriptor vector corresponding to each feature point;
the specific process is as follows:
first, for the feature point σ i Calculating the scale factor mu i The following formula:
offset is constant; lambda (lambda) i To adjust the variable of the scale range;
then, by the scale factor mu i Multiplying the standard quantification k to obtain an improved adaptive multi-scale PIIFD descriptor detection neighborhood size of (kμ) × (kμ);
finally, in the detection neighborhood of size (kμ) × (kμ), computing improved PIIFD descriptor extraction descriptors;
the method comprises the following steps: extracting 16 small square regions from the square neighborhood of (kμ) × (kμ), each small square region beingCorresponding to a direction histogram;
the feature point σ is obtained by converting the 16 direction histograms covering 0 to 2 pi (0 °,22.5 °,..once, 337.5 °) on average into 8 degradation direction histograms covering 0 to pi (0 °,22.5 °,..once, 157.5 °) on average by calculating the sum of opposite directions, and then normalizing it with a linear descriptor constructed by the degradation direction histogram and the row vector of its 180 ° rotation matrix i A corresponding descriptor vector of length 128;
thirdly, adopting a BBF method, carrying out bidirectional initial matching by utilizing a feature descriptor vector, carrying out mismatching elimination by utilizing a feature main direction, and eliminating wrong matching by adopting a RANSAC method to obtain a fine matching point pair;
the method comprises the following steps:
step 301, sequentially calculating Euclidean distances between the feature point a and all feature points in a reference image aiming at the current feature point a of the image to be registered, selecting the nearest feature point meeting the condition, matching with the feature point a, and adding the nearest feature point to an initial matching result;
the conditions satisfied are as follows:
the nearest feature point and the next nearest feature point in the reference image satisfy the following formula:
wherein d is 1 For the Euclidean distance d between the feature point a and the nearest feature point 2 The Euclidean distance between the feature point a and the next-nearest feature point is defined as eta as a preset threshold value;
step 302, counting initial matching results, and eliminating rotation errors by utilizing a characteristic main direction;
the method comprises the following steps:
randomly selecting two groups of characteristic points corresponding to initial matching, wherein the main directions of the two groups of characteristic points are respectivelyAnd->N is the number of matched pairs, ">For the principal direction angles of two sets of matching feature points, the feature principal direction angle difference +.>The method comprises the following steps:
after twisting the images to the same direction, rejecting the unsatisfied imagesThe matching pairs which are matched are left, namely the matching result after the rotation error is eliminated;
step 303, rejecting the error matching by using RANSAC;
the method specifically comprises the following steps: for the current iteration, randomly extracting 4 non-collinear matching point pairs from the matching pairs excluding the rotation error, and taking the 4 non-collinear matching point pairs as sample data to calculate a transformation matrix H, and marking the transformation matrix H as a matrix model M;
then, calculating the cost function error of each matching pair and the matrix model M in sequence, and adding the current matching pair into the inner point set I if the error is smaller than a set threshold value;
the iteration number is increased by 1, 4 non-collinear matching point pairs are randomly extracted again to serve as samples, a matrix model M1 is calculated, the cost function error of each matching point pair and the matrix model M1 is calculated similarly, and matching points with the error smaller than a set threshold value are added into an inner point set I1;
until the set iteration times are reached, selecting an inner point set with the largest number of matching point pairs as a fine matching point pair;
step four, from the fine matching point pairs, selecting a corresponding number of fine matching point pairs according to different transformation modes to calculate a transformation matrix, and optimizing parameters of the transformation matrix by using a least square method;
and fifthly, multiplying the transformation matrix with optimized parameters with the image to be registered to obtain a final registration result.
2. The method for multi-source remote sensing image registration based on improved PIIFD feature description of claim 1, wherein the first step is as follows:
firstly, respectively constructing a nonlinear scale space of an image to be registered or a reference image based on nonlinear filtering;
the nonlinear scale space comprises O scale space group numbers and S sub-layer numbers;
then, for the i-th group i-th sublayer (o i ,s i ) Random pixel points in the image are compared with the same layerNine pixel points at the same positions corresponding to the eight neighboring pixel points around (a) and the upper and lower layers, and when the pixel point is larger or smaller than all neighboring pixel points, the pixel point is an extreme point;
finally, calculating a response value by scale normalization Hessian determinant, wherein an extremum corresponding to the response value is a KAZE characteristic point;
the Hessian formula is:
wherein L is xx And L yy Is the second partial derivative of the brightness L in the x or y direction, L xy Is the mixed second partial derivative of brightness L in x and y directions, sigma 2 Is the number of sets of Octave scale space.
3. The method for multi-source remote sensing image registration based on improved PIIFD feature description of claim 1, wherein in the second step, a variable λ of a scale range is adjusted i The sub-pixel approximate coordinates of the feature points are calculated as follows:
x＝(x,y,λ) T ，
where L (x) is an approximation of the laplace operator,calculating approximate coordinates for the space coordinate approximation value through the above formula, so as to calculate scale factors; lambda is [ -1,1]Is a random variable of (a).
4. The method for registering a multisource remote sensing image based on an improved PIIFD feature description as claimed in claim 1, wherein in the fourth step, the transformation means comprises: similarity transformation, affine transformation or projective transformation;
the similarity transformation corresponds to two groups of fine matching point pairs, the affine transformation needs three groups of fine matching point pairs, and the projection transformation needs four groups of fine matching point pairs.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2022111298076 | 2022-09-16 | ||
| CN202211129807 | 2022-09-16 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117115214A true CN117115214A (en) | 2023-11-24 |
Family
ID=88802807
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310875072.XA Pending CN117115214A (en) | 2022-09-16 | 2023-07-17 | Multisource remote sensing image registration method based on improved PIIFD feature description |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117115214A (en) |
-
2023
- 2023-07-17 CN CN202310875072.XA patent/CN117115214A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110097093B (en) | Method for accurately matching heterogeneous images | |
| CN109409292B (en) | Heterogeneous image matching method based on refined feature optimization extraction | |
| CN111080529A (en) | Unmanned aerial vehicle aerial image splicing method for enhancing robustness | |
| CN112085772B (en) | Remote sensing image registration method and device | |
| CN111079556A (en) | Multi-temporal unmanned aerial vehicle video image change area detection and classification method | |
| CN107145829B (en) | Palm vein identification method integrating textural features and scale invariant features | |
| CN111369605B (en) | Infrared and visible light image registration method and system based on edge features | |
| CN112017223B (en) | Heterologous image registration method based on improved SIFT-Delaunay | |
| CN111767960A (en) | Image matching method and system applied to image three-dimensional reconstruction | |
| CN113256653B (en) | Heterogeneous high-resolution remote sensing image registration method for high-rise ground object | |
| CN109785370B (en) | Weak texture image registration method based on space time sequence model | |
| CN108491498A (en) | A kind of bayonet image object searching method based on multiple features detection | |
| CN113888461A (en) | Method, system and equipment for detecting defects of hardware parts based on deep learning | |
| CN112614167A (en) | Rock slice image alignment method combining single-polarization and orthogonal-polarization images | |
| CN115601407A (en) | Infrared and visible light image registration method | |
| CN116433733A (en) | Registration method and device between optical image and infrared image of circuit board | |
| CN114358166B (en) | Multi-target positioning method based on self-adaptive k-means clustering | |
| CN116664892A (en) | Multi-temporal remote sensing image registration method based on cross attention and deformable convolution | |
| CN111563896A (en) | Image processing method for catenary anomaly detection | |
| CN112288758A (en) | Infrared and visible light image registration method for power equipment | |
| CN114463397A (en) | Multi-modal image registration method based on progressive filtering | |
| CN109086350B (en) | Mixed image retrieval method based on WiFi | |
| CN113128518B (en) | Sift mismatch detection method based on twin convolution network and feature mixing | |
| CN113095385A (en) | Multimode image matching method based on global and local feature description | |
| CN107886530A (en) | A kind of improved image registration algorithm based on SIFT feature |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |