CN113096168B - Optical remote sensing image registration method and system combining SIFT points and control line pairs - Google Patents
Optical remote sensing image registration method and system combining SIFT points and control line pairs Download PDFInfo
- Publication number
- CN113096168B CN113096168B CN202110287621.2A CN202110287621A CN113096168B CN 113096168 B CN113096168 B CN 113096168B CN 202110287621 A CN202110287621 A CN 202110287621A CN 113096168 B CN113096168 B CN 113096168B
- Authority
- CN
- China
- Prior art keywords
- straight line
- image
- matching
- registration
- feature set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 230000003287 optical effect Effects 0.000 title claims abstract description 22
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 56
- 230000009466 transformation Effects 0.000 claims description 25
- 230000000007 visual effect Effects 0.000 claims description 18
- 239000011159 matrix material Substances 0.000 claims description 17
- 238000012216 screening Methods 0.000 claims description 13
- 238000000605 extraction Methods 0.000 claims description 8
- 238000005259 measurement Methods 0.000 claims description 7
- 238000013519 translation Methods 0.000 claims description 6
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 3
- 238000005314 correlation function Methods 0.000 claims description 3
- 238000011426 transformation method Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 description 38
- 238000010586 diagram Methods 0.000 description 22
- 238000004590 computer program Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 7
- 230000006872 improvement Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000004927 fusion Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 230000002457 bidirectional effect Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 241000282326 Felis catus Species 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000036039 immunity Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20061—Hough transform
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an optical remote sensing image registration method and system combining SIFT points and control line pairs, wherein the method comprises the following steps: respectively acquiring linear feature sets of a reference image and an image to be registered; obtaining an initial registration image based on the linear feature set of the reference image and the linear feature set of the image to be registered; acquiring a linear feature set of the initial registration image; respectively counting intersecting straight line pairs in a straight line feature set of a reference image and a straight line feature set of an initial registration image to obtain a statistical result, and obtaining a candidate matching set based on the statistical result; obtaining a coarse registration result based on the candidate matching set; and obtaining a precise registration result of the reference image and the rough registration image by using a SIFT key point registration algorithm, and finishing the registration of the optical remote sensing image. The method disclosed by the invention is fast in speed and good in robustness, and can be used for registering the visible light remote sensing image with the typical linear target.
Description
Technical Field
The invention belongs to the technical field of aerospace, relates to the field of optical remote sensing image registration, and particularly relates to an optical remote sensing image registration method and system combining SIFT points and control line pairs.
Background
In aerospace research, abundant remote sensing data of the earth can be obtained by utilizing aerospace vehicles with different heights. Remote sensing data is widely applied to various fields such as battlefield detection, aircraft navigation, unmanned aerial vehicle landing, horizon detection and the like by virtue of timeliness and practicability. The rapid development of satellite remote sensing technology enables the space remote sensing data volume to be rapidly increased, and how to use massive space remote sensing data to meet different production and living demands becomes one of the outstanding problems faced by remote sensing application. The information fusion can comprehensively utilize different remote sensing data of a single sensor or a plurality of sensors, and through organic integration of complementary information, ambiguity, incompleteness, uncertainty and errors possibly existing in the explanation of a single signal source to a perceived object or an environment are reduced or suppressed, so that the application efficiency of the remote sensing data is greatly improved, and the reliability in aspects of feature extraction, classification, target identification and the like is improved.
Image fusion is one of important research directions of information fusion, and is to cooperatively utilize two or more images obtained by imaging the same scene to realize information complementation, so as to obtain finer, comprehensive and reliable description of the scene, thereby facilitating cognition and subsequent interpretation processing. The image registration is used as a necessary foundation for image fusion, and geometrical inconsistency of multi-source images is eliminated through a space conversion mechanism, so that a foundation is laid for fusion applications such as subsequent target identification and classification. Current image registration algorithms fall broadly into two main categories: an image registration method based on gray information and an image registration method based on features.
The registration method based on gray information mainly utilizes gray statistical information of images to search for the region with closest similarity between a reference image and an image to be registered so as to determine the optimal geometric transformation parameters. Kern JP proposes a registration algorithm for multispectral remote sensing images based on the mutual information method, which model can control and optimize the selection of image transformation parameters by correlating the curved shape of the mutual information with the frequency domain characteristics of the image. Li Jiao combines the cat swarm algorithm and the normalized cross-correlation matching algorithm, reduces the sensitivity of the original normalized cross-correlation matching algorithm to image rotation and scaling transformation, and effectively improves the accuracy of image registration. In general, the registration algorithm based on gray information directly utilizes the gray information of the image, so that the robustness of the registration method is improved, but the gray correlation is low due to the difference of image data acquisition environments, and the method is difficult to obtain a satisfactory effect.
The feature-based image registration algorithm mainly utilizes salient features such as points, lines, edges or areas of an image target to establish a mapping relation between images, and overcomes the defect of sensitivity of the gray information algorithm. In point feature based image registration algorithms, scale-invariant feature transforms (Scale-invariant Feature Transform, SIFT), accelerated robust features (Speeded Up Robust Features, SURF) and their improved algorithms are most widely used. The classical algorithm-SIFT algorithm based on image features is perfected by Lowe D.G. and is used for extracting interest points which remain unchanged in the image transformation process as feature description by comparing extreme points of images in different scale spaces, and finally, image thought measurement is carried out on the determined local features to realize accurate registration of the images. The strong stability and high efficiency of the SIFT algorithm plays a vital role in the field of image registration. Bay H. et al propose a brand new stable feature acceleration algorithm, SURF algorithm, which combines the advantages of feature detection of the Heisen matrix and feature descriptors based on distribution, and obviously improves the matching precision and speed of images. In order to further reduce the event complexity of the SIFT algorithm, the rectangular area of the Feng Jiajiang key points is changed into a concentric circle area, and the Gaussian blur process of the SIFT algorithm is simplified. Dan Yasun proposes a modified SURF algorithm that limits the sign points to certain areas of the image edges and secondarily constrains the registration points. The improved algorithm not only improves the accuracy of image registration, but also has stronger robustness to image alignment with strong noise. Cheng Dezhi et al determine an image similarity measurement function with better performance according to the feature vector generated by the SIFT algorithm, so that the feature matching accuracy of the SIFT algorithm is effectively improved, the registration time is greatly shortened, and a new method is provided for engineering real-time tasks. Yang Ji et al apply an improved SURF algorithm to the registration process of the remote sensing images, and the accuracy of the SURF algorithm in the registration of the remote sensing images is improved by iteratively removing mismatching points existing in the SURF algorithm. However, the improved SURF algorithm registration is slower. Li et al combine Haar-like features and SURF features to provide a new feature descriptor for registration of remote sensing images, the new feature descriptor adding to the description of the features, and multi-level extraction of the features to obtain more accurate registration results.
The line features have higher-level semantic information than the point features, and parameters such as length, direction and the like can be used for establishing mathematical constraint or similarity measurement when calculating the matching relationship. Therefore, a large number of image matching algorithms based on straight line features have been proposed in recent years. Stamos i. et al propose to use the slope of the feature line and the distribution of the edge profile to obtain the registration parameters, but when there are multiple lines approximating the slope, one-to-one correspondence is not easy to be achieved, and mismatching is not easy to be avoided. Kim Y.S. et al register using point feature and edge feature direction information, but Canny operators are used for both the original remote sensing image and the visible light image, and the characteristic of susceptibility to noise of the remote sensing image is not considered. Su Juan et al construct control points by using image straight line characteristics, design a control point matching degree function, realize automatic registration from coarse to fine, obtain good effect, but consume long time, and are difficult to be suitable for application with strong instantaneity. Li Ying et al construct candidate matching sets by using three straight line edges as reference models under certain constraint conditions, and determine the corresponding relation between the straight line edges on the basis of the candidate matching sets, and experimental results show that the operation time is short, but the constraint conditions are complex to select, a reference with general applicability is not obtained, and when the straight line edges are similar in position, direction and length, mismatching is easy to occur, and more constraint conditions need to be introduced.
In summary, most of the image registration algorithms based on features at the present stage adopt single features, and although the operation time of the algorithms is shortened, the robustness of the algorithms is not good for the remote sensing images with more noise.
Disclosure of Invention
The invention aims to provide an optical remote sensing image registration method and system combining SIFT points and control line pairs, so as to solve one or more technical problems. The method has high speed and good robustness, and can realize the registration of the visible light remote sensing image with the typical linear target.
In order to achieve the above purpose, the invention adopts the following technical scheme:
the invention discloses an optical remote sensing image registration method combining SIFT points and control line pairs, which comprises the following steps:
respectively acquiring linear feature sets of a reference image and an image to be registered;
obtaining an initial registration image based on the linear feature set of the reference image and the linear feature set of the image to be registered; acquiring a linear feature set of the initial registration image;
respectively counting intersecting straight line pairs in a straight line feature set of a reference image and a straight line feature set of an initial registration image to obtain a statistical result, and obtaining a candidate matching set based on the statistical result;
Obtaining a coarse registration result based on the candidate matching set;
and obtaining a precise registration result of the reference image and the rough registration image by using a SIFT key point registration algorithm, and finishing the registration of the optical remote sensing image.
The invention further improves that the step of respectively acquiring the straight line characteristic sets of the reference image and the image to be registered specifically comprises the following steps:
extraction of reference image I by LSD algorithm 1 And an image I to be registered 2 Is an alternative straight line segment set L 1 And L 2 ；
For the alternative straight line segment set L according to the visual saliency theory 1 And L 2 Screening to obtain an effective linear feature set C 1 And C 2 ；
Connecting the effective straight line feature set C by Hough transformation 1 And C 2 Long straight lines of medium fracture to obtain a straight line characteristic set Q 1 And Q 2 。
A further improvement of the invention is that the set of alternative straight line segments L is based on visual saliency theory 1 And L 2 Screening to obtain an effective linear feature set C 1 And C 2 The method comprises the steps of carrying out a first treatment on the surface of the Connecting the effective straight line feature set C by Hough transformation 1 And C 2 Long straight lines of medium fracture to obtain a straight line characteristic set Q 1 And Q 2 The method specifically comprises the following steps:
judging an alternative straight line segment set L 1 Or L 2 If the difference between the slope of a certain line segment C and the slope of the assumed straight line meets the significance criterion represented by the formula (1), judging the slope of the certain line segment C as an available characteristic straight line segment, determining the visual significance characteristic value of the straight line as length, and obtaining an effective straight line characteristic set C of the reference image and the image to be registered 1 And C 2 ；
Using Hough transformation method to obtain effective straight line feature set C 1 And C 2 The straight line segments with the same slope in the two coordinate spaces are mapped to peak points of another coordinate space, and the straight line feature set Q is obtained by connecting the feature straight line segments through statistics of the peak points 1 And Q 2 。
The invention further improves that the initial registration image is obtained based on the linear feature set of the reference image and the linear feature set of the image to be registered; the step of acquiring the linear feature set of the initial registration image specifically includes:
from a rectilinear feature set Q 1 And Q 2 An initial registration image I is obtained by adopting an initial registration method based on feature consistency 3 Extracting and obtaining an initial registration image I by utilizing LSD algorithm 3 Is an alternative straight line segment set L 3 For the alternative straight line segment set L according to the visual saliency theory 3 Screening to obtain an effective linear feature set C 3 Connecting the effective straight line feature set C by adopting Hough transformation 3 Long straight lines of medium fracture to obtain a straight line characteristic set Q 3 。
A further development of the invention is that the linear feature set Q 1 And Q 2 An initial registration image I is obtained by adopting an initial registration method based on feature consistency 3 The method specifically comprises the following steps:
the initial registration method based on feature consistency utilizes a four-parameter affine transformation model, and the model is composed of a scaling factor s, a rotation angle beta and a translation parameter t x And t y Composition, according to this transformation model, the point p= (x, y) in the reference image and its corresponding point in the image to be registeredThe following relationship is satisfied:
wherein the scaling factor s is determined from the resolution of the image; assume thatAnd->Straight line with the same name>And->The included angles between the horizontal axis and the horizontal axis are respectively->And->Then satisfy->
Scaling and rotating the image to be registered according to the scaling factor s and the rotation angle beta, calculating a cross-correlation function of the reference image and the image to be registered, wherein the x-axis and y-axis coordinates corresponding to the peak value are translation distances t x And t y 。
The invention further improves that the step of respectively counting the intersecting straight line pairs in the straight line feature set of the reference image and the straight line feature set of the initial registration image to obtain a statistical result and obtaining a candidate matching set based on the statistical result specifically comprises the following steps:
respectively counting straight line characteristic sets Q 1 And Q 3 Respectively storing the intersecting straight line pairs into the sets LB and LW; if the absolute value of the difference between the included angle of a certain intersecting straight line pair LB in LB and the included angle of a certain intersecting straight line pair LW in LW is smaller than a preset threshold value, the straight line pair LB and the straight line pair LW are considered to be identical-name straight line pairs, and all identical-name straight line pairs meeting the relation form a candidate matching set G.
A further improvement of the invention is that the respective statistical straight line feature set Q 1 And Q 3 The specific steps of storing intersecting line pairs in the sets LB and LW respectively include:
for straight line feature set Q 1 And Q 3 The line segments with the included angles greater than or equal to 30 degrees form a straight line pair, and an intersection point is generated and recorded as:
LB＝{LB ij |0＜i,j≤N,i≠j}，LW＝{LW ij |0＜i,j≤N,i≠j}；
wherein LB is ij ＝{(l i ,l j ,P ij ,θ ij )|0＜i,j≤N,i≠j}；LW ij ＝{(l i ,l j ,P ij ,θ ij )|0＜i,j≤N,i≠j}；
Wherein N represents the number of straight line features in the image; l (L) i And l j Is two straight lines forming a line pair; p (P) ij Is the intersection of the pair of straight lines; θ ij Representing the included angle between the straight line pairs;
if the absolute value of the difference between the included angle of a certain intersecting straight line pair LB in LB and the included angle of a certain intersecting straight line pair LW in LW is smaller than a preset threshold, the straight line pair LB and the straight line pair LW are considered to be identical-name straight line pairs, and the step of forming the candidate matching set G by all identical-name straight line pairs meeting the relation specifically comprises:
assume thatAnd->Is a reference image I 1 The middle intersection is in a straight line pair at a point p, and the included angle is theta b ；/>And->Is the initial registered image I 3 Straight line pair with middle intersection at point p' and included angle theta w ；
Constructing the following similarity measurement criteria, if the straight line pairsAnd straight line pair->If the following expression is satisfied, the points p and p' are considered to be matching control point pairs, and the expression is:
d θ (l b ,l w )＝|θ b -θ w |,0＜d θ (l b ,l w )＜d θmax ，
wherein,represents the angular relationship between the pairs of lines, d θmax A threshold value for characterizing the linear-to-angular relationship;
the straight line pairs meeting the conditions form a candidate matching set, which is expressed as:
wherein,and p i Is a straight line pair in the reference image and an intersection point thereof; />And p i ' is the straight line pair in the image to be registered and its intersection point.
The invention further improves that the step of obtaining a coarse registration result based on the candidate matching set specifically comprises the following steps:
defining homonymy line segment matching matrixes MB and MW for representing the matching relation of homonymy line pairs; constructing a two-way matching relationship according to the homonymous line segment matching matrix MB and MW, and removing repeated matching to obtain a one-to-one matching point set; obtaining a reference image I by a least square method 1 And initial registration image I 3 The optimal transformation matrix between the two is utilized to obtain a coarse registration result I by reverse interpolation 4 。
The invention is further improved in that the definition homonymous line segment matching matrixes MB and MW are used for representing the matching relation of homonymous straight line pairs; constructing a two-way matching relationship according to the homonymous line segment matching matrix MB and MW, removing repeated matching, and obtaining a one-to-one matching point set specifically comprises the following steps:
let N in candidate matching set G 1 Straight lines from the reference image, N 2 The straight line comes from the image to be registered; if the ith element in G contains the jth line in the reference image, mb ij =1; otherwise, mb ij =0; if the ith element in G contains the jth line in the image to be registered, mw ij =1; otherwise, mw ij ＝0；
Matrix MB represents unidirectional matching of the image to be registered to the reference image; matrix MW represents unidirectional matching of the reference image to the image to be registered;
and counting the same rows in the matrixes MB and MW, wherein the same rows represent repeated matching, calculating the distance between the straight lines with the same names corresponding to the same rows, and only reserving the straight line pair with the shortest distance.
The invention discloses an optical remote sensing image registration method combining SIFT points and control line pairs, which comprises the following steps of:
the linear feature set acquisition module is used for respectively acquiring linear feature sets of the reference image and the image to be registered;
the linear feature set acquisition module is used for acquiring an initial registration image according to the linear feature set of the reference image and the linear feature set of the image to be registered; acquiring a linear feature set of the initial registration image;
the candidate matching set acquisition module is used for respectively counting intersecting straight line pairs in the straight line feature set of the reference image and the straight line feature set of the initial registration image to obtain a statistical result, and obtaining a candidate matching set based on the statistical result;
The rough registration result acquisition module is used for acquiring a rough registration result according to the candidate matching set;
the accurate registration result acquisition module is used for obtaining an accurate registration result of the reference image and the rough registration image by using a SIFT key point registration algorithm and finishing the registration of the optical remote sensing image.
Compared with the prior art, the invention has the following beneficial effects:
the invention provides an image registration algorithm based on feature fusion, which can realize advantage complementation and overcome the defect of low robustness of the traditional registration method. Specifically, the invention provides an optical remote sensing image registration algorithm combining SIFT points and control line pairs, which not only utilizes a visual saliency theory to optimize a straight line segment detection algorithm (Line Segment Detection, LSD) so as to accurately position line features and reduce extra line interference in a coarse registration process, but also utilizes the SIFT algorithm to extract point features with rotation and scale invariance so as to realize an accurate registration process. The invention provides a coarse-to-fine matching method, which expands the thought of image matching and improves the matching precision.
Optical remote sensing images with typical linear targets (including airports, buildings, bridges and roads) all contain a large number of linear features, so efficient and accurate linear feature extraction is a key to registration of such optical remote sensing images. The LSD algorithm can realize line segment detection with sub-pixel level precision, linear features in an image can be effectively extracted, but due to the influence of noise, contrast and brightness, breakage and discontinuity are easy to occur in the image by directly detecting the image through the LSD algorithm, so that the invention combines a visual saliency model and Hough change to remove the interference of a miscellaneous line segment, obtain a long straight line, reduce a matching control line pair and improve the matching efficiency.
The line features are ideal matching primitives, but in the actual extraction and matching process, the obtained linear features are often broken, and accurate image registration is difficult to realize by directly utilizing the linear features. Intersecting straight lines tend to have a more stable geometric property, with the most common and invariance property being the directional characteristic of a straight line. Therefore, the method starts from the direction information between the straight line pairs, and finds the intersection points of the possibly matched straight line pairs with the same name as the registration control points, so that more accurate registration is realized.
In the invention, a candidate matching set is formed by acquiring homonymous straight line pairs of a reference image and an image to be registered, a bidirectional matching relation is established by utilizing homonymous line segment matching matrixes, and error matching is removed.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description of the embodiments or the drawings used in the description of the prior art will make a brief description; it will be apparent to those of ordinary skill in the art that the drawings in the following description are of some embodiments of the invention and that other drawings may be derived from them without undue effort.
Fig. 1 is a schematic flow chart of a visible light remote sensing image registration method combining SIFT points and control line pairs according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of distances between pairs of straight lines of the same name in an embodiment of the present invention.
FIG. 3 is a schematic diagram showing a first set of effects of the experimental diagram in comparison, wherein the images are visible light remote sensing images with the size of 993×993; wherein (a) in fig. 3 is a reference image, (b) in fig. 3 is an image to be registered, (c) in fig. 3 is an initial registration image of (b) in fig. 3, (d) in fig. 3 is a LSD straight line segment detection map (lsd_a) in fig. 3, (e) in fig. 3 is a Hough transform detection control map (hf_d) in (d) in fig. 3, (f) in fig. 3 is a LSD straight line segment detection map (vslsd_a) based on the improvement of the present invention, (g) in fig. 3 is a Hough transform detection control map (hf_f) in fig. 3, h) in fig. 3 is a LSD straight line segment detection map (lsd_c) in fig. 3, i) in fig. 3 is a Hough transform detection control map (hf_h) in fig. 3, j) in fig. 3 is a Hough transform detection control map (hf_h) in fig. 3, and (f) in fig. 3 is a checkerboard of the improvement of the present invention, the registration result (f) in fig. 3 is a checkerboard of the fig. 3;
FIG. 4 is a schematic diagram showing a second set of effects of the experimental diagram in comparison, wherein the images are visible light remote sensing images with the size of 220×291; wherein (a) in fig. 4 is a reference image, (b) in fig. 4 is an image to be registered, (c) in fig. 4 is an initial registration image of (b) in fig. 4, (d) in fig. 4 is a LSD straight line segment detection map (lsd_a), (e) in fig. 4 is a Hough transform detection control map (hf_d) in (d) in fig. 4, (f) in fig. 4 is a LSD straight line segment detection map (vslsd_a) based on the improvement of the present invention, (g) in fig. 4 is a Hough transform detection control map (hf_f) in fig. 4, (h) in fig. 4 is a LSD straight line segment detection map (lsd_c) in fig. 4, (i) in fig. 4 is a Hough transform detection control map (hf_h) in fig. 4, (j) in fig. 4 is a Hough transform detection control map (hf_h) based on the improvement of the present invention, and (f) in fig. 4 is a checkerboard registration result (fig. 4) in fig. 4;
FIG. 5 is a schematic diagram showing a third set of effects of the experimental diagram in comparison, wherein the images are visible light remote sensing images with the size of 196×491; where (a) in fig. 5 is a reference image, (b) in fig. 5 is an image to be registered, (c) in fig. 5 is an initial registration image of (b) in fig. 5, (d) in fig. 5 is a LSD straight line segment detection map (lsd_a), (e) in fig. 5 is a Hough transform detection control map (hf_d) in (d) in fig. 5, f) in fig. 5 is a LSD straight line segment detection map (vslsd_a) based on the improvement of the present invention, g) in fig. 5 is a Hough transform detection control map (hf_f) in fig. 5, h) in fig. 5 is a LSD straight line segment detection map (lsd_c) in fig. 5, i) in fig. 5 is a Hough transform detection control map (hf_h) in fig. 5, j) in fig. 5 is a Hough transform detection control map (hf_h) in fig. 5, and (j) in fig. 5 is a checkerboard of the present invention, the registration result is displayed.
Detailed Description
In order to make the purposes, technical effects and technical solutions of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention are clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention; it will be apparent that the described embodiments are some of the embodiments of the present invention. Other embodiments, which may be made by those of ordinary skill in the art based on the disclosed embodiments without undue burden, are within the scope of the present invention.
Referring to fig. 1, a method for registering a visible light remote sensing image by combining SIFT points and control line pairs according to an embodiment of the present invention includes the following steps:
step 1: extraction of reference image I using LSD algorithm 1 And an image I to be registered 2 Is an alternative straight line segment set L 1 And L 2 Alternative straight line segment set L for LSD output according to visual saliency theory 1 And L 2 Screening to remove interference of mixed lines, and extracting effective linear feature set C 1 And C 2 Connecting the filtered linear feature set C by adopting Hough transformation 1 And C 2 Long straight lines of medium fracture to obtain corresponding straight line characteristic sets Q 1 And Q 2 。
Step 2: from a rectilinear feature set Q 1 And Q 2 Using initial feature-based agreementThe registration method obtains an initial registration image I 3 Extracting an initial registration image I using LSD algorithm 3 Is an alternative straight line segment set L 3 Alternative straight line segment set L for LSD output according to visual saliency theory 3 Screening, eliminating interference of mixed lines, and extracting effective linear feature set C 3 Connecting the filtered linear feature set C by adopting Hough transformation 3 Long straight lines of medium fracture to obtain corresponding straight line characteristic sets Q 3 。
Step 3: statistics of Linear feature set Q 1 And Q 3 The intersecting line pairs of (a) are stored in sets LB and LW, respectively. If the absolute value of the difference between the included angle of a certain intersecting straight line pair LB in LB and the included angle of a certain intersecting straight line pair LW in LW is smaller than a threshold value, the straight line pair LB and the straight line pair LW are considered to be identical-name straight line pairs, and all the identical-name straight line pairs meeting the relation form a candidate matching set G.
Step 4: if the candidate matching set G is formed only by using the angular relationship between the pairs of straight lines of the same name, it is difficult to satisfy the "one-to-one" matching relationship, and there is also a possibility of erroneous matching. In order to improve the matching precision and remove repeated matching, the invention establishes a bidirectional matching relationship by using the homonymy line segment matching matrix to realize the accurate matching between homonymy line segments. The homonymy segment matching matrices MB and MW are defined for representing the matching relationship of homonymy line pairs. And constructing a two-way matching relationship according to the homonymous line segment matching matrixes MB and MW, and removing repeated matching to obtain a one-to-one matching point set. Obtaining a reference image I by a least square method 1 And initial registration image I 3 The optimal transformation matrix between the two is utilized to obtain a coarse registration result I by reverse interpolation 4 。
Step 5: obtaining a reference image I by using SIFT key point registration algorithm 1 And coarse registration image I 4 Is a precise registration result I of (1) 5 。
In step 1 of the embodiment of the invention, an alternative straight line segment set L output to the LSD according to the visual saliency theory 1 And L 2 Screening to remove interference of mixed lines, and extracting effective linear feature set C 1 And C 2 Connecting the linear characteristics after screening by adopting Hough transformationSet C 1 And C 2 Long straight lines of medium fracture to obtain corresponding straight line characteristic sets Q 1 And Q 2 The method specifically comprises the following steps: judging an alternative straight line segment set L of LSD output 1 Or L 2 If yes, the difference between the slope of a certain line segment C and the slope of the assumed straight line accords with the significance criterion represented by the following formula, and if yes, the difference indicates that the visual characteristic of human eyes is met when the length of the line segment C is at least length, the line segment C is judged to be a usable characteristic straight line segment, the visual significance characteristic value of the straight line is determined to be length, and a straight line characteristic set C of the reference image and the image to be registered is obtained 1 And C 2 。
Straight line feature set C by Hough transformation method 1 ，C 2 The straight line segments with the same slope are mapped to peak points of another coordinate space, and the characteristic straight line segments are connected to obtain a registration control line set Q through statistics of the peak points 1 And Q 2 。
In step 2 of the embodiment of the present invention, according to the straight line feature set Q 1 And Q 2 An initial registration image I is obtained by adopting an initial registration method based on feature consistency 3 The method specifically comprises the following steps: the initial registration method based on feature consistency utilizes a four-parameter affine transformation model, and the model is composed of a scaling factor s, a rotation angle beta and a translation parameter t x And t y Composition, according to this transformation model, the point p= (x, y) in the reference image and its corresponding point in the image to be registered The following relationship is satisfied:
wherein,
in general, the scaling factor s can be directly derived from the resolution of the image. Assume thatAnd->Straight line with the same name>And->The included angles between the horizontal axis and the horizontal axis are respectively->And->Then satisfy->
Scaling and rotating the image to be registered according to the scaling factor s and the rotation angle beta, and then calculating a cross-correlation function of the reference image and the image to be registered, wherein the coordinates of the x axis and the y axis corresponding to the peak value are the translation distance t x And t y 。
In step 3 of the embodiment of the present invention, the straight line feature set Q is counted 1 And Q 3 The storing of the intersecting line pairs into the sets LB and LW specifically includes: using reference image I 1 And initial registration image I 3 Acquiring a straight line feature set Q 1 And Q 3 For straight line feature set Q 1 And Q 3 The line segments with the included angles not smaller than 30 degrees form a straight line pair, and an intersection point is generated and recorded as:
LB＝{LB ij |0＜i,j≤N,i≠j}，LW＝{LW ij |0＜i,j≤N,i≠j}
wherein LB is ij ＝{(l i ,l j ,P ij ,θ ij )|0＜i,j≤N,i≠j}；LW ij ＝{(l i ,l j ,P ij ,θ ij ) I 0 < i, j is less than or equal to N, i is not equal to j; n represents the number of straight line features in the image; l (L) i And l j Is two straight lines forming a line pair; p (P) ij Is the intersection of the pair of straight lines; θ ij Representing the angle between the pair of lines.
In step 3 of the embodiment of the present invention, if the absolute value of the difference between the included angle of a certain intersecting line pair LB in LB and the included angle of a certain intersecting line pair LW in LW is smaller than a threshold value, the line pair LB and the line pair LW are considered to be homonymous line pairs, and all homonymous line pairs satisfying the above relationship form a candidate matching set G specifically including: assume that And->Is a reference image I 1 The middle intersection is in a straight line pair at a point p, and the included angle is theta b ；/>And->Is the initial registered image I 3 Straight line pair with middle intersection at point p' and included angle theta w . Constructing the following similarity measurement criteria, if the straight line is for +.>And straight line pair->Satisfying the following equation, the points p and p are consideredp' is the matching control point pair, namely:
d θ (l b ,l w )＝|θ b -θ w |,0＜d θ (l b ,l w )＜d θmax
wherein,represents the angular relationship between the pairs of lines, d θmax To characterize the threshold of the linear versus angular relationship.
The pairs of lines that will satisfy the above conditions form a candidate matching set, which can be expressed as:
wherein,and p i Is a straight line pair in the reference image and an intersection point thereof; />And p i ' is the straight line pair in the image to be registered and its intersection point.
In step 3 of the embodiment of the present invention, generating a candidate matching set by using a positional relationship between intersecting straight line pairs specifically includes: if I 1 And I 2 Representing the reference image and the image to be registered respectively,and->Is image I 1 The middle intersection is in a straight line pair at a point p, and the included angle is theta b ；/>And->Is image I 2 Straight line pair with middle intersection at point p' and included angle theta w . Constructing the following similarity measurement criteria, if the straight line is for +.> And straight line pair->If the following equation is satisfied, the points p and p' are considered to be matching control point pairs, i.e.:
d θ (l b ,l w )＝|θ b -θ w |,0＜d θ (l b ,l w )＜d θmax ，
represents the angular relationship between the pairs of lines, d θmax To characterize the threshold of the linear versus angular relationship.
The pairs of lines that will satisfy the above conditions form a candidate matching set, which can be expressed as:
wherein,and p i Is a straight line pair in the reference image and an intersection point thereof; />And p' i Is the straight line pair and the intersection point thereof in the image to be registered.
In step 4 of the embodiment of the invention, homonymous line segment matching matrices MB and MW are defined for representing the matching relationship of homonymous line pairs. Constructing a two-way matching gateway according to the homonymous line segment matching matrixes MB and MWRemoving repeated matching to obtain a one-to-one matching point set specifically comprises: let N in candidate matching set G 1 Straight lines from the reference image, N 2 The straight line comes from the image to be registered. If the ith element in the set G contains the jth line in the reference image, mb ij =1; otherwise, mb ij =0. Similarly, if the ith element in the set G contains the jth line in the image to be registered, mw ij =1; otherwise, mw ij ＝0。
Matrix MB represents unidirectional matching of the image to be registered to the reference image; the matrix MW represents the unidirectional matching of the reference image to the image to be registered. And counting the same rows in the matrixes MB and MW, wherein the same rows represent repeated matching, calculating the distance between the lines with the same names corresponding to the same rows, and only reserving the line pair with the shortest distance.
As shown in the figure 2 of the drawings, And->Is the intersecting straight line pair in the reference image, +.>And->Is the intersecting straight line pair in the image to be registered, supposing +.>And->Is a homonymous line segment>And->Is a line segment with the same name, d 1 Representation->And->Distance between d 2 Representation->And->Distance between, when d 1 And d 2 The assumption is considered correct when the minimum distance between all duplicate matching straight line pairs is the same name. Keep-> And->To correctly match the straight line pairs, the remaining sums are deleted>A pair of lines having a matching relationship. The matching line pairs obtained after screening in the matrixes MB and MW are stored in a set LP b And LP w In calculating LP b Match line pair in (1) and corresponding LP w Whether the match line pairs are identical. If the matching is consistent, the bidirectional matching is successful, and a matched line pair is reserved; otherwise, the two-way matching fails, and the matching line is deletedFor each pair.
Referring to fig. 3 to 5, in the embodiment of the present invention, three sets of remote sensing images are selected for simulation experiments, and compared with an image registration algorithm based on SIFT key points. The experimental CPU is configured as intel Core i7-8750H 2.20GHz, the experimental operating system is Windows 10, and the compiling environment is MATLAB2018b. The experimental results are compared with the graph as follows: fig. (a) is a reference image, fig. (b) is an image to be registered, fig. (c) is an initial registration image of fig. (b), fig. (d) is a Hough transform detection control diagram (hf_d) of fig. (a) LSD straight line segment detection diagram (lsd_a), fig. (e) is a Hough transform detection control diagram (hf_j) of fig. (d), fig. (a) is a LSD straight line segment detection diagram (vslsd_a) improved based on the present invention, fig. (g) is a Hough transform detection control diagram (hf_f) of fig. (f), fig. (h) is a Hough transform detection control diagram (lsd_c) of fig. (c), fig. (i) is a Hough transform detection control diagram (hf_h) of fig. (h), fig. (c) is a d straight line segment detection diagram (vslsd_c) improved based on the present invention, fig. (k) is a Hough transform detection control diagram (hf_j), fig. (l) is a checkerboard display SIFT result, fig. (l) is a checkerboard display registration result; from the three groups of simulation comparative experiments, it can be seen that: the algorithm registration effect provided by the invention is superior to that of the traditional SIFT key point-based registration algorithm. The line segments detected by using only the LSD are easily affected by noise, and problems such as incomplete line segment identification and false identification are easily caused. Although the Hough transformation detection method can overcome the influence of noise, the robustness is not strong because the number of the output line segments is required to be set manually, and the length of the identified line segments exceeds the length of the actual target line segments, so that the control is not easy. The invention provides a novel line segment detection algorithm based on the LSD algorithm, hough transformation and visual saliency model, the noise immunity of the LSD algorithm can be improved, the line segment breakage and other conditions are avoided, and the effectiveness of the algorithm is verified by the experimental result.
The embodiment of the invention discloses an optical remote sensing image registration method combining SIFT points and control line pairs, which comprises the following steps:
the linear feature set acquisition module is used for respectively acquiring linear feature sets of the reference image and the image to be registered;
the linear feature set acquisition module is used for acquiring an initial registration image according to the linear feature set of the reference image and the linear feature set of the image to be registered; acquiring a linear feature set of the initial registration image;
the candidate matching set acquisition module is used for respectively counting intersecting straight line pairs in the straight line feature set of the reference image and the straight line feature set of the initial registration image to obtain a statistical result, and obtaining a candidate matching set based on the statistical result;
the rough registration result acquisition module is used for acquiring a rough registration result according to the candidate matching set;
the accurate registration result acquisition module is used for obtaining an accurate registration result of the reference image and the rough registration image by using a SIFT key point registration algorithm and finishing the registration of the optical remote sensing image.
The invention aims at the registration method research by taking an optical remote sensing image containing typical linear targets (such as airports, buildings, bridges, roads and the like) as an object.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the above embodiments, one skilled in the art may make modifications and equivalents to the specific embodiments of the present invention, and any modifications and equivalents not departing from the spirit and scope of the present invention are within the scope of the claims of the present invention.
Claims (6)
1. An optical remote sensing image registration method combining SIFT points and control line pairs is characterized by comprising the following steps of:
respectively acquiring linear feature sets of a reference image and an image to be registered;
obtaining an initial registration image based on the linear feature set of the reference image and the linear feature set of the image to be registered; acquiring a linear feature set of the initial registration image;
respectively counting intersecting straight line pairs in a straight line feature set of a reference image and a straight line feature set of an initial registration image to obtain a statistical result, and obtaining a candidate matching set based on the statistical result;
obtaining a coarse registration result based on the candidate matching set;
obtaining a precise registration result of the reference image and the rough registration image by using a SIFT key point registration algorithm, and finishing optical remote sensing image registration;
the step of respectively acquiring the linear feature sets of the reference image and the image to be registered specifically comprises the following steps:
extraction of reference image I by LSD algorithm 1 And an image I to be registered 2 Is an alternative straight line segment set L 1 And L 2 ；
For the alternative straight line segment set L according to the visual saliency theory 1 And L 2 Screening to obtain an effective linear feature set C 1 And C 2 ；
Connecting the effective straight line feature set C by Hough transformation 1 And C 2 Long straight lines of medium fracture to obtain a straight line characteristic set Q 1 And Q 2 ；
The linear feature set based on the reference image and the linear feature set of the image to be registered obtain an initial registration image; the step of acquiring the linear feature set of the initial registration image specifically includes:
from a rectilinear feature set Q 1 And Q 2 An initial registration image I is obtained by adopting an initial registration method based on feature consistency 3 Extracting and obtaining an initial registration image I by utilizing LSD algorithm 3 Is an alternative straight line segment set L 3 For the alternative straight line segment set L according to the visual saliency theory 3 Screening to obtain an effective linear feature set C 3 Connecting the effective straight line feature set C by adopting Hough transformation 3 Long straight lines of medium fracture to obtain a straight line characteristic set Q 3 ；
The step of respectively counting intersecting straight line pairs in the straight line feature set of the reference image and the straight line feature set of the initial registration image to obtain a statistical result, and obtaining a candidate matching set based on the statistical result specifically comprises the following steps:
respectively counting straight line characteristic sets Q 1 And Q 3 Respectively storing the intersecting straight line pairs into the sets LB and LW; if the absolute value of the difference between the included angle of a certain intersecting straight line pair LB in LB and the included angle of a certain intersecting straight line pair LW in LW is smaller than a preset threshold value, the straight line pair LB and the straight line pair LW are considered to be identical-name straight line pairs, and all identical-name straight line pairs meeting the relation form a candidate matching set G;
The step of obtaining a coarse registration result based on the candidate matching set specifically includes:
defining homonymy line segment matching matrixes MB and MW for representing the matching relation of homonymy line pairs; constructing a two-way matching relationship according to the homonymous line segment matching matrix MB and MW, and removing repeated matching to obtain a one-to-one matching point set; obtaining a reference image I by a least square method 1 And initial registration image I 3 The optimal transformation matrix between the two is utilized to obtain a coarse registration result I by reverse interpolation 4 。
2. The method of claim 1, wherein the candidate straight line segment set L is based on visual saliency theory 1 And L 2 Screening to obtain an effective linear feature set C 1 And C 2 The method comprises the steps of carrying out a first treatment on the surface of the Connecting the effective straight line feature set C by Hough transformation 1 And C 2 Long straight lines of medium fracture to obtain a straight line characteristic set Q 1 And Q 2 The method specifically comprises the following steps:
judging an alternative straight line segment set L 1 Or L 2 If the difference between the slope of a certain line segment C and the slope of the assumed straight line meets the significance criterion represented by the formula (1), judging the slope of the certain line segment C as an available characteristic straight line segment, determining the visual significance characteristic value of the straight line as length, and obtaining an effective straight line characteristic set C of the reference image and the image to be registered 1 And C 2 ；
Using Hough transformation method to obtain effective straight line feature set C 1 And C 2 The straight line segments with the same slope in the two coordinate spaces are mapped to peak points of another coordinate space, and the straight line feature set Q is obtained by connecting the feature straight line segments through statistics of the peak points 1 And Q 2 。
3. The method of claim 1, wherein the method is based on a set of straight line features Q 1 And Q 2 An initial registration image I is obtained by adopting an initial registration method based on feature consistency 3 The method specifically comprises the following steps:
the initial registration method based on feature consistency utilizes a four-parameter affine transformation model, and the model is composed of a scaling factor s, a rotation angle beta and a translation parameter t x And t y Composition, according to this transformation model, the point p= (x, y) in the reference image and its corresponding point in the image to be registeredThe following relationship is satisfied:
wherein the scaling factor s is determined from the resolution of the image; assume thatAnd->Straight line with the same name>And->The included angles between the horizontal axis and the horizontal axis are respectively->And->Then satisfy->
Scaling and rotating the image to be registered according to the scaling factor s and the rotation angle beta, calculating a cross-correlation function of the reference image and the image to be registered, wherein the x-axis and y-axis coordinates corresponding to the peak value are translation distances t x And t y 。
4. The method of claim 1, wherein the respective sets of statistical rectilinear features Q 1 And Q 3 The specific steps of storing intersecting line pairs in the sets LB and LW respectively include:
for straight line feature set Q 1 And Q 3 The line segments with the included angles greater than or equal to 30 degrees form a straight line pair, and an intersection point is generated and recorded as:
LB＝{LB ij |0＜i,j≤N,i≠j}，LW＝{LW ij |0＜i,j≤N,i≠j}；
wherein LB is ij ＝{(l i ,l j ,P ij ,θ ij )|0＜i,j≤N,i≠j}；LW ij ＝{(l i ,l j ,P ij ,θ ij )|0＜i,j≤N,i≠j}；
Wherein N represents the number of straight line features in the image; l (L) i And l j Is two straight lines forming a line pair; p (P) ij Is straight lineIntersection of pairs; θ ij Representing the included angle between the straight line pairs;
if the absolute value of the difference between the included angle of a certain intersecting straight line pair LB in LB and the included angle of a certain intersecting straight line pair LW in LW is smaller than a preset threshold, the straight line pair LB and the straight line pair LW are considered to be identical-name straight line pairs, and the step of forming the candidate matching set G by all identical-name straight line pairs meeting the relation specifically comprises:
assume thatAnd->Is a reference image I 1 The middle intersection is in a straight line pair at a point p, and the included angle is theta b ；/>And->Is the initial registered image I 3 Straight line pair with middle intersection at point p' and included angle theta w ；
Constructing the following similarity measurement criteria, if the straight line pairsAnd straight line pair->If the following expression is satisfied, the points p and p' are considered to be matching control point pairs, and the expression is:
d θ (l b ,l w )＝|θ b -θ w |,0＜d θ (l b ,l w )＜d θmax ，
Wherein,represents the angular relationship between the pairs of lines, d θmax A threshold value for characterizing the linear-to-angular relationship;
the straight line pairs meeting the conditions form a candidate matching set, which is expressed as:
wherein,and p i Is a straight line pair in the reference image and an intersection point thereof; />And p i ' is the straight line pair in the image to be registered and its intersection point.
5. The method of claim 1, wherein the defining homonymous line segment matching matrices MB and MW is used for representing matching relations of homonymous straight line pairs; constructing a two-way matching relationship according to the homonymous line segment matching matrix MB and MW, removing repeated matching, and obtaining a one-to-one matching point set specifically comprises the following steps:
let N in candidate matching set G 1 Straight lines from the reference image, N 2 The straight line comes from the image to be registered; if the ith element in G contains the jth line in the reference image, mb ij =1; otherwise, mb ij =0; if the ith element in G contains the jth line in the image to be registered, mw ij =1; otherwise, mw ij ＝0；
Matrix MB represents unidirectional matching of the image to be registered to the reference image; matrix MW represents unidirectional matching of the reference image to the image to be registered;
and counting the same rows in the matrixes MB and MW, wherein the same rows represent repeated matching, calculating the distance between the straight lines with the same names corresponding to the same rows, and only reserving the straight line pair with the shortest distance.
6. An optical remote sensing image registration method combining SIFT points and control line pairs is characterized by comprising the following steps of:
the linear feature set acquisition module is used for respectively acquiring linear feature sets of the reference image and the image to be registered;
the linear feature set acquisition module is used for acquiring an initial registration image according to the linear feature set of the reference image and the linear feature set of the image to be registered; acquiring a linear feature set of the initial registration image;
the candidate matching set acquisition module is used for respectively counting intersecting straight line pairs in the straight line feature set of the reference image and the straight line feature set of the initial registration image to obtain a statistical result, and obtaining a candidate matching set based on the statistical result;
the rough registration result acquisition module is used for acquiring a rough registration result according to the candidate matching set;
the accurate registration result acquisition module is used for obtaining an accurate registration result of the reference image and the rough registration image by using a SIFT key point registration algorithm and finishing the registration of the optical remote sensing image;
the step of respectively acquiring the linear feature sets of the reference image and the image to be registered specifically comprises the following steps:
Extraction of reference image I by LSD algorithm 1 And an image I to be registered 2 Is an alternative straight line segment set L 1 And L 2 ；
For the alternative straight line segment set L according to the visual saliency theory 1 And L 2 Screening to obtain an effective linear feature set C 1 And C 2 ；
Using Hough transformConnecting an effective straight line feature set C 1 And C 2 Long straight lines of medium fracture to obtain a straight line characteristic set Q 1 And Q 2 ；
The linear feature set based on the reference image and the linear feature set of the image to be registered obtain an initial registration image; the step of acquiring the linear feature set of the initial registration image specifically includes:
from a rectilinear feature set Q 1 And Q 2 An initial registration image I is obtained by adopting an initial registration method based on feature consistency 3 Extracting and obtaining an initial registration image I by utilizing LSD algorithm 3 Is an alternative straight line segment set L 3 For the alternative straight line segment set L according to the visual saliency theory 3 Screening to obtain an effective linear feature set C 3 Connecting the effective straight line feature set C by adopting Hough transformation 3 Long straight lines of medium fracture to obtain a straight line characteristic set Q 3 ；
The step of respectively counting intersecting straight line pairs in the straight line feature set of the reference image and the straight line feature set of the initial registration image to obtain a statistical result, and obtaining a candidate matching set based on the statistical result specifically comprises the following steps:
Respectively counting straight line characteristic sets Q 1 And Q 3 Respectively storing the intersecting straight line pairs into the sets LB and LW; if the absolute value of the difference between the included angle of a certain intersecting straight line pair LB in LB and the included angle of a certain intersecting straight line pair LW in LW is smaller than a preset threshold value, the straight line pair LB and the straight line pair LW are considered to be identical-name straight line pairs, and all identical-name straight line pairs meeting the relation form a candidate matching set G;
the step of obtaining a coarse registration result based on the candidate matching set specifically includes:
defining homonymy line segment matching matrixes MB and MW for representing the matching relation of homonymy line pairs; constructing a two-way matching relationship according to the homonymous line segment matching matrix MB and MW, and removing repeated matching to obtain a one-to-one matching point set; obtaining a reference image I by a least square method 1 And initial registration image I 3 The optimal transformation matrix between the two is utilized to obtain a coarse registration result I by reverse interpolation 4 。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110287621.2A CN113096168B (en) | 2021-03-17 | 2021-03-17 | Optical remote sensing image registration method and system combining SIFT points and control line pairs |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110287621.2A CN113096168B (en) | 2021-03-17 | 2021-03-17 | Optical remote sensing image registration method and system combining SIFT points and control line pairs |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113096168A CN113096168A (en) | 2021-07-09 |
| CN113096168B true CN113096168B (en) | 2024-04-02 |
Family
ID=76668426
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110287621.2A Active CN113096168B (en) | 2021-03-17 | 2021-03-17 | Optical remote sensing image registration method and system combining SIFT points and control line pairs |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113096168B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103514606A (en) * | 2013-10-14 | 2014-01-15 | 武汉大学 | Heterology remote sensing image registration method |
| CN104992431A (en) * | 2015-06-19 | 2015-10-21 | 北京邮电大学 | Method and device for multispectral image registration |
| WO2020206903A1 (en) * | 2019-04-08 | 2020-10-15 | 平安科技（深圳）有限公司 | Image matching method and device, and computer readable storage medium |
-
2021
- 2021-03-17 CN CN202110287621.2A patent/CN113096168B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103514606A (en) * | 2013-10-14 | 2014-01-15 | 武汉大学 | Heterology remote sensing image registration method |
| CN104992431A (en) * | 2015-06-19 | 2015-10-21 | 北京邮电大学 | Method and device for multispectral image registration |
| WO2020206903A1 (en) * | 2019-04-08 | 2020-10-15 | 平安科技（深圳）有限公司 | Image matching method and device, and computer readable storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 一种可靠的高分辨率光学卫星遥感影像匹配方法;戴激光;宋伟东;李建军;;遥感信息(第01期);全文 * |
| 基于SIFT的全自动遥感图像配准算法;余婷;厉小润;;机电工程(第01期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113096168A (en) | 2021-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | Point cloud registration based on one-point ransac and scale-annealing biweight estimation | |
| Mousavi et al. | A two-step descriptor-based keypoint filtering algorithm for robust image matching | |
| Yang et al. | Fast and accurate vanishing point detection and its application in inverse perspective mapping of structured road | |
| Shekhar et al. | Multisensor image registration by feature consensus | |
| Hu et al. | Efficient and automatic plane detection approach for 3-D rock mass point clouds | |
| CN112419497A (en) | Monocular vision-based SLAM method combining feature method and direct method | |
| Hu et al. | An automatic 3D registration method for rock mass point clouds based on plane detection and polygon matching | |
| CN116485856B (en) | Unmanned aerial vehicle image geographic registration method based on semantic segmentation and related equipment | |
| Zhu et al. | A review of 6d object pose estimation | |
| Liu et al. | Image-translation-based road marking extraction from mobile laser point clouds | |
| Cao et al. | Parallel K nearest neighbor matching for 3D reconstruction | |
| Chen et al. | Robust local structure visualization for remote sensing image registration | |
| Zhang et al. | Efficient Pairwise 3-D Registration of Urban Scenes via Hybrid Structural Descriptors. | |
| Li et al. | TextSLAM: Visual SLAM With Semantic Planar Text Features | |
| CN110246165A (en) | It improves visible images and SAR image matches the method and system of Quasi velosity | |
| Zhang et al. | Accurate real-time SLAM based on two-step registration and multimodal loop detection | |
| Zheng et al. | Lattice-point mutually guided ground-to-aerial feature matching for urban scene images | |
| CN117870659A (en) | Visual inertial integrated navigation algorithm based on dotted line characteristics | |
| CN113704276A (en) | Map updating method and device, electronic equipment and computer readable storage medium | |
| Hwang et al. | Optimized clustering scheme-based robust vanishing point detection | |
| CN113096168B (en) | Optical remote sensing image registration method and system combining SIFT points and control line pairs | |
| Chen et al. | Extracting and matching lines of low-textured region in close-range navigation for tethered space robot | |
| Wu et al. | Enhanced 3D mapping with an RGB-D sensor via integration of depth measurements and image sequences | |
| Yang et al. | Fast urban aerial image matching based on rectangular building extraction | |
| Ren et al. | SAR image matching method based on improved SIFT for navigation system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |