CN116129146B - Heterogeneous image matching method and system based on local feature consistency - Google Patents
Heterogeneous image matching method and system based on local feature consistency Download PDFInfo
- Publication number
- CN116129146B CN116129146B CN202310317745.XA CN202310317745A CN116129146B CN 116129146 B CN116129146 B CN 116129146B CN 202310317745 A CN202310317745 A CN 202310317745A CN 116129146 B CN116129146 B CN 116129146B
- Authority
- CN
- China
- Prior art keywords
- key
- image
- phase
- feature
- representing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a heterogeneous image matching method and system based on local feature consistency, belongs to the technical field of image processing and remote sensing image analysis, and solves the problems of low heterogeneous image matching precision and efficiency in the prior art. The invention extracts key points of each heterogeneous image based on a characteristic extraction method of local information; extracting phase information of each heterogeneous image, and establishing a local phase feature descriptor of each heterogeneous image based on the key points of each heterogeneous image and local information of the neighborhood of the key points of each heterogeneous image; filtering the key points by using a mean shift algorithm based on the key points of each heterogeneous image and the phase characteristic descriptors thereof, and eliminating repeated modes in the key points; based on the key points and the feature descriptors of the two heterogeneous images to be matched after the repeated mode is removed, performing feature matching of the two heterogeneous images by adopting a method for calculating the Euclidean distance of the feature descriptors; and then the nearest neighbor ratio adjusting method is adopted to eliminate mismatching. The method is used for precisely matching the heterogeneous images.
Description
Technical Field
A heterogeneous image matching method and system based on local feature consistency are used for realizing accurate matching of image data acquired by different types of imaging sensors, and belong to the technical fields of image processing and remote sensing image analysis.
Background
In image acquisition, a single type of sensor can only obtain information of a certain aspect of a scene, a plurality of types of sensors can obtain information of various aspects of the scene, for example, an optical image can obtain spectrum information and texture information of the scene, an infrared image can obtain thermal radiation information of the scene, and a laser radar image can obtain distance information and reflection intensity information of the scene. The premise of synthesizing complementary information in the heterologous image is that precise matching of the heterologous image needs to be achieved.
Heterologous image matching has been a hotspot and difficulty problem in the field of image processing. For the heterogeneous images of the same scene, due to the imaging principle difference of the sensor, obvious nonlinear differences exist between the heterogeneous images in image texture and spectral characteristics. Therefore, the conventional heterogeneous image matching is mostly realized by adopting a key point matching method based on image local characteristics, firstly, key point detection is carried out, then, characteristic description is carried out on key points by utilizing local gradient statistical characteristics near the key points, and finally, euclidean distance between two image key point characteristic descriptors is calculated to carry out key point matching. The method has limited capability of overcoming the spectral characteristics of the heterogeneous image and the nonlinear differences of the image textures due to the adoption of the gradient information of the image. In addition, because only image local information is utilized in image matching, the image matching is easily interfered by repeated patterns in the image, a plurality of repeated key points are generated, and mismatching is caused.
In summary, the heterologous image matching method in the prior art has the following technical problems:
1. is sensitive to image noise and is easily interfered by secondary detail information in an image, so that the robustness of a feature descriptor is low;
2. the capability of overcoming the nonlinear difference of the spectrum characteristic and the image texture of the heterogeneous image is limited, so that the problems of low matching precision and efficiency are easily caused;
3. only image local information is utilized in image matching, and the image local information is easily interfered by repeated modes in the image, so that a plurality of repeated key point feature descriptors are generated, and mismatching is caused.
Disclosure of Invention
Aiming at the problems of the researches, the invention aims to provide a heterogeneous image matching method and system based on local feature consistency, which solve the problems that the prior art is sensitive to image noise and is easily interfered by secondary detail information in an image, so that the robustness of a feature descriptor is low, and the like.
In order to achieve the above purpose, the invention adopts the following technical scheme:
a heterologous image matching method based on local feature consistency comprises the following steps:
step 1: and (3) key point extraction: extracting key points of each heterogeneous image by a feature extraction method based on local information;
step 2: phase characterization: extracting phase information of each heterogeneous image by using a Log-Gabor filter, and establishing a local phase feature descriptor of each heterogeneous image based on the key points of each heterogeneous image and local information of a neighborhood thereof;
step 3: and (5) key point filtering: filtering the key points by using a mean shift algorithm based on the key points of each heterogeneous image and the phase characteristic descriptors thereof, and eliminating repeated modes in the key points;
step 4: key point matching: based on the key points and the phase feature descriptors of the two heterogeneous images to be matched after the repeated mode is eliminated, the method for calculating the Euclidean distance of the phase feature descriptors is adopted to carry out feature matching on the two heterogeneous images, and after matching, the nearest neighbor ratio adjusting method is adopted to further eliminate mismatching.
Further, the specific steps of the step 1 are as follows:
step 1.1: using a circular template with a radius of 3 pixels, enabling the center point of the circular template to slide on each heterologous image, traversing the whole heterologous image, and when the center of the circular template is overlapped with the pixels of the heterologous image, using one pixelpThe center is a circular neighborhood with radius of 3 pixels, and 16 pixel points are arranged on the circular neighborhoodp 1 、p 2 、...、p 16 ）；
Step 1.2: calculating key candidate points: defining a threshold valuetSequentially calculating the slavep 1 To the point ofp 16 These 16 points and center pixelspIf there are at least 9 consecutive pixel differences exceeding the thresholdtThen the center pixel ispDefining as key candidate points;
step 1.3: non-maximum threshold: by key candidate pointspIs the middle warmerIn a 5×5 neighborhood of the heart, if there are multiple key candidate points, the formula is used in turnThe pixels exceeding the threshold value in each key candidate point are subjected to difference accumulation summation to obtain all the key candidate pointssValue, finally remainsThe key candidate point with the largest value is taken as the key point of each heterogeneous image.
Further, the specific steps of the step 2 are as follows:
step 2.1: calculating a structural feature map of each heterologous image: by using structural characteristic formulasCalculating structural feature map of each heterologous image, wherein +.>Representing a structural feature map, ">Representing an input image +.>Representing the radius of the input heterologous image as +.>Gray mean of local neighborhood of>Represents the radius of a circular template, m is 5, (-)>，/>) A pixel point representing a heterologous image,iandjrespectively representxAndypixel offset values in the direction, max () represents a maximum value taking function;
step 2.2: using Log-Gabor filtersLGTransforming the structural feature diagram to obtain a phase consistency model, and firstly utilizing a formulaObtaining odd-numbered parts of the heterologous image in the frequency domainOAnd an even number partEThen based on odd partsOAnd an even number partE，Using the formula->Obtaining amplitude components of a heterologous imageAAnd phase componentθFinally based on amplitude componentAAnd phase componentθ，Using the formulaCalculating a phase consistency model, wherein +.>Representation->Filter even component extractor, < >>Representation->Odd component extractor of filter, < >>Representing pixel dot +.>Weight parameter of->Representing estimated noise threshold, +_>Representing the scale in a Log-Gabor filtersAnd direction ofoUpper pixel +.>Variance function of phase component of>Representing the scale in a Log-Gabor filtersAnd direction ofoUpper pixel +.>Wherein the subscriptsAndorepresenting the dimensions and direction of the Log-Gabor filter, respectively, < >>Representing a small positive number to avoid denominator 0;
step 2.3: based on the structural feature map and the direction in the phase consistency modeloAnd respectively setting the filter directions as 0 degree, 45 degree, 90 degree and 135 degree to obtain 4 phase feature images, selecting a 4×4 neighborhood around the key point, arranging the phase feature values into a 4-dimensional feature vector according to the 4 directions of 0 degree, 45 degree, 90 degree and 135 degree for each pixel in the neighborhood, and combining the feature vectors of 16 pixels to obtain a final 64-dimensional feature vector, namely a phase feature descriptor of the key point.
Further, the specific steps of the step 3 are as follows:
step 3.1: clustering key points: based on the phase feature descriptors, calculating a mean shift quantity by using a mean shift algorithm to obtain a clustering center to cluster the phase feature descriptors of the key points, wherein the mean shift quantity is expressed by a formulaCalculating, wherein->Phase feature descriptors representing keypoints, +.>Indicate->Phase characteristic descriptor of key point, +.>The number of key points is represented and,grepresenting a kernel function->Representing the mean shift amount;
step 3.2: repeated keypoint filtration: setting a distance threshold and using Euclidean distance formulaCalculating the distance between any two clustering centers, according to priori knowledge, if two key points belong to a repeated mode, obtaining a smaller distance, removing the repeated key points smaller than a threshold value, and reserving a key point set with the distance larger than the threshold value, wherein the key point set with the distance larger than the threshold value is selected by the user>Indicate->Phase characteristic descriptor of key point, +.>Indicate->Phase characteristic descriptors of key points.
Further, the specific steps of the step 4 are as follows:
step 4.1: key point matching: based on the key points of two to-be-matched heterologous images after the repeated mode is eliminated, taking one to-be-matched heterologous image as a reference image, respectively traversing each key point in the reference image through all key points in the other to-be-matched heterologous image, calculating Euclidean distance of phase feature descriptors of the key points of the two images, and reserving two values with the minimum distance as matching candidate points;
step 4.2: mismatch cancellation: using the formulaCalculating whether the Euclidean distance of the nearest neighbor point pair of each group of matching candidate points is smaller than the Euclidean distance ratio adjustment of the next nearest neighbor point pairPost-ganglion values, whereγRepresents the adjustment ratio->Hours represent the Euclidean distance of the nearest neighbor pair,/->And (3) representing the Euclidean distance of the next-neighbor point pair, and reserving the point pair with the Euclidean distance of the nearest-neighbor point pair smaller than the value after Euclidean distance ratio adjustment of the next-neighbor point pair as a final heterologous image characteristic matching point pair.
A local feature consistency-based heterologous image matching system, comprising:
and a key point extraction module: extracting key points of each heterogeneous image by a feature extraction method based on local information;
and the phase characteristic description module is used for: extracting phase information of each heterogeneous image by using a Log-Gabor filter, and establishing a local phase feature descriptor of each heterogeneous image based on the key points of each heterogeneous image and local information of a neighborhood thereof;
and the key point filtering module is used for: filtering the key points by using a mean shift algorithm based on the key points of each heterogeneous image and the phase characteristic descriptors thereof, and eliminating repeated modes in the key points;
and a key point matching module: based on the key points and the phase feature descriptors of the two heterogeneous images to be matched after the repeated mode is eliminated, the method for calculating the Euclidean distance of the phase feature descriptors is adopted to carry out feature matching on the two heterogeneous images, and after matching, the nearest neighbor ratio adjusting method is adopted to further eliminate mismatching.
Further, the specific implementation steps of the key point extraction module are as follows:
step 1.1: using a circular template with a radius of 3 pixels, enabling the center point of the circular template to slide on each heterologous image, traversing the whole heterologous image, and when the center of the circular template is overlapped with the pixels of the heterologous image, using one pixelpThe center is a circular neighborhood with radius of 3 pixels, and 16 pixel points are arranged on the circular neighborhoodp 1 、p 2 、...、p 16 ）；
Step 1.2: calculating key candidate points: defining a threshold valuetSequentially calculating the slavep 1 To the point ofp 16 These 16 points and center pixelspIf there are at least 9 consecutive pixel differences exceeding the thresholdtThen the center pixel ispDefining as key candidate points;
step 1.3: non-maximum threshold: by key candidate pointspIn a 5×5 neighborhood with the center, if there are multiple key candidate points, the formula is used in turnThe pixels exceeding the threshold value in each key candidate point are subjected to difference accumulation summation to obtain all the key candidate pointssValue, finally remainsThe key candidate point with the largest value is taken as the key point of each heterogeneous image.
Further, the specific implementation steps of the phase characteristic description module are as follows:
step 2.1: calculating a structural feature map of each heterologous image: by using structural characteristic formulasCalculating structural feature map of each heterologous image, wherein +.>Representing a structural feature map, ">Representing an input image +.>Representing the radius of the input heterologous image as +.>Gray mean of local neighborhood of>Represents the radius of a circular template, m is 5, (-)>，/>) A pixel point representing a heterologous image,iandjrespectively representxAndypixel offset values in the direction, max () represents a maximum value taking function;
step 2.2: using Log-Gabor filtersLGTransforming the structural feature diagram to obtain a phase consistency model, and firstly utilizing a formulaObtaining odd-numbered parts of the heterologous image in the frequency domainOAnd an even number partEThen based on odd partsOAnd an even number partE，Using the formula->Obtaining amplitude components of a heterologous imageAAnd phase componentθFinally based on amplitude componentAAnd phase componentθ，Using the formulaCalculating a phase consistency model, wherein +.>Representation->Filter even component extractor, < >>Representation->Odd component extractor of filter, < >>Representing pixel dot +.>Weight parameter of->Representing estimated noise threshold, +_>Representing the scale in a Log-Gabor filtersAnd direction ofoUpper pixel +.>Variance function of phase component of>Representing the scale in a Log-Gabor filtersAnd direction ofoUpper pixel +.>Wherein the subscriptsAndorepresenting the dimensions and direction of the Log-Gabor filter, respectively, < >>Representing a small positive number to avoid denominator 0;
step 2.3: based on the structural feature map and the direction in the phase consistency modeloAnd respectively setting the filter directions as 0 degree, 45 degree, 90 degree and 135 degree to obtain 4 phase feature images, selecting a 4×4 neighborhood around the key point, arranging the phase feature values into a 4-dimensional feature vector according to the 4 directions of 0 degree, 45 degree, 90 degree and 135 degree for each pixel in the neighborhood, and combining the feature vectors of 16 pixels to obtain a final 64-dimensional feature vector, namely a phase feature descriptor of the key point.
Further, the specific implementation steps of the key point filtering module are as follows:
step 3.1: clustering key points: based on the phase feature descriptors, calculating a mean shift quantity by using a mean shift algorithm to obtain a clustering center to cluster the phase feature descriptors of the key points, wherein the mean shift quantity is expressed by a formulaCalculating, wherein->Phase feature descriptors representing keypoints, +.>Indicate->Phase characteristic descriptor of key point, +.>The number of key points is represented and,grepresenting a kernel function->Representing the mean shift amount;
step 3.2: repeated keypoint filtration: setting a distance threshold and using Euclidean distance formulaCalculating the distance between any two clustering centers, according to priori knowledge, if two key points belong to a repeated mode, obtaining a smaller distance, removing the repeated key points smaller than a threshold value, and reserving a key point set with the distance larger than the threshold value, wherein the key point set with the distance larger than the threshold value is selected by the user>Indicate->Phase characteristic descriptor of key point, +.>Indicate->Phase characteristic descriptors of key points.
Further, the specific implementation steps of the key point matching module are as follows:
step 4.1: key point matching: based on the key points of two to-be-matched heterologous images after the repeated mode is eliminated, taking one to-be-matched heterologous image as a reference image, respectively traversing each key point in the reference image through all key points in the other to-be-matched heterologous image, calculating Euclidean distance of phase feature descriptors of the key points of the two images, and reserving two values with the minimum distance as matching candidate points;
step 4.2: mismatch cancellation: using the formulaCalculating whether the Euclidean distance of the nearest neighbor point pair of each group of matching candidate points is smaller than the Euclidean distance ratio adjusted value of the next nearest neighbor point pair, whereinγRepresents the adjustment ratio->Hours represent the Euclidean distance of the nearest neighbor pair,/->And (3) representing the Euclidean distance of the next-neighbor point pair, and reserving the point pair with the Euclidean distance of the nearest-neighbor point pair smaller than the value after Euclidean distance ratio adjustment of the next-neighbor point pair as a final heterologous image characteristic matching point pair.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention provides a method for establishing a structural feature map based on local contrast, which reserves strong structural information with high consistency in a heterogeneous image, ignores details such as brightness, color and the like of the image with larger difference, and can greatly improve the robustness of a phase feature descriptor of a key point;
2. the method is based on the structural feature diagram, phase consistency information is extracted by utilizing Log-Gabor transformation, and compared with gradient information, interference such as spectrum characteristics of heterogeneous images, nonlinear differences of image textures and the like can be effectively overcome, and matching precision and efficiency of pairwise heterogeneous images are improved;
3. after the characteristic description, the invention provides a key point filtering technology based on mean shift clustering, which can effectively remove most of key points in a repeated mode before matching, and greatly improve the accuracy and efficiency of the matching of the heterogeneous images;
4. the invention adopts nearest neighbor ratio adjustment in feature matching, can effectively reduce the number of mismatching and improve the matching precision of heterogeneous images;
5. the invention provides a heterogeneous image matching method based on local feature consistency, which utilizes local contrast and Log-Gabor transformation to reduce interference of image noise and secondary details on heterogeneous image matching; compared with the existing method, the technical scheme provided by the invention can obtain the matching result of the heterologous image with high robustness, high efficiency and high precision under the conditions that noise interference, nonlinear difference, repeated mode interference and the like exist in the heterologous image.
Drawings
FIG. 1 is a schematic flow chart of the present invention.
Detailed Description
The invention will be further described with reference to the drawings and detailed description.
A heterologous image matching method based on local feature consistency comprises the following steps:
step 1: and (3) key point extraction: extracting key points of each heterogeneous image by a feature extraction method based on local information; the method comprises the following specific steps:
step 1.1: using a circular template with a radius of 3 pixels, enabling the center point of the circular template to slide on each heterologous image, traversing the whole heterologous image, and when the center of the circular template is overlapped with the pixels of the heterologous image, using one pixelpThe center is a circular neighborhood with radius of 3 pixels, and 16 pixel points are arranged on the circular neighborhoodp 1 、p 2 、...、p 16 ）；
Step 1.2: calculating key candidate points: defining a threshold valuetSequentially calculating the slavep 1 To the point ofp 16 These 16 points and center pixelspIf there are at least 9 consecutive pixel differences exceeding the thresholdtThen the center pixel ispDefining as key candidate points;
step 1.3: non-maximum threshold: by key candidate pointspOne at the centerIn the 5×5 neighborhood, if there are multiple key candidate points, the formula is used in turnThe pixels exceeding the threshold value in each key candidate point are subjected to difference accumulation summation to obtain all the key candidate pointssValue, finally remainsThe key candidate point with the largest value is taken as the key point of each heterogeneous image.
Step 2: phase characterization: extracting phase information of each heterogeneous image by using a Log-Gabor filter, and establishing a local phase feature descriptor of each heterogeneous image based on the key points of each heterogeneous image and local information of a neighborhood thereof; the method comprises the following specific steps:
step 2.1: calculating a structural feature map of each heterologous image: by using structural characteristic formulasCalculating structural feature map of each heterologous image, wherein +.>Representing a structural feature map, ">Representing an input image +.>Representing the radius of the input heterologous image as +.>Gray mean of local neighborhood of>Represents the radius of a circular template, m is 5, (-)>，/>) A pixel point representing a heterologous image,iandjrespectively representxAndypixel offset value in direction, max () represents the maximum valueA large value function;
step 2.2: using Log-Gabor filtersLGTransforming the structural feature diagram to obtain a phase consistency model, and firstly utilizing a formulaObtaining odd-numbered parts of the heterologous image in the frequency domainOAnd an even number partEThen based on odd partsOAnd an even number partE，Using the formula->Obtaining amplitude components of a heterologous imageAAnd phase componentθFinally based on amplitude componentAAnd phase componentθ，Using the formulaCalculating a phase consistency model, wherein +.>Representation->Filter even component extractor, < >>Representation->Odd component extractor of filter, < >>Representing pixel dot +.>Weight parameter of->Representing estimated noise threshold, +_>Representing the scale in a Log-Gabor filtersAnd direction ofoUpper pixel +.>Variance function of phase component of>Representing the scale in a Log-Gabor filtersAnd direction ofoUpper pixel +.>Wherein the subscriptsAndorepresenting the dimensions and direction of the Log-Gabor filter, respectively, < >>Representing a small positive number to avoid denominator 0;
step 2.3: based on the structural feature map and the direction in the phase consistency modeloAnd respectively setting the filter directions as 0 degree, 45 degree, 90 degree and 135 degree to obtain 4 phase feature images, selecting a 4×4 neighborhood around the key point, arranging the phase feature values into a 4-dimensional feature vector according to the 4 directions of 0 degree, 45 degree, 90 degree and 135 degree for each pixel in the neighborhood, and combining the feature vectors of 16 pixels to obtain a final 64-dimensional feature vector, namely a phase feature descriptor of the key point.
Step 3: and (5) key point filtering: filtering the key points by using a mean shift algorithm based on the key points of each heterogeneous image and the phase characteristic descriptors thereof, and eliminating repeated modes in the key points; the method comprises the following specific steps:
step 3.1: clustering key points: based on the phase feature descriptors, calculating a mean shift quantity by using a mean shift algorithm to obtain a clustering center to cluster the phase feature descriptors of the key points, wherein the mean shift quantity is expressed by a formulaCalculating, wherein->Phase feature descriptors representing keypoints, +.>Indicate->Phase characteristic descriptor of key point, +.>The number of key points is represented and,grepresenting a kernel function->Representing the mean shift amount;
step 3.2: repeated keypoint filtration: setting a distance threshold and using Euclidean distance formulaCalculating the distance between any two clustering centers, according to priori knowledge, if two key points belong to a repeated mode, obtaining a smaller distance, removing the repeated key points smaller than a threshold value, and reserving a key point set with the distance larger than the threshold value, wherein the key point set with the distance larger than the threshold value is selected by the user>Indicate->Phase characteristic descriptor of key point, +.>Indicate->Phase characteristic descriptors of key points.
Step 4: key point matching: based on the key points and the phase feature descriptors of the two heterogeneous images to be matched after the repeated mode is eliminated, the method for calculating the Euclidean distance of the phase feature descriptors is adopted to carry out feature matching on the two heterogeneous images, and after matching, the nearest neighbor ratio adjusting method is adopted to further eliminate mismatching. The method comprises the following specific steps:
step 4.1: key point matching: based on the key points of two to-be-matched heterologous images after the repeated mode is eliminated, taking one to-be-matched heterologous image as a reference image, respectively traversing each key point in the reference image through all key points in the other to-be-matched heterologous image, calculating Euclidean distance of phase feature descriptors of the key points of the two images, and reserving two values with the minimum distance as matching candidate points;
step 4.2: mismatch cancellation: using the formulaCalculating whether the Euclidean distance of the nearest neighbor point pair of each group of matching candidate points is smaller than the Euclidean distance ratio adjusted value of the next nearest neighbor point pair, whereinγRepresents the adjustment ratio->Hours represent the Euclidean distance of the nearest neighbor pair,/->And (3) representing the Euclidean distance of the next-neighbor point pair, and reserving the point pair with the Euclidean distance of the nearest-neighbor point pair smaller than the value after Euclidean distance ratio adjustment of the next-neighbor point pair as a final heterologous image characteristic matching point pair.
Examples
Fig. 1 is a schematic flow chart of a heterologous image matching method based on local feature consistency, and can be seen from the figure: in the embodiment of the invention, the camera and the laser radar acquire the optical image and the radar image (radar intensity image) pair of the same scene;
first, key point extraction is performed: extracting key points on the laser radar intensity image and the optical image by adopting a characteristic extraction method based on local information respectively;
then, phase characterization is performed: firstly, calculating a structural feature map on a laser radar intensity image and an optical image by utilizing local contrast, extracting phase information of the laser radar intensity image and the optical image by utilizing a Log-Gabor filter, and then respectively establishing phase feature descriptors of the laser radar intensity image and the optical image based on key points of the laser radar intensity image and the optical image and local information of a neighborhood thereof, namely the phase feature descriptors of the key points, wherein the specific steps are as follows: the method comprises the steps of adopting 4 filtering directions of 0 degree, 45 degree, 90 degree and 135 degree to a structural feature map to obtain 4 phase feature maps, selecting a 4×4 neighborhood around a key point, arranging phase feature values into a 4-dimensional feature vector according to 4 directions of 0 degree, 45 degree, 90 degree and 135 degree for each pixel in the neighborhood, and combining feature vectors of 16 pixels to obtain a 64-dimensional feature vector;
then, the key point filtering is performed: based on key points of laser radar intensity images and optical images and phase feature descriptors thereof, clustering the key points by using a mean shift clustering algorithm, setting a distance threshold, calculating the distance between any two clustering centers by using a Euclidean distance formula, removing repeated key points with the distance value smaller than the threshold, and reserving a key point set with the distance larger than the threshold so as to eliminate repeated modes in the key points;
finally, key point matching is carried out: based on the key points of the filtered laser radar intensity image and the optical image and the phase feature descriptors thereof, a method for calculating the Euclidean distance of the phase feature descriptors of the key points is adopted to establish feature matching candidate points of the laser radar intensity image and the optical image, and a nearest neighbor ratio adjustment method is adopted to further eliminate mismatching, so that a final feature matching result is obtained.
The method has universality and is used for precisely matching heterogeneous images. The above is merely representative examples of numerous specific applications of the present invention and should not be construed as limiting the scope of the invention in any way. All technical schemes formed by adopting transformation or equivalent substitution fall within the protection scope of the invention.
Claims (8)
1. The heterologous image matching method based on local feature consistency is characterized by comprising the following steps of:
step 1: and (3) key point extraction: extracting key points of each heterogeneous image by a feature extraction method based on local information;
step 2: phase characterization: extracting phase information of each heterogeneous image by using a Log-Gabor filter, and establishing a local phase feature descriptor of each heterogeneous image based on the key points of each heterogeneous image and local information of a neighborhood thereof;
step 3: and (5) key point filtering: filtering the key points by using a mean shift algorithm based on the key points of each heterogeneous image and the phase characteristic descriptors thereof, and eliminating repeated modes in the key points;
step 4: key point matching: based on the key points and the phase feature descriptors of the two heterogeneous images to be matched after the repeated mode is eliminated, performing feature matching of the two heterogeneous images by adopting a method for calculating the Euclidean distance of the phase feature descriptors, and further eliminating mismatching by adopting a nearest neighbor ratio adjusting method after matching;
the specific steps of the step 2 are as follows:
step 2.1: calculating a structural feature map of each heterologous image: by using structural characteristic formulasCalculating structural feature map of each heterologous image, wherein +.>Representing a structural feature map, ">Representing an input image +.>Representing the radius of the input heterologous image as +.>Gray mean of local neighborhood of>Representing the radius of the circular template +.>The value is 5, (-)>，/>) A pixel point representing a heterologous image,iandjrespectively representxAndypixel offset values in the direction, max () represents a maximum value taking function;
step 2.2: using Log-Gabor filtersLGTransforming the structural feature diagram to obtain a phase consistency model, and firstly utilizing a formulaObtaining odd-numbered parts of the heterologous image in the frequency domainOAnd an even number partEThen based on odd partsOAnd an even number partE，Using the formula->Obtaining amplitude components of a heterologous imageAAnd phase componentθFinally based on amplitude componentAAnd phase componentθ，Using the formulaCalculating a phase consistency model, wherein +.>Representation->Filter even component extractor, < >>Representation->Odd component extractor of filter, < >>Representing pixel dot +.>Weight parameter of->Representing estimated noise threshold, +_>Representing the scale in a Log-Gabor filtersAnd direction ofoUpper pixel +.>Variance function of phase component of>Representing the scale in a Log-Gabor filtersAnd direction ofoUpper pixel +.>Wherein the subscriptsAndorepresenting the dimensions and direction of the Log-Gabor filter, respectively, < >>Representing a small positive number to avoid denominator 0;
step 2.3: based on the structural feature map and the direction in the phase consistency modeloAnd respectively setting the filter directions as 0 degree, 45 degree, 90 degree and 135 degree to obtain 4 phase feature images, selecting a 4×4 neighborhood around the key point, arranging the phase feature values into a 4-dimensional feature vector according to the 4 directions of 0 degree, 45 degree, 90 degree and 135 degree for each pixel in the neighborhood, and combining the feature vectors of 16 pixels to obtain a final 64-dimensional feature vector, namely a phase feature descriptor of the key point.
2. The method for matching heterologous images based on local feature consistency according to claim 1, wherein the specific steps of step 1 are as follows:
step 1.1: using a circular template with a radius of 3 pixels, enabling the center point of the circular template to slide on each heterologous image, traversing the whole heterologous image, and when the center of the circular template is overlapped with the pixels of the heterologous image, using one pixelpThe center is a circular neighborhood with radius of 3 pixels, and 16 pixel points are arranged on the circular neighborhoodp 1 、p 2 、...、p 16 ）；
Step 1.2: calculating key candidate points: defining a threshold valuetSequentially calculating the slavep 1 To the point ofp 16 These 16 points and center pixelspIf there are at least 9 consecutive pixel differences exceeding the thresholdtThen the center pixel ispDefining as key candidate points;
step 1.3: non-maximum threshold: by key candidate pointspIn a 5×5 neighborhood with the center, if there are multiple key candidate points, the formula is used in turn
The pixels exceeding the threshold value in each key candidate point are subjected to difference accumulation summation to obtain all the key candidate pointssValue, finally remainsThe key candidate point with the largest value is taken as the key point of each heterogeneous image.
3. The method for matching heterologous images based on local feature consistency according to claim 2, wherein the specific steps of step 3 are as follows:
step 3.1: clustering key points: based on the phase feature descriptors, calculating a mean shift quantity by using a mean shift algorithm to obtain a clustering center to cluster the phase feature descriptors of the key points, wherein the mean shift quantity is expressed by a formulaCalculating, wherein->Phase feature descriptors representing keypoints, +.>Indicate->Phase characteristic descriptor of key point, +.>The number of key points is represented and,grepresenting a kernel function->Representing the mean shift amount;
step 3.2: repeated keypoint filtration: setting a distance threshold and using Euclidean distance formulaCalculating the distance between any two clustering centers, according to priori knowledge, if two key points belong to a repeated mode, obtaining a smaller distance, removing the repeated key points smaller than a threshold value, and reserving a key point set with the distance larger than the threshold value, wherein the key point set with the distance larger than the threshold value is selected by the user>Indicate->Phase characteristic descriptor of key point, +.>Indicate->Phase characteristic descriptors of key points.
4. The method for matching heterologous images based on local feature consistency according to claim 3, wherein the specific steps of step 4 are as follows:
step 4.1: key point matching: based on the key points of two to-be-matched heterologous images after the repeated mode is eliminated, taking one to-be-matched heterologous image as a reference image, respectively traversing each key point in the reference image through all key points in the other to-be-matched heterologous image, calculating Euclidean distance of phase feature descriptors of the key points of the two images, and reserving two values with the minimum distance as matching candidate points;
step 4.2: mismatch cancellation: using the formulaCalculating whether the Euclidean distance of the nearest neighbor point pair of each group of matching candidate points is smaller than the Euclidean distance ratio adjusted value of the next nearest neighbor point pair, whereinγIndicating that the adjustment ratio is to be made,hours represent the Euclidean distance of the nearest neighbor pair,/->And (3) representing the Euclidean distance of the next-neighbor point pair, and reserving the point pair with the Euclidean distance of the nearest-neighbor point pair smaller than the value after Euclidean distance ratio adjustment of the next-neighbor point pair as a final heterologous image characteristic matching point pair.
5. A heterologous image matching system based on local feature consistency, comprising:
and a key point extraction module: extracting key points of each heterogeneous image by a feature extraction method based on local information;
and the phase characteristic description module is used for: extracting phase information of each heterogeneous image by using a Log-Gabor filter, and establishing a local phase feature descriptor of each heterogeneous image based on the key points of each heterogeneous image and local information of a neighborhood thereof;
and the key point filtering module is used for: filtering the key points by using a mean shift algorithm based on the key points of each heterogeneous image and the phase characteristic descriptors thereof, and eliminating repeated modes in the key points;
and a key point matching module: based on the key points and the phase feature descriptors of the two heterogeneous images to be matched after the repeated mode is eliminated, performing feature matching of the two heterogeneous images by adopting a method for calculating the Euclidean distance of the phase feature descriptors, and further eliminating mismatching by adopting a nearest neighbor ratio adjusting method after matching;
the specific implementation steps of the phase characteristic description module are as follows:
step 2.1: calculating a structural feature map of each heterologous image: by using structural characteristic formulasCalculating structural feature map of each heterologous image, wherein +.>Representing a structural feature map, ">Representing an input image +.>Representing the radius of the input heterologous image as +.>Gray mean of local neighborhood of>Represents the radius of a circular template, m is 5, (-)>，/>) A pixel point representing a heterologous image,iandjrespectively representxAndypixel offset values in the direction, max () represents a maximum value taking function;
step 2.2: using Log-Gabor filtersLGStructural featuresTransforming the graph to obtain a phase consistency model, and firstly utilizing a formulaObtaining odd-numbered parts of the heterologous image in the frequency domainOAnd an even number partEThen based on odd partsOAnd an even number partE，Using the formula->Obtaining amplitude components of a heterologous imageAAnd phase componentθFinally based on amplitude componentAAnd phase componentθ，Using the formulaCalculating a phase consistency model, wherein +.>Representation->Filter even component extractor, < >>Representation->Odd component extractor of filter, < >>Representing pixel dot +.>Weight parameter of->Representing estimated noise threshold, +_>Representing the scale in a Log-Gabor filtersAnd direction ofoUpper pixel +.>Variance function of phase component of>Representing the scale in a Log-Gabor filtersAnd direction ofoUpper pixel +.>Wherein the subscriptsAndorepresenting the dimensions and direction of the Log-Gabor filter, respectively, < >>Representing a small positive number to avoid denominator 0;
step 2.3: based on the structural feature map and the direction in the phase consistency modeloAnd respectively setting the filter directions as 0 degree, 45 degree, 90 degree and 135 degree to obtain 4 phase feature images, selecting a 4×4 neighborhood around the key point, arranging the phase feature values into a 4-dimensional feature vector according to the 4 directions of 0 degree, 45 degree, 90 degree and 135 degree for each pixel in the neighborhood, and combining the feature vectors of 16 pixels to obtain a final 64-dimensional feature vector, namely a phase feature descriptor of the key point.
6. The heterologous image matching system based on local feature consistency according to claim 5, wherein the specific implementation steps of the key point extraction module are as follows:
step 1.1: using a circular template with a radius of 3 pixels, enabling the center point of the circular template to slide on each heterologous image, traversing the whole heterologous image, and when the center of the circular template is overlapped with the pixels of the heterologous image, using one pixelpThe center is a circular neighborhood with radius of 3 pixels, and 16 pixel points are arranged on the circular neighborhoodp 1 、p 2 、...、p 16 ）；
Step 1.2: calculating key candidate points: defining a threshold valuetSequentially calculating the slavep 1 To the point ofp 16 These 16 points and center pixelspIf there are at least 9 consecutive pixel differences exceeding the thresholdtThen the center pixel ispDefining as key candidate points;
step 1.3: non-maximum threshold: by key candidate pointspIn a 5×5 neighborhood with the center, if there are multiple key candidate points, the formula is used in turn
The pixels exceeding the threshold value in each key candidate point are subjected to difference accumulation summation to obtain all the key candidate pointssValue, finally remainsThe key candidate point with the largest value is taken as the key point of each heterogeneous image.
7. The heterologous image matching system based on local feature consistency of claim 6, wherein the specific implementation steps of the key point filtering module are as follows:
step 3.1: clustering key points: based on the phase feature descriptors, calculating a mean shift quantity by using a mean shift algorithm to obtain a clustering center to cluster the phase feature descriptors of the key points, wherein the mean shift quantity is expressed by a formula
And (3) calculating, wherein,phase feature descriptors representing keypoints, +.>Indicate->Phase characterization of keypointsSon (S)/(S)>The number of key points is represented and,grepresenting a kernel function->Representing the mean shift amount;
step 3.2: repeated keypoint filtration: setting a distance threshold and using Euclidean distance formulaCalculating the distance between any two clustering centers, according to priori knowledge, if two key points belong to a repeated mode, obtaining a smaller distance, removing the repeated key points smaller than a threshold value, and reserving a key point set with the distance larger than the threshold value, wherein the key point set with the distance larger than the threshold value is selected by the user>Indicate->Phase characteristic descriptor of key point, +.>Indicate->Phase characteristic descriptors of key points.
8. The heterologous image matching system based on local feature consistency of claim 7, wherein the specific implementation steps of the key point matching module are as follows:
step 4.1: key point matching: based on the key points of two to-be-matched heterologous images after the repeated mode is eliminated, taking one to-be-matched heterologous image as a reference image, respectively traversing each key point in the reference image through all key points in the other to-be-matched heterologous image, calculating Euclidean distance of phase feature descriptors of the key points of the two images, and reserving two values with the minimum distance as matching candidate points;
step 4.2: mismatch cancellation: using the formulaCalculating whether the Euclidean distance of the nearest neighbor point pair of each group of matching candidate points is smaller than the Euclidean distance ratio adjusted value of the next nearest neighbor point pair, whereinγIndicating that the adjustment ratio is to be made,hours represent the Euclidean distance of the nearest neighbor pair,/->And (3) representing the Euclidean distance of the next-neighbor point pair, and reserving the point pair with the Euclidean distance of the nearest-neighbor point pair smaller than the value after Euclidean distance ratio adjustment of the next-neighbor point pair as a final heterologous image characteristic matching point pair.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310317745.XA CN116129146B (en) | 2023-03-29 | 2023-03-29 | Heterogeneous image matching method and system based on local feature consistency |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310317745.XA CN116129146B (en) | 2023-03-29 | 2023-03-29 | Heterogeneous image matching method and system based on local feature consistency |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116129146A CN116129146A (en) | 2023-05-16 |
| CN116129146B true CN116129146B (en) | 2023-09-01 |
Family
ID=86295799
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310317745.XA Active CN116129146B (en) | 2023-03-29 | 2023-03-29 | Heterogeneous image matching method and system based on local feature consistency |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116129146B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117115336A (en) * | 2023-07-13 | 2023-11-24 | 中国工程物理研究院计算机应用研究所 | Point cloud reconstruction method based on remote sensing stereoscopic image |
| CN117237680B (en) * | 2023-08-18 | 2024-03-12 | 暨南大学 | Heterogeneous model fitting-based multi-source image matching method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109409292A (en) * | 2018-10-26 | 2019-03-01 | 西安电子科技大学 | The heterologous image matching method extracted based on fining characteristic optimization |
| CN110321925A (en) * | 2019-05-24 | 2019-10-11 | 中国工程物理研究院计算机应用研究所 | A kind of more granularity similarity comparison methods of text based on semantics fusion fingerprint |
| CN112712510A (en) * | 2020-12-31 | 2021-04-27 | 中国电子科技集团公司第十四研究所 | Different-source image matching method based on gradient and phase consistency |
| CN113643334A (en) * | 2021-07-09 | 2021-11-12 | 西安电子科技大学 | Different-source remote sensing image registration method based on structural similarity |
| CN115601569A (en) * | 2022-10-17 | 2023-01-13 | 安徽工业大学（Cn） | Different-source image optimization matching method and system based on improved PIIFD |
-
2023
- 2023-03-29 CN CN202310317745.XA patent/CN116129146B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109409292A (en) * | 2018-10-26 | 2019-03-01 | 西安电子科技大学 | The heterologous image matching method extracted based on fining characteristic optimization |
| CN110321925A (en) * | 2019-05-24 | 2019-10-11 | 中国工程物理研究院计算机应用研究所 | A kind of more granularity similarity comparison methods of text based on semantics fusion fingerprint |
| CN112712510A (en) * | 2020-12-31 | 2021-04-27 | 中国电子科技集团公司第十四研究所 | Different-source image matching method based on gradient and phase consistency |
| CN113643334A (en) * | 2021-07-09 | 2021-11-12 | 西安电子科技大学 | Different-source remote sensing image registration method based on structural similarity |
| CN115601569A (en) * | 2022-10-17 | 2023-01-13 | 安徽工业大学（Cn） | Different-source image optimization matching method and system based on improved PIIFD |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116129146A (en) | 2023-05-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116129146B (en) | Heterogeneous image matching method and system based on local feature consistency | |
| CN103679714B (en) | A kind of optics and SAR automatic image registration method based on gradient cross-correlation | |
| CN104751465A (en) | ORB (oriented brief) image feature registration method based on LK (Lucas-Kanade) optical flow constraint | |
| CN107403134B (en) | Local gradient trilateral-based image domain multi-scale infrared dim target detection method | |
| CN107358161B (en) | Coastline extraction method and coastline extraction system based on remote sensing image classification | |
| CN104732231B (en) | A kind of recognition methods of valuable bills | |
| CN103400384A (en) | Large viewing angle image matching method capable of combining region matching and point matching | |
| CN105894513B (en) | Take the remote sensing image variation detection method and system of imaged object change in time and space into account | |
| Badr et al. | A robust copy-move forgery detection in digital image forensics using SURF | |
| CN105513094A (en) | Stereo vision tracking method and stereo vision tracking system based on 3D Delaunay triangulation | |
| CN115937160A (en) | Explosion fireball contour detection method based on convex hull algorithm | |
| Xu et al. | A robust deformed image matching method for multi-source image matching | |
| CN116664892A (en) | Multi-temporal remote sensing image registration method based on cross attention and deformable convolution | |
| CN112419198B (en) | Non-local mean weighting method for SAR interferogram filtering | |
| CN112907748B (en) | Three-dimensional shape reconstruction method based on non-down-sampling shear wave transformation and depth image texture feature clustering | |
| CN111595758B (en) | Oval cell detection device and detection method based on lens-free imaging | |
| Li et al. | Robust optical and SAR image registration based on phase congruency scale space | |
| CN113610906B (en) | Multi-parallax image sequence registration method based on fusion image guidance | |
| CN104463182A (en) | NBI gastroscope image processing method based on key point detection | |
| CN115424249B (en) | Self-adaptive detection method for small and weak targets in air under complex background | |
| Liu et al. | Identification Method of Sunflower Leaf Disease Based on SIFT Point | |
| Pakdelazar et al. | Improvement of BM3D algorithm and Employment to satellite and CFA images Denoising | |
| CN111709434A (en) | Robust multi-scale template matching method based on nearest neighbor feature point matching | |
| Tang et al. | An SIFT-Based Fast Image Alignment Algorithm for High-Resolution Image | |
| Hui et al. | Image fusion based on complex wavelets and region segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |