CN112861672B - Heterogeneous remote sensing image matching method based on optical-SAR - Google Patents
Heterogeneous remote sensing image matching method based on optical-SAR Download PDFInfo
- Publication number
- CN112861672B CN112861672B CN202110111049.4A CN202110111049A CN112861672B CN 112861672 B CN112861672 B CN 112861672B CN 202110111049 A CN202110111049 A CN 202110111049A CN 112861672 B CN112861672 B CN 112861672B
- Authority
- CN
- China
- Prior art keywords
- image
- optical
- sar
- matching
- local neighborhood
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Image Analysis (AREA)
Abstract
The invention provides an optical-SAR-based heterogeneous remote sensing image matching method, which comprises the following steps: s1, collecting training data containing a positive sample and a negative sample; s2, training a full convolution backbone network by using training data to obtain an image matching model; and S3, matching the optical image to be matched with the SAR image by using the image matching model. The invention can self-adaptively learn the detail characteristics of the image by using a deep learning method, can improve the accuracy and efficiency of remote sensing image matching, and can be truly commercialized.
Description
Technical Field
The invention relates to the technical field of heterogeneous remote sensing matching, in particular to an optical-SAR-based heterogeneous remote sensing image matching method.
Background
At present, the heterogeneous remote sensing matching mostly uses a manual design method, the detail characteristics of heterogeneous remote sensing images cannot be well quantized and utilized, great precision loss exists, and the method does not meet the production requirements.
Disclosure of Invention
The invention aims to provide an optical-SAR-based heterogeneous remote sensing image matching method, and aims to solve the problems that the existing heterogeneous remote sensing matching method is mostly designed manually, cannot be quantized well, utilizes the detail characteristics of heterogeneous remote sensing images, has large precision loss and does not meet the requirements of productization.
The invention provides an optical-SAR-based heterogeneous remote sensing image matching method, which comprises the following steps:
s1, collecting training data containing a positive sample and a negative sample;
s2, training a full convolution backbone network by using training data to obtain an image matching model;
and S3, matching the optical image to be matched with the SAR image by using the image matching model.
Further, step S1 includes the following sub-steps:
s11, acquiring optical images and SAR images of a plurality of same areas;
s12, carrying out image preprocessing on the optical image and the SAR image;
s13, extracting SAR characteristic points and optical homonymy characteristic points of the SAR characteristic points; the SAR characteristic point and the optical homonymous characteristic point are called homonymous point pairs;
s14, collecting local neighborhood image blocks in a preset shape with SAR characteristic points as centers in the preprocessed SAR image;
s15, collecting a local neighborhood image block of a preset shape taking an optical homonymous feature point as a center in the preprocessed optical image;
s16, matching the local neighborhood image blocks obtained in the step S14 and the step S15 in pairs, wherein the local neighborhood image blocks corresponding to the same-name point pairs are called same-name image pairs, and the rest local neighborhood image blocks matched in pairs are called heterogeneous image pairs;
s17, the homonym image pair is used as a positive sample of the training data, and the heteronym image pair is used as a negative sample of the training data.
Further, step S13 includes the following sub-steps:
s131, searching feature points in the preprocessed optical image and SAR image by using a Harris algorithm, detecting the feature points by using an SIFT algorithm, and extracting the optical feature points and the SAR feature points;
s132, collecting local neighborhood image blocks in a preset shape by taking the optical characteristic points and the SAR characteristic points as centers respectively;
s133, generating 128-dimensional feature vectors for each local neighborhood image block by using a deep neural network;
s134, calculating Euclidean spatial similarity of feature vectors of image blocks of local neighborhoods of different sources by using a K-means algorithm;
s135, according to the calculated European spatial similarity, taking the SAR characteristic point as a reference, and taking the nearest neighbor optical characteristic point as an optical homonymy characteristic point;
s136, eliminating false-reported optical homonymous feature points by using a RANSAC algorithm to obtain an affine transformation matrix of a matching point set;
s137, calculating an accurate regression coordinate by using the affine transformation matrix of the matching point set, and constructing a matching block with the radius of H by taking the coordinate as a center; secondly, by taking the SAR image characteristic points as a reference, re-matching the optical characteristic points in the matching block as new optical homonymous characteristic points;
and S138, repeating the steps S136 to S137 for a plurality of times to obtain the required optical homonymous feature points.
Further, the network structure of the fully-convolutional backbone network in step S2 includes 1 local response normalization layer LRN and 11 network modules, each network module is composed of a convolutional layer Conv and a batch normalization layer BN, and each network module is followed by 1 activation function Relu; the partial response normalization layer LRN is connected after the last activation function Relu.
Further, the proportion of the positive samples and the negative samples of the training data input into the full convolution backbone network is controlled to be 1: 1.
Further, the method for training the full convolution backbone network in step S2 includes: the method comprises the steps of pre-training the full convolution backbone network by using Brown and HPatches data sets, and then training by using positive samples and negative samples of training data to obtain an image matching model, so that the problem of small data volume can be solved to a certain extent.
Further, step S3 includes the following sub-steps:
s31, acquiring an optical image and an SAR image to be matched;
s32, carrying out image preprocessing on the optical image to be matched and the SAR image;
s33, searching characteristic points in the preprocessed SAR image to be matched by using a Harris algorithm, detecting the characteristic points by using an SIFT algorithm, and extracting SAR characteristic points;
s34, collecting local neighborhood image blocks in a preset shape with SAR feature points as centers in the preprocessed SAR image to be matched;
s35, acquiring local neighborhood image blocks in different positions and in a preset shape in the preprocessed optical image to be matched;
s36, inputting the optical image to be matched and the local neighborhood image block of the SAR image into the image block pair matching model to obtain the local neighborhood image block of the matched optical image; the optical characteristic point at the center of the local neighborhood image block of the matched optical image is the optical homonymous characteristic point.
Further, the image preprocessing method comprises the following steps:
(1) improving the local contrast ratio by using image equalization on the optical image and the SAR image;
(2) gaussian filtering is also used on the SAR image to filter out image noise.
Further, the preset shape of the local neighborhood image block is a square.
Further, the size of the local neighborhood image block is 16 × 16, 32 × 32, or 64 × 64.
In summary, due to the adoption of the technical scheme, the invention has the beneficial effects that:
1. the method for matching the heterogeneous remote sensing image based on the optical-SAR can adaptively learn the detail characteristics of the image by using a deep learning method, can improve the accuracy and efficiency of matching the remote sensing image, and can be truly commercialized.
2. According to the method, the effect of characteristic point extraction and the reliability of a characteristic extraction part can be improved by image equalization on the quality of the optical image and the SAR image; and by additionally using Gaussian filtering on the SAR image, false recognition can be avoided when the feature points are extracted.
3. The method uses Harris algorithm and SIFT algorithm to extract the feature points, and can avoid the problem that the feature points with different names are detected on the optical image and the SAR image.
4. The invention eliminates the false-reported optical homonymous feature points by using the RANSAC algorithm, and can obtain more accurate homonymous feature points.
5. According to the method, the ratio of the positive sample to the negative sample of the training data is controlled to be 1:1, so that the trained image matching model can be ensured to be accurately matched with a target image block which is most matched with the local neighborhood image block of the SAR image from the local neighborhood image block of the optical image containing a large number of negative samples, and the situation that the image matching model cannot be converged due to excessive negative samples is avoided.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention, and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a flow chart of an optical-SAR heterogeneous remote sensing image matching method according to an embodiment of the present invention.
FIG. 2 is a block diagram of a process for collecting training data including positive and negative examples according to an embodiment of the present invention
Fig. 3 is a block diagram of a process of extracting SAR feature points and optical homonymous feature points according to an embodiment of the present invention.
Fig. 4 is a schematic diagram of a pair of homonymous images and homonymous feature points according to an embodiment of the present invention.
FIG. 5 is a schematic diagram of a homonymous image pair and a heterologous image pair of an embodiment of the invention.
Fig. 6 is a schematic network structure diagram of the full-convolution backbone network according to the present invention.
Fig. 7 is a block diagram of a process of matching an optical image to be matched with an SAR image by using an image matching model according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Examples
Referring to fig. 1, the present embodiment provides an optical-SAR-based heterogeneous remote sensing image matching method, where the matching method includes the following steps:
s1, collecting training data containing positive and negative samples, see fig. 2:
s11, acquiring optical images and SAR images of a plurality of same areas;
s12, image preprocessing is carried out on the optical image and the SAR image: (1) improving the local contrast ratio by using image equalization on the optical image and the SAR image; (2) gaussian filtering is also used on the SAR image to filter out image noise. The quality of an original optical image and an SAR image is poor, an overexposed image or an overexposed image often appears, the effect of feature point extraction can be improved and the reliability of a feature extraction part can be improved for the quality of the optical image and the SAR image through image equalization; the SAR image is additionally subjected to Gaussian filtering, so that false recognition during feature point extraction can be avoided;
s13, extracting SAR characteristic points and optical homonymy characteristic points of the SAR characteristic points; the SAR feature points and the optical homonymous feature points are called homonymous point pairs, see fig. 3:
s131, searching feature points in the preprocessed optical image and SAR image by using a Harris algorithm, detecting the feature points by using an SIFT algorithm, and extracting the optical feature points and the SAR feature points; for an optical image and an SAR image, the gray value is generally mutated in the edge and other areas of an object, the Harris algorithm can well detect the characteristic points, but the Harris algorithm requires that the gray value of each direction of a picture is obviously changed, so that the detection quantity of the characteristic points is too small, and even the characteristic points with the same name are detected on the optical image and the SAR image, therefore, the SIFT algorithm is used for assisting in extraction to solve the problem;
s132, collecting local neighborhood image blocks in a preset shape by taking the optical characteristic points and the SAR characteristic points as centers respectively; in this embodiment, the preset shape of the local neighborhood image block is a square; generally, the size of the local neighborhood image block is 16 × 16, 32 × 32 or 64 × 64;
s133, a 128-dimensional feature vector is generated for each local neighborhood image block by using a deep neural network, and the method for extracting the feature vector in the step can adopt the prior art and is not described any more;
s134, calculating Euclidean spatial similarity of feature vectors of local neighborhood image blocks of different sources (namely from the optical image and the SAR image respectively) by using a K-means algorithm; the present embodiment uses the L2 distance as the euclidean space similarity;
s135, according to the calculated European spatial similarity, taking the SAR characteristic point as a reference, and taking the nearest neighbor optical characteristic point as an optical homonymy characteristic point;
s136, eliminating false-reported optical homonymous feature points by using a RANSAC algorithm to obtain an affine transformation matrix of a matching point set; the accuracy of the optical homonymous feature points obtained in step S135 needs to be further improved, so the present invention uses the RANSAC algorithm to eliminate the misreported optical homonymous feature points, where the RANSAC algorithm is the prior art, and in this embodiment: and randomly selecting a plurality of optical homonymous feature points, calculating an affine transformation matrix by using a least square method, and quantifying the model goodness according to the number of homonymous points which can be matched by the affine transformation matrix. Repeating the cycle until a model meeting the requirement is obtained or the maximum iteration times is reached, and then withdrawing the cycle, wherein the obtained affine transformation matrix is the affine transformation matrix of the matching point set;
s137, calculating an accurate regression coordinate by using the affine transformation matrix of the matching point set, and constructing a matching block with the radius of H by taking the coordinate as a center; secondly, by taking the SAR image characteristic points as a reference, re-matching the optical characteristic points in the matching block as new optical homonymous characteristic points;
s138, repeatedly executing the steps S136-S137 for a plurality of times to obtain the required optical homonymous feature points;
s14, collecting local neighborhood image blocks in a preset shape with SAR characteristic points as centers in the preprocessed SAR image; as before, the preset shape of the local neighborhood image block here is also square; likewise, the size of the local neighborhood image block is 16 × 16, 32 × 32 or 64 × 64;
s15, collecting a local neighborhood image block in a preset shape with an optical homonymous feature point as a center in the preprocessed optical image; as before, the preset shape of the local neighborhood image block here is also square; likewise, the size of the local neighborhood image block is 16 × 16, 32 × 32 or 64 × 64;
s16, matching the local neighborhood image blocks obtained in the step S14 and the step S15 in pairs, wherein the local neighborhood image blocks corresponding to the same-name point pairs are called same-name image pairs, and the rest local neighborhood image blocks matched in pairs are called heterogeneous image pairs; as shown in fig. 4, the square frame in the optical image and the SAR image is a pair of homonymous images, and the center of the pair of homonymous images is a pair of homonymous points.
S17, the homonym image pair is used as a positive sample of the training data, and the heteronym image pair is used as a negative sample of the training data.
S2, training a full convolution backbone network by using training data to obtain an image matching model;
as shown in fig. 5, for positive samples far lower in number than negative samples, any two heterogeneous image pairs can constitute a negative sample in addition to the homonymous image pair; however, for hundreds or thousands of homonymous image pairs in a batch, traversing all negative examples would increase the computational complexity significantly, and having many negative examples would not actually result in an effective gradient update. Therefore, the proportion of the positive samples and the negative samples of the training data input into the full convolution backbone network is controlled to be 1:1, so that the trained image matching model can ensure that the target image block which is most matched with the local neighborhood image block of the SAR image can be accurately matched from the local neighborhood image block of the optical image containing a large number of negative samples, and the situation that the image matching model cannot be converged due to excessive negative samples is avoided.
As shown in fig. 6, the network structure of the fully-convolutional backbone network in step S2 includes 1 local response normalization layer LRN and 11 network modules, each network module is composed of a convolutional layer Conv and a batch normalization layer BN, and each network module is followed by 1 activation function Relu; the partial response normalization layer LRN is connected after the last activation function Relu. And defining a fine label loss function, inputting a positive sample and a negative sample of training data into the full convolution backbone network, and training to obtain an image matching model by using the defined loss function. The loss function implementation is as follows:
having a unique representation of a remembered image block
τ i I-1 represents the optical image, i-2 represents the SAR image,
represented as the original image tau i The image block with the upper index j,
for image blocks
Coordinates of the center point of (a). Image block pair assumed to participate in training
From the same pair of heterogeneous image pairs (tau) i ,τ i ) The distance between the image block and the central point (i.e. the coordinates of the key feature point corresponding to the image block) is x ij . For training samples
Obtaining the descriptive feature vector after the improved L2-Net
Where n represents the number of same-name image block pairs (batch size) in a batch during training, and p represents the feature vector output by L2-Net. Obtaining a space distance matrix after sampling
Distance matrix D ═ D from feature ij ] n×n Wherein:
in the above formula x ij Smaller represents more overlapping areas, they are more similar when x ij Larger than the block radius b represents a non-overlapping area. Similarly, when the training samples do not belong to the same pair of different source image pairs, they do not have an overlapping region, and accordingly, a spatial distance matrix L ═ L is obtained ij ] n×n As shown in the following formula:
where θ is a magnification factor, where θ is set to 1, the larger θ is, the more sensitive the key feature point distance change is. According to the image homonymy information, a label matrix Y ═ Y can be obtained ij ] n×n ：
The fine tag loss function, which combines the feature distance and the spatial distance, is then:
in some embodiments, the full convolution backbone network may be pre-trained using Brown and HPatches data sets, and then trained using positive and negative samples of training data to obtain an image matching model, which may eliminate the problem of small data size to some extent.
S3, matching the optical image to be matched with the SAR image by using the image matching model, see fig. 7:
s31, acquiring an optical image and an SAR image to be matched;
s32, image preprocessing is carried out on the optical image to be matched and the SAR image, and the steps are the same as the step S12: (1) using image equalization to the optical image and the SAR image to be matched to improve the local contrast; (2) and filtering image noise by using Gaussian filtering on the SAR image to be matched.
S33, searching characteristic points in the preprocessed SAR image to be matched by using a Harris algorithm, detecting the characteristic points by using an SIFT algorithm, and extracting SAR characteristic points;
s34, collecting local neighborhood image blocks in a preset shape with SAR feature points as centers in the preprocessed SAR image to be matched; similarly, the preset shape of the local neighborhood image block is a square, and the size of the local neighborhood image block is 16 × 16, 32 × 32 or 64 × 64;
s35, collecting local neighborhood image blocks in different positions and in a preset shape in the preprocessed optical image to be matched; similarly, the preset shape of the local neighborhood image block is a square, and the size of the local neighborhood image block is 16 × 16, 32 × 32 or 64 × 64;
s36, inputting the optical image to be matched and the local neighborhood image block of the SAR image into the image block pair matching model to obtain the local neighborhood image block of the matched optical image; the optical characteristic point at the center of the local neighborhood image block of the matched optical image is the optical homonymous characteristic point.
According to the method for matching the remote sensing image based on the optical-SAR heterogeneous remote sensing image, disclosed by the invention, the image detail characteristics are self-adaptively learned by using a deep learning method, so that the accuracy and the efficiency of matching the remote sensing image can be improved, and the remote sensing image is truly commercialized.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (8)
1. A heterogeneous remote sensing image matching method based on optical-SAR is characterized by comprising the following steps:
s1, collecting training data containing a positive sample and a negative sample;
s2, training a full convolution backbone network by using training data to obtain an image matching model;
s3, matching the optical image to be matched with the SAR image by using the image matching model;
step S1 includes the following sub-steps:
s11, acquiring optical images and SAR images of a plurality of same areas;
s12, carrying out image preprocessing on the optical image and the SAR image;
s13, extracting SAR characteristic points and optical homonymy characteristic points of the SAR characteristic points; the SAR characteristic point and the optical homonymous characteristic point are called homonymous point pairs;
s14, collecting local neighborhood image blocks in a preset shape with SAR characteristic points as centers in the preprocessed SAR image;
s15, collecting a local neighborhood image block in a preset shape with an optical homonymous feature point as a center in the preprocessed optical image;
s16, matching the local neighborhood image blocks obtained in the step S14 and the step S15 in pairs, wherein the local neighborhood image blocks corresponding to the same-name point pairs are called same-name image pairs, and the rest local neighborhood image blocks matched in pairs are called heterogeneous image pairs;
s17, using the homonymy image pair as a positive sample of the training data, and using the heterogenous image pair as a negative sample of the training data;
step S13 includes the following sub-steps:
s131, searching feature points in the preprocessed optical image and SAR image by using a Harris algorithm, detecting the feature points by using an SIFT algorithm, and extracting the optical feature points and the SAR feature points;
s132, respectively collecting a local neighborhood image block in a preset shape by taking the optical characteristic point and the SAR characteristic point as centers;
s133, generating 128-dimensional feature vectors for each local neighborhood image block by using a deep neural network;
s134, calculating Euclidean spatial similarity of feature vectors of image blocks of local neighborhoods of different sources by using a K-means algorithm;
s135, according to the calculated European spatial similarity, taking the SAR characteristic point as a reference, and taking the nearest neighbor optical characteristic point as an optical homonymy characteristic point;
s136, eliminating false-reported optical homonymous feature points by using a RANSAC algorithm to obtain an affine transformation matrix of a matching point set;
s137, calculating an accurate regression coordinate by using the affine transformation matrix of the matching point set, and constructing a matching block with the radius of H by taking the coordinate as a center; secondly, by taking the SAR image characteristic points as a reference, re-matching the optical characteristic points in the matching block as new optical homonymous characteristic points;
and S138, repeating the steps S136 to S137 for a plurality of times to obtain the required optical homonymous feature points.
2. The optical-SAR-based heterogeneous remote sensing image matching method according to claim 1, wherein the network structure of the fully-convoluted backbone network in step S2 comprises 1 local response normalization layer LRN and 11 network modules, each network module is composed of one convolution layer Conv and batch normalization layer BN, and each network module is followed by 1 activation function Relu; the partial response normalization layer LRN is connected after the last activation function Relu.
3. The method for matching the images based on the optical-SAR heterogeneous remote sensing as claimed in claim 2, wherein the ratio of the positive sample and the negative sample of the training data input into the full convolution backbone network is controlled to be 1: 1.
4. The method for matching the image based on the optical-SAR heterogeneous remote sensing of claim 3, wherein the method for training the full convolution backbone network in the step S2 comprises the following steps: the method comprises the steps of pre-training the full convolution backbone network by using Brown and HPatches data sets, and then training by using positive samples and negative samples of training data to obtain an image matching model, so that the problem of small data volume can be solved to a certain extent.
5. The optical-SAR-based heterogeneous remote sensing image matching method according to claim 4, wherein the step S3 comprises the following sub-steps:
s31, acquiring an optical image and an SAR image to be matched;
s32, carrying out image preprocessing on the optical image to be matched and the SAR image;
s33, searching characteristic points in the preprocessed SAR image to be matched by using a Harris algorithm, detecting the characteristic points by using an SIFT algorithm, and extracting SAR characteristic points;
s34, collecting local neighborhood image blocks in a preset shape with SAR feature points as centers in the preprocessed SAR image to be matched;
s35, collecting local neighborhood image blocks in different positions and in a preset shape in the preprocessed optical image to be matched;
s36, inputting the optical image to be matched and the local neighborhood image block of the SAR image into the image block pair matching model to obtain the local neighborhood image block of the matched optical image; the optical characteristic point at the center of the local neighborhood image block of the matched optical image is the optical homonymous characteristic point.
6. The optical-SAR-based heterogeneous remote sensing image matching method according to claim 5, wherein the image preprocessing method comprises the following steps:
(1) improving the local contrast ratio by using image equalization on the optical image and the SAR image;
(2) gaussian filtering is also used on the SAR image to filter out image noise.
7. The optical-SAR-based heterogeneous remote sensing image matching method according to any one of claims 1-6, wherein the preset shape of the local neighborhood image block is a square.
8. The method for matching the images based on the optical-SAR heterogeneous remote sensing as claimed in claim 7, wherein the size of the image blocks of the local neighborhood is 16 x 16, 32 x 32 or 64 x 64.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110111049.4A CN112861672B (en) | 2021-01-27 | 2021-01-27 | Heterogeneous remote sensing image matching method based on optical-SAR |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110111049.4A CN112861672B (en) | 2021-01-27 | 2021-01-27 | Heterogeneous remote sensing image matching method based on optical-SAR |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112861672A CN112861672A (en) | 2021-05-28 |
| CN112861672B true CN112861672B (en) | 2022-08-05 |
Family
ID=76009596
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110111049.4A Active CN112861672B (en) | 2021-01-27 | 2021-01-27 | Heterogeneous remote sensing image matching method based on optical-SAR |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112861672B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113538536B (en) * | 2021-07-21 | 2022-06-07 | 中国人民解放军国防科技大学 | SAR image information-assisted remote sensing optical image dense cloud detection method and system |
| CN114581771B (en) * | 2022-02-23 | 2023-04-25 | 南京信息工程大学 | Method for detecting collapse building by high-resolution heterogeneous remote sensing |
| CN114565653B (en) * | 2022-03-02 | 2023-07-21 | 哈尔滨工业大学 | Heterologous remote sensing image matching method with rotation change and scale difference |
| CN115019071B (en) * | 2022-05-19 | 2023-09-19 | 昆明理工大学 | Optical image and SAR image matching method and device, electronic equipment and medium |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1736928A1 (en) * | 2005-06-20 | 2006-12-27 | Mitsubishi Electric Information Technology Centre Europe B.V. | Robust image registration |
| CN108510532B (en) * | 2018-03-30 | 2022-07-15 | 西安电子科技大学 | Optical and SAR image registration method based on deep convolution GAN |
| CN109035315A (en) * | 2018-08-28 | 2018-12-18 | 武汉大学 | Merge the remote sensing image registration method and system of SIFT feature and CNN feature |
| CN109492108B (en) * | 2018-11-22 | 2020-12-15 | 上海唯识律简信息科技有限公司 | Deep learning-based multi-level fusion document classification method and system |
| US11494615B2 (en) * | 2019-03-28 | 2022-11-08 | Baidu Usa Llc | Systems and methods for deep skip-gram network based text classification |
| CN110609897B (en) * | 2019-08-12 | 2023-08-04 | 北京化工大学 | Multi-category Chinese text classification method integrating global and local features |
| CN111476251A (en) * | 2020-03-26 | 2020-07-31 | 中国人民解放军战略支援部队信息工程大学 | Remote sensing image matching method and device |
| CN111709980A (en) * | 2020-06-10 | 2020-09-25 | 北京理工大学 | Multi-scale image registration method and device based on deep learning |
| CN112085772B (en) * | 2020-08-24 | 2022-10-18 | 南京邮电大学 | Remote sensing image registration method and device |
-
2021
- 2021-01-27 CN CN202110111049.4A patent/CN112861672B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112861672A (en) | 2021-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112861672B (en) | Heterogeneous remote sensing image matching method based on optical-SAR | |
| CN108460356B (en) | Face image automatic processing system based on monitoring system | |
| CN108090472B (en) | Pedestrian re-identification method and system based on multi-channel consistency characteristics | |
| CN110598692B (en) | Ellipse identification method based on deep learning | |
| CN109740679A (en) | A kind of target identification method based on convolutional neural networks and naive Bayesian | |
| US11715288B2 (en) | Optical character recognition using specialized confidence functions | |
| CN115205521B (en) | Kitchen waste detection method based on neural network | |
| CN114782997B (en) | Pedestrian re-recognition method and system based on multi-loss attention self-adaptive network | |
| CN116071560A (en) | Fruit identification method based on convolutional neural network | |
| WO2015146113A1 (en) | Identification dictionary learning system, identification dictionary learning method, and recording medium | |
| CN109993213A (en) | A kind of automatic identifying method for garment elements figure | |
| CN111369526A (en) | Multi-type old bridge crack identification method based on semi-supervised deep learning | |
| CN112559791A (en) | Cloth classification retrieval method based on deep learning | |
| CN113128518B (en) | Sift mismatch detection method based on twin convolution network and feature mixing | |
| CN110647897B (en) | Zero sample image classification and identification method based on multi-part attention mechanism | |
| CN108960005B (en) | Method and system for establishing and displaying object visual label in intelligent visual Internet of things | |
| CN117079272A (en) | Bullet bottom socket mark feature identification method combining manual features and learning features | |
| CN116128800A (en) | ViG-based mobile phone microphone part defect detection and segmentation method | |
| CN115019367A (en) | Genetic disease face recognition device and method | |
| CN113705731A (en) | End-to-end image template matching method based on twin network | |
| CN114926420A (en) | Identification and counting method of target naan based on cross-level feature enhancement | |
| CN111382685B (en) | Scene recognition method and system based on deep learning | |
| CN114419529A (en) | Cross-modal pedestrian re-identification method and system based on distribution space alignment | |
| CN113268625A (en) | Fine-grained cloth image retrieval method based on cross entropy loss function | |
| CN113688735A (en) | Image classification method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |