CN113743515A - Remote sensing image feature matching method based on self-supervision and self-learning feature points - Google Patents
Remote sensing image feature matching method based on self-supervision and self-learning feature points Download PDFInfo
- Publication number
- CN113743515A CN113743515A CN202111050440.4A CN202111050440A CN113743515A CN 113743515 A CN113743515 A CN 113743515A CN 202111050440 A CN202111050440 A CN 202111050440A CN 113743515 A CN113743515 A CN 113743515A
- Authority
- CN
- China
- Prior art keywords
- remote sensing
- sensing image
- feature
- training
- characteristic point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000000605 extraction Methods 0.000 claims abstract description 74
- 238000005070 sampling Methods 0.000 claims description 27
- 238000012360 testing method Methods 0.000 claims description 18
- 238000011176 pooling Methods 0.000 claims description 12
- 238000007781 pre-processing Methods 0.000 claims description 6
- 230000011218 segmentation Effects 0.000 claims description 6
- 238000010008 shearing Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 description 13
- 230000008569 process Effects 0.000 description 9
- 238000013135 deep learning Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000002372 labelling Methods 0.000 description 3
- 238000005286 illumination Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Molecular Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
Abstract
本发明公开了一种基于自监督自学习特征点的遥感影像特征匹配方法，A、采集得到三维数据库，B、构建特征点提取网络并对训练集的三维数据进行特征点提取训练，训练得到特征点模型m1；C、通过孪生特征点提取网络以随机选取两个遥感影像对为训练数据、特征点label1为真值进行特征点训练；D、通过孪生特征点提取网络以随机选取两个遥感影像对为训练数据、特征点labelk为真值进行特征点训练；E、采用特征点提取网络进行特征点提取训练并得到特征点labeln，通过特征匹配网络对遥感影像对进行特征点匹配训练。本发明基于真实特征点以及遥感影像对之间配准匹配关系作为特征点的描述子，利用特征匹配网络来实现遥感影像的匹配功能，提高了特征匹配效率和精准度。
The invention discloses a remote sensing image feature matching method based on self-supervision and self-learning feature points. A. collecting and obtaining a three-dimensional database; Point model m1; C. Through the twin feature point extraction network, randomly select two remote sensing image pairs as training data, and feature point label1 as the true value for feature point training; D. Through the twin feature point extraction network, randomly select two remote sensing images Perform feature point training on training data and feature point labelk as the true value; E. Use feature point extraction network to perform feature point extraction training and obtain feature point labeln, and perform feature point matching training on remote sensing image pairs through feature matching network. Based on the real feature points and the registration matching relationship between the remote sensing image pairs as the descriptors of the feature points, the invention realizes the matching function of the remote sensing images by using the feature matching network, and improves the feature matching efficiency and accuracy.
Description
Technical Field
The invention relates to the field of remote sensing image feature matching, in particular to a remote sensing image feature matching method based on self-supervision and self-learning feature points.
Background
The image matching aims to identify, align and match the content or the structure with the same or similar attributes in the two images on the pixel. Generally, the images to be matched are usually taken from the same or similar scene or object, or other types of image pairs having the same shape or semantic information, so as to have certain matchability. Because the deep learning method has excellent learning and expression capability on deep features of images, preliminary results on the image matching problem are obtained at present. The main application of deep learning in image matching is to directly learn a pixel-level matching relationship from image pairs containing the same or similar structural content, and the main application form of the deep learning comprises (1) directly designing an end-to-end matching network for learning a feature point set which is more accurate and reliable to detect from an image, learning a main direction or a main scale of each feature point and a feature descriptor which has more distinguishing and matching capabilities. (2) The deep learning method can acquire deep features among image blocks, measure similarity among the features to establish a corresponding relationship, and is generally used for extracting good feature points, constructing descriptors, image retrieval, image registration and the like.
However, the current image matching based on deep learning seriously depends on a large amount of artificial labels as real feature points which can be used for training, for remote sensing images, factors such as a large amount of image data generated by characteristics of multiple time phases, multiple sensors and the like, the illumination angle, shooting conditions and the like not only bring error influence to the image matching process, but also increase the labeling difficulty in the process of artificially labeling the real feature points, and greatly improve the labeling cost.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a remote sensing image feature matching method based on self-supervision and self-learning feature points, a remote sensing image pair is obtained by selecting and registering remote sensing image data in a three-dimensional database, a feature point extraction network is used for carrying out feature point extraction training and obtaining a feature point model m1, data repeated training is carried out on the basis of a feature point model m1 through the feature point extraction network to obtain a feature point model mn and a feature point labeln, and then training is carried out through the feature matching network to obtain a remote sensing image matching model s 1.
The purpose of the invention is realized by the following technical scheme:
a remote sensing image feature matching method based on self-supervision and self-learning feature points comprises the following steps:
A. acquiring a three-dimensional database, wherein three-dimensional data in the three-dimensional database is remote sensing image data; selecting three-dimensional data of data registration from a three-dimensional database to form remote sensing image pairs, wherein each remote sensing image pair comprises two three-dimensional data, a plurality of remote sensing image pairs are formed, and ID numbering is carried out on the remote sensing image pairs; performing data preprocessing operation on all remote sensing image pairs, wherein the data preprocessing operation comprises simultaneous shearing of the image pairs, rotating of mirror images, adjustment of image definition contrast and image Gaussian blur;
B. dividing and classifying all remote sensing image pairs into a training set and a testing set, wherein the number ratio of the remote sensing image pairs in the training set to the remote sensing image pairs in the testing set is 8-9: 1; constructing a feature point extraction network, performing feature point extraction training on the three-dimensional data in the training set, and obtaining a feature point model m1 after the training is finished;
C. randomly selecting two remote sensing image pairs from the training set, and performing characteristic point extraction training on a first remote sensing image pair of the randomly selected two remote sensing image pairs by adopting a characteristic point extraction network based on a characteristic point model m1 to obtain a characteristic point label1 of the remote sensing image pair; constructing a twin characteristic point extraction network, and training characteristic points by randomly selecting two remote sensing image pairs as training data and characteristic point label1 as a true value through the twin characteristic point extraction network to obtain a characteristic point model m 2;
D. randomly selecting two remote sensing image pairs from the training set, and performing characteristic point extraction training on a first remote sensing image pair of the randomly selected two remote sensing image pairs by adopting a characteristic point extraction network based on a characteristic point model m2 to obtain a characteristic point label2 of the remote sensing image pair; randomly selecting two remote sensing image pairs as training data and a characteristic point label2 as a true value through a twin characteristic point extraction network to perform characteristic point training, and training to obtain a characteristic point model m 3; … …, randomly selecting two remote sensing image pairs from the training set, and performing feature point extraction training on a first remote sensing image pair of the two randomly selected remote sensing image pairs by adopting a feature point extraction network based on a feature point model mk to obtain a feature point labelk of the remote sensing image pair; randomly selecting two remote sensing image pairs as training data and a characteristic point label as a true value through a twin characteristic point extraction network to perform characteristic point training, and training to obtain a characteristic point model mn, wherein n is k + 1;
E. carrying out feature point extraction training on any remote sensing image pair in the test set or the training set by adopting a feature point extraction network based on a feature point model mn to obtain a feature point labeln of the remote sensing image pair; and (3) constructing a feature matching network, performing feature point matching training on the remote sensing image pairs in the test set or/and the training set by using the feature matching network and taking the feature point labeln as a true value, constraining the feature point descriptors by adopting a matching relationship during matching, and finishing training to generate a remote sensing image matching model s 1.
The invention also comprises the following method:
F. and carrying out feature point matching test on the remote sensing image pair in the test set based on the generated remote sensing image matching model s 1.
Preferably, the feature point extraction network structure in step B of the present invention is an encoder-decoder structure based on semantic segmentation, the encoder-decoder structure includes an encoder portion and a decoder portion, the encoder portion adopts a VGG type full convolution network, the encoder portion includes eight convolution layers and four maximum pooling layers, and the decoder portion includes an oftmax feature point function sampling model and a reshape feature point sampling model.
Preferably, the twin feature point extraction network structure in step D of the present invention is a twin encoder-decoder structure based on semantic segmentation, the twin encoder-decoder structure includes a twin encoder portion and a merging decoder portion, the twin encoder portion includes an encoder unit using two weight sharing, the encoder unit uses a VGG type full convolution network, the encoder unit includes eight convolution layers and four maximum pooling layers; and the merging decoder part is used for merging data of the two encoder units and comprises an soft ofmax characteristic point function sampling model and a reshape characteristic point sampling model.
Preferably, in step E of the present invention, the feature matching network structure adopts an encoder-decoder network matching structure, the encoder-decoder network matching structure includes two encoder units and two decoder units, the two encoder units of the encoder-decoder network matching structure correspond to the two decoder units one by one, the encoder units adopt a VGG type full convolution network, and the encoder units include eight convolution layers and four maximum pooling layers; the decoder unit comprises an soft max characteristic point function sampling model and a reshape characteristic point sampling model, and the decoder unit is provided with a descriptor generation network.
Preferably, the data registration requirement of the remote sensing image pair selected from the three-dimensional database in the step A of the invention is more than 90%, and the data registration index comprises the number of the characteristic points and the positions of the characteristic points.
Preferably, the three-dimensional data in the three-dimensional database in the step a of the present invention is derived from a remote sensing image device, the remote sensing image is cut to enable the length and width of the cut remote sensing image to be a multiple of 8 when the three-dimensional data is acquired, and the cut remote sensing image is stored in the three-dimensional database.
Preferably, in step a of the present invention, the remote sensing image data includes a geometric structure, and the geometric structure includes a point, a line, a plane, and a cube.
Compared with the prior art, the invention has the following advantages and beneficial effects:
(1) according to the method, remote sensing image pairs are obtained through selecting and registering remote sensing image data in a three-dimensional database, feature point extraction training is carried out through a feature point extraction network to obtain a feature point model m1, data repeated training is carried out through the feature point extraction network based on the feature point model m1 to obtain a feature point model mn and a feature point labeln, and then training is carried out through a feature matching network to obtain a remote sensing image matching model s 1.
(2) The invention extracts real feature points in the remote sensing image pairs by using a self-supervision feature point learning method, takes the registration matching relationship between the real feature points and the remote sensing image pairs as descriptors of the feature points, and realizes the matching function of the remote sensing images by using an improved super-point feature matching network, thereby improving the feature matching efficiency and accuracy.
Drawings
Fig. 1 is a schematic diagram of a feature point extraction network according to this embodiment;
FIG. 2 is a schematic diagram of a twin feature point extraction network according to the present embodiment;
FIG. 3 is a schematic diagram of a feature matching network according to the present embodiment;
fig. 4 is a schematic diagram of the feature point extraction network and the descriptor learning network in fig. 3.
Detailed Description
The present invention will be described in further detail with reference to the following examples:
examples
As shown in fig. 1 to 4, a remote sensing image feature matching method based on self-supervision and self-learning feature points includes the following steps:
A. the method includes the steps of acquiring a three-dimensional database, wherein three-dimensional data in the three-dimensional database is remote sensing image data (the three-dimensional data is also called remote sensing image data, the three-dimensional data can be from various remote sensing image devices, the three-dimensional data can also be virtual three-dimensional data, and the three-dimensional data is widely recorded with remote sensing images of three-dimensional buildings and the like). Selecting three-dimensional data of data registration from a three-dimensional database to form remote sensing image pairs, wherein each remote sensing image pair comprises two three-dimensional data, a plurality of remote sensing image pairs are formed, and ID numbering is carried out on the remote sensing image pairs; and performing data preprocessing operation on all remote sensing image pairs, wherein the data preprocessing operation comprises simultaneous shearing of the image pairs, rotating of mirror images, adjustment of image definition contrast, Gaussian image blurring and the like.
Preferably, in this embodiment, the data registration requirement of the remote sensing image pair (the remote sensing image pair may be a remote sensing image pair in the same area in different time phases and different illumination) selected from the three-dimensional database in step a is over 90%, and the data registration index mainly includes the number of feature points and the position of the feature points. The remote sensing image pair selected by the embodiment has strict requirements on data registration, and the selected remote sensing image pair (especially the remote sensing image pair in the training set) is strictly registered (the data registration requirement is more than 90%).
Preferably, the three-dimensional data in the three-dimensional database in step a of this embodiment is derived from a remote sensing image device, the remote sensing image needs to be cut to acquire the three-dimensional data, the length and the width of the cut remote sensing image are multiples of 8, and the cut remote sensing image is stored in the three-dimensional database. The length and width multiples of the remote sensing image are set according to the remote sensing image equipment from which the remote sensing image comes, the difference of hardware equipment and the like are considered, the length and width of the image pair can be only the multiples of 8 when the image pair is cut, and the specific cutting size can be automatically adjusted according to the condition of the hardware equipment.
B. Dividing and classifying all remote sensing image pairs into a training set and a testing set, wherein the number ratio of the remote sensing image pairs in the training set to the remote sensing image pairs in the testing set is 8-9: 1; and constructing a feature point extraction network, performing feature point extraction training on the three-dimensional data in the training set, and finishing the training to obtain a feature point model m 1. The principle of the feature point extraction network structure adopted in this embodiment is shown in fig. 1, where input refers to input of a remote sensing image (the remote sensing image is data, the remote sensing image is simply referred to as an image, and the remote sensing image data is also referred to as three-dimensional data); output refers to outputting an image with feature points; h denotes the height of the input image; w indicates the width of the input image; h/8 means that the height of the image is one eighth of the original image; w/8 means that the width of the image is one eighth of the original image; the Encoder refers to a network coding structure in the feature point extraction process, and the embodiment mainly adopts a VGG type network and consists of eight convolutional layers and four maximum pooling layers. The Decoder refers to a network decoding structure in a feature point extraction process, and mainly comprises a convolution, an oftmax feature point function sampling model (whether each pixel point of an image is a feature point or not is represented in a probability mode by adopting an index function softmax), and a reshape feature point sampling model (reshape refers to an up-sampling process of an image, and the feature point image with the width being one eighth of the original image is up-sampled to the size of the original image). conv refers to the convolution process.
Preferably, the feature point extraction network structure in step B of this embodiment is an encoder-decoder structure based on semantic segmentation, the encoder-decoder structure includes an encoder portion and a decoder portion, the encoder portion adopts a VGG type full convolution network, the encoder portion includes eight convolution layers and four maximum pooling layers, and the decoder portion includes an oftmax feature point function sampling model and a reshape feature point sampling model.
C. Randomly selecting two remote sensing image pairs from the training set, and performing characteristic point extraction training on a first remote sensing image pair of the randomly selected two remote sensing image pairs by adopting a characteristic point extraction network based on a characteristic point model m1 to obtain a characteristic point label1 of the remote sensing image pair; and (3) constructing a twin characteristic point extraction network, and training the characteristic points by randomly selecting two remote sensing image pairs as training data and the characteristic point label1 as a true value through the twin characteristic point extraction network to obtain a characteristic point model m 2.
D. Randomly selecting two remote sensing image pairs from the training set, and performing characteristic point extraction training on a first remote sensing image pair of the randomly selected two remote sensing image pairs by adopting a characteristic point extraction network based on a characteristic point model m2 to obtain a characteristic point label2 of the remote sensing image pair; randomly selecting two remote sensing image pairs as training data and a characteristic point label2 as a true value through a twin characteristic point extraction network to perform characteristic point training, and training to obtain a characteristic point model m 3; … … (training in sequence according to the method described in step D, thus obtaining label3, label4 … …, a feature point model m4, a feature point model m5 … …, and taking label3 and a feature point model m4 as examples), selecting two remote sensing image pairs randomly from the training set, using a feature point extraction network to perform feature point extraction training on the first remote sensing image pair randomly selected from the two remote sensing image pairs based on the feature point model m3 to obtain the feature point label3 of the remote sensing image pair, using a twin feature point extraction network to perform feature point training on the two remote sensing image pairs randomly selected as training data and the feature point label3 as a true value to obtain the feature point model m4, selecting two remote sensing image pairs randomly from the training set, using the feature point extraction network to perform feature point extraction training on the first remote sensing image pair randomly selected from the two remote sensing image pairs based on the feature point model mk, obtaining a characteristic point labelk of a remote sensing image pair; randomly selecting two remote sensing image pairs as training data and a characteristic point label as a true value through a twin characteristic point extraction network to perform characteristic point training, and training to obtain a characteristic point model mn, wherein n is k + 1;
preferably, the twin feature point extraction network structure in step D of this embodiment is a twin encoder-decoder structure based on semantic segmentation, the twin encoder-decoder structure includes a twin encoder portion and a merging decoder portion, the twin encoder portion includes an encoder unit using two weight shares, the encoder unit uses a VGG-type full convolution network, and the encoder unit includes eight convolution layers and four maximum pooling layers; and the merging decoder part is used for merging data of the two encoder units and comprises an soft ofmax characteristic point function sampling model and a reshape characteristic point sampling model.
The twin feature point extraction network structure principle adopted in this embodiment is shown in fig. 2, where input1 or input2 indicates that a remote sensing image (the remote sensing image is data, the remote sensing image is simply referred to as an image, and the remote sensing image data is also referred to as three-dimensional data) is input; output refers to outputting an image with feature points; h denotes the height of the input image; w indicates the width of the input image; h/8 means that the height of the image is one eighth of the original image; w/8 means that the width of the image is one eighth of the original image; the Encoder refers to a network coding structure in the feature point extraction process, and the embodiment mainly adopts a VGG type network, and consists of eight convolutional layers and four maximum pooling layers, and the network coding structure comprises two Encoder units. The Decoder refers to a network decoding structure in a feature point extraction process, and mainly comprises a convolution, an oftmax feature point function sampling model (whether each pixel point of an image is a feature point or not is represented in a probability mode by adopting an index function softmax), and a reshape feature point sampling model (reshape refers to an up-sampling process of an image, and the feature point image with the width being one eighth of the original image is up-sampled to the size of the original image). conv refers to the convolution process; add refers to the addition of two input images between channels.
E. Carrying out feature point extraction training on any remote sensing image pair in the test set by adopting a feature point extraction network based on a feature point model mn to obtain a feature point labeln (which can be used for feature point extraction and feature descriptor generation) of the remote sensing image pair; and (3) constructing a feature matching network, performing feature point matching training on the remote sensing image pair in the test set training set by using the feature matching network and taking the feature point labeln as a true value, constraining the feature point descriptor by adopting a matching relation during matching, and finishing training to generate a remote sensing image matching model s 1.
Preferably, in step E of this embodiment, the feature matching network structure adopts an encoder-decoder network matching structure, the encoder-decoder network matching structure includes two encoder units and two decoder units, the two encoder units of the encoder-decoder network matching structure correspond to the two decoder units one by one, the encoder unit adopts a VGG-type full convolution network, and the encoder unit includes eight convolution layers and four maximum pooling layers; the decoder unit comprises an soft max characteristic point function sampling model and a reshape characteristic point sampling model, and the decoder unit is provided with a descriptor generation network.
The structural principle of the feature matching network adopted in the embodiment is shown in fig. 3 and 4, in which input1 or input2 indicates that a remote sensing image (the remote sensing image is data, the remote sensing image is simply referred to as an image, and the remote sensing image data is also referred to as three-dimensional data) is input; output1 or output 2 indicates that images with feature points are output; h denotes the height of the input image; w indicates the width of the input image; h/8 means that the height of the image is one eighth of the original image; w/8 means that the width of the image is one eighth of the original image; the Encoder refers to a network coding structure in the feature point extraction process, and the embodiment mainly adopts a VGG type network, and consists of eight convolutional layers and four maximum pooling layers, and the network coding structure comprises two Encoder units. The Decoder refers to a network decoding structure in a feature point extraction process, and mainly comprises a convolution, an oftmax feature point function sampling model (whether each pixel point of an image is a feature point or not is represented in a probability mode by adopting an index function softmax), and a reshape feature point sampling model (reshape refers to an up-sampling process of an image, and the feature point image with the width being one eighth of the original image is up-sampled to the size of the original image). conv refers to the convolution process; add refers to the addition of two input images between channels. Interest Points Network refers to a feature point extraction Network; descriptors Networks refers to descriptor generation Networks. Bi-Cubic Interpolate refers to a bicubic interpolation process; L2-Norm refers to the L2 Norm.
F. And carrying out feature point matching test on the remote sensing image pair in the test set based on the generated remote sensing image matching model s 1.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (8)
1. A remote sensing image feature matching method based on self-supervision and self-learning feature points is characterized by comprising the following steps: the method comprises the following steps:
A. acquiring a three-dimensional database, wherein three-dimensional data in the three-dimensional database is remote sensing image data; selecting three-dimensional data of data registration from a three-dimensional database to form remote sensing image pairs, wherein each remote sensing image pair comprises two three-dimensional data, a plurality of remote sensing image pairs are formed, and ID numbering is carried out on the remote sensing image pairs; performing data preprocessing operation on all remote sensing image pairs, wherein the data preprocessing operation comprises simultaneous shearing of the image pairs, rotating of mirror images, adjustment of image definition contrast and image Gaussian blur;
B. dividing and classifying all remote sensing image pairs into a training set and a testing set, wherein the number ratio of the remote sensing image pairs in the training set to the remote sensing image pairs in the testing set is 8-9: 1; constructing a feature point extraction network, performing feature point extraction training on the three-dimensional data in the training set, and obtaining a feature point model m1 after the training is finished;
C. randomly selecting two remote sensing image pairs from the training set, and performing characteristic point extraction training on a first remote sensing image pair of the randomly selected two remote sensing image pairs by adopting a characteristic point extraction network based on a characteristic point model m1 to obtain a characteristic point label1 of the remote sensing image pair; constructing a twin characteristic point extraction network, and training characteristic points by randomly selecting two remote sensing image pairs as training data and characteristic point label1 as a true value through the twin characteristic point extraction network to obtain a characteristic point model m 2;
D. randomly selecting two remote sensing image pairs from the training set, and performing characteristic point extraction training on a first remote sensing image pair of the randomly selected two remote sensing image pairs by adopting a characteristic point extraction network based on a characteristic point model m2 to obtain a characteristic point label2 of the remote sensing image pair; randomly selecting two remote sensing image pairs as training data and a characteristic point label2 as a true value through a twin characteristic point extraction network to perform characteristic point training, and training to obtain a characteristic point model m 3; … …, randomly selecting two remote sensing image pairs from the training set, and performing feature point extraction training on a first remote sensing image pair of the two randomly selected remote sensing image pairs by adopting a feature point extraction network based on a feature point model mk to obtain a feature point labelk of the remote sensing image pair; randomly selecting two remote sensing image pairs as training data and a characteristic point label as a true value through a twin characteristic point extraction network to perform characteristic point training, and training to obtain a characteristic point model mn, wherein n is k + 1;
E. carrying out feature point extraction training on any remote sensing image pair in the test set or the training set by adopting a feature point extraction network based on a feature point model mn to obtain a feature point labeln of the remote sensing image pair; and (3) constructing a feature matching network, performing feature point matching training on the remote sensing image pairs in the test set or/and the training set by using the feature matching network and taking the feature point labeln as a true value, constraining the feature point descriptors by adopting a matching relationship during matching, and finishing training to generate a remote sensing image matching model s 1.
2. The remote sensing image feature matching method based on the self-supervision and self-learning feature points as claimed in claim 1, characterized in that: the method also comprises the following steps:
F. and carrying out feature point matching test on the remote sensing image pair in the test set based on the generated remote sensing image matching model s 1.
3. The remote sensing image feature matching method based on the self-supervision and self-learning feature points as claimed in claim 1, characterized in that: and B, the characteristic point extraction network structure in the step B is an encoder-decoder structure based on semantic segmentation, the encoder-decoder structure comprises an encoder part and a decoder part, the encoder part adopts a VGG type full convolution network, the encoder part comprises eight convolution layers and four maximum pooling layers, and the decoder part comprises an Microsoft max characteristic point function sampling model and a reshape characteristic point sampling model.
4. The remote sensing image feature matching method based on the self-supervision and self-learning feature points as claimed in claim 1, characterized in that: the twin feature point extraction network structure in the step D is a twin encoder-decoder structure based on semantic segmentation, the twin encoder-decoder structure comprises a twin encoder part and a merging decoder part, the twin encoder part comprises an encoder unit adopting two weight sharing, the encoder unit adopts a VGG type full convolution network, and the encoder unit comprises eight convolution layers and four maximum pooling layers; and the merging decoder part is used for merging data of the two encoder units and comprises an soft ofmax characteristic point function sampling model and a reshape characteristic point sampling model.
5. The remote sensing image feature matching method based on the self-supervision and self-learning feature points as claimed in claim 1, characterized in that: the characteristic matching network structure in the step E adopts an encoder-decoder network matching structure, the encoder-decoder network matching structure comprises two encoder units and two decoder units, the two encoder units of the encoder-decoder network matching structure correspond to the two decoder units one by one, the encoder units adopt VGG type full convolution networks, and the encoder units comprise eight convolution layers and four maximum pooling layers; the decoder unit comprises an soft max characteristic point function sampling model and a reshape characteristic point sampling model, and the decoder unit is provided with a descriptor generation network.
6. The remote sensing image feature matching method based on the self-supervision and self-learning feature points as claimed in claim 1, characterized in that: and B, the data registration requirement of the remote sensing image pair selected from the three-dimensional database in the step A is more than 90%, and the data registration index comprises the number of the characteristic points and the positions of the characteristic points.
7. The remote sensing image feature matching method based on the self-supervision and self-learning feature points as claimed in claim 1, characterized in that: and D, the three-dimensional data in the three-dimensional database in the step A is from a remote sensing image device, the remote sensing image is required to be cut when the three-dimensional data is acquired, the length and the width of the cut remote sensing image are multiples of 8, and the cut remote sensing image is stored in the three-dimensional database.
8. The remote sensing image feature matching method based on the self-supervision and self-learning feature points as claimed in claim 1, characterized in that: the remote sensing image data in the step A comprises a geometric structure, wherein the geometric structure comprises points, lines, surfaces and a vertical surface body.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111050440.4A CN113743515B (en) | 2021-09-08 | 2021-09-08 | Remote sensing image feature matching method based on self-supervised and self-learning feature points |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111050440.4A CN113743515B (en) | 2021-09-08 | 2021-09-08 | Remote sensing image feature matching method based on self-supervised and self-learning feature points |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113743515A true CN113743515A (en) | 2021-12-03 |
| CN113743515B CN113743515B (en) | 2022-03-11 |
Family
ID=78737087
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111050440.4A Active CN113743515B (en) | 2021-09-08 | 2021-09-08 | Remote sensing image feature matching method based on self-supervised and self-learning feature points |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113743515B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114463547A (en) * | 2021-12-29 | 2022-05-10 | 广州极飞科技股份有限公司 | Training method of feature extraction network model, image processing method and device thereof |
| CN114463549A (en) * | 2021-12-29 | 2022-05-10 | 广州极飞科技股份有限公司 | Training method of feature extraction network model, image processing method and device thereof |
| CN115186119A (en) * | 2022-09-07 | 2022-10-14 | 深圳市华曦达科技股份有限公司 | Picture processing method and system based on picture and text combination and readable storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109784223A (en) * | 2018-12-28 | 2019-05-21 | 珠海大横琴科技发展有限公司 | A kind of multi-temporal remote sensing image matching process and system based on convolutional neural networks |
| CN111028277A (en) * | 2019-12-10 | 2020-04-17 | 中国电子科技集团公司第五十四研究所 | SAR and optical remote sensing image registration method based on pseudo-twin convolutional neural network |
| CN111127538A (en) * | 2019-12-17 | 2020-05-08 | 武汉大学 | A 3D reconstruction method for multi-view images based on convolutional cyclic encoding-decoding structure |
| CN111161218A (en) * | 2019-12-10 | 2020-05-15 | 核工业北京地质研究院 | High-resolution remote sensing image change detection method based on twin convolutional neural network |
| CN111369601A (en) * | 2020-02-12 | 2020-07-03 | 西北工业大学 | A Remote Sensing Image Registration Method Based on Siamese Network |
| CN111612066A (en) * | 2020-05-21 | 2020-09-01 | 成都理工大学 | Remote sensing image classification method based on deep fusion convolutional neural network |
| CN111832501A (en) * | 2020-07-20 | 2020-10-27 | 中国人民解放军战略支援部队航天工程大学 | An intelligent text description method for remote sensing images for satellite in-orbit applications |
| CN112668494A (en) * | 2020-12-31 | 2021-04-16 | 西安电子科技大学 | Small sample change detection method based on multi-scale feature extraction |
| CN112883850A (en) * | 2021-02-03 | 2021-06-01 | 湖北工业大学 | Multi-view aerospace remote sensing image matching method based on convolutional neural network |
| CN113223068A (en) * | 2021-05-31 | 2021-08-06 | 西安电子科技大学 | Multi-modal image registration method and system based on depth global features |
-
2021
- 2021-09-08 CN CN202111050440.4A patent/CN113743515B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109784223A (en) * | 2018-12-28 | 2019-05-21 | 珠海大横琴科技发展有限公司 | A kind of multi-temporal remote sensing image matching process and system based on convolutional neural networks |
| CN111028277A (en) * | 2019-12-10 | 2020-04-17 | 中国电子科技集团公司第五十四研究所 | SAR and optical remote sensing image registration method based on pseudo-twin convolutional neural network |
| CN111161218A (en) * | 2019-12-10 | 2020-05-15 | 核工业北京地质研究院 | High-resolution remote sensing image change detection method based on twin convolutional neural network |
| CN111127538A (en) * | 2019-12-17 | 2020-05-08 | 武汉大学 | A 3D reconstruction method for multi-view images based on convolutional cyclic encoding-decoding structure |
| CN111369601A (en) * | 2020-02-12 | 2020-07-03 | 西北工业大学 | A Remote Sensing Image Registration Method Based on Siamese Network |
| CN111612066A (en) * | 2020-05-21 | 2020-09-01 | 成都理工大学 | Remote sensing image classification method based on deep fusion convolutional neural network |
| CN111832501A (en) * | 2020-07-20 | 2020-10-27 | 中国人民解放军战略支援部队航天工程大学 | An intelligent text description method for remote sensing images for satellite in-orbit applications |
| CN112668494A (en) * | 2020-12-31 | 2021-04-16 | 西安电子科技大学 | Small sample change detection method based on multi-scale feature extraction |
| CN112883850A (en) * | 2021-02-03 | 2021-06-01 | 湖北工业大学 | Multi-view aerospace remote sensing image matching method based on convolutional neural network |
| CN113223068A (en) * | 2021-05-31 | 2021-08-06 | 西安电子科技大学 | Multi-modal image registration method and system based on depth global features |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114463547A (en) * | 2021-12-29 | 2022-05-10 | 广州极飞科技股份有限公司 | Training method of feature extraction network model, image processing method and device thereof |
| CN114463549A (en) * | 2021-12-29 | 2022-05-10 | 广州极飞科技股份有限公司 | Training method of feature extraction network model, image processing method and device thereof |
| CN115186119A (en) * | 2022-09-07 | 2022-10-14 | 深圳市华曦达科技股份有限公司 | Picture processing method and system based on picture and text combination and readable storage medium |
| CN115186119B (en) * | 2022-09-07 | 2022-12-06 | 深圳市华曦达科技股份有限公司 | Picture processing method and system based on picture and text combination and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113743515B (en) | 2022-03-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113743515B (en) | Remote sensing image feature matching method based on self-supervised and self-learning feature points | |
| CN111050219B (en) | Method and system for processing video content using a spatio-temporal memory network | |
| CN114359283B (en) | Defect detection method based on Transformer and electronic equipment | |
| CN109902748A (en) | An Image Semantic Segmentation Method Based on Multi-layer Information Fusion Fully Convolutional Neural Network | |
| CN111127538B (en) | A 3D reconstruction method of multi-view images based on convolutional cyclic encoding-decoding structure | |
| CN110866934B (en) | Method and system for segmentation of complex point cloud based on normative coding | |
| CN106780546B (en) | The personal identification method of motion blur encoded point based on convolutional neural networks | |
| CN110727819B (en) | Method for retrieving scale-adaptive pathological full-section image database | |
| CN110677639A (en) | Non-reference video quality evaluation method based on feature fusion and recurrent neural network | |
| CN111768452A (en) | A non-contact automatic mapping method based on deep learning | |
| CN113160291B (en) | Change detection method based on image registration | |
| CN110197505A (en) | Remote sensing images binocular solid matching process based on depth network and semantic information | |
| CN114913498A (en) | Parallel multi-scale feature aggregation lane line detection method based on key point estimation | |
| CN116524189A (en) | High-resolution remote sensing image semantic segmentation method based on coding and decoding indexing edge characterization | |
| GB2579262A (en) | Space-time memory network for locating target object in video content | |
| CN112149535A (en) | A lane line detection method and device combining SegNet and U-Net | |
| CN114638866B (en) | A point cloud registration method and system based on local feature learning | |
| CN112906675B (en) | Method and system for detecting non-supervision human body key points in fixed scene | |
| CN110458132A (en) | An End-to-End Text Recognition Method of Indefinite Length | |
| CN113313031A (en) | Deep learning-based lane line detection and vehicle transverse positioning method | |
| CN118351538A (en) | A remote sensing image road segmentation method combining channel attention mechanism and multi-layer axial Transformer feature fusion structure | |
| CN112288626A (en) | A face illusion method and system based on dual-path deep fusion | |
| CN116049469A (en) | Multi-matching search and super-resolution reconstruction method based on reference diagram | |
| CN115035193A (en) | Bulk grain random sampling method based on binocular vision and image segmentation technology | |
| CN113763539A (en) | An Implicit Function 3D Reconstruction Method Based on Image and 3D Input |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |