CN114494152A - Unsupervised change detection method based on associated learning model - Google Patents
Unsupervised change detection method based on associated learning model Download PDFInfo
- Publication number
- CN114494152A CN114494152A CN202111663288.7A CN202111663288A CN114494152A CN 114494152 A CN114494152 A CN 114494152A CN 202111663288 A CN202111663288 A CN 202111663288A CN 114494152 A CN114494152 A CN 114494152A
- Authority
- CN
- China
- Prior art keywords
- change detection
- learning model
- probability
- data
- gradient
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an unsupervised change detection method based on an associated learning model, which comprises the following steps: constructing a symmetrical bilateral associative network structure; establishing an association learning model aiming at change detection; optimizing the associated learning model based on a contrast divergence optimization method; and extracting an optimization result and outputting a change area. The invention defines an associated learning model aiming at the change detection problem, trains the neural network in an unsupervised mode, has stronger adaptability and can adapt to various data types including homologous, heterologous, heterogeneous and other remote sensing data.
Description
Technical Field
The invention belongs to the field of change detection, and particularly relates to an unsupervised change detection method based on an associated learning model.
Background
The change detection is to detect a changed region of the same scene image taken at different times. It is of great interest in many applications, including video monitoring, medical diagnosis and treatment, particularly in remote monitoring and land use analysis. Learning is based on whether the change detection method requires a manually annotated sample, and can be classified into supervised, semi-supervised, and unsupervised detection methods. Unsupervised methods typically analyze multi-temporal images to detect changing regions, which can be detected without manually labeling data. Whereas supervised or semi-supervised uses artificially labeled data to learn distributions, classifications, or other a priori information in order to generate accurate regions of variation in complex scenes. However, due to the large scale and diversity of the remotely sensed images, models learned from limited label data are difficult to adapt to most scenes. Due to the variety of images in practical problems, their applications are often limited. For example, models trained from natural images are difficult to apply directly to remote sensing images. Unsupervised methods are widely used, but their accuracy depends on the effectiveness of preprocessing methods such as geometry adjustment (co-registration) and radiation correction (de-noising, atmospheric correction, normalization).
The widely used change detection method can be divided into two steps, given co-registered multi-temporal images 1) pixel or region comparison generates difference images; 2) the difference image is analyzed to generate a binary image showing the changed area. We refer to this as a comparison-based approach. Another widely used change detection prototype is a classification-based change detection method in which multi-temporal images are compared based on the classification results. Classification-based methods are very robust to co-registration errors because they compare the objects of the images. They can be applied to many scenarios. However, to train accurate classifiers, classification-based methods are typically supervised or semi-supervised.
For comparison-based methods, multi-temporal images can be compared by transforming features to avoid false positives. For the same type of image acquired by different sensors (including SAR for example for multi-sensor images), the intensities can be assumed to be linearly related. Since linear relationships are easy to handle due to signal or spectral characteristics considerations, many unsupervised change detection methods for multi-sensor images have been proposed. For example, a prediction-based non-parametric regression method is proposed for multi-sensor images of different spectral bands to convert one image into the spectral domain of another image.
Feature-based change detection methods focus more on the comparison process to produce significant changes in complex scenes. Some methods learn comparable features of multi-source images. Presdes et al propose a physical model based on a multidimensional distribution mixed with available invariant samples. Since change detection can be a classification problem, many supervised methods have been developed based on deep learning methods that learn to compare multi-time images to trainable hierarchical features. However, to train a deep network, enough labeled training samples are required, which limits their widespread use. Feature-based change detection methods can avoid many insignificant changes caused by sensor noise, illumination changes, non-uniform attenuation, atmospheric absorption, and even heterogeneous sensors. However, for unsupervised methods, they require a higher alignment accuracy, i.e. a co-registration method, since they typically compare local features.
Given two images, most co-registration methods take the captured scene as a plane and transform the images using a fixed transformation template, such as shift, rotation, and affine transformation. Therefore, for high resolution, images captured from different angles are difficult to align perfectly. Such images are common in many situations, such as Very High Resolution (VHR) optical remote sensing images and those captured by a moving unmanned aerial vehicle (UVA). Therefore, in many change detection scenarios, a method that is robust against co-registration errors is needed. The target-based change detection method first classifies objects in the image and then compares the objects, which is robust to co-registration errors. In order to generate accurate regions of variation, the classification method should be specially designed. The intuitive approach is to classify the multiple time images separately, then compare the corresponding classes, and generate the changed regions. The accuracy of these methods depends on the accuracy of the classification method, respectively, and there is error propagation. Even with high self-matching and robustness to co-registration errors, object-based methods are typically supervised for learning accurate classifiers.
Disclosure of Invention
The invention aims to provide an unsupervised change detection method based on an associated learning model.
The technical solution for realizing the invention is as follows: an unsupervised change detection method based on an associated learning model comprises the following steps:
firstly, a symmetrical bilateral network structure is constructed for association learning, two registered pictures are input at two ends of the network, and feature graphs are obtained through a plurality of layers of convolution layers respectively for comparison and association;
secondly, defining a correlation learning model aiming at change detection, wherein the model has two parameters to be optimized, including network parameters and a change probability graph;
optimizing the association learning model, and optimizing a multi-layer association model by designing a new sampling method by using a contrast divergence optimization method;
and step four, normalizing the optimized change probability graph to obtain a difference graph, and segmenting the difference graph to obtain a change detection result.
An electronic device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the above unsupervised change detection method based on the associative learning model.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, implements the above-mentioned unsupervised change detection method based on a associative learning model.
Compared with the prior art, the invention has the remarkable characteristics that: (1) defining a bilateral convolution neural network architecture; (2) defining an associated learning model aiming at change detection, and utilizing two input images to learn network parameters in an unsupervised mode; (3) and optimizing the association learning model, and optimizing the multilayer association model by designing a new sampling method by using a contrast divergence optimization method.
The present invention is described in further detail below with reference to the attached drawing figures.
Drawings
FIG. 1 is a flow chart of the present invention.
Fig. 2 is a schematic diagram of a content encoding network structure according to the present invention.
FIG. 3 is a graph of ROC and PR curves for different methods on a homogeneous remote sensing dataset, wherein (a) the PR curve and (b) the ROC curve.
FIG. 4 shows ROC and PR curves for different methods on heterogeneous remote sensing data, wherein (a) PR curve and (b) ROC curve.
FIG. 5 shows two given images X1And X2Schematic diagram of the whole parameter learning process.
Detailed Description
The invention provides an unsupervised change detection method based on an associated learning model, which is characterized in that a bilateral convolutional neural network is constructed, a probability model based on a characteristic distance square energy function is established, the difference degree of each pixel point is obtained through the optimization of the model, model parameters are learned in an unsupervised mode, change information is obtained from a change mask, and finally a change detection result is obtained.
With reference to fig. 1, the method comprises the following steps:
firstly, a symmetrical bilateral network structure is constructed for association learning, two registered pictures are input at two ends of the network, and feature graphs are obtained through a plurality of layers of convolution layers respectively for comparison and association. The network structure is shown in fig. 2 and is composed of a plurality of convolutional layers, the input and output of each convolutional layer are respectively a plurality of characteristic graphs, and the characteristic graph of the output layer is generated by the input layer through convolution weighting and an activation function.
And secondly, defining an associated learning model aiming at change detection, wherein the model has two parameters to be optimized, including network parameters, namely weight and bias and a change probability graph.
(1) The associated learning model is a probability model, and based on the probability model, the probability of the input data is defined as:
where Z is the partition function, θ is the set of network parameters including weights and biases, and u is the variation probability map, the probability model is optimized by maximizing the log-likelihood of the probability, which means that the input data (i.e., X) is increased1And X2) While reducing the energy of all other data, the network can then specifically learn the relationship between the two images.
(2) To enable change detection, the energy function E (X) is designed by defining the change detection1,X2(ii) a θ, u), we define the energy function as the square of the characteristic distance between the neural networks on both sides, expressed as follows:
wherein, FLAnd FRRespectively representing the output profile of the network, i being the index of the elements in the profile, u being a pseudo-probability map representing the invariant probability of each content object, which is a trainable parameter, the objective of the probability model being to generate an energy field distributed with the data throughout the data space and thus will be trained together with the network parameter set θ.
Thirdly, optimizing the association learning model, and optimizing a multi-layer association model by designing a new sampling method by using a contrast divergence optimization method;
using gradient descent to maximize the log-likelihood, the negative gradient- Δ (θ, u) of the likelihood function with respect to two parameters θ and u is derived as follows:
where Δ is the derivative sign, the first term is the observed X1,X2The gradient is easily calculated because the calculation formula is the Euclidean distance, and the second term is the gradient expectation of all possible data energies, whereinAll possible data in a data space are represented, the data space is large and cannot be estimated, the bilateral architecture causes difficulty in probability derivation, and therefore by adopting the reverse sampling data, low-probability input can be driven to move to a high-probability area, which is equivalent to an optimization problem:
the first term is easily calculated by back-propagation algorithm, Z is a function of θ, u, X1,X2Since the change in Z does not affect the value of Z, in the second term, Z can be taken as a constant with a gradient of 0. Then, the sampling data is obtained through multiple iterations of the gradient descent method, however, the sampling process seriously affects the optimization efficiency of the model, and we can also consider the sampling process as a process of generating balanced distribution by innumerable gradient descent, so in the model, the contrast divergence can also be minimized, and the gradient formula can be updated as follows:
whereinRespectively represent from X1,X2Sample data obtained by one step gradient descent, i.e. Wherein alpha isxIs the learning rate. The gradient counter-propagates as follows:
in the method, u is taken as a sigmoid unconstrained variable function, and then optimization can be carried out through gradient:
After optimization, u can represent the change probability of different pixel points, a difference graph representing the change degree can be obtained through normalization, and the difference graph is segmented to obtain a change detection result.
And step four, normalizing the optimized change probability graph to obtain a difference graph, and segmenting the difference graph to obtain a change detection result.
FIG. 5 shows two given images X1And X2The whole parameter learning process.
The invention utilizes two input images for learning network parameters in an unsupervised manner. And automatically acquiring the difference degree of each pixel point by optimizing the model, and further identifying. Probabilistic model learning captures the distribution in an unsupervised manner. Therefore, the method can adapt to various scenes without training of the marked data. The method has the advantages that the method is proved by the fact that the method is verified under different data types and scenes, and the robustness of the co-registration error is improved.
The effect of the invention can be further illustrated by the following simulation experiment:
simulation conditions
The simulation experiment used 2 data sets: homogeneous remote sensing data set and heterogeneous remote sensing data. Detecting changes in single-band optical images with similar climatic conditions is a simple task, even subtraction operators can produce accurate results. Here, this dataset was used to verify the feasibility of the proposed probabilistic model in change detection and compare the capabilities of the four unsupervised change detection methods in processing image details, the simulation experiments were all configured in Inteli7-8700KCPU (3.7GHz) and RTX2080Ti GPU under Windows operating system, and the programs were written using C + + and Visualstudio 2017.
The evaluation indexes adopted by the invention are an evaluation method of clustering Precision (ACC), Precision recall rate (PR), Receiver Operating Characteristic curve (ROC), average accuracy rate (AP) and kappa coefficient.
Emulated content
The invention adopts a real homogeneous remote sensing data set and a heterogeneous remote sensing data set to check the performance of the algorithm. In order to test the performance of the algorithm, the unsupervised change detection method based on the associative learning is compared with the change detection algorithm popular internationally at present. The comparison method comprises the following steps: subtrraction, CDL, SCCN, Fast Map.
Analysis of simulation experiment results
Table 1 shows the comparison results of different evaluation indexes under different change detection algorithms in two data sets, and it can be seen from table 1 that, in a homogeneous data set, the unsupervised change detection method based on the association learning model provided by the present invention has strong adaptability, can adapt to various data types, can highlight changed areas, and avoid the influence of unchanged buildings, and compared with subtrraction, CDL, SCCN, Fast Map, the accuracy in different evaluation indexes is significantly improved. Table 2 shows the run times under this method. The effect graphs of the method of the invention on different data sets are shown in fig. 3 and fig. 4, and the simulation experiment results of the above real data sets show the effectiveness of the method of the invention.
TABLE 1 different algorithmic quantitative evaluation of the data set (AP, AUR, ACC, Kappa)
TABLE 2 time cost per data set
Data set | Homogeneous remote sensing data set | Heterogeneous remote sensing data set |
Time(s) | 22.9 | 80.6 |
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (5)
1. An unsupervised change detection method based on a correlation learning model is characterized by comprising the following steps:
firstly, a symmetrical bilateral network structure is constructed for association learning, two registered pictures are input at two ends of the network, and feature graphs are obtained through a plurality of layers of convolution layers respectively for comparison and association;
secondly, defining a correlation learning model aiming at change detection, wherein the model has two parameters to be optimized, including network parameters and a change probability graph;
optimizing the association learning model, and optimizing a multi-layer association model by designing a new sampling method by using a contrast divergence optimization method;
and step four, normalizing the optimized change probability graph to obtain a difference graph, and segmenting the difference graph to obtain a change detection result.
2. The unsupervised change detection method based on the associative learning model according to claim 1, wherein the associative learning model is defined in the second step, and the specific process is as follows:
(1) the associated learning model is a probability model, and based on the probability model, the probability of the input data is defined as:
wherein Z is a distribution function, theta is a network parameter set comprising weight and bias, and u is a variation probability graph;
(2) designing an energy function E (X) by defining a change detection1,X2(ii) a θ, u), defining the energy function as the square of the characteristic distance between the neural networks on both sides, expressed as follows:
wherein, FLAnd FRThe output profiles of the network are represented separately, i is the index of the elements in the profile, u is the variation probability map, representing the invariant probability of each content object, which is a trainable parameter, the objective of the probability model is to generate an energy field distributed with the data throughout the data space, and thus will be trained together with the network parameter set θ.
3. The unsupervised change detection method based on associative learning model according to claim 1, characterized in that in the third step the associative learning model is optimized, using gradient descent to maximize the log-likelihood, the negative gradient- Δ (θ, u) of the likelihood function with respect to two parameters θ and u is derived as follows:
where Δ is the derivative sign, the first term is the observed X1,X2The second term is the gradient expectation of all possible data energies, whereinRepresenting all possible data in the data space; and (3) driving the low-probability input to move to a high-probability region by adopting reverse sampling data, which is equivalent to an optimization problem:
the first term is calculated by a back propagation algorithm, Z is a function of θ, u, X1,X2So, in the second term, Z is taken as a constant with a gradient of 0; then, the sampling data is obtained through a plurality of iterations of the gradient descent method, the sampling process is regarded as a process of generating balanced distribution by innumerable gradient descent, therefore, in the model, the contrast divergence can be minimized, and the gradient formula is updated as follows:
whereinRespectively represent from X1,X2Sample data obtained by one step gradient descent, i.e. Wherein alpha isxIs the learning rate; the gradient counter-propagates as follows:
redefining an unconstrained intermediate variable T, enabling u to be sigmoid (T), and then optimizing through a gradient:
4. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the association learning model based unsupervised change detection method of any of claims 1-3 when executing the program.
5. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a method for unsupervised change detection based on a associative learning model according to any one of claims 1 to 3.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111663288.7A CN114494152A (en) | 2021-12-31 | 2021-12-31 | Unsupervised change detection method based on associated learning model |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111663288.7A CN114494152A (en) | 2021-12-31 | 2021-12-31 | Unsupervised change detection method based on associated learning model |
Publications (1)
Publication Number | Publication Date |
---|---|
CN114494152A true CN114494152A (en) | 2022-05-13 |
Family
ID=81508990
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202111663288.7A Pending CN114494152A (en) | 2021-12-31 | 2021-12-31 | Unsupervised change detection method based on associated learning model |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114494152A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115908987A (en) * | 2023-01-17 | 2023-04-04 | 南京理工大学 | Target detection method based on hierarchical automatic association learning |
-
2021
- 2021-12-31 CN CN202111663288.7A patent/CN114494152A/en active Pending
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115908987A (en) * | 2023-01-17 | 2023-04-04 | 南京理工大学 | Target detection method based on hierarchical automatic association learning |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109961089B (en) | Small sample and zero sample image classification method based on metric learning and meta learning | |
Liu et al. | A deep convolutional coupling network for change detection based on heterogeneous optical and radar images | |
CN108470332B (en) | Multi-target tracking method and device | |
CN107316013B (en) | Hyperspectral image classification method based on NSCT (non-subsampled Contourlet transform) and DCNN (data-to-neural network) | |
US11373064B2 (en) | Cross-modality automatic target recognition | |
CN113723255B (en) | Hyperspectral image classification method and storage medium | |
CN110619352A (en) | Typical infrared target classification method based on deep convolutional neural network | |
CN109492700A (en) | A kind of Target under Complicated Background recognition methods based on multidimensional information fusion | |
Goumiri et al. | A new hybrid model of convolutional neural networks and hidden Markov chains for image classification | |
CN111242028A (en) | Remote sensing image ground object segmentation method based on U-Net | |
CN114494152A (en) | Unsupervised change detection method based on associated learning model | |
Jing et al. | A tomato disease identification method based on leaf image automatic labeling algorithm and improved YOLOv5 model | |
Dai et al. | Research on hyper-spectral remote sensing image classification by applying stacked de-noising auto-encoders neural network | |
CN108960005B (en) | Method and system for establishing and displaying object visual label in intelligent visual Internet of things | |
CN116502899B (en) | Risk rating model generation method, device and storage medium based on artificial intelligence | |
US20230229897A1 (en) | Distances between distributions for the belonging-to-the-distribution measurement of the image | |
Roy et al. | AWDMC-Net: classification of adversarial weather degraded multiclass scenes using a convolution neural network | |
Sreedevi et al. | Development of weighted ensemble transfer learning for tomato leaf disease classification solving low resolution problems | |
CN116434074A (en) | Target identification method based on adjacent branch complementation significance and multiple priori sparse representation | |
CN114998731A (en) | Intelligent terminal navigation scene perception identification method | |
Rezvanian et al. | Patch-based sparse and convolutional autoencoders for anomaly detection in hyperspectral images | |
Jun et al. | Two-view correspondence learning via complex information extraction | |
CN114494154A (en) | Unsupervised change detection method based on content coding | |
Tian et al. | Recognition Method of Corn and Rice Crop Growth State Based on Computer Image Processing Technology | |
CN116388933B (en) | Communication signal blind identification system based on deep learning |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |