CN109409263B - Method for detecting urban ground feature change of remote sensing image based on Siamese convolutional network - Google Patents
Method for detecting urban ground feature change of remote sensing image based on Siamese convolutional network Download PDFInfo
- Publication number
- CN109409263B CN109409263B CN201811191860.2A CN201811191860A CN109409263B CN 109409263 B CN109409263 B CN 109409263B CN 201811191860 A CN201811191860 A CN 201811191860A CN 109409263 B CN109409263 B CN 109409263B
- Authority
- CN
- China
- Prior art keywords
- scnn
- network
- neural network
- image
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/35—Categorising the entire scene, e.g. birthday party or wedding scene
- G06V20/38—Outdoor scenes
- G06V20/39—Urban scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method for detecting urban ground feature change of remote sensing images based on a Simese convolution network, wherein the Simese convolution network is a twin convolution neural network SCNN, the method comprises the steps of selecting an initial sample set from two registered time-phase urban images based on a data enhancement technology, setting the twin convolution neural network SCNN, training the twin convolution neural network SCNN based on the initial sample set, and expanding the initial sample set by adopting the data enhancement technology; and training the twin convolutional neural network SCNN based on the extended sample set to obtain a trained SCNN model, and realizing the change detection of urban ground objects. The invention realizes the expansion of the sample by a data enhancement technology and designs a Simase convolutional neural network, thereby avoiding the fussy step of manually designing the characteristics in the traditional change detection method and realizing the end-to-end operation; the spatial attribute of the image is fully considered, and the precision and the reliability of change detection are improved.
Description
Technical Field
The invention belongs to the technical field of remote sensing image change detection, and particularly relates to a change detection method of urban ground objects.
Background
The change detection is a process of finding changes on the earth surface by using two or more remote sensing images acquired in the same geographical area at different time points. The change detection is an important means for maintaining the situation of geographic information data, and is an important research direction in the field of remote sensing application. In recent years, the urbanization process of China is accelerated continuously, and the change of urban land features is changing day by day. The change detection of urban land features plays an important role in grasping urban change rules, updating urban maps, assisting urban planning and design, making government decisions and the like.
The traditional change detection method needs manual design features, which is time-consuming and labor-consuming work and needs strong professional knowledge. And it is difficult to design a universal feature that is applicable to all terrain types. In recent years, the deep learning technology is developed rapidly, and the method is applied to the fields of image recognition and change detection to a certain extent. The multilayer nonlinear mapping of the deep neural network enables the deep neural network to have the capability of fitting any function, so that a high-dimensional classification surface can be constructed, and a pattern classification and identification task can be finished with high quality. The invention mainly researches and utilizes the deep neural network to realize end-to-end change detection on urban ground features, avoids the process of manually designing features and improves the precision of the change detection. (references: Tewkesbury A P, Comber A J, Tate N J, et al, a critical synthesis of removed sensitive image change detection technique, removed Sensing of Environment, 2015; Ian Goodfellow, Yoshua Bengio and Aaron Corville, Deep Learning, MITPress, 2016)
At present, the deep neural network models commonly used for detecting the change of the high-resolution image comprise a self-coding network, a deep confidence network, a convolutional neural network and the like. The self-coding network and the depth confidence network convert the two-dimensional image into a one-dimensional vector to be input into the network model, and the spatial information of the image is lost. The convolutional neural network adopts the idea of local connection, takes a local receptive field as the minimum unit for feature extraction, and fully considers the spatial information of the image. However, the current research methods are based on pixel change detection, and the range of each pixel neighborhood of an image is used as training input to train a deep neural network to obtain a model, and then change detection is performed on the whole image. The pixel-by-pixel change detection method adopts a moving window with a fixed size as the input of the neural network, the contained information amount is limited, and the advantage of deep neural network learning complex characteristics cannot be fully exerted. And with pixels as the analysis unit, there is a large amount of spurious variations of fragmentation. The current commonly used analysis unit based on scene change detection uses a scene as change detection, the scene needs to be captured, and the image has more non-scene areas and cannot be subjected to change detection, so that the image cannot be subjected to 'full coverage' change detection. (references: Xu Y, Xiang S, Huo C, et al, Change Detection Based on auto-encoder model for VHR Images, Proceedings of SPIE-The International Society for Optical Engineering, 2013; Argyris A, Argilias D P, Building Change Detection third multi-scale, scale GEOBIA adaptation by integrating Detection later processes with reflection Networks, International Journal of Image & Data Fusion, 2016; Liu J, Gong M, Qin K, et al, A content Coupling protocol for Change Detection result, characterization analysis and mapping, method of conversion of samples, method of conversion, method of conversion, conversion of samples, method of conversion, conversion of conversion system, conversion of conversion system, conversion of conversion
Deep learning is a data-driven machine learning method. Because the number of layers of the deep neural network is large and the number of parameters is large, a large number of samples are needed for training the deep neural network. Making a set of labeled samples is a time-consuming and labor-intensive task and requires a certain amount of expertise. To solve this problem, Alex Krizhevsky et al extend the training samples using a "data enhancement" technique. Typical data enhancement techniques include operations such as gray-scale transformation, rotation, arbitrary cropping, color dithering, etc. to expand the sample. The data adding technology achieves better effects in the fields of image classification, recognition and target detection. But the technology is difficult to be effective when applied to the field of change detection. The reason is that two or more remote sensing images acquired at different time points in the same geographical area have large difference due to different shooting time, climate conditions, light conditions, shooting angles and the like, and the diversity of samples cannot be increased by simply utilizing operations such as gray level conversion, rotation, arbitrary cutting, color dithering and the like. Therefore, when the sample for change detection is expanded, the characteristic that the difference exists among the multi-temporal remote sensing images is fully considered, and a suitable sample expansion method is sought by combining the characteristic of change data. (references: Krizhevsky A, Sutskeeper I, Hinton G E, ImageNet classification with deep comparative Processing systems, Current Associates Inc., 2012; Zhong Y, Large batch comparative Neural networks for the same, Journal of Applied Remote classification, 2016.)
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides an urban ground feature change detection technology based on a Siamese convolutional network, and provides a special data enhancement method aiming at the change detection problem.
The technical scheme adopted by the invention is a method for detecting urban ground feature change of remote sensing images based on a Simese convolutional network, wherein the Simese convolutional network is a twin convolutional neural network (SCNN) and comprises the following steps based on a data enhancement technology,
step 1, selecting an initial sample set from the two-time phase city image which is registered, wherein the initial sample set comprises selecting variable and invariable blocks, dividing the blocks into division blocks with fixed sizes respectively, and giving a label to each division block pair to obtain two-time phase sample pairs;
step 2, setting a twin convolutional neural network SCNN, and training the twin convolutional neural network SCNN based on the initial sample set obtained in the step 1 to obtain the twin convolutional neural network SCNN after initial training;
the twin convolutional neural network SCNN consists of two identical branch networks and a decision layer network, wherein the branch networks are positioned at the lower layer of the twin convolutional neural network SCNN, the two branch networks have identical structures and parameters, the two branch networks respectively extract the characteristics of the two time phase image samples, the extracted characteristics are connected through the characteristics to generate the relative overall characteristics of the two time phase samples, and the overall characteristics are input into the decision layer network at the top layer; a decision layer network realizes a similarity measurement model and carries out similarity measurement on the input overall characteristics;
step 3, based on the twin convolutional neural network SCNN obtained in the step 2 after the initial training, expanding the initial sample set by adopting a data enhancement technology;
step 4, training the twin convolutional neural network SCNN based on the extended sample set obtained in the step 3 to obtain a trained SCNN model;
and 5, testing a new urban area based on the SCNN model trained in the step 4, and realizing the change detection of urban ground features.
Furthermore, step 3, which employs a data enhancement technique to extend the initial sample set, comprises the following sub-steps,
step 3.1, based on the rule that the number of the variable regions and the number of the invariable regions in the two-time phase image are few, performing data enhancement on the invariable sample;
and 3.2, based on a rule, each piece of the changed partition block data in the relative time phase 1 and the partition blocks in the time phase 2, which are positioned at different positions, can form a changed sample pair with the changed partition blocks, and data enhancement is carried out on the changed samples.
Furthermore, step 3.1 comprises the following sub-steps,
step 3.1.1, inputting the twin convolutional neural network SCNN obtained in the step 2 after the initial training;
step 3.1.2, selecting the extended area image of two time phases, and dividing the extended area image into division blocks with fixed sizes;
step 3.1.3, inputting each segmentation block obtained in the step 3.1.2 into the twin convolutional neural network SCNN after initial training, carrying out change detection to obtain a result, and determining each segmentation block as a change area or an invariant area;
step 3.1.4, selecting an invariant region from the result obtained in the step 3.1.3, and adjusting a threshold parameter T to ensure that the selected invariant region does not contain a variant region, wherein when the probability of the invariant sample is greater than T, the sample is an invariant sample, and each of other segmented blocks is a variant region;
and 3.1.5, adding the selected invariant region into the invariant samples of the initial sample set obtained in the step 1 to realize the extension of the invariant region.
Furthermore, step 3.2 comprises the following sub-steps,
step 3.2.1, copying all the change segmentation blocks in the time phase 1 image for num times based on the change area obtained in the step 3.1.4, and renaming, wherein the change sample of the time phase 1 is expanded to be (num +1) times of the original change sample, and num is a preset numerical value;
step 3.2.2, based on the change area obtained in the step 3.1.4, corresponding to each change partition block in the time phase 1 image, randomly selecting num partition blocks which are positioned at different positions from the num partition blocks in the time phase 2 image, renaming the num partition blocks, and forming a num pair of change image pairs with the corresponding partition blocks of the time phase 1 copied in the step 3.2.1;
and 3.2.3, adding the change sample obtained in the step 3.2.2 into the change sample of the initial sample set obtained in the step 1 to realize expansion.
Furthermore, each branch network of the twin convolutional neural network SCNN contains 5 convolutional layers, 3 pooling layers, and one fully-connected layer, respectively.
Moreover, the decision layer network of the twin convolutional neural network SCNN is composed of three fully connected layers.
Moreover, when the twin convolutional neural network SCNN is trained, a transfer learning strategy is adopted, the trained CaffeNet model parameters are adopted, branch network parameters are initialized, and the network training speed and the change detection precision are improved.
The invention realizes the expansion of samples through a specific data enhancement technology, designs a Siamese Convolutional Neural Network (SCNN) and realizes the change detection of ground objects in two-time high-resolution image urban areas. Based on the rule that the variable area is a few and the invariable area is a majority in the two-time phase image, the method adopts the mode of 'iterative training SCNN-sample selection-sample expansion' to realize the expansion of the invariable sample; based on the fact that the segmentation blocks located at different positions in two time phases can form a change sample pair, the expansion of the change samples is achieved, and the problem of dependence of deep learning on big data is solved. The invention has the beneficial effects that: the invention avoids the complicated steps of manual design of characteristics in the traditional change detection method and realizes the end-to-end operation; the method solves the problem that the spatial information of the image is lost by the change detection method based on the self-coding network and the depth confidence network, fully considers the spatial attribute of the image, and improves the precision and the reliability of the change detection.
Drawings
FIG. 1 is a flow chart of a training phase according to an embodiment of the present invention.
FIG. 2 is a flow chart of a testing phase according to an embodiment of the present invention.
Detailed Description
In order to facilitate the understanding and implementation of the present invention for those of ordinary skill in the art, the present invention is further described in detail with reference to the accompanying drawings and examples, it is to be understood that the embodiments described herein are merely illustrative and explanatory of the present invention and are not restrictive thereof.
Referring to fig. 1 and fig. 2, an embodiment of the invention provides a technology for detecting urban surface feature changes of remote sensing images based on a Siamese convolutional network, which includes the following steps:
step 1: selecting an initial sample set from the two registered time-phase urban images;
the specific implementation of the initial sample set selection of the embodiment of the invention comprises the following substeps:
step 1.1: the method comprises the steps of inputting two time-phase images, wherein the urban images are researched, the two time-phase images of a certain urban area can be selected and recorded as initial area images, and in the specific implementation, a change detection sample selection tool can be provided, and the two time-phase images are displayed in a two-window linkage manner;
step 1.2: comparing the changed and unchanged blocks of the images in the two time phases, wherein the changed and unchanged blocks can be selected by a user in a frame mode or the changed and unchanged blocks can be predetermined;
step 1.3: the method comprises the steps of dividing changed and unchanged blocks in an image of two time phases into divided blocks with fixed sizes respectively (in the embodiment, 64 pixels by 64 pixels are adopted), forming divided block pairs, endowing each divided block pair with a label, obtaining two time phase sample pairs, and realizing construction of a sample set, wherein in the embodiment, the label (1,0) represents a changed area, and the label (0,1) represents an unchanged area.
Step 2: designing a twin Convolutional Neural Network (SCNN), and training the twin Convolutional Neural Network (SCNN) based on the initial sample set obtained in the step 1 to obtain an initially trained SCNN model;
the SCNN is composed of two identical branch networks and a decision layer network, wherein the branch networks are located at the lower layer of the SCNN, the two branch networks have identical structures and parameters, and feature extraction is respectively carried out on two time phase image samples. The extracted features are connected through features to generate overall features of two time phase samples, and the overall features are input into a decision layer network at the top layer. The decision layer network is essentially a similarity measurement model, and similarity measurement is carried out on the input overall characteristics.
The SCNN structure design of the embodiment of the invention is as follows:
the two branch networks of the SCNN lower layer adopt a convolution layer and a first full connection layer of a convolution neural network framework (CaffeNet), and have the same structure and parameters, and respectively comprise 5 convolution layers, 3 pooling layers and a full connection layer;
performing characteristic connection on the two time-phase image sample characteristics extracted by the two branch networks to generate the relative overall characteristics of the two time-phase samples;
the decision layer network of the SCNN upper layer is composed of three fully connected layers, the decision layer network is substantially a similarity measurement model, similarity measurement is performed on the input overall features, the output of the SCNN is (prob1, prob2), the probability that the sample is a changed sample is prob1, and the probability that the sample is an unchanged sample is prob2(prob1+ prob2 ═ 1).
When the twin convolutional neural network SCNN is trained based on the initial sample set obtained in step 1, the training process is as follows,
firstly, dividing an extended sample set into a training sample set (80%) and a verification sample set (20%), and segmenting in a random mode; (the training sample set is input into the SCNN later, and iterative training is carried out based on gradient descent and back propagation algorithm, when the iteration number is T1Then, verifying the sample set to verify the model trained by the training set to obtain verification precision; when the number of iterations is T2Preservation model (T)2＝nT1)。)
In specific implementation, T can be preset by a user1And T2Can execute T in each round1And (4) performing iteration, and saving the model after n rounds of execution.
The training sample set and the verification sample set respectively contain a variation region and an invariant region.
Then, inputting the training sample set into the SCNN for iterative training to obtain a trained SCNN model;
the specific implementation of the SCNN iterative training of the embodiment of the invention comprises the following substeps:
step a: respectively importing model parameters of a convolution layer and a first full connection layer of a convolution neural network framework (CaffeNet) into two branch networks of the SCNN to realize the initialization of the parameters of the two branch networks;
in the step, the model parameters adopt empirical values, the trained CaffeNet model parameters are transferred, and the branch network parameters are initialized, so that the network training speed and the change detection precision can be effectively improved.
Step b: randomly initializing decision layer network parameters of the SCNN;
step c: the method comprises the following steps of taking a sample pair corresponding to two time phases as input, taking a change detection result as output, setting network hyper-parameters, and adopting empirical values during specific implementation, for example: number of samples in small batches: 1000, parts by weight; learning rate: 0.01; proportion of randomly inactivated neurons: and 0.5, carrying out iterative training on the SCNN based on a random gradient descent and back propagation algorithm until the model is judged to be converged through verification accuracy, and storing the optimal SCNN model. In specific implementation, the judgment can be carried out according to the rising and falling conditions of the precision function curve and the loss function curve, and the curve converges when reaching the stability.
Stochastic gradient descent and back propagation algorithms are prior art and are not described in detail herein.
And step 3: the initial sample set is expanded through the data enhancement technology provided by the invention;
the specific implementation of the sample set expansion of the embodiment of the invention comprises the following substeps:
step 3.1: based on the rule that the variation area is few and the invariable area is majority in the two-time phase image, data enhancement is carried out on the invariable sample;
the specific implementation process of the data expansion of the invariant sample in this embodiment is as follows:
step 3.1.1: inputting the initially trained SCNN model obtained in the step 2, namely, training a twin convolutional neural network (SCNN) based on the initial sample set in the step 2;
step 3.1.2: selecting an image of a large area in a city, and dividing the image of the large area into blocks (64 × 64 pixels in the embodiment) corresponding to the division size in the step 1;
and 1.1, the two time phases of the initial area image are consistent, and the two time phase images of another larger area in the city can be selected as the extended area image in the step.
Step 3.1.3: inputting each segmented block image obtained in the step 3.1.2 into the SCNN model obtained in the step 3.1.1, and performing change detection to obtain a result (each segmented block is a change area or an invariant area);
step 3.1.4: selecting an invariant region from the result obtained in the step 3.1.3, and adjusting a threshold parameter T to ensure that the selected invariant region does not contain a variant region, wherein when the probability of the invariant sample is greater than T, the sample is an invariant sample, and each of other segmented blocks is a variant region;
in an embodiment, the threshold T is set to 0.90, and when the probability of an invariant sample is greater than 0.90, the sample is an invariant sample.
Step 3.1.5: and adding the selected invariant region into the invariant samples of the obtained initial sample set in the step 1 to realize the extension of the invariant region.
Step 3.2: for each changed tile data in phase 1, the tile located at a different position in phase 2 may form a changed sample pair with the changed tile. Based on the theory, data enhancement is performed on the variation samples.
The specific implementation process of the data expansion of the change sample in the embodiment is as follows:
step 3.2.1: copying all the changed segmentation block images in the time phase 1 image for num times based on the changed area obtained in the step 3.1.4, and renaming, wherein the changed sample of the time phase 1 is expanded to be (num +1) times of the original sample;
step 3.2.2: based on the change area obtained in the step 3.1.4, corresponding to each change segmentation block in the time phase 1 image, randomly selecting num segmentation blocks which are positioned at different positions from the num segmentation blocks in the time phase 2 image, renaming the num segmentation blocks, and forming num pairs of change images with the segmentation blocks of the corresponding time phase 1 copied in the step 3.2.1;
step 3.2.3: and adding the variation sample obtained in the step 3.2.2 into the variation sample of the initial sample set obtained in the step 1 to realize the extension of the variation sample.
After all the changed segmentation blocks in the time phase 1 image are processed in the steps of 3.2.1 and 3.2.2 respectively, the changed sample of the time phase 2 is expanded to be (num +1) times of the original sample, and then the changed sample is added into the changed sample of the initial sample set.
In specific implementation, num can be preset. For the two time phase images of the same region, each changed sample in the time phase 1, and the sample located at a different position in the time phase 2 can form a new changed sample pair with the changed sample. Based on this assumption, a change sample is selected in phase 1 image and is replicated num times, num samples at different positions are randomly selected in phase 2 image, and the phase 1 image and phase 1 image form umn pairs of change images. This operation is performed on all the original change samples, which are expanded by a factor of (num + 1). For example, 4 samples are copied from a phase 1 sample, 4 samples located at different positions are randomly selected from phase 2 samples, and 4 pairs of changing images are formed with the 4 samples of the phase 1. And adding an original changed image pair, and 5 changed image pairs in total. Thus, the change samples are expanded by a factor of 5.
And 4, step 4: and (4) training the twin convolutional neural network (SCNN) based on the extended sample set obtained in the step (3) to obtain a trained SCNN model.
The mode of retraining is the same as the mode of training for the first time in step 2, namely:
firstly, dividing an extended sample set into a training sample set (80%) and a verification sample set (20%); (the training sample set is input into the SCNN later, and iterative training is carried out based on gradient descent and back propagation algorithm, when the iteration number is T1While, verifying the sampleVerifying the model trained by the training set to obtain verification precision; when the number of iterations is T2Time, save model (T)2＝nT1). ) At this time, T1And T2The value of (c) can be adjusted to be different from that of the first training in step 2.
Then, inputting the training sample set into the SCNN for iterative training to obtain a trained SCNN model;
the specific implementation of the SCNN iterative training of the embodiment of the invention comprises the following substeps:
step a, respectively importing model parameters of a convolution layer and a first full connection layer of a convolution neural network framework (CaffeNet) into two branch networks of the SCNN to realize the initialization of the parameters of the two branch networks;
in the step, the model parameters also adopt empirical values, the trained CaffeNet model parameters are transferred, and the branch network parameters are initialized, so that the network training speed and the change detection precision can be effectively improved. In specific implementation, the same initial parameters as those in step 2 can be adopted, and the expanded training samples are adopted to retrain the main optimized decision-making layer network parameters.
Step b, randomly initializing decision layer network parameters of the SCNN;
and c, setting a proper network hyper-parameter by taking the sample pair corresponding to the two time phases as input and the change detection result as output, wherein the same value as that in the step 2 can be set: number of samples in small batches: 1000, parts by weight; learning rate: 0.01; proportion of randomly inactivated neurons: and 0.5, carrying out iterative training on the SCNN based on a random gradient descent and back propagation algorithm until the model is converged, and storing the optimal SCNN model.
And 5: based on the SCNN model trained in the step 4, testing is carried out, and the method can carry out 'segmentation-change detection-precision evaluation' on a new large-area urban area so as to realize the change detection on urban land features;
the method and the device for detecting the ground feature change of the large-area city area are based on the trained SCNN model. Referring to fig. 2, the specific implementation process is as follows:
step 5.1: selecting a large-area urban area which is not overlapped with the training sample data, and respectively dividing the two-time phase image of the area into divided blocks (64 × 64 pixels in the embodiment) with the sizes corresponding to the divided blocks in the step 1;
and 1.1, selecting two time phases of the image of the initial area in the city, wherein the two time phases of the image of the other large area in the city are consistent with the two time phases of the image of the initial area and the image of the expansion area, and the two time phases of the image of the other large area in the city are not overlapped with the image of the initial area and the image of the expansion area and are marked.
Step 5.2: inputting the two time phase image segmentation blocks into the SCNN model trained in the step 4, carrying out change detection, and outputting a result;
step 5.3: and (5) carrying out precision evaluation and result visualization on the change detection result obtained in the step (5.2).
The visualization result of the change detection is represented by a binary image, white represents a changed segmentation block, and black represents an unchanged segmentation block. According to the ground change reference map, the number of the correctly and wrongly detected segmentation blocks can be counted according to the reference map and the change detection result, and precision evaluation indexes such as drawing precision, user precision, missing detection rate, false detection rate, overall precision, Kappa coefficient and the like are calculated so as to test the precision of the change detection result and verify the effectiveness of the method provided by the invention.
In specific implementation, the automatic operation of the processes can be realized by adopting a computer software technology.
The technical scheme of the embodiment of the invention is utilized to carry out experiments, and a change detection result visualization graph is extracted:
taking an image (a), namely a 2010 remote sensing image of a certain area in Wuhan city, which is a 17-level image obtained from an ArcGIS platform, wherein the resolution of the image is 1.19 meters per pixel; taking an image (b), namely a remote sensing image of 2013 in the same area, wherein the image is a 17-level image obtained from a Microsoft map platform, and the resolution of the image is 1.19 meters per pixel; image (c) -the ground variation reference picture is a real variation image picture of two time phases obtained by artificial visual interpretation and actual investigation, wherein white represents a variation area and black represents a constant area; by adopting the above process provided by the invention, the image (d), namely the change detection result graph detected by the change detection method can be finally obtained. The effectiveness of the invention can be confirmed by changing the detection result graph.
It should be understood that parts of the specification not set forth in detail are well within the prior art.
It should be understood that the above description of the preferred embodiments is given for clarity and not for any purpose of limitation, and that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (6)
1. A method for detecting urban ground feature change of remote sensing images based on a Siamese convolutional network is characterized by comprising the following steps: the siense convolutional network is a twin convolutional neural network SCNN, and is based on a data enhancement technology and comprises the following steps,
step 1, selecting an initial sample set from the two-time phase city image which is registered, wherein the initial sample set comprises selecting variable and invariable blocks, dividing the blocks into division blocks with fixed sizes respectively, and giving a label to each division block pair to obtain two-time phase sample pairs;
step 2, setting a twin convolutional neural network SCNN, and training the twin convolutional neural network SCNN based on the initial sample set obtained in the step 1 to obtain the twin convolutional neural network SCNN after initial training;
the twin convolutional neural network SCNN consists of two identical branch networks and a decision layer network, wherein the branch networks are positioned at the lower layer of the twin convolutional neural network SCNN, the two branch networks have identical structures and parameters, the two branch networks respectively extract the characteristics of the two time phase image samples, the extracted characteristics are connected through the characteristics to generate the relative overall characteristics of the two time phase samples, and the overall characteristics are input into the decision layer network at the top layer; a decision layer network realizes a similarity measurement model and carries out similarity measurement on the input overall characteristics;
and 3, expanding the initial sample set by adopting a data enhancement technology based on the twin convolutional neural network SCNN after the initial training obtained in the step 2, and comprising the following substeps,
step 3.1, based on the rule that the number of the variable regions and the number of the invariable regions in the two-time phase image are few, performing data enhancement on the invariable sample;
step 3.2, based on a rule, each piece of the changed partition block data in the relative time phase 1 and the partition blocks in the time phase 2, which are positioned at different positions, can form a changed sample pair with the changed partition blocks, and data enhancement is carried out on the changed samples;
step 4, training the twin convolutional neural network SCNN based on the extended sample set obtained in the step 3 to obtain a trained SCNN model;
and 5, testing a new urban area based on the SCNN model trained in the step 4, and realizing the change detection of urban ground features.
2. The method for detecting urban terrain variation of remote sensing images based on the Siamese convolutional network as claimed in claim 1, wherein: step 3.1 comprises the sub-steps of,
step 3.1.1, inputting the twin convolutional neural network SCNN obtained in the step 2 after the initial training;
step 3.1.2, selecting the extended area image of two time phases, and dividing the extended area image into division blocks with fixed sizes;
step 3.1.3, inputting each segmentation block obtained in the step 3.1.2 into the twin convolutional neural network SCNN after initial training, carrying out change detection to obtain a result, and determining each segmentation block as a change area or an invariant area;
step 3.1.4, selecting an invariant region from the result obtained in the step 3.1.3, and adjusting a threshold parameter T to ensure that the selected invariant region does not contain a variant region, wherein when the probability of the invariant sample is greater than T, the sample is an invariant sample, and each of other segmented blocks is a variant region;
and 3.1.5, adding the selected invariant region into the invariant samples of the initial sample set obtained in the step 1 to realize the extension of the invariant region.
3. The method for detecting urban terrain variation based on the remotely sensed image of the Siamese convolutional network as claimed in claim 2, wherein: step 3.2 comprises the sub-steps of,
step 3.2.1, copying all the change segmentation blocks in the time phase 1 image for num times based on the change area obtained in the step 3.1.4, and renaming, wherein the change sample of the time phase 1 is expanded to be (num +1) times of the original change sample, and num is a preset numerical value;
step 3.2.2, based on the change area obtained in the step 3.1.4, corresponding to each change partition block in the time phase 1 image, randomly selecting num partition blocks which are positioned at different positions from the num partition blocks in the time phase 2 image, renaming the num partition blocks, and forming a num pair of change image pairs with the corresponding partition blocks of the time phase 1 copied in the step 3.2.1;
and 3.2.3, adding the change sample obtained in the step 3.2.2 into the change sample of the initial sample set obtained in the step 1 to realize expansion.
4. The method for detecting urban terrain variation based on the remotely sensed image of the siemese convolutional network as claimed in claim 1, 2 or 3, wherein: each branch network of the twin convolutional neural network SCNN includes 5 convolutional layers, 3 pooling layers, and one fully-connected layer, respectively.
5. The method for detecting urban terrain variation based on the remotely sensed image of the siemese convolutional network as claimed in claim 1, 2 or 3, wherein: the decision layer network of the twin convolutional neural network SCNN is composed of three fully connected layers.
6. The method for detecting urban terrain variation based on the remotely sensed image of the siemese convolutional network as claimed in claim 1, 2 or 3, wherein: when the twin convolutional neural network SCNN is trained, a transfer learning strategy is adopted, the trained CaffeNet model parameters are adopted, branch network parameters are initialized, and the speed of network training and the precision of change detection are improved.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811191860.2A CN109409263B (en) | 2018-10-12 | 2018-10-12 | Method for detecting urban ground feature change of remote sensing image based on Siamese convolutional network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811191860.2A CN109409263B (en) | 2018-10-12 | 2018-10-12 | Method for detecting urban ground feature change of remote sensing image based on Siamese convolutional network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109409263A CN109409263A (en) | 2019-03-01 |
| CN109409263B true CN109409263B (en) | 2021-05-04 |
Family
ID=65467086
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811191860.2A Active CN109409263B (en) | 2018-10-12 | 2018-10-12 | Method for detecting urban ground feature change of remote sensing image based on Siamese convolutional network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109409263B (en) |
Families Citing this family (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109934166A (en) * | 2019-03-12 | 2019-06-25 | 中山大学 | Unmanned plane image change detection method based on semantic segmentation and twin neural network |
| CN111738972B (en) * | 2019-03-19 | 2024-05-28 | 阿里巴巴集团控股有限公司 | Building detection system, method and device |
| CN110059658B (en) * | 2019-04-26 | 2020-11-24 | 北京理工大学 | Remote sensing satellite image multi-temporal change detection method based on three-dimensional convolutional neural network |
| CN110070140B (en) * | 2019-04-28 | 2021-03-23 | 清华大学 | User similarity determination method and device based on multi-category information |
| CN110378224B (en) * | 2019-06-14 | 2021-01-05 | 香港理工大学深圳研究院 | Detection method and detection system for ground feature change and terminal |
| CN110897630A (en) * | 2019-06-27 | 2020-03-24 | 上海数创医疗科技有限公司 | Ventricular tachycardia and heart rhythm identification method and device based on transfer learning |
| CN110610191A (en) * | 2019-08-05 | 2019-12-24 | 深圳优地科技有限公司 | Elevator floor identification method and device and terminal equipment |
| CN110570397B (en) * | 2019-08-13 | 2020-12-04 | 创新奇智(重庆)科技有限公司 | Method for detecting ready-made clothes printing defects based on deep learning template matching algorithm |
| CN110969088B (en) * | 2019-11-01 | 2023-07-25 | 华东师范大学 | Remote sensing image change detection method based on significance detection and deep twin neural network |
| CN111161218A (en) * | 2019-12-10 | 2020-05-15 | 核工业北京地质研究院 | High-resolution remote sensing image change detection method based on twin convolutional neural network |
| CN111079671B (en) * | 2019-12-20 | 2020-11-03 | 深圳集智数字科技有限公司 | Method and device for detecting abnormal articles in scene |
| CN111127532B (en) * | 2019-12-31 | 2020-12-22 | 成都信息工程大学 | Medical image deformation registration method and system based on deep learning characteristic optical flow |
| CN111325134B (en) * | 2020-02-17 | 2023-04-07 | 武汉大学 | Remote sensing image change detection method based on cross-layer connection convolutional neural network |
| CN111640159B (en) * | 2020-05-11 | 2022-04-01 | 武汉大学 | Remote sensing image change detection method based on twin convolutional neural network |
| CN112016436A (en) * | 2020-08-28 | 2020-12-01 | 北京国遥新天地信息技术有限公司 | Remote sensing image change detection method based on deep learning |
| CN112215085A (en) * | 2020-09-17 | 2021-01-12 | 云南电网有限责任公司昆明供电局 | Power transmission corridor foreign matter detection method and system based on twin network |
| CN112396594B (en) * | 2020-11-27 | 2024-03-29 | 广东电网有限责任公司肇庆供电局 | Method and device for acquiring change detection model, change detection method, computer equipment and readable storage medium |
| CN113326724B (en) * | 2021-02-07 | 2024-02-02 | 海南长光卫星信息技术有限公司 | Remote sensing image change detection method, device, equipment and readable storage medium |
| CN112784929B (en) * | 2021-03-14 | 2023-03-28 | 西北工业大学 | Small sample image classification method and device based on double-element group expansion |
| CN113158855B (en) * | 2021-04-08 | 2023-04-18 | 成都国星宇航科技股份有限公司 | Remote sensing image auxiliary processing method and device based on online learning |
| CN113609889B (en) * | 2021-05-18 | 2024-02-13 | 武汉大学 | High-resolution remote sensing image vegetation extraction method based on sensitive characteristic focusing perception |
| CN114385860A (en) * | 2021-12-10 | 2022-04-22 | 阿里巴巴达摩院(杭州)科技有限公司 | Image processing method, image processing device, storage medium and computer equipment |
| CN115019186B (en) * | 2022-08-08 | 2022-11-22 | 中科星图测控技术(合肥)有限公司 | Method and system for detecting remote sensing change |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105740894A (en) * | 2016-01-28 | 2016-07-06 | 北京航空航天大学 | Semantic annotation method for hyperspectral remote sensing image |
| CN106897681A (en) * | 2017-02-15 | 2017-06-27 | 武汉喜恩卓科技有限责任公司 | A kind of remote sensing images comparative analysis method and system |
| CN108573276A (en) * | 2018-03-12 | 2018-09-25 | 浙江大学 | A kind of change detecting method based on high-resolution remote sensing image |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9965717B2 (en) * | 2015-11-13 | 2018-05-08 | Adobe Systems Incorporated | Learning image representation by distilling from multi-task networks |
-
2018
- 2018-10-12 CN CN201811191860.2A patent/CN109409263B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105740894A (en) * | 2016-01-28 | 2016-07-06 | 北京航空航天大学 | Semantic annotation method for hyperspectral remote sensing image |
| CN106897681A (en) * | 2017-02-15 | 2017-06-27 | 武汉喜恩卓科技有限责任公司 | A kind of remote sensing images comparative analysis method and system |
| CN108573276A (en) * | 2018-03-12 | 2018-09-25 | 浙江大学 | A kind of change detecting method based on high-resolution remote sensing image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109409263A (en) | 2019-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109409263B (en) | Method for detecting urban ground feature change of remote sensing image based on Siamese convolutional network | |
| CN111986099B (en) | Tillage monitoring method and system based on convolutional neural network with residual error correction fused | |
| EP3614308B1 (en) | Joint deep learning for land cover and land use classification | |
| CN108304873B (en) | Target detection method and system based on high-resolution optical satellite remote sensing image | |
| CN113160192B (en) | Visual sense-based snow pressing vehicle appearance defect detection method and device under complex background | |
| CN109598241B (en) | Satellite image marine ship identification method based on Faster R-CNN | |
| CN108596055B (en) | Airport target detection method of high-resolution remote sensing image under complex background | |
| CN110163213B (en) | Remote sensing image segmentation method based on disparity map and multi-scale depth network model | |
| WO2018214195A1 (en) | Remote sensing imaging bridge detection method based on convolutional neural network | |
| CN108596108B (en) | Aerial remote sensing image change detection method based on triple semantic relation learning | |
| CN108256424A (en) | A kind of high-resolution remote sensing image method for extracting roads based on deep learning | |
| Hormese et al. | Automated road extraction from high resolution satellite images | |
| JP6397379B2 (en) | CHANGE AREA DETECTION DEVICE, METHOD, AND PROGRAM | |
| CN110334762A (en) | A kind of feature matching method combining ORB and SIFT based on quaternary tree | |
| CN113076871A (en) | Fish shoal automatic detection method based on target shielding compensation | |
| CN109829423B (en) | Infrared imaging detection method for frozen lake | |
| Crommelinck et al. | Interactive cadastral boundary delineation from UAV data | |
| CN112733711B (en) | Remote sensing image damaged building extraction method based on multi-scale scene change detection | |
| JP2019185787A (en) | Remote determination of containers in geographical region | |
| CN116758421A (en) | Remote sensing image directed target detection method based on weak supervised learning | |
| CN115512247A (en) | Regional building damage grade assessment method based on image multi-parameter extraction | |
| Lv et al. | Novel automatic approach for land cover change detection by using VHR remote sensing images | |
| Zhiyong et al. | Automatic landslide inventory mapping approach based on change detection technique with very-high-resolution images | |
| Lari et al. | Automated building extraction from high-resolution satellite imagery using spectral and structural information based on artificial neural networks | |
| CN114332644A (en) | Large-view-field traffic density acquisition method based on video satellite data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |