CN115861823B - Remote sensing change detection method and device based on self-supervision deep learning - Google Patents
Remote sensing change detection method and device based on self-supervision deep learning Download PDFInfo
- Publication number
- CN115861823B CN115861823B CN202310140461.8A CN202310140461A CN115861823B CN 115861823 B CN115861823 B CN 115861823B CN 202310140461 A CN202310140461 A CN 202310140461A CN 115861823 B CN115861823 B CN 115861823B
- Authority
- CN
- China
- Prior art keywords
- image
- surface reflectivity
- deep learning
- self
- image pair
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a remote sensing change detection method and a remote sensing change detection device based on self-supervision deep learning, which relate to the technical field of change detection and comprise the following steps: acquiring a sample earth surface reflectivity image pair; processing the sample surface reflectivity image pair by using a preset algorithm to obtain a pre-training image, wherein the preset algorithm comprises: a CCA-EM self-supervision image transformation algorithm, a typical association analysis algorithm, a expectation maximization algorithm and an Ojin segmentation algorithm; performing image visual enhancement processing on the sample surface reflectivity image pair to obtain a target surface reflectivity image pair; training the deep learning model by utilizing the pre-training image and the target surface reflectivity image to obtain a self-supervision deep learning model; after the surface reflectivity image pair to be processed is obtained, a self-supervision deep learning model is utilized to determine the change pattern spots in the surface reflectivity image pair to be processed, and the technical problems of low engineering efficiency and poor applicability caused by the fact that an existing remote sensing change detection method is seriously dependent on an artificial sample are solved.
Description
Technical Field
The invention relates to the technical field of change detection, in particular to a remote sensing change detection method and device based on self-supervision deep learning.
Background
The change detection is to analyze and determine the characteristics and the process of the surface change by utilizing the multi-temporal remote sensing image data covering the same surface area. The remote sensing image change detection technology is widely applied to various fields such as land investigation, urban research, ecological system monitoring, disaster monitoring and evaluation, military reconnaissance and the like.
The remote sensing change detection algorithm represented by deep learning is one of the most widely applied methods at present, and the deep learning change detection method can be divided into three types according to different network structures, namely a deep confidence network (Deep Belief Networks, DBN), a generating countermeasure network (Generative Adversarial Networks, GAN) and a convolutional neural network (Convolutional Neural Networks, CNN), wherein the great model of the deep learning method completely depends on manually interpreted samples and needs a large number of high-quality training samples to drive, so that the characterization capability of the neural network is improved, otherwise, fitting is easy, and the accuracy of change detection is seriously affected. However, on one hand, the remote sensing images have various fields and complex manual interpretation procedures, and the sample preparation requires a large amount of labor and time cost, and on the other hand, the remote sensing images are easily interfered by unfavorable weather such as cloud, rain, fog and the like, so that the difficulty of obtaining high-quality samples is increased, the main problem in deep learning remote sensing change detection engineering application is caused, namely, massive samples with high credibility are difficult to automatically obtain, and further, the technical problems of low engineering efficiency and poor applicability are caused.
An effective solution to the above-mentioned problems has not been proposed yet.
Disclosure of Invention
In view of the above, the present invention aims to provide a remote sensing change detection method and device based on self-supervision deep learning, so as to alleviate the technical problems of low engineering efficiency and poor applicability caused by the serious dependence on artificial samples in the existing remote sensing change detection method.
In a first aspect, an embodiment of the present invention provides a remote sensing change detection method based on self-supervised deep learning, including: acquiring a sample earth surface reflectivity image pair; processing the sample surface reflectivity image pair by using a preset algorithm to obtain a pre-training image, wherein the preset algorithm comprises: a CCA-EM self-supervision image transformation algorithm, a typical association analysis algorithm, a expectation maximization algorithm and an Ojin segmentation algorithm; performing image visual enhancement processing on the sample surface reflectivity image pair to obtain a target surface reflectivity image pair; training a deep learning model by utilizing the pre-training image and the target surface reflectivity image to obtain a self-supervision deep learning model; and after the surface reflectivity image pair to be processed is obtained, determining the change pattern spots in the surface reflectivity image pair to be processed by using the self-supervision deep learning model.
Further, the sample surface reflectivity image pair includes: a front time phase sample remote sensing image and a rear time phase sample remote sensing image; processing the sample surface reflectivity image pair by using a preset algorithm to obtain a pre-training image, wherein the processing comprises the following steps: converting the front time phase sample remote sensing image and the rear time phase sample remote sensing image into a first two-dimensional array and a second two-dimensional array respectively by utilizing an array size conversion algorithm; converting the first two-dimensional array and the second two-dimensional array into a target array by utilizing a CCA-EM self-supervision image transformation algorithm; performing size transformation on the target array to obtain a transformed image; and dividing the transformed image by using the Ojin division algorithm to obtain the pre-training image.
Further, performing image visual enhancement processing on the sample surface reflectivity image pair to obtain a target surface reflectivity image pair, including: performing target processing on the front time phase sample remote sensing image and the rear time phase sample remote sensing image respectively to obtain the target surface reflectivity image pair, wherein the target processing comprises: vegetation enhancement treatment, nonlinear stretching treatment, gamma enhancement treatment and band extraction treatment.
Further, training the deep learning model by using the pre-training image and the target surface reflectivity image to obtain a self-supervision deep learning model, including: according to a preset size, slicing the pre-training image and the target surface reflectivity image pair respectively to obtain a pre-training image after slicing and a target surface reflectivity image pair after slicing; generating a slice sample based on the degree of overlap between the pre-training image after the slicing process and the target surface reflectivity image pair after the slicing process; and training the deep learning model by using the slice sample to obtain the self-supervision deep learning model.
Further, determining the change pattern spots in the to-be-processed surface reflectivity image pair by using the self-supervision deep learning model comprises the following steps: performing visual enhancement on the surface reflectivity image pair to be processed to obtain an intermediate surface reflectivity image pair; slicing the middle surface reflectivity image pair according to the preset size to obtain a sliced image; inputting the slice image into the self-supervision deep learning model, and determining the change pattern spots in the surface reflectivity image pair to be processed.
Further, the method further comprises: and performing open operation processing on the change pattern spots in the to-be-processed surface reflectivity image pair to obtain a change detection result.
Further, before processing the pair of sample surface reflectivity images by using a preset algorithm to obtain a pre-training image, the method further includes: and resampling the sample earth surface reflectivity image pair and cutting the intersection area.
In a second aspect, an embodiment of the present invention further provides a remote sensing change detection device based on self-supervised deep learning, including: the acquisition unit is used for acquiring a sample earth surface reflectivity image pair; the first processing unit is used for processing the sample surface reflectivity image pair by using a preset algorithm to obtain a pre-training image, wherein the preset algorithm comprises: a CCA-EM self-supervision image transformation algorithm, a typical association analysis algorithm, a expectation maximization algorithm and an Ojin segmentation algorithm; the second processing unit is used for performing image visual enhancement processing on the sample surface reflectivity image pair to obtain a target surface reflectivity image pair; the training unit is used for training the deep learning model by utilizing the pre-training image and the target surface reflectivity image to obtain a self-supervision deep learning model; and the detection unit is used for determining the change pattern spots in the surface reflectivity image pair to be processed by utilizing the self-supervision deep learning model after the surface reflectivity image pair to be processed is acquired.
In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, where the memory is configured to store a program for supporting the processor to execute the method described in the first aspect, and the processor is configured to execute the program stored in the memory.
In a fourth aspect, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon.
In the embodiment of the invention, a sample surface reflectivity image pair is obtained; processing the sample surface reflectivity image pair by using a preset algorithm to obtain a pre-training image, wherein the preset algorithm comprises: a CCA-EM self-supervision image transformation algorithm, a typical association analysis algorithm, a expectation maximization algorithm and an Ojin segmentation algorithm; performing image visual enhancement processing on the sample surface reflectivity image pair to obtain a target surface reflectivity image pair; training a deep learning model by utilizing the pre-training image and the target surface reflectivity image to obtain a self-supervision deep learning model; after the surface reflectivity image pair to be processed is obtained, the self-supervision deep learning model is utilized to determine the change pattern spots in the surface reflectivity image pair to be processed, the purpose of carrying out change detection by utilizing the self-supervision segmentation and deep learning network model is achieved, and further the technical problems of low engineering efficiency and poor applicability caused by the serious dependence on artificial samples in the existing remote sensing change detection method are solved, and the technical effects of improving the detection efficiency and applicability of the remote sensing change detection method are achieved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
In order to make the above objects, features and advantages of the present invention more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a remote sensing change detection method based on self-supervised deep learning provided by an embodiment of the invention;
FIG. 2 is a schematic diagram of a BIT-CD deep learning framework according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a remote sensing change detection device based on self-supervised deep learning according to an embodiment of the present invention;
fig. 4 is a schematic diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Embodiment one:
in accordance with an embodiment of the present invention, there is provided an embodiment of a remote sensing change detection method based on self-supervised deep learning, it being noted that the steps illustrated in the flowchart of the figures may be performed in a computer system, such as a set of computer executable instructions, and that, although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order other than that illustrated herein.
FIG. 1 is a flow chart of a remote sensing change detection method based on self-supervised deep learning according to an embodiment of the present invention, as shown in FIG. 1, the method comprises the steps of:
step S102, obtaining a sample earth surface reflectivity image pair;
step S104, processing the sample surface reflectivity image pair by using a preset algorithm to obtain a pre-training image, where the preset algorithm includes: a CCA-EM self-supervision image transformation algorithm, a typical association analysis algorithm, a expectation maximization algorithm and an Ojin segmentation algorithm;
step S106, performing image visual enhancement processing on the sample surface reflectivity image pair to obtain a target surface reflectivity image pair;
step S108, training a deep learning model by utilizing the pre-training image and the target surface reflectivity image to obtain a self-supervision deep learning model;
step S110, after obtaining the surface reflectivity image pair to be processed, determining the change pattern spots in the surface reflectivity image pair to be processed by using the self-supervision deep learning model.
In the embodiment of the invention, a sample surface reflectivity image pair is obtained; processing the sample surface reflectivity image pair by using a preset algorithm to obtain a pre-training image, wherein the preset algorithm comprises: a CCA-EM self-supervision image transformation algorithm, a typical association analysis algorithm, a expectation maximization algorithm and an Ojin segmentation algorithm; performing image visual enhancement processing on the sample surface reflectivity image pair to obtain a target surface reflectivity image pair; training a deep learning model by utilizing the pre-training image and the target surface reflectivity image to obtain a self-supervision deep learning model; after the surface reflectivity image pair to be processed is obtained, the self-supervision deep learning model is utilized to determine the change pattern spots in the surface reflectivity image pair to be processed, the purpose of carrying out change detection by utilizing the self-supervision segmentation and deep learning network model is achieved, and further the technical problems of low engineering efficiency and poor applicability caused by the serious dependence on artificial samples in the existing remote sensing change detection method are solved, and the technical effects of improving the detection efficiency and applicability of the remote sensing change detection method are achieved.
In the embodiment of the invention, before the sample surface reflectivity image pair is processed by using a preset algorithm to obtain the pre-training image, the method further comprises the following steps:
and step S103, resampling processing and intersection region cutting processing are carried out on the sample surface reflectivity image pair.
In the embodiment of the invention, after a sample surface reflectivity image pair is obtained, the sample surface reflectivity image pair is uniformly resampled, the sample surface reflectivity image pair is changed into an image with the resolution ratio of R, then an intersecting region is extracted from the sample surface reflectivity image pair, two-period images are cut based on the intersecting region, the sample surface reflectivity image pair which is subjected to resampling and cutting in the intersecting region is marked as X and Y, the X and Y are in the same region and have the same size, resolution ratio and band number, the threshold range of X and Y is 0-1, the size of the X and Y is (N, L and W), N is the number of image bands, and L and W are the length and width of the image respectively.
In an embodiment of the present invention, the pair of sample surface reflectivity images includes: the step S104 includes the steps of:
processing the sample surface reflectivity image pair by using a preset algorithm to obtain a pre-training image, wherein the processing comprises the following steps:
converting the front time phase sample remote sensing image and the rear time phase sample remote sensing image into a first two-dimensional array and a second two-dimensional array respectively by utilizing an array size conversion algorithm;
converting the first two-dimensional array and the second two-dimensional array into a target array by utilizing a CCA-EM self-supervision image transformation algorithm;
performing size transformation on the target array to obtain a transformed image;
and dividing the transformed image by using the Ojin division algorithm to obtain the pre-training image.
In the embodiment of the invention, because the acquisition time and imaging conditions of the pre-change time phase sample remote sensing image and the post-change time phase sample remote sensing image are different, the same ground object can generate chromatic aberration on two-phase images, which is the main noise influencing the change detection, in order to reduce chromatic aberration interference, and simultaneously, the spectral characteristics of remote sensing data are utilized to the greatest extent, a CCA-EM self-supervision image transformation algorithm is constructed, and the algorithm is combined with a typical association analysis algorithm (Canonical Correlation Analysis, CCA) and an expected maximization algorithm (Expectation Maximization, EM) to generate a transformation image through continuous iteration, and then the self-adaptive threshold segmentation is utilized to generate a pre-training change image.
The corresponding pre-training change image S is generated by the X and the Y, and the specific implementation process is as follows:
the dimensions of the X and Y three-dimensional arrays are reduced to two-dimensional arrays by array size transformation, the size is (N, M), M=L×W, and the two-dimensional arrays are marked as A and B.
Wherein i is the ith band,
and->
The arrays of the i-th wave bands of A and B respectively.
Inputting A and B into a CCA-EM self-supervision image transformation algorithm, setting the maximum iteration number of the EM as max_item and the minimum residual as min_residual. Creating a transformation coefficient array C with the size of (N, 1), a weight array P with the size of (1, M), and initial elements of C and P being 1.
According to CCA-EM, a transformation array D is generated, CCA-EM comprises CCA image transformation and EM iteration, first, a first EM iteration is carried out, weighted average arrays A2 and B2 of A and B are calculated, weight covariance cov of A2 and B2 is calculated, then feature decomposition is carried out on cov, feature values sig and feature vectors corresponding to A2 and B2 are generated, sig is ordered from small to large, a CCA transformation array and a CHI-square array CHI are calculated, a weight array P is updated according to probability density functions distributed according to the CHI-square, residual of the first CCA transformation is calculated, CHI is the transformation array D if residual of the first time is smaller than min_residual, next EM iteration is carried out until all iterations are completed or residual conditions are met, and iteration is ended.
The D array is subjected to size transformation, the D array is changed from (1, M) to (L, W), a transformed image is generated, and then a segmentation image S is obtained by using an Ojin self-adaptive segmentation algorithm, wherein the formula is as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,
for the segmentation threshold +.>
For adaptive division of body fluid, the formula->
For the segmentation scale, STD (D) is the standard deviation of D.
And sequentially carrying out the processing on all the sample surface reflectivity image pairs to generate pre-training images corresponding to the sample surface reflectivity image pairs.
In the embodiment of the present invention, step S106 includes the following steps:
performing target processing on the front time phase sample remote sensing image and the rear time phase sample remote sensing image respectively to obtain the target surface reflectivity image pair, wherein the target processing comprises: vegetation enhancement treatment, nonlinear stretching treatment, gamma enhancement treatment and band extraction treatment.
In the embodiment of the invention, the deep learning framework is derived from a visual algorithm, the deep learning model is trained based on a large number of natural images, and the remote sensing image has the characteristics of multiple scales, multiple spectrums, different geographic distributions, complex ground features and the like, so that the recognition of vision is not facilitated, and in order to improve the recognition accuracy of the deep learning change detection model, the visual enhancement processing is performed on the remote sensing data of the feeding model in the data layer.
The sample earth surface reflectivity image pair comprises a front time phase sample remote sensing image X and a rear time phase sample remote sensing image Y, and vegetation enhancement, nonlinear stretching, gamma enhancement, band extraction and other treatments are sequentially carried out on the X and the Y to generate images X3 and Y3 using a visual recognition and deep learning framework. The method comprises the following steps:
as most of the ground features of the remote sensing image are vegetation, the enhancement of vegetation information can improve the visual recognition capability. The vegetation enhancement is mainly constructed according to the NDVI ladder threshold value to generate a new green wave band. The vegetation enhancement of the X image is carried out by firstly creating a new G array with the size of (L, W) and the size of 0 as initial data of a new green wave band of the X image, and then filling the data of the new wave band based on the NDVI threshold range, wherein the specific formula is as follows:
wherein the method comprises the steps of
The reflectivity of the X near infrared, red and green wave bands, i is the ith gradient, and the gradient number is T, +>
And->
Threshold for NDVI at i gradient, +.>
And->
Coefficients at i gradient for near infrared and green bands,/->
The vegetation enhanced green band surface reflectivity of the i gradient is obtained, and G is the newly constructed green band surface reflectivity.
And simultaneously carrying out the same treatment on Y, and respectively replacing the original green wave band data of X and Y with the newly constructed vegetation enhanced green wave band surface reflectivity to generate X1 and Y1 with the sizes of (N, L and W).
And performing nonlinear stretching and gamma enhancement on each wave band of the X1 and Y1 images to generate X2 and Y2, wherein the formula is as follows:
where i is the ith band of X1 or Y1,
for the i-band surface reflectivity +.>
Is->
The value of the percentile 2 is chosen,
is->
The value of percentile 98, r, is the gamma enhancement factor. After stretching->
A grant of 0.0001 less than 0, a grant of 1 greater than 1, and a stretch and gamma reinforcement +.>
The threshold value ranges from 0 to 1.
The sizes of X2 and Y2 are (N, L, W), X2 and Y2 contain multispectral data, and the deep learning framework can only identify the spectrum channel of a natural picture at present, namely, red, green and blue (RGB) three-band uint8 type data, namely, RGB bands need to be extracted from multispectral remote sensing and converted into uint8 type images, and the formula is as follows:
where i is the mid RGB band position of X1 or Y1.
After the wave band extraction and the data type conversion, the X2 and Y3 generate an image pair with the wave band number of 3 and the length and the width of L and W, which are marked as X3 and Y3.
The same visual enhancement process is performed on all of the multiple local surface reflectivity image pairs, generating image pairs (i.e., target surface reflectivity image pairs) suitable for deep learning and improving visual interpretation capabilities.
In the embodiment of the present invention, step S108 includes the steps of:
according to a preset size, slicing the pre-training image and the target surface reflectivity image pair respectively to obtain a pre-training image after slicing and a target surface reflectivity image pair after slicing;
generating a slice sample based on the degree of overlap between the pre-training image after the slicing process and the target surface reflectivity image pair after the slicing process;
and training the deep learning model by using the slice sample to obtain the self-supervision deep learning model.
In the embodiment of the invention, the self-supervision and segmentation adopts the image transformation and threshold segmentation technology, so that the obtained change region with higher reliability can not be avoided, the deep learning has a deep network, the result of the self-supervision and segmentation can be independently learned, the change characteristics of the result can be extracted, the region similar to the self-supervision and change region can be found, the problem of missed detection can be avoided to a great extent, and the deep learning change model can be further constructed in the process.
A deep learning model for change detection is constructed based on multiple sets of visually enhanced image pairs and the acquired pre-trained images of each image pair self-supervised segmentation. BIT-CD (Bitemporal Image Transformer Change Detection) is selected as a deep learning change detection framework, the trunk of the network is CNN-based, and the framework is combined with a transducer decoder to extract the change, so that the framework has higher precision and speed in the field of remote sensing change detection. Firstly, slicing an image pair and a pre-training image (binary label image) according to engineering experience and requirements of a framework, generating a slice sample according to a set slice size and overlapping degree, and then randomly dividing the sample into a training sample and a verification sample according to a certain proportion. And feeding the training sample into a BIT-CD framework to finally obtain the verified self-supervision deep learning change detection model.
The BIT-CD deep learning framework design is shown in fig. 2. BIT-CD is a method that uses the backbone CNN to extract high-level semantic features from the input image pairs and converts each temporal feature map into a set of compact spatiotemporal semantic tags. And then, connecting and correlating the space-time information based on the marks by using a transducer encoder, and performing projection conversion on the context-rich information obtained by the two-period time images to obtain a difference image. Finally, the difference image is input into a shallow CNN to generate pixel-level predictions.
The main super-parameters of the BIT-CD are a loss function loss, an optimizer, a learning rate lr and a maximum iteration number max_epochs. The BIT-CD is specifically processed as follows:
firstly defining a pair of slice images as S1 and S2, wherein the size is (3, size), the size is the slice size, 3 is the band number, and then carrying out feature extraction on the S1 and the S2 by using ResNet to obtain
And->
Then->
And->
Coding, will->
And->
The L wave bands are obtained by the convolution kernel sent into 1*1 and the split softmax operationSemantic information with the number N, recorded as +.>
And->
Then based on->
And->
Training weights are acquired and updated using a transducer encoder>
And
finally, the inverse convolution is used for the ∈>
And->
Treatment to obtain->
And->
Then extracting through two convolution layers, and finally obtaining a change image P with the size of 3 (size) through a Softmax layer, wherein the formula is as follows:
wherein P is the changing image,
is a Softmax layer, g is two convolution layers.
In the embodiment of the present invention, step S110 includes the following steps:
performing visual enhancement on the surface reflectivity image pair to be processed to obtain an intermediate surface reflectivity image pair;
slicing the middle surface reflectivity image pair according to the preset size to obtain a sliced image;
inputting the slice image into the self-supervision deep learning model, and determining the change pattern spots in the surface reflectivity image pair to be processed.
In the embodiment of the invention, after the surface reflectivity image pair to be processed is obtained, firstly, visual enhancement processing is carried out on the surface reflectivity image pair to be processed, then slicing processing is carried out, each slice is predicted by utilizing a self-supervision deep learning model to carry out change detection, and finally, the obtained result is inlaid into a change detection result image D (namely, a change pattern spot in the surface reflectivity image pair to be processed is determined) according to the slice coordinate information.
In the embodiment of the invention, after the change pattern spots in the surface reflectivity image pair to be processed are obtained, the method further comprises the following steps:
and performing open operation processing on the change pattern spots in the to-be-processed surface reflectivity image pair to obtain a change detection result.
In the embodiment of the invention, the problem of island or cavity of the change detection result image D is solved by using a mathematical morphological operation algorithm, and the outline of the pattern spots is optimized, so that the pattern spots are distributed and communicated reasonably. The mathematical morphological opening operation method determines whether a pixel is in the same group as surrounding pixels by analyzing 4 or 8 pixels around the pixel. If the number of pixels analyzed is less than the threshold of input, the pixels are deleted or filled.
The embodiment of the invention is suitable for most surface reflectivity image products, is sensitive to typical object changes of buildings, vegetation, water bodies and the like, and can autonomously generate samples for deep learning requirements in a self-supervision mode by utilizing self-supervision segmentation, visual enhancement, deep learning and mathematical morphology technologies, especially combining the self-supervision segmentation with a deep learning network, thereby reducing the dependence of the deep learning on the samples, enhancing the application capacity of the deep learning change detection engineering, having the advantages of strong applicability, high automation degree and the like, and providing basic technical support for the fields of national soil monitoring, ecological environment protection and the like.
Embodiment two:
the embodiment of the invention also provides a remote sensing change detection device based on self-supervision deep learning, which is used for executing the method provided by the embodiment of the invention, and the following is a specific introduction of the remote sensing change detection device based on self-supervision deep learning.
As shown in fig. 3, fig. 3 is a schematic diagram of the foregoing remote sensing change detection based on self-supervised deep learning, where the remote sensing change detection based on self-supervised deep learning includes:
an acquisition unit 10 for acquiring a sample surface reflectance image pair;
the first processing unit 20 is configured to process the pair of sample surface reflectivity images by using a preset algorithm to obtain a pre-training image, where the preset algorithm includes: a CCA-EM self-supervision image transformation algorithm, a typical association analysis algorithm, a expectation maximization algorithm and an Ojin segmentation algorithm;
a second processing unit 30, configured to perform image visual enhancement processing on the sample surface reflectivity image pair, so as to obtain a target surface reflectivity image pair;
a training unit 40, configured to train the deep learning model by using the pre-training image and the target surface reflectivity image, so as to obtain a self-supervised deep learning model;
the detection unit 50 is configured to determine, after acquiring the surface reflectivity image pair to be processed, a change map spot in the surface reflectivity image pair to be processed by using the self-supervised deep learning model.
In the embodiment of the invention, a sample surface reflectivity image pair is obtained; processing the sample surface reflectivity image pair by using a preset algorithm to obtain a pre-training image, wherein the preset algorithm comprises: a CCA-EM self-supervision image transformation algorithm, a typical association analysis algorithm, a expectation maximization algorithm and an Ojin segmentation algorithm; performing image visual enhancement processing on the sample surface reflectivity image pair to obtain a target surface reflectivity image pair; training a deep learning model by utilizing the pre-training image and the target surface reflectivity image to obtain a self-supervision deep learning model; after the surface reflectivity image pair to be processed is obtained, the self-supervision deep learning model is utilized to determine the change pattern spots in the surface reflectivity image pair to be processed, the purpose of carrying out change detection by utilizing the self-supervision segmentation and deep learning network model is achieved, and further the technical problems of low engineering efficiency and poor applicability caused by the serious dependence on artificial samples in the existing remote sensing change detection method are solved, and the technical effects of improving the detection efficiency and applicability of the remote sensing change detection method are achieved.
Embodiment III:
an embodiment of the present invention further provides an electronic device, including a memory and a processor, where the memory is configured to store a program that supports the processor to execute the method described in the first embodiment, and the processor is configured to execute the program stored in the memory.
Referring to fig. 4, an embodiment of the present invention further provides an electronic device 100, including: a processor 60, a memory 61, a bus 62 and a communication interface 63, the processor 60, the communication interface 63 and the memory 61 being connected by the bus 62; the processor 60 is arranged to execute executable modules, such as computer programs, stored in the memory 61.
The memory 61 may include a high-speed random access memory (RAM, random Access Memory), and may further include a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory. The communication connection between the system network element and at least one other network element is achieved via at least one communication interface 63 (which may be wired or wireless), and may use the internet, a wide area network, a local network, a metropolitan area network, etc.
Bus 62 may be an ISA bus, a PCI bus, an EISA bus, or the like. The buses may be classified as address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in FIG. 4, but not only one bus or type of bus.
The memory 61 is configured to store a program, and the processor 60 executes the program after receiving an execution instruction, and the method executed by the apparatus for flow defining disclosed in any of the foregoing embodiments of the present invention may be applied to the processor 60 or implemented by the processor 60.
The processor 60 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuitry in hardware or instructions in software in the processor 60. The processor 60 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but may also be a digital signal processor (Digital Signal Processing, DSP for short), application specific integrated circuit (Application Specific Integrated Circuit, ASIC for short), off-the-shelf programmable gate array (Field-ProgrammableGate Array, FPGA for short), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory 61 and the processor 60 reads the information in the memory 61 and in combination with its hardware performs the steps of the method described above.
Embodiment four:
the embodiment of the invention also provides a computer readable storage medium, and a computer program is stored on the computer readable storage medium, and when the computer program is executed by a processor, the steps of the method in the first embodiment are executed.
In addition, in the description of embodiments of the present invention, unless explicitly stated and limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art.
In the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. are based on the directions or positional relationships shown in the drawings, are merely for convenience of describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
Finally, it should be noted that: the above examples are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention, but it should be understood by those skilled in the art that the present invention is not limited thereto, and that the present invention is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (9)
1. The remote sensing change detection method based on self-supervision deep learning is characterized by comprising the following steps of:
acquiring a sample earth surface reflectivity image pair;
processing the sample surface reflectivity image pair by using a preset algorithm to obtain a pre-training image, wherein the preset algorithm comprises: a CCA-EM self-supervision image transformation algorithm and an Ojin segmentation algorithm, or the preset algorithm comprises: typical association analysis algorithms, expectation maximization algorithms, and oxford segmentation algorithms;
performing image visual enhancement processing on the sample surface reflectivity image pair to obtain a target surface reflectivity image pair;
training a deep learning model by utilizing the pre-training image and the target surface reflectivity image to obtain a self-supervision deep learning model, wherein the deep learning model is constructed based on a BIT-CD network;
after the surface reflectivity image pair to be processed is obtained, determining a change pattern spot in the surface reflectivity image pair to be processed by using the self-supervision deep learning model;
wherein, the sample earth surface reflectivity image pair includes: a front time phase sample remote sensing image and a rear time phase sample remote sensing image;
processing the sample surface reflectivity image pair by using a preset algorithm to obtain a pre-training image, wherein the processing comprises the following steps:
converting the front time phase sample remote sensing image and the rear time phase sample remote sensing image into a first two-dimensional array and a second two-dimensional array respectively by utilizing an array size conversion algorithm;
converting the first two-dimensional array and the second two-dimensional array into a target array by utilizing a CCA-EM self-supervision image transformation algorithm;
performing size transformation on the target array to obtain a transformed image;
and dividing the transformed image by using the Ojin division algorithm to obtain the pre-training image.
2. The method of claim 1, wherein performing image visual enhancement processing on the sample surface reflectance image pair to obtain a target surface reflectance image pair comprises:
performing target processing on the front time phase sample remote sensing image and the rear time phase sample remote sensing image respectively to obtain the target surface reflectivity image pair, wherein the target processing comprises: vegetation enhancement treatment, nonlinear stretching treatment, gamma enhancement treatment and band extraction treatment.
3. The method of claim 1, wherein training a deep learning model using the pre-training image and the target surface reflectivity image results in a self-supervised deep learning model, comprising:
according to a preset size, slicing the pre-training image and the target surface reflectivity image pair respectively to obtain a pre-training image after slicing and a target surface reflectivity image pair after slicing;
generating a slice sample based on the degree of overlap between the pre-training image after the slicing process and the target surface reflectivity image pair after the slicing process;
and training the deep learning model by using the slice sample to obtain the self-supervision deep learning model.
4. A method according to claim 3, wherein determining a change patch in the pair of surface reflectivity images to be processed using the self-supervised deep learning model comprises:
performing visual enhancement on the surface reflectivity image pair to be processed to obtain an intermediate surface reflectivity image pair;
slicing the middle surface reflectivity image pair according to the preset size to obtain a sliced image;
inputting the slice image into the self-supervision deep learning model, and determining the change pattern spots in the surface reflectivity image pair to be processed.
5. The method according to claim 1, wherein the method further comprises:
and performing open operation processing on the change pattern spots in the to-be-processed surface reflectivity image pair to obtain a change detection result.
6. The method of claim 1, wherein prior to processing the sample surface reflectance image pair using a predetermined algorithm to obtain a pre-trained image, the method further comprises:
and resampling the sample earth surface reflectivity image pair and cutting the intersection area.
7. Remote sensing change detection device based on self-supervision deep learning, characterized by comprising:
the acquisition unit is used for acquiring a sample earth surface reflectivity image pair;
the first processing unit is used for processing the sample surface reflectivity image pair by using a preset algorithm to obtain a pre-training image, wherein the preset algorithm comprises: a CCA-EM self-supervision image transformation algorithm and an Ojin segmentation algorithm, or the preset algorithm comprises: typical association analysis algorithms, expectation maximization algorithms, and oxford segmentation algorithms;
the second processing unit is used for performing image visual enhancement processing on the sample surface reflectivity image pair to obtain a target surface reflectivity image pair;
the training unit is used for training the deep learning model by utilizing the pre-training image and the target surface reflectivity image to obtain a self-supervision deep learning model, wherein the deep learning model is a deep learning model constructed based on a BIT-CD network;
the detection unit is used for determining a change pattern spot in the surface reflectivity image pair to be processed by utilizing the self-supervision deep learning model after the surface reflectivity image pair to be processed is acquired;
wherein, the sample earth surface reflectivity image pair includes: a front time phase sample remote sensing image and a rear time phase sample remote sensing image;
the first processing unit is used for:
converting the front time phase sample remote sensing image and the rear time phase sample remote sensing image into a first two-dimensional array and a second two-dimensional array respectively by utilizing an array size conversion algorithm;
converting the first two-dimensional array and the second two-dimensional array into a target array by utilizing a CCA-EM self-supervision image transformation algorithm;
performing size transformation on the target array to obtain a transformed image;
and dividing the transformed image by using the Ojin division algorithm to obtain the pre-training image.
8. An electronic device comprising a memory for storing a program supporting the processor to perform the method of any one of claims 1 to 6, and a processor configured to execute the program stored in the memory.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, performs the steps of the method according to any of the preceding claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310140461.8A CN115861823B (en) | 2023-02-21 | 2023-02-21 | Remote sensing change detection method and device based on self-supervision deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310140461.8A CN115861823B (en) | 2023-02-21 | 2023-02-21 | Remote sensing change detection method and device based on self-supervision deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115861823A CN115861823A (en) | 2023-03-28 |
| CN115861823B true CN115861823B (en) | 2023-05-09 |
Family
ID=85658550
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310140461.8A Active CN115861823B (en) | 2023-02-21 | 2023-02-21 | Remote sensing change detection method and device based on self-supervision deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115861823B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116091497B (en) * | 2023-04-07 | 2023-06-27 | 航天宏图信息技术股份有限公司 | Remote sensing change detection method, device, electronic equipment and storage medium |
| CN117218552B (en) * | 2023-08-04 | 2024-06-04 | 华南师范大学 | Estimation algorithm optimization method and device based on pixel change detection |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017024963A1 (en) * | 2015-08-11 | 2017-02-16 | 阿里巴巴集团控股有限公司 | Image recognition method, measure learning method and image source recognition method and device |
| CN110991248A (en) * | 2019-11-04 | 2020-04-10 | 同济大学 | High-resolution noctilucent remote sensing image automatic change detection method based on feature fusion |
| CN111476813A (en) * | 2020-04-28 | 2020-07-31 | 兰州交通大学 | Image change detection method, image change detection device, electronic equipment and storage medium |
| CN113554011A (en) * | 2021-09-22 | 2021-10-26 | 航天宏图信息技术股份有限公司 | Method, device and equipment for detecting change of remote sensing image and storage medium |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10346495B2 (en) * | 2013-02-19 | 2019-07-09 | Digitalglobe, Inc. | System and method for large scale crowdsourcing of map data cleanup and correction |
| US10176194B2 (en) * | 2013-02-19 | 2019-01-08 | Digitalglobe, Inc. | Enhanced crowdsourced search and locate platform |
| US20160379388A1 (en) * | 2014-07-16 | 2016-12-29 | Digitalglobe, Inc. | System and method for combining geographical and economic data extracted from satellite imagery for use in predictive modeling |
| US20180018757A1 (en) * | 2016-07-13 | 2018-01-18 | Kenji Suzuki | Transforming projection data in tomography by means of machine learning |
| CN109284692A (en) * | 2018-08-31 | 2019-01-29 | 河南工程学院 | Merge the face identification method of EM algorithm and probability two dimension CCA |
| CN110648347A (en) * | 2019-09-24 | 2020-01-03 | 北京航天宏图信息技术股份有限公司 | Coastline extraction method and device based on remote sensing image |
| CN113111974B (en) * | 2021-05-10 | 2021-12-14 | 清华大学 | Vision-laser radar fusion method and system based on depth canonical correlation analysis |
| CN114926511A (en) * | 2022-05-11 | 2022-08-19 | 西安电子科技大学 | High-resolution remote sensing image change detection method based on self-supervision learning |
-
2023
- 2023-02-21 CN CN202310140461.8A patent/CN115861823B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017024963A1 (en) * | 2015-08-11 | 2017-02-16 | 阿里巴巴集团控股有限公司 | Image recognition method, measure learning method and image source recognition method and device |
| CN110991248A (en) * | 2019-11-04 | 2020-04-10 | 同济大学 | High-resolution noctilucent remote sensing image automatic change detection method based on feature fusion |
| CN111476813A (en) * | 2020-04-28 | 2020-07-31 | 兰州交通大学 | Image change detection method, image change detection device, electronic equipment and storage medium |
| CN113554011A (en) * | 2021-09-22 | 2021-10-26 | 航天宏图信息技术股份有限公司 | Method, device and equipment for detecting change of remote sensing image and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| 基于深度置信网络与数学形态学融合的遥感影像建筑物变化检测;朱春宇等;科学技术与工程(08);全文 * |
| 结合深度学习和超像元的高分遥感影像变化检测;王艳恒等;中国图象图形学报(06);全文 * |
| 高分辨率遥感影像的深度学习变化检测方法;张鑫龙等;测绘学报(08);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115861823A (en) | 2023-03-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115861823B (en) | Remote sensing change detection method and device based on self-supervision deep learning | |
| CN111986099B (en) | Tillage monitoring method and system based on convolutional neural network with residual error correction fused | |
| US10015360B1 (en) | Image-based field boundary detection and identification | |
| CN109886997B (en) | Identification frame determining method and device based on target detection and terminal equipment | |
| CN110781756A (en) | Urban road extraction method and device based on remote sensing image | |
| CN116091497B (en) | Remote sensing change detection method, device, electronic equipment and storage medium | |
| CN104517110A (en) | Binarization method and system of two-dimensional code image | |
| CN111860398A (en) | Remote sensing image target detection method and system and terminal equipment | |
| CN115661144A (en) | Self-adaptive medical image segmentation method based on deformable U-Net | |
| CN112712535A (en) | Mask-RCNN landslide segmentation method based on simulation difficult sample | |
| US20220215656A1 (en) | Method, apparatus, device for image processing, and storage medium | |
| CN113239736B (en) | Land coverage classification annotation drawing acquisition method based on multi-source remote sensing data | |
| CN112966635B (en) | Low-resolution time sequence remote sensing image-oriented moving ship detection method and device | |
| CN115861409B (en) | Soybean leaf area measuring and calculating method, system, computer equipment and storage medium | |
| CN114943876A (en) | Cloud and cloud shadow detection method and device for multi-level semantic fusion and storage medium | |
| CN114494821A (en) | Remote sensing image cloud detection method based on feature multi-scale perception and self-adaptive aggregation | |
| CN113887472A (en) | Remote sensing image cloud detection method based on cascade color and texture feature attention | |
| US20200265227A1 (en) | Background suppression for anomaly detection | |
| CN114240940B (en) | Cloud and cloud shadow detection method and device based on remote sensing image | |
| CN113537017A (en) | Optical remote sensing image airplane detection method and device based on cascade regression correction | |
| CN116704324A (en) | Target detection method, system, equipment and storage medium based on underwater image | |
| CN111178186A (en) | Rice extraction method, device and equipment based on sentinel remote sensing data | |
| CN114120056A (en) | Small target identification method, small target identification device, electronic equipment, medium and product | |
| CN112633158A (en) | Power transmission line corridor vehicle identification method, device, equipment and storage medium | |
| CN111460965A (en) | Waterlogging area intelligent analysis system and method based on unmanned aerial vehicle image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |