CN115457385A - Building change detection method based on lightweight network - Google Patents
Building change detection method based on lightweight network Download PDFInfo
- Publication number
- CN115457385A CN115457385A CN202211027648.9A CN202211027648A CN115457385A CN 115457385 A CN115457385 A CN 115457385A CN 202211027648 A CN202211027648 A CN 202211027648A CN 115457385 A CN115457385 A CN 115457385A
- Authority
- CN
- China
- Prior art keywords
- change detection
- conv
- building
- network
- loss
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/176—Urban or other man-made structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Abstract
The invention provides a building change detection method based on a lightweight network, which comprises the following steps: acquiring remote sensing image pairs of different time phases of a region to be detected; respectively extracting multilevel features in the remote sensing image pairs; calculating a building change detection result graph on the basis of fusing image multilevel features; post-processing the building change detection result; and finally, vectorizing the post-processing result to obtain a final vector result of the building change detection. The invention uses the lightweight network to extract the building characteristics, reduces the parameter and the calculated amount of the network, and ensures that the network has smaller volume and faster running speed; meanwhile, in the feature fusion stage, the deconvolution dynamic learning weight parameters are utilized, so that the loss of useful information in the size recovery stage is reduced, and the precision of building change detection is ensured.
Description
Technical Field
The invention belongs to the technical field of building change detection, and particularly relates to a building change detection method based on a lightweight network.
Background
In recent years, building change detection methods based on deep learning develop rapidly, wherein a convolutional neural network provides a new idea for optimization of a building change detection algorithm by virtue of strong feature extraction capability of the convolutional neural network.
The existing building change detection method based on the convolutional neural network can be divided into a first classification and then comparison method and a direct comparison method according to design strategies, the first classification and then comparison method is to firstly use the convolutional neural network to extract the characteristics of images before and after change or the building results and then compare the characteristics or the building results, although the method for extracting the characteristics firstly can automatically learn the depth characteristics, the method is essentially the analysis pixel by pixel, and the problem of false change generated by different shooting angles is difficult to solve; for the method of outputting the building result first, the change label and the double temporal semantic label are needed, and the data set is less at present. The direct classification method can be classified into a single-stream direct classification and a double-stream direct classification according to the design of a network framework. In a single-flow structure, two-stage images are directly spliced according to channels or difference images processed by the two-stage images are input into a semantic segmentation network to obtain a change result; although the method utilizes rich features of multiple scales and multiple levels, independent features of a single image are ignored in the encoding and decoding processes, such as the boundary integrity of the single image and the internal compactness of a building, and the boundary error in the detection result and the internal hole of the building are caused. In the double-flow structure, two identical feature extractors are usually used for respectively extracting two-stage image features, and then a change result is obtained through a feature fusion network; the method has good performance in building change detection, but the network scale is huge, and when the characteristics are extracted by using a network structure with 'very deep', the problems of huge calculation amount and the like can be caused.
Disclosure of Invention
The invention aims to solve the technical problem of high calculation cost caused by a building change detection method using a complex deep network and multi-scale fusion and provides a building change detection method based on a lightweight network. The method can reduce the parameters and the calculated amount of the model and simultaneously ensure the precision of the change detection.
The invention adopts a change detection network with a double-flow structure, takes MobilenetV2 as a feature extractor, respectively extracts the multilevel features of two-stage images, then fuses the multilevel features, and finally obtains a change detection result.
The technical scheme adopted by the invention is as follows:
a building change detection method based on a lightweight network comprises the following steps:
step 1) obtaining a remote sensing image of a to-be-detected area, wherein the remote sensing image comprises a time t 1 And t 2 Respectively acquiring images A and B;
step 2) extracting the characteristics of the building change detection image pairs A and B to respectively obtain the characteristic AF of the image A and the characteristic BF of the image B:
the method comprises the steps of adopting a double-flow structure network with shared weight to respectively extract the characteristics of change detection image pairs A and B, wherein a characteristic extractor in a branch of the double-flow structure network is a lightweight network MobilenetV2, and a fourth layer and a seventeenth layer of the MobilenetV2 are used as low-level characteristics and high-level characteristics of the change detection image pairs A and B, so that the characteristics of the building change detection image pairs A and B are AF = { A = } Fi I =4,17 and BF = { B = | i = Fi |i＝4，17}；
Step 3) calculating a change result graph of the buildings in the images A and B:
3.1 Respectively fusing the low-level features and the high-level features of the images A and B in the step 2) to obtain fused features F l And F h Is represented as follows:
F l ＝conv k＝3 (conv k＝3 (concat(A F4 ，B F4 )))#(1)
F h ＝conv k＝3 (conv k＝3 (concat(A F17 ，B F17 )))#(2)
wherein, conv k＝3 Convolution operation with convolution kernel of 3x3, concat is splicing operation according to channels;
3.2 ) the advanced fusion feature F of step 3.1) h Further extracting multi-scale features through spatial pyramid pooling, and upsampling to a level that is fused with a low-level feature F l Uniform size, giving feature F' h ：
F′ h ＝Upsampling bilinear (ASPP(F h ))#(3)
Wherein, upesampling bilinear Representing an upsampling operation in a bilinear interpolation mode, and representing a spatial pyramid pooling operation by ASPP;
3.3 ) the low-level fusion signature F of step 3.1) l Conv by 1x1 convolution k＝1 Vitamin D is added to obtain F' l Expressed as follows:
F′ l ＝conv k＝1 (F l )#(4)
3.4 ) is prepared from step 3.3) by characteristic F' l And step 3.2) feature F' h Further fusion gave F':
F″＝conv k＝3 (conv k＝3 (concat(F′ l ，F′ h )))#(5)
and then processing the F 'by adopting two sets of deconvolution and convolution operations to obtain a characteristic F':
F′＝conv k＝3 (deconv k＝2，s＝2 (conv k＝3 (deconv k＝2，s＝2 (F″))))#(6)
wherein, deconv k＝2，s＝2 Deconvolution operation with a convolution kernel of 2x2 and a step number of 2 is represented;
3.5 ) classifying the features F' of step 3.4) using a convolution of 1x1, resulting in a predicted class probability result P:
P＝conv k＝1 (F′)#(7)
loss function loss employed in a change detection network cd As follows:
loss cd ＝loss ce +loss dice #(8)
wherein loss ce Is a cross entropy loss function of two classes, loss dice Is a Dice loss function;
3.6 Binarizing the prediction probability in step 3.5) to obtain a building change detection result graph I, wherein the binarization result I (p) of any pixel is calculated as follows:
wherein p is i The prediction type probability result P indicates the type probability of any pixel, I indicates the type, 0 indicates no change in pixel, 1 indicates a changed building, 255 indicates a changed building, and 0 indicates an unchanged area.
Step 4) carrying out post-processing on the building change detection result graph I in the step 3.6), wherein the post-processing comprises the operations of removing small-area communication areas and filling holes to obtain a final change detection result graph R;
and 5) vectorizing the change detection result diagram R in the step 4) to obtain a vector result of the final change building.
The invention has the following beneficial effects:
1) The invention uses the lightweight network in the stage of extracting the characteristic, reduces the parameter quantity and the calculated quantity of the network, ensures that the network has smaller volume and has higher running speed;
2) The invention utilizes deconvolution dynamic learning weight parameters in the feature fusion stage, reduces the loss of useful information in the size recovery stage, and ensures the precision of building change detection.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a diagram of an image before modification in an embodiment of the present invention;
FIG. 3 is a modified image of an embodiment of the present invention;
FIG. 4 is a diagram of a network architecture in an embodiment of the present invention;
FIG. 5 is a graph of an image versus the change detection prediction of FIGS. 2 and 3 using the method of the present invention, where white represents a changing building and black represents the background.
Detailed Description
The invention is further illustrated by the following examples in connection with the accompanying drawings.
A building change detection method based on a lightweight network comprises the following steps:
the method comprises the steps of adopting a double-flow structure network with shared weight to respectively extract the characteristics of change detection image pairs A and B, wherein a characteristic extractor in a branch of the double-flow structure network is a lightweight network MobilenetV2, and a fourth layer and a seventeenth layer of the MobilenetV2 are used as low-level characteristics and high-level characteristics of the change detection image pairs A and B, so that the characteristics of the building change detection image pairs A and B are AF = { A = } Fi I =4,17 and BF = { B = | i = Fi |i＝4，17}；
(31) Respectively fusing the low-level features and the high-level features of the images A and B in the step 2 to obtain fused features F l And F h Is represented as follows:
F l ＝conv k＝3 (conv k＝3 (concat(A F4 ，B F4 )))#(1)
F h ＝conv k＝3 (conv k＝3 (concat(A F17 ，B F17 )))#(2)
wherein, conv k＝3 Convolution operation with a convolution kernel of 3x3, concat is splicing operation according to channels;
(32) The advanced fusion feature F in the step (31) h Further extracting multi-scale features through spatial pyramid pooling, and performing up-samplingSample to lower fusion characteristics F l Consistent size, giving feature F' h ：
F′ h ＝Upsampling bilinear (ASPP(F h ))#(3)
Wherein, upsampling bilinear Representing an upsampling operation in a bilinear interpolation mode, and representing a spatial pyramid pooling operation by ASPP;
(33) The low-level fusion characteristics F of the step (31) l Conv by 1x1 convolution k＝1 Vitamin D is added to obtain F' l Expressed as follows:
F′ l ＝conv k＝1 (F l )#(4)
(34) Mixing feature F 'of step (33)' l And feature F 'of step (32)' h Further fusion gave F ″:
F″＝conv k＝3 (conv k＝3 (concat(F′ l ，F′ h )))#(5)
and processing the F 'by adopting two groups of deconvolution and convolution operations to obtain a characteristic F':
F′＝conv k＝3 (deconv k＝2，s＝2 (conv k＝3 (deconv k＝2，s＝2 (F″))))#(6)
wherein, deconv k＝2，s＝2 Deconvolution operation with a convolution kernel of 2x2 and a step number of 2 is represented;
(35) Classifying the features F' of step (34) by using convolution of 1x1 to obtain a prediction class probability result P:
P＝conv k＝1 (F′)#(7)
loss function loss employed in a change detection network cd As follows:
loss cd ＝loss ce +loss dice #(8)
therein, loss ce Is a cross entropy loss function of two classes, loss dice Is a Dice loss function;
(36) And (5) binarizing the prediction probability in the step (35) to obtain a building change detection result graph I, wherein the binarization result I (p) of any pixel is calculated as follows:
wherein p is i The prediction type probability result P indicates the type probability of any pixel, I indicates the type, 0 indicates no change in the pixel, 1 indicates a changed building, 255 indicates a changed building, and 0 indicates an unchanged area. The structure of the change detection network in this embodiment is shown in fig. 4.
And 4, post-processing the building change detection result graph I obtained in the step (36), wherein the post-processing comprises the operations of removing small-area communication areas and filling holes to obtain a final change detection result graph R, and the final result of the building change detection in the embodiment is shown in FIG. 5.
And 5, vectorizing the change detection result graph R in the step 4 to obtain a vector result of the final change building.
The foregoing is merely a description of embodiments of the invention and is not intended to limit the scope of the invention to the particular forms set forth, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims.
Claims (1)
1. A building change detection method based on a lightweight network is characterized by comprising the following steps:
step 1) obtaining a remote sensing image of a to-be-detected area, wherein the remote sensing image comprises a time t 1 And t 2 Respectively acquiring images A and B;
step 2) extracting the characteristics of the building change detection image pair A and B to respectively obtain the characteristic AF of the image A and the characteristic BF of the image B:
respectively extracting the characteristics of a change detection image pair A and a change detection image pair B by adopting a double-flow structure network with shared weight, wherein a characteristic extractor in a branch of the double-flow structure network is a lightweight network MobilenetV2, and a fourth layer and a seventeenth layer of the MobilenetV2 are used as change detection image pairsThe low-level features and the high-level features of a and B, the feature of the building change detection image pair a and B thus obtained is AF = { a = { (a) } Fi I =4,17} and BF = { B = Fi |i＝4,17}；
Step 3) calculating a change result graph of the buildings in the images A and B:
3.1 Respectively fusing the low-level features and the high-level features of the images A and B in the step 2) to obtain fused features F l And F h Is represented as follows:
F l ＝conv k＝3 (conv k＝3 (concat(A F4 ，B F4 )))#(1)
F h ＝conv k＝3 (conv k＝3 (concat(A F17 ，B F17 )))#(2)
wherein, conv k＝3 Convolution operation with convolution kernel of 3x3, concat is splicing operation according to channels;
3.2 ) the advanced fusion feature F of step 3.1) h Further extracting multi-scale features by spatial pyramid pooling, and upsampling to a level F that fuses with low-level features l Uniform size, giving feature F' h ：
F′ h ＝Upsampling bilinear (ASPP(F h ))#(3)
Wherein, upesampling bilinear Representing an upsampling operation in a bilinear interpolation mode, and representing a spatial pyramid pooling operation by ASPP;
3.3 C) the lower fusion signature F of step 3.1) l Conv by 1x1 convolution k＝1 Vitamin E is extracted to obtain F' l Expressed as follows:
F′ l ＝conv k＝1 (F l )#(4)
3.4 ) step 3.3) feature F' l And step 3.2) feature F' h Further fusion gave F ":
F”＝conv k＝3 (conv k＝3 (concat(F' l ，F′ h )))#(5)
and then processing the F 'by adopting two groups of deconvolution and convolution operations to obtain a characteristic F':
F＝conv k＝3 (deconv k＝2,s＝2 (conv k＝3 (deconv k＝2,s＝2 (F″))))#(6)
wherein, deconv k＝2,s＝2 Deconvolution operation with a convolution kernel of 2x2 and a step number of 2 is represented;
3.5 ) classifying the features F' of step 3.4) using a convolution of 1x1, resulting in a predicted class probability result P:
P＝conv k＝1 (F′)#(7)
loss function loss employed in a change detection network cd As follows:
loss cd ＝loss ce +loss dice #(8)
therein, loss ce Is a cross entropy loss function of two classes, loss dice Is a Dice loss function;
3.6 Binarizing the prediction probability in the step 3.5) to obtain a building change detection result graph I, wherein a binarization result I (p) of any pixel is calculated as follows:
wherein p is i A category probability of any pixel in the prediction category probability result P is represented, I represents a category, 0 represents no change in pixel, 1 represents a changed building, 255 represents a changed building, and 0 represents an unchanged area in the building change detection result graph I;
step 4) carrying out post-processing on the building change detection result graph I in the step 3.6), wherein the post-processing comprises the operations of removing small-area communication areas and filling holes to obtain a final change detection result graph R;
and 5) vectorizing the change detection result graph R obtained in the step 4) to obtain a vector result of the final change building.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211027648.9A CN115457385A (en) | 2022-08-25 | 2022-08-25 | Building change detection method based on lightweight network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211027648.9A CN115457385A (en) | 2022-08-25 | 2022-08-25 | Building change detection method based on lightweight network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115457385A true CN115457385A (en) | 2022-12-09 |
Family
ID=84297708
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211027648.9A Pending CN115457385A (en) | 2022-08-25 | 2022-08-25 | Building change detection method based on lightweight network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115457385A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116051519A (en) * | 2023-02-02 | 2023-05-02 | 广东国地规划科技股份有限公司 | Method, device, equipment and storage medium for detecting double-time-phase image building change |
-
2022
- 2022-08-25 CN CN202211027648.9A patent/CN115457385A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116051519A (en) * | 2023-02-02 | 2023-05-02 | 广东国地规划科技股份有限公司 | Method, device, equipment and storage medium for detecting double-time-phase image building change |
| CN116051519B (en) * | 2023-02-02 | 2023-08-22 | 广东国地规划科技股份有限公司 | Method, device, equipment and storage medium for detecting double-time-phase image building change |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109190752B (en) | Image semantic segmentation method based on global features and local features of deep learning | |
| CN108647585B (en) | Traffic identifier detection method based on multi-scale circulation attention network | |
| CN111047551B (en) | Remote sensing image change detection method and system based on U-net improved algorithm | |
| CN111582316B (en) | RGB-D significance target detection method | |
| CN110210539B (en) | RGB-T image saliency target detection method based on multi-level depth feature fusion | |
| CN112434721A (en) | Image classification method, system, storage medium and terminal based on small sample learning | |
| CN111325751A (en) | CT image segmentation system based on attention convolution neural network | |
| CN111126202A (en) | Optical remote sensing image target detection method based on void feature pyramid network | |
| CN111950453A (en) | Optional-shape text recognition method based on selective attention mechanism | |
| CN110059728B (en) | RGB-D image visual saliency detection method based on attention model | |
| CN110020658B (en) | Salient object detection method based on multitask deep learning | |
| CN110399840B (en) | Rapid lawn semantic segmentation and boundary detection method | |
| CN111797841B (en) | Visual saliency detection method based on depth residual error network | |
| CN114187520B (en) | Building extraction model construction and application method | |
| CN115035361A (en) | Target detection method and system based on attention mechanism and feature cross fusion | |
| CN110852327A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN113963170A (en) | RGBD image saliency detection method based on interactive feature fusion | |
| CN113936235A (en) | Video saliency target detection method based on quality evaluation | |
| CN115861756A (en) | Earth background small target identification method based on cascade combination network | |
| CN116542991A (en) | Network architecture for fracture image segmentation, training method and segmentation method thereof | |
| CN115908793A (en) | Coding and decoding structure semantic segmentation model based on position attention mechanism | |
| CN115457385A (en) | Building change detection method based on lightweight network | |
| CN114821350A (en) | Multi-stage information fusion high-resolution remote sensing image building change detection method | |
| WO2022222036A1 (en) | Method and apparatus for determining parking space | |
| Zong et al. | A cascaded refined rgb-d salient object detection network based on the attention mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |