CN113657351A - High-resolution remote sensing image forest and grass change detection device and method based on deep learning - Google Patents
High-resolution remote sensing image forest and grass change detection device and method based on deep learning Download PDFInfo
- Publication number
- CN113657351A CN113657351A CN202111140640.9A CN202111140640A CN113657351A CN 113657351 A CN113657351 A CN 113657351A CN 202111140640 A CN202111140640 A CN 202111140640A CN 113657351 A CN113657351 A CN 113657351A
- Authority
- CN
- China
- Prior art keywords
- image
- module
- forest
- grass
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 244000025254 Cannabis sativa Species 0.000 title claims abstract description 85
- 230000008859 change Effects 0.000 title claims abstract description 68
- 238000001514 detection method Methods 0.000 title claims abstract description 34
- 238000013135 deep learning Methods 0.000 title claims abstract description 21
- 238000000034 method Methods 0.000 title abstract description 41
- 230000011218 segmentation Effects 0.000 claims abstract description 45
- 238000004891 communication Methods 0.000 claims abstract description 41
- 238000012937 correction Methods 0.000 claims abstract description 41
- 238000012545 processing Methods 0.000 claims abstract description 39
- 238000003702 image correction Methods 0.000 claims abstract description 14
- 238000010276 construction Methods 0.000 claims abstract description 11
- 238000000605 extraction Methods 0.000 claims description 22
- 238000007781 pre-processing Methods 0.000 claims description 11
- 238000003709 image segmentation Methods 0.000 claims description 6
- 230000004927 fusion Effects 0.000 abstract description 5
- 230000005855 radiation Effects 0.000 description 13
- 238000010586 diagram Methods 0.000 description 11
- 239000000284 extract Substances 0.000 description 10
- 230000008569 process Effects 0.000 description 8
- 238000005070 sampling Methods 0.000 description 8
- 230000002776 aggregation Effects 0.000 description 7
- 238000004220 aggregation Methods 0.000 description 7
- 230000007246 mechanism Effects 0.000 description 7
- 238000006116 polymerization reaction Methods 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 230000015556 catabolic process Effects 0.000 description 3
- 238000006731 degradation reaction Methods 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000005484 gravity Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 206010047571 Visual impairment Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000013213 extrapolation Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000012417 linear regression Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012627 multivariate algorithm Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000000611 regression analysis Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G06T5/80—
-
- G06T5/90—
Abstract
The invention discloses a high-resolution remote sensing image forest and grass change detection device based on deep learning, which comprises: the system comprises an image correction unit, an image drawing unit, a data processing unit, an attention model construction unit, a communication unit and terminal equipment; the image correction unit is used for acquiring a high-resolution remote sensing image of the forest and grass and obtaining a forest and grass correction image; the image drawing unit is used for drawing a label graph; the data processing unit is used for carrying out segmentation processing on the forest and grass corrected image and the label graph; the attention model building unit is used for building a characteristic model through attention learning processing to obtain a change result of the forest and grass; the communication unit is used for transmitting the change result of the forest and grass to the terminal equipment. According to the method, the attention module is adopted, so that the weight of the change region information can be increased during feature dimension fusion, meanwhile, the non-change region feature information is restrained, noise and redundancy in input are ignored, and the anti-noise capability of the model is improved.
Description
Technical Field
The invention relates to the technical field of intelligent interpretation of remote sensing images, in particular to a device and a method for detecting forest and grass changes of high-resolution remote sensing images based on deep learning.
Background
With the continuous progress of the technology of the remote sensing satellite, it is easier to obtain the high-spatial-resolution remote sensing image by earth observation, and the high-resolution remote sensing image gradually becomes important image data for information processing of the remote sensing ground objects because the characteristic information of various textures, colors, spaces and the like of all the objects in the high-resolution remote sensing image is obvious.
Along with the rapid development of urban and rural construction, human activities are gradually aggravated, the speed of change of land utilization types is increased day by day, the amplitude is increased day by day, and how to rapidly and efficiently monitor and update the land utilization types becomes a problem to be solved urgently in the remote sensing subject. The most important link for updating the land use type is change region extraction, namely change detection. At present, the land use change detection mode is mainly manual visual interpretation, which not only consumes a large amount of manpower, but also has long time, poor effect and serious omission. With the increasing of the land change speed, the deepening of the environmental complexity and the increasing of the diversity of remote sensing data, the traditional method cannot well adapt to the requirement of change detection.
In order to solve the above problems, in recent years, research on detecting the change of features of a high-resolution remote sensing image by using a machine learning method is deepened, the method for using the machine learning method generally extracts the spectral, texture and structural features of the remote sensing image according to a feature descriptor set by an expert, and then performs feature classification (the feature classification method includes a supervised classification method based on shallow feature discrimination, such as a maximum likelihood method, a decision tree, a support vector machine and the like), the method generally needs to manually determine a feature value and a kernel function, is difficult to span semantic gap between bottom layer image data and high-level logic information, and is often poor in classification accuracy.
The traditional method based on deep learning (such as a full convolution network, a SegNet network, a U-type network and the like) can automatically learn a feature extraction and feature classification model from data through an end-to-end learning mechanism, so as to adaptively extract and classify the features of the remote sensing image. On the premise of ensuring sufficient labeled training samples, better classification precision can be obtained. However, the deep learning neural network is highly dependent on data learning, cannot fully utilize rich logic information (such as spatial relationship) between ground features to perform self-correction, has poor interpretability of obtaining forest and grass change results (the black box characteristic of the deep learning neural network is still very significant), and needs to be improved in precision.
Disclosure of Invention
Aiming at the problems, the invention provides a method and a device for detecting forest and grass changes of a high-resolution remote sensing image based on deep learning, which are used for solving the technical problems in the prior art, increasing the weight of changed region information during feature dimension fusion by adopting an attention module, inhibiting non-changed region feature information, neglecting noise and redundancy in input and improving the anti-noise capability of a model.
In order to achieve the purpose, the invention provides the following scheme: the invention provides a high-resolution remote sensing image forest and grass change detection device based on deep learning, which comprises: the system comprises an image correction unit, an image drawing unit, a data processing unit, an attention model construction unit, a communication unit and terminal equipment;
The image correction unit, the image drawing unit, the data processing unit, the attention model building unit, the communication unit and the terminal equipment are sequentially connected;
the image correction unit is used for acquiring a high-resolution remote sensing image of the forest and grass and preprocessing the high-resolution remote sensing image to obtain a forest and grass correction image;
the image drawing unit draws a label graph based on the forest and grass correction image;
the data processing unit is used for carrying out segmentation processing on the forest and grass corrected image and the label graph to obtain a segmentation data set;
the attention model building unit is used for performing convolution operation on the segmentation data set, extracting the characteristics of the segmentation data set, building a characteristic model through attention learning processing, and obtaining the change result of the forest and grass based on the characteristic model;
the communication unit is used for transmitting the change result of the forest and grass to the terminal equipment.
Preferably, the image correction unit comprises an acquisition module and a correction module; the acquisition module, the correction module and the image drawing unit are sequentially connected;
the acquisition module is used for acquiring high-resolution remote sensing images of the forest and grass;
and the correction module is used for preprocessing the high-resolution remote sensing image to obtain a forest and grass correction image.
Preferably, the image drawing unit comprises an interpretation module and a delineation module; the correction module, the interpretation module, the delineation module and the data processing unit are connected in sequence;
the interpretation module is used for loading and manually and visually interpreting the forest and grass correction image to obtain an interpreted image;
the drawing module is used for drawing the chart spots of forest and grass changes in the interpreted image to obtain a label chart.
Preferably, the data processing unit comprises a partitioning module and a collecting module; the interpretation module and the drawing module are both connected with the segmentation module; the segmentation module, the collection module and the attention model construction unit are connected in sequence;
the segmentation module is used for segmenting the forest and grass corrected image and the label graph to obtain a plurality of image segmentation graphs;
the collection module is used for collecting all the image segmentation maps to obtain a segmentation data set.
Preferably, the attention model building unit comprises a feature extraction module, an attention module and a model building module; the collection module, the feature extraction module, the attention module, the model construction module and the communication unit are connected in sequence;
The feature extraction module is used for performing convolution operation on the segmentation data set and extracting features of the segmentation data set;
the attention module is used for carrying out attention learning processing on the characteristics of the segmentation data set;
the model building module builds a feature model based on the features subjected to the attention learning processing, and obtains a change result of the forest and grass based on the feature model.
Preferably, the communication unit includes a first communication module and a second communication module; the first communication module is placed in the attention model building unit; the second communication module is arranged in the terminal equipment;
the first communication module is used for transmitting the forest and grass change result;
the second communication module is used for receiving the result of the forest and grass change.
Preferably, the first communication module and the second communication module are wirelessly connected through 2.4 g.
A high-resolution remote sensing image forest and grass change detection method based on deep learning comprises the following steps:
s1, collecting a high-resolution remote sensing image of the forest and grass, and preprocessing the high-resolution remote sensing image to obtain a forest and grass correction image;
s2, drawing a label graph based on the forest and grass corrected image;
S3, carrying out segmentation processing on the forest and grass corrected image and the label graph to obtain a segmentation data set;
s4, performing convolution operation on the segmentation data set, extracting the characteristics of the segmentation data set, constructing a characteristic model through attention learning processing, and acquiring the change result of the forest and grass based on the characteristic model.
The invention discloses the following technical effects:
(1) changing the convolution layer of the model in the up-sampling and down-sampling processes into a polymerization residual convolution layer, and changing single-channel extraction of a feature extraction channel into multi-channel extraction so that the network can fully learn the features of the image; the residual error module is introduced, so that the model can effectively train a deeper network structure, and the problems of model degradation and the like caused by network deepening are prevented; the attention module is adopted to increase the weight of the change region information during feature dimension fusion, inhibit the non-change region feature information, ignore noise and redundancy in input and improve the anti-noise capability of the model.
(2) The invention changes the original jump connection of the U-Net model, introduces the attention mechanism in the jump connection process, the attention mechanism can adjust the weight of each component in the feature diagram, inhibits the learning of the features by reducing the weight of the feature diagram which is irrelevant to the task, and enhances the learning of the features by increasing the weight of the features which are relevant to the task. In a change detection task of a high-resolution remote sensing image, the key point is to extract a change region from a two-phase image, so that the information weight of a change type is increased by introducing an attention mechanism, so that the model gravity learns the change region, meanwhile, the weight of an unchanged type is reduced, the sensitivity of the model to the change type is improved, and a more accurate forest and grass change condition can be obtained.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a block diagram of an apparatus of the present invention;
FIG. 2 is a flow chart of a method of the present invention;
FIG. 3 is a structural diagram of an attention model according to an embodiment of the present invention;
FIG. 4 is an overall flow chart in an embodiment of the invention;
FIG. 5 is a sample region map of before and after images in an embodiment of the present invention; wherein, (a) is the image of the previous stage; (b) is a later stage image map;
FIG. 6 is a diagram of a neural network architecture in an embodiment of the present invention;
FIG. 7 is a key block diagram in an embodiment of the invention;
FIG. 8 is a front-to-back image comparison chart in accordance with an embodiment of the present invention; wherein, (a) is the image of the previous stage; (b) is a later stage image map; (c) a label graph is sketched;
FIG. 9 is a diagram illustrating the detection results of different region models according to an embodiment of the present invention; wherein, (a) is a model detection result graph of the region 1; (b) is a model detection result graph of the region 2; (c) is a model detection result graph of the region 3;
FIG. 10 is a comparison of test results obtained by different methods according to the example of the present invention; wherein, (a) is a detection result graph adopting an FCN method; (b) a detection result graph adopting a SegNet method; (c) is a detection result diagram by adopting a Simunetdiff method; (d) is a detection result diagram adopting a U-net method; (e) is a detection result chart by adopting a ResNeXtATTUnet method.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Referring to fig. 1, the present embodiment provides a device for detecting forest and grass changes in high-resolution remote sensing images based on deep learning, including: the system comprises an image correction unit, an image drawing unit, a data processing unit, an attention model construction unit, a communication unit and terminal equipment; the image correction unit, the image drawing unit, the data processing unit, the attention model building unit, the communication unit and the terminal equipment are sequentially connected.
The image correction unit is used for acquiring a high-resolution remote sensing image of the forest and grass and preprocessing the high-resolution remote sensing image to obtain a forest and grass correction image; the image drawing unit draws a label graph based on the forest and grass corrected image; the data processing unit is used for carrying out segmentation processing on the forest and grass corrected image and the label graph to obtain a segmentation data set; the attention model building unit is used for performing convolution operation on the segmented data set, extracting the characteristics of the segmented data set, building a ResNeXtAttU-Net characteristic model through attention learning processing, and obtaining the change result of the forest grass based on the ResNeXtAttU-Net characteristic model; the communication unit is used for transmitting the change result of the forest and grass to the terminal equipment; and the terminal equipment is used for storing and checking the change result of the forest and grass.
The image correction unit comprises an acquisition module and a correction module; the acquisition module, the correction module and the image drawing unit are sequentially connected; the acquisition module is used for acquiring high-resolution remote sensing images of the forest and grass; the correction module is used for preprocessing the high-resolution remote sensing image to obtain a forest and grass correction image.
The image drawing unit comprises an interpretation module and a delineation module; the correction module, the interpretation module, the delineation module and the data processing unit are connected in sequence; the interpretation module is used for loading and manually and visually interpreting the forest and grass corrected image to obtain an interpreted image; the drawing module is used for drawing the image spots for interpreting forest and grass changes in the image to obtain a label image.
The data processing unit comprises a segmentation module and a collection module; the interpretation module and the drawing module are both connected with the segmentation module; the segmentation module, the collection module and the attention model construction unit are connected in sequence; the segmentation module is used for carrying out segmentation processing on the forest and grass corrected images and the label images to obtain a plurality of image segmentation images; the collection module is used for collecting all the image segmentation maps to obtain a segmentation data set.
The attention model building unit comprises a feature extraction module, an aggregation residual error module and a feature extraction module; the set module, the feature extraction module, the residual error aggregation module, the model construction module and the communication unit are sequentially connected; the characteristic extraction module is used for carrying out convolution operation on the segmentation data set and extracting the characteristics of the segmentation data set; the aggregation residual module is used for carrying out aggregation residual processing on the characteristics of the segmentation data set; the feature extraction module builds a ResNeXtATTU-Net model based on the features subjected to polymerization residual processing, and obtains a forest and grass change result based on the ResNeXtATTU-Net model.
The communication unit comprises a first communication module and a second communication module; the first communication module is arranged in the attention model building unit; the second communication module is arranged in the terminal equipment; the first communication module is used for transmitting the result of forest and grass change; the second communication module is used for receiving the result of the forest and grass change. The first communication module and the second communication module are wirelessly connected through 2.4 g.
Referring to fig. 2 to 10, the embodiment provides a method for detecting forest and grass changes of high-resolution remote sensing images based on deep learning, which includes the following steps:
and S1, collecting the high-resolution remote sensing image of the forest and grass, and preprocessing the high-resolution remote sensing image to obtain a forest and grass correction image.
In the imaging process of the satellite, due to different orbit positions, solar altitude angles and instantaneous field angles of the sensor at different moments, the obtained remote sensing image may have geometric distortion in position and cannot be directly used. Therefore, the first step in the remote sensing image interpretation is to pre-process the acquired image. The preprocessing generally comprises radiation correction, geometric correction, image enhancement and the like, and the influence on the change detection of the remote sensing image is weakened by preprocessing the image to eliminate 'pseudo change' caused by external factors.
The radiation correction is used for correcting or eliminating the phenomenon that when the sensor receives electromagnetic wave radiation energy emitted by an earth surface object, due to the influences of factors such as atmospheric action, illumination conditions and the like, a detection value received by the sensor is inconsistent with the spectral radiance actually emitted by the earth surface object, and the phenomenon of image gray level distortion, namely radiation error, is caused.
The radiation correction is further divided into absolute radiation correction and relative radiation correction. Wherein, the radiation correction adopts relative radiation correction, namely, one phase image in two phases of images is taken as a reference image, the other phase image is taken as an image to be corrected, and a regression analysis method is adopted to establish linear mapping y between the two phases of imagesiThe formula is as follows:
yi＝ki*xi+bi
in the formula, yiThe radiation brightness value x of the pixel of the ith waveband after radiation correction of the image to be corrected in the later periodiThe pixel radiation brightness value k of the image to be corrected in the ith wave bandi、biThe slope and intercept of the linear regression equation of the ith wave band. Selecting pseudo-invariant feature points in the two-phase image by adopting an iterative weighted multivariate algorithm, selecting a threshold value and a weighted value after multiple iterations, calculating the pseudo-invariant feature points by adopting a least square method, and further solving a slope k in the formulaiAnd intercept bi。
The geometric errors of the image are caused by a series of factors such as the height of a sensor platform, the curvature of the earth, the change of air refraction, the change of terrain and the like. The geometric correction is a geometric deformation error generated by the characteristics of geometric position, shape size, space position and the like of the same ground object when a certain type of information in the two-phase images is projected to a reference system in a specified image. The method utilizes the control points to carry out geometric correction on the image, the number of the control points is related to the times of using a polynomial model for geometric correction, for an nth-order polynomial, (n +1) × (n +2)/2 control points are at least needed theoretically, and in the process of actually selecting the control points, the number of the control points is at least larger than the lowest theoretical value. The selection of control points mainly follows the following principles: (1) the control point should select characteristic points which are easy to distinguish, permanent and fine in images, such as house corners, road intersections, airports and the like; (2) the area with large characteristic change on the image should select some control points; (3) control points are selected in the image edge area to avoid the corrected image extrapolation; (4) the selection of control points should be evenly distributed over the image. In the embodiment of the present application, the first-stage image is a reference image, the second-stage image is an image to be corrected, and the geometric correction is performed by using a second-order polynomial model, which has the following formula:
In the formula, xl、ylIs pixel coordinate, x, of the corrected later-stage image1And y1Is the pixel coordinate of the previous image.
And
the coefficients of the second-order polynomial correction model are obtained by the least square method through artificially selected control points, wherein i is 0, 1, 2, 3, 4 and 5.
There are many methods for image enhancement in remote sensing images, such as color enhancement, radiation enhancement, etc., and the final results obtained by different methods are different. The embodiment of the application mainly aims at performing gray level stretching on the acquired two-phase image. The gray scale stretching is a simple and efficient linear image enhancement method. The piecewise linear gray stretching can restrain a low-frequency part in an image, improve the contrast and brightness of the high-frequency part, improve the visual effect of the image more obviously, and extract more useful information for a current task from the image when the image is visually interpreted. 2% linear stretching is carried out on two images adopted in the experiment, namely the pixel gray value of the image gray value between 2% and 98% is linearly stretched, the gray values smaller than 2% and larger than 98% are set as 0, so that part of abnormal values can be abandoned, and the pixel values in the residual range are stretched again to the gray value range of 0-255, and the formula is as follows:
In the formula, g (x, y) represents the processed image, f (x, y) represents the input image, and V represents the image pixel gradation value.
And S2, drawing a label graph based on the forest and grass corrected image.
And respectively loading the two-stage images by using remote sensing image processing software, and drawing the pattern spots of forest and grass changes in the two-stage images in a mode of manual visual interpretation to serve as training samples of the model.
And S3, carrying out segmentation processing on the forest and grass corrected image and the label graph to obtain a segmentation data set.
And (4) dividing the processed two-stage images obtained in the S1 and the S2 and the label graph which is interpreted and sketched by human vision according to the same size, and dividing the divided images and the label graph into a training sample and a verification sample according to a certain proportion.
S4, performing convolution operation on the segmented data set, extracting the characteristics of the segmented data set, constructing a ResNeXtAttU-Net characteristic model through attention and mechanics learning processing, and acquiring the change result of the forest and grass based on the ResNeXt AttU-Net characteristic model.
Firstly, a convolution operation in a deep learning framework pytore is called to extract features of an input image, and one convolution kernel can only extract one feature map and can not extract all different features of the whole image, so each convolution layer extracts different types of features by a plurality of different convolution kernels, wherein the low-layer convolution layer mainly extracts shallow features of the image, such as information of boundaries, contours and the like, and the high-layer convolution layer extracts high-level features of the image, such as geometric relations, spatial relations and the like of the image by superposing and integrating the information extracted by the low-layer convolution. Second, a pooling operation in the pytorech is invoked. Because the feature map of the input image after passing through the convolution layer has high dimensionality and contains some unimportant high-frequency information, if the high-dimensional feature maps are directly input into the next convolution layer, the calculated amount of the model is increased, the dimensionality is overhigh, and the phenomenon of overfitting occurs. Therefore, a method for performing aggregation processing on the convolved feature maps, that is, describing a large-area region by using a feature with a small dimension, is needed, and the method can reduce the dimension of the feature maps, well retain the main features of the feature maps, effectively reduce the number of parameters, and prevent the occurrence of an overfitting phenomenon. However, the convolution operation only performs linear transformation on the input image, and no matter how many hidden layers are overlapped in the neural network, the output result is a combination of linear transformation, and only a simple mapping relation can be expressed. When the method faces complex task scenes such as remote sensing images, the model expression capability of linear transformation is insufficient, and the generalization capability is very limited. Therefore, in order to improve the expression capability and generalization capability of the model, it is necessary to introduce an activation function to map the linear features extracted by the convolutional layer into nonlinear features, so as to enhance the generalization capability of the model.
Aiming at the phenomena of missing detection, error detection and the like of the U-Net model in the change detection of the high-resolution image, the invention improves the U-Net model. In the up-down sampling stage of the original U-Net model, a polymerization residual module is introduced, and the accuracy of the model is improved by deepening or widening the network in the traditional method, but the difficulty of network design and the calculation cost are increased along with the increase of the number of hyper-parameters. The ResNeXt structure can improve the accuracy rate on the premise of not increasing the complexity of parameters, and simultaneously reduces the number of the hyper-parameters. When ResNeXt processes feature graphs with the same size, convolution kernels with the same size and number are adopted, when the resolution length and width of the feature graphs are reduced by two times, the number of feature channels is doubled, each branch in the block adopts the same structure, the increase of the number of the feature channels can fully extract feature information of images, the feature extraction time is saved, and the detection precision is improved (as shown in figure 3). The attention module is added at the jump connection layer, as the low-level feature map contains more position information and the high-level feature map contains rich category information, the module utilizes semantic information in the high-level feature map to strengthen the feature weight of forest and grass change areas in the low-level feature map, so that more detail information is added to the low-level feature map, the learning capacity of the model to be divided is enhanced, the dividing precision of the model is improved, in addition, the attention weight can play a selection role in the low-level feature map, and the low-level feature map can have more accurate position information while having rich category information.
The ResNeXtATTU-Net model is similar to the U-Net model in structure and is composed of an encoder, a decoder and a jump connection part. In the encoder part, a ResNeXtAttU-Net model consists of 4 convolutional layers and down-sampling layers, wherein the convolutional layers are partially added with the aggregation residual error module, the residual error module can solve the problem of model degradation caused by deepening of the number of network layers, the aggregation residual error of an improved version of the traditional residual error can reduce the network iteration times and the hyper-parameters, and meanwhile, the multi-channel block convolution extracts the characteristic information of an image, so that the characteristic calculation time can be reduced, the network training time is saved, and the change detection precision is improved; a jump connection of a merging attention module is adopted between each layer of encoder and decoder; the method has the advantages that low-dimensional and high-dimensional fusion is carried out on the deep-layer characteristic graph and the shallow-layer characteristic graph in a jumping connection mode, more dimensional position information of the image is reserved, and accordingly the problem of image edge detail blurring is solved.
Aiming at the problems of ground feature detail loss, under-segmentation and the like of a U-Net model in high-resolution remote sensing image change detection, a polymerization residual convolution module is introduced in the up-sampling and down-sampling stages of the U-Net model, so that the single-channel extraction is changed into multi-channel extraction of the characteristics, and the method also comprises a quick structure for directly performing identity mapping on input, so that the problem of model prediction accuracy reduction caused by gradient disappearance is avoided; the attention module is introduced into jump connection of the U-Net model, combines a feature map obtained from an encoder and a feature map obtained from a decoder through a jump connection layer, equivalently combines extracted low-dimensional features and extracted high-dimensional features through a plurality of convolution layers, enables the feature maps to contain more position information and category information, and meanwhile can have higher resolution, and finally obtains output through convolution and a classifier.
The invention discloses the following technical effects:
(1) changing the convolution layer of the model in the up-sampling and down-sampling processes into a polymerization residual convolution layer, and changing single-channel extraction of a feature extraction channel into multi-channel extraction so that the network can fully learn the features of the image; the residual error module is introduced, so that the model can effectively train a deeper network structure, and the problems of model degradation and the like caused by network deepening are prevented; and the attention module is adopted to increase the weight of the change region information during feature dimension fusion, inhibit the non-change region feature information, ignore the noise and redundancy in the input and improve the anti-noise capability of the model.
(2) The invention changes the original jump connection of the U-Net model, introduces the attention mechanism in the jump connection process, the attention mechanism can adjust the weight of each component in the feature diagram, inhibits the learning of the features by reducing the weight of the feature diagram which is irrelevant to the task, and enhances the learning of the features by increasing the weight of the features which are relevant to the task. In a change detection task of a high-resolution remote sensing image, the key point is to extract a change region from a two-phase image, so that the information weight of a change type is increased by introducing an attention mechanism, so that the model gravity learns the change region, meanwhile, the weight of an unchanged type is reduced, the sensitivity of the model to the change type is improved, and a more accurate forest and grass change condition can be obtained.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the present invention in its spirit and scope. Are intended to be covered by the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (8)
1. High branch remote sensing image forest and grass change detection device based on degree of deep learning, its characterized in that includes: the system comprises an image correction unit, an image drawing unit, a data processing unit, an attention model construction unit, a communication unit and terminal equipment;
the image correction unit, the image drawing unit, the data processing unit, the attention model building unit, the communication unit and the terminal equipment are sequentially connected;
the image correction unit is used for acquiring a high-resolution remote sensing image of the forest and grass and preprocessing the high-resolution remote sensing image to obtain a forest and grass correction image;
the image drawing unit draws a label graph based on the forest and grass correction image;
the data processing unit is used for carrying out segmentation processing on the forest and grass corrected image and the label graph to obtain a segmentation data set;
the attention model building unit is used for performing convolution operation on the segmentation data set, extracting the characteristics of the segmentation data set, building a characteristic model through attention learning processing, and obtaining the change result of the forest and grass based on the characteristic model;
the communication unit is used for transmitting the change result of the forest and grass to the terminal equipment.
2. The forest and grass change detection device based on the deep learning high-resolution remote sensing image is characterized in that the image correction unit comprises an acquisition module and a correction module; the acquisition module, the correction module and the image drawing unit are sequentially connected;
the acquisition module is used for acquiring high-resolution remote sensing images of the forest and grass;
and the correction module is used for preprocessing the high-resolution remote sensing image to obtain a forest and grass correction image.
3. The forest and grass change detection device based on the deep learning high-resolution remote sensing image is characterized in that the image drawing unit comprises an interpretation module and a delineation module; the correction module, the interpretation module, the delineation module and the data processing unit are connected in sequence;
the interpretation module is used for loading and manually and visually interpreting the forest and grass correction image to obtain an interpreted image;
the drawing module is used for drawing the chart spots of forest and grass changes in the interpreted image to obtain a label chart.
4. The forest and grass change detection device based on the deep learning high-resolution remote sensing image is characterized in that the data processing unit comprises a segmentation module and a combination module; the interpretation module and the drawing module are both connected with the segmentation module; the segmentation module, the collection module and the attention model construction unit are connected in sequence;
The segmentation module is used for segmenting the forest and grass corrected image and the label graph to obtain a plurality of image segmentation graphs;
the collection module is used for collecting all the image segmentation maps to obtain a segmentation data set.
5. The forest and grass change detection device based on the deep learning high-resolution remote sensing image is characterized in that the attention model building unit comprises a feature extraction module, an attention module and a model building module; the collection module, the feature extraction module, the attention module, the model construction module and the communication unit are connected in sequence;
the feature extraction module is used for performing convolution operation on the segmentation data set and extracting features of the segmentation data set;
the attention module is used for carrying out attention learning processing on the characteristics of the segmentation data set;
the model building module builds a feature model based on the features subjected to the attention learning processing, and obtains a change result of the forest and grass based on the feature model.
6. The forest and grass change detection device based on the deep learning high-resolution remote sensing image is characterized in that the communication unit comprises a first communication module and a second communication module; the first communication module is placed in the attention model building unit; the second communication module is arranged in the terminal equipment;
The first communication module is used for transmitting the forest and grass change result;
the second communication module is used for receiving the result of the forest and grass change.
7. The device for detecting forest and grass changes based on deep learning remote sensing images of claim 6, wherein the first communication module is in wireless connection with the second communication module through 2.4 g.
8. A high-resolution remote sensing image forest and grass change detection method based on deep learning is characterized by comprising the following steps:
s1, collecting a high-resolution remote sensing image of the forest and grass, and preprocessing the high-resolution remote sensing image to obtain a forest and grass correction image;
s2, drawing a label graph based on the forest and grass corrected image;
s3, carrying out segmentation processing on the forest and grass corrected image and the label graph to obtain a segmentation data set;
s4, performing convolution operation on the segmentation data set, extracting the characteristics of the segmentation data set, constructing a characteristic model through attention learning processing, and acquiring the change result of the forest and grass based on the characteristic model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111140640.9A CN113657351A (en) | 2021-09-28 | 2021-09-28 | High-resolution remote sensing image forest and grass change detection device and method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111140640.9A CN113657351A (en) | 2021-09-28 | 2021-09-28 | High-resolution remote sensing image forest and grass change detection device and method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113657351A true CN113657351A (en) | 2021-11-16 |
Family
ID=78494479
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111140640.9A Pending CN113657351A (en) | 2021-09-28 | 2021-09-28 | High-resolution remote sensing image forest and grass change detection device and method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113657351A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114937211A (en) * | 2022-07-25 | 2022-08-23 | 北京数慧时空信息技术有限公司 | Remote sensing image change detection method based on invariant object |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110705457A (en) * | 2019-09-29 | 2020-01-17 | 核工业北京地质研究院 | Remote sensing image building change detection method |
| CN112949549A (en) * | 2021-03-19 | 2021-06-11 | 中山大学 | Super-resolution-based change detection method for multi-resolution remote sensing image |
-
2021
- 2021-09-28 CN CN202111140640.9A patent/CN113657351A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110705457A (en) * | 2019-09-29 | 2020-01-17 | 核工业北京地质研究院 | Remote sensing image building change detection method |
| CN112949549A (en) * | 2021-03-19 | 2021-06-11 | 中山大学 | Super-resolution-based change detection method for multi-resolution remote sensing image |
Non-Patent Citations (2)
| Title |
|---|
| 徐风等: "基于卷积注意力模块的端到端遥感图像分类", 林业工程学报, no. 4 * |
| 李道纪等: "遥感影像地物分类多注意力融和U型网络法", 测绘学报, no. 8 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114937211A (en) * | 2022-07-25 | 2022-08-23 | 北京数慧时空信息技术有限公司 | Remote sensing image change detection method based on invariant object |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109934200B (en) | RGB color remote sensing image cloud detection method and system based on improved M-Net | |
| CN108573276B (en) | Change detection method based on high-resolution remote sensing image | |
| CN109102469B (en) | Remote sensing image panchromatic sharpening method based on convolutional neural network | |
| CN107392130A (en) | Classification of Multispectral Images method based on threshold adaptive and convolutional neural networks | |
| CN107967474A (en) | A kind of sea-surface target conspicuousness detection method based on convolutional neural networks | |
| CN115497005A (en) | YOLOV4 remote sensing target detection method integrating feature transfer and attention mechanism | |
| CN114187450A (en) | Remote sensing image semantic segmentation method based on deep learning | |
| CN113705538A (en) | High-resolution remote sensing image road change detection device and method based on deep learning | |
| CN114494821B (en) | Remote sensing image cloud detection method based on feature multi-scale perception and self-adaptive aggregation | |
| CN105405138A (en) | Water surface target tracking method based on saliency detection | |
| CN112001293A (en) | Remote sensing image ground object classification method combining multi-scale information and coding and decoding network | |
| CN112560717A (en) | Deep learning-based lane line detection method | |
| CN113569810A (en) | Remote sensing image building change detection system and method based on deep learning | |
| CN112733736A (en) | Class imbalance hyperspectral image classification method based on enhanced oversampling | |
| CN112950780A (en) | Intelligent network map generation method and system based on remote sensing image | |
| CN115457396A (en) | Surface target ground object detection method based on remote sensing image | |
| CN115861823A (en) | Remote sensing change detection method and device based on self-supervision deep learning | |
| CN106407975B (en) | Multiple dimensioned layering object detection method based on space-optical spectrum structural constraint | |
| CN113657351A (en) | High-resolution remote sensing image forest and grass change detection device and method based on deep learning | |
| CN112818920B (en) | Double-temporal hyperspectral image space spectrum joint change detection method | |
| CN114120036A (en) | Lightweight remote sensing image cloud detection method | |
| CN113743373A (en) | High-resolution remote sensing image cropland change detection device and method based on deep learning | |
| CN113378672A (en) | Multi-target detection method for defects of power transmission line based on improved YOLOv3 | |
| CN110084203B (en) | Full convolution network airplane level detection method based on context correlation | |
| CN112613354A (en) | Heterogeneous remote sensing image change detection method based on sparse noise reduction self-encoder |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |