CN115937599A - Deep learning remote sensing image classification method for improving self-adaptive pooling - Google Patents
Deep learning remote sensing image classification method for improving self-adaptive pooling Download PDFInfo
- Publication number
- CN115937599A CN115937599A CN202211669005.4A CN202211669005A CN115937599A CN 115937599 A CN115937599 A CN 115937599A CN 202211669005 A CN202211669005 A CN 202211669005A CN 115937599 A CN115937599 A CN 115937599A
- Authority
- CN
- China
- Prior art keywords
- remote sensing
- image
- sensing image
- pooling
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an improved self-adaptive pooling deep learning remote sensing image classification method, which comprises the following steps: collecting remote sensing image data; preparing an image data sample; inputting remote sensing image characteristics and constructing an improved self-adaptive pooling layer; constructing a remote sensing image classification network based on the self-adaptive pooling layer; training a network to obtain a trained model; and testing and evaluating image classification by using the trained model. According to the method, the improved self-adaptive pooling layer is formed by improving the pooling layer, so that the key features of image ground objects can be kept as much as possible while the receptive field is increased, the integration and extraction capability of the pooling layer on the key features on the large-amplitude remote sensing image is improved, the influence of the unbalance problem of different ground classes on the classification precision in the remote sensing image classification process is effectively relieved, and the classification precision and the model mobility are improved.
Description
Technical Field
The invention belongs to the technical field of digital image processing, and particularly relates to a deep learning remote sensing image classification method for improving self-adaptive pooling.
Background
The remote sensing image classification is to use a computer to analyze the spectral information and the spatial information of various ground objects in the remote sensing image and select effective characteristics, divide each pixel in the image into different categories according to a certain rule or algorithm, and then obtain the corresponding information of the actual ground objects in the remote sensing image, thereby realizing the classification of the pixels on the image. As an important component of the remote sensing image digital processing technology, the remote sensing image classification is a basic technology of tasks such as remote sensing image interpretation, target identification, anomaly detection, change detection and the like, and has great significance in the fields of agricultural and forestry remote sensing surveying and mapping, disaster prevention and reduction, city planning and supervision and the like.
The traditional remote sensing image classification method mainly adopts a manual design characteristic or an unsupervised clustering mode to obtain the category of the final pixel. The method for artificially designing the features mainly aims at the characteristics of the remote sensing image land types, designs feature description methods of different land features according to analysis of spectral features, land feature distribution conditions and the like among different land types, and determines classification results according to information of pixels on different positions of the image to be classified. As a demand-oriented method, a method of manually designing feature classification needs to be designed for an image to be classified. Due to the difference of different image sensors, the land features have obvious difference, so that the method for manually designing the features has poor mobility. The other classification method is an unsupervised clustering method, which is a classification method for classifying and merging by a computer according to the similarity of spectral features between pixels under the condition of no prior sample, namely the class features of ground objects in an image are not known in advance, such as a K-means classification method, an ISODATA algorithm and the like. The unsupervised clustering method does not need prior knowledge, and can automatically divide different land categories through a computer, so that the unsupervised clustering method is convenient and quick and has strong operability. However, under the conditions of complex land types, image conditions and the like, the unsupervised clustering method has certain accuracy limitation due to the situations of 'same-object different-spectrum', 'same-spectrum foreign matter' and the like of the remote sensing images.
With the development of deep learning and convolutional neural networks, more and more remote sensing image classification methods adopt a deep learning-based method. In 2015, with the introduction of a full convolution neural network (FCN), pixel-level image classification techniques have made breakthrough progress. The method restores the small-scale feature image to the original image size through the up-sampling operation on the basis of the common convolutional neural network, and effectively retains the position information of each pixel while extracting the depth feature. However, the remote sensing image has a large image size and a small image proportion occupied by ground objects, so that the full convolution network widely applied to natural images is difficult to be directly applied to remote sensing image classification. In order to obtain more effective depth features and obtain more accurate classification results, many researchers have further improved on the basis of the FCN network, such as a deconvolution network, a jump network, a hole convolution network, and the like. The representative of the deconvolution network is a SegNet network, the SegNet network is combined with multi-core convolution, the original classification network is improved from the perspective of multi-source characteristics, and a more effective classification result is obtained through classification of the multi-source characteristics. The jump network is additionally provided with jump connection on the basis of the original full convolution network, effectively combines feature maps of different levels, and effectively relieves the problem that the precision of classification results in remote sensing images is influenced due to weakening of position information caused by multi-level convolution by combining the feature maps of different levels. The space convolution network uses the cavity convolution operation on the basis of the common convolution neural network, namely, the space between the acting pixels in the convolution kernel is increased, the experience field of the convolution is effectively increased on the basis of not increasing parameters, and the problem of low precision caused by too small experience field is solved. Besides the deconvolution network, the jump network and the hole convolution network, some researchers add multi-scale processing, conditional random fields and the like on the basis of common networks, and certain achievements are obtained.
However, the basic convolutional neural network is a general image classification network, and although a great amount of experimental verification and results are obtained on natural scene images, due to the problems that the remote sensing images are large in image size, the image proportion of the ground class is small, obvious class imbalance exists and the like, the conventional method directly applies the natural scene image classification method to remote sensing image classification, and further research is still needed on how to improve the convolutional neural network to be better applied to remote sensing image classification.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides the deep learning remote sensing image classification method for improving the self-adaptive pooling, which can reserve the key features of the image ground objects while performing the network pooling of the remote sensing image classification based on the deep learning, enlarge the receptive field of the image and improve the feature extraction capability and the classification precision of the network.
In order to achieve the purpose of the invention, the invention adopts the technical scheme that:
the method for classifying the deep learning remote sensing images with the improved self-adaptive pooling comprises the following steps:
s1, collecting remote sensing image data, including collecting and correcting an original satellite image;
s2, preparing an image data sample, wherein the image data sample comprises the steps of cutting the obtained image and manufacturing an artificial marking sample for deep learning network training;
s3, inputting the characteristics of the remote sensing image and constructing an improved self-adaptive pooling layer, wherein the specific method comprises the following substeps:
s3-1, setting a size parameter r of a pooling sampling window; assuming that the length, the width and the channel number of the input features are H, W and C respectively, and selecting appropriate pooling sampling parameters according to the size of the remote sensing image and the depth of the feature layer where the remote sensing image is located;
s3-2, constructing a four-dimensional pooling block structure based on the original three-dimensional characteristics of length, width and channel number according to a size parameter r, wherein the size is (H/r, W/r, phi), and P = r 2 ；
S3-3, performing convolution on the pooling block through the conventional convolution layer, and outputting the feature after convolution as an output result of the self-adaptive pooling layer;
s3-4, performing dimensionality compression on the feature with the size of (H/r, W/r,1,) output in the S3-3 to obtain a new feature size of (H/r, W/r,);
s4, selecting an applicable remote sensing image classification network structure based on the convolutional neural network according to the image characteristics and the application types, replacing the pooling layer in the network with a creative improved self-adaptive pooling layer, and constructing a remote sensing image classification network based on the self-adaptive pooling layer;
s5, inputting a group of remote sensing images and classification truth value labels to the network constructed in the S4, calculating a loss value of the training network, reversely returning the network, and continuously inputting the next group of remote sensing images until iteration is completed to obtain a trained model;
and S6, inputting the image to be detected by using the trained model, outputting an image classification result, recording and storing the result and evaluating.
Further, the method of step S2 specifically includes the following sub-steps:
s2-1, expanding the image by rotating and translating the remote sensing satellite image obtained in the S1 to obtain a new remote sensing image and form double image data volume;
s2-2, cutting the size of the original remote sensing image to obtain an image with a fixed size as network input;
and S2-3, marking each position pixel category in the image as a sample label in a manual visual interpretation mode.
Further, step S3-2 includes:
partitioning the features in the length direction and the width direction according to the r size respectively, and setting feature values on coordinate positions (i, j) on an mth layer channel as follows: f. of i,j And the coordinate positions in the length and width directions of the partitioned blocks, namely the 1 st and 2 nd dimensions of the four-dimensional pooling blocks, are as follows: (i '= ceil (i/r), j' = ceil (j/r)); wherein the ceil () function represents the rounding up process on the division result; the formula for calculating the coordinates in the 3 rd dimension of the four-dimensional pooling block is as follows:
p＝(mod(i/r)-1)×r+mod(j/r)
and assigning the characteristic value of the original characteristic coordinate position (i, j, m) to the new coordinate position (i ', j', p, m), thus obtaining the four-dimensional pooling block characteristic.
Further, step S3-3 includes:
setting convolution kernels (k, k,1 and C) for the four-dimensional pooling block to perform convolution, wherein k represents the size of the convolution kernels, and different sizes are set according to actual conditions; during convolution, padding sets the same mode, and the feature points are supplemented adaptively around the feature so that the output feature is the same as the input image in length and width.
The invention has the beneficial effects that:
1. through the improvement to the pooling layer, form modified self-adaptation pooling layer, can remain the key feature of image ground thing as far as possible in the increase receptive field, promote pooling layer to the integration and the extraction ability of key feature on the remote sensing image by a wide margin, then effectively alleviate the influence of the unbalanced problem of different ground classes in the remote sensing image classification process to classification precision, promote the mobility of classification precision and model then.
2. In the remote sensing image classification network based on deep learning, different remote sensing image classification networks can be converted into the remote sensing image classification network based on the self-adaptive pooling layer by replacing the pooling layer, so that the image feature extraction is convenient and efficient, the receptive field of the features is increased, the context information among pixels in the image is better utilized, and the precision of the classification result is improved.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a flow chart of steps of constructing an improved adaptive pooling layer and constructing a remote sensing image classification network based on the adaptive pooling layer in the method of the present invention.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined in the appended claims, and all matters produced by the invention using the inventive concept are protected.
Examples
Referring to fig. 1-2, a method provided by an embodiment of the invention includes the following steps:
step 1, remote sensing image data acquisition.
The data required to be collected by the invention is mainly optical remote sensing image data collected from a remote sensing satellite. For the acquired original satellite image, conventional satellite image processing operations of the satellite image such as orthorectification, radiation correction and the like are required. The method adopts two image correction methods of orthorectification and radiation correction, and ensures that the processed remote sensing image can be used for deep learning sample training.
And 2, preparing an image data sample.
And the obtained remote sensing image data needs to be cut, so that the subsequent sample training based on deep learning is facilitated. Meanwhile, sample labels need to be marked manually, and preparation is provided for convolutional neural network training in subsequent steps. The specific process is as follows:
the first step is as follows: the remote sensing satellite image obtained in the step 1 can be expanded to a certain extent in a rotating mode, a translating mode and the like during specific implementation. The embodiment mainly adopts a rotation mode, and obtains a new remote sensing image by rotating the original remote sensing image by 180 degrees to form doubled image data volume.
The second step is that: and cutting the original size to obtain an image with a fixed size as network input. In the embodiment, the image is uniformly cut into 1024 × 1024 pixels, and the overlapping degree between adjacent images in the cutting process is set to be 0.
The third step: and (5) manufacturing a sample label. And marking each position pixel category in the image as a sample label in a manner of manual visual interpretation. In the embodiment, three categories of agricultural land, construction land and unused land are set, and are respectively marked as 0, 1 and 2 to serve as category labels.
And 3, constructing an improved self-adaptive pooling layer. The specific process is as follows:
in the first step, the size parameter r of the pooled sampling window is set. And (3) assuming that the length, the width and the channel number of the input features are H, W and C respectively, and selecting proper pooling sampling parameters according to the size of the remote sensing image and the depth of the feature layer where the remote sensing image is located. The remote sensing image used in the example sample was 1024 × 1024 in length and width, and all the improved adaptive pooling layer sampling parameters were set to: r =2, according to the characteristics of the constructed adaptive pooling layer, after the characteristics pass through one layer of adaptive pooling layer, the length and width of the characteristics are respectively reduced to 1/r =1/2 of the original length and width, and the number of channels is kept unchanged.
And secondly, constructing a four-dimensional pooling block structure on the basis of the original three-dimensional characteristics of length, width and channel number according to the size parameters. Inputting the characteristic size (H, W, C), firstly constructing a four-dimensional structure according to the size parameters set in the first step, wherein the size is (H/r, W/r, P, C). Wherein, P = r 2 。
Partitioning the features in the length and width directions according to the r size, and setting the features on the coordinate positions (i, j) of the m-th layer channelThe values are: f. of i,j The coordinate positions in the length and width directions (i.e. the 1 st and 2 nd dimensions of the four-dimensional pooling block) of the partitioned blocks are as follows: (i '= ceil (i/r), j' = ceil (j/r)). Wherein the ceil () function represents the rounding up process on the division result. The formula for calculating the coordinates in the 3 rd dimension of the four-dimensional pooling block is as follows:
p＝(mod(i/r)-1)×r+mod(j/r)
and assigning the characteristic value of the original characteristic coordinate position (i, j, m) to the new coordinate position (i ', i', p, m), thus obtaining the four-dimensional pooling block characteristic. The characteristics can effectively retain all information of the original characteristics, reduce the size of the characteristics in the length direction and the width direction and lay a foundation for self-adaptive pooling.
And thirdly, convolving the pooling blocks through a conventional convolution layer, and outputting the convolved characteristics as an output result of the adaptive pooling layer. The core of the step is to effectively utilize the information on the four-dimensional pooling block formed in the second step and generate the pooling layer output characteristics retaining the key characteristics of the image ground features on a large receptive field scale through convolution operation.
For the four-dimensional pooling block, convolution kernels (k, k,1, C) are set for convolution, wherein k represents the size of the convolution kernels, and different sizes can be set according to actual conditions. Meanwhile, in the convolution process, padding sets a same mode, and the feature points are supplemented around the features in a self-adaptive mode, so that the output features are the same as the input image in length and width.
In the embodiment, k =3 is set, and the receptive field of the network is further increased by setting a convolution kernel with a size larger than 1 on the basis of the previous step.
And fourthly, compressing the characteristic dimension. And performing dimension compression on the features with the sizes of (/ r,/r,1,) output by the third step to obtain new features with the sizes of (/ r,/r,).
And 4, selecting an applicable remote sensing image classification network structure based on the convolutional neural network according to the image characteristics and the application type, replacing the pooling layer in the network with a creative improved self-adaptive pooling layer, and constructing the remote sensing image classification network based on the self-adaptive pooling layer.
In the embodiment, a 16-layer VGG feature extraction network is selected as a remote sensing image classification feature extraction network, and a pooling layer in an original network is replaced by a creative improved self-adaptive pooling layer, so that the receptive field of the network can be enlarged, and key features of image ground objects can be more effectively reserved for large-size remote sensing images. And the extracted features continuously enlarge the size through a full connection layer to form an end-to-end network structure, finally outputting the probability value of each category at each position according to the classified category number n, and selecting the category number with the maximum probability as the prediction category of the pixel position. The whole network input is a remote sensing image, and the output is a classification result prediction value with the same length and width.
And 5, inputting a group of remote sensing images and classification truth value labels to the network constructed in the step 4, calculating a loss value of the training network, carrying out reverse return of the network, and continuously inputting the next group of remote sensing images until iteration is completed to obtain a trained model.
And 6, inputting the image to be detected by using the trained model, outputting an image classification result, recording and storing the result and evaluating.
According to the method, the improved self-adaptive pooling layer is formed by improving the pooling layer, so that the key features of image ground objects can be kept as much as possible while the receptive field is increased, the integration and extraction capability of the pooling layer on the key features on the large-amplitude remote sensing image is improved, the influence of the unbalance problem of different ground classes on the classification precision in the remote sensing image classification process is effectively relieved, and the classification precision and the model mobility are improved.
In the remote sensing image classification network based on deep learning, different remote sensing image classification networks can be converted into the remote sensing image classification network based on the self-adaptive pooling layer by replacing the pooling layer, so that the image feature extraction is convenient and efficient, the receptive field of the features is increased, the context information among pixels in the image is better utilized, and the precision of the classification result is improved.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.
Claims (4)
1. A deep learning remote sensing image classification method for improving self-adaptive pooling is characterized by comprising the following steps:
s1, collecting remote sensing image data, including collecting and correcting an original satellite image;
s2, preparing an image data sample, wherein the image data sample comprises the steps of cutting the obtained image and manufacturing an artificial marking sample for deep learning network training;
s3, inputting the characteristics of the remote sensing image and constructing an improved self-adaptive pooling layer, wherein the specific method comprises the following substeps:
s3-1, setting a size parameter r of a pooling sampling window; supposing that the length, the width and the channel number of the input features are H, W and C respectively, and selecting proper pooling sampling parameters according to the size of the remote sensing image and the depth of the feature layer where the remote sensing image is located;
s3-2, constructing a four-dimensional pooling block structure based on the original three-dimensional characteristics of length, width and channel number according to a size parameter r, wherein the size is (H/r, W/r, P, C), and P = r 2 ；
S3-3, performing convolution on the pooling block through the conventional convolution layer, and outputting the feature after convolution as an output result of the self-adaptive pooling layer;
s3-4, performing dimensionality compression on the features with the size of (H/r, W/r,1, C) output in the S3-3 to obtain new feature sizes of (H/r, W/r, C);
s4, selecting an applicable remote sensing image classification network structure based on the convolutional neural network according to image characteristics and application types, replacing a pooling layer in the network with a creative improved self-adaptive pooling layer, and constructing a remote sensing image classification network based on the self-adaptive pooling layer;
s5, inputting a group of remote sensing images and classification truth value labels to the network constructed in the step S4, calculating a loss value of the training network, carrying out reverse return of the network, and continuously inputting the next group of remote sensing images until iteration is completed to obtain a trained model;
and S6, inputting the image to be detected by using the trained model, outputting an image classification result, recording and storing the result and evaluating.
2. The method for classifying the deep learning remote sensing images based on the improved adaptive pooling as claimed in claim 1, wherein the method of step S2 specifically comprises the following substeps:
s2-1, expanding the image by rotating and translating the remote sensing satellite image obtained in the S1 to obtain a new remote sensing image and form double image data volume;
s2-2, cutting the size of the original remote sensing image to obtain an image with a fixed size as network input;
and S2-3, marking each position pixel category in the image as a sample label in a manual visual interpretation mode.
3. The improved self-adaptive pooling deep learning remote sensing image classification method according to claim 1, wherein the step S3-2 comprises:
partitioning the features in the length direction and the width direction according to the r size respectively, and setting feature values on coordinate positions (i, j) on an mth layer channel as follows: f. of i，j And the coordinate positions in the length and width directions of the partitioned blocks, namely the 1 st and 2 nd dimensions of the four-dimensional pooling blocks, are as follows: (i '= ceil (i/r), j'= ceil (j/r)); wherein the ceil () function represents the rounding up process on the division result; the formula for calculating the coordinates in the 3 rd dimension of the four-dimensional pooling block is as follows:
p＝(mod(i/r)-1)×r+mod(j/r)
and assigning the characteristic value of the original characteristic coordinate position (i, j, m) to the new coordinate position (i ', j', p, m), thus obtaining the four-dimensional pooling block characteristic.
4. The improved self-adaptive pooling deep learning remote sensing image classification method according to claim 1, wherein the step S3-3 comprises:
setting convolution kernels (k, k,1 and C) for the four-dimensional pooling block to perform convolution, wherein k represents the size of the convolution kernels, and different sizes are set according to actual conditions; in the convolution process, padding sets a same mode, and feature points are supplemented around the features in an adaptive mode, so that the output features are the same as the input image in length and width.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211669005.4A CN115937599A (en) | 2022-12-23 | 2022-12-23 | Deep learning remote sensing image classification method for improving self-adaptive pooling |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211669005.4A CN115937599A (en) | 2022-12-23 | 2022-12-23 | Deep learning remote sensing image classification method for improving self-adaptive pooling |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115937599A true CN115937599A (en) | 2023-04-07 |
Family
ID=86697556
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211669005.4A Pending CN115937599A (en) | 2022-12-23 | 2022-12-23 | Deep learning remote sensing image classification method for improving self-adaptive pooling |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115937599A (en) |
-
2022
- 2022-12-23 CN CN202211669005.4A patent/CN115937599A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110781775B (en) | Remote sensing image water body information accurate segmentation method supported by multi-scale features | |
| CN111986099B (en) | Tillage monitoring method and system based on convolutional neural network with residual error correction fused | |
| CN107424159B (en) | Image semantic segmentation method based on super-pixel edge and full convolution network | |
| CN109934826B (en) | Image feature segmentation method based on graph convolution network | |
| CN111860233B (en) | SAR image complex building extraction method and system based on attention network selection | |
| CN107564009B (en) | Outdoor scene multi-target segmentation method based on deep convolutional neural network | |
| CN112489054A (en) | Remote sensing image semantic segmentation method based on deep learning | |
| CN113705641B (en) | Hyperspectral image classification method based on rich context network | |
| Chen et al. | A new process for the segmentation of high resolution remote sensing imagery | |
| Moua et al. | Correcting the effect of sampling bias in species distribution modeling–A new method in the case of a low number of presence data | |
| CN114494821A (en) | Remote sensing image cloud detection method based on feature multi-scale perception and self-adaptive aggregation | |
| CN114863288B (en) | Building contour extraction and regularization method and device | |
| CN113223042A (en) | Intelligent acquisition method and equipment for remote sensing image deep learning sample | |
| CN111325134B (en) | Remote sensing image change detection method based on cross-layer connection convolutional neural network | |
| CN112906813A (en) | Flotation condition identification method based on density clustering and capsule neural network | |
| CN114782355B (en) | Gastric cancer digital pathological section detection method based on improved VGG16 network | |
| Chen et al. | BisDeNet: A New Lightweight Deep Learning-based Framework for Efficient Landslide Detection | |
| CN113989287A (en) | Urban road remote sensing image segmentation method and device, electronic equipment and storage medium | |
| CN115909077A (en) | Hyperspectral image change detection method based on unsupervised spectrum unmixing neural network | |
| CN115937599A (en) | Deep learning remote sensing image classification method for improving self-adaptive pooling | |
| Osei et al. | Long term monitoring of Ghana’s forest reserves Using Google Earth Engine | |
| CN114998587A (en) | Remote sensing image building semantic segmentation method and system | |
| CN114241470A (en) | Natural scene character detection method based on attention mechanism | |
| CN111986223A (en) | Method for extracting trees in outdoor point cloud scene based on energy function | |
| Cao et al. | Understanding 3D point cloud deep neural networks by visualization techniques |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |