CN111274865A - Remote sensing image cloud detection method and device based on full convolution neural network - Google Patents
Remote sensing image cloud detection method and device based on full convolution neural network Download PDFInfo
- Publication number
- CN111274865A CN111274865A CN201911286668.6A CN201911286668A CN111274865A CN 111274865 A CN111274865 A CN 111274865A CN 201911286668 A CN201911286668 A CN 201911286668A CN 111274865 A CN111274865 A CN 111274865A
- Authority
- CN
- China
- Prior art keywords
- network
- remote sensing
- sensing image
- cloud
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Astronomy & Astrophysics (AREA)
- Multimedia (AREA)
- Remote Sensing (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention relates to the field of remote sensing detection, in particular to a remote sensing image cloud detection method and device based on a full convolution neural network. The method and the device select RGB wave bands of the wind cloud meteorological satellite remote sensing image to construct a data set, and a training set is obtained in the data set; constructing an SP-HRNet network model, wherein the network model comprises a continuous parallel multi-resolution sub-network, a repeated multi-scale fusion module and a depth separable convolution combination module; inputting the training set into a network model for training to obtain parameters of the network model and form a network parameter model; and carrying out remote sensing image cloud detection by using the network parameter model. The method and the device can always maintain the sub-networks with a plurality of resolutions, so that information cannot be lost in the process of extracting the features of the image, the depth of the network is deepened, the extraction capability of the network on the features is improved by combining with the depth separable convolution, the detail information of the detection result is enriched, and the accuracy of cloud detection is improved.
Description
Technical Field
The invention relates to the field of remote sensing detection, in particular to a remote sensing image cloud detection method and device based on a full convolution neural network.
Background
Global cloud data provided by the international satellite cloud climate program ISCCP show that clouds cover more than 60% of the global surface of the earth. Therefore, the remote sensing image is extremely easy to be shielded by the cloud layer in the imaging process, so that the spectrum distortion of the original ground object is caused, and the information extraction of the image is greatly influenced. The cloud detection methods that have been conventionally used can be roughly summarized as a spectral threshold method, a spatial texture analysis method, a pattern recognition detection method, a machine learning method, and the like. The spectral threshold based cloud detection method is the least long-studied history, primarily by extracting various spectral features of each pixel, and then using one or more thresholds to determine the cloud mask. The method generally has the characteristic of simple calculation, but the method detects according to data of certain spectral bands, so the method is generally limited to specific remote sensing data and has poor universality. The cloud detection method based on the spatial texture mainly utilizes the mutual relation of pixel spatial information to carry out cloud detection, and the adaptability of the method to different sensors is superior to that of a spectral threshold method. However, the cloud has various morphological characteristics, and under the interference of the underlying surface and the cloud-like ground object, great difficulty still exists in how to select proper texture characteristics so as to obtain higher extraction accuracy. Pattern recognition relies on the correct combination of training data sets and features to determine the performance of the method. The cloud detection method based on machine learning generally regards a cloud layer as a ground surface coverage type, and a specific classifier is constructed through comprehensive spectrum and spatial characteristics of an existing sample data set to perform classification, identification and extraction on the cloud layer. Although the cloud detection method of machine learning can further improve the accuracy of cloud detection, the algorithm has the defects of time consumption, labor waste, difficulty in meeting automatic extraction of mass image data and the like in training of a classifier.
In recent years, with the success of deep learning in the field of image processing, the performance of deep learning convolutional neural networks, as networks specifically designed for image processing and computer vision, has surpassed traditional image methods in a large number of experiments. Xie et al apply the good classification ability of the convolutional neural network to remote sensing image cloud classification, have obtained very good detection results, but the classification based on neural network also has limitations, and can only output a classification result at last and can not accomplish to classify each pixel. The first semantic segmentation network FCN based on convolutional neural network modification proposed in 2015 overcomes this difficulty, and converts the full-link layer of the convolutional neural network for classification into a convolutional layer, which can adapt to any size input and can classify each pixel point. This network architecture is then widely used for remote sensing image classification. However, the convolutional neural network structure applied to cloud detection loses spatial information in the down-sampling process, so that the information cannot recover the detection result accurately when the resolution of the image is recovered by up-sampling. The semantic segmentation convolutional neural network HRNet (high Resolution Net) proposed in 2019 provides a solution for the problem, the HRNet keeps a plurality of sub-networks with Resolution ratios, and information can be fused among the sub-networks in the process of extracting features by the network, so that the loss of information is reduced, but the HRNet only uses three down-sampling layers, so that the network has insufficient extraction capability on image features, and the detection result is inaccurate.
According to the cloud detection method of the full convolution neural network based on the FCN network architecture, spatial information is lost in the process of down-sampling and feature extraction of an image, so that information cannot be recovered when the resolution of the image is recovered by up-sampling, and the obtained detection result is not accurate. The method has the defects that the resolution of the image is continuously reduced by performing pooling operation or other down-sampling operation on the image in the process of extracting the features of the image by the convolutional neural network, so that information is continuously lost in the process, and the lost information of the network cannot be completely recovered when the resolution of the image is recovered, so that the finally obtained detection result is inaccurate.
Disclosure of Invention
The embodiment of the invention provides a remote sensing image cloud detection method and device based on a full convolution neural network, and aims to at least solve the technical problem of low detection accuracy of the existing cloud detection method.
According to an embodiment of the invention, a remote sensing image cloud detection method based on a full convolution neural network is provided, which comprises the following steps:
selecting RGB wave bands of a wind cloud meteorological satellite remote sensing image to construct a data set, and acquiring a training set in the data set;
constructing an SP-HRNet network model, wherein the network model comprises a continuous parallel multi-resolution sub-network, a repeated multi-scale fusion module and a depth separable convolution combination module;
inputting the training set into a network model for training to obtain parameters of the network model and form a network parameter model;
and carrying out remote sensing image cloud detection by using the network parameter model.
Further, the consecutive parallel multi-resolution sub-networks comprise a plurality of sub-networks and are divided into a plurality of stages; the multiple sub-networks are distributed from high to low through the resolution and are connected in series in sequence, and multiple convolutions are formed; a down-sampling layer is arranged between two adjacent sub-networks, and the resolution is reduced by half; starting from a sub-network with the same resolution as the original image as a first stage, and then gradually increasing the sub-networks from high resolution to low resolution by convolution with a step size of 2 to form a new stage, and connecting the sub-networks of a plurality of resolutions in parallel.
Further, the iterative multi-scale fusion module is configured to: the network model introduces a switching unit between parallel sub-networks, and each stage of the sub-networks can repeatedly receive feature extraction information of other stages and perform multi-scale fusion, so that high-resolution information is fused into a low-resolution feature layer.
Further, the depth separable convolution in combination module is configured to: the depth separable convolution firstly uses a convolution kernel to map each channel of the feature map to a new space, then carries out convolution through another convolution kernel, and realizes the separation of the channel and the space through the depth separable convolution.
Further, the training set is input into the network model for training to obtain parameters of the network model, and the forming of the network parameter model includes: inputting the training set into the constructed network model, setting learning rate and iteration number super-parameters, setting a loss function for optimizing network parameters, adjusting the training process according to the trained loss curve, and finally obtaining the trained network parameter model.
Further, the method further comprises:
and acquiring a test set in the data set, inputting the test set into the network parameter model to obtain an extraction result of the test set, and evaluating the network model.
Further, the RGB wave bands of the wind cloud meteorological satellite remote sensing image are selected to cut the image into 256 × 256 size to construct a data set, and then the data set is divided into a training set and a testing set according to the ratio of 4: 1.
Further, the method further comprises:
and preprocessing the remote sensing image of the wind cloud meteorological satellite.
Further, the preprocessing of the wind cloud meteorological satellite remote sensing image comprises the following steps:
and performing radiation correction and spatial domain enhancement processing filtering on the remote sensing image of the wind cloud meteorological satellite by using arcgis and ENVI.
According to another embodiment of the invention, a remote sensing image cloud detection device based on a full convolution neural network is provided, which includes:
the data set acquisition unit is used for selecting RGB wave bands of the wind cloud meteorological satellite remote sensing image to construct a data set and acquiring a training set in the data set;
the network model building unit is used for building an SP-HRNet network model, and the network model comprises a multi-resolution sub-network, a repeated multi-scale fusion module and a depth separable convolution combination module which are continuously parallel;
the training unit is used for inputting the training set into the network model for training to obtain parameters of the network model and form a network parameter model;
and the detection unit is used for carrying out cloud detection on the remote sensing image by using the network parameter model.
The remote sensing image cloud detection method and device based on the full convolution neural network select RGB wave bands of a wind cloud meteorological satellite remote sensing image to construct a data set, and acquire a training set in the data set; constructing an SP-HRNet network model, wherein the network model comprises a continuous parallel multi-resolution sub-network, a repeated multi-scale fusion module and a depth separable convolution combination module; inputting the training set into a network model for training to obtain parameters of the network model and form a network parameter model; and carrying out remote sensing image cloud detection by using the network parameter model. The method and the device can always maintain the sub-networks with a plurality of resolutions, so that information cannot be lost in the process of extracting the features of the image, the depth of the network is increased, the depth separable convolution is combined, the feature extraction capability of the network is improved, the detail information of the detection result is enriched, and the accuracy of cloud detection is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention to a proper form. In the drawings:
FIG. 1 is a flow chart of a remote sensing image cloud detection method based on a full convolution neural network according to the invention;
FIG. 2 is a preferred flow chart of the remote sensing image cloud detection method based on the full convolution neural network of the present invention;
FIG. 3 is a diagram of the SP-HRNet network architecture of the present invention;
FIG. 4 is a diagram of a network comprising five consecutive parallel sub-networks according to the present invention;
FIG. 5 is a block diagram of a remote sensing image cloud detection device based on a full convolution neural network according to the present invention;
FIG. 6 is a preferred block diagram of the remote sensing image cloud detection device based on the full convolution neural network according to the invention;
FIG. 7 is a test result diagram of the remote sensing image cloud detection method based on the full convolution neural network.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The invention provides a remote sensing image cloud detection method SP-HRNet based on a full convolution neural network HRNet, which can always maintain a plurality of sub-networks with resolution ratios, so that information cannot be lost in the process of extracting the features of an image, the depth of the network is deepened, the depth separable convolution is combined, the extraction capability of the network on the features is improved, the detail information of the detection result is enriched, and the accuracy of cloud detection is improved.
Example 1
According to an embodiment of the present invention, a method for detecting a cloud of a remote sensing image based on a full convolution neural network is provided, referring to fig. 1, including the following steps:
s101, selecting RGB wave bands of a wind cloud meteorological satellite remote sensing image to construct a data set, and acquiring a training set in the data set;
s102, constructing an SP-HRNet network model, wherein the network model comprises a continuous parallel multi-resolution sub-network, a repeated multi-scale fusion module and a depth separable convolution combination module;
s103, inputting a training set into a network model for training to obtain parameters of the network model and form a network parameter model;
and S104, carrying out cloud detection on the remote sensing image by using the network parameter model.
The remote sensing image cloud detection method based on the full convolution neural network in the embodiment of the invention selects RGB wave bands of a wind cloud meteorological satellite remote sensing image to construct a data set, and obtains a training set in the data set; constructing an SP-HRNet network model, wherein the network model comprises a continuous parallel multi-resolution sub-network, a repeated multi-scale fusion module and a depth separable convolution combination module; inputting the training set into a network model for training to obtain parameters of the network model and form a network parameter model; and carrying out cloud detection on the remote sensing image by using the network parameter model. The method and the device can always maintain the sub-networks with a plurality of resolutions, so that information cannot be lost in the process of extracting the features of the image, the depth of the network is deepened, the extraction capability of the network on the features is improved by combining with depth separable convolution, the detail information of the detection result is enriched, and the accuracy of cloud detection is improved.
Specifically, referring to fig. 2, the method further comprises:
and S105, acquiring a test set in the data set, inputting the test set into the network parameter model to obtain an extraction result of the test set, and evaluating the network model.
Specifically, referring to fig. 2, the method further comprises:
and S100, preprocessing the remote sensing image of the wind cloud meteorological satellite.
The method for detecting the cloud of the remote sensing image based on the full convolution neural network is described in detail in the following by specific embodiments.
The remote sensing image cloud detection method based on the full convolution neural network comprises the following steps:
1. and (4) preprocessing the remote sensing image, performing radiation correction and spatial domain enhancement processing filtering by using arcgis and ENVI, eliminating interference, and obtaining data of real reflectivity.
2. The method comprises the steps of selecting RGB wave bands (1 st blue, 2 nd green and 3 rd red wave bands) of a wind cloud meteorological satellite remote sensing image, cutting the image into 256 × 256 sizes, constructing a data set, and dividing the data set into a training set and a testing set according to the ratio of 4: 1.
3. Providing an SP-HRNet network structure, and constructing a network model, wherein the network model comprises the following modules:
(1) a series of parallel multi-resolution sub-networks;
(2) repeating the multi-scale fusion module;
(3) and combining with the depth separable convolution module.
Compared with the HRNet, the SP-HRNet firstly deepens the depth of the network, and secondly improves the width of the network by combining the depth separable convolution, thereby being beneficial to the propagation of the characteristic information in the network and effectively improving the extraction capability of the characteristic information. The specific network structure is shown in fig. 3.
4. The training set is input into the network model for training to obtain parameters of the network model and form a network parameter model.
5. And inputting the test set into the network parameter model to obtain an extraction result of the test set image, and evaluating the network structure provided by the invention.
The network model comprises the following concrete contents:
(1) serial parallel multi-resolution sub-networks. The network contains 5 sub-networks and divide into 5 stages, compares in HRNet network, deepens the network layer number, is favorable to the extraction to the characteristic, and the network comprises high to low resolution ratio's sub-network through establishing ties, comprises a series of convolutions, has a down sample layer between two adjacent sub-networks, reduces resolution ratio by half. The network starts from a sub-network with the same resolution as the original as a first stage, and then gradually increases the sub-network from a high resolution to a low resolution by convolution with a step size of 2 to form a new stage, and connects the sub-networks of a plurality of resolutions in parallel. For example, please refer to fig. 4, which shows a network structure comprising five consecutive parallel sub-networks;
wherein N issrRepresenting the s-th sub-networkAnd r simultaneously represents the resolution index (the resolution of which represents the resolution of the original image)
). The downward slanted arrow indicates the downsampling layer, which is a convolution layer with a step size of 2, and reduces the resolution by half.
(2) And repeating the multi-scale fusion module. The network introduces an exchange unit between parallel sub-networks, for example, a part with crossed downward arrows and upward arrows in fig. 3, so that feature extraction information of other stages can be repeatedly received between each stage of the network, multi-scale fusion is fully performed, high-resolution information is fused into a low-resolution feature layer, and further detail information of cloud detection of remote sensing images is maintained.
(3) And combining the depth separable convolution with the module. The network changes the convolution in the original residual module into a depth-separable convolution, which first maps each channel of the feature map to a new space using a 3 x 3 convolution kernel, in the process, channel correlation is learned, convolution is carried out through a 1-by-1 convolution kernel, so that spatial correlation and correlation among channels are learned simultaneously, the operation is particularly effective in the remote sensing image processing, the texture information and the waveband information of the space in the remote sensing image are inseparable, the separation of the channel and the space can be realized through the deep separable convolution, the correlation of the channel and the space is learned, the required parameters are reduced compared with the linear low-rank convolution, the training speed of the network is increased, and the width of the network is further improved, so that more characteristic information can be transmitted in the network, and the reconstruction quality of the network is enhanced.
Inputting the training set into a network model for training to obtain parameters of the network model and form a network parameter model:
inputting the established training set into the established network model, setting the learning rate, the iteration times and other super parameters, setting a loss function for optimizing the network parameters, adjusting the training process according to the trained loss curve, and finally obtaining the trained network parameter model.
Inputting a test set, evaluating an extraction result:
and inputting the built test set into a network parameter model to obtain an extraction result of the test set image, and evaluating the network structure provided by the invention.
Example 2
According to another embodiment of the present invention, there is provided a remote sensing image cloud detection apparatus based on a full convolution neural network, referring to fig. 5, including:
the data set acquisition unit 201 is used for selecting RGB wave bands of the wind cloud meteorological satellite remote sensing image to construct a data set, and acquiring a training set in the data set;
the network model building unit 202 is used for building an SP-HRNet network model, and the network model comprises a continuous parallel multi-resolution sub-network, a repeated multi-scale fusion module and a depth separable convolution combination module;
the training unit 203 is used for inputting the training set into the network model for training to obtain parameters of the network model and form a network parameter model;
and the detection unit 204 is used for performing cloud detection on the remote sensing image by using the network parameter model.
The remote sensing image cloud detection device based on the full convolution neural network selects RGB wave bands of a wind cloud meteorological satellite remote sensing image to construct a data set, and obtains a training set in the data set; constructing an SP-HRNet network model, wherein the network model comprises a continuous parallel multi-resolution sub-network, a repeated multi-scale fusion module and a depth separable convolution combination module; inputting the training set into a network model for training to obtain parameters of the network model and form a network parameter model; and carrying out remote sensing image cloud detection by using the network parameter model. The method and the device can always maintain the sub-networks with a plurality of resolutions, so that information cannot be lost in the process of extracting the features of the image, the depth of the network is deepened, the extraction capability of the network on the features is improved by combining with depth separable convolution, the detail information of the detection result is enriched, and the accuracy of cloud detection is improved.
Specifically, referring to fig. 6, the apparatus further comprises:
the evaluation unit 205 is configured to obtain a test set in the data set, input the test set into the network parameter model, obtain an extraction result of the test set, and evaluate the network model.
Specifically, referring to fig. 6, the apparatus further comprises:
and the preprocessing unit 200 is used for preprocessing the remote sensing image of the wind cloud meteorological satellite.
The method for detecting the cloud of the remote sensing image based on the full convolution neural network is described in detail in the following by specific embodiments.
Remote sensing image cloud detection device based on full convolution neural network includes:
1. the preprocessing unit 200: and (4) preprocessing the remote sensing image, performing radiation correction and spatial domain enhancement processing filtering by using arcgis and ENVI, eliminating interference, and obtaining data of real reflectivity.
2. The data set acquisition unit 201: RGB wave bands (1 st blue, 2 nd green and 3 rd red wave bands) of the wind cloud meteorological satellite remote sensing image are selected, the image is cut into 256 × 256 sizes, a data set is constructed, and the data set is divided into a training set and a testing set according to the ratio of 4: 1.
3. Network model construction unit 202: providing an SP-HRNet network structure, and constructing a network model, wherein the network model comprises the following modules:
(1) a series of parallel multi-resolution sub-networks;
(2) repeating the multi-scale fusion module;
(3) and combining with the depth separable convolution module.
Compared with the HRNet, the SP-HRNet firstly deepens the depth of the network, and secondly improves the width of the network by combining the depth separable convolution, thereby being beneficial to the propagation of the characteristic information in the network and effectively improving the extraction capability of the characteristic information. The specific network structure is shown in fig. 3.
4. The training unit 203: the training set is input into a network model for training to obtain parameters of the network model and form a network parameter model.
5. The detection unit 204: and carrying out remote sensing image cloud detection by using the network parameter model.
6. The evaluation unit 205: and inputting the test set into the network parameter model to obtain the extraction result of the test set image, and evaluating the network structure provided by the invention.
The network model comprises the following concrete contents:
(1) serial parallel multi-resolution sub-networks. The network contains 5 sub-networks and divide into 5 stages, compares in HRNet network, deepens the network layer number, is favorable to the extraction to the characteristic, and the network comprises high to low resolution ratio's sub-network through establishing ties, comprises a series of convolutions, has a down sample layer between two adjacent sub-networks, reduces resolution ratio by half. The network starts from a sub-network with the same resolution as the original as a first stage, and then gradually increases the sub-network from a high resolution to a low resolution by convolution with a step size of 2 to form a new stage, and connects the sub-networks of a plurality of resolutions in parallel. For example, please refer to fig. 4, which shows a network structure comprising five consecutive parallel sub-networks;
wherein N issrRepresenting the r-th stage of the s-th sub-network, and r simultaneously represents the resolution index (the resolution of which represents the resolution of the original image)
). The downward slanted arrow indicates the downsampling layer, which is a convolution layer with a step size of 2, and reduces the resolution by half.
(2) And repeating the multi-scale fusion module. The network introduces an exchange unit between parallel sub-networks, for example, a part with crossed downward arrows and upward arrows in fig. 3, so that feature extraction information of other stages can be repeatedly received between each stage of the network, multi-scale fusion is fully performed, high-resolution information is fused into a low-resolution feature layer, and further detail information of cloud detection of remote sensing images is maintained.
(3) And combining the depth separable convolution with the module. The network changes the convolution in the original residual module into a depth-separable convolution, which first maps each channel of the feature map to a new space using a 3 x 3 convolution kernel, in the process, channel correlation is learned, convolution is carried out through a 1-by-1 convolution kernel, so that spatial correlation and correlation among channels are learned simultaneously, the operation is particularly effective in the remote sensing image processing, the texture information and the waveband information of the space in the remote sensing image are inseparable, the separation of the channel and the space can be realized through the deep separable convolution, the correlation of the channel and the space is learned, the required parameters are reduced compared with the linear low-rank convolution, the training speed of the network is increased, and the width of the network is further improved, so that more characteristic information can be transmitted in the network, and the reconstruction quality of the network is enhanced.
The training unit 203: the training set is input into the network model for training to obtain parameters of the network model and form a network parameter model.
Inputting the established training set into the established network model, setting the learning rate, the iteration times and other super parameters, setting a loss function for optimizing the network parameters, adjusting the training process according to the trained loss curve, and finally obtaining the trained network parameter model.
The evaluation unit 205: inputting the test set, and evaluating the extraction result.
And inputting the built test set into a network parameter model to obtain an extraction result of the test set image, and evaluating the network structure provided by the invention.
The invention has the innovative technical points that:
1. and 3, providing the SP-HRNet network structure in the step 3.
2. Implementation in a network architecture "combined with deep separable convolution" in step 3.
The advantages of the invention over the prior art are at least:
(1) compared with a network similar to an FCN network structure, the method and the device have the advantages that the down-sampling is performed firstly, then the up-sampling is performed, and the image resolution is restored, so that the detail information of the cloud detection result is enriched to a certain extent.
(2) Compared with the method for carrying out cloud detection by using the HRNet, the network provided by the invention deepens the network depth, and meanwhile, the depth separable convolution is combined, so that the feature extraction capability is improved, and the cloud detection result precision can be improved to a certain extent.
The SP-HRNet network model provided by the invention is trained by utilizing a training set formed by wind cloud meteorological satellite image data, the test set is tested, and meanwhile, the test set is compared with the DeepLab v3+ (the network with the best performance in the DeepLab series network provided by Google obtains excellent segmentation results on a plurality of public data sets) and the cloud detection result of the HRNet convolutional network, so that the aim of the invention is primarily realized.
Fig. 7(a) - (d) are an original image of a remote sensing image, a deep lab v3+ cloud detection result diagram, an HRNet cloud detection result diagram, and an SP-HRNet cloud detection result diagram, respectively. FIGS. 7(e) - (h) are respectively a region A, a deep Lab v3+ cloud detection result graph, an HRNet cloud detection result graph, and an SP-HRNet cloud detection result graph of the image in FIG. 7 (a). (i) - (l) are the B region, DeepLab v3+ cloud detection result map, HRNet cloud detection result map, and SP-HRNet cloud detection result map of the image in (a) in FIG. 7, respectively.
Compared with the DeepLab v3+ cloud detection result, the method greatly enriches the detail information of the cloud detection result. As shown in fig. 7, deep lab v3+ can only detect large cloud areas, and cannot detect internal detail information. And the SP-HRNet not only can detect the cloud region, but also can avoid the internal non-cloud region and accurately detect the cloud layer. Compared with the HRNet cloud detection result, as shown in the region a and the region B in fig. 7(a), the HRNet cloud detection result has the phenomena of false detection and missing detection, and the SP-HRNet can well store the detail information, so that the false detection and missing detection regions are effectively reduced.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described system embodiments are merely illustrative, and for example, a division of a unit may be a logical division, and an actual implementation may have another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk, which can store program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A remote sensing image cloud detection method based on a full convolution neural network is characterized by comprising the following steps:
selecting RGB wave bands of a wind cloud meteorological satellite remote sensing image to construct a data set, and acquiring a training set in the data set;
constructing an SP-HRNet network model, wherein the network model comprises a continuous parallel multi-resolution sub-network, a repeated multi-scale fusion module and a depth separable convolution combination module;
inputting the training set into a network model for training to obtain parameters of the network model and form a network parameter model;
and carrying out remote sensing image cloud detection by using the network parameter model.
2. The method for detecting the cloud of the remote sensing images based on the full convolution neural network as claimed in claim 1, wherein the continuous parallel multi-resolution sub-networks comprise a plurality of sub-networks and are divided into a plurality of stages; the multiple sub-networks are distributed from high to low through the resolution and are connected in series in sequence, and multiple convolutions are formed; a down-sampling layer is arranged between two adjacent sub-networks, and the resolution is reduced by half; starting from a sub-network with the same resolution as the original image as a first stage, and then gradually increasing the sub-network from a high resolution to a low resolution by convolution with a step size of 2 to form a new stage, and connecting the sub-networks of a plurality of resolutions in parallel.
3. The full convolution neural network-based remote sensing image cloud detection method according to claim 1, wherein the repeated multi-scale fusion module is configured to: the network model introduces switching units between parallel sub-networks, and each stage of the sub-networks can repeatedly receive feature extraction information of other stages, perform multi-scale fusion and fuse high-resolution information into a low-resolution feature layer.
4. The full convolution neural network-based remote sensing image cloud detection method according to claim 1, wherein the depth separable convolution combining module is configured to: the depth separable convolution firstly uses convolution kernels to map each channel of the feature map to a new space, then convolution is carried out through another convolution kernel, and separation of the channel and the space is achieved through the depth separable convolution.
5. The method for cloud detection of remote sensing images based on a full convolution neural network as claimed in claim 1, wherein the training set is input into a network model for training to obtain parameters of the network model, and forming the network parameter model includes: inputting the training set into the constructed network model, setting learning rate and iteration number hyperparameters, setting a loss function for optimizing network parameters, adjusting the training process according to the trained loss curve, and finally obtaining the trained network parameter model.
6. The method for cloud detection of remote sensing images based on full convolution neural network according to claim 1, wherein the method further comprises:
and acquiring a test set in the data set, inputting the test set into the network parameter model to obtain an extraction result of the test set, and evaluating the network model.
7. The remote sensing image cloud detection method based on the full convolution neural network is characterized in that RGB wave bands of wind cloud meteorological satellite remote sensing images are selected to cut the images into 256 × 256 size building data sets, and then the data sets are divided into training sets and testing sets according to the ratio of 4: 1.
8. The method for cloud detection of remote sensing images based on full convolution neural network according to claim 1, wherein the method further comprises:
and preprocessing the remote sensing image of the wind cloud meteorological satellite.
9. The remote sensing image cloud detection method based on the full convolution neural network of claim 8, wherein the preprocessing of the wind cloud meteorological satellite remote sensing image comprises:
and performing radiation correction and spatial domain enhancement processing filtering on the remote sensing image of the wind cloud meteorological satellite by using arcgis and ENVI.
10. A remote sensing image cloud detection device based on a full convolution neural network is characterized by comprising:
the data set acquisition unit is used for selecting RGB wave bands of the wind cloud meteorological satellite remote sensing image to construct a data set and acquiring a training set in the data set;
the network model building unit is used for building an SP-HRNet network model, and the network model comprises a continuous parallel multi-resolution sub-network, a repeated multi-scale fusion module and a depth separable convolution combination module;
the training unit is used for inputting the training set into the network model for training to obtain parameters of the network model and form a network parameter model;
and the detection unit is used for carrying out cloud detection on the remote sensing image by using the network parameter model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911286668.6A CN111274865B (en) | 2019-12-14 | 2019-12-14 | Remote sensing image cloud detection method and device based on full convolution neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911286668.6A CN111274865B (en) | 2019-12-14 | 2019-12-14 | Remote sensing image cloud detection method and device based on full convolution neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111274865A true CN111274865A (en) | 2020-06-12 |
| CN111274865B CN111274865B (en) | 2023-09-19 |
Family
ID=71111850
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911286668.6A Active CN111274865B (en) | 2019-12-14 | 2019-12-14 | Remote sensing image cloud detection method and device based on full convolution neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111274865B (en) |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111784623A (en) * | 2020-09-07 | 2020-10-16 | 腾讯科技（深圳）有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| CN111798461A (en) * | 2020-06-19 | 2020-10-20 | 武汉大学 | Pixel-level remote sensing image cloud area detection method for guiding deep learning by coarse-grained label |
| CN111985549A (en) * | 2020-08-12 | 2020-11-24 | 中国科学院光电技术研究所 | Deep learning method for automatic positioning and identification of components for given rigid body target |
| CN112348770A (en) * | 2020-09-09 | 2021-02-09 | 陕西师范大学 | Bridge crack detection method based on multi-resolution convolution network |
| CN112446859A (en) * | 2020-11-18 | 2021-03-05 | 中国科学院上海技术物理研究所 | Satellite-borne thermal infrared camera image cloud detection method based on deep learning |
| CN113239830A (en) * | 2021-05-20 | 2021-08-10 | 北京航空航天大学 | Remote sensing image cloud detection method based on full-scale feature fusion |
| CN113343942A (en) * | 2021-07-21 | 2021-09-03 | 西安电子科技大学 | Remote sensing image defect detection method |
| CN113570642A (en) * | 2021-06-10 | 2021-10-29 | 国家卫星气象中心(国家空间天气监测预警中心) | Stationary orbit satellite convection birth early warning method based on background field data and machine learning |
| CN113743300A (en) * | 2021-09-03 | 2021-12-03 | 中化现代农业有限公司 | Semantic segmentation based high-resolution remote sensing image cloud detection method and device |
| CN113936204A (en) * | 2021-11-22 | 2022-01-14 | 安徽师范大学 | High-resolution remote sensing image cloud and snow identification method and device fusing terrain data and deep neural network |
| CN113962925A (en) * | 2021-08-19 | 2022-01-21 | 中国人民解放军国防科技大学 | Method and device for detecting heterogeneous remote sensing image change based on satellite and unmanned aerial vehicle |
| CN114066755A (en) * | 2021-11-11 | 2022-02-18 | 南京航空航天大学 | Remote sensing image thin cloud removing method and system based on full-band feature fusion |
| CN114119621A (en) * | 2021-11-30 | 2022-03-01 | 云南电网有限责任公司输电分公司 | SAR remote sensing image water area segmentation method based on depth coding and decoding fusion network |
| CN114882380A (en) * | 2022-07-08 | 2022-08-09 | 山东省国土测绘院 | Wetland resource remote sensing identification algorithm based on improved hrnet model |
| WO2022260591A1 (en) * | 2021-06-08 | 2022-12-15 | Lemon Inc. | A multi-resolution neural network architecture search space for dense prediction tasks |
| CN115546671A (en) * | 2022-11-01 | 2022-12-30 | 北京数字政通科技股份有限公司 | Unmanned aerial vehicle change detection method and system based on multitask learning |
| CN115546033A (en) * | 2022-12-05 | 2022-12-30 | 耕宇牧星(北京)空间科技有限公司 | Image correction method based on remote sensing image |
| CN115830471A (en) * | 2023-01-04 | 2023-03-21 | 安徽大学 | Multi-scale feature fusion and alignment domain self-adaptive cloud detection method |
| CN115862014A (en) * | 2023-02-27 | 2023-03-28 | 中国地质大学（武汉） | Mining area semantic segmentation model construction method and device and mining area semantic segmentation method |
| KR20230092604A (en) * | 2021-12-17 | 2023-06-26 | 한국원자력연구원 | Intelligent cloud volume measurement apparatus and method |
| US11983239B2 (en) | 2021-06-08 | 2024-05-14 | Lemon Inc. | Lightweight transformer for high resolution images |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107610194A (en) * | 2017-08-14 | 2018-01-19 | 成都大学 | MRI super resolution ratio reconstruction method based on Multiscale Fusion CNN |
| CN108564109A (en) * | 2018-03-21 | 2018-09-21 | 天津大学 | A kind of Remote Sensing Target detection method based on deep learning |
| CN108805861A (en) * | 2018-04-28 | 2018-11-13 | 中国人民解放军国防科技大学 | Remote sensing image cloud detection method based on deep learning |
| CN109255334A (en) * | 2018-09-27 | 2019-01-22 | 中国电子科技集团公司第五十四研究所 | Remote sensing image terrain classification method based on deep learning semantic segmentation network |
| CN110084195A (en) * | 2019-04-26 | 2019-08-02 | 西安电子科技大学 | Remote Sensing Target detection method based on convolutional neural networks |
| CN110097090A (en) * | 2019-04-10 | 2019-08-06 | 东南大学 | A kind of image fine granularity recognition methods based on multi-scale feature fusion |
| CN110097554A (en) * | 2019-04-16 | 2019-08-06 | 东南大学 | The Segmentation Method of Retinal Blood Vessels of convolution is separated based on intensive convolution sum depth |
| CN110175575A (en) * | 2019-05-29 | 2019-08-27 | 南京邮电大学 | A kind of single Attitude estimation method based on novel high-resolution network model |
-
2019
- 2019-12-14 CN CN201911286668.6A patent/CN111274865B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107610194A (en) * | 2017-08-14 | 2018-01-19 | 成都大学 | MRI super resolution ratio reconstruction method based on Multiscale Fusion CNN |
| CN108564109A (en) * | 2018-03-21 | 2018-09-21 | 天津大学 | A kind of Remote Sensing Target detection method based on deep learning |
| CN108805861A (en) * | 2018-04-28 | 2018-11-13 | 中国人民解放军国防科技大学 | Remote sensing image cloud detection method based on deep learning |
| CN109255334A (en) * | 2018-09-27 | 2019-01-22 | 中国电子科技集团公司第五十四研究所 | Remote sensing image terrain classification method based on deep learning semantic segmentation network |
| CN110097090A (en) * | 2019-04-10 | 2019-08-06 | 东南大学 | A kind of image fine granularity recognition methods based on multi-scale feature fusion |
| CN110097554A (en) * | 2019-04-16 | 2019-08-06 | 东南大学 | The Segmentation Method of Retinal Blood Vessels of convolution is separated based on intensive convolution sum depth |
| CN110084195A (en) * | 2019-04-26 | 2019-08-02 | 西安电子科技大学 | Remote Sensing Target detection method based on convolutional neural networks |
| CN110175575A (en) * | 2019-05-29 | 2019-08-27 | 南京邮电大学 | A kind of single Attitude estimation method based on novel high-resolution network model |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111798461A (en) * | 2020-06-19 | 2020-10-20 | 武汉大学 | Pixel-level remote sensing image cloud area detection method for guiding deep learning by coarse-grained label |
| CN111985549A (en) * | 2020-08-12 | 2020-11-24 | 中国科学院光电技术研究所 | Deep learning method for automatic positioning and identification of components for given rigid body target |
| CN111985549B (en) * | 2020-08-12 | 2023-03-31 | 中国科学院光电技术研究所 | Deep learning method for automatic positioning and identification of components for given rigid body target |
| CN111784623A (en) * | 2020-09-07 | 2020-10-16 | 腾讯科技（深圳）有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| CN112348770A (en) * | 2020-09-09 | 2021-02-09 | 陕西师范大学 | Bridge crack detection method based on multi-resolution convolution network |
| CN112348770B (en) * | 2020-09-09 | 2024-03-01 | 陕西师范大学 | Bridge crack detection method based on multi-resolution convolutional network |
| CN112446859A (en) * | 2020-11-18 | 2021-03-05 | 中国科学院上海技术物理研究所 | Satellite-borne thermal infrared camera image cloud detection method based on deep learning |
| CN113239830A (en) * | 2021-05-20 | 2021-08-10 | 北京航空航天大学 | Remote sensing image cloud detection method based on full-scale feature fusion |
| CN113239830B (en) * | 2021-05-20 | 2023-01-17 | 北京航空航天大学 | Remote sensing image cloud detection method based on full-scale feature fusion |
| WO2022260591A1 (en) * | 2021-06-08 | 2022-12-15 | Lemon Inc. | A multi-resolution neural network architecture search space for dense prediction tasks |
| US11983239B2 (en) | 2021-06-08 | 2024-05-14 | Lemon Inc. | Lightweight transformer for high resolution images |
| CN113570642A (en) * | 2021-06-10 | 2021-10-29 | 国家卫星气象中心(国家空间天气监测预警中心) | Stationary orbit satellite convection birth early warning method based on background field data and machine learning |
| CN113570642B (en) * | 2021-06-10 | 2024-01-05 | 国家卫星气象中心(国家空间天气监测预警中心) | Static orbit satellite convection primary early warning method based on background field data and machine learning |
| CN113343942A (en) * | 2021-07-21 | 2021-09-03 | 西安电子科技大学 | Remote sensing image defect detection method |
| CN113962925B (en) * | 2021-08-19 | 2024-01-26 | 中国人民解放军国防科技大学 | Method and device for detecting heterogeneous remote sensing image change based on satellite and unmanned aerial vehicle |
| CN113962925A (en) * | 2021-08-19 | 2022-01-21 | 中国人民解放军国防科技大学 | Method and device for detecting heterogeneous remote sensing image change based on satellite and unmanned aerial vehicle |
| CN113743300A (en) * | 2021-09-03 | 2021-12-03 | 中化现代农业有限公司 | Semantic segmentation based high-resolution remote sensing image cloud detection method and device |
| CN114066755A (en) * | 2021-11-11 | 2022-02-18 | 南京航空航天大学 | Remote sensing image thin cloud removing method and system based on full-band feature fusion |
| CN113936204A (en) * | 2021-11-22 | 2022-01-14 | 安徽师范大学 | High-resolution remote sensing image cloud and snow identification method and device fusing terrain data and deep neural network |
| CN114119621A (en) * | 2021-11-30 | 2022-03-01 | 云南电网有限责任公司输电分公司 | SAR remote sensing image water area segmentation method based on depth coding and decoding fusion network |
| KR20230092604A (en) * | 2021-12-17 | 2023-06-26 | 한국원자력연구원 | Intelligent cloud volume measurement apparatus and method |
| KR102635911B1 (en) | 2021-12-17 | 2024-02-14 | 한국원자력연구원 | Intelligent cloud volume measurement apparatus and method |
| CN114882380A (en) * | 2022-07-08 | 2022-08-09 | 山东省国土测绘院 | Wetland resource remote sensing identification algorithm based on improved hrnet model |
| CN115546671A (en) * | 2022-11-01 | 2022-12-30 | 北京数字政通科技股份有限公司 | Unmanned aerial vehicle change detection method and system based on multitask learning |
| CN115546033A (en) * | 2022-12-05 | 2022-12-30 | 耕宇牧星(北京)空间科技有限公司 | Image correction method based on remote sensing image |
| CN115830471B (en) * | 2023-01-04 | 2023-06-13 | 安徽大学 | Multi-scale feature fusion and alignment domain self-adaptive cloud detection method |
| CN115830471A (en) * | 2023-01-04 | 2023-03-21 | 安徽大学 | Multi-scale feature fusion and alignment domain self-adaptive cloud detection method |
| CN115862014A (en) * | 2023-02-27 | 2023-03-28 | 中国地质大学（武汉） | Mining area semantic segmentation model construction method and device and mining area semantic segmentation method |
| CN115862014B (en) * | 2023-02-27 | 2023-04-28 | 中国地质大学（武汉） | Mining area semantic segmentation model construction method and device and mining area semantic segmentation method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111274865B (en) | 2023-09-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111274865B (en) | Remote sensing image cloud detection method and device based on full convolution neural network | |
| CN111986099B (en) | Tillage monitoring method and system based on convolutional neural network with residual error correction fused | |
| CN111428781A (en) | Remote sensing image ground object classification method and system | |
| CN108846835B (en) | Image change detection method based on depth separable convolutional network | |
| CN108596108B (en) | Aerial remote sensing image change detection method based on triple semantic relation learning | |
| CN106845321B (en) | Method and device for processing pavement marking information | |
| CN110163213B (en) | Remote sensing image segmentation method based on disparity map and multi-scale depth network model | |
| CN108960404B (en) | Image-based crowd counting method and device | |
| CN111797712B (en) | Remote sensing image cloud and cloud shadow detection method based on multi-scale feature fusion network | |
| CN113239830B (en) | Remote sensing image cloud detection method based on full-scale feature fusion | |
| CN108229425A (en) | A kind of identifying water boy method based on high-resolution remote sensing image | |
| CN110866494B (en) | Urban group extraction method and system based on optical remote sensing image | |
| CN107392122B (en) | Polarimetric SAR image target detection method based on multi-polarimetric feature and FCN-CRF fusion network | |
| CN103020265B (en) | The method and system of image retrieval | |
| CN107665324A (en) | A kind of image-recognizing method and terminal | |
| CN106548169A (en) | Fuzzy literal Enhancement Method and device based on deep neural network | |
| CN114494821B (en) | Remote sensing image cloud detection method based on feature multi-scale perception and self-adaptive aggregation | |
| CN106529472B (en) | Object detection method and device based on large scale high-resolution high spectrum image | |
| CN108932474B (en) | Remote sensing image cloud judgment method based on full convolution neural network composite characteristics | |
| CN113887472A (en) | Remote sensing image cloud detection method based on cascade color and texture feature attention | |
| CN112749578A (en) | Remote sensing image automatic road extraction method based on deep convolutional neural network | |
| CN111291818B (en) | Non-uniform class sample equalization method for cloud mask | |
| Zhu et al. | Towards automatic wild animal detection in low quality camera-trap images using two-channeled perceiving residual pyramid networks | |
| CN114266957A (en) | Hyperspectral image super-resolution restoration method based on multi-degradation mode data augmentation | |
| CN114943902A (en) | Urban vegetation unmanned aerial vehicle remote sensing classification method based on multi-scale feature perception network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |