CN107273938B - Multi-source remote sensing image ground object classification method based on two-channel convolution ladder network - Google Patents

Multi-source remote sensing image ground object classification method based on two-channel convolution ladder network Download PDF

Info

Publication number
CN107273938B
CN107273938B CN201710571057.0A CN201710571057A CN107273938B CN 107273938 B CN107273938 B CN 107273938B CN 201710571057 A CN201710571057 A CN 201710571057A CN 107273938 B CN107273938 B CN 107273938B
Authority
CN
China
Prior art keywords
remote sensing
data set
channel
layer
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710571057.0A
Other languages
Chinese (zh)
Other versions
CN107273938A (en
Inventor
焦李成
屈嵘
高倩
马文萍
杨淑媛
侯彪
刘芳
尚荣华
张向荣
张丹
唐旭
马晶晶
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xidian University
Original Assignee
Xidian University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xidian University filed Critical Xidian University
Priority to CN201710571057.0A priority Critical patent/CN107273938B/en
Publication of CN107273938A publication Critical patent/CN107273938A/en
Application granted granted Critical
Publication of CN107273938B publication Critical patent/CN107273938B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a multisource remote sensing image ground object classification method based on a two-channel convolution ladder network, which is characterized in that multispectral data of an area to be classified, which are obtained by a landsat-8 sensor and a sentinel-2 sensor, are respectively normalized by using ENVI software to obtain normalized multispectral data; taking each element of the normalized data, and taking a peripheral 28 multiplied by 28 block to represent an original element value to form a characteristic matrix based on the image block; randomly selecting a plurality of blocks in each class to form a training data set L and S; constructing a multi-source remote sensing image ground object classification model based on a two-channel convolution ladder network; training a two-channel convolution ladder network multisource remote sensing image ground object classification model by using a training data set L and a training data set S; and classifying the test data set by using the trained two-channel convolution ladder network multi-source remote sensing image surface feature classification model. The method can obtain high multi-source image classification precision by using a small number of samples with class marks, and can be used for target detection.

Description

Multi-source remote sensing image ground object classification method based on two-channel convolution ladder network
Technical Field
The invention belongs to the field of image processing, and particularly relates to a multisource remote sensing image ground object classification method based on a two-channel convolution ladder network.
Background
With the development of remote sensing technology, more and more multispectral, multi-resolution and multi-temporal image data of the same region are acquired by various remote sensors, and abundant and precious data are provided for natural resource investigation, environmental monitoring and the like. However, the image data obtained by various single remote sensing means has obvious limitations and differences in geometric, spectral and spatial resolutions, so that the capacity of the image data for classification is limited. It is obviously important to combine their respective advantages and complements for classification. The information fusion technology is a technology for comprehensive processing of multi-element information, and can synthesize multi-source information to generate more accurate and more complete estimation and judgment compared with unit information. The information fusion technology can be divided into fusion of a data layer, a feature layer and a decision layer according to a fusion level. The feature level fusion is a higher-level fusion, and the fusion technology firstly extracts features of various data sources and then comprehensively analyzes and fuses the features. The common feature level fusion method comprises Bayes estimation, Dempster-Shafer evidence theory, cluster analysis, artificial neural network and the like. The neural network has the functions of distributed storage, parallel processing, self-learning, self-organization and the like of information, integrates and processes a high-dimensional feature space formed by multi-source information features, and can effectively fuse multi-dimensional information for classification and other problems.
When the artificial neural network is used for realizing fusion and classification, a supervised classification method is often used, a large amount of classified standard data is needed, the cost is high, and a large amount of manpower and financial resources are consumed.
Disclosure of Invention
The invention aims to overcome the defects and provides a multi-source remote sensing image ground object classification method based on a two-channel convolution ladder network.
In order to achieve the above object, the present invention comprises the steps of:
step one, performing normalization processing on multispectral data of a plurality of areas to be detected, which are obtained by a landsat-8 sensor, to obtain normalized multispectral data, and marking the normalized multispectral data as landsat _ A, landsat _ B, landsat _ C, … … and landsat _ N;
step two, the multispectral data of a plurality of regions to be detected obtained by the sentinel-2 sensor are normalized to obtain normalized multispectral data, and l is marked as sentinel _ A, sentinel _ B, sentinel _ C, … … and sentinel _ N;
thirdly, taking a peripheral 28 multiplied by 28 block in each element in the landsat _ A to represent an original element value to form a feature matrix L1_ A based on the image block, and similarly, obtaining feature matrices L1_ B, L1_ C, … … and L1_ N;
step four, each element in the sentinel _ A takes a peripheral 28 multiplied by 28 block to represent an original element value to form a characteristic matrix S1_ A based on the image block, and similarly, characteristic matrices S1_ B, S1_ C, … … and S1_ N are obtained;
step five, in the characteristic L1_ A, a plurality of blocks are randomly selected from each type to form a set L2_ A, similarly, sets L2_ B, L2_ C, … … and L2_ N are respectively obtained from L1_ B, L1_ C, … … and L1_ N, and a training data set L is formed from L2_ A, L2_ B, L2_ C, … … and L2_ N;
step six, in S1_ A, selecting blocks corresponding to the blocks in L2_ A to form a set S2_ A, and similarly, obtaining sets S2_ B, S2_ C, … … and S2_ N respectively from S1_ B, S1_ C, … … and S1_ N, and forming a training data set S from S2_ A, S2_ B, S2_ C, … … and S2_ N;
constructing a two-channel convolution ladder network multi-source remote sensing image ground object classification model;
step eight, training the multi-source image ground feature classification model by using a training data set L and a training data set S to obtain a trained model;
and step nine, classifying the test data sets L1_ A, L1_ B, L1_ C, … …, L1_ N, S1_ A, S1_ B, S1_ C, … … and S1_ N by using the trained models to obtain the output of the model corresponding to each pixel point in the test data sets.
In the first step and the second step, the ENVI software is used for carrying out normalization processing on the data, and the selected normalization mode is equal.
In the third step, the specific method for obtaining the feature matrices L1_ A, L1_ B, L1_ C, … … and L1_ N is as follows:
the normalized feature matrix landsat _ a is used to take a block of 28 × 28 pixels around each element to represent the original element value, so that the class label of the block is still the class label of the original element value, and since the feature matrix is 9-dimensional, the size of each block is 28 × 28 × 9, so that the feature matrix L1_ a based on the image block is formed, and similarly, the feature matrices L1_ A, L1_ B, L1_ C, … … and L1_ N are obtained.
In the fourth step, the specific method for obtaining the feature matrices S1_ A, S1_ B, S1_ C, … … and S1_ N is as follows:
the normalized feature matrix sentinel _ a is used to take a block of 28 × 28 pixels around each element to represent the original element value, so that the class label of the block is still the class label of the original element value, and since the feature matrix is 10-dimensional, the size of each block is 28 × 28 × 10, thereby forming the feature matrix S1_ a based on image blocks, and similarly, the feature matrices S1_ A, S1_ B, S1_ C, … … and S1_ N are obtained.
In the fifth step, a specific method for forming the training data set L is as follows:
in L1_ A, samples are randomly disturbed, the first 10% of blocks of each type are selected to form a set L2_ A, similarly, sets L2_ B, L2_ C, … … and L2_ N are respectively obtained from L1_ B, L1_ C, … … and L1_ N, and a training data set L is formed from L2_ A, L2_ B, L2_ C, … … and L2_ N.
In the sixth step, a specific method for forming the training data set S is as follows:
s1_ A is disordered in the same way as L1_ A, the first 10% of blocks of each type are selected to form a set S2_ A, similarly, sets S2_ B, S2_ C … … and S2_ N are obtained from S1_ B, S1_ C, … … and S1_ N respectively, and a training data set S is formed by S2_ A, S2_ B, S2_ C, S … … and S2_ N.
In the seventh step, a specific method for constructing a dual-channel convolution ladder network multisource remote sensing image surface feature classification model is as follows:
first, a network encoder is constructed:
the encoder based on convolution contains clean part and lossy part, and two parts share a set of parameter, and the gaussian noise that the average is 0 standard deviation and is 0.3 is added to each layer of lossy part, and the structure of encoder is: input layer → first convolution layer → second convolution layer → third convolution layer → fourth convolution layer → softmax classifier;
secondly, constructing a decoder:
the decoder reconstructs the lossy part of the encoder for each layer from output to input in turn, the reconstruction function is as follows:
Figure BDA0001349677340000041
Figure BDA0001349677340000042
represents the output of the ith neuron of the ith layer of the decoder, i 1,2lWherein m islRepresents the number of neurons of the l-th layer,
Figure BDA0001349677340000043
represents the output of the ith neuron of the l-th layer of the lossy part of the encoder,
Figure BDA0001349677340000044
to represent
Figure BDA0001349677340000045
The weight of (a) is determined,
Figure BDA0001349677340000046
it is indicated that a priori,
Figure BDA0001349677340000047
is a noise reduction function whichIn the formula, i and l are positive integers;
thirdly, constructing a loss function:
Figure BDA0001349677340000048
Ccis a cross-entropy loss function representing the loss of the supervised part, t (n) represents class labels,
Figure BDA0001349677340000049
is the output of the lossy part of the encoder, x1(n) is the input of the first channel in the encoder, x2(n) is the input of a second channel in the encoder, CdDenotes the loss of the unsupervised part, λlRepresenting the weight of the l layers of reconstruction errors, N representing the number of samples input, mlNumber of neurons in layer I, z(l)Represents the output of the ith layer of the clean part of the encoder,
Figure BDA00013496773400000410
representing the output of the l-th layer reconstructed by the decoder.
In the step eight, a specific method for obtaining the trained model is as follows:
and respectively taking the training data set L and the training data set S as the input of a first channel and a second channel in the multi-source remote sensing image surface feature classification model, taking the category of each pixel point in the training data set as the output of the multi-source image surface feature classification model, and optimizing the network parameters of the classification model by solving the error between the category and the correct category marked manually and performing back propagation on the error to obtain the trained classification model.
In the ninth step, a specific method for obtaining the output of the model corresponding to each pixel point in the test data set is as follows:
the test data sets L1_ A, L1_ B, L1_ C, … … and L1_ N serve as input of a first channel of a trained multi-source remote sensing image ground feature classification model, S1_ A, S1_ B, S1_ C, … … and S1_ N serve as input of a second channel of the trained multi-source remote sensing image ground feature classification model, and output of the model is a classification category obtained by classifying each pixel point in the test data set of the region to be classified.
Compared with the prior art, the method has the advantages that the pixel-level features are expanded into the image block features, so that the spectral band information and the spatial information can be simultaneously obtained, and the method is more favorable for describing the features of different ground objects; according to the invention, the semi-supervised convolution ladder network applied to the natural image classification problem is applied to the multi-source remote sensing image ground feature classification, so that higher ground feature classification precision can be obtained under the condition of only using a small number of samples with class marks, and the consumption of manpower and financial resources is greatly reduced; compared with the traditional data fusion method, the method has the advantages that the fusion characteristics obtained by the method are more abstract and representative, the characteristics of different ground objects can be better described, and the classification precision of the multi-source remote sensing image is favorably improved; according to the method, the ground object target classification is realized by using the multi-source remote sensing data, the information complementation of different data is realized, and more complete characteristics of different ground objects are obtained.
Drawings
FIG. 1 is a flow chart of an implementation of the present invention;
FIG. 2 is a diagram of an artificial marker of an image to be detected according to the present invention; wherein a is Berlin; b is Paris; c is hong Kong; d is Rome; e is saint paul;
FIG. 3 is a diagram illustrating the classification result of an image to be classified according to the present invention; wherein a is Berlin; b is Paris; c is hong Kong; d is Rome; e is saint paul.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
Referring to fig. 1, the method of the invention comprises the following steps:
step 1, respectively carrying out normalization processing on multispectral data of five cities, namely Berlin, Paris, hong Kong, Roman and St Paul, obtained by a landsat-8 sensor by using ENVI software to obtain normalized multispectral data which are respectively marked as landsat _ berlin, landsat _ paris, landsat _ hong _ kong, landsat _ color and landsat _ sao _ paulo;
multispectral data of five cities, namely Berlin, Paris, hong Kong, Roman and St.Paul, obtained by a Landsat-8 sensor are all 9 wave bands, and the image sizes are 666 multiplied by 643, 988 multiplied by 1160, 529 multiplied by 528, 447 multiplied by 377 and 871 multiplied by 1067 respectively;
when the ENVI software is used for carrying out normalization processing on the data, the selected normalization mode is equal;
step 2, performing normalization processing on multispectral data of five cities, namely Berlin, Paris, hong Kong, Roman and St.Paul, obtained by a sensor of sentinel-2 by using ENVI software respectively to obtain normalized multispectral data which are respectively marked as sentinel _ berlin, sentinel _ part, sentinel _ hong _ kong, sentinel _ color and sentinel _ sao _ paulo;
multispectral data of five cities of Berlin, Paris, hong Kong, Roman and St Paul obtained by a sentinel-2 sensor are all 10 wave bands, and the image sizes are 666 multiplied by 643, 988 multiplied by 1160, 529 multiplied by 528, 447 multiplied by 377 and 871 multiplied by 1067 respectively;
when the ENVI software is used for carrying out normalization processing on the data, the selected normalization mode is equal;
step 3, taking a peripheral 28 multiplied by 28 block from each element in the landsat _ berlin to represent an original element value, and forming a feature matrix L1_ berlin based on the image block, and similarly, obtaining feature matrices L1_ paris, L1_ hong _ kong, L1_ color and L1_ sao _ paulo;
taking a block with 28 × 28 pixels around each element in the normalized feature matrix landsat _ berlin to represent an original element value, so that the class label of the block is still the class label of the original element value, and since the feature matrix is 9-dimensional, the size of each block is 28 × 28 × 9, so that an image block-based feature matrix L1_ berlin is formed, and similarly, feature matrices L1_ paris, L1_ hong _ kong, L1_ rope, L1_ sao _ paulo are obtained;
step 4, taking a peripheral 28 multiplied by 28 block from each element in the sentinel _ berlin to represent an original element value, and forming a feature matrix S1_ berlin based on the image block, and similarly, obtaining feature matrices S1_ paris, S1_ hong _ kong, S1_ rome and S1_ sao _ paulo;
each element in the normalized feature matrix sentinel _ berlin is used for taking a block with 28 × 28 pixels around to represent the original element value, so that the class mark of the block is still the class mark of the original element value, and the feature matrix is 10-dimensional, so that the size of each block is 28 × 28 × 10, thereby forming an image block-based feature matrix S1_ berlin, and similarly, feature matrices S1_ paris, S1_ hong _ kong, S1_ rope, S1_ sao _ paulo are obtained;
step 5, in the L1_ berlin, a plurality of blocks are randomly selected from each type to form a set L2_ berlin, similarly, sets L2_ paris, L2_ hong _ kong, L2_ rome and L2_ sao _ paulo are respectively obtained from L1_ paris, L1_ hong _ kong, L1_ rome and L1_ sao _ paulo, and a training data set L is formed by L2_ berlin, L2_ paris, L2_ hong _ kong, L2_ rome and L2_ sao _ paulo;
in L1_ berlin, randomly scrambling samples, selecting the first 10% of blocks of each type to form a set L2_ berlin, and similarly, respectively obtaining sets L2_ paris, L2_ hong _ kong, L2_ rome and L2_ sao _ paulo from L1_ paris, L1_ hong _ kong, L2_ rome and L2_ sao _ paulo, and forming a training data set L from L2_ berlin, L2_ paris, L2_ hong _ kong, L2_ rome and L2_ sao _ paulo;
step 6, in the S1_ berlin, selecting blocks corresponding to the L2_ berlin to form a set S2_ berlin, and similarly, obtaining sets S2_ paris, S2_ hong _ kong, S2_ rome and S2_ sao _ paulo by S1_ paris, S1_ hong _ kong, S1_ rome and S1_ sao _ paulo respectively, and forming a training data set S by S2_ berlin, S2_ paris, S2_ hong _ kong, S2_ rome and S2_ sao _ paulo;
the S1_ berlin is disturbed in the same way as the L1_ berlin, the first 10% of blocks of each type are selected to form a set S2_ berlin, and similarly, the sets S2_ paris, S2_ hong _ kong, S2_ rome and S2_ sao _ paulo are obtained by S1_ paris, S1_ hong _ kong, S1_ rock and S1_ sao _ paulo respectively, and a training data set S is formed by S2_ berlin, S2_ paris, S2_ hong _ kong, S2_ rock and S2_ sao _ paulo;
step 7, constructing a dual-channel convolution ladder network multi-source remote sensing image ground feature classification model, comprising the following steps of:
(7a) constructing a network encoder:
the convolution-based encoder contains a clean part and a lossy part, both of which share a set of parameters, and the lossy part adds 0 standard deviation 0.3 gaussian noise to each layer. The structure of the encoder is as follows: input layer → first convolution layer → second convolution layer → third convolution layer → fourth convolution layer → softmax classifier, each layer parameter is as follows:
first layer input layer: setting the number of feature maps to be 9 for the first channel and setting the number of feature maps to be 10 for the second channel;
a second first winding layer: the number of feature maps of the first channel and the second channel is 100;
third layer second convolution layer: the number of feature maps of the first channel and the second channel is 100;
fourth layer third convolutional layer: the number of feature maps of the first channel and the second channel is 17;
fifth layer fourth buildup layer: setting the number of feature maps to be 17;
a sixth-level softmax classifier: setting the number of feature maps to be 17;
(7b) constructing a decoder:
the decoder reconstructs the lossy part of the encoder for each layer from output to input in turn, the reconstruction function is as follows:
Figure BDA0001349677340000081
Figure BDA0001349677340000082
denotes the i (i ═ 1, 2. -, m) th layer of the decoderl) The output of each neuron, wherein mlRepresents the number of neurons of the l-th layer,
Figure BDA0001349677340000083
represents the output of the ith neuron of the l-th layer of the lossy part of the encoder,
Figure BDA0001349677340000084
to represent
Figure BDA0001349677340000085
The weight of (a) is determined,
Figure BDA0001349677340000086
it is indicated that a priori,
Figure BDA0001349677340000087
is a noise reduction function, where i, l are both positive integers;
(7c) constructing a loss function:
Figure BDA0001349677340000088
Ccis a cross-entropy loss function representing the loss of the supervised part, t (n) represents class labels,
Figure BDA0001349677340000089
is the output of the lossy part of the encoder, x1(n) is the input of the first channel in the encoder, x2(n) is an input to a second channel in the encoder. CdDenotes the loss of the unsupervised part, λlRepresenting the weight of the l layers of reconstruction errors, N representing the number of samples input, mlNumber of neurons in layer I, z(l)Represents the output of the ith layer of the clean part of the encoder,
Figure BDA00013496773400000810
representing the output of the l-th layer reconstructed by the decoder.
Step 8, training the dual-channel convolution ladder network multisource remote sensing image ground object classification model by using a training data set L and a training data set S to obtain a trained model;
respectively taking the training data sets L and S as the input of a first channel and a second channel in the dual-channel convolution ladder network multi-source remote sensing image ground feature classification model, taking the category of each pixel point in the training data sets as the output of the dual-channel convolution ladder network multi-source remote sensing image ground feature classification model, optimizing the network parameters of the classification model by solving the error between the category and the correct category of the artificial mark and performing back propagation on the error to obtain the trained classification model, wherein the correct category of the artificial mark is shown in figure 2.
And 9, classifying the test data sets L1_ berlin, L1_ paris, L1_ hong _ kong, L1_ round, L1_ sao _ paulo, S1_ bnerlin, S1_ paris, S1_ hong _ kong, S1_ round and S1_ sao _ paulo by using the trained model to obtain the output of the model corresponding to each pixel point in the test data sets.
The test data sets L1_ berlin, L1_ paris, L1_ hong _ kong, L1_ ring and L1_ sao _ paulo are used as the input of a first channel of a trained dual-channel convolution ladder network multi-source remote sensing image ground feature classification model, S1_ berlin, S1_ paris, S1_ hong _ kong, S1_ ring and S1_ sao _ paulo are used as the input of a second channel of the trained dual-channel convolution ladder network ground feature classification model, and the output of the model is a classification category obtained by classifying each pixel point in the test data sets of five cities.
The effect of the invention can be further illustrated by the following simulation experiment:
1. simulation conditions are as follows:
the hardware platform is as follows: HP-Z820.
The software platform is as follows: tensorflow.
2. Simulation content and results:
the experiment is carried out under the simulation condition by using the method, namely 10% of pixel points are randomly selected as training samples from multispectral data of five cities of Berlin, Paris, hong Kong, Roman and St Paul obtained from a landsat-8 sensor and a sentinel-2 sensor, and all the other marked pixel points are used as test samples to obtain the classification result shown in the figure 3. The data used in this experiment were classified into 16 types, which were: 1-dense high-rise buildings, 2-dense medium-rise buildings, 3-dense low-rise buildings, 4-open high-rise buildings, 5-open medium-rise buildings, 6-open low-rise buildings, 8-large low-rise buildings, 9-sparsely distributed buildings, 10-heavy industrial areas, 11-dense forests, 12-scattered trees, 13-shrubs and dwarf trees, 14-low vegetation, 15-bare rock, 16-bare soil and sandy soil, 17-water.
As can be seen from fig. 3: except for a few of wrongly-divided spots, each part has a good classified result of the classified mark area, the classified result of the water area is the best, the wrongly-divided spots exist in the classified result due to the similar characteristics of various buildings, and the classified mark areas of each part have good classified results.
The size of the training sample without the class marks is fixed to be 10 percent (about 8000 samples), the number of the samples with the class marks in the training sample is changed to make the samples with the class marks account for 10 percent, 5 percent and 3 percent of the total number, the classification precision of the test data set of the convolution ladder network with the single channel is compared, and the result is shown in a table 1:
TABLE 1
Figure BDA0001349677340000101
As can be seen from Table 1, when the training samples contain the class standard samples accounting for 10%, 5% and 3% of the total labeled samples, the classification accuracy of the test data set of each city is higher than the classification accuracy obtained by classifying the data of the landsat-8 or sentinel-2 sensor by using the single-channel convolution ladder network.
In conclusion, the multi-source remote sensing image is fused by using the two-channel convolution ladder network, so that classification is realized, more accurate and complete information is extracted, the expression capability of image characteristics is effectively improved, the generalization capability of a model is enhanced, and higher ground feature classification accuracy can be still achieved under the condition of few training samples.

Claims (8)

1. The multi-source remote sensing image ground object classification method based on the two-channel convolution ladder network is characterized by comprising the following steps of:
step one, performing normalization processing on multispectral data of a plurality of areas to be detected, which are obtained by a landsat-8 sensor, to obtain normalized multispectral data, and marking the normalized multispectral data as landsat _ A, landsat _ B, landsat _ C, … … and landsat _ N;
step two, the multispectral data of a plurality of regions to be detected obtained by the sentinel-2 sensor are normalized to obtain normalized multispectral data which are marked as sentinel _ A, sentinel-B, sentinel _ C, … … and sentinel _ N;
thirdly, taking a peripheral 28 multiplied by 28 block in each element in the landsat _ A to represent an original element value to form a feature matrix L1_ A based on the image block, and similarly, obtaining feature matrices L1_ B, L1_ C, … … and L1_ N;
step four, each element in the sentinel _ A takes a peripheral 28 multiplied by 28 block to represent an original element value to form a characteristic matrix S1_ A based on the image block, and similarly, characteristic matrices S1_ B, S1_ C, … … and S1_ N are obtained;
step five, in a feature matrix L1_ A, randomly selecting a plurality of blocks of each type to form a set L2_ A, similarly, obtaining sets L2_ B, L2_ C, … … and L2_ N respectively from L1_ B, L1_ C, … … and L1_ N, and forming a training data set L from L2_ A, L2_ B, L2_ C, … … and L2_ N;
step six, selecting blocks corresponding to the blocks in the L2_ A from the feature matrix S1_ A to form a set S2_ A, and similarly, obtaining sets S2_ B, S2_ C, … … and S2_ N from S1_ B, S1_ C, … … and S1_ N respectively, and forming a training data set S from S2_ A, S2_ B, S2_ C, … … and S2_ N;
step seven, constructing a two-channel convolution ladder network multi-source remote sensing image ground object classification model, wherein the specific method comprises the following steps:
first, a network encoder is constructed:
the encoder based on convolution contains clean part and lossy part, and two parts share a set of parameter, and the gaussian noise that the average is 0 standard deviation and is 0.3 is added to each layer of lossy part, and the structure of encoder is: input layer → first convolution layer → second convolution layer → third convolution layer → fourth convolution layer → softmax classifier;
secondly, constructing a decoder:
the decoder reconstructs the lossy part of the encoder for each layer from output to input in turn, the reconstruction function is as follows:
Figure FDA0002355151760000021
Figure FDA0002355151760000022
represents the output of the ith neuron of the ith layer of the decoder, i 1,2lWherein m islRepresents the number of neurons of the l-th layer,
Figure FDA0002355151760000023
represents the output of the ith neuron of the l-th layer of the lossy part of the encoder,
Figure FDA0002355151760000024
to represent
Figure FDA0002355151760000025
The weight of (a) is determined,
Figure FDA0002355151760000026
it is indicated that a priori,
Figure FDA0002355151760000027
is a noise reduction function, where i, l are both positive integers;
thirdly, constructing a loss function:
Figure FDA0002355151760000028
Ccis a cross-entropy loss function representing the loss of the supervised part, t (n) represents class labels,
Figure FDA0002355151760000029
is the output of the lossy part of the encoder, x1(n) is the input of the first channel in the encoder, x2(n) is the input of a second channel in the encoder, CdDenotes the loss of the unsupervised part, λlRepresenting the weight of the l layers of reconstruction errors, N representing the number of samples input, mlNumber of neurons in layer I, z(l)Represents the output of the ith layer of the clean part of the encoder,
Figure FDA00023551517600000210
represents the output of the l-th layer reconstructed by the decoder;
step eight, training the multi-source image ground feature classification model by using a training data set L and a training data set S to obtain a trained model;
and step nine, classifying the test data sets L1_ A, L1_ B, L1_ C, … …, L1_ N, S1_ A, S1_ B, S1_ C, … … and S1_ N by using the trained models to obtain the output of the model corresponding to each pixel point in the test data sets.
2. The multi-source remote sensing image terrain classification method based on the two-channel convolution ladder network as claimed in claim 1, wherein in the first step and the second step, ENVI software is used for carrying out normalization processing on data, and the selected normalization mode is equal.
3. The method for classifying land features of multi-source remote sensing images based on the dual-channel convolution ladder network as claimed in claim 1, wherein in the third step, the specific method for obtaining feature matrices L1_ A, L1_ B, L1_ C, … … and L1_ N is as follows:
the normalized feature matrix landsat _ a is used to take a block of 28 × 28 pixels around each element to represent the original element value, so that the class label of the block is still the class label of the original element value, and since the feature matrix is 9-dimensional, the size of each block is 28 × 28 × 9, so that the feature matrix L1_ a based on the image block is formed, and similarly, the feature matrices L1_ A, L1_ B, L1_ C, … … and L1_ N are obtained.
4. The method for classifying land features of multi-source remote sensing images based on the dual-channel convolution ladder network as claimed in claim 1, wherein in the fourth step, the specific method for obtaining feature matrices S1_ A, S1_ B, S1_ C, … … and S1_ N is as follows:
the normalized feature matrix sentinel _ a is used to take a block of 28 × 28 pixels around each element to represent the original element value, so that the class label of the block is still the class label of the original element value, and since the feature matrix is 10-dimensional, the size of each block is 28 × 28 × 10, thereby forming the feature matrix S1_ a based on image blocks, and similarly, the feature matrices S1_ A, S1_ B, S1_ C, … … and S1_ N are obtained.
5. The multi-source remote sensing image surface feature classification method based on the two-channel convolution ladder network according to claim 1, characterized in that in the fifth step, a specific method for forming a training data set L is as follows:
in L1_ A, samples are randomly disturbed, the first 10% of blocks of each type are selected to form a set L2_ A, similarly, sets L2_ B, L2_ C, … … and L2_ N are respectively obtained from L1_ B, L1_ C, … … and L1_ N, and a training data set L is formed from L2_ A, L2_ B, L2_ C, … … and L2_ N.
6. The multi-source remote sensing image surface feature classification method based on the two-channel convolution ladder network according to claim 1 is characterized in that in the sixth step, a specific method for forming a training data set S is as follows:
s1_ A is disordered in the same way as L1_ A, the first 10% of blocks of each type are selected to form a set S2_ A, similarly, sets S2_ B, S2_ C … … and S2_ N are obtained from S1_ B, S1_ C, … … and S1_ N respectively, and a training data set S is formed by S2_ A, S2_ B, S2_ C, S … … and S2_ N.
7. The multi-source remote sensing image surface feature classification method based on the two-channel convolution ladder network as claimed in claim 1, wherein in the eighth step, a specific method for obtaining the trained model is as follows:
and respectively taking the training data set L and the training data set S as the input of a first channel and a second channel in the multi-source remote sensing image surface feature classification model, taking the category of each pixel point in the training data set as the output of the multi-source image surface feature classification model, and optimizing the network parameters of the classification model by solving the error between the category and the correct category marked manually and performing back propagation on the error to obtain the trained classification model.
8. The multi-source remote sensing image ground object classification method based on the two-channel convolution ladder network according to claim 1 is characterized in that in the ninth step, a specific method for obtaining the output of the model corresponding to each pixel point in the test data set is as follows:
the test data sets L1_ A, L1_ B, L1_ C, … … and L1_ N serve as input of a first channel of a trained multi-source remote sensing image ground feature classification model, S1_ A, S1_ B, S1_ C, … … and S1_ N serve as input of a second channel of the trained multi-source remote sensing image ground feature classification model, and output of the model is a classification category obtained by classifying each pixel point in the test data set of the region to be classified.
CN201710571057.0A 2017-07-13 2017-07-13 Multi-source remote sensing image ground object classification method based on two-channel convolution ladder network Active CN107273938B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710571057.0A CN107273938B (en) 2017-07-13 2017-07-13 Multi-source remote sensing image ground object classification method based on two-channel convolution ladder network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710571057.0A CN107273938B (en) 2017-07-13 2017-07-13 Multi-source remote sensing image ground object classification method based on two-channel convolution ladder network

Publications (2)

Publication Number Publication Date
CN107273938A CN107273938A (en) 2017-10-20
CN107273938B true CN107273938B (en) 2020-05-29

Family

ID=60071838

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710571057.0A Active CN107273938B (en) 2017-07-13 2017-07-13 Multi-source remote sensing image ground object classification method based on two-channel convolution ladder network

Country Status (1)

Country Link
CN (1) CN107273938B (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107944483B (en) * 2017-11-17 2020-02-07 西安电子科技大学 Multispectral image classification method based on dual-channel DCGAN and feature fusion
CN107832797B (en) * 2017-11-17 2020-04-07 西安电子科技大学 Multispectral image classification method based on depth fusion residual error network
CN107967454B (en) * 2017-11-24 2021-10-15 武汉理工大学 Double-path convolution neural network remote sensing classification method considering spatial neighborhood relationship
CN108022222A (en) * 2017-12-15 2018-05-11 西北工业大学 A kind of thin cloud in remote sensing image minimizing technology based on convolution-deconvolution network
CN108846829B (en) * 2018-05-23 2021-03-23 平安科技（深圳）有限公司 Lesion site recognition device, computer device, and readable storage medium
CN108985365B (en) * 2018-07-05 2021-10-01 重庆大学 Multi-source heterogeneous data fusion method based on deep subspace switching ensemble learning
CN111291826B (en) * 2020-02-25 2023-06-06 西安电子科技大学 Pixel-by-pixel classification method of multisource remote sensing image based on correlation fusion network
CN111507388A (en) * 2020-04-10 2020-08-07 上海眼控科技股份有限公司 Weather image classification method and equipment
WO2021226977A1 (en) * 2020-05-15 2021-11-18 安徽中科智能感知产业技术研究院有限责任公司 Method and platform for dynamically monitoring typical ground features in mining on the basis of multi-source remote sensing data fusion and deep neural network
CN113837209A (en) * 2020-06-23 2021-12-24 乐达创意科技股份有限公司 Method and system for improved machine learning using data for training
CN112749670B (en) * 2021-01-18 2023-09-05 西安电子科技大学 Pixel-by-pixel classification method, medium and equipment for multi-source remote sensing image

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104021396A (en) * 2014-06-23 2014-09-03 哈尔滨工业大学 Hyperspectral remote sensing data classification method based on ensemble learning
CN105095864A (en) * 2015-07-16 2015-11-25 西安电子科技大学 Aurora image detection method based on deep learning two-dimensional principal component analysis network
CN106156793A (en) * 2016-06-27 2016-11-23 西北工业大学 Extract in conjunction with further feature and the classification method of medical image of shallow-layer feature extraction
CN106203523A (en) * 2016-07-17 2016-12-07 西安电子科技大学 The classification hyperspectral imagery of the semi-supervised algorithm fusion of decision tree is promoted based on gradient
CN106709421A (en) * 2016-11-16 2017-05-24 广西师范大学 Cell image recognition and classification method based on transform domain characteristics and CNN (Convolutional Neural Network)
CN106934419A (en) * 2017-03-09 2017-07-07 西安电子科技大学 Classification of Polarimetric SAR Image method based on plural profile ripple convolutional neural networks

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7634137B2 (en) * 2005-10-14 2009-12-15 Microsoft Corporation Unfolded convolution for fast feature extraction

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104021396A (en) * 2014-06-23 2014-09-03 哈尔滨工业大学 Hyperspectral remote sensing data classification method based on ensemble learning
CN105095864A (en) * 2015-07-16 2015-11-25 西安电子科技大学 Aurora image detection method based on deep learning two-dimensional principal component analysis network
CN106156793A (en) * 2016-06-27 2016-11-23 西北工业大学 Extract in conjunction with further feature and the classification method of medical image of shallow-layer feature extraction
CN106203523A (en) * 2016-07-17 2016-12-07 西安电子科技大学 The classification hyperspectral imagery of the semi-supervised algorithm fusion of decision tree is promoted based on gradient
CN106709421A (en) * 2016-11-16 2017-05-24 广西师范大学 Cell image recognition and classification method based on transform domain characteristics and CNN (Convolutional Neural Network)
CN106934419A (en) * 2017-03-09 2017-07-07 西安电子科技大学 Classification of Polarimetric SAR Image method based on plural profile ripple convolutional neural networks

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Semi-Supervised Learning with Ladder Networks;Rasmus, Antti.etc;《29th Annual Conference on Neural Information Processing Systems》;20151212;第1-9页 *
深度学习的半监督遥感图像检索;张洪群等;《遥感学报》;20160925;第406-414页 *

Also Published As

Publication number Publication date
CN107273938A (en) 2017-10-20

Similar Documents

Publication Publication Date Title
CN107273938B (en) Multi-source remote sensing image ground object classification method based on two-channel convolution ladder network
CN111259905B (en) Feature fusion remote sensing image semantic segmentation method based on downsampling
CN111914907B (en) Hyperspectral image classification method based on deep learning space-spectrum combined network
Pan et al. Land-cover classification of multispectral LiDAR data using CNN with optimized hyper-parameters
Saito et al. Multiple object extraction from aerial imagery with convolutional neural networks
CN110852225A (en) Remote sensing image mangrove forest extraction method and system based on deep convolutional neural network
CN111259828A (en) High-resolution remote sensing image multi-feature-based identification method
CN107832797B (en) Multispectral image classification method based on depth fusion residual error network
Cai et al. Residual-capsule networks with threshold convolution for segmentation of wheat plantation rows in UAV images
CN109344891A (en) A kind of high-spectrum remote sensing data classification method based on deep neural network
Tambe et al. Deep multi-feature learning architecture for water body segmentation from satellite images
CN113920442B (en) Hyperspectral classification method combining graph structure and convolutional neural network
CN110807485B (en) Method for fusing two-classification semantic segmentation maps into multi-classification semantic map based on high-resolution remote sensing image
CN113887472B (en) Remote sensing image cloud detection method based on cascade color and texture feature attention
CN107516103A (en) A kind of image classification method and system
CN116091940B (en) Crop classification and identification method based on high-resolution satellite remote sensing image
CN111738052B (en) Multi-feature fusion hyperspectral remote sensing ground object classification method based on deep learning
CN111640087B (en) SAR depth full convolution neural network-based image change detection method
Abdollahi et al. Short-time-series grassland mapping using Sentinel-2 imagery and deep learning-based architecture
CN114612883A (en) Forward vehicle distance detection method based on cascade SSD and monocular depth estimation
Du et al. Training SegNet for cropland classification of high resolution remote sensing images
Qiao et al. Rotation is all you need: Cross dimensional residual interaction for hyperspectral image classification
CN106971402B (en) SAR image change detection method based on optical assistance
Pan et al. Enhanced FCN for farmland extraction from remote sensing image
Lopez et al. Convolutional neural networks for semantic segmentation of multispectral remote sensing images

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant