CN110516596A - Empty spectrum attention hyperspectral image classification method based on Octave convolution - Google Patents
Empty spectrum attention hyperspectral image classification method based on Octave convolution Download PDFInfo
- Publication number
- CN110516596A CN110516596A CN201910797299.0A CN201910797299A CN110516596A CN 110516596 A CN110516596 A CN 110516596A CN 201910797299 A CN201910797299 A CN 201910797299A CN 110516596 A CN110516596 A CN 110516596A
- Authority
- CN
- China
- Prior art keywords
- octave
- convolutional neural
- neural networks
- loss
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/194—Terrestrial scenes using hyperspectral data, i.e. more or other wavelengths than RGB
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Molecular Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
A kind of empty spectrum attention hyperspectral image classification method based on Octave convolution disclosed by the invention, solves the problems, such as that prior art the same category spacing is big, different classes of spacing is small, classification accuracy is low.Scheme is: image to be classified input builds with data prediction, division training set and test set, Octave convolutional neural networks, determines Octave convolutional neural networks loss function, the training of Octave convolutional neural networks updates, test set data test, completes classification hyperspectral imagery.The present invention strengthens character representation using Octave convolution operation, introduces spatial attention mechanism and spectrum attention mechanism, finds network more accurately for classifying more advantageous and including the more comprehensive detailed region of information.Nicety of grading of the present invention is high, and strong robustness can be applied to the analysis and management of hyperspectral image data.
Description
Technical field
The invention belongs to technical field of image processing, are related specifically to the classifying content of high spectrum image, specifically a kind of
Spatial spectral attention hyperspectral image classification method based on Octave convolution, can be applied to the analysis of hyperspectral image data
And management.
Background technique
As the resolution ratio of high spectrum image pixel is continuously improved, can be obtained from high spectrum image more useful
Data and information.And it is directed to the demand of different application, also there is different requirements to the processing of high spectrum image, so in order to have
Effect ground is analyzed and is managed to these hyperspectral image data, needs to stick semanteme for each pixel of high spectrum image
Label.And classification hyperspectral imagery is exactly a kind of important channel for solving the problems, such as such.What classification hyperspectral imagery referred to be exactly from
The pixel with similar features is distinguished in one high spectrum image, and is correctly classified to these pixels.It compares
In natural image, high spectrum image itself has the characteristics of there is itself, sky of the classification results due to high spectrum image itself
Between resolution ratio limitation and the different spectrum of jljl, foreign matter with spectrum phenomenon presence, usually will cause the phenomenon that mistake is divided, this is by bloom
Caused by the complexity of spectrogram picture itself.Therefore, how more accurately high spectrum image progress Accurate classification to be also become and is worked as
A preceding huge challenge.
Classification based on convolutional neural networks, refer to will need training some data, in batches be input to convolution mind
In network, by the repetition training of high-volume data, so that objective optimization loss function constantly reduces, to realize classification
Purpose.Nowadays there are many more mature, famous convolutional neural networks to be suggested, such as mentioned by He Kaiming et al. within 2015
The depth residual error convolutional neural networks for image classification task out, are widely used by everybody.Depth residual error convolutional Neural net
Network effective solution image classification task extracted feature includes the problem of information is insufficient, network training gradient disappears, but
This network there are still because image data it is complicated caused by the same category spacing is big, different classes of spacing is small, image classification is quasi-
The true low problem of rate and because training sample it is few caused by network robustness is lower, is easily trapped into the problem of over-fitting.
Although existing convolutional neural networks can be realized the task of EO-1 hyperion Pixel-level classification, but in study image language
The deficiency of three aspects is still had when adopted information: first is that the positioning of the classification information as caused by high spectrum image complexity is not
Accurately, it carries out causing same category of spacing larger when classification task, different classes of spacing is smaller；Second is that for EO-1 hyperion spy
When sign is extracted, causes information to lose or retain excessive irrelevant information the utilization rate deficiency of the feature of extraction and cause information superfluous
It is remaining, classification results are influenced, while rolling up neural network can usually fall into local optimum region in training；Third is that available bloom
Modal data is fewer, and training convolutional neural networks usually require a large amount of training data, and a small amount of high-spectral data cannot expire
The data requirements of sufficient convolutional neural networks.These three deficiencies, which will lead to, there is robust in the assorting process of practical high spectrum image
Property it is poor and lead to the problem of mistake and divide.
Summary of the invention
Present invention aims at above-mentioned prior art there are aiming at the problem that, propose that a kind of classification accuracy is higher and be based on
The hyperspectral image classification method of the empty spectrum attention mechanism deep learning of Octave convolution.
The present invention is a kind of classification hyperspectral imagery side of empty spectrum attention mechanism deep learning based on Octave convolution
Method, which is characterized in that comprise the following steps that
(1) image input and data prediction: high spectrum image to be sorted is inputted, is carried out centered on each pixel
Point sliding pixel-by-pixel, all image blocks slided are for establishing high spectrum image library { I1,I2,…,In,…,IN, image
The corresponding classification of each image block is { Y in library1,Y2,…,Yn,…,YN, and normalizing is carried out to the high spectrum image library of foundation
Change is handled, wherein InN-th image block, Y in representative image librarynThe corresponding classification of n-th image block in representative image library, n are represented
N-th sample number in image library, n ∈ [0, N], N represent the image block total number in high spectrum image library；
(2) training set and test set are divided: selecting specified number at random from every class high spectrum image after normalized
The high spectrum image sample of amount constructs training sample set { T1,T2,…,Tj,…,TM, using remaining high spectrum image as survey
Try sample set { t1,t2,…td,…,tm, wherein TjIndicate j-th of sample in training sample, j ∈ [0, M], tdIndicate test specimens
D-th of sample in this, d ∈ [0, m], M are the total number of training sample, and m is the total number of test sample, m < N, M < N；
(3) Octave convolutional neural networks are built: building an Octave convolutional neural networks, the input terminal of network is
Octave convolution module, the output end of network are the output of full articulamentum as a result, wrapping between the input terminal and output end of network
Containing two branches, wherein a branch successively passes through space transforms power module and Pixel-level pays attention to power module, another branch according to
It is secondary to notice that power module and Pixel-level pay attention to power module by spectrum；
(4) Octave convolutional neural networks loss function loss is determinedop: setting loss function includes that will pass through Fusion Features
The feature extracted afterwards is input to the cross entropy loss of full articulamentum output category result and actual result obtained from1, will be through
It crosses the feature that space transforms power module and Pixel-level notice that power module is extracted and is input to full articulamentum output category obtained from
As a result with the cross entropy loss of actual result2, by by spectrum pay attention to power module and Pixel-level pay attention to power module extract feature
It is input to the cross entropy loss of full articulamentum output category result and actual result obtained from3With the convolution for having hyper parameter
Four part of L2 norm of neural network weight W, the loss function of network is successively added by above four part to be constituted；
(5) training updates: the number of iterations that network training is arranged is P, by gradient decline optimization to Octave convolution mind
It is iterated training through network, until loss function lossopDo not decline or exercise wheel number reaches the number of iterations, is trained
Octave convolutional neural networks；
(6) test sample collection after normalized data test: is input to trained Octave convolution mind
In network, classification results are obtained, complete image classification.
Present invention employs Octave convolution operations, and have incorporated attention mechanism, and being provided with one includes Octave volumes
The depth convolutional network model of product operation and attention mechanism, this model only need a small amount of training data that can train effect
The high frequency section of hyperspectral image data and low frequency part have been carried out effective knot by the preferable model of fruit, Octave convolution operation
It closes, the information that the feature of extraction includes is more comprehensively in detailed, and utilization rate is higher, while attention mechanism can promote network faster
Speed effectively finds characteristic area more favorable for classification task, keeps the information of network capture more comprehensively accurate, effectively solves
The problem that classification accuracy is lower, robustness is not strong in current classification hyperspectral imagery task of having determined.
The present invention has the advantage that compared with prior art
Character representation enhancing: Octave convolution operation is introduced into hyperspectral image classification method for the first time by the present invention,
It is specially equipped with Octave convolution module in Octave convolutional neural networks, obtains height by being then based on Octave convolution operation
Spectrum picture feature, the high-frequency information of high spectrum image and low-frequency information be rationally effectively combined together with, make height
The information that spectrum picture feature includes is more comprehensively and detailed, enhances the character representation of image.
Classification accuracy improves: attention mechanism is introduced into hyperspectral image classification method, In by the present invention
Spatial attention mechanism module, spectrum attention mechanism module and Pixel-level is specially equipped in Octave convolutional neural networks to pay attention to
Power mechanism module can promote network fast and accurately to find the high-spectrum obtained due to introducing attention mechanism principle
As feature region the most apparent in feature, make more to concentrate on more favorable feature of classifying a certain with obvious semantic
The region of information reduces the probability that loss function falls into local optimum, enhances the accuracy of classification hyperspectral imagery.
Robustness enhancing: the present invention devises a more effective loss function, and new loss function utilizes three intersections
Entropy loss function promotes e-learning to the more effective feature of classification hyperspectral imagery, strengthens the character representation of image, into
One step specifies classification task, and purpose is stronger, can adapt to complicated hyperspectral image data, greatly enhances network
Robustness.
Training sample is few: present invention only requires a small amount of samples can train the preferable network model of effect, to height
The data volume of spectrum picture requires smaller.
Detailed description of the invention
Fig. 1 is implementation flow chart of the invention；
Fig. 2 is the Octave convolutional neural networks structure chart constructed in the present invention；
Fig. 3 is the high spectrum image of experiment in the present invention, and wherein Fig. 3 (a) is original high spectrum image, and Fig. 3 (b) is
The class label of original high spectrum image corresponding pixel points；
Fig. 4 is Octave convolution module structure chart；
Fig. 5 is spatial attention mechanism module structure chart；
Fig. 6 is attention mechanism module structure chart between spectrum；
Fig. 7 is Pixel-level attention mechanism module structure chart.
Specific embodiment
Technical solutions and effects of the present invention is described in detail below in conjunction with attached drawing.
Embodiment 1
In recent decades, high spectral resolution provided useful information, EO-1 hyperion to distinguish different materials and object
Image classification method is widely used in earth observation, especially in urban development, precision agriculture, land change inspection
Survey, resource management etc. have great importance.Presently, there are classification hyperspectral imagery method in, referring to Fig. 1, pass through
Building high spectrum image library, divide training sample set and test sample collection, build and training convolutional neural networks model, to training
Good convolutional neural networks model carries out the classification tested etc. Wan Chengs for high spectrum image, but due to the complexity of high spectrum image
The problem of property, leads to classification information position inaccurate, and trained network is easily ensnared into local optimum；Simultaneously because network is deep
For degree compared with deep, network operation is more complicated, the feature of extraction includes that information is insufficient, leads to carry out same category occur when classification task
Larger, the different classes of small-pitch problem of spacing, the phenomenon that being easy to appear mistake classification, robustness is poor.
For this status, the present invention expands research and discussion, proposes that a kind of empty spectrum based on Octave convolution pays attention to
The hyperspectral image classification method of power mechanism deep learning comprises the following steps that referring to fig. 2
(1) image input and data prediction: inputting high spectrum image to be sorted, high spectrum image spectral band compared with
It is more, and different-waveband differs greatly, while being influenced by environmental factors such as illumination, temperature, and same category of EO-1 hyperion is belonged to
Image differs greatly, and the high spectrum image difference to belong to a different category is little；For hyperspectral image data, in high-spectrum
It is put sliding pixel-by-pixel centered on each pixel as on, the sliding sash of 9*9 size is chosen in this example, sliding sash size can basis
Actual conditions are adjusted, and are carried out point pixel-by-pixel using sliding sash and are slided, the distance slided every time is a pixel, is slided every time
The high spectrum image block of an all available 9*9 size, all high spectrum image blocks slided are for establishing high-spectrum
As library { I1,I2,…,In,…,IN, the corresponding classification of each image block is respectively { Y in image library1,Y2,…,Yn,…,YN},
For established high spectrum image library, the maximum value and minimum value of all pixels point in image library are found, all pixels are utilized
The high spectrum image library of foundation is normalized in the value of point and the most value of two pixels, wherein InIn representative image library
N-th image, YnThe corresponding classification of n-th image in representative image library, n-th of sample number in n representative image library, n ∈ [0,
N], N represents the picture total number in high spectrum image library.
(2) training set and test set are divided: being selected at random from the high spectrum image of each classification after normalized
The high spectrum image sample of specified quantity constructs training sample set { T1,T2,…,Tj,…,TM, abbreviation training set will be remaining
High spectrum image is as test sample collection { t1,t2,…td,…,tmThis test sample collection be normalized test sample collection,
Abbreviation test set.Training sample concentrates TjIndicate j-th of sample in training sample, j ∈ [0, M], test sample concentrates tdIt indicates to survey
D-th of sample in sample sheet, d ∈ [0, m], M are the total number of training sample, and m is the total number of test sample, m < N, M < N.
The mode selected at random is respectively adopted for each classification when selecting sample building training sample set by the present invention,
The training sample set chosen in this way may include the sample of all categories, while the institute for being included by each classification of maximum possible
It is possible that property sample is all divided among training sample set.
When constructing high spectrum image training sample set, the training sample for randomly selecting specified quantity is usually art technology
The Normal practice of personnel, is primarily due in hyperspectral image data, and high spectrum image sample is few and every other sample of type
Quantitative difference it is larger, if dividing training sample set and test sample collection in proportion will lead to point of the few EO-1 hyperion classification of sample
Class accuracy rate is very low.
(3) Octave convolutional neural networks are built: build an Octave convolutional neural networks, referring to fig. 2, network it is defeated
Entering end is Octave convolution module, the output end of network be the output of full articulamentum as a result, network input terminal and output end
Between include two branches, wherein a branch successively passes through space transforms power module and Pixel-level pays attention to power module, another
Branch successively passes through spectrum and notices that power module and Pixel-level pay attention to power module.
Octave convolution module of the present invention has fully considered the high-frequency information and low-frequency information of high spectrum image, passes through convolution
The high-frequency information of high spectrum image and low-frequency information are effectively combined by operation, are enhanced the character representation of image, are made
The information that characteristics of image includes is more comprehensive；Space transforms power module and Pixel-level notice that power module makes the attention of whole network
Concentrate on a certain region with obvious semantic information, make whole network during to hyperspectral classification, make full use of for
Classify best region, catches most apparent characteristic area, improve the classification accuracy of EO-1 hyperion；Spectrum pay attention to power module and
Pixel-level notices that power module makes whole network focus on spectral band the most useful for classification hyperspectral imagery, dashes forward
The more good spectral band of character representation out is conducive to improve hyperspectral classification accuracy rate, improves the robustness of network.
(4) Octave convolutional neural networks loss function loss is determinedop: setting loss function includes that will pass through Fusion Features
The feature extracted afterwards is input to the cross entropy loss of full articulamentum output category result and actual result obtained from1, will be through
It crosses the feature that space transforms power module and Pixel-level notice that power module is extracted and is input to full articulamentum output category obtained from
As a result with the cross entropy loss of actual result2, by by spectrum pay attention to power module and Pixel-level pay attention to power module extract feature
It is input to the cross entropy loss of full articulamentum output category result and actual result obtained from3With the convolution for having hyper parameter
Four part of L2 norm of neural network weight W, the loss function of network is successively added by above four part to be constituted.
Loss function loss of the inventionopPromote e-learning to classification hyperspectral imagery using three intersection entropy functions
More effective feature reinforces the character representation of high spectrum image, has further clarified classification task, and purpose is stronger, Neng Goushi
Complicated hyperspectral image data is answered, greatly enhances the robustness of network, while effectively reducing loss function in net
The probability of local optimum is fallen into network training process.
(5) training updates: the number of iterations that network training is arranged is P, by gradient decline optimization to Octave convolution mind
It is iterated training through network, until loss function lossopDo not decline or exercise wheel number reaches the number of iterations, is trained
Octave convolutional neural networks.The number of iterations P is to be manually set, and can be adjusted according to the training effect of network, make height
Spectrum picture classification accuracy highest.In network training process, the learning rate of network is gradually reduced with the training of network, just
Learning rate is larger when beginning to train, and with trained intensification, learning rate is gradually reduced, and network is made to effectively reduce the loss of network
Function falls into the probability of local optimum, is conducive to the raising of classification hyperspectral imagery accuracy rate, robustness enhancing.
(6) test sample collection after normalized data test: is input to trained Octave convolution mind
In network, classification results are obtained, complete image classification.
The high spectrum image point for the empty spectrum attention mechanism deep learning based on Octave convolution that the present invention provides one
The overall technical architecture of class method.
Technical thought of the invention is: building an Octave convolutional neural networks, is obtained using Octave convolution operation
For each pixel region, include the more comprehensive convolution feature of information；According to spatial attention mechanism principle and pixel
Grade attention mechanism principle, focuses on a certain region with obvious semantic information for the attention of whole network, finds and be conducive to
The region of the useful information of classification；According to spectrum attention mechanism principle and Pixel-level attention mechanism principle, feature is found more
For protrusion, the stronger spectral band of spectral information, the stronger character representation for being conducive to classification is obtained；Pass through above-mentioned attention mechanism
The attention of whole network is focused on for most effective region of classifying, full articulamentum network implementations image point is then passed through
Class.
The present invention solve present in current classification hyperspectral imagery the same category spacing is big, different classes of spacing is small,
The low problem of image classification accuracy rate and because training sample it is few caused by network robustness is lower, is easily trapped into over-fitting
Problem.
The present invention is effectively obtained by Octave convolution module comprising the more comprehensive convolution feature of information, is enhanced
It is more bright to notice that power module makes network find semantic information using space transforms power module and Pixel-level for the character representation of image
Aobvious, feature region outstanding, notices that power module and Pixel-level notice that power module finds the stronger spectrum of spectral information using spectrum
Wave band finds characteristic area more effective for classification hyperspectral imagery, improves height by the combination of several attention mechanism
The accuracy rate of spectrum picture classification, enhances the robustness of network.
Embodiment 2
The hyperspectral image classification method of empty spectrum attention mechanism deep learning based on Octave convolution with embodiment 1,
Octave convolutional neural networks described in step (3) of the present invention are built, and referring to fig. 2, the present invention forms Octave convolutional Neural
The Octave convolution module of network, space transforms power module, spectrum notice that power module, Pixel-level pay attention to power module and full connection
Layer respectively forms module parameter and is provided that
Octave convolution module, that is, input module is made of, each conventional part sequentially connected four conventional parts
It again include Octave convolution, referring to fig. 4, Batch Normalization and Relu activation primitive, in second and third convolution portion
/ there are one maximum pond layers.
High spectrum image is mainly first divided into two parts of high frequency section and low frequency part by Octave convolution operation, is passed through
Conventional convolution operation carries out convolution to high frequency section and low frequency part, obtains by high frequency to high frequency, high frequency to low frequency, low frequency to height
Frequently, low frequency is added to together to four convolution results of low frequency, then by the convolution results of high frequency to high and low frequency to high frequency, high frequency
It is added to together to low frequency and low frequency to the convolution results of low frequency, thus by the high-frequency information of high spectrum image and low-frequency information
Carried out effective connection and linked up, make the character representation obtained include information more comprehensively and in detail, enhance high-spectrum
The character representation of picture.
Spatial attention mechanism module activates letter by convolutional layer, Batch Normalization, Relu referring to Fig. 5
Number, matrix transposition are constituted with the layer that is multiplied, softmax layers and data transposition with layer is added.It will enter into spatial attention mechanism mould
The feature of block strengthens feature by a convolution operation, is input to matrix transposition and the layer that is multiplied, obtains a matrix, square
Each element represents the spatial relationship of any two position in high spectrum image in battle array；Recycle softmax layers to this matrix into
Row normalized normalizes to the value in matrix between 0 to 1, while by the matrix after normalization and passing through convolution operation
The feature obtained afterwards is multiplied, a certain region with obvious semantic information in prominent high spectrum image, obtains having obvious semantic
The characteristic pattern that information indicates；Finally by with obvious semantic information feature be input to the initial of spatial attention mechanism module
Feature be input to data transposition be added in layer, two features are added to the loss for preventing information together.By space transforms
The obtained last Feature Semantics information of power mechanism module is obvious, and network is made to be easily found the most apparent region of feature,
Improve the accuracy rate of classification hyperspectral imagery.
Spectrum attention mechanism module, referring to Fig. 6, by matrix transposition and the layer that is multiplied, softmax layers and data transposition with
Layer is added to constitute.The feature that will enter into spectrum attention mechanism module is input to matrix transposition and the layer that is multiplied, and obtains a square
Gust, each element represents the relationship in high spectrum image between the different spectral bands of any pixel in matrix；It recycles
Softmax layers are normalized this matrix, and the value in matrix is normalized between 0 to 1, while will be after normalization
Matrix notices that the initial characteristics of power module are multiplied with spectrum is input to, spectrum letter in each pixel in prominent high spectrum image
Strongest spectral band is ceased, the characteristic pattern of prominent most strong spectral information is obtained；Finally will the prominent most feature of strong spectral information with
Be input to spectrum pay attention to the initial characteristics of power module be input to data transposition be added layer, by two features be added to together, prevent
The only loss of information.The strongest spectrum wave of spectral information is highlighted by the last feature that spectrum attention mechanism module obtains
Section, makes network focus on the strongest region of spectral information, improves the accuracy rate of classification hyperspectral imagery, enhance net
The robustness of network.
Pixel-level attention mechanism module is activated referring to Fig. 7 by convolutional layer, Batch Normalization and Relu
Function is constituted.Classification hyperspectral imagery is classified to pixel each in high spectrum image, by Pixel-level attention machine
The feature that molding block obtains more has refined the feature of each pixel, enhances the character representation of each pixel, mentions
The high accuracy rate of classification hyperspectral imagery.
The full articulamentum is to be made of the first full articulamentum and the second full articulamentum and softmax layers, that is, exports
Layer.The input of first full articulamentum is the feature obtained after space transforms power module and Pixel-level pay attention to power module, passes through
Spectrum pays attention to the feature obtained after power module pays attention to power module with Pixel-level and obtains after being added fusion by two kinds of features
Fusion feature；The input of second full articulamentum is the output feature of the first full articulamentum；Softmax layers of input connects entirely for second
The output feature of layer is connect, softmax layers of output result indicates that training sample belongs to the probability of a certain classification in EO-1 hyperion,
Softmax layers of output result is the final output of whole network.
The present invention strengthens character representation using Octave convolution operation, introduces spatial attention mechanism and spectrum attention
Mechanism finds network more accurately for classifying more advantageous and including the more comprehensive detailed region of information, enhances height
The accuracy and network robustness of spectrum picture classification.
Embodiment 3
The hyperspectral image classification method of empty spectrum attention mechanism deep learning based on Octave convolution is the same as embodiment 1-
2, determination Octave convolutional neural networks loss function described in step 4 lossop, specifically comprise the following steps:
(4a) is by training image library { T1,T2,…,Tj,…,TMIt is input to Octave volumes of Octave convolutional neural networks
Volume module exports the last layer feature F of convolutional layer.
The last layer feature F is separately input to the space transforms power module and spectrum of Octave convolutional neural networks by (4b)
Pay attention to power module, output feature is respectively A and B, then output feature A and B are input to Pixel-level and pay attention to power module, exports feature
Respectively C and D.
The feature C and D that (4c) will be obtained, are input to the full articulamentum of Octave convolutional neural networks, and output utilizes feature C
The output category result obtained with D；Simultaneously then feature C and D are added pixel-by-pixel, are obtained respectively multiplied by a coefficient
Fused feature E, then feature E is input to the full articulamentum of Octave convolutional neural networks, output utilizes fused spy
The output category result that sign E is obtained, obtains the loss function loss of Octave convolutional neural networksop:
Wherein, loss1To utilize the friendship of fused feature E output category result and actual result after full articulamentum
Pitch entropy, loss2It is characterized the cross entropy of C output category result and actual result after full articulamentum, loss3It is characterized D process
The cross entropy of output category result and actual result after full articulamentum,For the L2 norm of convolutional neural networks weight vectors, η
ForHyper parameter.
Loss function loss of the inventionopPromote e-learning to classification hyperspectral imagery using three intersection entropy functions
More effective space characteristics and spectral signature, reinforce the character representation of high spectrum image, have further clarified classification task, purpose
Property it is stronger, can adapt to complicated hyperspectral image data, greatly enhance the robustness of network, effectively reduce simultaneously
Loss function falls into the probability of local optimum in network training process.
Embodiment 4
The hyperspectral image classification method of empty spectrum attention mechanism deep learning based on Octave convolution is the same as embodiment 1-
3, the cross entropy loss of output category result and actual result in (4c)1, formula is as follows:
Wherein, yjFor T in training image libraryjPrediction category probability, ojFor T in training image libraryjPractical category；
loss1Input be the prediction category probability that is obtained after full articulamentum of fused feature E；
Loss in the present invention2、loss3Principle and formula express and loss1It is identical, only loss2Input be characterized C
The prediction category probability obtained after full articulamentum, loss3Input be characterized the prediction class that D is obtained after full articulamentum
Mark probability.
Embodiment 5
The hyperspectral image classification method of empty spectrum attention mechanism deep learning based on Octave convolution is the same as embodiment 1-
4, step is normalized high spectrum image library in (1), is carried out by following formula:
Wherein VmaxFor the point maximum value of all pixels in high spectrum image library, VminFor all pixels in high spectrum image library
Point minimum value, VnFor the pixel value at any point in high spectrum image library, { I '1,I′2,…,I′n,…,I′NIt is at normalization
High spectrum image library after reason, I 'nFor n-th of sample of high spectrum image after normalized, n ∈ [0, N].
The present invention is restricted to -0.5 to 0.5 by the way that high spectrum image library is normalized, by EO-1 hyperion pixel value
Between, keep high spectrum image luminance distribution more balanced, effectively avoids subsequent processing bring from interfering, while by each pixel
The pixel value of point limits another unified section, prevents pixel value span larger, marginal information is erased.Since normalization makes
The pixel value of high spectrum image reduces, and reduces the calculation amount of network, while accelerating the convergence of network training.Pass through experiment
It also demonstrates to normalize high spectrum image pixel value and further improves the accurate of classification hyperspectral imagery between -0.5 to 0.5
Rate, while network training speed is greatly speeded up.
Embodiment 6
The hyperspectral image classification method of empty spectrum attention mechanism deep learning based on Octave convolution is the same as embodiment 1-
5, training is iterated to convolutional neural networks by gradient decline optimization in step (5), is accomplished by
The initial learning rate of (5a) setting training is L, attenuation rate β, by training image library { T1,T2,…,Tj,…,TM}
It is divided into the convolutional neural networks of G input building, the number of pictures inputted every time is Q, then:
Wherein M is the total number of training image library sample.
(5b) sets the corresponding learning rate l of input picture every time are as follows:
L=L* βG
(5c) carries out the update of G subparameter to convolutional neural networks by following formula, obtains updated weight vectors
Wnew；
Wherein, W is the weight vectors of convolutional neural networks parameter；
(5d) will train picture to input convolutional neural networks, loss function loss updated to weight vectors next timeop
It is updated, so that loss function lossopValue constantly decline；
(5e) repeats (5d), until loss function lossopNo longer decline, and current exercise wheel number is less than the iteration of setting
Number P then stops the training to the network, obtains trained convolutional neural networks；Otherwise, when training round reaches setting
When the number of iterations P, stops the training to the network, obtain trained convolutional neural networks.
The number of pictures Q inputted every time in the present invention is to be manually set, and can be adjusted according to the training effect of network,
Make classification hyperspectral imagery accuracy rate highest.The learning rate of network is the rate of e-learning validity feature, in network training
In the process, the learning rate of network is gradually reduced with the training of network, and learning rate is larger when initial training, promotes network quickly high
The main feature of the study high spectrum image of effect, with trained intensification, learning rate is gradually reduced, and e-learning speed slows down,
Promote e-learning to be conducive to the detailed features of classification hyperspectral imagery, while network being made to effectively reduce the loss letter of network
Number falls into the probability of local optimum, accelerates the speed of network training, accelerates the convergence of network training.
Provide a more detailed example again below, the present invention is further described:
Embodiment 7
The hyperspectral image classification method of empty spectrum attention mechanism deep learning based on Octave convolution is same to implement 1-6,
Referring to fig. 2, steps are as follows for realization of the invention:
Step 1, high spectrum image library is established, training sample and test sample are obtained.
1a) Indian Pines hyperspectral image data collection, Indian Pines high spectrum image are downloaded from related official website
Data set by airborne visual Infrared Imaging Spectrometer (AVIRIS) in 1992 to one piece of India pine tree of Indiana, USA into
Row imaging, referring to Fig. 3, Fig. 3 (a) is original Indian Pines high spectrum image, and Fig. 3 (b) is original Indian
The class label of Pines high spectrum image corresponding pixel points.It is carried out pixel-by-pixel centered on each pixel on high spectrum image
Point sliding, chooses the sliding sash of 13*13 size, carries out point pixel-by-pixel using sliding sash and slides, the distance slided every time is a pixel
Point, the high spectrum image block of all available 13*13 size of sliding, all high spectrum image blocks slided are used for every time
Establish high spectrum image library { I1,I2,…,In,…,IN, the corresponding classification of image library is { Y1,Y2,…,Yn,…,YN, wherein In
N-th image in representative image library, YnThe corresponding classification of n-th image in representative image library, n-th of sample in n representative image library
This number, n ∈ [0, N].
It 1b) is directed to established high spectrum image library, finds the maximum value and minimum value of all pixels point in image library, benefit
The high spectrum image library of foundation is normalized according to following formula with the value of all pixels point and the most value of two pixels
Processing:
Wherein VmaxFor the point maximum value of all pixels in high spectrum image library, VminFor all pixels in high spectrum image library
Point minimum value, VnFor the pixel value at any point in high spectrum image library, { I '1,I′2,…,I′n,…,I′NIt is at normalization
High spectrum image library after reason, I 'nFor n-th of sample of remote sensing images after normalized, n ∈ [0, N].
1c) select the high spectrum image of specified quantity at random from the high spectrum image of each classification after normalized
Sample constructs training sample set { T1,T2,…,Tj,…,TM, abbreviation training set, using remaining high spectrum image as test specimens
This collection { t1,t2,…td,…,tmAbbreviation test set, wherein TjIndicate j-th of sample in training sample, j ∈ [0, M], tdIt indicates
D-th of sample in test sample, d ∈ [0, m], M are the total number of training sample, and m is the total number of test sample, m < N, M < N.
Step 2, Octave convolutional neural networks are constructed.
Octave convolution module 2a) is set, is made of sequentially connected four conventional parts, each conventional part wraps again
Octave convolution is included, Batch Normalization and Relu activation primitive, there are also one between second and third conventional part
A maximum pond layer；
Referring to fig. 4, Octave convolution working principle of the invention is as follows:
Two parts of high and low frequency are divided the image into, wherein the one of the wide and a height of high frequency section of the image of low frequency part
Half；Carry out common convolution operation to high frequency section, obtain two convolution results, the width of medium-high frequency to high frequency convolution results and
High identical as high frequency section, the width and height of high frequency to low frequency convolution results are identical as low frequency part；Again to low frequency part into
The identical convolution operation of row, the wherein width of low frequency to high frequency convolution results and height, low frequency to low frequency convolution identical as high frequency section
As a result width and height is identical as low frequency part；Again by with identical wide and high results added to forming new radio-frequency head together
Point and low frequency part.
The Relu activation primitive are as follows:
Wherein x is the input function of Relu activation primitive.
2b) installation space attention mechanism module activates letter by convolutional layer, Batch Normalization, Relu
Number, matrix transposition are constituted with the layer that is multiplied, softmax layers and data transposition with layer is added, and structure is as shown in Figure 5.
Matrix transposition is the spy obtained by convolution with the layer that is multiplied, the feature of input in space transforms power module of the present invention
Sign, size are W × H × C, feature sizes are first converted to N × C, wherein N=W × H, then carry out square to the feature after conversion
Battle array transposition, the size of obtained feature are C × N, then obtained by the feature obtained by Feature Conversion and by matrix transposition
Feature carries out matrix multiple, obtains output matrix, and size is N × N.
Softmax layers in space transforms power module of the present invention, using softmax to the square of matrix transposition and the layer output that is multiplied
Battle array is normalized, and the value in matrix is normalized between 0 to 1, while by the matrix after normalization and passing through convolution
Operation is multiplied with the feature that matrix conversion obtains, and obtains output feature, and size is N × C.
In space transforms power module of the present invention data transposition be added layer, first softmax layers of output result is turned
It changes, size is W × H × C after converting, then will pass through the feature being converted to and be input to the initial of space transforms power module
Feature is added to the final output feature for obtaining space transforms power module together, and size is W × H × C.
Space transforms power module is by convolutional layer, Batch Normalization, Relu activation primitive, matrix transposition and phase
Multiply layer, softmax layers and data transposition and be sequentially connected composition with layer is added, network can be made to find band by space transforms power module
There is the region of obvious semantic information, and then improves the accuracy rate of classification hyperspectral imagery.
Spectrum attention mechanism module 2c) is set, by matrix transposition and the layer that is multiplied, softmax layers and data transposition and
It is added layer to constitute, structure is as shown in Figure 6.
Spectrum of the present invention pays attention to matrix transposition and the layer that is multiplied in power module, and the feature sizes of input are W × H × C, first will be special
Sign size is converted to C × N, wherein N=W × H, then carries out matrix transposition to the feature after conversion, and the size of obtained feature is N
× C, then will carry out matrix multiple by the feature that Feature Conversion and matrix transposition obtain, obtains output matrix, size be C ×
C。
Spectrum of the present invention pays attention in power module softmax layers, using softmax to the square of matrix transposition and the layer output that is multiplied
Battle array is normalized, and the value in matrix is normalized between 0 to 1, while by the matrix after normalization and passing through matrix
The feature being converted to is multiplied, and obtains output feature, and size is C × N.
Spectrum of the present invention pay attention in power module data transposition be added layer, first softmax layers of output result is turned
It changes, size is W × H × C after converting, then will pay attention to the initial of power module with spectrum is input to by the feature being converted to
Feature is added to and obtains the final output feature that spectrum pays attention to power module together, and size is W × H × C.
Spectrum notices that power module is sequentially connected by matrix transposition with the layer that is multiplied, softmax layers and data transposition with layer is added
It constitutes, notices that power module makes network find the stronger spectral band of spectral information by spectrum, and then improve high spectrum image point
The accuracy rate of class and the robustness of network.
Pixel-level attention mechanism module 2d) is set, is activated by convolutional layer, Batch Normalization and Relu
Function is sequentially connected composition, and structure is as shown in Figure 7.
Present invention pixel grade notices in power module in convolutional layer that setting convolution kernel size is 1*1, is 1*1 by convolution kernel
The convolution operation of size carries out characteristic strengthening to each pixel of high spectrum image, thus improves classification hyperspectral imagery
Accuracy rate.
Full articulamentum 2e) is set, full articulamentum is made of the first full articulamentum, the second full articulamentum and softmax layers,
The convolution kernel size of first full articulamentum is 9800 × 1024, and the convolution kernel size of the second full articulamentum is 1024 × 16.Wherein
16 be classification number total in the high spectrum image of input.
First full articulamentum in the full articulamentum of the present invention, it is first the every of input feature vector that it is 1 × 9800 that feature, which inputs size,
A value is multiplied by a coefficient, then is input to the first full articulamentum, and output feature sizes are 1 × 1024.Institute's multiplying factor is initial
One group changed meets the vector of Gaussian Profile, and coefficient can be updated during sample training.
Second full articulamentum in the full articulamentum of the present invention, it is 1 × 1024 that feature, which inputs size, and output feature sizes are 1
×16。
The Octave convolution module of above-mentioned setting, space transforms power module, spectrum 2f) are paid attention into power module and full articulamentum
It is sequentially connected, obtains Octave convolutional neural networks.
Step 3 determines the loss function of convolutional neural networks:
(3a) is by training sample set { T1,T2,…,Tj,…,TMIt is input to Octave volumes of Octave convolutional neural networks
Volume module exports the last layer feature F of convolutional layer.
The last layer feature F is separately input to the space transforms power module and spectrum of Octave convolutional neural networks by (3b)
Pay attention to power module, output feature is respectively A and B, then output feature A and B are input to Pixel-level and pay attention to power module, exports feature
Respectively C and D.
The feature C and D that (3c) will be obtained, are input to the full articulamentum of Octave convolutional neural networks, and output utilizes feature C
The output category result obtained with D；Simultaneously then feature C and D are added pixel-by-pixel, are obtained respectively multiplied by a coefficient
Fused feature E, then feature E is input to the full articulamentum of Octave convolutional neural networks, output utilizes fused spy
The output category result that sign E is obtained, obtains the loss function loss of Octave convolutional neural networksop:
Wherein:For the L2 norm of Octave convolutional neural networks weight vectors, η isHyper parameter；Indicate using fused feature E output category result and actual result after full articulamentum
Cross entropy, yjFor T in training image libraryjPrediction category probability, ojFor T in training image libraryjPractical category, loss1It is defeated
Enter the prediction category probability obtained after full articulamentum for fused feature E.
loss2、loss3Principle and formula express and loss1It is identical, loss2C is characterized to export after full articulamentum
The cross entropy of classification results and actual result, loss3It is characterized D output category result and actual result after full articulamentum
Cross entropy；loss2Input be characterized the prediction category probability that C is obtained after full articulamentum, loss3Input be characterized D warp
Cross the prediction category probability obtained after full articulamentum.
Step 4, training is iterated to convolutional neural networks.
Being iterated trained existing method to Octave convolutional neural networks has gradient optimization algorithm, Nesterov
Gradient acceleration method, Adagrad method use but are not limited only to gradient descent algorithm in this example, and implementation step is as follows:
4a) setting the number of iterations is P, and it is L, attenuation rate β that trained initial learning rate, which is arranged, by training image library { T1,
T2,…,Tj,…,TMDivide in the convolutional neural networks for being input to step 2 building, the number of pictures inputted every time is Q, and number is
G:
Wherein M is the total number of training image library sample.
4b) set the corresponding learning rate l of input picture every time are as follows:
L=L* βG
4c) by following formula to Octave convolutional neural networks carry out the update of G subparameter, obtain updated weight to
Measure Wnew:
Wherein, W is the weight vectors of Octave convolutional neural networks parameter.
By updated weight vectors WnewBring 3c into) in loss function lossop, obtain the updated damage of weight vectors
Lose function lossop。
4d) picture will be trained to be input to Octave convolutional neural networks next time, to the updated loss letter of weight vectors
Number lossopIt is updated, so that loss function lossopValue constantly decline.
4e) repeat 4d), until loss function lossopNo longer decline, and current exercise wheel number is less than the iteration time of setting
Number P then stops the training to the network, obtains trained Octave convolutional neural networks；Otherwise, it is set when training round reaches
When the number of iterations P set, stops the training to the network, obtain trained Octave convolutional neural networks.
Network is optimized using gradient descent algorithm in this example, finds optimal solution, but to network optimization when is unlimited
In gradient descent algorithm, other optimization algorithms such as genetic algorithm etc. can still be optimized network.
Step 5 classifies to test sample collection.
Test sample collection after normalized is input in trained Octave convolutional neural networks, from instruction
The Octave convolutional neural networks output perfected obtains the test sample collection classification results of the high spectrum image of input, completes to height
The precise classification of spectrum picture.
That present invention mainly solves prior art the same category spacing is big, different classes of spacing is small, classification accuracy is low asks
Topic.The present invention is by establishing high spectrum image library and the corresponding classification of image library, and from every class EO-1 hyperion after normalized
The high spectrum image sample building training sample set and test sample collection of specified quantity are selected in image at random；Constructing one includes
Octave convolution module, space transforms power module, spectrum notice that power module, Pixel-level pay attention to power module and full articulamentum
Octave convolutional neural networks；The training sample that training sample is concentrated is input in Octave convolutional neural networks and is trained
The classification results of sample, and determine the loss function of convolutional neural networks；By gradient descent method to loss function iteration more
Newly until penalty values stabilization, trained Octave convolutional neural networks are obtained；By the test to be sorted after normalization
Sample set is input to trained Octave convolutional neural networks and obtains classification results.Nicety of grading of the present invention is high, strong robustness,
It can be applied to the analysis and management of hyperspectral image data.
The effect of invention can be further illustrated by following emulation:
Embodiment 8
The hyperspectral image classification method of empty spectrum attention mechanism deep learning based on Octave convolution is the same as embodiment 1-
7,
Simulated conditions
This example in HP-Z840-Workstation with Xeon (R) CPU E5-2630, GeForce TITAN XP,
Under 64G RAM, Ubuntu system, on TensorFlow operation platform, the present invention and existing remote sensing images scene classification are completed
Emulation.
Simulation parameter setting is as follows, and iteration round P is 175 times, and initial learning rate L is 0.00001, η=0.0001, every time
Inputting picture number Q is 16, and attenuation rate β is 0.9.Learning sequence is in repetitive exercise each time, to category arbiter, classification
Difference optimizer, common training.
Emulation content
Download Indian Pines hyperspectral image data collection, Indian Pines hyperspectral image data collection by it is airborne can
One piece of India pine tree of Indiana, USA is imaged in 1992 depending on Infrared Imaging Spectrometer (AVIRIS), referring to Fig. 3,
Fig. 3 (a) is original Indian Pines high spectrum image, and Fig. 3 (b) is that original Indian Pines high spectrum image is corresponding
The class label of pixel.Centered on each pixel, the sliding sash of selection 13*13 size, the sliding put pixel-by-pixel, every time
Sliding obtains an image, and all images slided are for establishing high spectrum image library { I1,I2,…,In,…,IN}.Again
The high spectrum image library of foundation is normalized, i.e., first obtains high spectrum image library pixel maximum value V 'maxAnd pixel
The minimum value V ' of pointmin, then to the value of all pixels point in high spectrum image library divided by V 'maxWith V 'minDifference, obtain normalizing
Change treated high spectrum image library.
From a certain number of high spectrum images are selected after normalized in high spectrum image library at random as training sample
Collect DT, using remaining high spectrum image as test sample collection Dt。
The image that the training sample set and test sample are concentrated has 16 types, respectively Aflalfa, Corn-
notill、Corn-mintill、Corn、Grass-pasture、Grass-trees、Grass-pasture-mowed、Hay-
windrowed、Oats、Soybean-nottill、Soybean-mintill、Soybean-clean、Wheat、Woods、
Stone-Steel-Towers,Buildings-Grass-Trees-Drives；The other number of training of every type, test sample
Several and total number of samples is referring to table 1.
1 Indian Pines hyperspectral image data collection classification statistical form of table
Under above-mentioned simulated conditions, using training sample set DTRespectively with the present invention and existing representative three kinds of images point
Class model is trained, using test sample collection DtIt is tested, compares the accuracy rate of its classification, as a result such as table 2.
2 present invention of table and existing classification hyperspectral imagery model performance evaluation table
| Test model | Test sample accuracy rate |
| The present invention | 0.9898 |
| KFRC-CKIR | 0.9860 |
| 2-DCNN | 0.9888 |
| 3D-SRNet | 0.9720 |
KFRC-CKIR is the existing hyperspectral classification method merged based on core canonical and core in table 2, and 2-DCNN is existing base
In the hyperspectral image classification method of depth convolutional neural networks, 3D-SRNet can be divided and transfer learning to be existing based on three-dimensional
Hyperspectral image classification method.
From table 2 it can be seen that with the trained convolutional neural networks of the present invention to test sample collection DtClassify, it is quasi-
True rate highest, accurate rate than existing representative classification hyperspectral imagery model in all classification methods for participating in test is equal
There is promotion.
In conclusion a kind of empty spectrum attention classification hyperspectral imagery side based on Octave convolution disclosed by the invention
Method solves the problems, such as that prior art the same category spacing is big, different classes of spacing is small, classification accuracy is low.Scheme is: wait divide
The input of class image builds with data prediction, division training set and test set, Octave convolutional neural networks, determines Octave volumes
Product neural network loss function, the training update of Octave convolutional neural networks, test set data test, complete high spectrum image
Classification.The present invention strengthens character representation using Octave convolution operation, introduces spatial attention mechanism and spectrum attention machine
System finds network more accurately for classifying more advantageous and including the more comprehensive detailed region of information.Present invention classification
Precision is high, and strong robustness can be applied to the analysis and management of hyperspectral image data.
Claims (6)
1. a kind of hyperspectral image classification method of the empty spectrum attention mechanism deep learning based on Octave convolution, feature exist
In comprising the following steps that
(1) image input and data prediction: high spectrum image to be sorted is inputted, is carried out centered on each pixel by picture
Vegetarian refreshments sliding, all image blocks slided are for establishing high spectrum image library { I1,I2,…,In,…,IN, in image library
The corresponding classification of each image block is { Y1,Y2,…,Yn,…,YN, and place is normalized to the high spectrum image library of foundation
It manages, wherein InN-th image block, Y in representative image librarynThe corresponding classification of n-th image block in representative image library, n representative image
N-th of sample number in library, n ∈ [0, N], N represent the image block total number in high spectrum image library；
(2) training set and test set are divided: selecting specified quantity at random from every class high spectrum image after normalized
High spectrum image sample constructs training sample set { T1,T2,…,Tj,…,TM, using remaining high spectrum image as test specimens
This collection { t1,t2,…td,…,tmWherein TjIndicate j-th of sample in training sample, j ∈ [0, M], tdIt indicates the in test sample
D sample, d ∈ [0, m], M are the total number of training sample, and m is the total number of test sample, m < N, M < N；
(3) Octave convolutional neural networks are built: building an Octave convolutional neural networks, the input terminal of network is Octave
Convolution module, the output end of network are the output of full articulamentum as a result, including two between the input terminal and output end of network
Branch, wherein a branch successively passes through space transforms power module and Pixel-level pays attention to power module, another branch successively passes through
Spectrum notices that power module and Pixel-level pay attention to power module；
(4) Octave convolutional neural networks loss function loss is determinedop: setting loss function includes that will mention after Fusion Features
The feature taken is input to the cross entropy loss of full articulamentum output category result and actual result obtained from1, will by sky
Between notice that power module and Pixel-level notice that power module is extracted feature be input to full articulamentum output category result obtained from
With the cross entropy loss of actual result2, by by spectrum pay attention to power module and Pixel-level pay attention to power module extract feature input
To the cross entropy loss of full articulamentum output category result and actual result obtained from3With the convolutional Neural for having hyper parameter
Four part of L2 norm of network weight W, the loss function of network is successively added by above four part to be constituted；
(5) training updates: the number of iterations that network training is arranged is P, by gradient decline optimization to Octave convolutional Neural net
Network is iterated training, until loss function lossopDo not decline or exercise wheel number reaches the number of iterations, obtains trained
Octave convolutional neural networks；
(6) test sample collection after normalized data test: is input to trained Octave convolutional Neural net
In network, classification results are obtained, complete image classification.
2. the high spectrum image point of the empty spectrum attention mechanism deep learning according to claim 1 based on Octave convolution
Class method, which is characterized in that Octave convolutional neural networks described in step (3) are built, Octave convolution module therein,
Space transforms power module, spectrum notice that power module, Pixel-level notice that power module and full articulamentum, parameter setting are as follows:
Octave convolution module, that is, the input module is made of, each conventional part sequentially connected four conventional parts
It again include Octave convolution, Batch Normalization and Relu activation primitive is gone back between second and third conventional part
There is a maximum pond layer；
The spatial attention mechanism module is turned by convolutional layer, Batch Normalization, Relu activation primitive, matrix
It sets with the layer that is multiplied, softmax layers and data transposition with layer is added and constitutes；
The spectrum attention mechanism module, by matrix transposition and the layer that is multiplied, softmax layers and data transposition be added layer structure
At；
The Pixel-level attention mechanism module, by convolutional layer, Batch Normalization and Relu activation primitive structure
At；
The full articulamentum is made of the first full articulamentum and the second full articulamentum and softmax layers, i.e. output layer.
3. the high spectrum image point of the empty spectrum attention mechanism deep learning according to claim 1 based on Octave convolution
Class method, which is characterized in that determination Octave convolutional neural networks loss function loss described in step (4)op, specifically include
Following steps:
(4a) is by training image library { T1,T2,…,Tj,…,TMIt is input to the Octave convolution mould of Octave convolutional neural networks
Block exports the last layer feature F of convolutional layer；
(4b) the last layer feature F is separately input to the space transforms power module of Octave convolutional neural networks and spectrum pays attention to
Power module, output feature is respectively A and B, then output feature A and B are input to Pixel-level and pay attention to power module, output feature difference
For C and D.
The feature C and D that (4c) will be obtained, are input to the full articulamentum of Octave convolutional neural networks, and output utilizes feature C and D
Obtained output category result；Simultaneously then feature C and D are added pixel-by-pixel, are merged respectively multiplied by a coefficient
Feature E afterwards, then feature E is input to the full articulamentum of Octave convolutional neural networks, output is obtained using fused feature E
The output category result arrived obtains the loss function loss of Octave convolutional neural networksop:
Wherein, loss1For using fused feature E output category result and actual result after full articulamentum cross entropy,
loss2It is characterized the cross entropy of C output category result and actual result after full articulamentum, loss3D is characterized by connecting entirely
The cross entropy of output category result and actual result after layer is connect,For the L2 norm of convolutional neural networks weight vectors, η isHyper parameter.
4. the high spectrum image point of the empty spectrum attention mechanism deep learning according to claim 3 based on Octave convolution
Class method, which is characterized in that the output category result obtained using fused feature E and actual result in step (4c)
Cross entropy loss1, specific cross entropy formula is as follows:
Wherein, yjFor T in training image libraryjPrediction category probability, ojFor T in training image libraryjPractical category；loss1's
Input is the prediction category probability that fused feature E is obtained after full articulamentum；
loss2、loss3Principle and formula and loss1It is identical, loss2Input be characterized C obtained after full articulamentum it is pre-
Survey category probability, loss3Input be characterized the prediction category probability that D is obtained after full articulamentum.
5. the high spectrum image point of the empty spectrum attention mechanism deep learning according to claim 1 based on Octave convolution
Class method, which is characterized in that step is normalized high spectrum image library in (1), is carried out by following formula:
Wherein VmaxFor the point maximum value of all pixels in high spectrum image library, VminFor the point of all pixels in high spectrum image library
Minimum value, VnFor the pixel value at any point in high spectrum image library, { I '1,I′2,…,I′n,…,I′NFor after normalized
High spectrum image library, I 'nFor n-th of sample of high spectrum image after normalized, n ∈ [0, N].
6. the high spectrum image point of the empty spectrum attention mechanism deep learning according to claim 1 based on Octave convolution
Class method, which is characterized in that training is iterated to convolutional neural networks by gradient decline optimization in step (5), is realized
It is as follows:
The initial learning rate of (5a) setting training is L, attenuation rate β, by training image library { T1,T2,…,Tj,…,TMIt is divided into G
In the convolutional neural networks of secondary input building, the number of pictures inputted every time is Q, then:
Wherein M is the total number of training image library sample；
(5b) sets the corresponding learning rate l of input picture every time are as follows:
L=L* βG
(5c) carries out the update of G subparameter to convolutional neural networks by following formula, obtains updated weight vectors Wnew；
Wherein, W is the weight vectors of convolutional neural networks parameter；
(5d) will train picture to input convolutional neural networks, loss function loss updated to weight vectors next timeopIt carries out
It updates, so that loss function lossopValue constantly decline；
(5e) repeats (5d), until loss function lossopNo longer decline, and current exercise wheel number is less than the number of iterations of setting
P then stops the training to the network, obtains trained Octave convolutional neural networks；Otherwise, when training round reaches setting
The number of iterations P when, stop training to the network, obtain trained Octave convolutional neural networks.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910797299.0A CN110516596B (en) | 2019-08-27 | 2019-08-27 | Octave convolution-based spatial spectrum attention hyperspectral image classification method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910797299.0A CN110516596B (en) | 2019-08-27 | 2019-08-27 | Octave convolution-based spatial spectrum attention hyperspectral image classification method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110516596A true CN110516596A (en) | 2019-11-29 |
| CN110516596B CN110516596B (en) | 2023-03-24 |
Family
ID=68627278
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910797299.0A Active CN110516596B (en) | 2019-08-27 | 2019-08-27 | Octave convolution-based spatial spectrum attention hyperspectral image classification method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110516596B (en) |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111046962A (en) * | 2019-12-16 | 2020-04-21 | 中国人民解放军战略支援部队信息工程大学 | Sparse attention-based feature visualization method and system for convolutional neural network model |
| CN111160396A (en) * | 2019-12-06 | 2020-05-15 | 华南理工大学 | Hyperspectral image classification method based on graph convolution network with multi-graph structure |
| CN111259954A (en) * | 2020-01-15 | 2020-06-09 | 北京工业大学 | Hyperspectral traditional Chinese medicine tongue coating and tongue quality classification method based on D-Resnet |
| CN111274869A (en) * | 2020-01-07 | 2020-06-12 | 中国地质大学（武汉） | Method for classifying hyperspectral images based on parallel attention mechanism residual error network |
| CN111274988A (en) * | 2020-02-10 | 2020-06-12 | 安徽大学 | Multispectral-based vehicle weight identification method and device |
| CN111353531A (en) * | 2020-02-25 | 2020-06-30 | 西安电子科技大学 | Hyperspectral image classification method based on singular value decomposition and spatial spectral domain attention mechanism |
| CN111382676A (en) * | 2020-02-25 | 2020-07-07 | 南京大学 | Sand image classification method based on attention mechanism |
| CN111563520A (en) * | 2020-01-16 | 2020-08-21 | 北京航空航天大学 | Hyperspectral image classification method based on space-spectrum combined attention mechanism |
| CN111814607A (en) * | 2020-06-24 | 2020-10-23 | 中国地质大学（武汉） | Deep learning model suitable for small sample hyperspectral image classification |
| CN112052755A (en) * | 2020-08-24 | 2020-12-08 | 西安电子科技大学 | Semantic convolution hyperspectral image classification method based on multi-path attention mechanism |
| CN112052758A (en) * | 2020-08-25 | 2020-12-08 | 西安电子科技大学 | Hyperspectral image classification method based on attention mechanism and recurrent neural network |
| CN112232343A (en) * | 2020-09-03 | 2021-01-15 | 国家粮食和物资储备局科学研究院 | Neural network and method for recognizing grain mildewed grains |
| CN112464891A (en) * | 2020-12-14 | 2021-03-09 | 湖南大学 | Hyperspectral image classification method |
| CN112577747A (en) * | 2020-12-07 | 2021-03-30 | 东南大学 | Rolling bearing fault diagnosis method based on space pooling network |
| CN112801133A (en) * | 2020-12-30 | 2021-05-14 | 核工业北京地质研究院 | Spectrum identification and classification method based on keras model |
| CN112949592A (en) * | 2021-03-31 | 2021-06-11 | 云南大学 | Hyperspectral image classification method and device and electronic equipment |
| CN113095437A (en) * | 2021-04-29 | 2021-07-09 | 中国电子科技集团公司第五十四研究所 | Fire point detection method for Himapari-8 remote sensing data |
| CN113569660A (en) * | 2021-07-06 | 2021-10-29 | 河海大学 | Learning rate optimization algorithm discount coefficient method for hyperspectral image classification |
| CN114022716A (en) * | 2021-11-29 | 2022-02-08 | 云南师范大学 | Clothing attribute prediction method based on attention mechanism and graph relation network |
| CN114220002A (en) * | 2021-11-26 | 2022-03-22 | 通辽市气象台(通辽市气候生态环境监测中心) | Method and system for monitoring invasion of foreign plants based on convolutional neural network |
| CN114462596A (en) * | 2022-02-10 | 2022-05-10 | 黑龙江省农业科学院 | Disease and insect pest monitoring method and system for industrial hemp growth period |
| CN114492604A (en) * | 2022-01-11 | 2022-05-13 | 电子科技大学 | Radiation source individual identification method under small sample scene |
| CN114781534A (en) * | 2022-04-29 | 2022-07-22 | 中国人民解放军战略支援部队信息工程大学 | Image-level hyperspectral image classification method |
| CN114897814A (en) * | 2022-05-07 | 2022-08-12 | 中国石油大学(华东) | Hyperspectral image oil spilling detection method based on multi-level wavelet decomposition joint sealing network |
| CN114943251A (en) * | 2022-05-20 | 2022-08-26 | 电子科技大学 | Unmanned aerial vehicle target identification method based on fusion attention mechanism |
| CN114998725A (en) * | 2022-05-17 | 2022-09-02 | 北京理工大学 | Hyperspectral image classification method based on adaptive spatial spectrum attention kernel generation network |
| CN115457311A (en) * | 2022-08-23 | 2022-12-09 | 宁波大学 | Hyperspectral remote sensing image band selection method based on self-expression transfer learning |
| WO2023137783A1 (en) * | 2022-01-19 | 2023-07-27 | 深圳大学 | Spectral data processing method and apparatus, storage medium and electronic device |
| CN116824282A (en) * | 2023-08-31 | 2023-09-29 | 中国石油大学(华东) | Semi-supervised hyperspectral image classification method integrating multi-jump graph convolution and attention |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109784347A (en) * | 2018-12-17 | 2019-05-21 | 西北工业大学 | Image classification method based on multiple dimensioned dense convolutional neural networks and spectrum attention mechanism |
| CN109993220A (en) * | 2019-03-23 | 2019-07-09 | 西安电子科技大学 | Multi-source Remote Sensing Images Classification method based on two-way attention fused neural network |
| WO2019136946A1 (en) * | 2018-01-15 | 2019-07-18 | 中山大学 | Deep learning-based weakly supervised salient object detection method and system |
| CN110084159A (en) * | 2019-04-15 | 2019-08-02 | 西安电子科技大学 | Hyperspectral image classification method based on the multistage empty spectrum information CNN of joint |
-
2019
- 2019-08-27 CN CN201910797299.0A patent/CN110516596B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019136946A1 (en) * | 2018-01-15 | 2019-07-18 | 中山大学 | Deep learning-based weakly supervised salient object detection method and system |
| CN109784347A (en) * | 2018-12-17 | 2019-05-21 | 西北工业大学 | Image classification method based on multiple dimensioned dense convolutional neural networks and spectrum attention mechanism |
| CN109993220A (en) * | 2019-03-23 | 2019-07-09 | 西安电子科技大学 | Multi-source Remote Sensing Images Classification method based on two-way attention fused neural network |
| CN110084159A (en) * | 2019-04-15 | 2019-08-02 | 西安电子科技大学 | Hyperspectral image classification method based on the multistage empty spectrum information CNN of joint |
Cited By (49)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111160396A (en) * | 2019-12-06 | 2020-05-15 | 华南理工大学 | Hyperspectral image classification method based on graph convolution network with multi-graph structure |
| CN111160396B (en) * | 2019-12-06 | 2023-04-25 | 华南理工大学 | Hyperspectral image classification method of graph convolution network based on multi-graph structure |
| CN111046962A (en) * | 2019-12-16 | 2020-04-21 | 中国人民解放军战略支援部队信息工程大学 | Sparse attention-based feature visualization method and system for convolutional neural network model |
| CN111274869A (en) * | 2020-01-07 | 2020-06-12 | 中国地质大学（武汉） | Method for classifying hyperspectral images based on parallel attention mechanism residual error network |
| CN111259954A (en) * | 2020-01-15 | 2020-06-09 | 北京工业大学 | Hyperspectral traditional Chinese medicine tongue coating and tongue quality classification method based on D-Resnet |
| CN111259954B (en) * | 2020-01-15 | 2024-05-31 | 北京工业大学 | D-Resnet-based hyperspectral traditional Chinese medicine tongue fur and tongue quality classification method |
| CN111563520B (en) * | 2020-01-16 | 2023-01-13 | 北京航空航天大学 | Hyperspectral image classification method based on space-spectrum combined attention mechanism |
| CN111563520A (en) * | 2020-01-16 | 2020-08-21 | 北京航空航天大学 | Hyperspectral image classification method based on space-spectrum combined attention mechanism |
| CN111274988A (en) * | 2020-02-10 | 2020-06-12 | 安徽大学 | Multispectral-based vehicle weight identification method and device |
| CN111274988B (en) * | 2020-02-10 | 2023-03-24 | 安徽大学 | Multispectral-based vehicle weight identification method and device |
| CN111382676A (en) * | 2020-02-25 | 2020-07-07 | 南京大学 | Sand image classification method based on attention mechanism |
| CN111353531A (en) * | 2020-02-25 | 2020-06-30 | 西安电子科技大学 | Hyperspectral image classification method based on singular value decomposition and spatial spectral domain attention mechanism |
| CN111382676B (en) * | 2020-02-25 | 2023-12-22 | 南京大学 | Sand grain image classification method based on attention mechanism |
| CN111353531B (en) * | 2020-02-25 | 2023-03-28 | 西安电子科技大学 | Hyperspectral image classification method based on singular value decomposition and spatial spectral domain attention mechanism |
| CN111814607A (en) * | 2020-06-24 | 2020-10-23 | 中国地质大学（武汉） | Deep learning model suitable for small sample hyperspectral image classification |
| CN111814607B (en) * | 2020-06-24 | 2023-12-29 | 中国地质大学（武汉） | Deep learning model suitable for small sample hyperspectral image classification |
| CN112052755B (en) * | 2020-08-24 | 2023-06-02 | 西安电子科技大学 | Semantic convolution hyperspectral image classification method based on multipath attention mechanism |
| CN112052755A (en) * | 2020-08-24 | 2020-12-08 | 西安电子科技大学 | Semantic convolution hyperspectral image classification method based on multi-path attention mechanism |
| CN112052758A (en) * | 2020-08-25 | 2020-12-08 | 西安电子科技大学 | Hyperspectral image classification method based on attention mechanism and recurrent neural network |
| CN112052758B (en) * | 2020-08-25 | 2023-05-23 | 西安电子科技大学 | Hyperspectral image classification method based on attention mechanism and cyclic neural network |
| CN112232343B (en) * | 2020-09-03 | 2023-11-21 | 国家粮食和物资储备局科学研究院 | Grain mildew grain identification neural network and identification method |
| CN112232343A (en) * | 2020-09-03 | 2021-01-15 | 国家粮食和物资储备局科学研究院 | Neural network and method for recognizing grain mildewed grains |
| CN112577747A (en) * | 2020-12-07 | 2021-03-30 | 东南大学 | Rolling bearing fault diagnosis method based on space pooling network |
| CN112577747B (en) * | 2020-12-07 | 2023-08-08 | 东南大学 | Rolling bearing fault diagnosis method based on space pooling network |
| CN112464891B (en) * | 2020-12-14 | 2023-06-16 | 湖南大学 | Hyperspectral image classification method |
| CN112464891A (en) * | 2020-12-14 | 2021-03-09 | 湖南大学 | Hyperspectral image classification method |
| CN112801133A (en) * | 2020-12-30 | 2021-05-14 | 核工业北京地质研究院 | Spectrum identification and classification method based on keras model |
| CN112949592A (en) * | 2021-03-31 | 2021-06-11 | 云南大学 | Hyperspectral image classification method and device and electronic equipment |
| CN113095437B (en) * | 2021-04-29 | 2022-03-01 | 中国电子科技集团公司第五十四研究所 | Fire point detection method for Himapari-8 remote sensing data |
| CN113095437A (en) * | 2021-04-29 | 2021-07-09 | 中国电子科技集团公司第五十四研究所 | Fire point detection method for Himapari-8 remote sensing data |
| CN113569660B (en) * | 2021-07-06 | 2024-03-26 | 河海大学 | Learning rate optimization algorithm discount coefficient method for hyperspectral image classification |
| CN113569660A (en) * | 2021-07-06 | 2021-10-29 | 河海大学 | Learning rate optimization algorithm discount coefficient method for hyperspectral image classification |
| CN114220002A (en) * | 2021-11-26 | 2022-03-22 | 通辽市气象台(通辽市气候生态环境监测中心) | Method and system for monitoring invasion of foreign plants based on convolutional neural network |
| CN114022716B (en) * | 2021-11-29 | 2024-04-12 | 云南师范大学 | Clothes attribute prediction method based on attention mechanism and graph relation network |
| CN114022716A (en) * | 2021-11-29 | 2022-02-08 | 云南师范大学 | Clothing attribute prediction method based on attention mechanism and graph relation network |
| CN114492604A (en) * | 2022-01-11 | 2022-05-13 | 电子科技大学 | Radiation source individual identification method under small sample scene |
| WO2023137783A1 (en) * | 2022-01-19 | 2023-07-27 | 深圳大学 | Spectral data processing method and apparatus, storage medium and electronic device |
| CN114462596A (en) * | 2022-02-10 | 2022-05-10 | 黑龙江省农业科学院 | Disease and insect pest monitoring method and system for industrial hemp growth period |
| CN114781534A (en) * | 2022-04-29 | 2022-07-22 | 中国人民解放军战略支援部队信息工程大学 | Image-level hyperspectral image classification method |
| CN114897814A (en) * | 2022-05-07 | 2022-08-12 | 中国石油大学(华东) | Hyperspectral image oil spilling detection method based on multi-level wavelet decomposition joint sealing network |
| CN114897814B (en) * | 2022-05-07 | 2023-07-25 | 中国石油大学(华东) | Hyperspectral image oil spill detection method based on multistage wavelet decomposition close-coupled network |
| CN114998725A (en) * | 2022-05-17 | 2022-09-02 | 北京理工大学 | Hyperspectral image classification method based on adaptive spatial spectrum attention kernel generation network |
| CN114998725B (en) * | 2022-05-17 | 2024-07-19 | 北京理工大学 | Hyperspectral image classification method based on self-adaptive spatial spectrum attention kernel generation network |
| CN114943251B (en) * | 2022-05-20 | 2023-05-02 | 电子科技大学 | Unmanned aerial vehicle target recognition method based on fusion attention mechanism |
| CN114943251A (en) * | 2022-05-20 | 2022-08-26 | 电子科技大学 | Unmanned aerial vehicle target identification method based on fusion attention mechanism |
| CN115457311B (en) * | 2022-08-23 | 2023-08-29 | 宁波大学 | Hyperspectral remote sensing image band selection method based on self-expression transfer learning |
| CN115457311A (en) * | 2022-08-23 | 2022-12-09 | 宁波大学 | Hyperspectral remote sensing image band selection method based on self-expression transfer learning |
| CN116824282A (en) * | 2023-08-31 | 2023-09-29 | 中国石油大学(华东) | Semi-supervised hyperspectral image classification method integrating multi-jump graph convolution and attention |
| CN116824282B (en) * | 2023-08-31 | 2023-11-10 | 中国石油大学(华东) | Semi-supervised hyperspectral image classification method integrating multi-jump graph convolution and attention |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110516596B (en) | 2023-03-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110516596A (en) | Empty spectrum attention hyperspectral image classification method based on Octave convolution | |
| CN109993220B (en) | Multi-source remote sensing image classification method based on double-path attention fusion neural network | |
| CN112052755B (en) | Semantic convolution hyperspectral image classification method based on multipath attention mechanism | |
| CN110334765A (en) | Remote Image Classification based on the multiple dimensioned deep learning of attention mechanism | |
| CN111476713B (en) | Intelligent weather image identification method and system based on multi-depth convolution neural network fusion | |
| CN109034224B (en) | Hyperspectral classification method based on double branch network | |
| CN110321963A (en) | Based on the hyperspectral image classification method for merging multiple dimensioned multidimensional sky spectrum signature | |
| CN107832797B (en) | Multispectral image classification method based on depth fusion residual error network | |
| CN106203523A (en) | The classification hyperspectral imagery of the semi-supervised algorithm fusion of decision tree is promoted based on gradient | |
| CN110414577A (en) | A kind of laser radar point cloud multiple target Objects recognition method based on deep learning | |
| CN108460391B (en) | Hyperspectral image unsupervised feature extraction method based on generation countermeasure network | |
| CN106845418A (en) | A kind of hyperspectral image classification method based on deep learning | |
| CN105139028A (en) | SAR image classification method based on hierarchical sparse filtering convolutional neural network | |
| CN107563428A (en) | Classification of Polarimetric SAR Image method based on generation confrontation network | |
| CN110097145A (en) | One kind being based on CNN and the pyramidal traffic contraband recognition methods of feature | |
| CN106874956A (en) | The construction method of image classification convolutional neural networks structure | |
| CN108732172A (en) | Chinese medicine performance rating method, equipment and medium | |
| CN103839078B (en) | A kind of hyperspectral image classification method based on Active Learning | |
| CN108764138A (en) | A kind of highlands cloud snow sorting technique cascading forest based on the more granularities of multidimensional | |
| CN111222545B (en) | Image classification method based on linear programming incremental learning | |
| CN108416353A (en) | Crop field spike of rice fast partition method based on the full convolutional neural networks of depth | |
| CN109344891A (en) | A kind of high-spectrum remote sensing data classification method based on deep neural network | |
| CN110163101B (en) | Method for rapidly distinguishing seeds of traditional Chinese medicinal materials and rapidly judging grades of seeds | |
| CN106600595A (en) | Human body characteristic dimension automatic measuring method based on artificial intelligence algorithm | |
| CN112949738B (en) | Multi-class unbalanced hyperspectral image classification method based on EECNN algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |