CN110516596A - Empty spectrum attention hyperspectral image classification method based on Octave convolution - Google Patents

Empty spectrum attention hyperspectral image classification method based on Octave convolution Download PDF

Info

Publication number
CN110516596A
CN110516596A CN201910797299.0A CN201910797299A CN110516596A CN 110516596 A CN110516596 A CN 110516596A CN 201910797299 A CN201910797299 A CN 201910797299A CN 110516596 A CN110516596 A CN 110516596A
Authority
CN
China
Prior art keywords
octave
convolutional neural
neural networks
loss
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910797299.0A
Other languages
Chinese (zh)
Other versions
CN110516596B (en
Inventor
唐旭
孟凡波
马晶晶
焦李成
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian University of Electronic Science and Technology
Original Assignee
Xian University of Electronic Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian University of Electronic Science and Technology filed Critical Xian University of Electronic Science and Technology
Priority to CN201910797299.0A priority Critical patent/CN110516596B/en
Publication of CN110516596A publication Critical patent/CN110516596A/en
Application granted granted Critical
Publication of CN110516596B publication Critical patent/CN110516596B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/10Terrestrial scenes
    • G06V20/13Satellite images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/10Terrestrial scenes
    • G06V20/194Terrestrial scenes using hyperspectral data, i.e. more or other wavelengths than RGB
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/10Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Molecular Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Astronomy & Astrophysics (AREA)
  • Remote Sensing (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

A kind of empty spectrum attention hyperspectral image classification method based on Octave convolution disclosed by the invention, solves the problems, such as that prior art the same category spacing is big, different classes of spacing is small, classification accuracy is low.Scheme is: image to be classified input builds with data prediction, division training set and test set, Octave convolutional neural networks, determines Octave convolutional neural networks loss function, the training of Octave convolutional neural networks updates, test set data test, completes classification hyperspectral imagery.The present invention strengthens character representation using Octave convolution operation, introduces spatial attention mechanism and spectrum attention mechanism, finds network more accurately for classifying more advantageous and including the more comprehensive detailed region of information.Nicety of grading of the present invention is high, and strong robustness can be applied to the analysis and management of hyperspectral image data.

Description

Empty spectrum attention hyperspectral image classification method based on Octave convolution
Technical field
The invention belongs to technical field of image processing, are related specifically to the classifying content of high spectrum image, specifically a kind of Spatial spectral attention hyperspectral image classification method based on Octave convolution, can be applied to the analysis of hyperspectral image data And management.
Background technique
As the resolution ratio of high spectrum image pixel is continuously improved, can be obtained from high spectrum image more useful Data and information.And it is directed to the demand of different application, also there is different requirements to the processing of high spectrum image, so in order to have Effect ground is analyzed and is managed to these hyperspectral image data, needs to stick semanteme for each pixel of high spectrum image Label.And classification hyperspectral imagery is exactly a kind of important channel for solving the problems, such as such.What classification hyperspectral imagery referred to be exactly from The pixel with similar features is distinguished in one high spectrum image, and is correctly classified to these pixels.It compares In natural image, high spectrum image itself has the characteristics of there is itself, sky of the classification results due to high spectrum image itself Between resolution ratio limitation and the different spectrum of jljl, foreign matter with spectrum phenomenon presence, usually will cause the phenomenon that mistake is divided, this is by bloom Caused by the complexity of spectrogram picture itself.Therefore, how more accurately high spectrum image progress Accurate classification to be also become and is worked as A preceding huge challenge.
Classification based on convolutional neural networks, refer to will need training some data, in batches be input to convolution mind In network, by the repetition training of high-volume data, so that objective optimization loss function constantly reduces, to realize classification Purpose.Nowadays there are many more mature, famous convolutional neural networks to be suggested, such as mentioned by He Kaiming et al. within 2015 The depth residual error convolutional neural networks for image classification task out, are widely used by everybody.Depth residual error convolutional Neural net Network effective solution image classification task extracted feature includes the problem of information is insufficient, network training gradient disappears, but This network there are still because image data it is complicated caused by the same category spacing is big, different classes of spacing is small, image classification is quasi- The true low problem of rate and because training sample it is few caused by network robustness is lower, is easily trapped into the problem of over-fitting.
Although existing convolutional neural networks can be realized the task of EO-1 hyperion Pixel-level classification, but in study image language The deficiency of three aspects is still had when adopted information: first is that the positioning of the classification information as caused by high spectrum image complexity is not Accurately, it carries out causing same category of spacing larger when classification task, different classes of spacing is smaller;Second is that for EO-1 hyperion spy When sign is extracted, causes information to lose or retain excessive irrelevant information the utilization rate deficiency of the feature of extraction and cause information superfluous It is remaining, classification results are influenced, while rolling up neural network can usually fall into local optimum region in training;Third is that available bloom Modal data is fewer, and training convolutional neural networks usually require a large amount of training data, and a small amount of high-spectral data cannot expire The data requirements of sufficient convolutional neural networks.These three deficiencies, which will lead to, there is robust in the assorting process of practical high spectrum image Property it is poor and lead to the problem of mistake and divide.
Summary of the invention
Present invention aims at above-mentioned prior art there are aiming at the problem that, propose that a kind of classification accuracy is higher and be based on The hyperspectral image classification method of the empty spectrum attention mechanism deep learning of Octave convolution.
The present invention is a kind of classification hyperspectral imagery side of empty spectrum attention mechanism deep learning based on Octave convolution Method, which is characterized in that comprise the following steps that
(1) image input and data prediction: high spectrum image to be sorted is inputted, is carried out centered on each pixel Point sliding pixel-by-pixel, all image blocks slided are for establishing high spectrum image library { I1,I2,…,In,…,IN, image The corresponding classification of each image block is { Y in library1,Y2,…,Yn,…,YN, and normalizing is carried out to the high spectrum image library of foundation Change is handled, wherein InN-th image block, Y in representative image librarynThe corresponding classification of n-th image block in representative image library, n are represented N-th sample number in image library, n ∈ [0, N], N represent the image block total number in high spectrum image library;
(2) training set and test set are divided: selecting specified number at random from every class high spectrum image after normalized The high spectrum image sample of amount constructs training sample set { T1,T2,…,Tj,…,TM, using remaining high spectrum image as survey Try sample set { t1,t2,…td,…,tm, wherein TjIndicate j-th of sample in training sample, j ∈ [0, M], tdIndicate test specimens D-th of sample in this, d ∈ [0, m], M are the total number of training sample, and m is the total number of test sample, m < N, M < N;
(3) Octave convolutional neural networks are built: building an Octave convolutional neural networks, the input terminal of network is Octave convolution module, the output end of network are the output of full articulamentum as a result, wrapping between the input terminal and output end of network Containing two branches, wherein a branch successively passes through space transforms power module and Pixel-level pays attention to power module, another branch according to It is secondary to notice that power module and Pixel-level pay attention to power module by spectrum;
(4) Octave convolutional neural networks loss function loss is determinedop: setting loss function includes that will pass through Fusion Features The feature extracted afterwards is input to the cross entropy loss of full articulamentum output category result and actual result obtained from1, will be through It crosses the feature that space transforms power module and Pixel-level notice that power module is extracted and is input to full articulamentum output category obtained from As a result with the cross entropy loss of actual result2, by by spectrum pay attention to power module and Pixel-level pay attention to power module extract feature It is input to the cross entropy loss of full articulamentum output category result and actual result obtained from3With the convolution for having hyper parameter Four part of L2 norm of neural network weight W, the loss function of network is successively added by above four part to be constituted;
(5) training updates: the number of iterations that network training is arranged is P, by gradient decline optimization to Octave convolution mind It is iterated training through network, until loss function lossopDo not decline or exercise wheel number reaches the number of iterations, is trained Octave convolutional neural networks;
(6) test sample collection after normalized data test: is input to trained Octave convolution mind In network, classification results are obtained, complete image classification.
Present invention employs Octave convolution operations, and have incorporated attention mechanism, and being provided with one includes Octave volumes The depth convolutional network model of product operation and attention mechanism, this model only need a small amount of training data that can train effect The high frequency section of hyperspectral image data and low frequency part have been carried out effective knot by the preferable model of fruit, Octave convolution operation It closes, the information that the feature of extraction includes is more comprehensively in detailed, and utilization rate is higher, while attention mechanism can promote network faster Speed effectively finds characteristic area more favorable for classification task, keeps the information of network capture more comprehensively accurate, effectively solves The problem that classification accuracy is lower, robustness is not strong in current classification hyperspectral imagery task of having determined.
The present invention has the advantage that compared with prior art
Character representation enhancing: Octave convolution operation is introduced into hyperspectral image classification method for the first time by the present invention, It is specially equipped with Octave convolution module in Octave convolutional neural networks, obtains height by being then based on Octave convolution operation Spectrum picture feature, the high-frequency information of high spectrum image and low-frequency information be rationally effectively combined together with, make height The information that spectrum picture feature includes is more comprehensively and detailed, enhances the character representation of image.
Classification accuracy improves: attention mechanism is introduced into hyperspectral image classification method, In by the present invention Spatial attention mechanism module, spectrum attention mechanism module and Pixel-level is specially equipped in Octave convolutional neural networks to pay attention to Power mechanism module can promote network fast and accurately to find the high-spectrum obtained due to introducing attention mechanism principle As feature region the most apparent in feature, make more to concentrate on more favorable feature of classifying a certain with obvious semantic The region of information reduces the probability that loss function falls into local optimum, enhances the accuracy of classification hyperspectral imagery.
Robustness enhancing: the present invention devises a more effective loss function, and new loss function utilizes three intersections Entropy loss function promotes e-learning to the more effective feature of classification hyperspectral imagery, strengthens the character representation of image, into One step specifies classification task, and purpose is stronger, can adapt to complicated hyperspectral image data, greatly enhances network Robustness.
Training sample is few: present invention only requires a small amount of samples can train the preferable network model of effect, to height The data volume of spectrum picture requires smaller.
Detailed description of the invention
Fig. 1 is implementation flow chart of the invention;
Fig. 2 is the Octave convolutional neural networks structure chart constructed in the present invention;
Fig. 3 is the high spectrum image of experiment in the present invention, and wherein Fig. 3 (a) is original high spectrum image, and Fig. 3 (b) is The class label of original high spectrum image corresponding pixel points;
Fig. 4 is Octave convolution module structure chart;
Fig. 5 is spatial attention mechanism module structure chart;
Fig. 6 is attention mechanism module structure chart between spectrum;
Fig. 7 is Pixel-level attention mechanism module structure chart.
Specific embodiment
Technical solutions and effects of the present invention is described in detail below in conjunction with attached drawing.
Embodiment 1
In recent decades, high spectral resolution provided useful information, EO-1 hyperion to distinguish different materials and object Image classification method is widely used in earth observation, especially in urban development, precision agriculture, land change inspection Survey, resource management etc. have great importance.Presently, there are classification hyperspectral imagery method in, referring to Fig. 1, pass through Building high spectrum image library, divide training sample set and test sample collection, build and training convolutional neural networks model, to training Good convolutional neural networks model carries out the classification tested etc. Wan Chengs for high spectrum image, but due to the complexity of high spectrum image The problem of property, leads to classification information position inaccurate, and trained network is easily ensnared into local optimum;Simultaneously because network is deep For degree compared with deep, network operation is more complicated, the feature of extraction includes that information is insufficient, leads to carry out same category occur when classification task Larger, the different classes of small-pitch problem of spacing, the phenomenon that being easy to appear mistake classification, robustness is poor.
For this status, the present invention expands research and discussion, proposes that a kind of empty spectrum based on Octave convolution pays attention to The hyperspectral image classification method of power mechanism deep learning comprises the following steps that referring to fig. 2
(1) image input and data prediction: inputting high spectrum image to be sorted, high spectrum image spectral band compared with It is more, and different-waveband differs greatly, while being influenced by environmental factors such as illumination, temperature, and same category of EO-1 hyperion is belonged to Image differs greatly, and the high spectrum image difference to belong to a different category is little;For hyperspectral image data, in high-spectrum It is put sliding pixel-by-pixel centered on each pixel as on, the sliding sash of 9*9 size is chosen in this example, sliding sash size can basis Actual conditions are adjusted, and are carried out point pixel-by-pixel using sliding sash and are slided, the distance slided every time is a pixel, is slided every time The high spectrum image block of an all available 9*9 size, all high spectrum image blocks slided are for establishing high-spectrum As library { I1,I2,…,In,…,IN, the corresponding classification of each image block is respectively { Y in image library1,Y2,…,Yn,…,YN}, For established high spectrum image library, the maximum value and minimum value of all pixels point in image library are found, all pixels are utilized The high spectrum image library of foundation is normalized in the value of point and the most value of two pixels, wherein InIn representative image library N-th image, YnThe corresponding classification of n-th image in representative image library, n-th of sample number in n representative image library, n ∈ [0, N], N represents the picture total number in high spectrum image library.
(2) training set and test set are divided: being selected at random from the high spectrum image of each classification after normalized The high spectrum image sample of specified quantity constructs training sample set { T1,T2,…,Tj,…,TM, abbreviation training set will be remaining High spectrum image is as test sample collection { t1,t2,…td,…,tmThis test sample collection be normalized test sample collection, Abbreviation test set.Training sample concentrates TjIndicate j-th of sample in training sample, j ∈ [0, M], test sample concentrates tdIt indicates to survey D-th of sample in sample sheet, d ∈ [0, m], M are the total number of training sample, and m is the total number of test sample, m < N, M < N.
The mode selected at random is respectively adopted for each classification when selecting sample building training sample set by the present invention, The training sample set chosen in this way may include the sample of all categories, while the institute for being included by each classification of maximum possible It is possible that property sample is all divided among training sample set.
When constructing high spectrum image training sample set, the training sample for randomly selecting specified quantity is usually art technology The Normal practice of personnel, is primarily due in hyperspectral image data, and high spectrum image sample is few and every other sample of type Quantitative difference it is larger, if dividing training sample set and test sample collection in proportion will lead to point of the few EO-1 hyperion classification of sample Class accuracy rate is very low.
(3) Octave convolutional neural networks are built: build an Octave convolutional neural networks, referring to fig. 2, network it is defeated Entering end is Octave convolution module, the output end of network be the output of full articulamentum as a result, network input terminal and output end Between include two branches, wherein a branch successively passes through space transforms power module and Pixel-level pays attention to power module, another Branch successively passes through spectrum and notices that power module and Pixel-level pay attention to power module.
Octave convolution module of the present invention has fully considered the high-frequency information and low-frequency information of high spectrum image, passes through convolution The high-frequency information of high spectrum image and low-frequency information are effectively combined by operation, are enhanced the character representation of image, are made The information that characteristics of image includes is more comprehensive;Space transforms power module and Pixel-level notice that power module makes the attention of whole network Concentrate on a certain region with obvious semantic information, make whole network during to hyperspectral classification, make full use of for Classify best region, catches most apparent characteristic area, improve the classification accuracy of EO-1 hyperion;Spectrum pay attention to power module and Pixel-level notices that power module makes whole network focus on spectral band the most useful for classification hyperspectral imagery, dashes forward The more good spectral band of character representation out is conducive to improve hyperspectral classification accuracy rate, improves the robustness of network.
(4) Octave convolutional neural networks loss function loss is determinedop: setting loss function includes that will pass through Fusion Features The feature extracted afterwards is input to the cross entropy loss of full articulamentum output category result and actual result obtained from1, will be through It crosses the feature that space transforms power module and Pixel-level notice that power module is extracted and is input to full articulamentum output category obtained from As a result with the cross entropy loss of actual result2, by by spectrum pay attention to power module and Pixel-level pay attention to power module extract feature It is input to the cross entropy loss of full articulamentum output category result and actual result obtained from3With the convolution for having hyper parameter Four part of L2 norm of neural network weight W, the loss function of network is successively added by above four part to be constituted.
Loss function loss of the inventionopPromote e-learning to classification hyperspectral imagery using three intersection entropy functions More effective feature reinforces the character representation of high spectrum image, has further clarified classification task, and purpose is stronger, Neng Goushi Complicated hyperspectral image data is answered, greatly enhances the robustness of network, while effectively reducing loss function in net The probability of local optimum is fallen into network training process.
(5) training updates: the number of iterations that network training is arranged is P, by gradient decline optimization to Octave convolution mind It is iterated training through network, until loss function lossopDo not decline or exercise wheel number reaches the number of iterations, is trained Octave convolutional neural networks.The number of iterations P is to be manually set, and can be adjusted according to the training effect of network, make height Spectrum picture classification accuracy highest.In network training process, the learning rate of network is gradually reduced with the training of network, just Learning rate is larger when beginning to train, and with trained intensification, learning rate is gradually reduced, and network is made to effectively reduce the loss of network Function falls into the probability of local optimum, is conducive to the raising of classification hyperspectral imagery accuracy rate, robustness enhancing.
(6) test sample collection after normalized data test: is input to trained Octave convolution mind In network, classification results are obtained, complete image classification.
The high spectrum image point for the empty spectrum attention mechanism deep learning based on Octave convolution that the present invention provides one The overall technical architecture of class method.
Technical thought of the invention is: building an Octave convolutional neural networks, is obtained using Octave convolution operation For each pixel region, include the more comprehensive convolution feature of information;According to spatial attention mechanism principle and pixel Grade attention mechanism principle, focuses on a certain region with obvious semantic information for the attention of whole network, finds and be conducive to The region of the useful information of classification;According to spectrum attention mechanism principle and Pixel-level attention mechanism principle, feature is found more For protrusion, the stronger spectral band of spectral information, the stronger character representation for being conducive to classification is obtained;Pass through above-mentioned attention mechanism The attention of whole network is focused on for most effective region of classifying, full articulamentum network implementations image point is then passed through Class.
The present invention solve present in current classification hyperspectral imagery the same category spacing is big, different classes of spacing is small, The low problem of image classification accuracy rate and because training sample it is few caused by network robustness is lower, is easily trapped into over-fitting Problem.
The present invention is effectively obtained by Octave convolution module comprising the more comprehensive convolution feature of information, is enhanced It is more bright to notice that power module makes network find semantic information using space transforms power module and Pixel-level for the character representation of image Aobvious, feature region outstanding, notices that power module and Pixel-level notice that power module finds the stronger spectrum of spectral information using spectrum Wave band finds characteristic area more effective for classification hyperspectral imagery, improves height by the combination of several attention mechanism The accuracy rate of spectrum picture classification, enhances the robustness of network.
Embodiment 2
The hyperspectral image classification method of empty spectrum attention mechanism deep learning based on Octave convolution with embodiment 1, Octave convolutional neural networks described in step (3) of the present invention are built, and referring to fig. 2, the present invention forms Octave convolutional Neural The Octave convolution module of network, space transforms power module, spectrum notice that power module, Pixel-level pay attention to power module and full connection Layer respectively forms module parameter and is provided that
Octave convolution module, that is, input module is made of, each conventional part sequentially connected four conventional parts It again include Octave convolution, referring to fig. 4, Batch Normalization and Relu activation primitive, in second and third convolution portion / there are one maximum pond layers.
High spectrum image is mainly first divided into two parts of high frequency section and low frequency part by Octave convolution operation, is passed through Conventional convolution operation carries out convolution to high frequency section and low frequency part, obtains by high frequency to high frequency, high frequency to low frequency, low frequency to height Frequently, low frequency is added to together to four convolution results of low frequency, then by the convolution results of high frequency to high and low frequency to high frequency, high frequency It is added to together to low frequency and low frequency to the convolution results of low frequency, thus by the high-frequency information of high spectrum image and low-frequency information Carried out effective connection and linked up, make the character representation obtained include information more comprehensively and in detail, enhance high-spectrum The character representation of picture.
Spatial attention mechanism module activates letter by convolutional layer, Batch Normalization, Relu referring to Fig. 5 Number, matrix transposition are constituted with the layer that is multiplied, softmax layers and data transposition with layer is added.It will enter into spatial attention mechanism mould The feature of block strengthens feature by a convolution operation, is input to matrix transposition and the layer that is multiplied, obtains a matrix, square Each element represents the spatial relationship of any two position in high spectrum image in battle array;Recycle softmax layers to this matrix into Row normalized normalizes to the value in matrix between 0 to 1, while by the matrix after normalization and passing through convolution operation The feature obtained afterwards is multiplied, a certain region with obvious semantic information in prominent high spectrum image, obtains having obvious semantic The characteristic pattern that information indicates;Finally by with obvious semantic information feature be input to the initial of spatial attention mechanism module Feature be input to data transposition be added in layer, two features are added to the loss for preventing information together.By space transforms The obtained last Feature Semantics information of power mechanism module is obvious, and network is made to be easily found the most apparent region of feature, Improve the accuracy rate of classification hyperspectral imagery.
Spectrum attention mechanism module, referring to Fig. 6, by matrix transposition and the layer that is multiplied, softmax layers and data transposition with Layer is added to constitute.The feature that will enter into spectrum attention mechanism module is input to matrix transposition and the layer that is multiplied, and obtains a square Gust, each element represents the relationship in high spectrum image between the different spectral bands of any pixel in matrix;It recycles Softmax layers are normalized this matrix, and the value in matrix is normalized between 0 to 1, while will be after normalization Matrix notices that the initial characteristics of power module are multiplied with spectrum is input to, spectrum letter in each pixel in prominent high spectrum image Strongest spectral band is ceased, the characteristic pattern of prominent most strong spectral information is obtained;Finally will the prominent most feature of strong spectral information with Be input to spectrum pay attention to the initial characteristics of power module be input to data transposition be added layer, by two features be added to together, prevent The only loss of information.The strongest spectrum wave of spectral information is highlighted by the last feature that spectrum attention mechanism module obtains Section, makes network focus on the strongest region of spectral information, improves the accuracy rate of classification hyperspectral imagery, enhance net The robustness of network.
Pixel-level attention mechanism module is activated referring to Fig. 7 by convolutional layer, Batch Normalization and Relu Function is constituted.Classification hyperspectral imagery is classified to pixel each in high spectrum image, by Pixel-level attention machine The feature that molding block obtains more has refined the feature of each pixel, enhances the character representation of each pixel, mentions The high accuracy rate of classification hyperspectral imagery.
The full articulamentum is to be made of the first full articulamentum and the second full articulamentum and softmax layers, that is, exports Layer.The input of first full articulamentum is the feature obtained after space transforms power module and Pixel-level pay attention to power module, passes through Spectrum pays attention to the feature obtained after power module pays attention to power module with Pixel-level and obtains after being added fusion by two kinds of features Fusion feature;The input of second full articulamentum is the output feature of the first full articulamentum;Softmax layers of input connects entirely for second The output feature of layer is connect, softmax layers of output result indicates that training sample belongs to the probability of a certain classification in EO-1 hyperion, Softmax layers of output result is the final output of whole network.
The present invention strengthens character representation using Octave convolution operation, introduces spatial attention mechanism and spectrum attention Mechanism finds network more accurately for classifying more advantageous and including the more comprehensive detailed region of information, enhances height The accuracy and network robustness of spectrum picture classification.
Embodiment 3
The hyperspectral image classification method of empty spectrum attention mechanism deep learning based on Octave convolution is the same as embodiment 1- 2, determination Octave convolutional neural networks loss function described in step 4 lossop, specifically comprise the following steps:
(4a) is by training image library { T1,T2,…,Tj,…,TMIt is input to Octave volumes of Octave convolutional neural networks Volume module exports the last layer feature F of convolutional layer.
The last layer feature F is separately input to the space transforms power module and spectrum of Octave convolutional neural networks by (4b) Pay attention to power module, output feature is respectively A and B, then output feature A and B are input to Pixel-level and pay attention to power module, exports feature Respectively C and D.
The feature C and D that (4c) will be obtained, are input to the full articulamentum of Octave convolutional neural networks, and output utilizes feature C The output category result obtained with D;Simultaneously then feature C and D are added pixel-by-pixel, are obtained respectively multiplied by a coefficient Fused feature E, then feature E is input to the full articulamentum of Octave convolutional neural networks, output utilizes fused spy The output category result that sign E is obtained, obtains the loss function loss of Octave convolutional neural networksop:
Wherein, loss1To utilize the friendship of fused feature E output category result and actual result after full articulamentum Pitch entropy, loss2It is characterized the cross entropy of C output category result and actual result after full articulamentum, loss3It is characterized D process The cross entropy of output category result and actual result after full articulamentum,For the L2 norm of convolutional neural networks weight vectors, η ForHyper parameter.
Loss function loss of the inventionopPromote e-learning to classification hyperspectral imagery using three intersection entropy functions More effective space characteristics and spectral signature, reinforce the character representation of high spectrum image, have further clarified classification task, purpose Property it is stronger, can adapt to complicated hyperspectral image data, greatly enhance the robustness of network, effectively reduce simultaneously Loss function falls into the probability of local optimum in network training process.
Embodiment 4
The hyperspectral image classification method of empty spectrum attention mechanism deep learning based on Octave convolution is the same as embodiment 1- 3, the cross entropy loss of output category result and actual result in (4c)1, formula is as follows:
Wherein, yjFor T in training image libraryjPrediction category probability, ojFor T in training image libraryjPractical category; loss1Input be the prediction category probability that is obtained after full articulamentum of fused feature E;
Loss in the present invention2、loss3Principle and formula express and loss1It is identical, only loss2Input be characterized C The prediction category probability obtained after full articulamentum, loss3Input be characterized the prediction class that D is obtained after full articulamentum Mark probability.
Embodiment 5
The hyperspectral image classification method of empty spectrum attention mechanism deep learning based on Octave convolution is the same as embodiment 1- 4, step is normalized high spectrum image library in (1), is carried out by following formula:
Wherein VmaxFor the point maximum value of all pixels in high spectrum image library, VminFor all pixels in high spectrum image library Point minimum value, VnFor the pixel value at any point in high spectrum image library, { I '1,I′2,…,I′n,…,I′NIt is at normalization High spectrum image library after reason, I 'nFor n-th of sample of high spectrum image after normalized, n ∈ [0, N].
The present invention is restricted to -0.5 to 0.5 by the way that high spectrum image library is normalized, by EO-1 hyperion pixel value Between, keep high spectrum image luminance distribution more balanced, effectively avoids subsequent processing bring from interfering, while by each pixel The pixel value of point limits another unified section, prevents pixel value span larger, marginal information is erased.Since normalization makes The pixel value of high spectrum image reduces, and reduces the calculation amount of network, while accelerating the convergence of network training.Pass through experiment It also demonstrates to normalize high spectrum image pixel value and further improves the accurate of classification hyperspectral imagery between -0.5 to 0.5 Rate, while network training speed is greatly speeded up.
Embodiment 6
The hyperspectral image classification method of empty spectrum attention mechanism deep learning based on Octave convolution is the same as embodiment 1- 5, training is iterated to convolutional neural networks by gradient decline optimization in step (5), is accomplished by
The initial learning rate of (5a) setting training is L, attenuation rate β, by training image library { T1,T2,…,Tj,…,TM} It is divided into the convolutional neural networks of G input building, the number of pictures inputted every time is Q, then:
Wherein M is the total number of training image library sample.
(5b) sets the corresponding learning rate l of input picture every time are as follows:
L=L* βG
(5c) carries out the update of G subparameter to convolutional neural networks by following formula, obtains updated weight vectors Wnew
Wherein, W is the weight vectors of convolutional neural networks parameter;
(5d) will train picture to input convolutional neural networks, loss function loss updated to weight vectors next timeop It is updated, so that loss function lossopValue constantly decline;
(5e) repeats (5d), until loss function lossopNo longer decline, and current exercise wheel number is less than the iteration of setting Number P then stops the training to the network, obtains trained convolutional neural networks;Otherwise, when training round reaches setting When the number of iterations P, stops the training to the network, obtain trained convolutional neural networks.
The number of pictures Q inputted every time in the present invention is to be manually set, and can be adjusted according to the training effect of network, Make classification hyperspectral imagery accuracy rate highest.The learning rate of network is the rate of e-learning validity feature, in network training In the process, the learning rate of network is gradually reduced with the training of network, and learning rate is larger when initial training, promotes network quickly high The main feature of the study high spectrum image of effect, with trained intensification, learning rate is gradually reduced, and e-learning speed slows down, Promote e-learning to be conducive to the detailed features of classification hyperspectral imagery, while network being made to effectively reduce the loss letter of network Number falls into the probability of local optimum, accelerates the speed of network training, accelerates the convergence of network training.
Provide a more detailed example again below, the present invention is further described:
Embodiment 7
The hyperspectral image classification method of empty spectrum attention mechanism deep learning based on Octave convolution is same to implement 1-6, Referring to fig. 2, steps are as follows for realization of the invention:
Step 1, high spectrum image library is established, training sample and test sample are obtained.
1a) Indian Pines hyperspectral image data collection, Indian Pines high spectrum image are downloaded from related official website Data set by airborne visual Infrared Imaging Spectrometer (AVIRIS) in 1992 to one piece of India pine tree of Indiana, USA into Row imaging, referring to Fig. 3, Fig. 3 (a) is original Indian Pines high spectrum image, and Fig. 3 (b) is original Indian The class label of Pines high spectrum image corresponding pixel points.It is carried out pixel-by-pixel centered on each pixel on high spectrum image Point sliding, chooses the sliding sash of 13*13 size, carries out point pixel-by-pixel using sliding sash and slides, the distance slided every time is a pixel Point, the high spectrum image block of all available 13*13 size of sliding, all high spectrum image blocks slided are used for every time Establish high spectrum image library { I1,I2,…,In,…,IN, the corresponding classification of image library is { Y1,Y2,…,Yn,…,YN, wherein In N-th image in representative image library, YnThe corresponding classification of n-th image in representative image library, n-th of sample in n representative image library This number, n ∈ [0, N].
It 1b) is directed to established high spectrum image library, finds the maximum value and minimum value of all pixels point in image library, benefit The high spectrum image library of foundation is normalized according to following formula with the value of all pixels point and the most value of two pixels Processing:
Wherein VmaxFor the point maximum value of all pixels in high spectrum image library, VminFor all pixels in high spectrum image library Point minimum value, VnFor the pixel value at any point in high spectrum image library, { I '1,I′2,…,I′n,…,I′NIt is at normalization High spectrum image library after reason, I 'nFor n-th of sample of remote sensing images after normalized, n ∈ [0, N].
1c) select the high spectrum image of specified quantity at random from the high spectrum image of each classification after normalized Sample constructs training sample set { T1,T2,…,Tj,…,TM, abbreviation training set, using remaining high spectrum image as test specimens This collection { t1,t2,…td,…,tmAbbreviation test set, wherein TjIndicate j-th of sample in training sample, j ∈ [0, M], tdIt indicates D-th of sample in test sample, d ∈ [0, m], M are the total number of training sample, and m is the total number of test sample, m < N, M < N.
Step 2, Octave convolutional neural networks are constructed.
Octave convolution module 2a) is set, is made of sequentially connected four conventional parts, each conventional part wraps again Octave convolution is included, Batch Normalization and Relu activation primitive, there are also one between second and third conventional part A maximum pond layer;
Referring to fig. 4, Octave convolution working principle of the invention is as follows:
Two parts of high and low frequency are divided the image into, wherein the one of the wide and a height of high frequency section of the image of low frequency part Half;Carry out common convolution operation to high frequency section, obtain two convolution results, the width of medium-high frequency to high frequency convolution results and High identical as high frequency section, the width and height of high frequency to low frequency convolution results are identical as low frequency part;Again to low frequency part into The identical convolution operation of row, the wherein width of low frequency to high frequency convolution results and height, low frequency to low frequency convolution identical as high frequency section As a result width and height is identical as low frequency part;Again by with identical wide and high results added to forming new radio-frequency head together Point and low frequency part.
The Relu activation primitive are as follows:
Wherein x is the input function of Relu activation primitive.
2b) installation space attention mechanism module activates letter by convolutional layer, Batch Normalization, Relu Number, matrix transposition are constituted with the layer that is multiplied, softmax layers and data transposition with layer is added, and structure is as shown in Figure 5.
Matrix transposition is the spy obtained by convolution with the layer that is multiplied, the feature of input in space transforms power module of the present invention Sign, size are W × H × C, feature sizes are first converted to N × C, wherein N=W × H, then carry out square to the feature after conversion Battle array transposition, the size of obtained feature are C × N, then obtained by the feature obtained by Feature Conversion and by matrix transposition Feature carries out matrix multiple, obtains output matrix, and size is N × N.
Softmax layers in space transforms power module of the present invention, using softmax to the square of matrix transposition and the layer output that is multiplied Battle array is normalized, and the value in matrix is normalized between 0 to 1, while by the matrix after normalization and passing through convolution Operation is multiplied with the feature that matrix conversion obtains, and obtains output feature, and size is N × C.
In space transforms power module of the present invention data transposition be added layer, first softmax layers of output result is turned It changes, size is W × H × C after converting, then will pass through the feature being converted to and be input to the initial of space transforms power module Feature is added to the final output feature for obtaining space transforms power module together, and size is W × H × C.
Space transforms power module is by convolutional layer, Batch Normalization, Relu activation primitive, matrix transposition and phase Multiply layer, softmax layers and data transposition and be sequentially connected composition with layer is added, network can be made to find band by space transforms power module There is the region of obvious semantic information, and then improves the accuracy rate of classification hyperspectral imagery.
Spectrum attention mechanism module 2c) is set, by matrix transposition and the layer that is multiplied, softmax layers and data transposition and It is added layer to constitute, structure is as shown in Figure 6.
Spectrum of the present invention pays attention to matrix transposition and the layer that is multiplied in power module, and the feature sizes of input are W × H × C, first will be special Sign size is converted to C × N, wherein N=W × H, then carries out matrix transposition to the feature after conversion, and the size of obtained feature is N × C, then will carry out matrix multiple by the feature that Feature Conversion and matrix transposition obtain, obtains output matrix, size be C × C。
Spectrum of the present invention pays attention in power module softmax layers, using softmax to the square of matrix transposition and the layer output that is multiplied Battle array is normalized, and the value in matrix is normalized between 0 to 1, while by the matrix after normalization and passing through matrix The feature being converted to is multiplied, and obtains output feature, and size is C × N.
Spectrum of the present invention pay attention in power module data transposition be added layer, first softmax layers of output result is turned It changes, size is W × H × C after converting, then will pay attention to the initial of power module with spectrum is input to by the feature being converted to Feature is added to and obtains the final output feature that spectrum pays attention to power module together, and size is W × H × C.
Spectrum notices that power module is sequentially connected by matrix transposition with the layer that is multiplied, softmax layers and data transposition with layer is added It constitutes, notices that power module makes network find the stronger spectral band of spectral information by spectrum, and then improve high spectrum image point The accuracy rate of class and the robustness of network.
Pixel-level attention mechanism module 2d) is set, is activated by convolutional layer, Batch Normalization and Relu Function is sequentially connected composition, and structure is as shown in Figure 7.
Present invention pixel grade notices in power module in convolutional layer that setting convolution kernel size is 1*1, is 1*1 by convolution kernel The convolution operation of size carries out characteristic strengthening to each pixel of high spectrum image, thus improves classification hyperspectral imagery Accuracy rate.
Full articulamentum 2e) is set, full articulamentum is made of the first full articulamentum, the second full articulamentum and softmax layers, The convolution kernel size of first full articulamentum is 9800 × 1024, and the convolution kernel size of the second full articulamentum is 1024 × 16.Wherein 16 be classification number total in the high spectrum image of input.
First full articulamentum in the full articulamentum of the present invention, it is first the every of input feature vector that it is 1 × 9800 that feature, which inputs size, A value is multiplied by a coefficient, then is input to the first full articulamentum, and output feature sizes are 1 × 1024.Institute's multiplying factor is initial One group changed meets the vector of Gaussian Profile, and coefficient can be updated during sample training.
Second full articulamentum in the full articulamentum of the present invention, it is 1 × 1024 that feature, which inputs size, and output feature sizes are 1 ×16。
The Octave convolution module of above-mentioned setting, space transforms power module, spectrum 2f) are paid attention into power module and full articulamentum It is sequentially connected, obtains Octave convolutional neural networks.
Step 3 determines the loss function of convolutional neural networks:
(3a) is by training sample set { T1,T2,…,Tj,…,TMIt is input to Octave volumes of Octave convolutional neural networks Volume module exports the last layer feature F of convolutional layer.
The last layer feature F is separately input to the space transforms power module and spectrum of Octave convolutional neural networks by (3b) Pay attention to power module, output feature is respectively A and B, then output feature A and B are input to Pixel-level and pay attention to power module, exports feature Respectively C and D.
The feature C and D that (3c) will be obtained, are input to the full articulamentum of Octave convolutional neural networks, and output utilizes feature C The output category result obtained with D;Simultaneously then feature C and D are added pixel-by-pixel, are obtained respectively multiplied by a coefficient Fused feature E, then feature E is input to the full articulamentum of Octave convolutional neural networks, output utilizes fused spy The output category result that sign E is obtained, obtains the loss function loss of Octave convolutional neural networksop:
Wherein:For the L2 norm of Octave convolutional neural networks weight vectors, η isHyper parameter;Indicate using fused feature E output category result and actual result after full articulamentum Cross entropy, yjFor T in training image libraryjPrediction category probability, ojFor T in training image libraryjPractical category, loss1It is defeated Enter the prediction category probability obtained after full articulamentum for fused feature E.
loss2、loss3Principle and formula express and loss1It is identical, loss2C is characterized to export after full articulamentum The cross entropy of classification results and actual result, loss3It is characterized D output category result and actual result after full articulamentum Cross entropy;loss2Input be characterized the prediction category probability that C is obtained after full articulamentum, loss3Input be characterized D warp Cross the prediction category probability obtained after full articulamentum.
Step 4, training is iterated to convolutional neural networks.
Being iterated trained existing method to Octave convolutional neural networks has gradient optimization algorithm, Nesterov Gradient acceleration method, Adagrad method use but are not limited only to gradient descent algorithm in this example, and implementation step is as follows:
4a) setting the number of iterations is P, and it is L, attenuation rate β that trained initial learning rate, which is arranged, by training image library { T1, T2,…,Tj,…,TMDivide in the convolutional neural networks for being input to step 2 building, the number of pictures inputted every time is Q, and number is G:
Wherein M is the total number of training image library sample.
4b) set the corresponding learning rate l of input picture every time are as follows:
L=L* βG
4c) by following formula to Octave convolutional neural networks carry out the update of G subparameter, obtain updated weight to Measure Wnew:
Wherein, W is the weight vectors of Octave convolutional neural networks parameter.
By updated weight vectors WnewBring 3c into) in loss function lossop, obtain the updated damage of weight vectors Lose function lossop
4d) picture will be trained to be input to Octave convolutional neural networks next time, to the updated loss letter of weight vectors Number lossopIt is updated, so that loss function lossopValue constantly decline.
4e) repeat 4d), until loss function lossopNo longer decline, and current exercise wheel number is less than the iteration time of setting Number P then stops the training to the network, obtains trained Octave convolutional neural networks;Otherwise, it is set when training round reaches When the number of iterations P set, stops the training to the network, obtain trained Octave convolutional neural networks.
Network is optimized using gradient descent algorithm in this example, finds optimal solution, but to network optimization when is unlimited In gradient descent algorithm, other optimization algorithms such as genetic algorithm etc. can still be optimized network.
Step 5 classifies to test sample collection.
Test sample collection after normalized is input in trained Octave convolutional neural networks, from instruction The Octave convolutional neural networks output perfected obtains the test sample collection classification results of the high spectrum image of input, completes to height The precise classification of spectrum picture.
That present invention mainly solves prior art the same category spacing is big, different classes of spacing is small, classification accuracy is low asks Topic.The present invention is by establishing high spectrum image library and the corresponding classification of image library, and from every class EO-1 hyperion after normalized The high spectrum image sample building training sample set and test sample collection of specified quantity are selected in image at random;Constructing one includes Octave convolution module, space transforms power module, spectrum notice that power module, Pixel-level pay attention to power module and full articulamentum Octave convolutional neural networks;The training sample that training sample is concentrated is input in Octave convolutional neural networks and is trained The classification results of sample, and determine the loss function of convolutional neural networks;By gradient descent method to loss function iteration more Newly until penalty values stabilization, trained Octave convolutional neural networks are obtained;By the test to be sorted after normalization Sample set is input to trained Octave convolutional neural networks and obtains classification results.Nicety of grading of the present invention is high, strong robustness, It can be applied to the analysis and management of hyperspectral image data.
The effect of invention can be further illustrated by following emulation:
Embodiment 8
The hyperspectral image classification method of empty spectrum attention mechanism deep learning based on Octave convolution is the same as embodiment 1- 7,
Simulated conditions
This example in HP-Z840-Workstation with Xeon (R) CPU E5-2630, GeForce TITAN XP, Under 64G RAM, Ubuntu system, on TensorFlow operation platform, the present invention and existing remote sensing images scene classification are completed Emulation.
Simulation parameter setting is as follows, and iteration round P is 175 times, and initial learning rate L is 0.00001, η=0.0001, every time Inputting picture number Q is 16, and attenuation rate β is 0.9.Learning sequence is in repetitive exercise each time, to category arbiter, classification Difference optimizer, common training.
Emulation content
Download Indian Pines hyperspectral image data collection, Indian Pines hyperspectral image data collection by it is airborne can One piece of India pine tree of Indiana, USA is imaged in 1992 depending on Infrared Imaging Spectrometer (AVIRIS), referring to Fig. 3, Fig. 3 (a) is original Indian Pines high spectrum image, and Fig. 3 (b) is that original Indian Pines high spectrum image is corresponding The class label of pixel.Centered on each pixel, the sliding sash of selection 13*13 size, the sliding put pixel-by-pixel, every time Sliding obtains an image, and all images slided are for establishing high spectrum image library { I1,I2,…,In,…,IN}.Again The high spectrum image library of foundation is normalized, i.e., first obtains high spectrum image library pixel maximum value V 'maxAnd pixel The minimum value V ' of pointmin, then to the value of all pixels point in high spectrum image library divided by V 'maxWith V 'minDifference, obtain normalizing Change treated high spectrum image library.
From a certain number of high spectrum images are selected after normalized in high spectrum image library at random as training sample Collect DT, using remaining high spectrum image as test sample collection Dt
The image that the training sample set and test sample are concentrated has 16 types, respectively Aflalfa, Corn- notill、Corn-mintill、Corn、Grass-pasture、Grass-trees、Grass-pasture-mowed、Hay- windrowed、Oats、Soybean-nottill、Soybean-mintill、Soybean-clean、Wheat、Woods、 Stone-Steel-Towers,Buildings-Grass-Trees-Drives;The other number of training of every type, test sample Several and total number of samples is referring to table 1.
1 Indian Pines hyperspectral image data collection classification statistical form of table
Under above-mentioned simulated conditions, using training sample set DTRespectively with the present invention and existing representative three kinds of images point Class model is trained, using test sample collection DtIt is tested, compares the accuracy rate of its classification, as a result such as table 2.
2 present invention of table and existing classification hyperspectral imagery model performance evaluation table
Test model Test sample accuracy rate
The present invention 0.9898
KFRC-CKIR 0.9860
2-DCNN 0.9888
3D-SRNet 0.9720
KFRC-CKIR is the existing hyperspectral classification method merged based on core canonical and core in table 2, and 2-DCNN is existing base In the hyperspectral image classification method of depth convolutional neural networks, 3D-SRNet can be divided and transfer learning to be existing based on three-dimensional Hyperspectral image classification method.
From table 2 it can be seen that with the trained convolutional neural networks of the present invention to test sample collection DtClassify, it is quasi- True rate highest, accurate rate than existing representative classification hyperspectral imagery model in all classification methods for participating in test is equal There is promotion.
In conclusion a kind of empty spectrum attention classification hyperspectral imagery side based on Octave convolution disclosed by the invention Method solves the problems, such as that prior art the same category spacing is big, different classes of spacing is small, classification accuracy is low.Scheme is: wait divide The input of class image builds with data prediction, division training set and test set, Octave convolutional neural networks, determines Octave volumes Product neural network loss function, the training update of Octave convolutional neural networks, test set data test, complete high spectrum image Classification.The present invention strengthens character representation using Octave convolution operation, introduces spatial attention mechanism and spectrum attention machine System finds network more accurately for classifying more advantageous and including the more comprehensive detailed region of information.Present invention classification Precision is high, and strong robustness can be applied to the analysis and management of hyperspectral image data.

Claims (6)

1. a kind of hyperspectral image classification method of the empty spectrum attention mechanism deep learning based on Octave convolution, feature exist In comprising the following steps that
(1) image input and data prediction: high spectrum image to be sorted is inputted, is carried out centered on each pixel by picture Vegetarian refreshments sliding, all image blocks slided are for establishing high spectrum image library { I1,I2,…,In,…,IN, in image library The corresponding classification of each image block is { Y1,Y2,…,Yn,…,YN, and place is normalized to the high spectrum image library of foundation It manages, wherein InN-th image block, Y in representative image librarynThe corresponding classification of n-th image block in representative image library, n representative image N-th of sample number in library, n ∈ [0, N], N represent the image block total number in high spectrum image library;
(2) training set and test set are divided: selecting specified quantity at random from every class high spectrum image after normalized High spectrum image sample constructs training sample set { T1,T2,…,Tj,…,TM, using remaining high spectrum image as test specimens This collection { t1,t2,…td,…,tmWherein TjIndicate j-th of sample in training sample, j ∈ [0, M], tdIt indicates the in test sample D sample, d ∈ [0, m], M are the total number of training sample, and m is the total number of test sample, m < N, M < N;
(3) Octave convolutional neural networks are built: building an Octave convolutional neural networks, the input terminal of network is Octave Convolution module, the output end of network are the output of full articulamentum as a result, including two between the input terminal and output end of network Branch, wherein a branch successively passes through space transforms power module and Pixel-level pays attention to power module, another branch successively passes through Spectrum notices that power module and Pixel-level pay attention to power module;
(4) Octave convolutional neural networks loss function loss is determinedop: setting loss function includes that will mention after Fusion Features The feature taken is input to the cross entropy loss of full articulamentum output category result and actual result obtained from1, will by sky Between notice that power module and Pixel-level notice that power module is extracted feature be input to full articulamentum output category result obtained from With the cross entropy loss of actual result2, by by spectrum pay attention to power module and Pixel-level pay attention to power module extract feature input To the cross entropy loss of full articulamentum output category result and actual result obtained from3With the convolutional Neural for having hyper parameter Four part of L2 norm of network weight W, the loss function of network is successively added by above four part to be constituted;
(5) training updates: the number of iterations that network training is arranged is P, by gradient decline optimization to Octave convolutional Neural net Network is iterated training, until loss function lossopDo not decline or exercise wheel number reaches the number of iterations, obtains trained Octave convolutional neural networks;
(6) test sample collection after normalized data test: is input to trained Octave convolutional Neural net In network, classification results are obtained, complete image classification.
2. the high spectrum image point of the empty spectrum attention mechanism deep learning according to claim 1 based on Octave convolution Class method, which is characterized in that Octave convolutional neural networks described in step (3) are built, Octave convolution module therein, Space transforms power module, spectrum notice that power module, Pixel-level notice that power module and full articulamentum, parameter setting are as follows:
Octave convolution module, that is, the input module is made of, each conventional part sequentially connected four conventional parts It again include Octave convolution, Batch Normalization and Relu activation primitive is gone back between second and third conventional part There is a maximum pond layer;
The spatial attention mechanism module is turned by convolutional layer, Batch Normalization, Relu activation primitive, matrix It sets with the layer that is multiplied, softmax layers and data transposition with layer is added and constitutes;
The spectrum attention mechanism module, by matrix transposition and the layer that is multiplied, softmax layers and data transposition be added layer structure At;
The Pixel-level attention mechanism module, by convolutional layer, Batch Normalization and Relu activation primitive structure At;
The full articulamentum is made of the first full articulamentum and the second full articulamentum and softmax layers, i.e. output layer.
3. the high spectrum image point of the empty spectrum attention mechanism deep learning according to claim 1 based on Octave convolution Class method, which is characterized in that determination Octave convolutional neural networks loss function loss described in step (4)op, specifically include Following steps:
(4a) is by training image library { T1,T2,…,Tj,…,TMIt is input to the Octave convolution mould of Octave convolutional neural networks Block exports the last layer feature F of convolutional layer;
(4b) the last layer feature F is separately input to the space transforms power module of Octave convolutional neural networks and spectrum pays attention to Power module, output feature is respectively A and B, then output feature A and B are input to Pixel-level and pay attention to power module, output feature difference For C and D.
The feature C and D that (4c) will be obtained, are input to the full articulamentum of Octave convolutional neural networks, and output utilizes feature C and D Obtained output category result;Simultaneously then feature C and D are added pixel-by-pixel, are merged respectively multiplied by a coefficient Feature E afterwards, then feature E is input to the full articulamentum of Octave convolutional neural networks, output is obtained using fused feature E The output category result arrived obtains the loss function loss of Octave convolutional neural networksop:
Wherein, loss1For using fused feature E output category result and actual result after full articulamentum cross entropy, loss2It is characterized the cross entropy of C output category result and actual result after full articulamentum, loss3D is characterized by connecting entirely The cross entropy of output category result and actual result after layer is connect,For the L2 norm of convolutional neural networks weight vectors, η isHyper parameter.
4. the high spectrum image point of the empty spectrum attention mechanism deep learning according to claim 3 based on Octave convolution Class method, which is characterized in that the output category result obtained using fused feature E and actual result in step (4c) Cross entropy loss1, specific cross entropy formula is as follows:
Wherein, yjFor T in training image libraryjPrediction category probability, ojFor T in training image libraryjPractical category;loss1's Input is the prediction category probability that fused feature E is obtained after full articulamentum;
loss2、loss3Principle and formula and loss1It is identical, loss2Input be characterized C obtained after full articulamentum it is pre- Survey category probability, loss3Input be characterized the prediction category probability that D is obtained after full articulamentum.
5. the high spectrum image point of the empty spectrum attention mechanism deep learning according to claim 1 based on Octave convolution Class method, which is characterized in that step is normalized high spectrum image library in (1), is carried out by following formula:
Wherein VmaxFor the point maximum value of all pixels in high spectrum image library, VminFor the point of all pixels in high spectrum image library Minimum value, VnFor the pixel value at any point in high spectrum image library, { I '1,I′2,…,I′n,…,I′NFor after normalized High spectrum image library, I 'nFor n-th of sample of high spectrum image after normalized, n ∈ [0, N].
6. the high spectrum image point of the empty spectrum attention mechanism deep learning according to claim 1 based on Octave convolution Class method, which is characterized in that training is iterated to convolutional neural networks by gradient decline optimization in step (5), is realized It is as follows:
The initial learning rate of (5a) setting training is L, attenuation rate β, by training image library { T1,T2,…,Tj,…,TMIt is divided into G In the convolutional neural networks of secondary input building, the number of pictures inputted every time is Q, then:
Wherein M is the total number of training image library sample;
(5b) sets the corresponding learning rate l of input picture every time are as follows:
L=L* βG
(5c) carries out the update of G subparameter to convolutional neural networks by following formula, obtains updated weight vectors Wnew
Wherein, W is the weight vectors of convolutional neural networks parameter;
(5d) will train picture to input convolutional neural networks, loss function loss updated to weight vectors next timeopIt carries out It updates, so that loss function lossopValue constantly decline;
(5e) repeats (5d), until loss function lossopNo longer decline, and current exercise wheel number is less than the number of iterations of setting P then stops the training to the network, obtains trained Octave convolutional neural networks;Otherwise, when training round reaches setting The number of iterations P when, stop training to the network, obtain trained Octave convolutional neural networks.
CN201910797299.0A 2019-08-27 2019-08-27 Octave convolution-based spatial spectrum attention hyperspectral image classification method Active CN110516596B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910797299.0A CN110516596B (en) 2019-08-27 2019-08-27 Octave convolution-based spatial spectrum attention hyperspectral image classification method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910797299.0A CN110516596B (en) 2019-08-27 2019-08-27 Octave convolution-based spatial spectrum attention hyperspectral image classification method

Publications (2)

Publication Number Publication Date
CN110516596A true CN110516596A (en) 2019-11-29
CN110516596B CN110516596B (en) 2023-03-24

Family

ID=68627278

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910797299.0A Active CN110516596B (en) 2019-08-27 2019-08-27 Octave convolution-based spatial spectrum attention hyperspectral image classification method

Country Status (1)

Country Link
CN (1) CN110516596B (en)

Cited By (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111046962A (en) * 2019-12-16 2020-04-21 中国人民解放军战略支援部队信息工程大学 Sparse attention-based feature visualization method and system for convolutional neural network model
CN111160396A (en) * 2019-12-06 2020-05-15 华南理工大学 Hyperspectral image classification method based on graph convolution network with multi-graph structure
CN111259954A (en) * 2020-01-15 2020-06-09 北京工业大学 Hyperspectral traditional Chinese medicine tongue coating and tongue quality classification method based on D-Resnet
CN111274869A (en) * 2020-01-07 2020-06-12 中国地质大学(武汉) Method for classifying hyperspectral images based on parallel attention mechanism residual error network
CN111274988A (en) * 2020-02-10 2020-06-12 安徽大学 Multispectral-based vehicle weight identification method and device
CN111353531A (en) * 2020-02-25 2020-06-30 西安电子科技大学 Hyperspectral image classification method based on singular value decomposition and spatial spectral domain attention mechanism
CN111382676A (en) * 2020-02-25 2020-07-07 南京大学 Sand image classification method based on attention mechanism
CN111563520A (en) * 2020-01-16 2020-08-21 北京航空航天大学 Hyperspectral image classification method based on space-spectrum combined attention mechanism
CN111814607A (en) * 2020-06-24 2020-10-23 中国地质大学(武汉) Deep learning model suitable for small sample hyperspectral image classification
CN112052755A (en) * 2020-08-24 2020-12-08 西安电子科技大学 Semantic convolution hyperspectral image classification method based on multi-path attention mechanism
CN112052758A (en) * 2020-08-25 2020-12-08 西安电子科技大学 Hyperspectral image classification method based on attention mechanism and recurrent neural network
CN112232343A (en) * 2020-09-03 2021-01-15 国家粮食和物资储备局科学研究院 Neural network and method for recognizing grain mildewed grains
CN112464891A (en) * 2020-12-14 2021-03-09 湖南大学 Hyperspectral image classification method
CN112577747A (en) * 2020-12-07 2021-03-30 东南大学 Rolling bearing fault diagnosis method based on space pooling network
CN112801133A (en) * 2020-12-30 2021-05-14 核工业北京地质研究院 Spectrum identification and classification method based on keras model
CN112949592A (en) * 2021-03-31 2021-06-11 云南大学 Hyperspectral image classification method and device and electronic equipment
CN113095437A (en) * 2021-04-29 2021-07-09 中国电子科技集团公司第五十四研究所 Fire point detection method for Himapari-8 remote sensing data
CN113569660A (en) * 2021-07-06 2021-10-29 河海大学 Learning rate optimization algorithm discount coefficient method for hyperspectral image classification
CN114022716A (en) * 2021-11-29 2022-02-08 云南师范大学 Clothing attribute prediction method based on attention mechanism and graph relation network
CN114220002A (en) * 2021-11-26 2022-03-22 通辽市气象台(通辽市气候生态环境监测中心) Method and system for monitoring invasion of foreign plants based on convolutional neural network
CN114462596A (en) * 2022-02-10 2022-05-10 黑龙江省农业科学院 Disease and insect pest monitoring method and system for industrial hemp growth period
CN114492604A (en) * 2022-01-11 2022-05-13 电子科技大学 Radiation source individual identification method under small sample scene
CN114781534A (en) * 2022-04-29 2022-07-22 中国人民解放军战略支援部队信息工程大学 Image-level hyperspectral image classification method
CN114897814A (en) * 2022-05-07 2022-08-12 中国石油大学(华东) Hyperspectral image oil spilling detection method based on multi-level wavelet decomposition joint sealing network
CN114943251A (en) * 2022-05-20 2022-08-26 电子科技大学 Unmanned aerial vehicle target identification method based on fusion attention mechanism
CN114998725A (en) * 2022-05-17 2022-09-02 北京理工大学 Hyperspectral image classification method based on adaptive spatial spectrum attention kernel generation network
CN115457311A (en) * 2022-08-23 2022-12-09 宁波大学 Hyperspectral remote sensing image band selection method based on self-expression transfer learning
WO2023137783A1 (en) * 2022-01-19 2023-07-27 深圳大学 Spectral data processing method and apparatus, storage medium and electronic device
CN116824282A (en) * 2023-08-31 2023-09-29 中国石油大学(华东) Semi-supervised hyperspectral image classification method integrating multi-jump graph convolution and attention

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109784347A (en) * 2018-12-17 2019-05-21 西北工业大学 Image classification method based on multiple dimensioned dense convolutional neural networks and spectrum attention mechanism
CN109993220A (en) * 2019-03-23 2019-07-09 西安电子科技大学 Multi-source Remote Sensing Images Classification method based on two-way attention fused neural network
WO2019136946A1 (en) * 2018-01-15 2019-07-18 中山大学 Deep learning-based weakly supervised salient object detection method and system
CN110084159A (en) * 2019-04-15 2019-08-02 西安电子科技大学 Hyperspectral image classification method based on the multistage empty spectrum information CNN of joint

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019136946A1 (en) * 2018-01-15 2019-07-18 中山大学 Deep learning-based weakly supervised salient object detection method and system
CN109784347A (en) * 2018-12-17 2019-05-21 西北工业大学 Image classification method based on multiple dimensioned dense convolutional neural networks and spectrum attention mechanism
CN109993220A (en) * 2019-03-23 2019-07-09 西安电子科技大学 Multi-source Remote Sensing Images Classification method based on two-way attention fused neural network
CN110084159A (en) * 2019-04-15 2019-08-02 西安电子科技大学 Hyperspectral image classification method based on the multistage empty spectrum information CNN of joint

Cited By (49)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111160396A (en) * 2019-12-06 2020-05-15 华南理工大学 Hyperspectral image classification method based on graph convolution network with multi-graph structure
CN111160396B (en) * 2019-12-06 2023-04-25 华南理工大学 Hyperspectral image classification method of graph convolution network based on multi-graph structure
CN111046962A (en) * 2019-12-16 2020-04-21 中国人民解放军战略支援部队信息工程大学 Sparse attention-based feature visualization method and system for convolutional neural network model
CN111274869A (en) * 2020-01-07 2020-06-12 中国地质大学(武汉) Method for classifying hyperspectral images based on parallel attention mechanism residual error network
CN111259954A (en) * 2020-01-15 2020-06-09 北京工业大学 Hyperspectral traditional Chinese medicine tongue coating and tongue quality classification method based on D-Resnet
CN111259954B (en) * 2020-01-15 2024-05-31 北京工业大学 D-Resnet-based hyperspectral traditional Chinese medicine tongue fur and tongue quality classification method
CN111563520B (en) * 2020-01-16 2023-01-13 北京航空航天大学 Hyperspectral image classification method based on space-spectrum combined attention mechanism
CN111563520A (en) * 2020-01-16 2020-08-21 北京航空航天大学 Hyperspectral image classification method based on space-spectrum combined attention mechanism
CN111274988A (en) * 2020-02-10 2020-06-12 安徽大学 Multispectral-based vehicle weight identification method and device
CN111274988B (en) * 2020-02-10 2023-03-24 安徽大学 Multispectral-based vehicle weight identification method and device
CN111382676A (en) * 2020-02-25 2020-07-07 南京大学 Sand image classification method based on attention mechanism
CN111353531A (en) * 2020-02-25 2020-06-30 西安电子科技大学 Hyperspectral image classification method based on singular value decomposition and spatial spectral domain attention mechanism
CN111382676B (en) * 2020-02-25 2023-12-22 南京大学 Sand grain image classification method based on attention mechanism
CN111353531B (en) * 2020-02-25 2023-03-28 西安电子科技大学 Hyperspectral image classification method based on singular value decomposition and spatial spectral domain attention mechanism
CN111814607A (en) * 2020-06-24 2020-10-23 中国地质大学(武汉) Deep learning model suitable for small sample hyperspectral image classification
CN111814607B (en) * 2020-06-24 2023-12-29 中国地质大学(武汉) Deep learning model suitable for small sample hyperspectral image classification
CN112052755B (en) * 2020-08-24 2023-06-02 西安电子科技大学 Semantic convolution hyperspectral image classification method based on multipath attention mechanism
CN112052755A (en) * 2020-08-24 2020-12-08 西安电子科技大学 Semantic convolution hyperspectral image classification method based on multi-path attention mechanism
CN112052758A (en) * 2020-08-25 2020-12-08 西安电子科技大学 Hyperspectral image classification method based on attention mechanism and recurrent neural network
CN112052758B (en) * 2020-08-25 2023-05-23 西安电子科技大学 Hyperspectral image classification method based on attention mechanism and cyclic neural network
CN112232343B (en) * 2020-09-03 2023-11-21 国家粮食和物资储备局科学研究院 Grain mildew grain identification neural network and identification method
CN112232343A (en) * 2020-09-03 2021-01-15 国家粮食和物资储备局科学研究院 Neural network and method for recognizing grain mildewed grains
CN112577747A (en) * 2020-12-07 2021-03-30 东南大学 Rolling bearing fault diagnosis method based on space pooling network
CN112577747B (en) * 2020-12-07 2023-08-08 东南大学 Rolling bearing fault diagnosis method based on space pooling network
CN112464891B (en) * 2020-12-14 2023-06-16 湖南大学 Hyperspectral image classification method
CN112464891A (en) * 2020-12-14 2021-03-09 湖南大学 Hyperspectral image classification method
CN112801133A (en) * 2020-12-30 2021-05-14 核工业北京地质研究院 Spectrum identification and classification method based on keras model
CN112949592A (en) * 2021-03-31 2021-06-11 云南大学 Hyperspectral image classification method and device and electronic equipment
CN113095437B (en) * 2021-04-29 2022-03-01 中国电子科技集团公司第五十四研究所 Fire point detection method for Himapari-8 remote sensing data
CN113095437A (en) * 2021-04-29 2021-07-09 中国电子科技集团公司第五十四研究所 Fire point detection method for Himapari-8 remote sensing data
CN113569660B (en) * 2021-07-06 2024-03-26 河海大学 Learning rate optimization algorithm discount coefficient method for hyperspectral image classification
CN113569660A (en) * 2021-07-06 2021-10-29 河海大学 Learning rate optimization algorithm discount coefficient method for hyperspectral image classification
CN114220002A (en) * 2021-11-26 2022-03-22 通辽市气象台(通辽市气候生态环境监测中心) Method and system for monitoring invasion of foreign plants based on convolutional neural network
CN114022716B (en) * 2021-11-29 2024-04-12 云南师范大学 Clothes attribute prediction method based on attention mechanism and graph relation network
CN114022716A (en) * 2021-11-29 2022-02-08 云南师范大学 Clothing attribute prediction method based on attention mechanism and graph relation network
CN114492604A (en) * 2022-01-11 2022-05-13 电子科技大学 Radiation source individual identification method under small sample scene
WO2023137783A1 (en) * 2022-01-19 2023-07-27 深圳大学 Spectral data processing method and apparatus, storage medium and electronic device
CN114462596A (en) * 2022-02-10 2022-05-10 黑龙江省农业科学院 Disease and insect pest monitoring method and system for industrial hemp growth period
CN114781534A (en) * 2022-04-29 2022-07-22 中国人民解放军战略支援部队信息工程大学 Image-level hyperspectral image classification method
CN114897814A (en) * 2022-05-07 2022-08-12 中国石油大学(华东) Hyperspectral image oil spilling detection method based on multi-level wavelet decomposition joint sealing network
CN114897814B (en) * 2022-05-07 2023-07-25 中国石油大学(华东) Hyperspectral image oil spill detection method based on multistage wavelet decomposition close-coupled network
CN114998725A (en) * 2022-05-17 2022-09-02 北京理工大学 Hyperspectral image classification method based on adaptive spatial spectrum attention kernel generation network
CN114998725B (en) * 2022-05-17 2024-07-19 北京理工大学 Hyperspectral image classification method based on self-adaptive spatial spectrum attention kernel generation network
CN114943251B (en) * 2022-05-20 2023-05-02 电子科技大学 Unmanned aerial vehicle target recognition method based on fusion attention mechanism
CN114943251A (en) * 2022-05-20 2022-08-26 电子科技大学 Unmanned aerial vehicle target identification method based on fusion attention mechanism
CN115457311B (en) * 2022-08-23 2023-08-29 宁波大学 Hyperspectral remote sensing image band selection method based on self-expression transfer learning
CN115457311A (en) * 2022-08-23 2022-12-09 宁波大学 Hyperspectral remote sensing image band selection method based on self-expression transfer learning
CN116824282A (en) * 2023-08-31 2023-09-29 中国石油大学(华东) Semi-supervised hyperspectral image classification method integrating multi-jump graph convolution and attention
CN116824282B (en) * 2023-08-31 2023-11-10 中国石油大学(华东) Semi-supervised hyperspectral image classification method integrating multi-jump graph convolution and attention

Also Published As

Publication number Publication date
CN110516596B (en) 2023-03-24

Similar Documents

Publication Publication Date Title
CN110516596A (en) Empty spectrum attention hyperspectral image classification method based on Octave convolution
CN109993220B (en) Multi-source remote sensing image classification method based on double-path attention fusion neural network
CN112052755B (en) Semantic convolution hyperspectral image classification method based on multipath attention mechanism
CN110334765A (en) Remote Image Classification based on the multiple dimensioned deep learning of attention mechanism
CN111476713B (en) Intelligent weather image identification method and system based on multi-depth convolution neural network fusion
CN109034224B (en) Hyperspectral classification method based on double branch network
CN110321963A (en) Based on the hyperspectral image classification method for merging multiple dimensioned multidimensional sky spectrum signature
CN107832797B (en) Multispectral image classification method based on depth fusion residual error network
CN106203523A (en) The classification hyperspectral imagery of the semi-supervised algorithm fusion of decision tree is promoted based on gradient
CN110414577A (en) A kind of laser radar point cloud multiple target Objects recognition method based on deep learning
CN108460391B (en) Hyperspectral image unsupervised feature extraction method based on generation countermeasure network
CN106845418A (en) A kind of hyperspectral image classification method based on deep learning
CN105139028A (en) SAR image classification method based on hierarchical sparse filtering convolutional neural network
CN107563428A (en) Classification of Polarimetric SAR Image method based on generation confrontation network
CN110097145A (en) One kind being based on CNN and the pyramidal traffic contraband recognition methods of feature
CN106874956A (en) The construction method of image classification convolutional neural networks structure
CN108732172A (en) Chinese medicine performance rating method, equipment and medium
CN103839078B (en) A kind of hyperspectral image classification method based on Active Learning
CN108764138A (en) A kind of highlands cloud snow sorting technique cascading forest based on the more granularities of multidimensional
CN111222545B (en) Image classification method based on linear programming incremental learning
CN108416353A (en) Crop field spike of rice fast partition method based on the full convolutional neural networks of depth
CN109344891A (en) A kind of high-spectrum remote sensing data classification method based on deep neural network
CN110163101B (en) Method for rapidly distinguishing seeds of traditional Chinese medicinal materials and rapidly judging grades of seeds
CN106600595A (en) Human body characteristic dimension automatic measuring method based on artificial intelligence algorithm
CN112949738B (en) Multi-class unbalanced hyperspectral image classification method based on EECNN algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant