CN113947712A

CN113947712A - Hyperspectral anomaly detection method and system based on capsule differential countermeasure network

Info

Publication number: CN113947712A
Application number: CN202111028713.5A
Authority: CN
Inventors: 王佳宁; 郭思颖; 黄润虎; 刘一琛; 胡金雨; 李林昊; 杨攀泉; 焦李成
Original assignee: Xidian University
Current assignee: Xidian University
Priority date: 2021-09-02
Filing date: 2021-09-02
Publication date: 2022-01-18

Abstract

The invention relates to a capsule differential countermeasure network-based hyperspectral anomaly detection method and system, wherein the method comprises the following steps: acquiring a hyperspectral image to be detected; inputting the hyperspectral image to be tested into a generated countermeasure network which is trained in advance, and obtaining a reconstructed image corresponding to the hyperspectral image to be tested and the abnormal probability of each pixel point in the reconstructed image; the generation countermeasure network is obtained based on training of a background sample training set and comprises a generator and a discriminator which are connected in series, wherein the generator is used for generating reconstructed image data which are similar to the input image data distribution of the generator, and the discriminator is used for judging the truth of the input data; the generator and the discriminator are both in a one-dimensional capsule network structure. According to the method, the generated countermeasure network which is trained in advance is used for detecting the abnormality of the hyperspectral image to be detected, and compared with the traditional convolutional neural network, the generated countermeasure network can effectively relieve the problem of overfitting, and can be better suitable for a hyperspectral abnormality detection scene with unbalanced data.

Description

Hyperspectral anomaly detection method and system based on capsule differential countermeasure network

Technical Field

The invention belongs to the technical field of image information processing, and particularly relates to a capsule differential countermeasure network-based hyperspectral anomaly detection method and system.

Background

The hyperspectral image is a high-dimensional image which is acquired by a spectral imager and contains hundreds of spectral channels, so that each pixel point is a continuous spectral curve, and a specific waveband can be selected or extracted as required to highlight target features. The hyperspectral imager simultaneously detects two-dimensional geometric space information and one-dimensional spectral information of a target, so that hyperspectral data has an image cube structure, and the characteristics and advantages of map integration are embodied.

Due to the unique physical and chemical characteristics of different regions of the earth surface, different electromagnetic radiation can be emitted, reflected and absorbed, and the spectrum discrimination information in the hyperspectral image can be used for identifying different materials. With abundant spectrum, space and time information, the hyperspectral image is widely applied to various fields such as mineral exploration, environment monitoring, precision agriculture and national defense. The goal of hyperspectral anomaly detection is to identify objects that are significantly different from their surrounding background in space or spectrum without any prior knowledge of the background and target, which is one of the most active research fields in hyperspectrum.

In recent years, depth model algorithms are receiving more and more attention in hyperspectral image processing. The convolutional neural network shows excellent performance in a depth model, however, the modeling capability of the convolutional neural network-based scheme is quite limited, and if input data shows rotation, inclination or changes to any other direction, the convolutional neural network may show poor recognition performance, so that the hyperspectral anomaly detection effect is poor.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides a hyperspectral anomaly detection method and system based on a capsule differential countermeasure network. The technical problem to be solved by the invention is realized by the following technical scheme:

the invention provides a hyperspectral anomaly detection method based on a capsule differential countermeasure network, which comprises the following steps:

acquiring a hyperspectral image to be detected;

inputting the hyperspectral image to be tested to a generated countermeasure network which is trained in advance, and obtaining a reconstructed image corresponding to the hyperspectral image to be tested and the abnormal probability of each pixel point in the reconstructed image;

the generation countermeasure network is obtained based on training of a background sample training set, and comprises a generator and a discriminator which are connected in series, wherein the generator is used for generating reconstructed image data which is similar to the input image data distribution of the generator, and the discriminator is used for judging the truth of the input data of the discriminator; the generator and the discriminator are both in a one-dimensional capsule network structure.

In one embodiment of the invention, the generator comprises a cascade of an encoder for mapping input raw data into a potential space and a decoder for mapping data in the potential space to spectral vectors of the same size as the raw data; wherein the content of the first and second substances,

the encoder comprises a plurality of first fully-connected layers, a first primary capsule layer and a first dense capsule layer which are sequentially cascaded, and the first primary capsule layer is connected with the first dense capsule layer through a dynamic routing strategy;

the decoder comprises a number of cascaded second fully-connected layers.

In an embodiment of the present invention, the discriminator includes a plurality of one-dimensional convolution layers, a second primary capsule layer, and a second dense capsule layer, which are sequentially cascaded, and the second primary capsule layer and the second dense capsule layer are connected by a dynamic routing policy.

In one embodiment of the present invention, the training method for generating the countermeasure network includes:

acquiring a background sample training set;

inputting the background sample training set into the generated countermeasure network for training, and calculating a loss function;

and optimizing and updating the generated countermeasure network according to the loss function to obtain the trained generated countermeasure network.

In one embodiment of the present invention, obtaining a training set of background samples comprises:

acquiring a hyperspectral anomaly detection data set, wherein the hyperspectral anomaly detection data set comprises a plurality of hyperspectral images;

calculating the cosine similarity between each pixel and adjacent pixels in the hyperspectral image, and recording the coordinate set of the pixel points with the cosine similarity being more than or equal to 0.99 as a background sample coordinate set X_index；

Performing spatial-spectral mean fusion processing on each pixel in the hyperspectral image to obtain a fusion data set dataSpeSpa;

extracting coordinates in the background sample coordinate set X in the fused data set dataSpeSpa_indexThe pixel points in (1) form the background sample training set X_BKG。

In one embodiment of the present invention, inputting the training set of background samples into the generative confrontation network for training, and calculating a loss function, includes:

inputting the background sample training set into the generator for generating the countermeasure network for training, and calculating the loss function L of the generator_GThe generator outputs a reconstruction image set corresponding to the input of the generator, and meanwhile, the reconstruction image set is subjected to micro-data enhancement processing to obtain enhanced reconstructionAn image set;

performing micro data enhancement processing on the background sample training set to obtain an enhanced background sample set;

inputting the enhanced background sample set and the enhanced reconstructed image set into the discriminator for generating the countermeasure network for training, and calculating a loss function L of the discriminator_D。

In one embodiment of the invention, the loss function L of the generator_GComprises the following steps:

wherein the content of the first and second substances,

where E represents the mathematical expectation, X represents the background sample in the training set of background samples, p (X)_BKG) The method comprises the steps of representing data distribution of a background sample training set, E (x) representing output of an encoder, G (E (x)) representing a reconstructed image output by a decoder, Diff (G (E (x)) representing an enhanced reconstructed image after micro data enhancement processing is carried out on the reconstructed image, D (Diff (G (E (x))) representing a prediction result of input sample Diff (G (E (x))) true probability output by a discriminator, and MSE representing a mean square error loss function.

In one embodiment of the invention, the loss function L of the discriminator_DComprises the following steps: .

Wherein the content of the first and second substances,

where E represents the mathematical expectation, X represents the background sample in the training set of background samples, p (X)_BKG) Data distribution of a background sample training set is shown, E (x) shows the output of an encoder, G (E (x)) shows a reconstructed image output by a decoder, and Diff (G (E (x)) shows the reconstructed image after micro data enhancement processing is carried out on the reconstructed imageAnd enhancing the reconstructed image, wherein D (Diff (G (E (x))) represents a prediction result of the input sample Diff (G (E (x))) true probability output by the discriminator, Diff (x)) represents an enhanced background sample obtained by enhancing the background sample by the micro data, and D (Diff (x)) represents a prediction result of the input sample Diff (x)) true probability output by the discriminator.

In an embodiment of the present invention, the expression of the abnormal probability of each pixel point in the reconstructed image is as follows:

Pic_(i,j)＝MSE(X_(i,j),G(E(X_(i,j)))),(0≤i≤m,0≤j≤n)

wherein MSE represents a mean square error loss function, m represents the number of rows of the hyperspectral image data to be measured, n represents the number of columns of the hyperspectral image data to be measured, and X represents the number of rows of the hyperspectral image data to be measured_(i,j)Representing a pixel point in the hyperspectral image to be measured, E (X)_(i,j)) Representing the encoder with respect to input X_(i,j)Output of G (E (X)_(i,j)) Representing pixel points in a reconstructed image corresponding to the hyperspectral image to be measured.

The invention provides a hyperspectral anomaly detection system based on a capsule differential countermeasure network, which comprises:

the image input module is used for inputting a hyperspectral image to be detected;

the detection module is used for detecting the input hyperspectral image to be detected according to a generated countermeasure network which is stored in the detection module and trained in advance, and obtaining a reconstructed image corresponding to the hyperspectral image to be detected and the abnormal probability of each pixel point in the reconstructed image;

Compared with the prior art, the invention has the beneficial effects that:

1. according to the hyperspectral anomaly detection method based on the capsule differential countermeasure network, the anomaly of the hyperspectral image to be detected is detected by the generated countermeasure network which is trained in advance, the problem of overfitting can be effectively solved by the generated countermeasure network compared with the traditional convolutional neural network, in anomaly detection data, only a small part of anomaly points are usually generated, and the generated countermeasure network can be better suitable for scenes with data imbalance.

2. According to the hyperspectral anomaly detection method based on the capsule differential countermeasure network, the generated countermeasure network is of a capsule network structure, the attitude parameters of the sample can be stored, and the generalization performance of the model is improved.

3. According to the capsule differential countermeasure network-based hyperspectral anomaly detection method, in the training process of the countermeasure network, the spectral information and the spatial information of the hyperspectral image are processed, the spatial information is effectively merged and fully utilized, and the detection precision of the countermeasure network can be improved.

4. According to the capsule differential countermeasure network-based hyperspectral anomaly detection method, in the training process of the countermeasure network, the sample is enhanced and processed by using the micro data, so that the detection performance of the network can be further improved.

The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical means of the present invention more clearly understood, the present invention may be implemented in accordance with the content of the description, and in order to make the above and other objects, features, and advantages of the present invention more clearly understood, the following preferred embodiments are described in detail with reference to the accompanying drawings.

Drawings

FIG. 1 is a flow chart of a hyperspectral anomaly detection method based on a capsule differential countermeasure network according to an embodiment of the invention;

fig. 2 is a schematic structural diagram of a generation countermeasure network according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a training process for generating a countermeasure network according to an embodiment of the present invention;

fig. 4 is a diagram of an anomaly detection result in a urban scenario according to an embodiment of the present invention;

fig. 5 is a diagram of an anomaly detection result in a beacon scene according to an embodiment of the present invention;

fig. 6 is a diagram of an anomaly detection result in an airport scenario according to the embodiment of the present invention.

Detailed Description

In order to further explain the technical means and effects of the present invention adopted to achieve the predetermined invention objective, the following detailed description will be made of a hyperspectral anomaly detection method and system based on a capsule differential countermeasure network according to the present invention with reference to the accompanying drawings and the detailed implementation manners.

The foregoing and other technical matters, features and effects of the present invention will be apparent from the following detailed description of the embodiments, which is to be read in connection with the accompanying drawings. The technical means and effects of the present invention adopted to achieve the predetermined purpose can be more deeply and specifically understood through the description of the specific embodiments, however, the attached drawings are provided for reference and description only and are not used for limiting the technical scheme of the present invention.

Example one

Referring to fig. 1, fig. 1 is a flowchart of a hyperspectral anomaly detection method based on a capsule differential countermeasure network according to an embodiment of the present invention, where as shown in the figure, the hyperspectral anomaly detection method based on the capsule differential countermeasure network according to the embodiment includes:

step 1: acquiring a hyperspectral image to be detected;

step 2: and inputting the hyperspectral image to be tested into a generated countermeasure network which is trained in advance, and obtaining a reconstructed image corresponding to the hyperspectral image to be tested and the abnormal probability of each pixel point in the reconstructed image.

In the embodiment, the generation of the confrontation network is obtained based on training of the background sample training set.

Specifically, the generated countermeasure network is a generative model, and includes a generator and a discriminator in cascade, the generator is used for generating reconstructed image data with similar distribution to the input image data, and the discriminator is used for judging the truth of the input data, namely distinguishing whether the input data is real training data or the reconstructed data generated by the generator. The output of the discriminator is a true-false prediction value for its input sample, and the larger the value, the truer the sample is considered, and conversely, the smaller the value, the false the sample is considered.

In the generative confrontation network, the generator and the discriminator play a game against each other in a confrontational manner, and therefore, the generative confrontation network can generate realistic reconstruction data (pseudo data) that can be used for background pixel reconstruction in anomaly detection.

In this embodiment, the generator and the discriminator are both one-dimensional capsule network structures. The capsule network is a novel network structure, overcomes the defects of the convolutional neural network, and enhances the identification capability. The capsule network encodes the data relationships into vectors (rather than scalars), replacing scalar neurons with vector neurons, the norm of which represents the probability of the presence of a feature, and the direction represents the pose information (position, color, orientation, etc.) of the feature.

According to the hyperspectral anomaly detection method based on the capsule differential countermeasure network, the anomaly of the hyperspectral image to be detected is detected by the generation countermeasure network which is trained in advance, the problem of overfitting can be effectively solved by the generation countermeasure network compared with the traditional convolutional neural network, in anomaly detection data, only a small part of anomaly points are usually generated, and the generation countermeasure network can be better suitable for scenes with unbalanced data.

Further, referring to fig. 2, fig. 2 is a schematic structural diagram of a generative countermeasure network according to an embodiment of the present invention, as shown in the figure, in the generative countermeasure network according to the embodiment of the present invention, the generator includes a cascaded encoder and decoder, the encoder is configured to map input original data into a potential space, and the decoder is configured to map data in the potential space into a spectral vector having the same size as the original data.

In order to improve the modeling capability of the network, the encoder is designed to be a one-dimensional capsule network structure in the implementation, specifically, the encoder comprises a plurality of first full-connection layers, a first primary capsule layer and a first dense capsule layer which are sequentially cascaded, and the first primary capsule layer and the first dense capsule layer are connected through a dynamic routing strategy. The decoder comprises several cascaded second fully connected layers.

Similar to the encoder of the generator, since the capsule network has excellent feature modeling capability, the discriminator is also designed as a one-dimensional capsule network structure in this embodiment, and different from the encoder of the generator that includes fully connected layers, the discriminator includes a plurality of one-dimensional convolution layers, a second primary capsule layer and a second dense capsule layer which are sequentially cascaded, and the second primary capsule layer and the second dense capsule layer are connected through a dynamic routing strategy.

Further, a training method for generating the countermeasure network, which is constructed as shown in fig. 2, is specifically described, please refer to fig. 3, and fig. 3 is a schematic diagram of a training process for generating the countermeasure network according to an embodiment of the present invention. As shown in the figure, the training method for generating an antagonistic network of the present embodiment includes:

step a: acquiring a background sample training set;

the goal of anomaly detection is to identify objects that differ significantly in space or spectrum from their surrounding background without any prior knowledge of the background and object. In order to make the network learn the distribution of the background pixels, a preliminary background pixel screening is first required according to the spectral information.

Specifically, step a includes:

step a 1: acquiring a hyperspectral anomaly detection data set, wherein the hyperspectral anomaly detection data set comprises a plurality of hyperspectral images;

in this embodiment, the hyperspectral image data is a data cube with a size (m, n, channels), where m represents a row number of the hyperspectral image data, n represents a column number of the hyperspectral image data, and channels represent a spectrum channel number of the hyperspectral image data, and thus each pixel point is a spectrum curve with a length of channels.

For example, the data of the hyperspectral image abu-airport-1 is a data cube with the size (100,100,205), wherein two 100 respectively represent the row number and the column number of the abu-airport-1 data, and 205 represents the channel number of the abu-airport-1 data, so that each pixel point is a spectral curve with the length of 205.

Alternatively, the hyperspectral anomaly detection dataset may be obtained from an existing public dataset.

Step a 2: calculating the cosine similarity between each pixel and adjacent pixels in the hyperspectral image, and recording the coordinate set of the pixel points with the cosine similarity being more than or equal to 0.99 as a background sample coordinate set X_index；

It should be noted that, in the task of anomaly detection, the spectral curves of the anomalous pixel and the background pixel have a large difference, and the spectral curve difference between the adjacent background pixels is small. Therefore, the cosine similarity between each pixel and the adjacent pixels in the hyperspectral image is calculated, and the type of the pixel (namely the background pixel or the abnormal pixel) can be judged according to the cosine similarity.

Specifically, in this embodiment, if the cosine similarity is less than 0.99, the pixel is considered to have a larger difference from its neighboring pixels, and the pixel is considered to be an abnormal pixel, and if the cosine similarity is greater than or equal to 0.99, the coordinates of the pixel are added to the background sample coordinate set X_index。

Suppose X_indexIn which there are N elements, then

For example, for the hyperspectral image abu-airport-1, its background sample coordinate set

Step a 3: performing spatial-spectral mean fusion processing on each pixel in the hyperspectral image to obtain a fusion data set dataSpeSpa;

specifically, in this embodiment, each pixel in the hyperspectral image is taken as a center, a spatial block with a size of (a, a, channels) is defined, global average pooling is performed to obtain one-dimensional data with a size of (1,1, channels) and fused spatial information, and then the one-dimensional data is spliced with the original spectrum to obtain new data of (1,1,2 channels). This operation is performed on each pixel in the hyperspectral image, and a fused data set dataSpeSpa with the size of (m, n,2channels) fused with the spectral information of the pixel and the spatial information of the adjacent pixels is obtained.

For example, taking the data (100,100,205) of the hyperspectral image abu-airport-1 as an example, a spatial block with the size of (3, 205) is defined by taking each pixel as the center, global average pooling is performed to obtain one-dimensional data with the size of (1, 205) and fused spatial information, and the one-dimensional data is spliced with the original spectrum to obtain a spectral vector of (1, 410). This is done for each pixel in the hyperspectral image, resulting in a fused dataset dataSpeSpa, of size (100,100,410), fused with its own spectral information and spatial information for neighboring pixels,

in the training process of the generation countermeasure network, the spectral information and the spatial information of the hyperspectral image of the training sample are processed, the spatial spectral information is effectively merged and fully utilized, and the detection precision of the generation countermeasure network can be improved.

Step a 4: extraction of coordinates in the background sample coordinate set X in the fused dataset dataspectspa_indexThe pixel points in (1) form a background sample training set X_BKG。

In the present embodiment, it is preferred that,

step b: inputting a background sample training set into a generated countermeasure network for training, and calculating a loss function;

specifically, the method comprises the following steps:

step b 1: inputting the background sample training set into a generator for generating a countermeasure network for training, and calculating a loss function L of the generator_GThe generator outputs a reconstruction image set corresponding to the input of the generator, and meanwhile, data enhancement processing is carried out on the reconstruction image set to obtain an enhanced reconstruction image set;

specifically, first, a background sample training set X is set_BKGThe background sample x in (1) is input into the encoder of the generator, and the encoder based on the one-dimensional capsule network structure maps the background spectrum vector to a potential vector of 200 dimensions, denoted as e (x).

Next, the decoder reconstructs the potential vector into a reconstructed image G (e (x)) of 2channels dimension through the mapping of the encoder.

The reconstructed image G (e (x)) thus obtained is subjected to the enhancement processing of the micromanipulation data and then input to the discriminator, the enhanced reconstructed image obtained by subjecting the reconstructed image to the enhancement processing of the micromanipulation data is denoted as Diff (G (e (x)), and the discriminator outputs a prediction result of the true probability of the input sample Diff (G (e (x)) and denoted as D (Diff (G (e (x))).

Finally, the network loss is calculated, and in the present embodiment, the network loss here includes two parts: reconstruction loss and prediction loss.

Specifically, a loss value between the input background sample x and the reconstructed image G (e (x)) generated by the generator is calculated using a mean square error loss function.

The game training is carried out between the generator and the discriminator for generating the countermeasure network. In training the generator, the generator spoofs the discriminator to make the discriminator think the reconstructed image is a real sample, so the loss (-D (Diff (G (E (x)))) is minimized using a gradient descent algorithm.

And carrying out back propagation training on the generated countermeasure network by utilizing the two losses.

In this embodiment, the loss function L of the generator_GExpressed as:

wherein the content of the first and second substances,

Step b 2: carrying out micro data enhancement processing on the background sample training set to obtain an enhanced background sample set;

in the embodiment, in order to prevent the over-fitting of the discriminator for generating the countermeasure network and improve the performance of generating the countermeasure network, the training set is subjected to the enhancement of the micro data, and the micro data enhancement method is not only used for the training set of the background sample, but also used for the reconstructed image set generated by the generator, so as to avoid the damage to the dynamic balance between the generator and the discriminator.

Specifically, the method for enhancing the micro data is denoted as Diff, and then, a training set X is applied to the background sample_BKGThe background sample x in (1) is marked as Diff (x) after the enhancement of the micro data.

It is noted that the differentiable data enhancement method Diff includes data enhancement for colors (e.g., image brightness, saturation, and/or contrast variation) and data enhancement for shifts (e.g., movement in the X and/or Y directions).

In the training process of the generation countermeasure network, the detection performance of the network can be further improved by using the micro data to enhance the training sample.

Step b 3: inputting the enhanced background sample set and the enhanced reconstructed image set into a discriminator for generating a countermeasure network for training, and calculating a loss function L of the discriminator_D。

Specifically, the enhanced background sample diff (x) is input into a discriminator to be nonlinearly mapped, and a prediction result D (diff (x)) of the true probability of diff (x) is output.

Next, the enhanced reconstructed image Diff (G (e (x))) is input to a discriminator to be subjected to nonlinear mapping, and a prediction result D (Diff (G (e (x))) of true probability with respect to Diff (G (e (x))) is output.

Finally, the network loss is calculated, and in the present embodiment, the loss here includes two parts: enhancing the prediction loss of the background sample and enhancing the prediction loss of the reconstructed image.

When training the discriminators, in order to enhance the discriminative power of the discriminators, the loss (-D (Diff (x))) and D (Diff (G (e (x)))) are minimized by a gradient descent algorithm, and the generated countermeasure network is trained by using these two losses in a back propagation manner.

In bookIn an embodiment, the penalty function L of the discriminator_DExpressed as: .

Wherein the content of the first and second substances,

where E represents the mathematical expectation, X represents the background sample in the training set of background samples, p (X)_BKG) The method comprises the steps of representing data distribution of a training set of background samples, E (x) representing output of an encoder, G (E (x)) representing a reconstructed image output by a decoder, Diff (G (E (x)) representing an enhanced reconstructed image obtained after micro data enhancement processing is carried out on the reconstructed image, D (Diff (G (E (x))) representing a prediction result of the true probability of an input sample Diff (G (E (x))) output by a discriminator, Diff (x) representing an enhanced background sample obtained after micro data enhancement processing is carried out on the background sample, and D (Diff (x))) representing a prediction result of the true probability of the input sample Diff (x) output by the discriminator.

Step c: and optimally updating the generated countermeasure network according to the loss function to obtain the trained generated countermeasure network.

In this embodiment, 5000 times of iterative training is performed to complete training, so as to obtain a trained generative confrontation network.

And further, carrying out anomaly detection on the hyperspectral image to be detected by using the generated countermeasure network after training is finished.

Since the generation countermeasure network has learned the distribution information of the background pixels, similar pseudo data (reconstructed image) can be output for the input background pixels, but for the abnormal pixels, the output of the generation countermeasure network will be greatly different from the original data. Therefore, the reconstruction error of the abnormal pixel is higher than that of the background pixel, and the probability that the pixel belongs to the abnormal pixel can be obtained by calculating the reconstruction error of the confrontation network to the pixel.

In this embodiment, the generator in the countermeasure network can reconstruct the spectral vector of each pixel in the input hyperspectral image to be measured. Due to the trainingThe finished generation of the countermeasure network already grasps the data distribution of the background pixels, but does not know the data distribution of the abnormal pixels, so that all pixels of the hyperspectral image to be measured are input to the generation of the countermeasure network, and each pixel point x in the hyperspectral image to be measured is calculated_(i,j)(i is more than or equal to 0 and less than or equal to m, j is more than or equal to 0 and less than or equal to n) and a pixel point G (E (x) in the reconstructed image corresponding to the hyperspectral image to be detected_(i,j)) The loss value of the background pixel is small, and the loss value of the abnormal pixel is large. And taking the loss value as the probability that the pixel in the hyperspectral image to be detected is an abnormal pixel, completing abnormal detection, and outputting a detection result, wherein the detection result is a reconstructed image corresponding to the hyperspectral image to be detected and the abnormal probability of each pixel point in the reconstructed image.

Specifically, the expression of the abnormal probability of each pixel point in the reconstructed image is as follows:

Pic_(i,j)＝MSE(X_(i,j),G(E(X_(i,j)))),(0≤i≤m,0≤j≤n)

According to the hyperspectral anomaly detection method based on the capsule differential countermeasure network, the anomaly of the hyperspectral image to be detected is detected by utilizing the generation countermeasure network which is trained in advance, the generation countermeasure network utilizes space spectrum mean value fusion, the space spectrum information of the hyperspectral image is fully utilized and extracted, and a micro data enhancement method is adopted in the network training process.

Example two

The embodiment verifies and explains the effect of the first hyperspectral anomaly detection method based on the capsule differential countermeasure network through a simulation experiment.

1. Conditions of the experiment

In this embodiment, an ABU (Airport-beacon-Urban) dataset for experimental verification is divided into three scenes, namely, a Urban scene, a beacon scene and an Airport, wherein the Urban scene has 5 hyperspectral images, the beacon scene has 4 hyperspectral images, the Airport scene has 4 hyperspectral images, and specific hyperspectral image data are shown in table 1.

TABLE 1 Hyperspectral image data

Data set	Line number	Number of rows	Number of spectral bands
				abu-urban-1	100	100	204
abu-urban-2	100	100	207
				abu-urban-3	100	100	191
abu-urban-4	100	100	205
				abu-urban-5	100	100	205
abu-beach-1	150	150	188
				abu-beach-2	100	100	193
abu-beach-3	100	100	188
				abu-beach-4	150	150	102
abu-airport-1	100	100	205
				abu-airport-2	100	100	205
abu-airport-3	100	100	205
				abu-airport-4	100	100	191

Respectively utilizing a capsule differential countermeasure network-based hyperspectral anomaly detection method implemented in one step and four existing hyperspectral anomaly detection methods: Reed-Xiioli (RX), kernel-RX (KRX), Local RX (LRX) and Collaborative Representation-based Detector (CRD), ABU data sets were tested under the same experimental conditions, and AUC values of the test results were analyzed and evaluated, respectively.

The AUC (area Under curve) value is defined as the area enclosed by the coordinate axes Under the ROC curve, and since the ROC curve is generally located above the line y ═ x, the AUC ranges between 0.5 and 1. The closer the AUC is to 1, the higher the authenticity of the detection method is; and when the value is equal to 0.5, the authenticity is lowest, and the application value is not high.

2. Analysis of Experimental results

Please refer to table 2, where table 2 shows AUC scores of the detection results of the hyperspectral images by the detection methods.

TABLE 2 AUC scores of the results of the detection of the hyperspectral images by the detection methods

Data set	RX	KRX	LRX	CRD	Method for producing a composite material
						abu-urban-1	0.9906	0.9906	0.98181	0.98013	0.9917
abu-urban-2	0.9945	0.9945	0.98016	0.94706	0.9993
						abu-urban-3	0.9524	0.9524	0.93676	0.94726	0.9811
abu-urban-4	0.9896	0.9896	0.96898	0.95938	0.9967
						abu-urban-5	0.9694	0.9694	0.96939	0.93383	0.9754
abu-beach-1	0.9731	0.9731	0.98256	0.97268	0.9912
						abu-beach-2	0.9103	0.9103	0.91683	0.89812	0.9015
abu-beach-3	0.9997	0.9997	0.99608	0.98973	0.9999
						abu-beach-4	0.9893	0.9893	0.9787	0.97113	0.9988
abu-airport-1	0.8213	0.8213	0.94432	0.88994	0.9331
						abu-airport-2	0.8413	0.8413	0.9754	0.90564	0.9292
abu-airport-3	0.9283	0.9283	0.93761	0.83916	0.9673
						abu-airport-4	0.9508	0.9508	0.98959	0.90017	0.9923

Referring to fig. 4 to 6 in combination, fig. 4 is an abnormal detection result diagram in an urban scene provided by an embodiment of the present invention, where (a) diagram- (e) diagram sequentially shows an abnormal detection result diagram of a hyperspectral image abu-urban-1-hyperspectral image abu-urban-5, and from left to right, the abnormal detection result diagram is a pseudo-color diagram, a groudtruth diagram, a detection result diagram of a method of the present invention, an RX detection result diagram, a KRX detection result diagram, an LRX detection result diagram, and a CRD method detection result diagram of a hyperspectral image, respectively. Fig. 5 is an anomaly detection result diagram in a beacon scene provided in an embodiment of the present invention, where the diagram (a) and the diagram (d) are an anomaly detection result diagram of a hyperspectral image abu-beacon-1-hyperspectral image abu-beacon-4 in sequence, and a pseudo-color diagram, a groudtruth diagram, a detection result diagram of a method of the present invention, an RX detection result diagram, a KRX detection result diagram, an LRX detection result diagram, and a CRD method detection result diagram of a hyperspectral image are respectively from left to right. Fig. 6 is an abnormality detection result diagram in an airport scene provided in an embodiment of the present invention, where the diagram (a) and the diagram (d) are sequentially an abnormality detection result diagram of the hyperspectral image abu-airport-1-the hyperspectral image abu-airport-4, and a pseudo-color diagram, a groudtruth diagram, a detection result diagram of the method of the present invention, an RX detection result diagram, a KRX detection result diagram, an LRX detection result diagram, and a CRD method detection result diagram of the hyperspectral image are respectively from left to right.

As can be seen from the table 2 and the figures 4 to 6, on most data sets, the capsule differential countermeasure network-based hyperspectral anomaly detection method is superior to other traditional detection methods, and has stable and excellent performance. Because the spectrum of some abnormal pixels in the detection data is closer to the spectrum of background pixels, the algorithm may determine the abnormal pixels as the background or determine the background pixels as abnormal, in which case, the RX, LRX and CRD algorithms have a large number of abnormal missing detection situations, and the abnormal detection rate is low. Although the KRX algorithm detects most abnormal pixels, many background pixels are detected as abnormal at the same time, namely KRX has higher false alarm rate. The method of the invention achieves the balance between the detection rate and the false alarm rate, and reduces the false detection rate while detecting abnormal pixels as comprehensively as possible.

According to the hyperspectral anomaly detection method based on the capsule differential countermeasure network, the capsule network is used for replacing a widely used convolutional neural network, the generation of the countermeasure network is combined, the discriminative information in hyperspectral data is effectively utilized and fused, meanwhile, the micro data enhancement method is utilized, the overfitting of a discriminator is relieved, and the performance of the generation of the countermeasure network is improved.

EXAMPLE III

Based on the same invention concept, the embodiment of the invention provides a capsule differential countermeasure network-based hyperspectral anomaly detection system, which comprises:

the detection module is used for detecting the input hyperspectral image to be detected according to a generated countermeasure network which is stored in the detection module and is trained in advance, and obtaining a reconstructed image corresponding to the hyperspectral image to be detected and the abnormal probability of each pixel point in the reconstructed image;

the generation countermeasure network is obtained based on training of a background sample training set and comprises a generator and a discriminator which are connected in series, wherein the generator is used for generating reconstructed image data which are similar to the input image data distribution of the generator, and the discriminator is used for judging the truth of the input data; the generator and the discriminator are both in a one-dimensional capsule network structure.

The hyperspectral anomaly detection system based on the capsule differential countermeasure network provided by the embodiment of the invention can execute the method embodiment, the realization principle and the technical effect are similar, and the detailed description is omitted.

The invention also provides a terminal device comprising a processor and a memory for storing a computer program comprising program instructions, the processor being configured to execute the program instructions stored by the computer storage medium. The Processor may be a Central Processing Unit (CPU), or may be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable gate array (FPGA) or other Programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc., which is a computing core and a control core of the terminal, and is adapted to implement one or more instructions, and is specifically adapted to load and execute one or more instructions to implement a corresponding method flow or a corresponding function; the processor of the embodiment of the invention can realize the method steps of any capsule differential countermeasure network-based hyperspectral anomaly detection method, or realize the function realized by any generation countermeasure network.

The present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a Memory device in a terminal device and is used for storing programs and data. It is understood that the computer readable storage medium herein may include a built-in storage medium in the terminal device, and may also include an extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space storing an operating system of the terminal. Also, one or more instructions, which may be one or more computer programs (including program code), are stored in the memory space and are adapted to be loaded and executed by the processor. It should be noted that the computer-readable storage medium may be a high-speed RAM memory, or may be a non-volatile memory (non-volatile memory), such as at least one disk memory.

The processor may load and execute one or more instructions stored in the computer-readable storage medium to implement the method steps of any of the above methods for detecting hyperspectral abnormality based on capsule differential countermeasure networks, or to implement the functions implemented by any of the above methods for generating a countermeasure network.

For the electronic device/storage medium/computer program product embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, refer to the partial description of the method embodiment.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that an article or device that comprises a list of elements does not include only those elements but may include other elements not expressly listed. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of additional like elements in the article or device comprising the element. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.

Claims

1. A hyperspectral anomaly detection method based on a capsule differential countermeasure network is characterized by comprising the following steps:

acquiring a hyperspectral image to be detected;

2. The capsule differential countermeasure network-based hyperspectral anomaly detection method according to claim 1, wherein the generator comprises a cascade of an encoder and a decoder, the encoder is used for mapping input raw data into a potential space, and the decoder is used for mapping data in the potential space to a spectral vector with the same size as the raw data; wherein the content of the first and second substances,

the decoder comprises a number of cascaded second fully-connected layers.

3. The hyperspectral anomaly detection method based on the capsule differential countermeasure network according to claim 1, wherein the discriminator comprises a plurality of one-dimensional convolution layers, a second primary capsule layer and a second dense capsule layer which are sequentially cascaded, and the second primary capsule layer is connected with the second dense capsule layer through a dynamic routing strategy.

4. The capsule differential countermeasure network-based hyperspectral anomaly detection method according to claim 1, wherein the training method for generating the countermeasure network comprises:

acquiring a background sample training set;

5. The capsule differential countermeasure network-based hyperspectral anomaly detection method according to claim 4, wherein obtaining a background sample training set comprises:

6. The capsule differential countermeasure network-based hyperspectral anomaly detection method according to claim 4, wherein the inputting the background sample training set into the generation countermeasure network for training and calculating a loss function comprises:

inputting the background sample training set into the generator for generating the countermeasure network for training, and calculating the loss function L of the generator_GThe generator outputs a reconstructed image set corresponding to the input of the generator, and meanwhile, the reconstructed image set is subjected to micro-data enhancement processing to obtain an enhanced reconstructed image set;

7. The capsule differential countermeasure network-based hyperspectral anomaly detection method according to claim 6, wherein the loss function L of the generator_GComprises the following steps:

wherein the content of the first and second substances,

where E represents the mathematical expectation, X represents the background sample in the training set of background samples, p (X)_BKG) Representing background sample trainingThe data distribution of the training set, E (x) represents the output of the encoder, G (E (x)) represents the reconstructed image output by the decoder, Diff (G (E (x)) represents the enhanced reconstructed image after the micro-data enhancement processing is carried out on the reconstructed image, D (Diff (G (E (x))) represents the prediction result of the true probability of input samples Diff (G (E (x))) output by the discriminator, and MSE represents the mean square error loss function.

8. The hyperspectral anomaly detection method based on capsule differential countermeasure network according to claim 6, characterized in that the loss function L of the discriminator_DComprises the following steps: .

Wherein the content of the first and second substances,

9. The capsule differential countermeasure network-based hyperspectral anomaly detection method according to claim 1, wherein the expression of the anomaly probability of each pixel point in the reconstructed image is as follows:

Pic_(i，j)＝MSE(X_(i，j)，G(E(X_(i，j))))，(0≤i≤m，0≤j≤n)

wherein MSE represents a mean square error loss function, m represents the number of hyperspectral images to be measuredThe row number of the data, n represents the column number of the hyperspectral image data to be measured, X_(i，j)Representing a pixel point in the hyperspectral image to be measured, E (X)_(i，j)) Representing the encoder with respect to input X_(i，j)Output of G (E (x)_(i，j)) Representing pixel points in a reconstructed image corresponding to the hyperspectral image to be measured.

10. A hyperspectral anomaly detection system based on a capsule differential countermeasure network is characterized by comprising: