CN117131376A - Hyperspectral cross-domain robust anomaly detection method, system, equipment and medium for continuous learning based on visual transformation combined generation countermeasure network - Google Patents
Hyperspectral cross-domain robust anomaly detection method, system, equipment and medium for continuous learning based on visual transformation combined generation countermeasure network Download PDFInfo
- Publication number
- CN117131376A CN117131376A CN202311115700.0A CN202311115700A CN117131376A CN 117131376 A CN117131376 A CN 117131376A CN 202311115700 A CN202311115700 A CN 202311115700A CN 117131376 A CN117131376 A CN 117131376A
- Authority
- CN
- China
- Prior art keywords
- generator
- background
- task
- hyperspectral
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000009466 transformation Effects 0.000 title claims abstract description 81
- 238000001514 detection method Methods 0.000 title claims abstract description 78
- 230000000007 visual effect Effects 0.000 title claims abstract description 69
- 238000012549 training Methods 0.000 claims abstract description 96
- 238000012360 testing method Methods 0.000 claims abstract description 51
- 238000000034 method Methods 0.000 claims abstract description 45
- 238000001228 spectrum Methods 0.000 claims abstract description 36
- 238000012216 screening Methods 0.000 claims abstract description 12
- 239000013598 vector Substances 0.000 claims description 44
- 230000006870 function Effects 0.000 claims description 37
- 239000011159 matrix material Substances 0.000 claims description 35
- 230000002159 abnormal effect Effects 0.000 claims description 30
- 238000004590 computer program Methods 0.000 claims description 25
- 230000004927 fusion Effects 0.000 claims description 24
- 238000013507 mapping Methods 0.000 claims description 24
- 238000010276 construction Methods 0.000 claims description 21
- 230000003595 spectral effect Effects 0.000 claims description 21
- 238000004422 calculation algorithm Methods 0.000 claims description 10
- 238000010606 normalization Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 210000002569 neuron Anatomy 0.000 claims description 5
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 230000000873 masking effect Effects 0.000 claims description 3
- 238000004321 preservation Methods 0.000 claims description 3
- 230000008485 antagonism Effects 0.000 claims description 2
- 230000005856 abnormality Effects 0.000 abstract 1
- 230000002085 persistent effect Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 238000013461 design Methods 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000003491 array Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 230000000875 corresponding effect Effects 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000007786 learning performance Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 206010027175 memory impairment Diseases 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
A hyperspectral cross-domain robust anomaly detection method, system, device and medium for continuous learning based on visual transformation combination generation countermeasure network, wherein the method comprises the following steps: constructing a continuous learning task, carrying out spatial spectrum background screening pretreatment on data in each task to obtain a background set, replaying the background set of a first task as a training set, using the obtained replay set and the background set as training sets, alternately training ViT by using the training sets to generate an countermeasure network model, constructing F norm regularization loss items for a generator of a current task and a generator of a previous task, finally inputting data before the current task as a test set into a trained generator, adding 1 to the task number after a final result is obtained by testing, and returning to execute pretreatment; the system, the device and the medium are used for realizing a hyperspectral cross-domain robust anomaly detection method for continuously learning based on the combination of visual transformation and generation of an countermeasure network; the method has the characteristic of continuously detecting the abnormality in the cross-domain hyperspectral image.
Description
Technical Field
The invention belongs to the image information processing technology, and particularly relates to a hyperspectral cross-domain robust anomaly detection method, a system, equipment and a medium for continuous learning based on visual transformation combined generation countermeasure network.
Background
The hyperspectral image is a high-dimensional image acquired by a spectrum imager and comprises hundreds of spectrum channels, so that each pixel point is a continuous and high-dimensional spectrum curve, and a specific wave band can be selected or extracted according to the requirement to highlight the target characteristics. The hyperspectral imager detects the two-dimensional geometric space information and the one-dimensional spectrum information of the target at the same time, so that the hyperspectral data has a structure of an image cube, and the characteristics and advantages of map integration are reflected. Currently, hyperspectral images have been widely used in agriculture, military, astronomy, and other fields.
The hyperspectral remote sensing image has the most outstanding characteristic that the space image dimension information and the spectrum dimension information can be effectively fused. Therefore, according to the characteristics and application requirements of the hyperspectral remote sensing data, the targeted spatial spectrum characteristic extraction is carried out, and the effect of the hyperspectral remote sensing technology in practical application can be effectively improved. In addition, in geological exploration and disaster response applications, hyperspectral remote sensing images require large-area visits by related experts for calibration, so that marked samples available in practical applications are very limited. Therefore, the high-dimensional spectral characteristics, the small quantity of marked samples and the highly correlated spatial characteristics provide a series of challenges for hyperspectral remote sensing image classification and anomaly detection.
For hyperspectral anomaly detection tasks, due to the fact that priori knowledge of anomaly samples is very limited and the number of background samples and anomaly samples is very unbalanced, many anomaly detection methods in the prior art find different features or modes from hyperspectral data compared with normal samples. These anomalies may represent potential faults, abnormal events, or other unusual phenomena.
The patent application with publication number of [ CN114005044A ] provides a hyperspectral image anomaly detection method based on superpixel and progressive low-rank representation, which mainly aims at solving the problems of low and incomplete purity and poor detection result of a background dictionary constructed in the existing low-rank representation process. Comprising the following steps: dividing the hyperspectral image to be detected by adopting a super-pixel segmentation method based on orthogonal projection divergence to obtain a homogeneous region set; constructing a background dictionary by taking the centroid of the obtained homogeneous region as an atom, carrying out low-rank representation on the image, calculating a detection result by using the obtained abnormal part, screening the pure homogeneous region by taking the detection result as a reference, forming a new background dictionary by taking the centroid as the atom, and continuing the low-rank representation operation; repeating the above process until the detection result is unchanged, and obtaining the final detection result. However, the method performs low-rank representation on the image, so that the problems of high data redundancy and image information loss can be caused, and meanwhile, the input image is the whole hyperspectral image data, so that the calculated amount is large, and the calculation complexity is high; because of the characteristics of the hyperspectral image, only utilizing the spectrum information can cause a phenomenon of single knowledge, thereby affecting the detection accuracy; moreover, the model can only detect the abnormal point of one image at a time, and the phenomenon of time and resource waste can be caused in the reproduction of the real application.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to provide a hyperspectral cross-domain robust anomaly detection method, a system, equipment and a medium for continuously learning by combining visual transformation with generation of an countermeasure network, wherein data after feature enhancement is obtained through spatial spectrum background screening, a replay sample set of each task is obtained by using a clustering algorithm, an F norm regularization loss term is constructed, a model combining ViT with generation of the countermeasure network is established, reconstruction and capture of anomalies are carried out on a background of an image to be detected, forgetting of knowledge in a previous hyperspectral image can be prevented, the anomaly detection precision of the hyperspectral image to be detected currently and the hyperspectral image is ensured, and the characteristic of continuously detecting the hyperspectral image anomalies is realized.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
a hyperspectral cross-domain robust anomaly detection method for continuously learning based on view transformation combination generation countermeasure network comprises the following steps:
step 1, constructing t tasks meeting the continuous learning of hyperspectral images;
step 2, carrying out spatial spectrum background screening pretreatment on each data in the task constructed in the step 1 to obtain a background set;
Step 3, when t=1, taking the background set obtained in the step 2 as a training set, and executing the step 4;
otherwise, performing replay operation on t-1 task data by using a clustering algorithm to obtain a replay sample set, taking the union set of the background set obtained in the step 2 and the replay sample set obtained in the step 3 as a training set, and executing the step 4;
step 4, building ViT with a cascaded generator and discriminator structure to generate an countermeasure network model;
step 5, when t=1, alternately training the generator and the discriminator of the ViT generated countermeasure network model in the step 4 by using the training set in the step 3 to obtain a trained ViT generated countermeasure network model, and executing the step 6;
otherwise, the ViT in the step 4 is trained alternately by utilizing the training set in the step 3 to generate a generator and a discriminator of the countermeasure network model, and F norm regularization loss items are built for the generator of the current task and the generator of the last task at the same time, and the step 6 is executed;
and 6, taking data under t tasks as a test set, inputting the test set into the ViT trained in the step 5 to generate an countermeasure network model, testing to obtain a final result, and returning t=t+1 to execute the step 2.
The step 1 specifically comprises the following steps:
Constructing t tasks, and respectively marking data as Y 1 、Y 2 、…、Y m 、…Y t Wherein Y is m (m.epsilon.1, 2, …, t) represents the data set of the mth training scenario in the open scenario.
The step 2 specifically comprises the following steps:
step 2.1, setting a hyperspectral image (HSI) containing M×N pixels each having O channels, reducing the dimension of the channels to C dimension (C.ltoreq.O) by PCA, expressed as Wherein (1)>Represents Y m Spectral vector with middle coordinates (i, j), Y m Can be divided into an abnormal sample set A m And background set, i.e. Y m ＝[A m ，B m ]，A m ∪B m ＝Y m ，/>After capturing representative features of the background set, use +.>Obtaining a reconstructed background sample such that +.>Wherein G represents a generator in the visual transformation generation countermeasure network, B represents the background set, m represents the current task number, m e 1,2, t;
step 2.2, for Y in step 2.1 m (m.epsilon.1, 2,.,. T.) calculating the distance of adjacent pixels within the local area using cosine similarity, the cosine similarity calculation being expressed as:pixels with similarity greater than the set threshold are kept as background samples, and other samples are marked as abnormal samples, i.e. +.> Obtaining a coarse background mask matrix->Where τ is the threshold of background pixel similarity;
step 2.3, for Y in step 2.1 m (m.epsilon.1, 2,.,. T) performing block-based spatial-spectral feature fusion to enhance spatial information, computing an average vector of local regions of size w×w:where r is the index of the spectral vector in each w×w local region, the average vector is stitched with the spectral vector to enhance spatial information:wherein (1)>Vector representing original spectral information and local spatial information comprising (i, j) coordinates, +.>Representing splicing operation to obtain a space spectrum feature matrix of the HSI:
step 2.4 masking the matrix K with the coarse background of step 2.2 m M is 1,2, t is an index, and the space spectrum feature matrix F in the step 2.3 is obtained m M.epsilon.1, 2.,. T division differenceConstant sample set A m And background set, abnormal sample set A m Expressed as:the background set is expressed as: />Wherein a is i Representing an abnormal sample set A m The ith sample, n a Representing an abnormal sample set A m The number of samples in b i Represents the ith sample in the background set, n b Representing the number of samples in the background set, B represents the background set, m represents the current task number, m e 1,2,..t.
The step 3 specifically comprises the following steps:
step 3.1, data Y of the current task m Is of N m =m×n pixels, using K-means pairs Y m Clustering into N groups, each group comprising N i，m (i=1, 2,., n) pixels, the closest to the cluster center in each groupThe individual samples are taken as representative background samples in a playback sample pool, the playback samples being expressed as:
wherein P represents the slave current task data Y m Number of replay samples selected in KM i Group i representing K-means clusters, replay sample set e when there is a new task input m M.epsilon.1, 2.,. T is according to e m ←e m-1 ∪s(Y m ) (e when t=1) m-1 =Φ) is updated continuously, and a playback sample set is constructed as follows: e, e m ＝{s(Y 1 )，s(Y 2 )，...，s(Y m ) Preservation of a representative background sample in the historical task;
step 3.2, judging whether the currently delivered task is the first task, namely Y t ＝Y 1 Whether or not to establish; if so, the training set is the background set, i.e., X_train_AD 1 ＝B 1 The method comprises the steps of carrying out a first treatment on the surface of the If not, the sample set e is replayed in step 3.1 m And the union of the background set as training set, i.e. X_train_AD m ＝B m ∪e m ，m∈1，2，...，t。
The step 4 specifically comprises the following steps:
step 4.1, building a generator: the generator comprises an encoder 1, a decoder and an encoder 2 connected in series, wherein the encoder 1 and the encoder 2 have the same structure, and the encoder 1 and the encoder 2 are utilized to assist in training the decoder:
step 4.1.1, adopting three full connection layers and one depth separable convolution layer to form an encoder 1, and mapping the data of the training set in the step 3 into a potential space by using the encoder 1;
Step 4.1.2, adopting four fully connected layers to form a decoder, and mapping the data in the potential space in the step 4.1.1 to spectral vectors with the same size as the training set in the step 3 by using the decoder;
step 4.1.3, adopting three full-connection layers and a depth separable convolution layer to form an encoder 2, and mapping the spectral vector in step 4.1.2 by using the encoder 2 to obtain a potential space vector;
step 4.2, building a discriminator with a ViT network structure: the arbiter comprises a multi-scale convolution layer, a visual transformation layer and an output layer which are sequentially arranged according to the data logic processing sequence, and the arbiter is utilized to assist a training generator:
step 4.2.1, the multi-scale convolution layer comprises three parallel-connected one-dimensional convolution layers with different convolution kernel sizes and one-dimensional convolution layer, and the multi-scale convolution layer is utilized to obtain an intermediate vector fusing multi-level features;
step 4.2.2 the visual transformation layer comprises L 2 The multi-head self-attention, normalization layer and residual connection structure, the intermediate vector in the step 4.2.1 is mapped to the attention vector containing key characteristics by utilizing the visual transformation layer; the visual transformation layer can be expressed as:
where LN is a layer normalization function, expressed as:mu and sigma represent the mean and standard deviation, respectively, of the output of the neuron in batch, E is a small constant, i represents one of the neurons of the layer, L2_MHSA represents L 2 Multi-head self-attention;
step 4.2.3, adopting a Sigmoid layer to form an output layer, and mapping the attention vector in the step 4.2.2 by using the output layer to obtain a true and false judgment result;
and step 4.3, cascading the generator constructed in the step 4.1 with the discriminator constructed in the step 4.2 to obtain a ViT generated countermeasure network model.
The step 5 specifically comprises the following steps:
step 5.1, constructing a generator loss function: when t=1, the loss function of the generator is expressed as:
when t+.1, the loss function of the generator is expressed as:
wherein,wherein E represents a mathematical expectation, p (X_train_AD) m ) Representing the data distribution of the training set, E1 (X_train_AD m ) Represents the output of the encoder 1, G (E1 (X_train_AD) m ) Representing the reconstructed image output by the decoder, diff (G (E1 (x_train_ad) m ) (v)) means that the reconstructed image output from the decoder is subjected to a microtransparent enhancement to obtain an enhanced reconstructed sample set, E2 (G (E1 (x_train_ad) m ) And) represents the potential characteristics of the reconstructed image obtained by the encoder 2, D (Diff (G (E1 (x_train_ad) m ) A) represents the predicted outcome of the true probability of the reconstructed samples output by the arbiter, MSE represents the mean square error loss function.
Formula L Gt L of (3) f Representing F-norm regularization loss, forcing the reconstructed image of the t-1 th task to be closer to the t-th task, thereby making the parameter update amplitude smaller, for the t-th task, assuming that the replay sample is at the current generator G t The reconstructed samples generated are denoted as Z t The output of each data point is expressed asThe reconstructed samples generated by the generator at the time of the t-1 st task are denoted as Z t-1 Representing Z from the output t Calculating a covariance matrix of the reconstructed samples generated by the current generator, expressed as:
wherein b represents batch_size;
according to Z t-1 The position (i, j) where the middle pixel is zero is at Z t The pixel matrix in (a) is denoted as Z t [mask]The corresponding pixel number is numbered as num, and a covariance matrix related to the category is calculated:
updating the accumulated covariance matrix as:
P_W+＝P_X+P_C-I，
i represents an identity matrix of the same dimension as p_c.
The F-norm regularization loss of each task t is finally obtained as:
L f ＝λ*||P_W-I|| F ；
step 5.2, constructing a loss function of the discriminator: loss function L of said arbiter D Expressed as:
wherein,wherein E represents a mathematical expectation, p (X_train_AD) m ) Data distribution, E1 (X_train_AD, representing background sample training set m ) Represents the output of the encoder 1, G (E1 (X_train_AD) m ) Representing the reconstructed image output by the decoder, diff (G (E1 (x_train_ad) m ) ) means that the reconstructed image output by the decoder is subjected to a microtransparent enhancement to obtain an enhanced reconstructed sample set, D (Diff (x_train_ad) m ) A prediction result obtained by feeding the enhanced background sample set obtained by enhancing the micro data of the background sample set into the discriminator is shown;
Step 5.3, training the generator and the arbiter alternately by using the loss function of the generator in step 5.1 and the loss function of the arbiter in step 5.2:
step 5.3.1, training generator: the training set is input into a generator for nonlinear mapping to generate a reconstructed background sample G (E1 (x_train_ad) m ) And reconstructing the background sample G (E1 (x_train_ad) using the discriminator m ) Non-linear mapping), outputting the prediction result of true and false reconstructed background samples, and recording as true_G (E1 (X_train_AD) m ) A) is provided; calculating a loss value L of a prediction result of reconstructing the true or false of a background sample by using a mean square error loss function G The method comprises the steps of carrying out a first treatment on the surface of the By means of loss value L G Performing back propagation training on the generator;
step 5.3.2, training the discriminator: nonlinear mapping is carried out on the training set by utilizing a discriminator, and output is carried outThe true and false prediction result of the training set sample is recorded as true_x, and the mean square error loss is utilized to calculate the true and false prediction result loss value L of the training set sample D_true The method comprises the steps of carrying out a first treatment on the surface of the Nonlinear mapping is carried out on the reconstructed background sample by utilizing a discriminator, a prediction result of the reconstructed background sample is output and recorded as false_x, and a true and false prediction result loss value L of the training set is calculated by utilizing the mean square error loss D_false Will L D_true And L D_false Is added up as the total discriminator loss L D By L D Performing back propagation training on the discriminator; the model training is completed.
The step 6 specifically comprises the following steps:
step 6.1, constructing a test set Y_test 1 ＝Y 1 ，Y_test 2 ＝Y 2 ，...，Y_test m ＝Y m M=1, 2,.. all task data included before the current task; background set T_B for constructing test set 1 ＝B 1 ，T_B 2 ＝B 2 ，...，T_B m ＝B m ，m＝1，2，...，t；
Step 6.2, using the trained generator, for the background set T_B of the test set of step 6.1 m Reconstructing the background sample, calculating the reconstructed background sample and Y_test by using the mean square error loss m Obtaining the abnormal probability of each pixel point in the reconstructed image by the loss value of the data in the reconstructed image, wherein the abnormal probability expression of each pixel point is as follows: m (i, j) m ＝MSE(Y_test(i，j) m ，G m (E1(B(i，j) m ) (0.ltoreq.i.ltoreq.M, 0.ltoreq.j.ltoreq.N), wherein G m Representing an mth task trained generator, sequentially testing the previous m-1 tasks, Y_test (i, j) m Representing pixel points in the hyperspectral image to be measured of the mth task;
and 6.3, increasing the value of m by 1, and returning to the step 2.
A hyperspectral cross-domain robust anomaly detection system for continuous learning based on visual transformation in combination with generation of an countermeasure network, comprising:
the continuous learning task construction module: constructing t tasks meeting the continuous learning of hyperspectral images;
and a spatial spectrum feature fusion module: carrying out spatial spectrum feature fusion on task data constructed in the continuous learning task construction module;
Rough background mask matrix construction module: performing background screening pretreatment on the result of the spatial feature fusion module;
sample replay module: the data in the empty spectrum feature fusion module is utilized, a clustering algorithm is used for obtaining a replay sample set, and the replay sample set is obtained and combined with a background set in the rough background mask matrix construction module and is fed into models of the generator module and the discriminator module;
the generator module: constructing an encoder 1, a decoder and an encoder 2 which are connected in series to form a generator;
the discriminator module: constructing a discriminator comprising a multi-scale convolution layer, a visual transformation layer and an output layer, and using the discriminator to assist the training of the generator;
generator and arbiter loss module: the training generator module and the discriminator module are used for obtaining a trained ViT to generate an countermeasure network model;
and a testing module: an antagonism net model test is generated using the trained ViT in the generator and arbiter loss module.
A hyperspectral cross-domain robust anomaly detection device for continuous learning based on visual transformation in combination with generation of an countermeasure network, comprising:
a memory: the computer program is used for storing and realizing the hyperspectral cross-domain robust anomaly detection method for continuously learning based on the visual transformation combination generation countermeasure network;
A processor: the hyperspectral cross-domain robust anomaly detection method is used for realizing continuous learning based on the visual transformation combination generation countermeasure network when the computer program is executed.
A computer readable storage medium storing a computer program which when executed by a processor implements the steps of a method for continuously learning hyperspectral cross-domain robust anomaly detection based on visual transformation in combination with generation of a countermeasure network.
Compared with the prior art, the invention has the beneficial effects that:
1. according to the hyperspectral cross-domain robust anomaly detection method for continuously learning based on the visual transformation combined generation countermeasure network, the historical data is utilized to obtain the replay sample set, the memory degree of the model on old knowledge is enhanced, and the problem of catastrophic forgetting of the model is effectively solved.
2. According to the hyperspectral cross-domain robust anomaly detection method for continuously learning based on the visual transformation combination generation countermeasure network, an F norm regularization loss term is designed, a model is promoted to learn new knowledge and keep previous knowledge information, the model can reduce the correlation between the currently learned characteristics and the previous characteristics, so that the collision among samples is reduced, and the learning capacity of the model on the new knowledge is enhanced.
3. The invention relates to a hyperspectral cross-domain robust anomaly detection method for continuous learning based on a visual transformation combined generation countermeasure network, which designs a model for ViT combined generation countermeasure network, and can utilize L in ViT 2 The multi-head self-attention captures deeper sample characteristics, the reconstruction capability of a generator on a background sample is improved by alternately training the generator and a discriminator in a generating countermeasure network, the model can improve the learning performance of a cross-scene task of a hyperspectral image, and the actual application of a structure based on deep learning in hyperspectral interpretation is promoted.
To sum up, the present invention uses the historical data to obtain the replay sample set, designs an F-norm regularization loss term, and designs a ViT combined model for generating the countermeasure network, which can use L in ViT 2 The multi-head self-attention captures deeper sample characteristics, the generator and the discriminator in the generated countermeasure network are used for training alternately, the reconstruction capability of the generator on background samples is improved, the memory degree of the model on old knowledge and the learning capability of the model on new knowledge are enhanced, the learning performance of the hyperspectral image across scene tasks is improved, and the practical application of the structure based on deep learning in hyperspectral interpretation is promoted.
Drawings
FIG. 1 is a schematic flow chart of the method of the present invention.
FIG. 2 is a process diagram of hyperspectral persistent anomaly detection in the method of the present invention.
FIG. 3 is a model diagram of the hyperspectral persistent anomaly detection building vision transformation generated against a network in accordance with the method of the present invention.
Detailed Description
Referring to fig. 1, a hyperspectral cross-domain robust anomaly detection method for continuous learning based on view transformation combined generation countermeasure network includes the following steps:
step 1, constructing t tasks meeting the continuous learning of hyperspectral images;
step 2, carrying out spatial spectrum background screening pretreatment on each data in the task constructed in the step 1 to obtain a background set;
step 3, when t=1, taking the background set obtained in the step 2 as a training set, and executing the step 4;
otherwise, performing replay operation on t-1 task data by using a clustering algorithm to obtain a replay sample set, taking the union set of the background set obtained in the step 2 and the replay sample set obtained in the step 3 as a training set, and executing the step 4;
step 4, building ViT with a cascaded generator and discriminator structure to generate an countermeasure network model;
step 5, when t=1, alternately training the generator and the discriminator of the ViT generated countermeasure network model in the step 4 by using the training set in the step 3 to obtain a trained ViT generated countermeasure network model, namely, the generator, and executing the step 6;
Otherwise, the training set in the step 3 is utilized to alternately train ViT in the step 4 to generate a generator and a discriminator of the countermeasure network model, F norm regularization loss items are built for the generator of the current task and the generator of the last task, a trained generator is obtained, and the step 6 is executed;
and 6, taking the data under t tasks as a test set, inputting the test set into the trained generator in the step 5, testing to obtain a final result, enabling t=t+1, and returning to the step 2.
The step 1 specifically comprises the following steps:
construction of 5 ArbitrariesThe data are recorded as Y 1 、Y 2 、...、Y m 、...Y 5 Wherein Y is m (m.epsilon.1, 2,.,. 5) represents the dataset of the mth training scenario in the open scenario.
The step 2 specifically comprises the following steps:
step 2.1, setting a hyperspectral image (HSI) comprising M×N pixels each having O channels, reducing the channel dimension to C dimension (C.ltoreq.O) by PCA, M representing the number of lines of the hyperspectral image, N representing the number of columns of the hyperspectral image, expressed asWherein (1)>Represents Y m Spectral vector with middle coordinates (i, j), Y m Can be divided into an abnormal sample set A m And background set, i.e. Y m ＝[A m ，B m ]，A m ∪B m ＝Y m ，After capturing representative features of the background set, use +.>Obtaining a reconstructed background sample such that +. >Wherein G represents a generator in the visual transformation generation countermeasure network, B represents the background set, m represents the current task number, m e 1,2, 5;
for example, the data of the hyperspectral image abu-uban-4 is a data cube of size (100, 100, 205), where two 100 represent the number of rows and columns of abu-uban-4 data, respectively, and 205 represent the number of channels of abu-uban-4 data, the size of abu-uban-4 data is updated to (100, 100, 376) using PCA dimension reduction to 188 dimensions.
Step 2.2,For Y in step 2.1 m (m.epsilon.1, 2.,. 5) calculating the distance of neighboring pixels within the local area using cosine similarity, the cosine similarity calculation being expressed as:pixels with similarity greater than the set threshold are kept as background samples, and other samples are marked as abnormal samples, i.e. +.> Obtaining a coarse background mask matrix->Where τ is the threshold of background pixel similarity;
specifically, in this embodiment, τ is 0.99, if cosine similarity is less than 0.99, the pixel is considered to have a large difference from its neighboring pixels, the pixel is considered to be an abnormal sample, if cosine similarity is greater than or equal to 0.99, the pixel is considered to be a background sample, and if P samples are assumed to be in the background set, the pixel is considered to be a background sample Then, for example, for hyperspectral image abu-uban-4, its background set +.>
Step 2.3, for Y in step 2.1 m (m.epsilon.1, 2.,. 5) performing block-based spatial-spectral feature fusion to enhance spatial information, calculating an average vector of local regions of size w×w:where r is the index of the spectral vector in each w×w local region, the value of w takes 3, and the average vector is spliced with the spectral vector to enhance the spatial information:wherein->Vector representing original spectral information and local spatial information comprising (i, j) coordinates, +.>Representing splicing operation to obtain a space spectrum feature matrix of the HSI:
for example, for a partial region of the hyperspectral image abu-uban-4, 3×3, an average vector of size (1, 188) is calculated, and the average vector is spliced with the spectral vector (1, 188) to obtain f of size (1, 376) i，j Executing the operation on each pixel point in the hyperspectral image to obtain a spatial spectrum characteristic matrix F which fuses original spectrum information and local space information;
step 2.4 masking the matrix K with the coarse background of step 2.2 m M.epsilon.1, 2..5 is index, for the spatial spectrum feature matrix F of step 2.3 m M.epsilon.1, 2..5 dividing the abnormal sample set A m And background set, abnormal sample set A m Expressed as:the background set is expressed as: />Wherein a is i Representing an abnormal sample set A m The ith sample, n a Representing an abnormal sample set A m The number of samples in b i Represents the ith sample in the background set, n b Representing the number of samples in the background set, B represents the background set, m represents the current task number, m e 1,2,..5.
The step 3 specifically comprises the following steps:
step 3.1, seeFIG. 2 is a process diagram of hyperspectral persistent anomaly detection according to the present invention, as shown in FIG. 2, data Y of the current task m Is of N m =m×n pixels, using K-means pairs Y m Clustering into 3 groups, each group comprising N i，m (i=1, 2, 3) pixels, the nearest cluster center in each groupThe individual samples are taken as representative background samples in a playback sample pool, the playback samples being expressed as:
wherein P represents the slave current task data Y m Number of replay samples selected in KM i Group i representing K-means clusters, replay sample set e when there is a new task input m M.epsilon.1, 2..5. According to e m ←e m-1 ∪s(Y m ) (e when t=1) m-1 =Φ) is updated continuously, and a playback sample set is constructed as follows: e, e m ＝{s(Y 1 )，s(Y 2 )，...，s(Y m )，...，s(Y 5 ) Preservation of a representative background sample in the historical task;
step 3.2, judging whether the currently delivered task is the first task, namely Y t ＝Y 1 Whether or not to establish; if so, the training set is the background set, i.e., X_train_AD 1 ＝B 1 The method comprises the steps of carrying out a first treatment on the surface of the If not, the sample set e is replayed in step 3.1 m And the union of the background set as training set, i.e. X_train_AD m ＝B m ∪e m ，m∈1，2，...，5。
The step 4 specifically comprises the following steps:
referring to fig. 3, fig. 3 is a model diagram of a hyperspectral persistent anomaly detection building vision transformation generation countermeasure network according to the present invention, as shown in the accompanying drawings:
step 4.1, building a generator: the generator comprises an encoder 1, a decoder and an encoder 2 connected in series, wherein the encoder 1 and the encoder 2 have the same structure, and the encoder 1 and the encoder 2 are utilized to assist in training the decoder:
step 4.1.1, constructing encoder 1 by three full connection layers and one depth separable convolution layer, and transmitting the training set data X_train_AD in step 3 m In the input encoder 1, the data X_train_AD of the training set is input m Mapping to 100-dimensional potential vector, denoted E1 (X_train_AD m )；
Step 4.1.2, using four fully connected layers to form a decoder, using the decoder to map the data in the potential space in step 4.1.1 to spectral vectors of the same size as the training set in step 3, and using the encoder mapping, the decoder reconstructing the potential vectors into a 2C-dimensional reconstructed image, denoted G (E1 (x_train_ad) m ))；
Step 4.1.3, using three fully-connected layers and one depth-separable convolutional layer to form encoder 2, the spectral vectors in step 4.1.2 are mapped by encoder 2 to obtain potential spatial vectors, denoted as E2 (G (E1 (x_train_ad) 1 )))；
Step 4.2, building a discriminator with a ViT network structure: the discriminator includes a multi-scale convolution layer, a visual conversion layer, and an output layer sequentially arranged in the data logic processing order, and the discriminator is used to assist the training generator to train the reconstructed image G (E1 (x_train_ad) m ) After the microdata enhancement processing, the enhanced reconstructed image is recorded as Diff (G (E1 (x_train_ad) m ) The arbiter outputs a prediction result of the true probability of the enhanced reconstructed image as D (Diff (G (E1 (x_train_ad) m ))))；
Step 4.2.1, the multi-scale convolution layer comprises three parallel-connected one-dimensional convolution layers with different convolution kernel sizes and one-dimensional convolution layer, and an intermediate vector fused with multi-level features is obtained by utilizing the multi-scale convolution layer and is marked as d i ；
Step 4.2.2 the visual transformation layer comprises L 2 Multi-head self-attention, normalization layer and residual error connection structure, and intermediate vector d in step 4.2.1 is obtained by utilizing a visual transformation layer i Mapping to an attention vector containing key features; the visual transformation layer can be expressed as:
Where LN is a layer normalization function, expressed as:mu and sigma represent the mean and standard deviation, respectively, of the output of the neuron in batch, E is a small constant, i represents one of the neurons of the layer, L2_MHSA represents L 2 Multi-head self-attention, fetch 64;
step 4.2.3, adopting a Sigmoid layer to form an output layer, and mapping the attention vector in the step 4.2.2 by using the output layer to obtain a result, namely predicting true, marking real, predicting false and marking false;
and step 4.3, cascading the generator constructed in the step 4.1 with the discriminator constructed in the step 4.2 to obtain a ViT generated countermeasure network model.
The step 5 specifically comprises the following steps:
referring to fig. 3, fig. 3 is a model diagram of a hyperspectral persistent anomaly detection building vision transformation generation countermeasure network according to the present invention, as shown in the accompanying drawings:
step 5.1, constructing a generator loss function: when t=1, the loss function of the generator is expressed as:
when t+.1, the loss function of the generator is expressed as:
wherein,wherein E represents a mathematical expectation, p (X_train_AD) m ) Representing the data distribution of the training set, E1 (X_train_AD m ) Represents the output of the encoder 1, G (E1 (X_train_AD) m ) Representing the reconstructed image output by the decoder, diff (G (E1 (x_train_ad) m ) (v)) means that the reconstructed image output from the decoder is subjected to a microtransparent enhancement to obtain an enhanced reconstructed sample set, E2 (G (E1 (x_train_ad) m ) And) represents the potential characteristics of the reconstructed image obtained by the encoder 2, D (Diff (G (E1 (x_train_ad) m ) A) represents the predicted outcome of the true probability of the reconstructed samples output by the arbiter, MSE represents the mean square error loss function.
Formula L Gt L of (3) f Representing F-norm regularization loss, forcing the reconstructed image of the t-1 th task to be closer to the t-th task, thereby making the parameter update amplitude smaller, for the t-th task, assuming that the replay sample is at the current generator G t The reconstructed samples generated are denoted as Z t The output of each data point is expressed asThe reconstructed samples generated by the generator at the time of the t-1 st task are denoted as Z t-1 Representing Z from the output t Calculating a covariance matrix of the reconstructed samples generated by the current generator, expressed as:
wherein b represents batch_size, taken 64;
according to Z t-1 The position (i, j) where the middle pixel is zero is at Z t The pixel matrix in (a) is denoted as Z t [mask]The corresponding number of pixels is numbered as num, and class-related coordination is calculatedVariance matrix:
updating the accumulated covariance matrix as:
P_W+＝P_X+P_C-I，
i represents an identity matrix of the same dimension as p_c.
The F-norm regularization loss of each task t is finally obtained as:
L f ＝λ*||P_W-I|| F ，
The loss of the final generator is noted as L G ；
Step 5.2, constructing a loss function of the discriminator: loss function L of said arbiter D Expressed as:
wherein,wherein E represents a mathematical expectation, p (X_train_AD) m ) Data distribution, E1 (X_train_AD, representing background sample training set m ) Represents the output of the encoder 1, G (E1 (X_train_AD) m ) Representing the reconstructed image output by the decoder, diff (G (E1 (x_train_ad) m ) ) means that the reconstructed image output by the decoder is subjected to a microtransparent enhancement to obtain an enhanced reconstructed sample set, D (Diff (x_train_ad) m ) A prediction result obtained by feeding the enhanced background sample set obtained by enhancing the micro data of the background sample set into the discriminator is shown;
step 5.3, alternately training the generator and the discriminator by using the loss function of the generator in step 5.1 and the loss function of the discriminator in step 5.2, and finally obtaining a trained generator:
step 5.3.1, training generator: the training set is input into a generator for nonlinear mapping to generate a reconstructed background sample G (E1 (x_train_ad) m ) And reconstructing the background sample G (E1 (x_train_ad) using the discriminator m ) Non-linear mapping), outputting the prediction result of true and false reconstructed background samples, and recording as true_G (E1 (X_train_AD) m ) A) is provided; calculating a loss value L of a prediction result of reconstructing the true or false of a background sample by using a mean square error loss function G The method comprises the steps of carrying out a first treatment on the surface of the By means of loss value L G Performing back propagation training on the generator;
step 5.3.2, training the discriminator: nonlinear mapping is carried out on the training set by utilizing a discriminator, a prediction result of true and false of the training set sample is output and recorded as true_x, and a prediction result loss value L of true and false of the training set sample is calculated by utilizing the mean square error loss D_true The method comprises the steps of carrying out a first treatment on the surface of the Nonlinear mapping is carried out on the reconstructed background sample by utilizing a discriminator, a prediction result of the reconstructed background sample is output and recorded as false_x, and a true and false prediction result loss value L of the training set is calculated by utilizing the mean square error loss D_false Will L D_true And L D_false Is added up as the total discriminator loss L D By L D Performing back propagation training on the discriminator; model iterative training 5000 times, this model training is completed.
The step 6 specifically comprises the following steps:
step 6.1, constructing a test set Y_test 1 ＝Y 1 ，Y_test 2 ＝Y 2 ，...，m=1, 2,..5, including all task data prior to the current task; background set T_B for constructing test set 1 ＝B 1 ，T_B 2 ＝B 2 ，...，T_B m ＝B m ，m＝1，2，...，5；
Step 6.2, using the trained generator, for the background set T_B of the test set of step 6.1 m Reconstructing the background sample, calculating the reconstructed background sample and Y_test by using the mean square error loss m Obtaining the abnormal probability of each pixel point in the reconstructed image by the loss value of the data in the reconstructed image, wherein the abnormal probability expression of each pixel point is as follows: m (i, j) m ＝MSE(Y_test(i，j) m ，G m (E1(B(i，j) m ) (0.ltoreq.i.ltoreq.M, 0.ltoreq.j.ltoreq.N), wherein，G m Representing an mth task trained generator, sequentially testing the previous m-1 tasks, Y_test (i, j) m Representing pixel points in the hyperspectral image to be measured of the mth task;
specifically, for example, m=3, the dataset of the current task is abu-beacon-3, the dataset of the 2 nd task is abu-uban-5, the dataset of the 1 st task is abu-uban-4, and step 2 is performed on abu-beacon-3 to obtain a background set, denoted as t_b 3 Feeding the background set into a trained generator to obtain a reconstructed image G 3 (T_B 3 ) Obtaining abnormal probability M (i, j) of each pixel point by utilizing the mean square error loss 3 (0.ltoreq.i.ltoreq.100, 0.ltoreq.j.ltoreq.100), followed by step 2 on abu-uban-5 to obtain a background set, denoted as T_B 2 Feeding the background set into a trained generator to obtain a reconstructed image G 2 (T_B 2 ) Obtaining abnormal probability M (i, j) of each pixel point by utilizing the mean square error loss 2 (i is more than or equal to 0 and less than or equal to 100, j is more than or equal to 0 and less than or equal to 100), and finally, step 2 is performed on abu-uban-4 to obtain a background set, which is expressed as T_B 1 Feeding the background set into a trained generator to obtain a reconstructed image G 3 (T_B 1 ) Obtaining abnormal probability M (i, j) of each pixel point by utilizing the mean square error loss 1 ，(0≤i≤100，0≤j≤100)；
And 6.3, increasing the value of m by 1, and returning to the step 2.
A hyperspectral cross-domain robust anomaly detection system for continuous learning based on visual transformation in combination with generation of an countermeasure network, comprising:
the continuous learning task construction module: constructing t tasks meeting the requirement of continuous learning of hyperspectral images, and realizing a step 1 of a hyperspectral cross-domain robust anomaly detection method for continuous learning based on the combination of visual transformation and generation of an countermeasure network;
and a spatial spectrum feature fusion module: performing spatial spectrum feature fusion on task data constructed in a continuous learning task construction module, wherein the spatial spectrum feature fusion is used for realizing a hyperspectral cross-domain robust anomaly detection method for continuous learning based on view transformation combined generation of an countermeasure network;
rough background mask matrix construction module: a step 2 of performing background screening pretreatment on the result of the spatial feature fusion module, wherein the background screening pretreatment is used for realizing a hyperspectral cross-domain robust anomaly detection method for continuously learning based on the combination of visual transformation and generation of an countermeasure network;
sample replay module: obtaining a replay sample set by using a clustering algorithm by utilizing data in the spatial spectrum feature fusion module, and then obtaining a union set with a background set in the rough background mask matrix construction module, and feeding the union set into models of the generator module and the discriminator module to realize a hyperspectral cross-domain robust anomaly detection method for continuously learning based on view transformation combination generation of an countermeasure network;
The generator module: constructing an encoder 1, a decoder and an encoder 2 which are connected in series to form a generator, wherein the generator is used for realizing a hyperspectral cross-domain robust anomaly detection method for continuously learning based on the combination of visual transformation and generation of an countermeasure network;
the discriminator module: constructing a discriminator comprising a multi-scale convolution layer, a visual transformation layer and an output layer, wherein the discriminator is used for assisting the training of a generator and is used for realizing a hyperspectral cross-domain robust anomaly detection method for continuously learning based on the combination of visual transformation and generation of an countermeasure network;
generator and arbiter loss module: the training generator module and the discriminator module are used for obtaining a trained ViT generation countermeasure network model for realizing a hyperspectral cross-domain robust anomaly detection method for continuously learning based on the combination of visual transformation and generation countermeasure network;
and a testing module: and 6, generating an countermeasure network model test by using a trained ViT in the generator and the discriminant loss module, wherein the test is used for realizing a hyperspectral cross-domain robust anomaly detection method for continuously learning by generating a countermeasure network based on visual transformation combination.
A hyperspectral cross-domain robust anomaly detection device for continuous learning based on visual transformation in combination with generation of an countermeasure network, comprising:
A memory: the computer program is used for storing and realizing the hyperspectral cross-domain robust anomaly detection method for continuously learning based on the visual transformation combination generation countermeasure network;
a processor: the hyperspectral cross-domain robust anomaly detection method is used for realizing continuous learning based on the visual transformation combination generation countermeasure network when the computer program is executed.
The processor may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. The general processor may be a microprocessor or the processor may also be any conventional processor, etc., and the processor is a control center of the device for generating the hyperspectral cross-domain robust anomaly detection method for continuous learning of the countermeasure network based on the combination of the vision transformation, and various interfaces and lines are used to connect various parts of the whole device for generating the hyperspectral cross-domain robust anomaly detection method for continuous learning of the countermeasure network based on the combination of the vision transformation.
The method for detecting the hyperspectral cross-domain robust anomaly based on the visual transformation combination to generate the countermeasure network for continuous learning is realized when the processor executes the computer program.
Alternatively, the processor may implement functions of each module in the above system when executing the computer program, for example: the continuous learning task construction module: constructing t tasks meeting the continuous learning of hyperspectral images; and a spatial spectrum feature fusion module: carrying out spatial spectrum feature fusion on task data constructed in the continuous learning task construction module; rough background mask matrix construction module: performing background screening pretreatment on the result of the spatial feature fusion module; sample replay module: the data in the empty spectrum feature fusion module is utilized, a clustering algorithm is used for obtaining a replay sample set, and the replay sample set is obtained and combined with a background set in the rough background mask matrix construction module and is fed into models of the generator module and the discriminator module; the generator module: constructing an encoder 1, a decoder and an encoder 2 which are connected in series to form a generator; the discriminator module: constructing a discriminator comprising a multi-scale convolution layer, a visual transformation layer and an output layer, and using the discriminator to assist the training of the generator; generator and arbiter loss module: the training generator module and the discriminator module are used for obtaining a trained ViT to generate an countermeasure network model; and a testing module: generating an countermeasure network model test by using the trained ViT in the generator and the discriminant loss module; and outputting a result of the hyperspectral cross-domain robust anomaly detection method for continuously learning the countermeasure network based on the combination of the vision transformation.
The computer program may be divided into one or more modules/units, which are stored in the memory and executed by the processor to accomplish the present invention, for example. The one or more modules/units may be a series of computer program instruction segments capable of performing a preset function, the instruction segments being used to describe the execution of the computer program in the apparatus of the method for detecting hyperspectral cross-domain robust anomaly based on visual transformation in combination with generation of a countermeasure network for continuous learning. For example, the computer program may be divided into modules, the modules and specific functions being as follows: the continuous learning task construction module: constructing t tasks meeting the continuous learning of hyperspectral images; and a spatial spectrum feature fusion module: carrying out spatial spectrum feature fusion on task data constructed in the continuous learning task construction module; rough background mask matrix construction module: performing background screening pretreatment on the result of the spatial feature fusion module; sample replay module: the data in the empty spectrum feature fusion module is utilized, a clustering algorithm is used for obtaining a replay sample set, and the replay sample set is obtained and combined with a background set in the rough background mask matrix construction module and is fed into models of the generator module and the discriminator module; the generator module: constructing an encoder 1, a decoder and an encoder 2 which are connected in series to form a generator; the discriminator module: constructing a discriminator comprising a multi-scale convolution layer, a visual transformation layer and an output layer, and using the discriminator to assist the training of the generator; generator and arbiter loss module: the training generator module and the discriminator module are used for obtaining a trained ViT to generate an countermeasure network model; and a testing module: and generating an countermeasure network model test by using the trained ViT in the generator and the discriminant loss module, and outputting to obtain a result of the hyperspectral cross-domain robust anomaly detection method for continuously learning based on the combination of the vision transformation and the generation of the countermeasure network.
The equipment of the hyperspectral cross-domain robust anomaly detection method for continuously learning based on the visual transformation combination generation countermeasure network can be computing equipment such as a desktop computer, a notebook computer, a palm computer and a cloud server. The device for generating the hyperspectral cross-domain robust anomaly detection method for continuous learning against a network based on visual transformation can comprise, but is not limited to, a processor and a memory. It will be appreciated by those skilled in the art that the foregoing is an example of a device for generating a method for continuously learning a hyperspectral cross-domain robust anomaly detection based on a visual transformation combination, and is not limiting of a device for generating a method for continuously learning a hyperspectral cross-domain robust anomaly detection based on a visual transformation combination, and may include more components than the foregoing, or may combine certain components, or different components, for example, the device for generating a method for continuously learning a hyperspectral cross-domain robust anomaly detection based on a visual transformation combination may further include an input/output device, a network access device, a bus, and so on.
The memory may be used to store the computer program and/or the module, and the processor may implement various functions of the apparatus for generating a hyperspectral cross-domain robust anomaly detection method for continuous learning against a network based on visual transformation in combination by running or executing the computer program and/or the module stored in the memory and invoking data stored in the memory.
The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart Media Card (SMC), secure Digital (SD) Card, flash Card (Flash Card), at least one disk storage device, flash memory device, or other volatile solid-state storage device.
The invention also provides a computer readable storage medium storing a computer program which when executed by a processor implements the steps of the hyperspectral cross-domain robust anomaly detection method for continuous learning based on visual transformation combined generation countermeasure network.
The system-integrated module/unit for generating a hyperspectral cross-domain robust anomaly detection method for continuous learning against a network based on visual transformation can be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a stand-alone product.
The invention realizes all or part of the flow in the hyperspectral cross-domain robust anomaly detection method for continuously learning based on the visual transformation combination generation countermeasure network, and can also be completed by instructing related hardware through a computer program, wherein the computer program can be stored in a computer readable storage medium, and the computer program can realize the steps of the hyperspectral cross-domain robust anomaly detection method for continuously learning based on the visual transformation combination generation countermeasure network when being executed by a processor. The computer program comprises computer program code, and the computer program code can be in a source code form, an object code form, an executable file or a preset intermediate form and the like.
The computer readable storage medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth.
It should be noted that the computer readable storage medium may include content that is subject to appropriate increases and decreases as required by jurisdictions and by jurisdictions in which such computer readable storage medium does not include electrical carrier signals and telecommunications signals.
It should be noted that the embodiments of the present invention can be realized in hardware, software, or a combination of software and hardware. The hardware portion may be implemented using dedicated logic; the software portions may be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or special purpose design hardware.
Those of ordinary skill in the art will appreciate that the apparatus and methods described above may be implemented using computer executable instructions and/or embodied in processor control code, such as provided on a carrier medium such as a magnetic disk, CD or DVD-ROM, a programmable memory such as read only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The device of the present invention and its modules may be implemented by hardware circuitry, such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., as well as software executed by various types of processors, or by a combination of the above hardware circuitry and software, such as firmware.
The foregoing is merely illustrative of specific embodiments of the present invention, and the scope of the invention is not limited thereto, but any modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present invention will be apparent to those skilled in the art within the scope of the present invention.
Simulation experiment
The simulation experiment verifies and illustrates the effect of hyperspectral cross-domain robust anomaly detection of the visual transformation generation resisting continuous learning in the specific embodiment:
1. experimental conditions
In the simulation experiment, for the experiment of continuous anomaly detection of hyperspectral images, the data sets of 5 hyperspectral images used for experimental verification are BU (beacon-Urban), namely two scenes of the uban and the beacon, wherein the uban scene has two hyperspectral images, and the beacon scene has three hyperspectral images. In the experiment of hyperspectral image continuous anomaly detection, five tasks appear in sequence: (1) abu-uban-4, (2) abu-uban-5, (3) abu-beacon-3, (4) abu-beacon-2, and (5) abu-beacon-1.
Table 1 hyperspectral image persistent anomaly detection data
| Data set | Number of lines | Column number | Spectral band number |
| abu-urban-4 | 100 | 100 | 205 |
| abu-urban-5 | 100 | 100 | 205 |
| abu-beach-3 | 100 | 100 | 188 |
| abu-beach-2 | 100 | 100 | 193 |
| abu-beach-1 | 150 | 150 | 188 |
The hyperspectral cross-domain robust anomaly detection method for continuously learning based on the visual transformation combination generation countermeasure network is used for respectively comparing with a pure model (Trans GAN) method, a fine tuning (FT-Trans GAN) based method, a regularization (D-Trans GAN) based method, a replay (R-Trans GAN) based method and a joint learning (J-Trans GAN) based anomaly detection task in a cross-scene in a hyperspectral image continuous anomaly detection task. The detection is carried out under the same experimental conditions, and the AUC value and BWT value of the detection result are respectively analyzed.
AUC (Area Under Curve) is defined as the area under the ROC curve enclosed by the coordinate axes, and the AUC is in the range between 0.5 and 1 because the ROC curve is generally above the line y=x. The closer the AUC is to 1, the higher the detection method authenticity; when the value is equal to 0.5, the authenticity is the lowest, and the application value is not provided. BWT (Backward Transfer) is defined as a backward transition indicator to evaluate the impact of a new task on a historical task. The larger the BWT index, the better the proposed continuous learning training strategy performs in alleviating the catastrophic forgetfulness phenomenon. ACC represents the average of the sum of AUC values of the model trained in the current task t over previous tasks, with larger values representing more ability to continuously learn new knowledge.
The calculation formulas of the ACC value and the BWT value are respectively as follows:
wherein AUC t,i Representing the AUC value of the current model on task i after training t tasks.
2. Analysis of experimental results
Referring to table 2, table 2 shows ACC scores and BWT values for the persistent anomaly detection results for each detection setting.
TABLE 2 ACC score and BWT value for results of persistent anomaly detection and classification at each detection setting
Referring to Table 2,1-2 Tasks means that abu-uban-4 is performed first, then abu-uban-5 is performed, 1-3 Tasks means that abu-uban-4 is performed first, then abu-uban-5 is performed, finally abu-beacon-3 is performed, 1-4 Tasks means that abu-uban-4 is performed first, then abu-uban-5 is performed, then abu-beacon-3 is performed, finally abu-beacon-2 is performed, 1-5 Tasks means that abu-uban-4 is performed first, then abu-uban-5 is performed, then abu-beacon-3 is performed, then abu-beacon-2 is performed, and finally abu-beacon-1 is performed;
In the experimental setting, the Trans-GAN and the J-Trans-GAN have no memory capability and cannot alleviate catastrophic forgetting, the fine tuning method FT-Trans-GAN has unstable performance along with the increase of the task number, and the BWT value is negative, which means that when a new task is trained, the model is more biased to remember the knowledge of the new task and forget the knowledge of the old task, namely the model can suffer from serious catastrophic forgetting phenomenon. The regularization-based method D-TransGAN is used for losing the regularization term of the F norm regularization loss term and is used for shortening the distance between the parameters of the old model and the parameters of the new model, so that the forgetting phenomenon is prevented, but specific experiments show that the regularization loss term suffers from the forgetting phenomenon along with the increase of the number of tasks. The replay-based method R-fransgan uses a representative sample in the old task as an auxiliary sample for training the network, so as the task increases, the ACC value decreases less than that of other models, indicating that the models can memorize the knowledge learned before, but in the case of 1-3tasks and 1-4tasks, the models suffer from great forgetting, and in the case of 1-5tasks, the performance of the models is improved, indicating that the replay-based method can increase the plasticity of the network. The method CL-TransGAN using replay and F norm regularization loss terms is stable in performance in all comparison algorithms, and can be compared with a joint learning method along with the increase of the task number, and even the ACC evaluation indexes all exceed the joint learning. Experiments show that the self-adaptive sample replay strategy and the F norm regularization loss term provided by the invention have a strong effect of relieving catastrophic forgetting, and meanwhile, the vision transformation generation countermeasure network can play a role in solving the problem of hyperspectral anomaly detection.
Claims (10)
1. A hyperspectral cross-domain robust anomaly detection method for continuously learning based on view transformation combination generation countermeasure network is characterized by comprising the following steps:
step 1, constructing t tasks meeting the continuous learning of hyperspectral images;
step 2, carrying out spatial spectrum background screening pretreatment on each data in the task constructed in the step 1 to obtain a background set;
step 3, when t=1, taking the background set obtained in the step 2 as a training set, and executing the step 4;
otherwise, performing replay operation on t-1 task data by using a clustering algorithm to obtain a replay sample set, taking the union set of the background set obtained in the step 2 and the replay sample set obtained in the step 3 as a training set, and executing the step 4;
step 4, building ViT with a cascaded generator and discriminator structure to generate an countermeasure network model;
step 5, when t=1, alternately training the generator and the discriminator of the ViT generated countermeasure network model in the step 4 by using the training set in the step 3 to obtain a trained ViT generated countermeasure network model, and executing the step 6;
otherwise, the ViT in the step 4 is trained alternately by utilizing the training set in the step 3 to generate a generator and a discriminator of the countermeasure network model, and F norm regularization loss items are built for the generator of the current task and the generator of the last task at the same time, and the step 6 is executed;
And 6, taking data under t tasks as a test set, inputting the test set into the ViT trained in the step 5 to generate an countermeasure network model, testing to obtain a final result, and returning t=t+1 to execute the step 2.
2. The method for detecting hyperspectral cross-domain robust anomaly based on visual transformation combination generation and continuous learning of an countermeasure network according to claim 1, wherein the step 1 is specifically as follows:
constructing t tasks, and respectively marking data as Y 1 、Y 2 、…、Y m 、…Y t Wherein Y is m (m.epsilon.1, 2, …, t) represents the data set of the mth training scenario in the open scenario.
3. The method for detecting hyperspectral cross-domain robust anomaly based on visual transformation combination generation and continuous learning against a network according to claim 1, wherein the step 2 specifically comprises the following steps:
step 2.1, setting a hyperspectral image (HSI) containing M×N pixels each having O channels, reducing the dimension of the channels to C dimension (C.ltoreq.O) by PCA, expressed as Wherein (1)>Represents Y m Spectral vector with middle coordinates (i, j), Y m Can be divided into an abnormal sample set A m And background set, i.e. Y m ＝[A m ,B m ],A m ∪B m ＝Y m ,/>After capturing representative features of the background set, use +.>Obtaining a reconstructed background sample such that +. >Wherein G represents a generator in the visual transformation generation countermeasure network, B represents the background set, m represents the current task number, m epsilon 1,2, …, t;
step 2.2, for Y in step 2.1 m (m.epsilon.1, 2, …, t) calculating the distance of neighboring pixels within the local area using cosine similarity, the cosine similarity calculation being expressed as:pixels with similarity greater than the set threshold are kept as background samples, and other samples are marked as abnormal samples, i.e. +.> Obtaining a coarse background mask matrix->Where τ is the threshold of background pixel similarity;
step 2.3, for Y in step 2.1 m (m.epsilon.1, 2, …, t) block-based spatial signature fusion to enhance spatial information, computing an average vector of local regions of size w×w:wherein r isIndex of spectral vectors in each w x w local region, concatenating the average vector with the spectral vector to enhance spatial information:wherein (1)>Vector representing original spectral information and local spatial information comprising (i, j) coordinates, +.>Representing splicing operation to obtain a space spectrum feature matrix of the HSI:
step 2.4 masking the matrix K with the coarse background of step 2.2 m M is 1,2, …, t is index, for the space spectrum characteristic matrix F of step 2.3 m M epsilon 1,2, …, t divides the abnormal sample set A m And background set, abnormal sample set A m Expressed as:the background set is expressed as: />Wherein a is i Representing an abnormal sample set A m The ith sample, n a Representing an abnormal sample set A m The number of samples in b i Represents the ith sample in the background set, n b Representing the number of samples in the background set, B represents the background set, m represents the current task number, m ε 1,2, …, t.
4. The method for detecting hyperspectral cross-domain robust anomaly based on visual transformation combination generation and continuous learning against a network according to claim 1, wherein the step 3 specifically comprises the following steps:
step 3.1, data Y of the current task m Is of N m =m×n pixels, using K-means pairs Y m Clustering into N groups, each group comprising N i,m (i=1, 2, …, n) pixels, the closest to the cluster center in each groupThe individual samples are taken as representative background samples in a playback sample pool, the playback samples being expressed as:
wherein P represents the slave current task data Y m Number of replay samples selected in KM i Group i representing K-means clusters, replay sample set e when there is a new task input m M.epsilon.1, 2, …, t is according to e m ←e m-1 ∪s(Y m ) (e when t=1) m-1 =Φ) is updated continuously, and a playback sample set is constructed as follows: e, e m ＝{s(Y 1 ),s(Y 2 ),…,s(Y m ) Preservation of a representative background sample in the historical task;
step 3.2, judging whether the currently delivered task is the first task, namely Y t ＝Y 1 Whether or not to establish; if so, the training set is the background set, i.e., X_train_AD 1 ＝B 1 The method comprises the steps of carrying out a first treatment on the surface of the If not, the sample set e is replayed in step 3.1 m And the union of the background set as training set, i.e. X_train_AD m ＝B m ∪e m ,m∈1,2,…,t。
5. The method for detecting hyperspectral cross-domain robust anomaly based on visual transformation combination generation and continuous learning against a network according to claim 1, wherein the step 4 specifically comprises the following steps:
step 4.1, building a generator: the generator comprises an encoder 1, a decoder and an encoder 2 connected in series, wherein the encoder 1 and the encoder 2 have the same structure, and the encoder 1 and the encoder 2 are utilized to assist in training the decoder:
step 4.1.1, adopting three full connection layers and one depth separable convolution layer to form an encoder 1, and mapping the data of the training set in the step 3 into a potential space by using the encoder 1;
step 4.1.2, adopting four fully connected layers to form a decoder, and mapping the data in the potential space in the step 4.1.1 to spectral vectors with the same size as the training set in the step 3 by using the decoder;
Step 4.1.3, adopting three full-connection layers and a depth separable convolution layer to form an encoder 2, and mapping the spectral vector in step 4.1.2 by using the encoder 2 to obtain a potential space vector;
step 4.2, building a discriminator with a ViT network structure: the arbiter comprises a multi-scale convolution layer, a visual transformation layer and an output layer which are sequentially arranged according to the data logic processing sequence, and the arbiter is utilized to assist a training generator:
step 4.2.1, the multi-scale convolution layer comprises three parallel-connected one-dimensional convolution layers with different convolution kernel sizes and one-dimensional convolution layer, and the multi-scale convolution layer is utilized to obtain an intermediate vector fusing multi-level features;
step 4.2.2 the visual transformation layer comprises L 2 The multi-head self-attention, normalization layer and residual connection structure, the intermediate vector in the step 4.2.1 is mapped to the attention vector containing key characteristics by utilizing the visual transformation layer; the visual transformation layer can be expressed as:
where LN is a layer normalization function, expressed as:mu and sigma respectively represent the neuronThe average and standard deviation of the outputs in batch, e being a small constant, i representing one of the neurons of the layer, L2_MHSA representing L 2 Multi-head self-attention;
step 4.2.3, adopting a Sigmoid layer to form an output layer, and mapping the attention vector in the step 4.2.2 by using the output layer to obtain a true and false judgment result;
And step 4.3, cascading the generator constructed in the step 4.1 with the discriminator constructed in the step 4.2 to obtain a ViT generated countermeasure network model.
6. The method for detecting hyperspectral cross-domain robust anomaly based on visual transformation combination generation and continuous learning against a network according to claim 1, wherein the step 5 specifically comprises the following steps:
step 5.1, constructing a generator loss function: when t=1, the loss function of the generator is expressed as:
when t+.1, the loss function of the generator is expressed as:
wherein,wherein E represents a mathematical expectation, p (X_train_AD) m ) Representing the data distribution of the training set, E1 (X_train_AD m ) Represents the output of the encoder 1, G (E1 (X_train_AD) m ) Representing the reconstructed image output by the decoder, diff (G (E1 (x_train_ad) m ) (v)) means that the reconstructed image output from the decoder is subjected to a microtransparent enhancement to obtain an enhanced reconstructed sample set, E2 (G (E1 (x_train_ad) m ) And) represents the potential characteristics of the reconstructed image obtained by the encoder 2, D (Diff (G (E1 (x_train_ad) m ) ) table)The predicted result of the real probability of the reconstructed sample output by the discriminator is shown, and the MSE represents a mean square error loss function.
Formula L Gt L of (3) f Representing F-norm regularization loss, forcing the reconstructed image of the t-1 th task to be closer to the t-th task, thereby making the parameter update amplitude smaller, for the t-th task, assuming that the replay sample is at the current generator G t The reconstructed samples generated are denoted as Z t The output of each data point is expressed asThe reconstructed samples generated by the generator at the time of the t-1 st task are denoted as Z t-1 Representing Z from the output t Calculating a covariance matrix of the reconstructed samples generated by the current generator, expressed as:
wherein b represents batch_size;
according to Z t-1 The position (i, j) where the middle pixel is zero is at Z t The pixel matrix in (a) is denoted as Z t [mask]The corresponding pixel number is numbered as num, and a covariance matrix related to the category is calculated:
updating the accumulated covariance matrix as:
P_W+＝P_X+P_C–I，
i represents an identity matrix of the same dimension as p_c.
The F-norm regularization loss of each task t is finally obtained as:
L f ＝λ*||P_W-I|| F ；
step 5.2, constructing a loss function of the discriminator: loss function L of said arbiter D Expressed as:
wherein,wherein E represents a mathematical expectation, p (X_train_AD) m ) Data distribution, E1 (X_train_AD, representing background sample training set m ) Represents the output of the encoder 1, G (E1 (X_train_AD) m ) Representing the reconstructed image output by the decoder, diff (G (E1 (x_train_ad) m ) ) means that the reconstructed image output by the decoder is subjected to a microtransparent enhancement to obtain an enhanced reconstructed sample set, D (Diff (x_train_ad) m ) A prediction result obtained by feeding the enhanced background sample set obtained by enhancing the micro data of the background sample set into the discriminator is shown;
Step 5.3, training the generator and the arbiter alternately by using the loss function of the generator in step 5.1 and the loss function of the arbiter in step 5.2:
step 5.3.1, training generator: the training set is input into a generator for nonlinear mapping to generate a reconstructed background sample G (E1 (x_train_ad) m ) And reconstructing the background sample G (E1 (x_train_ad) using the discriminator m ) Non-linear mapping), outputting the prediction result of true and false reconstructed background samples, and recording as true_G (E1 (X_train_AD) m ) A) is provided; calculating a loss value L of a prediction result of reconstructing the true or false of a background sample by using a mean square error loss function G The method comprises the steps of carrying out a first treatment on the surface of the By means of loss value L G Performing back propagation training on the generator;
step 5.3.2, training the discriminator: nonlinear mapping is carried out on the training set by utilizing a discriminator, a prediction result of true and false of the training set sample is output and recorded as true_x, and a prediction result loss value L of true and false of the training set sample is calculated by utilizing the mean square error loss D_true The method comprises the steps of carrying out a first treatment on the surface of the Nonlinear mapping is carried out on the reconstructed background sample by utilizing a discriminator, a prediction result of the reconstructed background sample is output and recorded as false_x, and a true and false prediction result loss value L of the training set is calculated by utilizing the mean square error loss D_false Will L D_true And L D_false Is added up as the total discriminator loss L D By L D Performing back propagation training on the discriminator; the model training is completed.
7. The method for detecting hyperspectral cross-domain robust anomaly based on visual transformation combination generation and continuous learning against a network according to claim 1, wherein the step 6 specifically comprises the following steps:
step 6.1, constructing a test set Y_test 1 ＝Y 1 ,Y_test 2 ＝Y 2 ,…,Y_test m ＝Y m M=1, 2, …, t, including all task data prior to the current task; background set T_B for constructing test set 1 ＝B 1 ,T_B 2 ＝B 2 ,…,T_B m ＝B m ,m＝1,2,…,t；
Step 6.2, using the trained generator, for the background set T_B of the test set of step 6.1 m Reconstructing the background sample, calculating the reconstructed background sample and Y_test by using the mean square error loss m Obtaining the abnormal probability of each pixel point in the reconstructed image by the loss value of the data in the reconstructed image, wherein the abnormal probability expression of each pixel point is as follows: m (i, j) m ＝MSE(Y_test(i,j) m ,G m (E1(B(i,j) m ) (0.ltoreq.i.ltoreq.M, 0.ltoreq.j.ltoreq.N), wherein G m Representing an mth task trained generator, sequentially testing the previous m-1 tasks, Y_test (i, j) m Representing pixel points in the hyperspectral image to be measured of the mth task;
and 6.3, increasing the value of m by 1, and returning to the step 2.
8. A hyperspectral cross-domain robust anomaly detection system for continuous learning based on visual transformation in combination with generation of an countermeasure network, comprising:
The continuous learning task construction module: constructing t tasks meeting the continuous learning of hyperspectral images;
and a spatial spectrum feature fusion module: carrying out spatial spectrum feature fusion on task data constructed in the continuous learning task construction module;
rough background mask matrix construction module: performing background screening pretreatment on the result of the spatial feature fusion module;
sample replay module: the data in the empty spectrum feature fusion module is utilized, a clustering algorithm is used for obtaining a replay sample set, and the replay sample set is obtained and combined with a background set in the rough background mask matrix construction module and is fed into models of the generator module and the discriminator module;
the generator module: constructing an encoder 1, a decoder and an encoder 2 which are connected in series to form a generator;
the discriminator module: constructing a discriminator comprising a multi-scale convolution layer, a visual transformation layer and an output layer, and using the discriminator to assist the training of the generator;
generator and arbiter loss module: the training generator module and the discriminator module are used for obtaining a trained ViT to generate an countermeasure network model;
and a testing module: an antagonism net model test is generated using the trained ViT in the generator and arbiter loss module.
9. A hyperspectral cross-domain robust anomaly detection device for continuous learning based on visual transformation in combination with generation of an countermeasure network, comprising:
A memory: a computer program for storing a method for implementing a hyperspectral cross-domain robust anomaly detection method for continuous learning based on visual transformation in combination with generation of an countermeasure network as claimed in any one of claims 1 to 7;
a processor: a hyperspectral cross-domain robust anomaly detection method for implementing continuous learning against a network based on visual transformation combined generation as claimed in any one of claims 1 to 7 when said computer program is executed.
10. A computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program when executed by a processor implements the steps of a method for detecting hyperspectral cross-domain robust anomalies based on visual transformation in combination with generation of a countermeasure network for continuous learning.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311115700.0A CN117131376A (en) | 2023-08-31 | 2023-08-31 | Hyperspectral cross-domain robust anomaly detection method, system, equipment and medium for continuous learning based on visual transformation combined generation countermeasure network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311115700.0A CN117131376A (en) | 2023-08-31 | 2023-08-31 | Hyperspectral cross-domain robust anomaly detection method, system, equipment and medium for continuous learning based on visual transformation combined generation countermeasure network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117131376A true CN117131376A (en) | 2023-11-28 |
Family
ID=88852505
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311115700.0A Pending CN117131376A (en) | 2023-08-31 | 2023-08-31 | Hyperspectral cross-domain robust anomaly detection method, system, equipment and medium for continuous learning based on visual transformation combined generation countermeasure network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117131376A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117934978A (en) * | 2024-03-22 | 2024-04-26 | 安徽大学 | Hyperspectral and laser radar multilayer fusion classification method based on countermeasure learning |
| CN117949045A (en) * | 2024-03-13 | 2024-04-30 | 山东星科智能科技股份有限公司 | Digital monitoring method and system for new energy motor production line |
-
2023
- 2023-08-31 CN CN202311115700.0A patent/CN117131376A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117949045A (en) * | 2024-03-13 | 2024-04-30 | 山东星科智能科技股份有限公司 | Digital monitoring method and system for new energy motor production line |
| CN117949045B (en) * | 2024-03-13 | 2024-06-11 | 山东星科智能科技股份有限公司 | Digital monitoring method and system for new energy motor production line |
| CN117934978A (en) * | 2024-03-22 | 2024-04-26 | 安徽大学 | Hyperspectral and laser radar multilayer fusion classification method based on countermeasure learning |
| CN117934978B (en) * | 2024-03-22 | 2024-06-11 | 安徽大学 | Hyperspectral and laser radar multilayer fusion classification method based on countermeasure learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Gao et al. | Multiscale residual network with mixed depthwise convolution for hyperspectral image classification | |
| Sun et al. | Spectral–spatial feature tokenization transformer for hyperspectral image classification | |
| CN110378381B (en) | Object detection method, device and computer storage medium | |
| Sameen et al. | Classification of very high resolution aerial photos using spectral‐spatial convolutional neural networks | |
| CN111199214B (en) | Residual network multispectral image ground object classification method | |
| US9811718B2 (en) | Method and a system for face verification | |
| Barroso-Laguna et al. | Key. net: Keypoint detection by handcrafted and learned cnn filters revisited | |
| CN117131376A (en) | Hyperspectral cross-domain robust anomaly detection method, system, equipment and medium for continuous learning based on visual transformation combined generation countermeasure network | |
| CN111860398B (en) | Remote sensing image target detection method and system and terminal equipment | |
| CN111027576B (en) | Cooperative significance detection method based on cooperative significance generation type countermeasure network | |
| Yang et al. | A deep multiscale pyramid network enhanced with spatial–spectral residual attention for hyperspectral image change detection | |
| CN114549913B (en) | Semantic segmentation method and device, computer equipment and storage medium | |
| CN107545276A (en) | The various visual angles learning method of joint low-rank representation and sparse regression | |
| CN112580480B (en) | Hyperspectral remote sensing image classification method and device | |
| Wu et al. | A deep residual convolutional neural network for facial keypoint detection with missing labels | |
| CN111860124A (en) | Remote sensing image classification method based on space spectrum capsule generation countermeasure network | |
| CN112036381B (en) | Visual tracking method, video monitoring method and terminal equipment | |
| JP7502972B2 (en) | Pruning management device, pruning management system, and pruning management method | |
| CN114419406A (en) | Image change detection method, training method, device and computer equipment | |
| CN111179270A (en) | Image co-segmentation method and device based on attention mechanism | |
| Akhyar et al. | A beneficial dual transformation approach for deep learning networks used in steel surface defect detection | |
| CN112749576B (en) | Image recognition method and device, computing equipment and computer storage medium | |
| Zhou et al. | Identification of Rice Leaf Disease Using Improved ShuffleNet V2. | |
| Shi et al. | Hyperspectral image classification based on dual-branch spectral multiscale attention network | |
| CN116310899A (en) | YOLOv 5-based improved target detection method and device and training method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |