CN110738249A - aurora image clustering method based on deep neural network - Google Patents

Abstract

The invention discloses an aurora image clustering method based on a deep neural network, which mainly solves the problem that the existing automatic classification of aurora images is basically supervised classification.

Description

aurora image clustering method based on deep neural network

Technical Field

The invention belongs to the technical field of image processing, relates to a deep learning algorithm and an image clustering method, and particularly relates to aurora image clustering methods based on a deep neural network, which can be used for clustering large-scale aurora images.

Background

The aurora is only macroscopic high-altitude atmospheric phenomenon in the south and north extreme latitude areas of the earth, which can directly reflect the connection between the change of the earth magnetic field and the solar activity.

Early aurora image classification work was visually observed by scientists and manually labeled according to the classification mechanism criteria set at The quantity of aurora images of only arctic yellow river stations reaches millions of levels every year, and the traditional manual classification method cannot meet the requirement of analyzing a large amount of aurora data. With the development of computer technology, image processing and machine learning techniques are beginning to be applied to the study of aurora image classification. 2004, the year

In the literature of et al "

M T, Donovan EF.Dinal aurora observational science induced via a mechanization, 2004,22(4): 1103-. In 2010, Wang et al in the document "Wang Q, Liang J, Hu Z J, et al, spatial temporal textual automatic classification of daylight autora in all-sky images.J. Atmos.Sol. -Terr.Phys.,2010,72:498 and 508" propose a spatial texture-based characterization method to characterize the global shape and local texture of an aurora image and classify a daytime aurora image observed overwintering at a yellow river station.

The concept of "deep learning" was proposed in the document "Hinton G E, Osindero S, Teh YW. A fast learning algorithm for deep belief nets. neural Compout, 2006,18: 1527-. Until 2012 AlexNet took champions on the Large-Scale Visual Recognition Challenge (Large-Scale Visual Recognition Challenge) of ImageNet, and from then on the Deep Convolutional Neural Network (DCNN) gradually replaced the traditional machine learning method. In 2018, Clausen and Nickisch used an inclusion-V4 deep neural network to divide aurora images into 6 categories of electrodeless, cloud, moon, arc aurora, diffuse aurora and discrete aurora. In recent years, although aurora research methods based on manual features are still continued, the classical manual features have been gradually replaced by an intermediate feature layer of a deep convolutional neural network, and superior performance to the conventional manual features is achieved.

However, the aurora image classification mechanism is not uniform , the workload of marking data is large and tedious, and the accuracy and objectivity of a label are difficult to guarantee by manual marking.

Disclosure of Invention

Aiming at the defects of the aurora image supervision and classification method based on the deep learning technology, the invention provides kinds of aurora image unsupervised clustering networks, which do not need to manually appoint a classification mechanism and data labels, can automatically classify aurora images and greatly improve the aurora classification efficiency.

In order to achieve the purpose, the invention adopts the following technical scheme:

aurora image clustering method based on deep neural network, comprising the following steps:

step 1: pre-training a VGGNet 16-based depth convolution self-encoder DCAE _ VGG using all aurora images to be clustered;

step 2: extracting a feature vector of each aurora image by using the trained DCAE _ VGG;

and step 3: the feature vectors are clustered using a clustering network consisting of siemes, SpectralNet, and k-means clustering.

Step , step 1 specifically includes:

step 1.1: combining VGGNet16 with an auto-encoder to build a DCAE _ VGG model structure of the deep convolutional auto-encoder;

step 1.2: adjusting the size of the aurora image to be clustered to 64 x 64, inputting the size into DCAE _ VGG, training the DCAE _ VGG until a loss function is converged, wherein the DCAE _ VGG optimizes parameter values by using a random gradient descent method, and selects a mean square error as the loss function;

step 1.3: and saving the model structure and the model parameters of the DCAE _ VGG.

, the DCAE _ VGG model structure in step 1.1 is composed of an encoding part, a hidden layer and a decoding part, wherein the first five groups of neural networks and corresponding pooling layers of VGGNet16 are reserved in the encoding part, full-connection layers with the length of 512 are used for replacing the original three full-connection layers and softmax layers, the hidden layer is full-connection layers with the length of 10 and used for representing low-dimensional feature vectors of the aurora image, and the decoding part is composed of full-connection layers with the length of 512 and four deconvolution layers.

, the step 2 is to call the model structure and model parameters of the DCAE _ VGG, to input the n polar light images to be clustered into the coding part of DCAE _ VGG only by using the model structure and parameters of the coding part and hidden layer part of DCAE _ VGG, to obtain the feature vectors { x } of these images through the hidden layer of DCAE _ VGG₁，x₂，...，x_n}。

Step , step 3 specifically includes:

step 3.1: training a Siamese network by constructing a positive/negative similarity pair training set, wherein the trained Siamese network is used for calculating a similarity matrix W in the SpectralNet in step 3.2;

step 3.2: training the SpectralNet to obtain input { x₁，x₂，...，x_nThe corresponding mapping output y₁，y₂，...，y_nI.e. y ═ F_θ(x) Theta is a parameter of SpectralNet;

step 3.3: for { y₁，y₂，...，y_nAnd (5) clustering by using a k-means clustering method to obtain a clustering result.

Step , step 3.1 specifically includes:

step 3.1.1: constructing a positive/negative similarity pair training set for the Siemese network by calculating the Euclidean distance of two feature vectors, marking k nearest neighbor points of each point as positive similarity pairs by the Siemese network, randomly selecting equivalent non-neighbor points as negative similarity pairs, and determining the value of k by setting the number of the Siemese network similarity pairs;

step 3.1.2: training the Siamese network to obtain the mapping relation

Wherein theta is_SiameseParameters representing the Siemese network will have feature vectors x every time_iMapping to z in another spaces_iDetermined by minimizing a loss function during trainingLoss function L for fixed mapping, Siamese networks_siameseIs defined as:

where c is a constant, after the Siamese network training is completed, | | | z is used in step 3.2_i-z_jI replaces Euclidean distance x_i-x_jThe similarity matrix W in SpextralNet is calculated.

Step , step 3.2 specifically includes:

step 3.2.1: feature vector from all aurora images { x₁，x₂，...，x_n groups of m small-batch feature samples { x } are randomly extracted₁，x₂，...，x_mAnd obtaining corresponding mapping output of the feature samples of the m aurora images after the feature samples pass through an orthogonal layer of the SpectralNet

Step 3.2.2: computing Cholesky factorization

Step 3.2.3: set the weights of the SpectralNet orthogonal layers to

Step 3.2.4: feature vector from all aurora images { x₁，x₂，...，x_n groups of m small-batch feature samples { x } are randomly extracted₁，x₂，...，x_mInputting the characteristic samples of the m aurora images into a trained Siemese network to obtain a similarity matrix W with the output of m multiplied by m;

step 3.2.5: feature sample { x ] of aurora image₁，x₂，...，x_mInput SpectralNet, calculate the loss function L of SpectralNet_SpectralNet(theta) adjustment by its gradientWeights, L, of other layers than the orthogonal layer of SpectralNet_SpectralNet(θ) is defined as:

step 3.2.6: repeating the steps 3.2.1-3.2.5 while losing the function L_SpectralNet(theta) when convergence is carried out, determining the parameters and the weight of the SpectralNet, and finishing the training of the SpectralNet;

step 3.2.7: feature vector { x of aurora image to be clustered₁，x₂，...，x_nInputting the trained SpectralNet, namely obtaining the corresponding mapping output y₁，y₂，...，y_n}。

Compared with the prior art, the invention has the following beneficial technical effects:

, subject crossing, the invention utilizes the side polar light observation data of the arctic yellow river station, automatically classifies the polar light images by means of a computer image processing technology and a deep learning technology, applies the deep learning technology to the clustering of the polar light images for the first time, and is helpful for researching the generation mechanism of various polar lights in the step .

Secondly, the method does not depend on any prior classification mechanism and does not need to manually mark data, the aurora image classification based on the deep learning technology is supervised classification at present, and partial data need to be manually marked according to the prior classification mechanism for model training. The workload of marking data is large and tedious, and manual marking is difficult to ensure the objectivity and accuracy of the label. According to the invention, the features of the aurora images are automatically extracted through DCAE _ VGG, and the clustering network consisting of Siemese, SpectralNet and k-means clustering is used for unsupervised classification of the aurora images, so that the problems existing in the current aurora image supervised classification are solved, and the aurora image classification efficiency is greatly improved.

Thirdly, the optimal classification of the aurora images is explored, no aurora classification mechanism of the system exists at present, the potential features of the aurora images are extracted through a deep neural network, unsupervised clustering is carried out on the aurora images, and the aurora images with high morphological similarity are automatically classified into classes.

Drawings

FIG. 1 is a flow chart of an implementation of the present invention;

FIG. 2 is a schematic diagram of a deep convolutional auto-encoder DCAE _ VGG network structure based on VGGNet16 in the present invention;

FIG. 3 is a schematic view of a visualization of a feature vector for DCAE _ VGG extraction;

FIG. 4 is an aurora occurrence time profile based on class 2 tag labels;

FIG. 5 is a distribution diagram of aurora occurrence time based on class 2 results;

fig. 6 is an exemplary diagram of a clustering result.

Detailed Description

The following describes the implementation steps and technical effects of the present invention in detail with reference to the accompanying drawings.

Referring to fig. 1, the implementation steps of the invention are as follows:

step 1: the depth convolution self-encoder DCAE _ VGG based on VGGNet16 was pre-trained using all the aurora images to be clustered.

1.1) combining VGGNet16 with a self-encoder, building network structures such as a deep convolutional self-encoder DCAE _ VGG.DCAE _ VGG shown in FIG. 2, wherein the deep convolutional self-encoder DCAE _ VGG is composed of an encoding part, a hidden layer (characteristic vector) and a decoding part, the first five groups of neural networks of VGGNet16 and corresponding pooling layers are reserved in the encoding part, full-connection layers with the length of 512 are used for replacing the original three full-connection layers and softmax layers, the hidden layer is full-connection layers with the length of 10 and is used for representing low-dimensional characteristic vectors of an aurora image, and the decoding part is composed of full-connection layers with the length of 512 and four deconvolution layers.

1.2) the size of all aurora images to be clustered is adjusted to 64 x 64 and input into DCAE _ VGG, and the DCAE _ VGG is trained until the loss function converges. The DCAE _ VGG optimizes parameter values by using a random gradient descent method, and selects a mean square error as a loss function.

1.3) saving the model structure and model parameters of the DCAE _ VGG.

Step 2: and extracting a feature vector of each aurora image by using the trained DCAE _ VGG. Call securityThe model structure and model parameters of the stored DCAE _ VGG are only used when the feature vector is extracted, and the model structure and model parameters of the coding part and the hidden layer part of the DCAE _ VGG are only used when the feature vector is extracted. All n aurora images to be clustered are input into a coding part of DCAE _ VGG, and feature vectors { x ] of the images are obtained through a hidden layer of the DCAE _ VGG₁，x₂，...，x_n}。

And step 3: the feature vectors are clustered using a clustering network consisting of siemes, SpectralNet, and k-means clustering.

3.1) training the Siamese network by constructing a training set of positive/negative similarity pairs, the trained Siamese network will be used to calculate the similarity matrix W in SpectralNet in step 3.2:

3.1.1) through calculating the Euclidean distance of two eigenvectors, construct the training set of positive/negative similar pairs for the Siemese network, the Siemese network marks k nearest neighbor points of each point as positive similar pairs, randomly selects equivalent non-neighbor points as negative similar pairs, the numerical value of k is determined by setting the number of the Siemese network similar pairs, and the number of the similar pairs in the experiment is set as 3200.

3.1.2) train the Siamese network, which will train the Siamese network every feature vectors x_iMapping to z in another spaces_iThe mapping relation is

θ_SiameseRepresenting parameters of the Siamese network. The size of the network structure of the Siemese network is 512-4, the mapping relation is determined by minimizing a loss function in the training process, and the loss function L of the Siemese network_siameseIs defined as:

where the constant c is set to 1. After the siense network training is completed, | | z is used when calculating the similarity matrix W in SpextralNet in step 3.2_i-z_jI replaces Euclidean distance x_i-x_jL, wherein

σ is set to 25.

3.2) training SpectralNet:

the network structure size of the SpectralNet is 512-4, and the number of cluster clusters is set to be 2. Input { x is obtained by SpectralNet₁，x₂，...，x_nThe corresponding mapping output y₁，y₂，...，y_nI.e. y ═ F_θ(x) And theta is a parameter of SpectralNet.

3.2.1) feature vectors from all aurora images { x₁，x₂，...，x_n groups of m small-batch feature samples { x } are randomly extracted₁，x₂，...，x_mAnd after the characteristic samples of the m aurora images pass through an orthogonal layer of the SpectralNet, obtaining corresponding mapping output of the characteristic samples

3.2.2) calculating Cholesky factorization

3.2.3) set the weights of the SpectralNet orthogonal layers to

3.2.4) feature vector from all aurora images { x₁，x₂，...，x_n groups of m small-batch feature samples { x } are randomly extracted₁，x₂，...，x_mInputting the characteristic samples of the m aurora images into a trained Siemese network, and calculating a similarity matrix W;

3.2.5) sample features of an aurora image { x }₁，x₂，...，x_mInput SpectralNet, calculate the loss function L of SpectralNet_SpectralNet(θ) whose gradient is used to adjust the weights of the other layers of the SpectralNet except the orthogonal layerHeavy, L_SpectralNet(θ) is defined as:

3.2.6) repeat steps 3.2.1-3.2.5 when the function L is lost_SpectralNet(theta) when convergence is carried out, determining the parameters and the weight of the SpectralNet, and finishing the training of the SpectralNet;

3.2.7) feature vector { x) of aurora images to be clustered₁，x₂，...，x_nInputting the trained SpectralNet to obtain the corresponding mapping output y₁，y₂，...，y_n}。

3.3) pairs { y₁，y₂，...，y_nAnd (5) clustering by using a k-means clustering method to obtain a clustering result.

The effects of the present invention can be further described by the following simulation experiments .

Experiment 1: the DCAE _ VGG model extracts a feature vector of the aurora image.

The experimental conditions are as follows: the method uses 4000 aurora images acquired by the arctic yellow river station 2003-2008 overwintering observation for experiments.

The experimental contents are as follows: the DCAE _ VGG model is trained by using 4000 aurora images, and the structure and the model parameters of the DCAE _ VGG model are saved after the training is finished. And extracting the feature vector of the aurora image by using the trained DCAE _ VGG model, and visualizing the feature vector by using a t-SNE algorithm, wherein the visualization result is shown in FIG. 3.

As can be seen from fig. 3, the feature vectors are distributed in two clusters, which indicates that the potential features of the aurora image are distributed in 2 types, and in addition, the two groups of features have good separability, which indicates that types of highly distinguishable representations of the aurora image are extracted by DCAE _ VGG.

Experiment 2: the aurora images are grouped into 2 types by the invention.

The experimental conditions are as follows: the experiment uses the characteristic vectors of 4000 aurora images obtained in experiment 1. Based on the document Hu et al "Hu Z J, Yang H, Huang D, et al. synthetic distribution of polysaccharide aurora: Multiple-wavelet H all-sky observation at Yellow River Station in Ny-

The polar light classification mechanism of the north-yellow river station day side provided in svalbard.j.attmos.sol. -terr.phys.,2009,71:794-804 ", wherein the 4000 images comprise 1000 images of arc, valance crown, hot spot and radiation crown, and the four types of polar light are combined into 2 types in steps, namely, the arc polar light image and the hot spot polar light image are mixed and labeled into a large type , the valance crown polar light image and the radiation crown polar light image are mixed and labeled into a large type , respectively labeled as type 1 and type 2, and an polar light occurrence time distribution diagram based on the type 2 label is drawn, which is detailed in fig. 4.

The experimental contents are as follows: referring to the visualization result of the feature vector in experiment 1, 4000 aurora images were grouped into 2 types using the method proposed by the present invention. The distribution of the occurrence time of aurora based on the results of class 2 was plotted, and details are shown in FIG. 5. From the clustering results, 20 aurora images of each type were randomly selected for frame-by-frame comparison, as shown in fig. 6.

Comparing fig. 4 and fig. 5, it can be seen that the curve trend of the distribution of the aurora occurrence time is basically , from the clustering result fig. 5, it can be seen that the -th aurora in the clustering result shows a "bimodal" distribution in the front and back of the moon, and rarely occurs near the noon, the second aurora mainly occurs in the front and near the noon, and rarely occurs on the dim side, from fig. 6, it can be seen that the morphology of the aurora image aggregated into the -th category has the characteristics of a single arc, a multi-arc structure, a bright spot structure, and the like, and the morphology of the aurora image aggregated into the second category is overall darker and has the characteristics of weak and uniform aurora intensity.

Claims (7)

1, aurora image clustering method based on deep neural network, characterized by comprising the following steps:

step 1: pre-training a VGGNet 16-based depth convolution self-encoder DCAE _ VGG using all aurora images to be clustered;

step 2: extracting a feature vector of each aurora image by using the trained DCAE _ VGG;

and step 3: the feature vectors are clustered using a clustering network consisting of siemes, SpectralNet, and k-means clustering.

2. The aurora image clustering method based on the deep neural network as claimed in claim 1, wherein step 1 specifically includes:

step 1.1: combining VGGNet16 with an auto-encoder to build a DCAE _ VGG model structure of the deep convolutional auto-encoder;

step 1.2: adjusting the size of the aurora image to be clustered to 64 x 64, inputting the size into DCAE _ VGG, training the DCAE _ VGG until a loss function is converged, wherein the DCAE _ VGG optimizes parameter values by using a random gradient descent method, and selects a mean square error as the loss function;

step 1.3: and saving the model structure and the model parameters of the DCAE _ VGG.

3. The aurora image clustering method based on deep neural network as claimed in claim 2, wherein in step 1.1, the DCAE _ VGG model structure is composed of a coding part, a hidden layer and a decoding part, wherein the coding part reserves the first five groups of neural networks of VGGNet16 and corresponding pooling layers, the original three full-connected layers and softmax layer are replaced by full-connected layers with length of 512, the hidden layer is full-connected layers with length of 10 and is used for representing low-dimensional feature vectors of aurora images, and the decoding part is composed of full-connected layers with length of 512 and four deconvolution layers.

4. The aurora image clustering method based on deep neural network of claim 3, wherein the step 2 is to call the model structure and model parameters of the saved DCAE _ VGG, to input the n aurora images to be clustered into the coding part of DCAE _ VGG by using only the model structure and parameters of the coding part and hidden layer part of DCAE _ VGG, to obtain the feature vector { x ] of these images through the hidden layer of DCAE _ VGG₁，x₂，...，x_n}。

5. The aurora image clustering method based on the deep neural network as claimed in claim 4, wherein step 3 specifically includes:

step 3.1: training a Siamese network by constructing a positive/negative similarity pair training set, wherein the trained Siamese network is used for calculating a similarity matrix W in the SpectralNet in step 3.2;

step 3.2: training the SpectralNet to obtain input { x₁，x₂，...，x_nThe corresponding mapping output y₁，y₂，...，y_nI.e. y ═ F_θ(x) Theta is a parameter of SpectralNet;

step 3.3: for { y₁，y₂，...，y_nAnd (5) clustering by using a k-means clustering method to obtain a clustering result.

6. The aurora image clustering method based on the deep neural network as claimed in claim 5, wherein step 3.1 specifically includes:

step 3.1.1: constructing a positive/negative similarity pair training set for the Siemese network by calculating the Euclidean distance of two feature vectors, marking k nearest neighbor points of each point as positive similarity pairs by the Siemese network, randomly selecting equivalent non-neighbor points as negative similarity pairs, and determining the value of k by setting the number of the Siemese network similarity pairs;

step 3.1.2: training the Siamese network to obtain the mapping relation

Wherein theta is_SiameseParameters representing the siemese network, which will be every eigenvectors x_iMapping to z in another spaces_iThe mapping relation is determined by minimizing the loss function in the training process, the loss function L of the Siamese network_siameseIs defined as:

where c is a constant, after the Siamese network training is completed, | | | z is used in step 3.2_i-z_jI replaces Euclidean distance x_i-x_jThe similarity matrix W in SpextralNet is calculated.

7. The aurora image clustering method based on the deep neural network as claimed in claim 6, wherein step 3.2 specifically includes:

step 3.2.1: feature vector from all aurora images { x₁，x₂，...，x_n groups of m small-batch feature samples { x } are randomly extracted₁，x₂，...，x_mAnd obtaining corresponding mapping output of the feature samples of the m aurora images after the feature samples pass through an orthogonal layer of the SpectralNet

Step 3.2.2: computing Cholesky factorization

Step 3.2.3: set the weights of the SpectralNet orthogonal layers to

Step 3.2.4: feature vector from all aurora images { x₁，x₂，...，x_n groups of m small-batch feature samples { x } are randomly extracted₁，x₂，...，x_mInputting the characteristic samples of the m aurora images into a trained Siemese network to obtain a similarity matrix W with the output of m multiplied by m;

step 3.2.5: feature sample { x ] of aurora image₁，x₂，...，x_mInput SpectralNet, calculate the loss function L of SpectralNet_SpectralNet(theta) adjustment by its gradientWeights, L, of other layers than the orthogonal layer of SpectralNet_SpectralNet(θ) is defined as:

step 3.2.6: repeating the steps 3.2.1-3.2.5 while losing the function L_SpectralNet(theta) when convergence is carried out, determining the parameters and the weight of the SpectralNet, and finishing the training of the SpectralNet;

step 3.2.7: feature vector { x of aurora image to be clustered₁，x₂，...，x_nInputting the trained SpectralNet, namely obtaining the corresponding mapping output y₁，y₂，...，y_n}。

Images