CN110188612B - Aurora egg intensity image modeling method based on generating type countermeasure network - Google Patents

Abstract

The invention discloses an aurora ova intensity image modeling method based on a generating countermeasure network, which mainly solves the problem that the prediction result of the conventional model on the aurora ova intensity is inaccurate. The method comprises the following implementation steps: 1) Selecting aurora egg image data from an ultraviolet aurora image shot by a Polar satellite, selecting a spatial parameter corresponding to aurora egg image time from an OMNI database, and preprocessing the spatial parameter, 2) corresponding the preprocessed aurora egg image data and the spatial parameter data into an aurora egg image data and spatial parameter data pair one by one according to a time relationship, and separating training data and testing data from the data pairs; 3) Training the generative confrontation network by using training data to obtain a trained generator G and a trained discriminator D; 4) And inputting the spatial parameters in the test data into a trained generator G to obtain an aurora egg intensity image. The method improves the accuracy of aurora egg intensity prediction, and can be used for predicting aurora egg intensity.

Description

Aurora egg intensity image modeling method based on generative adversarial network

technical field

The invention belongs to the technical field of image processing, and further relates to an image modeling method, which can be used for modeling an ultraviolet aurora egg intensity image.

Background technique

The aurora is the gorgeous brilliance produced by the interaction of the falling particles along the magnetic field lines and the earth's upper atmosphere when the solar wind is injected into the earth's magnetosphere through the polar gap region on the sun side. From a physical point of view, the aurora is produced by the interaction of high-energy charged particles in the sun with atoms and molecules in the upper atmosphere of the polar region under the action of the earth's magnetic field, that is to say, the solar wind, the magnetic field of the earth and the upper atmosphere of the polar region are Necessary conditions for the formation of aurora. Therefore, the occurrence of aurora reflects the dynamic relationship between the sun and geomagnetic activity, which helps people understand the way and degree of the sun's influence on the earth. Secondly, some radio waves radiated when the aurora occurs will directly affect radio communication, navigation, positioning and circuit transmission on the earth. However, when the aurora occurs, the energy erupted in the earth's atmosphere can almost reach the sum of the electricity generated by power plants in all countries in the world. Therefore, how to use the huge energy generated by the aurora to benefit mankind has become an important research topic in the field of science today. Research and facts have shown that the auroral phenomenon is a common phenomenon of magnetar bodies in the solar system. The Hubble Space Telescope has been able to clearly see the aurora on the two planets Jupiter and Saturn. Therefore, the study of the aurora on the earth will help humans study the aurora phenomena on other planets.

The intensity and spatial position of the auroral eggs are important physical quantities for studying the dynamics of the magnetosphere and the space atmosphere, and can be used to predict substorms and hemispheric energy, and can also help people further understand the interaction between the solar wind and the Earth's magnetosphere .

The UVI of the Polar satellite can obtain the global auroral oval information, and since the launch of the Polar satellite, a large number of auroral oval images have been obtained. NASA OMNI data includes 32 spatial parameters and geomagnetic environment parameters, but many of them have similar functions and some have basically no relationship with the aurora. In the OMNI data, the three components of the interplanetary magnetic field (Bx, By, Bz), the solar wind velocity Vp, the solar wind density Np, and the geomagnetic index AE that are closely related to the auroral substorm are closely related to the auroral egg. Can be used to model the intensity of the auroral oval. However, in the current modeling methods for the intensity of the auroral oval, more useful univariate analysis methods are used to model the intensity of the auroral oval. The more classic one is that Y. Zhang et al. established the KP The aurora egg model of , which predicts the energy flux as well as global information about the mean electron energy. However, because the KP index used can only reflect part of the factors affecting the auroral oval, and the time resolution is low, the prediction result of the model for the intensity of the auroral oval is not accurate enough.

Contents of the invention

The purpose of the present invention is to address the deficiencies in the prior art above, and propose a method for modeling aurora oval intensity images based on generative confrontation networks, so as to improve the accuracy of the model in predicting the intensity of aurora ovals.

The technical idea of the present invention is: take 6 space environment parameters that closely affect the intensity of the aurora egg as input, and use the method of deep learning to model, and its implementation steps include the following:

(1) Select the auroral oval image data from the ultraviolet aurora image captured by the ultraviolet imager carried by the Polar satellite, and select the space environment parameters corresponding to the time of the auroral oval image from the OMNI database;

(2) Preprocess the selected aurora egg image data, that is, first convert the original image into a coordinate system centered on the geomagnetic pole, remove the data points whose geomagnetic latitude is less than 50, and then clear the negative points in the image And smooth and denoise the image;

(3) Get the moving average of 11 minutes to the space environment parameter selected in (1), obtain the space environment parameter data after preprocessing;

(4) Correspond the preprocessed aurora egg image data in (2) and the preprocessed space environment parameter data in (3) into pairs of aurora egg image data and space environment parameter data according to the time relationship, and from the aurora egg 70% of image data and space environment parameter data pairs are selected as training data, and the remaining 30% are used as test data;

(5) Use the training data to train the generator G and the discriminator D in the generative confrontation network through alternate iterations, and obtain the trained generator G* and the trained discriminator D*;

(6) Input the spatial environment parameters in the test data into the trained generator G* to obtain the predicted intensity image of the aurora egg.

Compared with the prior art, the present invention has the following advantages:

First, the present invention uses six spatial environment parameters that closely affect the intensity of the aurora egg as the input of the prediction model, which can more comprehensively reflect the factors that affect the change of the intensity of the aurora egg.

Second, the present invention uses the generative confrontation network in deep learning to model the intensity of the aurora egg, avoiding the error introduced by artificially setting the objective function of the deep learning model.

Third, the present invention adds traditional L1 objective function items and SSIM objective function items on the basis of the objective function of the generative confrontation network, thereby improving the accuracy of model prediction.

Description of drawings

Fig. 1 is the realization flowchart of the present invention;

Fig. 2 is the preprocessed aurora egg intensity image and the aurora egg intensity image without preprocessing in the present invention;

Fig. 3 is the sub-flow chart of the training to the generative confrontation network in the present invention;

Fig. 4 is a generator structure diagram of a generative confrontation network in the present invention;

Fig. 5 is a discriminator structural diagram of a generative confrontation network in the present invention;

Fig. 6 is a comparison diagram of the intensity of the aurora egg predicted in the simulation experiment 1 based on the present invention and the existing GRNN model;

Fig. 7 is a comparison diagram of the intensity of the aurora egg predicted in the simulation experiment 2 based on the present invention and the existing GRNN model;

Detailed ways

The embodiments and effects of the present invention will be further described in detail below in conjunction with the accompanying drawings.

Referring to Figure 1, the implementation steps of this example are as follows:

Step 1. Select the ultraviolet aurora egg image data and the space environment parameter data.

Select the auroral oval image data from the ultraviolet auroral images taken by the ultraviolet imager carried by the Polar satellite;

The spatial environment parameters corresponding to the time of the aurora egg image are selected from the OMNI database.

Step 2, preprocessing the image data of Aurora Oval.

First, convert the original image shown in Figure 2(a) into a coordinate system centered on the geomagnetic pole, and remove the data points whose geomagnetic latitude is less than 50; then clear the negative points in the image, and smooth the image Noise, that is, divide an appropriate neighborhood window with any pixel point (i, j) in the aurora egg image as the center, and calculate the average value of all pixels in the neighborhood window, and use the calculated average value as the smooth denoising The image corresponds to the value of the pixel point (i,j):

Among them, P is the number of pixels in the neighborhood window, H is the size of the window in the selected area, a(i,j) represents the pixel value of the pixel point (i,j) of the aurora egg image, b(i,j) is the smoothing The pixel value of the image pixel point (i, j) after noise;

The preprocessed UV aurora egg image is shown in Fig. 2(b).

Step 3, preprocessing the spatial environment parameter data.

Take the 11-minute moving average of the space environment parameters selected in step 1 to obtain the preprocessed space environment parameter data:

In this example, an 11-minute sliding window is taken as the center of any sample F(d) in the spatial environment parameter data before preprocessing, and the average value of the spatial environmental parameters of all samples in the sliding window is calculated, and the obtained average value is used as the preprocessing The following spatial environment parameter data corresponds to the value of sample B(w):

Step 4, divide training data and test data.

The aurora egg image data preprocessed in step 2 and the space environment parameter data after step 3 preprocessing are corresponding to the aurora egg image data and the space environment parameter data pair according to the time relationship;

Then select training data and test data from the pair of aurora egg image data and space environment parameter data. There are two selection methods:

The first is to randomly select 70% as training data and the remaining 30% as testing data.

The second is to select the first 70% as training data in chronological order, and the remaining 30% as test data.

Step 5, use the training data to train the generative confrontation network.

The generative confrontation network is a deep learning model, which consists of a generator G and a discriminator D, and continuously improves the performance of the generator G and the discriminator D through the mutual game learning of the generator G and the discriminator D.

The structure of the generator G is shown in Figure 4, in which the 1*1*6 space environment parameters are deconvolved 8 times, and the 2*2*512 matrix, 4*4*512 matrix, 8*8 *512 matrix, 16*16*512 matrix, 32*32*256 matrix, 64*64*128 matrix, 128*128*64 matrix and 256*256*1 aurora egg intensity images.

The structure of the discriminator D is shown in Figure 5, wherein the 1*1*6 space environment parameters are deconvolved 5 times, and the 2*2*8 matrix, 4*4*8 matrix, 8*8 matrix can be obtained in turn. *8 matrix, 16*16*8 matrix and 32*32*8 matrix. Perform 3 convolutions on the 256*256*1 aurora egg intensity image to obtain a 128*128*64 matrix, a 64*64*128 matrix and a 32*32*256 matrix in sequence. Connect the 32*32*8 matrix obtained by deconvolution and the 32*32*256 matrix obtained by convolution in the third dimension to obtain a 32*32*264 matrix, and perform convolution on the connected 32*32*264 matrix The product gets a 31*31*1 matrix.

This example uses the training data to train the generator G and the discriminator D in the generative confrontation network through alternate iterations, and obtains the trained generator G* and the trained discriminator D*,

Referring to Figure 3, the specific implementation of this step is as follows:

(5a) Define the objective function L:

L=αM(G,D)+βN(G)+λS(G),

Among them, M(G,D)=E _x,y～p(x,y) [log D(x,y)]+E _x～p(x) [log(1-D(x,G(x) )] is the objective function item of the generative confrontation network, x is the spatial environment parameter, y is the aurora egg intensity image, p(x,y) is the training data, D(x,y) is the spatial environment parameter x and the aurora egg intensity image When y is used as the output of the discriminator D, p(x) is the spatial environment parameter data in the training data, G(x) is the generated aurora egg intensity image, and D(x,G(x)) is the spatial environment parameter x And the generated aurora egg intensity image G(x) is used as the output of the discriminator D when input;

N(G)＝E _x,y～p(x,y) [||yG(x)|| ₁ ] is the L1 objective function item, and ||.|| ₁ means 1 norm;

是相似度目标函数项，

是两图像间相似度函数；

is the similarity objective function item,

is the similarity function between two images;

α, β and λ are the weights of M(G,D), N(G) and S(G) in the overall objective function respectively, and the values of α, β and λ are determined through experiments;

(5b) Calculate the similarity function between two images in the similarity objective function item S(G)

(5b1) Input the spatial environment parameter x into the generator G to obtain the generated aurora egg intensity image

在亮度上的相似度

(5b2) Calculate the real auroral oval intensity image y and the generated auroral oval intensity image

similarity in brightness

表示生成的极光卵强度图像

的均值，c₁是一个小于0.00001的常数；where u _y represents the mean value of the real auroral oval intensity image y,

Represents the resulting auroral oval intensity image

The mean value of , c ₁ is a constant less than 0.00001;

在对比度上的相似度

(5b3) Calculate the real auroral oval intensity image y and the generated auroral oval intensity image

similarity in contrast

表示真实极光卵强度图像y的方差、

表示生成的极光卵强度图像

的方差，c₂是一个小于0.00001的常数；in

Indicates the variance of the real auroral oval intensity image y,

Represents the resulting auroral oval intensity image

The variance of , c ₂ is a constant less than 0.00001;

在结构上的相似度

(5b4) Calculate the real auroral egg intensity image y and the generated auroral egg intensity image

similarity in structure

表示生成的极光卵强度图像

和真实极光卵强度图像y的协方差，

表示生成的极光卵强度图像

中第i行第j列的像素值，y_ij表示真实极光卵强度图像y中第i行第j列的像素值，Q表示极光卵强度图像的长，R表示极光卵图像的宽，c₃是一个小于0.00001的常数；in

Represents the resulting auroral oval intensity image

and the covariance of the real auroral oval intensity image y,

Represents the resulting auroral oval intensity image

The pixel value of row i and column j in , y _ij represents the pixel value of row i and column j in the real auroral egg intensity image y, Q represents the length of the auroral egg intensity image, R represents the width of the auroral egg image, c ₃ is a constant less than 0.00001;

(5b5) Calculate the similarity function between two images

在两图像间相似度函数

中的权重；Among them, a, b and e are respectively

The similarity function between two images

weight in

(5c) Keep the parameters in the discriminator D unchanged, and update the parameters in the generator G to minimize the objective function L;

(5d) Taking the spatial environment parameter x as input, use the updated generator G to generate an auroral oval intensity image G(x);

(5e) Keep the parameters of the generator G unchanged, use the spatial environment parameter x and the real aurora egg intensity image y data pair {x, y} and the spatial environment parameter x and the generated aurora egg intensity image G(x) data pair {x ,G(x)} as input, update the parameters of the discriminator D to maximize the objective function L;

(5f) Iteratively carry out the process of (5c)-(5e) until the iteration stop condition is satisfied, and the trained generator G* and the trained discriminator D* are obtained;

Step 6. Predict the intensity image of the auroral oval.

Input the spatial environment parameters in the test data into the trained generator G* to get the predicted intensity image of the aurora egg.

Effect of the present invention is further illustrated by following experiments:

1. Experimental conditions

Experimental hardware equipment: Linux 3.19.0TIAITAN

Experimental software platform: Tensorflow 1.2.0Python3.6.3

Experimental data: 141 images of the auroral oval image data of the Polar satellite and the corresponding space environment parameters.

2. Experimental content

Experiment Simulation 1,

70% of the pairs of aurora egg image data and space environment parameter data are randomly selected as training data, and the remaining 30% are used as test data.

Using the model in the present invention and the existing GRNN-based model to predict the intensity image of the auroral oval, the results are shown in Figure 6.

Wherein Fig. 6 (a) is the ultraviolet auroral oval intensity image after preprocessing, Fig. 6 (b) is the auroral oval intensity image obtained based on the prediction of the existing GRNN model, and Fig. 6 (c) is the auroral oval predicted by the present invention intensity image. It can be seen from Fig. 6 that the auroral oval intensity image predicted by the present invention is the most similar to the preprocessed ultraviolet auroral oval intensity image, indicating that the auroral oval intensity image predicted by the present invention is the most accurate.

Experiment Simulation 2,

Select the first 70% from the pair of aurora egg image data and space environment parameter data in chronological order as training data, and the remaining 30% as test data.

Use the model in the present invention and the existing GRNN-based model to predict the intensity image of the auroral oval, and the results are shown in Figure 7.

Among them, Fig. 7(a) is the preprocessed ultraviolet auroral oval intensity image, Fig. 7(b) is the auroral oval intensity image predicted based on the GRNN model, and Fig. 7(c) is the auroral oval predicted by the present invention intensity image. It can be seen from Fig. 7 that the intensity image of the auroral oval predicted by the present invention is the most similar to the preprocessed ultraviolet auroral oval intensity image, indicating that the intensity image of the auroral oval predicted by the present invention is the most accurate.

3. Evaluation of simulation results

和KL散度对上述实验仿真1和实验仿真2做客观评价。Use the similarity function between images

and KL divergence to make an objective evaluation of the above experimental simulation 1 and experimental simulation 2.

The calculation formula of KL divergence is as follows:

Where p is the distribution of brightness values of the aurora egg brightness image, q is the distribution of brightness values of the predicted aurora egg brightness image, and v is the brightness value. The smaller the KL value, the better the prediction result.

以及KL散度的平均值，结果如表1所示。Calculate the similarity function between the images of the method of the present invention and the existing GRNN model-based prediction of the intensity image of the aurora oval in Experimental Simulation 1

And the average value of KL divergence, the results are shown in Table 1.

和KL散度的平均值Table 1 Similarity function between two model images

and the mean of the KL divergence

It can be seen from Table 1 that the auroral oval intensity image predicted by the present invention is better than the auroral oval intensity image predicted based on the GRNN model.

以及KL散度的平均值，结果如表2所示。Calculate the similarity function between the images of the method of the present invention and the existing GRNN model-based prediction of the intensity image of the aurora oval in the experimental simulation 2

And the average value of KL divergence, the results are shown in Table 2.

和KL散度的平均值Table 2 Similarity function between two model images

and the mean of the KL divergence

It can be seen from Table 2 that the auroral oval intensity image predicted by the present invention is better than the auroral oval intensity image predicted based on the GRNN model, indicating that the present invention can predict the auroral oval intensity image more accurately.

Claims (3)

1. The aurora egg intensity image modeling method based on the generative countermeasure network comprises the following steps:

(1) Selecting aurora egg image data from an ultraviolet aurora image shot by an ultraviolet imager carried by a Polar satellite, and selecting a space environment parameter corresponding to the aurora egg image time from an OMNI database;

(2) Preprocessing the selected aurora oval image data, namely converting an original image into a coordinate system taking a geomagnetic pole as a center, removing data points with the geomagnetic latitude smaller than 50, clearing negative values in the image and carrying out smooth denoising on the image;

(3) Taking 11-minute sliding average of the spatial environment parameters selected in the step (1) to obtain preprocessed spatial environment parameter data;

(4) Correspondingly one-to-one correspondence is made between the aurora egg image data preprocessed in the step (2) and the space environment parameter data preprocessed in the step (3) according to a time relation, 70% of the aurora egg image data and the space environment parameter data are selected as training data, and the remaining 30% of the aurora egg image data and the space environment parameter data are used as test data;

(5) Training a generator G and a discriminator D in the generative confrontation network by using training data in an alternating iteration mode to obtain a trained generator G and a trained discriminator D; the generator G and the discriminator D in the generative confrontation network are trained in an alternating iteration mode, and the training is carried out according to the following steps:

(5a) The objective function L is defined as follows:

L＝αM(G,D)+βN(G)+λS(G)

M(G,D)＝E _x,y～p(x,y) [log D(x,y)]+E _x～p(x) [log(1-D(x,G(x))]，

N(G)＝E _x,y～p(x,y) [||y-G(x)|| ₁ ]

wherein M (G, D) is a generative confrontation network objective function term, N (G) is an L1 objective function term, S (G) is a similarity objective function term, x is a spatial environment parameter, y is a true aurora ovum intensity image, and p (x, y) is training data; α, β and λ are the weights of M (G, D), N (G) and S (G), respectively, in the overall objective function, the values of α, β and λ being determined experimentally;

(5b) Keeping the parameters in the discriminator D unchanged, and updating the parameters in the generator G to minimize the target function L;

(5c) Generating an aurora egg intensity image G (x) using the updated generator G with the spatial environment parameter x as input;

(5d) Keeping the generator G parameter unchanged, using the space environment parameter x and the real aurora ovum intensity image y data pair { x, y } and the space environment parameter x and the generated aurora ovum intensity image G (x) data pair { x, G (x) } as input, and updating the parameter of the discriminator D to maximize the target function L;

(5e) Iteratively performing the processes (5 b) - (5D) until an iteration stop condition is met, and obtaining a trained generator G and a trained discriminator D;

(6) And inputting the spatial environment parameters in the test data into a trained generator G to obtain a predicted aurora egg intensity image.

2. The method according to claim 1, wherein in (2), the smooth denoising is performed on the aurora ovum image, an appropriate neighborhood window is divided by taking any pixel point (i, j) in the aurora image as a center, and an average value of all pixel points in the neighborhood window is calculated, and the calculated average value is taken as a value of a corresponding pixel point (i, j) of the image after smooth denoising:

wherein P is the number of pixels in the neighborhood window, H is the size of the selected domain window, a (i, j) represents the pixel value of the pixel point of the aurora egg image, and b (i, j) represents the pixel value of the pixel point of the image after smooth denoising.

3. The method of claim 1, wherein the structural similarity objective function term S (G) in (5 a) is implemented as follows:

(5a1) Inputting the space environment parameter x into a generator G to obtain a generated aurora egg intensity image

(5a2) Calculating a true aurora egg intensity image y and generating an aurora egg intensity image

Similarity in luminance

Wherein

u _y Respectively represent

Mean value of y, c ₁ Is a constant less than 0.00001;

(5a3) Calculating true aurora ovumIntensity image y and the resulting aurora egg intensity image

Similarity in contrast

Wherein

Respectively represent

Variance of y, c ₂ Is a constant less than 0.00001;

(5a4) Calculating a true aurora egg intensity image y and generating an aurora egg intensity image

Similarity in structure

Wherein

Representing the generated aurora egg intensity image

And the covariance of the true aurora egg intensity image y,

representing the generated aurora egg intensity image

Pixel value of ith row and jth column in the row, y _ij Representing the pixel value of the ith row and the jth column in the true aurora egg intensity image y, Q representing the length of the aurora egg intensity image, R representing the width of the aurora egg image, c ₃ Is a constant less than 0.00001;

(5a5) Calculating the similarity function between two images

Wherein a, b and e are each

Similarity function between two images

The weight of (1);

(5a6) Calculating a similarity objective function term S (G):

Images