CN117493786B - Remote sensing data reconstruction method combining countermeasure generation network and graph neural network - Google Patents

Remote sensing data reconstruction method combining countermeasure generation network and graph neural network Download PDF

Info

Publication number
CN117493786B
CN117493786B CN202311854062.4A CN202311854062A CN117493786B CN 117493786 B CN117493786 B CN 117493786B CN 202311854062 A CN202311854062 A CN 202311854062A CN 117493786 B CN117493786 B CN 117493786B
Authority
CN
China
Prior art keywords
time
space
matrix
remote sensing
coding
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311854062.4A
Other languages
Chinese (zh)
Other versions
CN117493786A (en
Inventor
周玮辰
隋艺
唐丹玲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southern Marine Science and Engineering Guangdong Laboratory Guangzhou
Original Assignee
Southern Marine Science and Engineering Guangdong Laboratory Guangzhou
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southern Marine Science and Engineering Guangdong Laboratory Guangzhou filed Critical Southern Marine Science and Engineering Guangdong Laboratory Guangzhou
Priority to CN202311854062.4A priority Critical patent/CN117493786B/en
Publication of CN117493786A publication Critical patent/CN117493786A/en
Application granted granted Critical
Publication of CN117493786B publication Critical patent/CN117493786B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/10—Pre-processing; Data cleansing
    • G06F18/15—Statistical pre-processing, e.g. techniques for normalisation or restoring missing data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/042—Knowledge-based neural networks; Logical representations of neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0475—Generative networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/094—Adversarial learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

The application provides a remote sensing data reconstruction method combining an countermeasure generation network and a graph neural network, which belongs to the field of artificial intelligent data reconstruction algorithms, wherein the countermeasure generation network calculation method comprises a feature extraction module and a countermeasure generation model; and the feature extraction module performs graph node feature calculation on the remote sensing data to acquire the space-time variation features of the remote sensing data. The feature extraction module performs feature extraction on the remote sensing data graph by using a time-space coding and space-time attention model, and reduces the long-time sequence long-error propagation effect by using a transducer attention module. The countermeasure generation model consists of a discriminator and a generator, and the acquired space-time variation characteristics of the remote sensing data and the remote sensing observation data are used as input information to generate a matrix which is close to real distribution to fill the missing value of the remote sensing data, so that the task of reconstructing the remote sensing data is completed.

Description

Remote sensing data reconstruction method combining countermeasure generation network and graph neural network
Technical Field
The invention relates to the field of artificial intelligent data reconstruction algorithms, in particular to a remote sensing data reconstruction method combining an countermeasure generation network and a graph neural network.
Background
The remote sensing data reconstruction method based on the countermeasure generation network and the graph neural network comprises a space-time feature coding module, a time feature extraction module, a space feature extraction module, a transformation (transducer) attention mechanism module and a countermeasure generation reconstruction module, and is used for reconstructing water color remote sensing data.
The remote sensing data can be influenced by various interference (such as cloud cover, bad weather, sensor faults and other unreliability factors) to cause data deletion, such as sea surface temperature and sea surface chlorophyll a concentration data, and the data deletion rate is high due to various interference, so that the satellite data quality is greatly reduced, and the application of the satellite data is hindered; a plurality of different data reconstruction algorithms such as a statistical-based method, a traditional method based on a machine learning algorithm, a deep learning method and the like have been proposed for the problem; however, the reconstruction algorithm based on statistics and machine learning cannot fully mine hidden information in the long-time sequence water color remote sensing data, has limitation on extraction of different time periods and spatial variation characteristics, and results in poor data complement performance; some deep learning algorithms have been used in satellite data interpolation and reconstruction and perform better than traditional algorithms, however there are still problems of difficult convergence of training, accumulation of long timing errors, etc.
In general, how to better and faster extract the space-time variation features in satellite data and reduce the error of reconstructed data is one of the important technical subjects in the art.
Disclosure of Invention
The invention aims to provide a remote sensing data reconstruction method combining an countermeasure generation network and a graph neural network, which is used for solving the problems in the prior art.
The remote sensing data reconstruction method combining the countermeasure generation network and the graph neural network provided by the embodiment of the invention comprises the following steps:
acquiring remote sensing data of a plurality of time stamps; the remote sensing data are data of a plurality of grid points after dividing the geographic position by longitude and latitude; the remote sensing data contains missing data;
taking grid points of the remote sensing data as nodes in the dynamic graph to obtain a graph structure;
based on the graph structure, the space information and the time information of the nodes are correspondingly encoded to obtain a space-time encoding matrix;
inputting the space-time coding matrix into a first space-time attention model, and performing feature coding to obtain space-time coding features;
based on the space-time coding feature, reducing long time sequence accumulated errors through a transformation attention model to obtain a transformation space-time coding feature;
decoding through a second space-time attention model based on the transformation space-time coding to obtain space-time decoding characteristics;
inputting the space-time decoding characteristics into a generator of an countermeasure generation model to generate missing remote sensing data, so as to obtain a reconstruction matrix;
and generating reconstructed remote sensing data according to the reconstruction matrix and the graph structure.
Optionally, the reducing long time sequence accumulated error based on the space-time coding feature through a transformation attention model to obtain a transformation space-time coding feature includes:
dividing the plurality of timestamps into a historical timestamp and a future timestamp; the historical time stamp is a time stamp which is an observed value in the graph structure; the future time stamp bitmap structure is a time stamp of a missing value;
dividing the space-time coding matrix into historical space-time coding matrices according to historical timestamps;
inputting the historical space-time coding matrix into a transformation network to generate space-time coding features of future time stamps, so as to obtain transformation space-time coding features;
the transformation space-time coding characteristic is specifically obtained by the following formula calculation mode:
wherein,representing future time stamps corresponding to space-time coding matrix to be reconstructed, and t represents calendarShi Shikong the historical timestamp corresponding to the encoding matrix; />And->For non-linear transformation in a module +.>Representing node->And->Time of day, i.e. history timestamp, l is input data of the change attention mechanism layer, +.>Representing node->And->Time, i.e. future time stamp, i-round transforms the output data of the attention mechanism layer, +.>Representing the inner product operation,/->，/>For the length of the space-time coding matrix +.>，/>For the number of multi-head operations, < >>Representation ofNode->And->Time of day, i.e., space-time coding of future time stamps; />Representing node->And->Time of day, i.e., historical timestamp; history timestamp->。
Optionally, the inputting the space-time coding matrix into the first space-time attention model, performing feature coding to obtain space-time coding features, includes:
the first time-space attention model comprises a spatial attention mechanism, a time attention mechanism and a gating mechanism;
based on the space-time coding matrix, a time feature extraction module is carried out through a time attention module, and time coding feature output is obtained;
based on the space-time coding matrix, a space feature extraction module is carried out through a space attention module, and space coding feature output is obtained; the spatial attention mechanism comprises a plurality of spatial attention layers;
and based on the time coding feature output and the space coding feature output, fusing the space and time features through a gating mechanism to obtain space-time coding features.
Optionally, the performing, by the time attention module, the time feature extraction module based on the space-time coding matrix to obtain a time coding feature output includes:
the time attention mechanism is a multi-head attention mechanism;
obtaining a time dependence coefficient according to the space-time coding matrix and the mask matrix of the corresponding graph structure;
obtaining a time attention normalization coefficient according to the time dependence coefficient;
obtaining a time dependent characteristic according to the time attention normalization coefficient;
the time dependent features are output as time encoded features.
Optionally, based on the graph structure, the space information and the time information of the node are correspondingly encoded to obtain a space-time encoding matrix, which includes:
based on the remote sensing data, constructing nodes into space vectors through random walk;
inputting the space vector into a first full connection layer for coding to obtain a space coding vector;
based on the remote sensing data, superposing the date and month as the value of the corresponding node of the time vector through single-hot coding to obtain the time vector;
inputting the time vector into a second full connection layer for coding to obtain a time coding vector;
and combining according to the space coding vector and the time coding vector to obtain a space-time coding matrix.
Optionally, the obtaining the graph structure by using the grid points of the remote sensing data as nodes in the dynamic graph includes:
taking the grid points as nodes of the dynamic graph to obtain a node set;
obtaining the distance between every two nodes of the dynamic graph through a Euclidean distance algorithm to obtain the edge distance;
if the edge distance is greater than or equal to the sparsity threshold, an edge is established between two nodes, and an edge set is obtained;
obtaining an adjacency matrix according to the edge set and the node set;
acquiring values corresponding to a plurality of time stamps of nodes in a node set as observation values to obtain an observation matrix;
obtaining a mask matrix; the value of the position with the missing value in the mask matrix is set to 0, and the value of the position with the observed value is set to 1;
the node set, the edge set, the adjacency matrix, the observation matrix and the mask matrix are used as a graph structure.
Optionally, the training method of the countermeasure generation model includes:
acquiring pure noise and a prompt matrix;
taking pure noise, a prompt matrix, an observation matrix in a graph structure and space-time decoding characteristics as generatorsPredicting missing data to obtain a prediction matrix;
taking an observation matrix, a prompt matrix and a prediction matrix in the graph structure as inputs of a discriminator to obtain the authenticity probability;
and solving the loss according to the authenticity probability, the observation matrix and the prediction matrix in the graph structure, and training the generator and the discriminator.
Optionally, the generating the reconstructed remote sensing data according to the reconstruction matrix and the graph structure includes:
acquiring an identity matrix; the identity matrix is a matrix with all values of 1;
obtaining reconstructed remote sensing data based on the identity matrix, the reconstruction matrix, the observation matrix in the graph structure and the mask matrix in the graph structure;
the reconstructed remote sensing data is specifically obtained by the following formula calculation mode:
wherein,for the observation matrix in the graph structure, +.>For the reconstruction matrix,/a>Is a unitary matrix->Is the Hadamard product of the two kinds of Chinese characters,and reconstructing the remote sensing data.
Optionally, the second spatiotemporal attention model is identical in structure to the first spatiotemporal attention model and different in parameters.
Compared with the prior art, the embodiment of the invention achieves the following beneficial effects:
the embodiment of the invention also provides a remote sensing data reconstruction method and a remote sensing data reconstruction system combining an countermeasure generation network and a graph neural network, wherein the method comprises the following steps: acquiring remote sensing data of a plurality of time stamps; the remote sensing data are data of a plurality of grid points after dividing the geographic position by longitude and latitude; the remote sensing data contains missing data; taking grid points of the remote sensing data as nodes in the dynamic graph to obtain a graph structure; based on the graph structure, the space information and the time information of the nodes are correspondingly encoded to obtain a space-time encoding matrix; inputting the space-time coding matrix into a first space-time attention model, and performing feature coding to obtain space-time coding features; based on the space-time coding feature, reducing long time sequence accumulated errors through a transformation attention model to obtain a transformation space-time coding feature; decoding through a second space-time attention model based on the transformation space-time coding to obtain space-time decoding characteristics; inputting the space-time decoding characteristics into a generator of an countermeasure generation model to generate missing remote sensing data, so as to obtain a reconstruction matrix; and generating reconstructed remote sensing data according to the reconstruction matrix and the graph structure.
The invention provides a remote sensing data reconstruction method based on an countermeasure generation network and a graph neural network, which is used for reducing satellite data reconstruction errors. In an embodiment according to the present disclosure, the method for reconstructing remote sensing data by combining an countermeasure generation network and a graph neural network includes a feature extraction module and a countermeasure generation model. The feature extraction module comprises a space-time coding, a space-time attention model, a full connection layer and a transformation (transducer) attention mechanism module. The challenge-generating model includes a generator and a discriminator. The pixels of the telemetry data that need to be reconstructed will be considered nodes in the neural network and the missing values in the data will be filled with 0 values. The spatio-temporal feature encoding module will encode the spatio-temporal features of each node. The multi-head space attention mechanism in each node learns the space-time characteristics of the node, outputs the space characteristic vector and the time vector characteristic of the node, and extracts the short-term space-time characteristics and the dynamic space characteristics of the remote sensing data. The node value and the space feature vector are overlapped with the time vector feature (Concate), and are input into a conversion (transducer) attention mechanism module, and the module is used for aggregating the space-time features of the data to be reconstructed, reducing the long time sequence accumulated error and outputting the reconstructed data features to an anti-generation reconstruction module. And filling the missing value in the remote sensing data with initial noise, inputting the initial noise into an countermeasure generation reconstruction module, and generating optimal remote sensing reconstruction data by the module through countermeasure training of a generator and a discriminator. According to the remote sensing reconstruction calculation method, the space-time characteristic coding module can effectively extract space-time information of remote sensing data, and the model convergence speed is increased. The technical effect of effectively improving the quality of the reconstructed data is achieved.
Drawings
FIG. 1 is a flow chart of a remote sensing data reconstruction method combining an countermeasure generation network and a graph neural network provided by an embodiment of the invention;
FIG. 2 is a schematic diagram of a remote sensing data space-time feature encoding module according to an embodiment of the present disclosure;
FIG. 3 is a schematic flow diagram of an algorithm according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of a spatiotemporal attention model in accordance with an embodiment of the disclosure;
fig. 5 is a schematic diagram of a spatiotemporal attention mechanism calculation method according to an embodiment of the disclosure.
Detailed Description
The present invention will be described in detail with reference to the accompanying drawings.
Examples
As shown in fig. 1, an embodiment of the present invention provides a method for reconstructing remote sensing data by combining an countermeasure generation network and a graph neural network, where the method includes:
s101: acquiring remote sensing data of a plurality of time stamps; the remote sensing data are data of a plurality of grid points after dividing the geographic position by longitude and latitude; the remote sensing data contains missing data;
s102: taking grid points of the remote sensing data as nodes in the dynamic graph to obtain a graph structure;
s103: based on the graph structure, the space information and the time information of the nodes are correspondingly encoded to obtain a space-time encoding matrix;
s104: and inputting the space-time coding matrix into a first space-time attention model, and performing feature coding to obtain space-time coding features.
S105: based on the space-time coding feature, reducing long time sequence accumulated errors through a transformation attention model to obtain a transformation space-time coding feature;
s106, decoding through a second space-time attention model based on the transformation space-time coding to obtain space-time decoding characteristics;
the schematic structure of the first space-time attention model and the second space-time attention model is shown in fig. 4.
S107: inputting the space-time decoding characteristics into a generator of an countermeasure generation model to generate missing remote sensing data, so as to obtain a reconstruction matrix;
s108: and generating reconstructed remote sensing data according to the reconstruction matrix and the graph structure.
The feature extraction module comprises a space-time coding module, a full connection layer, a space-time attention model and a transducer attention module.
Optionally, the reducing long time sequence accumulated error based on the space-time coding feature through a transformation attention model to obtain a transformation space-time coding feature includes:
dividing the plurality of timestamps into a historical timestamp and a future timestamp; the historical time stamp is a time stamp which is an observed value in the graph structure; the future time stamp bitmap structure is a time stamp of a missing value;
dividing the space-time coding matrix into historical space-time coding matrices according to historical timestamps;
wherein the future time stamp corresponding to the space-time coded dataHistorical time stampData.
And inputting the historical space-time coding matrix into a transformation network, and generating space-time coding features of future time stamps to obtain transformation space-time coding features.
In order to mitigate the effect of error propagation between different prediction time steps over a long time frame, a transform attention layer is added between the encoder and decoder. It models the direct relationship between each future time step and each historical time step to transform the encoded flow characteristics to generate a future representation as input to the decoder.
The transformation space-time coding characteristic is specifically obtained by the following formula calculation mode:
wherein,representing future time stamps corresponding to the space-time coding matrix to be reconstructed, and t represents historical time stamps corresponding to the historical space-time coding matrix; />And->For non-linear transformation in a module +.>Representing node->And->(historical timestamp) time l round of transforming input data of the attention mechanism layer, +.>Representing node->And->(future timestamp) time l round transforming the output data of the attention mechanism layer, +.>Representing the inner product operation,/->，/>For the length of the space-time coding matrix +.>，/>For the number of multi-head operations, < >>Representing node->And->Space-time coding of (future time stamp) instants; />Representing node->And->(historical timestamp) time of day; history timestamp->。
Optionally, the inputting the space-time coding matrix into the first space-time attention model, performing feature coding to obtain space-time coding features, includes:
the first time-space attention model comprises a spatial attention mechanism, a time attention mechanism and a gating mechanism
Wherein it is shown in fig. 4.
Based on the space-time coding matrix, a time feature extraction module is carried out through a time attention module, and time coding feature output is obtained; the temporal attention mechanism includes a plurality of temporal attention layers;
based on the space-time coding matrix, a space feature extraction module is carried out through a space attention module, and space coding feature output is obtained; the spatial attention mechanism comprises a plurality of spatial attention layers;
and based on the time coding feature output and the space coding feature output, fusing the space and time features through a gating mechanism to obtain space-time coding features.
Optionally, the performing, by the time attention module, the time feature extraction module based on the space-time coding matrix to obtain a time coding feature output includes:
the time attention mechanism is a multi-head attention mechanism;
obtaining a time dependence coefficient according to the space-time coding matrix and the mask matrix of the corresponding graph structure;
obtaining a time attention normalization coefficient according to the time dependence coefficient;
obtaining a time dependent characteristic according to the time attention normalization coefficient;
the time dependent features are output as time encoded features.
Wherein the temporal attention layer will extract each nodeIn the diagram structure +.>Is lazy in nature and dynamically assigns weights between different nodes. Node->Time-dependent characteristics->Represented by formula (4).
（4）
Wherein the method comprises the steps ofRepresenting Sigmoid activation function,/->Representation->Wheel node->At->Output value of time>Representing influence node->Is>Is at node->At->And->Time attention normalization coefficient of time of day, wherein +.>The sum of (2) is 1./>The expression of (2) is shown in formula (5).
（5）
The time-dependent coefficient is expressed by formula (6).
（6）
Representing a ReLU nonlinear activation function, +.>Is vector join operator (connect),>time of yesTraining weights of the inter-attention module. />And->Respectively show in->And->Mask value at time. Fig. 5 illustrates the calculation of the time-attention mechanism. />Time node->At->Spatial dependence characteristic of time of day->Can be represented by the expressionCalculated (obtained) by (I)>When (I)>Is->。
To stabilize the learning process, we extend the temporal attention layer to hs multiple head attention mechanisms, formulas see (7) (8) (9):
（7）
（8）
（9）
of the formula (I)Indicating the connection operation +_>Representing the inner product operation,/->，/>For the length of the space-time coding matrix +.>，/>The number of multi-head operations is set to 8 in the algorithm. />Representing a set of all time stamps of the telemetry data. />、/>Expressed as +.>A non-linear projection of the representation.
By the method, a time attention mechanism is designed to simulate the nonlinear relation between different time steps of the observation point in a self-adaptive mode.
Optionally, based on the graph structure, the space information and the time information of the node are correspondingly encoded to obtain a space-time encoding matrix, which includes:
based on the remote sensing data, constructing a space vector by the data of each node through random walk;
wherein each Node is represented by a static vector by a random walk (Node 2 Vec) method.
And inputting the space vector into a first full-connection layer for coding to obtain a space coding vector.
Wherein the space vector is subjected to full connection layer to obtain space coding, which is expressed as a vectorWherein。
Based on the remote sensing data, superposing the date and month as the value of the corresponding node of the time vector through single-hot coding to obtain the time vector;
wherein, the time code displays the date and month in a single-hot coding mode, and the month T and date of the remote sensing data are overlapped to obtain a vector。
Inputting the time vector into a second full connection layer for coding to obtain a time coding vector;
wherein the time vector is input into a second full connection layer to obtain a vector。
And combining according to the space coding vector and the time coding vector to obtain a space-time coding matrix.
Wherein,node of time->Space-time coding to obtain->. Space-time coding of all nodes in a subsequence +.>E contains the graph structure and time information.
By the method, the static space-time coding information of each node comprises the graph structure and time information of remote sensing data, and the static space-time coding information is used for a space-time attention model.
Optionally, the obtaining the graph structure by using the grid points of the remote sensing data as nodes in the dynamic graph includes:
taking the grid points as nodes of the dynamic graph to obtain a node set;
wherein a dynamic diagram is usedTo represent satellite data, wherein->Is->And each node corresponds to one grid point in the remote sensing data.
Obtaining the distance between every two nodes of the dynamic graph through a Euclidean distance algorithm to obtain the edge distance;
wherein,is a set of edges, and if the spatial distance of two nodes is less than the sparseness threshold, they will have one edge. Wherein the sparsity threshold is a manually given threshold. Node->(/>) And->(/>) Grid point positions of (2) are respectively->And->Representing, then node->And->Distance of->Can be represented by the following formula (1):
（1）
if the edge distance is greater than or equal to the sparsity threshold, an edge is established between two nodes, and an edge set is obtained;
wherein an adjacency matrix is usedThe topological relation of the dynamic graph G is shown. Wherein->Let =1 denote node +.>And node->With edges, otherwise->And=0 indicates that the two nodes have no edges, as indicated by the discrimination formula (2).
（2）
Wherein,is a scaling factor->Is a control adjacency matrix->Threshold of sparsity. If the scaling distance of two nodes is greater than or equal to the threshold +.>They will be linked together.
Obtaining an adjacency matrix according to the edge set and the node set;
in this embodiment, the dynamic diagramThe topology of (a) does not change over time, i.e. the model uses a fixed adjacency matrix +.>。
Acquiring values corresponding to a plurality of time stamps of nodes in a node set as observation values to obtain an observation matrix;
wherein at each moment in timeIn (I)>A graph structure representing the corresponding time instants, the graph structure also comprising missing data, e.g.>Middle node->And node->With observations and the remaining nodes are missing data. For the whole dynamic diagram structure->We use +.>Representing +.>The individual nodes are->The value of the individual moments, wherein->Representation->Satellite observations of all nodes at the moment, +.>Then indicate +.>The individual nodes are->Observations of time of day.
Obtaining a mask matrix; the value of the position with the missing value in the mask matrix is set to 0, and the value of the position with the observed value is set to 1;
wherein the mask matrixRepresentation->Whether the data of the corresponding position is a missing value +.>If you are->For the missing value->And 1 is expressed as an observed value.
The node set, the edge set, the adjacency matrix, the observation matrix and the mask matrix are used as a graph structure.
Wherein it is shown in fig. 2.
Optionally, the training method of the countermeasure generation model includes:
acquiring pure noise and a prompt matrix;
taking pure noise, a prompt matrix, an observation matrix in a graph structure and space-time decoding characteristics as generatorsPredicting missing data to obtain a prediction matrix;
taking an observation matrix, a prompt matrix and a prediction matrix in the graph structure as inputs of a discriminator to obtain the authenticity probability;
and solving the loss according to the authenticity probability, the observation matrix and the prediction matrix in the graph structure, and training the generator and the discriminator.
The countermeasure generation module consists of a generator and a discriminator, as shown in fig. 3. We will sense the true observations of the dataMask matrix->Pure noise->And space-time feature matrix->As generator->Generates a matrix approximating the distribution of real data. Remote sensing observation value->Prompt matrix->Sum generator generated value->As input to the discriminator, the probability of authenticity of the generated result is obtained>。
（21）
（22）
Prompt matrixIs a dependent mask matrix->Based on a certain proportion, the random variables of the mask matrix change the observed value recognized in the mask matrix into a generated value to achieve the function of a confusion discriminator. Typically this ratio is set to 0.5.
The algorithm improves the reconstruction accuracy of the model by reducing generator loss and arbiter loss. Wherein the generator loses functionBy reconstruction loss->And discriminatingIs->The composition is shown in the formula (23).
) （23）
Wherein the method comprises the steps ofIs a super parameter and is typically set to 100./>Is generated by a generator->An authenticity probability value in the arbiter. />Is generated by observation value and generator>Root mean square error between them, equation (24).
（24）
Wherein the method comprises the steps ofFor obtaining a set of nodes of observations in the remote sensing data, < >>For observations->The result generated by the generator. The purpose of the discriminator is to distinguish between real data and generated data, the discriminator loss function +.>The formula is (25).
（25）
Is the probability value of the authenticity of the observed value in the arbiter. The model optimizes the loss functions of the generator and the arbiter by optimizing the corresponding parameters using a back propagation method. Finally, the generated matrix is used>Filling the missing data to obtain final reconstruction data +.>。
Optionally, the generating the reconstructed remote sensing data according to the reconstruction matrix and the graph structure includes:
acquiring an identity matrix; the identity matrix is a matrix with all values of 1;
obtaining reconstructed remote sensing data based on the identity matrix, the reconstruction matrix, the observation matrix in the graph structure and the mask matrix in the graph structure;
the reconstructed remote sensing data is specifically obtained by the following formula calculation mode:
wherein,for the observation matrix in the graph structure, +.>For the reconstruction matrix,/a>Is a unitary matrix->Is the Hadamard product of the two kinds of Chinese characters,and reconstructing the remote sensing data.
Wherein the model will input incomplete satellite dataAnd a mask matrix corresponding thereto>Obtain and->Close reconstruction matrix->. Reconstruction matrix->The missing value in the satellite data is filled up, and the filled up data is ensured to be close to the real data.
Optionally, the second spatiotemporal attention model is identical in structure to the first spatiotemporal attention model and different in parameters.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that some or all of the functions of some or all of the components in an apparatus according to embodiments of the present invention may be implemented in practice using a microprocessor or Digital Signal Processor (DSP). The present invention can also be implemented as an apparatus or device program (e.g., a computer program and a computer program product) for performing a portion or all of the methods described herein. Such a program embodying the present invention may be stored on a computer readable medium, or may have the form of one or more signals. Such signals may be downloaded from an internet website, provided on a carrier signal, or provided in any other form.

Claims (8)

1. A method for reconstructing remote sensing data by combining an countermeasure generation network and a graph neural network, comprising the steps of:
acquiring remote sensing data of a plurality of time stamps; the remote sensing data are data of a plurality of grid points after dividing the geographic position by longitude and latitude; the remote sensing data contains missing data;
taking grid points of the remote sensing data as nodes in the dynamic graph to obtain a graph structure;
based on the graph structure, the space information and the time information of the nodes are correspondingly encoded to obtain a space-time encoding matrix;
inputting the space-time coding matrix into a first space-time attention model, and performing feature coding to obtain space-time coding features;
based on the space-time coding feature, reducing long time sequence accumulated errors through a transformation attention model to obtain a transformation space-time coding feature;
decoding through a second space-time attention model based on the transformation space-time coding to obtain space-time decoding characteristics;
inputting the space-time decoding characteristics into a generator of an countermeasure generation model to generate missing remote sensing data, so as to obtain a reconstruction matrix;
generating reconstructed remote sensing data according to the reconstruction matrix and the graph structure;
the method for reducing long time sequence accumulated errors based on the space-time coding features through the transformation attention model to obtain the transformation space-time coding features comprises the following steps:
dividing the plurality of timestamps into a historical timestamp and a future timestamp; the historical time stamp is a time stamp which is an observed value in the graph structure; the future time stamp bitmap structure is a time stamp of a missing value;
dividing the space-time coding matrix into historical space-time coding matrices according to historical timestamps;
inputting the historical space-time coding matrix into a transformation network to generate space-time coding features of future time stamps, so as to obtain transformation space-time coding features;
the transformation space-time coding characteristic is specifically obtained by the following formula calculation mode:
wherein,representing future time stamps corresponding to the space-time coding matrix to be reconstructed, and t represents historical time stamps corresponding to the historical space-time coding matrix; />And->For non-linear transformation in a module +.>Representing node->And->At the moment of time of day,i.e. history timestamp, input data of the attention mechanism layer is transformed by the round i,/o>Representing node->And->Time, i.e. future time stamp, i-round transforms the output data of the attention mechanism layer, +.>Representing the inner product operation,/->，/>For the length of the space-time coding matrix +.>，/>For the number of multi-head operations, < >>Representing node->And->Time of day, i.e., space-time coding of future time stamps; />Representing node->And->Time of day, i.e., historical timestamp; history timestamp->。
2. The method for reconstructing remote sensing data combined by an countermeasure generation network and a graphic neural network according to claim 1, wherein the step of inputting a space-time coding matrix into a first space-time attention model to perform feature coding to obtain space-time coding features comprises the steps of:
the first time-space attention model comprises a spatial attention mechanism, a time attention mechanism and a gating mechanism;
based on the space-time coding matrix, a time feature extraction module is carried out through a time attention module, and time coding feature output is obtained;
based on the space-time coding matrix, a space feature extraction module is carried out through a space attention module, and space coding feature output is obtained; the spatial attention mechanism comprises a plurality of spatial attention layers;
and based on the time coding feature output and the space coding feature output, fusing the space and time features through a gating mechanism to obtain space-time coding features.
3. The method for reconstructing remote sensing data by combining an countermeasure generation network and a graph neural network according to claim 2, wherein the performing a temporal feature extraction module based on the space-time coding matrix through a temporal attention module to obtain a temporal coding feature output comprises:
the time attention mechanism is a multi-head attention mechanism;
obtaining a time dependence coefficient according to the space-time coding matrix and the mask matrix of the corresponding graph structure;
obtaining a time attention normalization coefficient according to the time dependence coefficient;
obtaining a time dependent characteristic according to the time attention normalization coefficient;
the time dependent features are output as time encoded features.
4. The method for reconstructing remote sensing data by combining an countermeasure generation network and a graph neural network according to claim 1, wherein the encoding the spatial information of the nodes and the time information correspondingly based on the graph structure to obtain a space-time encoding matrix comprises:
based on the remote sensing data, constructing nodes into space vectors through random walk;
inputting the space vector into a first full connection layer for coding to obtain a space coding vector;
based on the remote sensing data, superposing the date and month as the value of the corresponding node of the time vector through single-hot coding to obtain the time vector;
inputting the time vector into a second full connection layer for coding to obtain a time coding vector;
and combining according to the space coding vector and the time coding vector to obtain a space-time coding matrix.
5. The method for reconstructing remote sensing data by combining an countermeasure generation network and a graph neural network according to claim 1, wherein the step of using grid points of the remote sensing data as nodes in a dynamic graph to obtain a graph structure includes:
taking the grid points as nodes of the dynamic graph to obtain a node set;
obtaining the distance between every two nodes of the dynamic graph through a Euclidean distance algorithm to obtain the edge distance;
if the edge distance is greater than or equal to the sparsity threshold, an edge is established between two nodes, and an edge set is obtained;
obtaining an adjacency matrix according to the edge set and the node set;
acquiring values corresponding to a plurality of time stamps of nodes in a node set as observation values to obtain an observation matrix;
obtaining a mask matrix; the value of the position with the missing value in the mask matrix is set to 0, and the value of the position with the observed value is set to 1;
the node set, the edge set, the adjacency matrix, the observation matrix and the mask matrix are used as a graph structure.
6. A method of reconstructing telemetry data in combination with an countermeasure generation network and a graph neural network as claimed in claim 1, wherein the method of training the countermeasure generation model comprises:
acquiring pure noise and a prompt matrix;
taking pure noise, a prompt matrix, an observation matrix in a graph structure and space-time decoding characteristics as generatorsPredicting missing data to obtain a prediction matrix;
taking an observation matrix, a prompt matrix and a prediction matrix in the graph structure as inputs of a discriminator to obtain the authenticity probability;
and solving the loss according to the authenticity probability, the observation matrix and the prediction matrix in the graph structure, and training the generator and the discriminator.
7. The method for reconstructing remote sensing data combined with an countermeasure generation network and a graph neural network according to claim 1, wherein the generating the reconstructed remote sensing data according to the reconstruction matrix and the graph structure comprises:
acquiring an identity matrix; the identity matrix is a matrix with all values of 1;
obtaining reconstructed remote sensing data based on the identity matrix, the reconstruction matrix, the observation matrix in the graph structure and the mask matrix in the graph structure;
the reconstructed remote sensing data is specifically obtained by the following formula calculation mode:
wherein,for the observation matrix in the graph structure, +.>For the reconstruction matrix,/a>Is a unitary matrix->Is Hadamard product, is->And reconstructing the remote sensing data.
8. A method of reconstructing telemetry data in combination with an countermeasure generation network and a graph neural network as claimed in claim 1, wherein the second spatiotemporal attention model is structurally identical to the first spatiotemporal attention model and has different parameters.
CN202311854062.4A 2023-12-29 2023-12-29 Remote sensing data reconstruction method combining countermeasure generation network and graph neural network Active CN117493786B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311854062.4A CN117493786B (en) 2023-12-29 2023-12-29 Remote sensing data reconstruction method combining countermeasure generation network and graph neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311854062.4A CN117493786B (en) 2023-12-29 2023-12-29 Remote sensing data reconstruction method combining countermeasure generation network and graph neural network

Publications (2)

Publication Number Publication Date
CN117493786A CN117493786A (en) 2024-02-02
CN117493786B true CN117493786B (en) 2024-04-09

Family

ID=89669417

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311854062.4A Active CN117493786B (en) 2023-12-29 2023-12-29 Remote sensing data reconstruction method combining countermeasure generation network and graph neural network

Country Status (1)

Country Link
CN (1) CN117493786B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115470201A (en) * 2022-08-30 2022-12-13 同济大学 Intelligent ocean remote sensing missing data completion method based on graph attention network
CN115659797A (en) * 2022-10-24 2023-01-31 大连理工大学 Self-learning method for generating anti-multi-head attention neural network aiming at aeroengine data reconstruction
CN116910463A (en) * 2023-08-01 2023-10-20 西安长天长软件股份有限公司 Filling method for air pollution monitoring data missing value
WO2023217163A1 (en) * 2022-05-11 2023-11-16 华能澜沧江水电股份有限公司 Dam defect time-sequence image description method based on local self-attention mechanism

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115033662A (en) * 2022-06-10 2022-09-09 华中科技大学 Distributed attention time sequence knowledge graph reasoning method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023217163A1 (en) * 2022-05-11 2023-11-16 华能澜沧江水电股份有限公司 Dam defect time-sequence image description method based on local self-attention mechanism
CN115470201A (en) * 2022-08-30 2022-12-13 同济大学 Intelligent ocean remote sensing missing data completion method based on graph attention network
CN115659797A (en) * 2022-10-24 2023-01-31 大连理工大学 Self-learning method for generating anti-multi-head attention neural network aiming at aeroengine data reconstruction
CN116910463A (en) * 2023-08-01 2023-10-20 西安长天长软件股份有限公司 Filling method for air pollution monitoring data missing value

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
mf-dfnet：a deep learning method for pixel-wise classification of very high-resolution remote sensing images;Shichao Zhang;《international journal of remote sensing》;20220127;第330-348页 *

Also Published As

Publication number Publication date
CN117493786A (en) 2024-02-02

Similar Documents

Publication Publication Date Title
CN101795344B (en) Digital hologram compression method and system, decoding method and system, and transmission method and system
Liu et al. Climate downscaling using YNet: A deep convolutional network with skip connections and fusion
CN109636721B (en) Video super-resolution method based on countermeasure learning and attention mechanism
Li et al. Consecutively missing seismic data interpolation based on coordinate attention unet
CN110728219A (en) 3D face generation method based on multi-column multi-scale graph convolution neural network
CN111127325B (en) Satellite video super-resolution reconstruction method and system based on cyclic neural network
CN112766568B (en) Ultra-short-term distributed photovoltaic power prediction method based on satellite cloud picture
CN111833277A (en) Marine image defogging method with non-paired multi-scale hybrid coding and decoding structure
CN110084181B (en) Remote sensing image ship target detection method based on sparse MobileNet V2 network
CN113469884A (en) Video super-resolution method, system, equipment and storage medium based on data simulation
CN113793261A (en) Spectrum reconstruction method based on 3D attention mechanism full-channel fusion network
CN114581341A (en) Image style migration method and system based on deep learning
CN116151459A (en) Power grid flood prevention risk probability prediction method and system based on improved Transformer
CN117493786B (en) Remote sensing data reconstruction method combining countermeasure generation network and graph neural network
CN114494387A (en) Data set network generation model and fog map generation method
CN116306780B (en) Dynamic graph link generation method
CN113706670A (en) Method and device for generating dynamic three-dimensional human body mesh model sequence
CN113298131A (en) Attention mechanism-based time sequence data missing value interpolation method
CN117093830A (en) User load data restoration method considering local and global
CN115439849B (en) Instrument digital identification method and system based on dynamic multi-strategy GAN network
CN116402719A (en) Human blind face image recovery system and method based on potential diffusion model
CN115840261A (en) Typhoon precipitation short-term prediction model optimization and prediction method
CN113537573A (en) Wind power operation trend prediction method based on dual space-time feature extraction
CN110381313B (en) Video compression sensing reconstruction method based on LSTM network and image group quality blind evaluation
CN113761395A (en) Trajectory generation model training method, trajectory generation method and apparatus

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant