CN116612396A - Ocean surface temperature sequence prediction method based on space-time double-flow non-stationary sensing - Google Patents

Ocean surface temperature sequence prediction method based on space-time double-flow non-stationary sensing Download PDF

Info

Publication number
CN116612396A
CN116612396A CN202310607658.8A CN202310607658A CN116612396A CN 116612396 A CN116612396 A CN 116612396A CN 202310607658 A CN202310607658 A CN 202310607658A CN 116612396 A CN116612396 A CN 116612396A
Authority
CN
China
Prior art keywords
space
time
stationary
feature
surface temperature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310607658.8A
Other languages
Chinese (zh)
Inventor
聂婕
杜家豪
温琦
叶敏
左子杰
宋宁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ocean University of China
Original Assignee
Ocean University of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ocean University of China filed Critical Ocean University of China
Priority to CN202310607658.8A priority Critical patent/CN116612396A/en
Publication of CN116612396A publication Critical patent/CN116612396A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

The invention belongs to the technical field of image processing, and discloses a marine surface temperature sequence prediction method based on space-time double flow non-stationary perception.

Description

Ocean surface temperature sequence prediction method based on space-time double-flow non-stationary sensing
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a marine surface temperature sequence prediction method based on space-time double-flow non-stationary perception.
Background
Ocean surface temperature (SST) prediction methods can be divided into two major categories, numerical methods, which are mathematical models built by a series of complex physical, chemical and biological parameters to predict sea temperature changes, and data-driven methods, which generally require significant computational costs due to excessive parameters. The data driving method aims at solving the SST prediction problem by taking data as a center, and builds an SST prediction model by learning the change rule of SST from historical SST data. The data driven method does not require much knowledge of the ocean and atmosphere domain and can predict sea surface temperature with higher resolution on a smaller scale than the numerical method.
The main data driven methods include traditional statistical methods and machine learning methods. With the development of deep learning, the existing SST prediction method mostly adopts a space-time prediction model based on a cyclic neural network, captures non-stationary characteristics through simple state transition, establishes a simple spatial dependency while establishing a time dependency, however, the existing method has the following problems: first, focusing on building correlations in time series, full mining of ocean surface temperature space information is not achieved. The existing ocean surface temperature prediction model structure mostly adopts a circulating structure, and a method for performing simple feature extraction on space information and then performing time feature aggregation. However, the spatio-temporal data has spatial autocorrelation, and the potential spatial correlation information cannot be captured sufficiently by simple feature extraction, so that the spatial information needs to be further mined deeply according to features of different scales. Second, marine physics space-time shows complex non-stationarity in both time and space, such as el nino and southern billow phenomena, which cause abnormal increases in ocean surface temperature. The existing method cannot fully model the non-stationarity in the ocean surface temperature in time and space at the same time, so that uncertainty in space-time dynamics cannot be inferred.
Disclosure of Invention
Aiming at the defects existing in the prior art, the invention provides a marine surface temperature sequence prediction method based on space-time double-flow non-stationary perception, a prediction network based on double flow is constructed, and firstly, the time non-stationary characteristics in the space-time sequence are captured through a diagonal cyclic neural network stacked by MIM modules; then constructing a parallel time non-stationary feature modeling sub-network and a space non-stationary feature modeling sub-network, and simultaneously mining non-stationary features in time and space; and finally, the time comprehensive characteristics and the space comprehensive characteristics obtained from the two subnetworks are adaptively learned by a space-time fusion module based on stacked cross attention, so that the space-time correlation of the sea surface temperature is effectively excavated by the weight of the time dependence and the space dependence on the sea temperature field, a final SST prediction result is obtained, and the accuracy of SST prediction is improved.
In order to solve the technical problems, the invention adopts the following technical scheme:
a marine surface temperature sequence prediction method based on space-time double-flow non-stationary perception comprises the following steps: s1, acquiring a satellite remote sensing image of ocean surface temperature;
s2, constructing and training a marine surface temperature sequence prediction network based on space-time double-flow non-stationary perception, wherein the marine surface temperature sequence prediction network comprises a space non-stationary characteristic modeling sub-network, a time non-stationary characteristic modeling sub-network and a stacking cross attention space-time fusion module;
the time non-stationary characteristic modeling network consists of ST-LSTM modules and MIM modules, and comprises multiple layers of network structures with the same structure, wherein each layer of network structure comprises a plurality of layers of network structuresThe network structure comprises an ST-LSTM module and a plurality of MIM modules, wherein the modules of the upper layer and the modules of the lower layer are stacked and interconnected through a saw-tooth path, the ST-LSTM module or the MIM module at the corresponding position of each layer is connected through a vertical path, and an input ocean surface temperature satellite remote sensing image X is input 1 ,X 2 ,...,X t The method comprises the steps of obtaining the SST time sequence characteristics of the ocean surface temperature through ST-LSTM of each layer, transmitting the output hidden states of two vertically adjacent modules to the next layer of modules through diagonal paths, capturing non-stationary information in the SST time sequence characteristics of the ocean surface temperature in MIM modules by a differential-based method, capturing the non-stationary information through stacked diagonal structures, modeling the time sequence non-stationary characteristics of the SST of the ocean surface temperature, and finally outputting y of the last MIM module of each layer 1 ,y 2 ,...,y t Polymerizing to obtain SST time comprehensive characteristics beta;
the space non-stationary characteristic modeling sub-network is composed of a space non-stationary characteristic extraction module and a space characteristic fusion module, and specifically comprises the following components:
in a space non-stationary feature extraction module, downsampling a marine surface temperature satellite remote sensing image with the size of M to form a feature map with three scales of M, (M) M/4, (M) M/8, performing differential operation on the satellite remote sensing images with the sizes of (M) M/4 and (M) M/8 through convolution operation, and capturing a small-scale non-stationary feature n; similarly, performing a differential operation on the satellite remote sensing images of M and (M) M/4 through convolution operation, and capturing a large-scale non-stationary characteristic N;
in a space feature fusion module, aggregating an original feature map S, a small-scale non-stable feature N and a large-scale non-stable feature N of M, respectively performing sigmoid function processing and tanh function processing on the obtained space aggregation features, reinforcing the non-obvious non-stable features in space by the sigmoid function to prevent loss, reserving complete space comprehensive features, and multiplying corresponding elements of two function processing results to obtain SST space comprehensive features alpha;
the SST time comprehensive feature beta obtained from the time non-stationary feature modeling sub-network is fused with the SST space comprehensive feature alpha in the space non-stationary feature modeling sub-network through a stacking-based cross attention time-space fusion module, and the SST space comprehensive feature alpha and the SST time comprehensive feature beta self-adaptively learn the weight of the SST space comprehensive feature alpha and the SST time comprehensive feature beta to the ocean surface temperature field in the stacking-based cross attention time-space fusion module, and fuse time and space information;
and S3, inputting the ocean surface temperature satellite remote sensing image to be processed into a trained ocean surface temperature sequence prediction network based on space-time double flow non-stationary sensing, and outputting an ocean surface temperature prediction result.
Further, the calculation flow of the SST spatial synthesis feature α is as follows:
o＝tanh(W os *S+W oN *N+W on *n)
g＝σ(W gs *S+W gN *N+W gn *n)
wherein W is os 、W oN 、W on 、W gs 、W gN 、W gn Representing a convolution parameter; * Representing a convolution operation; sigma is a sigmoid function, an activation function applied to the input weighted sum; tanh represents a tanh activation function; n represents a small-scale non-stationary feature; n represents a large scale non-stationary feature; s represents an original feature map;representing a Hadamard product; o represents the result of the tan h function after the original feature map S, the small-scale non-stationary feature N and the large-scale non-stationary feature N are aggregated; g represents the result of the original feature map S, the small-scale non-stationary feature N and the large-scale non-stationary feature N after aggregation by a sigmoid function.
Further, in the space-time fusion module based on stacked cross attention, specifically:
firstly, SST space integrated feature alpha in (M, M) dimension is flattened and mapped through an attention coding mechanism to be respectively used as query Q in (M multiplied by M, 1) dimension α Sum value V α Matrix, SST time synthesisThe feature β likewise flattens the mapping by the attention encoding mechanism as keys K in the (m×m, 1) dimension, respectively β Sum value V β Matrix:
Q α ＝W q *α,V α ＝W v *α
K β ＝W k *β,V β ＝W v *β
wherein W is q ，W k ，W v Represents a convolution parameter, represents a convolution operation, Q α Representing a spatial query matrix; v (V) α Representing a matrix of spatial values, K β Representing a time key matrix, V β Representing a time value matrix;
for Q α And K β And performing matrix multiplication and modeling through each grid point of a softmax function to obtain a correlation score e of the SST time comprehensive feature beta and the SST space comprehensive feature alpha, wherein the dimension is (M multiplied by M ) and the formula is as follows:
where softmax represents the normalized exponential function,representing a matrix multiplication;
then, the space value matrix V α Matrix multiplication with e is performed on a space value matrix V α Modeling each grid point on the model and remodelling the result through a reshape function to obtain a spatial attention characteristic Z through the (M, M) dimension α Similarly, the time value matrix V β Matrix multiplication with e is performed on a time value matrix V β Modeling each grid point on the model and remodelling the result through a reshape function to the (M, M) dimension to obtain a time attention feature Z β ：
Where reshape denotes a function that converts tensor dimensions,represents matrix multiplication, e represents correlation scores on temporal and spatial features, V α Representing a matrix of spatial values, V β Representing a matrix of time values, Z α Representing spatial attention features, Z β Representing a temporal attention feature;
finally, a final ocean surface temperature satellite remote sensing image prediction result is obtained through the aggregation characteristics and the multi-layer perceptron, and is expressed as follows:
Z＝MLP(W z [Z α ,Z β ])
wherein MLP represents a multi-layer perceptron, W Z Representing the convolution parameters, Z α Representing spatial attention features, Z β And Z represents the prediction result of the marine surface temperature satellite remote sensing image.
Compared with the prior art, the invention has the advantages that:
(1) A spatially non-stationary feature modeling sub-network is designed to model spatially non-stationary features in SST data. Firstly, down-sampling an input SST space-time sequence to construct space features of three scales of small, medium and large; then differential modeling is carried out on the multi-scale space features, so that full excavation on small-scale and large-scale non-stable features and large-scale stable feature changes is realized; and finally, designing a spatial feature fusion module based on a gating mechanism, and dynamically carrying out association modeling of non-stationary features and stationary feature changes.
(2) The invention can effectively extract the non-stationary characteristics in time and space simultaneously. The invention constructs a prediction network based on double flow, firstly, capturing time non-stationary characteristics in a time-space sequence through a diagonal cyclic neural network stacked by MIM modules; then constructing a parallel time non-stationary feature modeling sub-network and a space non-stationary feature modeling sub-network, and simultaneously mining non-stationary features in time and space; and finally, the time comprehensive features and the space comprehensive features obtained from the two subnetworks are adaptively learned by a space-time fusion module based on stacked cross attention, the weight of time dependence and space dependence on sea temperature fields is effectively excavated by the space-time fusion module based on stacked cross attention, so that a final SST prediction result is obtained, and the accuracy of SST prediction is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a diagram of a network architecture of the present invention;
FIG. 2 is a schematic diagram of a stacked cross-attention based spatiotemporal fusion module of the present invention.
Detailed Description
The invention will be further described with reference to the accompanying drawings and specific examples.
The embodiment provides a marine surface temperature sequence prediction method based on space-time double-flow non-stationary sensing, which comprises the following steps:
s1, acquiring a satellite remote sensing image of ocean surface temperature;
and S2, constructing and training a marine surface temperature sequence prediction network based on space-time double-flow non-stationary sensing, and inputting a marine surface temperature satellite remote sensing image to be processed into the trained marine surface temperature sequence prediction network based on space-time double-flow non-stationary sensing, and outputting a marine surface temperature prediction result.
The following describes the structure of the marine surface temperature sequence prediction network based on space-time double-flow non-stationary sensing constructed by the embodiment with reference to fig. 1, wherein the marine surface temperature sequence prediction network comprises a space non-stationary characteristic modeling sub-network, a time non-stationary characteristic modeling sub-network and a stacked cross-attention space-time fusion module, and the composition, the function and the data processing process of each module are respectively described.
The method comprises the steps that a time non-stationary characteristic modeling sub-network is built, a circulating neural network stacked by MIM modules is used for capturing the time non-stationary characteristic, the time non-stationary characteristic is formed by ST-LSTM modules and MIM modules, the time non-stationary characteristic modeling sub-network comprises a multi-layer network structure with the same structure, each layer of network structure comprises an ST-LSTM module and a plurality of MIM modules, the modules of the upper layer are stacked and interconnected with the modules of the lower layer through sawtooth paths, the ST-LSTM modules or the MIM modules of the corresponding positions of each layer are connected through vertical paths, and the MIM modules are connected and interconnected through diagonal paths and used for modeling differential information in space-time prediction; by stacking multiple MIM modules, non-stationarity features can potentially be captured from spatio-temporal dynamics, gradually smoothing the spatio-temporal process, making predicted future SST time integration features more accurate.
Input ocean surface temperature satellite remote sensing image X 1 ,X 2 ,...,X t The SST time sequence characteristics of the ocean surface temperature are obtained through ST-LSTM of each layer respectively, but the SST has non-stationary information in natural time and space, so that the output hidden states of two vertically adjacent modules are transmitted to the next layer of modules through diagonal paths, the non-stationary information in the SST time sequence characteristics of the ocean surface temperature is captured by differential operation in the MIM modules, the non-stationary information is captured by stacked diagonal structures, the time sequence non-stationary characteristics of the SST of the ocean surface temperature are modeled, and finally the output y of the last MIM module of each layer is obtained 1 ,y 2 ,...,y t And polymerizing to obtain SST time comprehensive characteristics beta.
The space non-stationary characteristic modeling sub-network consists of a space non-stationary characteristic extraction module and a space characteristic fusion module, and specifically comprises the following components:
firstly, in a space non-stationary feature extraction module, due to the characteristic that the physical process change is slow in the marine physical field, a marine surface temperature satellite remote sensing image with the size of M is firstly downsampled into feature images with three scales of M, M/4, M/8, and the larger the scale of the remote sensing image is, the more abundant and obvious the features are contained. Performing differential operation on satellite remote sensing images of (M×M)/4 and (M×M)/8 through convolution operation to capture small-scale non-stationary features n; similarly, a difference operation is carried out on the satellite remote sensing images of M, M and (M, M)/4 through convolution operation, and a large-scale non-stationary characteristic N is captured. Because any non-stationary process can be decomposed into a plurality of deterministic time-varying polynomials and a zero-mean random value, the order of the time-varying polynomials can be reduced by properly applying differential operation, and the time-varying polynomials gradually tend to be stationary, so that deterministic complex trend information becomes predictable, and the problem that the non-stationary features of the space cannot be captured is solved.
And then, in a spatial feature fusion module, aggregating the original feature map S, the small-scale non-stable feature N and the large-scale non-stable feature N of M, respectively performing sigmoid function processing and tanh function processing on the obtained spatial aggregation features, reinforcing the non-stable features which are not obvious in the space by the sigmoid function to prevent loss, reserving the complete spatial comprehensive features, and finally multiplying corresponding elements of the two function processing results to obtain the SST spatial comprehensive feature alpha. The calculation flow of the SST spatial synthesis feature α is as follows:
o＝tanh(W os *S+W oN *N+W on *n)
g＝σ(W gs *S+W gN *N+W gn *n)
wherein W is os 、W oN 、W on 、W gs 、W gN 、W gn Representing a convolution parameter; * Representing a convolution operation; sigma is a sigmoid function, an activation function applied to the input weighted sum; tanh represents a tanh activation function; n represents a small-scale non-stationary feature; n represents a large scale non-stationary feature; s represents an original feature map;representing a Hadamard product; o represents the result of the tan h function after the original feature map S, the small-scale non-stationary feature N and the large-scale non-stationary feature N are aggregatedThe method comprises the steps of carrying out a first treatment on the surface of the g represents the result of the original feature map S, the small-scale non-stationary feature N and the large-scale non-stationary feature N after aggregation by a sigmoid function.
And finally, fusing the SST time comprehensive characteristic beta obtained from the time non-stationary characteristic modeling sub-network with the SST space comprehensive characteristic alpha in the space non-stationary characteristic modeling sub-network through a stacking cross attention time-space fusion module, and adaptively learning the weight of the SST space comprehensive characteristic alpha and the SST time comprehensive characteristic beta on the ocean surface temperature field in the stacking cross attention time-space fusion module, fusing important information in time and space, and filtering unimportant information, so that the prediction result is more accurate and reliable.
In conjunction with the illustration of fig. 2, within the stacked cross-attention based spatiotemporal fusion module, in particular:
firstly, SST space integrated feature alpha in (M, M) dimension is flattened and mapped through an attention coding mechanism to be respectively used as query Q in (M multiplied by M, 1) dimension α Sum value V α Matrix, SST time complex feature β likewise flattens the mapping by means of an attention coding mechanism as keys K in the (M1) dimension, respectively β Sum value V β Matrix:
Q α ＝W q *α,V α ＝W v *α
K β ＝W k *β,V β ＝W v *β
wherein W is q ，W k ，W v Represents a convolution parameter, represents a convolution operation, Q α Representing a spatial query matrix; v (V) α Representing a matrix of spatial values, K β Representing a time key matrix, V β Representing a time value matrix;
for Q α And K β And performing matrix multiplication and modeling through each grid point of a softmax function to obtain a correlation score e of the SST time comprehensive feature beta and the SST space comprehensive feature alpha, wherein the dimension is (M multiplied by M ) and the formula is as follows:
where softmax represents the normalized exponential function,representing a matrix multiplication;
then, the space value matrix V α Matrix multiplication with e is performed on a space value matrix V α Modeling each grid point on the model and remodelling the result through a reshape function to obtain a spatial attention characteristic Z through the (M, M) dimension α Similarly, the time value matrix V β Matrix multiplication with e is performed on a time value matrix V β Modeling each grid point on the model and remodelling the result through a reshape function to the (M, M) dimension to obtain a time attention feature Z β ：
Where reshape denotes a function that converts tensor dimensions,represents matrix multiplication, e represents correlation scores on temporal and spatial features, V α Representing a matrix of spatial values, V β Representing a matrix of time values, Z α Representing spatial attention features, Z β Representing a temporal attention feature;
finally, a final ocean surface temperature satellite remote sensing image prediction result is obtained through the aggregation characteristics and the multi-layer perceptron, and is expressed as follows:
Z＝MLP(W z [Z α ,Z β ])
wherein MLP represents a multi-layer perceptron, W Z Representing the convolution parameters, Z α Representing spatial attention features, Z β And Z represents the prediction result of the marine surface temperature satellite remote sensing image.
The network model models the uncertain trend in the SST dynamic space-time process with non-stationary information, so that the model is better fitted with the SST natural space-time process, and the prediction accuracy is improved.
Wherein the training of the network model is not a design gist of the present invention, and is not described in detail herein.
In summary, the invention (1) designs a spatially non-stationary feature modeling sub-network to model spatially non-stationary features in SST data. Firstly, down-sampling an input SST space-time sequence to construct space features of three scales of small, medium and large; then differential modeling is carried out on the multi-scale space features, so that full excavation on small-scale and large-scale non-stable features and large-scale stable feature changes is realized; and finally, designing a spatial feature fusion module based on a gating mechanism, and dynamically carrying out association modeling of non-stationary features and stationary feature changes. (2) constructing a predictive network based on double flow. First, capturing temporal non-stationary features in a spatio-temporal sequence by a diagonal recurrent neural network stacked by MIM modules; then constructing a parallel time non-stationary feature modeling sub-network and a space non-stationary feature modeling sub-network, and simultaneously mining non-stationary features in time and space; and finally, the time comprehensive features and the space comprehensive features obtained from the two sub-networks are adaptively learned by a space-time fusion module based on stacked cross attention, so that the weight of time dependence and space dependence on a sea temperature field is obtained, a final SST prediction result is obtained, and the accuracy of SST prediction is improved.
It should be understood that the above description is not intended to limit the invention to the particular embodiments disclosed, but to limit the invention to the particular embodiments disclosed, and that various changes, modifications, additions and substitutions can be made by those skilled in the art without departing from the spirit and scope of the invention.

Claims (3)

1. The marine surface temperature sequence prediction method based on space-time double-flow non-stationary perception is characterized by comprising the following steps of:
s1, acquiring a satellite remote sensing image of ocean surface temperature;
s2, constructing and training a marine surface temperature sequence prediction network based on space-time double-flow non-stationary perception, wherein the marine surface temperature sequence prediction network comprises a space non-stationary characteristic modeling sub-network, a time non-stationary characteristic modeling sub-network and a stacking cross attention space-time fusion module;
the time non-stationary characteristic modeling network consists of ST-LSTM modules and MIM modules, and comprises a multi-layer network structure with the same structure, wherein each layer of network structure comprises one ST-LSTM module and a plurality of MIM modules, the modules of the upper layer and the modules of the lower layer are stacked and interconnected through a saw-tooth path, the ST-LSTM modules or the MIM modules of the corresponding positions of each layer are connected through a vertical path, and an input ocean surface temperature satellite remote sensing image X is input 1 ,X 2 ,...,X t The method comprises the steps of obtaining the SST time sequence characteristics of the ocean surface temperature through ST-LSTM of each layer, transmitting the output hidden states of two vertically adjacent modules to the next layer of modules through diagonal paths, capturing non-stationary information in the SST time sequence characteristics of the ocean surface temperature in MIM modules by a differential-based method, capturing the non-stationary information through stacked diagonal structures, modeling the time sequence non-stationary characteristics of the SST of the ocean surface temperature, and finally outputting y of the last MIM module of each layer 1 ,y 2 ,...,y t Polymerizing to obtain SST time comprehensive characteristics beta;
the space non-stationary characteristic modeling sub-network is composed of a space non-stationary characteristic extraction module and a space characteristic fusion module, and specifically comprises the following components:
in a space non-stationary feature extraction module, downsampling a marine surface temperature satellite remote sensing image with the size of M to form a feature map with three scales of M, (M) M/4, (M) M/8, performing differential operation on the satellite remote sensing images with the sizes of (M) M/4 and (M) M/8 through convolution operation, and capturing a small-scale non-stationary feature n; similarly, performing a differential operation on the satellite remote sensing images of M and (M) M/4 through convolution operation, and capturing a large-scale non-stationary characteristic N;
in a space feature fusion module, aggregating an original feature map S, a small-scale non-stable feature N and a large-scale non-stable feature N of M, respectively performing sigmoid function processing and tanh function processing on the obtained space aggregation features, reinforcing the non-obvious non-stable features in space by the sigmoid function to prevent loss, reserving complete space comprehensive features, and multiplying corresponding elements of two function processing results to obtain SST space comprehensive features alpha;
the SST time comprehensive feature beta obtained from the time non-stationary feature modeling sub-network is fused with the SST space comprehensive feature alpha in the space non-stationary feature modeling sub-network through a stacking-based cross attention time-space fusion module, and the SST space comprehensive feature alpha and the SST time comprehensive feature beta self-adaptively learn the weight of the SST space comprehensive feature alpha and the SST time comprehensive feature beta to the ocean surface temperature field in the stacking-based cross attention time-space fusion module, and fuse time and space information;
and S3, inputting the ocean surface temperature satellite remote sensing image to be processed into a trained ocean surface temperature sequence prediction network based on space-time double flow non-stationary sensing, and outputting an ocean surface temperature prediction result.
2. The marine surface temperature sequence prediction method based on space-time double flow non-stationary sensing as set forth in claim 1, wherein the calculation flow of the SST spatial integrated feature α is as follows:
o＝tanh(W os *S+W oN *N+W on *n)
g＝σ(W gs *S+W gN *N+W gn *n)
wherein W is os 、W oN 、W on 、W gs 、W gN 、W gn Representing a convolution parameter; * Representing a convolution operation; sigma is a sigmoid function, an activation function applied to the input weighted sum; tanh represents a tanh activation function; n represents a small-scale non-stationary feature; n represents a large scale non-stationary feature; s represents an original feature map;representing a Hadamard product; o represents the result of the tan h function after the original feature map S, the small-scale non-stationary feature N and the large-scale non-stationary feature N are aggregated; g represents the result of the original feature map S, the small-scale non-stationary feature N and the large-scale non-stationary feature N after aggregation by a sigmoid function.
3. The marine surface temperature sequence prediction method based on space-time double flow non-stationary perception according to claim 2, wherein in the stacked cross-attention based space-time fusion module, specifically:
firstly, SST space integrated feature alpha in (M, M) dimension is flattened and mapped through an attention coding mechanism to be respectively used as query Q in (M multiplied by M, 1) dimension α Sum value V α Matrix, SST time complex feature β likewise flattens the mapping by means of an attention coding mechanism as keys K in the (M1) dimension, respectively β Sum value V β Matrix:
Q α ＝W q *α,V α ＝W v *α
K β ＝W k *β,V β ＝W v *β
wherein W is q ，W k ，W v Represents a convolution parameter, represents a convolution operation, Q α Representing a spatial query matrix; v (V) α Representing a matrix of spatial values, K β Representing a time key matrix, V β Representing a time value matrix;
for Q α And K β And performing matrix multiplication and modeling through each grid point of a softmax function to obtain a correlation score e of the SST time comprehensive feature beta and the SST space comprehensive feature alpha, wherein the dimension is (M multiplied by M ) and the formula is as follows:
where softmax represents the normalized exponential function,representing a matrix multiplication;
then, the space value matrix V α Matrix multiplication with e is performed on a space value matrix V α Modeling each grid point on the model and remodelling the result through a reshape function to obtain a spatial attention characteristic Z through the (M, M) dimension α Similarly, the time value matrix V β Matrix multiplication with e is performed on a time value matrix V β Modeling each grid point on the model and remodelling the result through a reshape function to the (M, M) dimension to obtain a time attention feature Z β ：
Where reshape denotes a function that converts tensor dimensions,represents matrix multiplication, e represents correlation scores on temporal and spatial features, V α Representing a matrix of spatial values, V β Representing a matrix of time values, Z α Representing spatial attention features, Z β Representing a temporal attention feature;
finally, a final ocean surface temperature satellite remote sensing image prediction result is obtained through the aggregation characteristics and the multi-layer perceptron, and is expressed as follows:
Z＝MLP(W z [Z α ,Z β ])
wherein MLP represents a multi-layer perceptron, W Z Representing the convolution parameters, Z α Representing spatial attention features, Z β And Z represents the prediction result of the marine surface temperature satellite remote sensing image.
CN202310607658.8A 2023-05-26 2023-05-26 Ocean surface temperature sequence prediction method based on space-time double-flow non-stationary sensing Pending CN116612396A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310607658.8A CN116612396A (en) 2023-05-26 2023-05-26 Ocean surface temperature sequence prediction method based on space-time double-flow non-stationary sensing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310607658.8A CN116612396A (en) 2023-05-26 2023-05-26 Ocean surface temperature sequence prediction method based on space-time double-flow non-stationary sensing

Publications (1)

Publication Number Publication Date
CN116612396A true CN116612396A (en) 2023-08-18

Family

ID=87681477

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310607658.8A Pending CN116612396A (en) 2023-05-26 2023-05-26 Ocean surface temperature sequence prediction method based on space-time double-flow non-stationary sensing

Country Status (1)

Country Link
CN (1) CN116612396A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116822382A (en) * 2023-08-30 2023-09-29 中国海洋大学 Sea surface temperature prediction method and network based on space-time multiple characteristic diagram convolution
CN117237781A (en) * 2023-11-16 2023-12-15 哈尔滨工业大学（威海） Attention mechanism-based double-element fusion space-time prediction method
CN117350171A (en) * 2023-12-04 2024-01-05 山东省计算中心（国家超级计算济南中心） Mesoscale vortex three-dimensional subsurface structure inversion method and system based on double-flow model

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116822382A (en) * 2023-08-30 2023-09-29 中国海洋大学 Sea surface temperature prediction method and network based on space-time multiple characteristic diagram convolution
CN116822382B (en) * 2023-08-30 2023-11-21 中国海洋大学 Sea surface temperature prediction method and network based on space-time multiple characteristic diagram convolution
CN117237781A (en) * 2023-11-16 2023-12-15 哈尔滨工业大学（威海） Attention mechanism-based double-element fusion space-time prediction method
CN117237781B (en) * 2023-11-16 2024-03-19 哈尔滨工业大学（威海） Attention mechanism-based double-element fusion space-time prediction method
CN117350171A (en) * 2023-12-04 2024-01-05 山东省计算中心（国家超级计算济南中心） Mesoscale vortex three-dimensional subsurface structure inversion method and system based on double-flow model
CN117350171B (en) * 2023-12-04 2024-03-12 山东省计算中心（国家超级计算济南中心） Mesoscale vortex three-dimensional subsurface structure inversion method and system based on double-flow model

Similar Documents

Publication Publication Date Title
CN116612396A (en) Ocean surface temperature sequence prediction method based on space-time double-flow non-stationary sensing
Jin et al. Spatio-temporal graph neural networks for predictive learning in urban computing: A survey
Wang et al. Memory in memory: A predictive neural network for learning higher-order non-stationarity from spatiotemporal dynamics
WO2021068528A1 (en) Attention weight calculation method and apparatus based on convolutional neural network, and device
CN104662526B (en) Apparatus and method for efficiently updating spiking neuron network
CN111027686B (en) Landslide displacement prediction method, device and equipment
CN112733656B (en) Skeleton action recognition method based on multiflow space attention diagram convolution SRU network
CN114818515A (en) Multidimensional time sequence prediction method based on self-attention mechanism and graph convolution network
CN112767682A (en) Multi-scale traffic flow prediction method based on graph convolution neural network
CN116822382B (en) Sea surface temperature prediction method and network based on space-time multiple characteristic diagram convolution
CN113780662B (en) Flow prediction method, device, equipment and medium
CN116524419B (en) Video prediction method and system based on space-time decoupling and self-attention difference LSTM
CN116316591A (en) Short-term photovoltaic power prediction method and system based on hybrid bidirectional gating cycle
CN116844041A (en) Cultivated land extraction method based on bidirectional convolution time self-attention mechanism
CN115933010A (en) Radar echo extrapolation near weather prediction method
CN114817773A (en) Time sequence prediction system and method based on multi-stage decomposition and fusion
CN115641720A (en) Traffic prediction method and system based on space-time fusion graph neural network
CN112241802A (en) Interval prediction method for wind power
Qin et al. Remaining useful life prediction using temporal deep degradation network for complex machinery with attention-based feature extraction
CN113821724A (en) Graph neural network recommendation method based on time interval enhancement
CN116306780B (en) Dynamic graph link generation method
CN112784831A (en) Character recognition method for enhancing attention mechanism by fusing multilayer features
CN117197632A (en) Transformer-based electron microscope pollen image target detection method
CN117011668A (en) Weather radar echo extrapolation method based on time sequence prediction neural network
CN117009785A (en) Security monitoring method, device, server and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination