CN114385619A - Multi-channel ocean observation time sequence scalar data missing value prediction method and system - Google Patents
Multi-channel ocean observation time sequence scalar data missing value prediction method and system Download PDFInfo
- Publication number
- CN114385619A CN114385619A CN202210285171.8A CN202210285171A CN114385619A CN 114385619 A CN114385619 A CN 114385619A CN 202210285171 A CN202210285171 A CN 202210285171A CN 114385619 A CN114385619 A CN 114385619A
- Authority
- CN
- China
- Prior art keywords
- sequence
- time
- scalar data
- attention module
- observation time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 230000002123 temporal effect Effects 0.000 claims abstract description 10
- 238000007670 refining Methods 0.000 claims abstract description 6
- ATNHDLDRLWWWCB-AENOIHSZSA-M chlorophyll a Chemical group C1([C@@H](C(=O)OC)C(=O)C2=C3C)=C2N2C3=CC(C(CC)=C3C)=[N+]4C3=CC3=C(C=C)C(C)=C5N3[Mg-2]42[N+]2=C1[C@@H](CCC(=O)OC\C=C(/C)CCC[C@H](C)CCC[C@H](C)CCCC(C)C)[C@H](C)C2=C5 ATNHDLDRLWWWCB-AENOIHSZSA-M 0.000 claims description 53
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 claims description 26
- 229910052760 oxygen Inorganic materials 0.000 claims description 24
- 239000001301 oxygen Substances 0.000 claims description 24
- 239000013598 vector Substances 0.000 claims description 18
- 238000013507 mapping Methods 0.000 claims description 14
- 230000007812 deficiency Effects 0.000 claims description 8
- 238000007781 pre-processing Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 5
- 150000003839 salts Chemical group 0.000 claims 2
- 238000004364 calculation method Methods 0.000 abstract description 4
- 229930002875 chlorophyll Natural products 0.000 description 25
- 235000019804 chlorophyll Nutrition 0.000 description 25
- 230000006870 function Effects 0.000 description 17
- 238000012217 deletion Methods 0.000 description 15
- 230000037430 deletion Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 14
- 238000011176 pooling Methods 0.000 description 9
- 239000011159 matrix material Substances 0.000 description 8
- 230000000306 recurrent effect Effects 0.000 description 6
- 230000007246 mechanism Effects 0.000 description 5
- 238000003062 neural network model Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000012512 characterization method Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 230000004913 activation Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 230000004931 aggregating effect Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000379 polymerizing effect Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Fuzzy Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention belongs to the field of computer systems based on specific calculation models, and provides a method and a system for predicting missing values of multi-channel ocean observation time sequence scalar data, which are used for acquiring ocean observation time sequence scalar data with ocean missing values; obtaining a marine missing value prediction result by adopting a TA-RNN model based on the marine observation time sequence scalar data; the TA-RNN model comprises a convolution attention module, a space attention module and a time attention module, wherein the convolution attention module is used for refining the ocean observation time series scalar data; the space attention module is used for capturing dynamic space correlation of the refined ocean observation time sequence scalar data; the temporal attention module is configured to capture dynamic temporal correlations between different time intervals in the spatial attention module output data.
Description
Technical Field
The invention belongs to the field of computer systems based on specific calculation models, and particularly relates to a method and a system for predicting missing values of multi-channel ocean observation time sequence scalar data.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
Ocean monitoring relies on widely deployed ocean buoys and observation stations that integrate various types of ocean sensors. The marine ecosystem has a complex structure, so that marine observation data has complexity and diversity. Missing values refer to clustering, grouping, deletion, or truncation of data due to lack of information in the original data, which means that some characteristic value or values in the data are incomplete. Due to the fact that the ocean observation data such as chlorophyll, wind speed, dissolved oxygen, salinity, temperature, oxygen content, wind speed and turbidity are collected in a cooperative mode through a buoy system, a navigation system and a database system, all collection systems are easily interfered by external environment factors, and missing values exist in the data. These data have an impact on the accuracy of downstream applications, such as ocean data assimilation and intelligent data mining. The traditional methods such as mathematical statistics, empirical prediction and the like cannot achieve the expected target on ocean observation data with the characteristics of multiple factors, irregularity, complexity and the like. Therefore, the accurate marine observation data prediction model is researched by taking data as drive, and irreplaceable effects are exerted on filling missing values of marine observation time sequence scalar data.
Disclosure of Invention
In order to solve the technical problems in the background art, the invention provides a method and a system for predicting missing values of multi-channel ocean observation time sequence scalar data, which predict the future change trend of the multi-channel ocean observation time sequence scalar data through historical data of the multi-channel ocean observation time sequence scalar data, and use the predicted data in filling of the missing values.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention provides a method for predicting missing values of multi-channel ocean observation time sequence scalar data.
A multi-channel ocean observation time sequence scalar data missing value prediction method comprises the following steps:
obtaining ocean observation time sequence scalar data with ocean deficiency values;
obtaining a marine missing value prediction result by adopting a TA-RNN model based on the marine observation time sequence scalar data;
the TA-RNN model comprises a convolution attention module, a space attention module and a time attention module, wherein the convolution attention module is used for refining the ocean observation time series scalar data; the space attention module is used for capturing dynamic space correlation of the refined ocean observation time sequence scalar data; the temporal attention module is configured to capture dynamic temporal correlations between different time intervals in the spatial attention module output data.
A second aspect of the invention provides a multi-channel ocean observation time series scalar data missing value prediction system.
A multi-channel ocean observation time series scalar data missing value prediction system, comprising:
a data acquisition module configured to: obtaining ocean observation time sequence scalar data with ocean deficiency values;
a prediction module configured to: obtaining a marine missing value prediction result by adopting a TA-RNN model based on the marine observation time sequence scalar data;
the TA-RNN model comprises a convolution attention module, a space attention module and a time attention module, wherein the convolution attention module is used for refining the ocean observation time series scalar data; the space attention module is used for capturing dynamic space correlation of the refined ocean observation time sequence scalar data; the temporal attention module is configured to capture dynamic temporal correlations between different time intervals in the spatial attention module output data.
Compared with the prior art, the invention has the beneficial effects that:
according to the three-stage attention-based recurrent neural network (TA-RNN) model, in the first stage, a convolution attention module is adopted to carry out thinning operation on an input sequence, so that the new input sequence has stronger representation capability; in the second stage, a space attention module is adopted to enable the model to selectively capture the dynamic correlation among different input sequences; and in the third stage, a time attention module is adopted, so that the model can adaptively capture the dynamic time correlation between different time intervals in the input sequence.
The method and the device can accurately fill the missing value, thereby avoiding the problems of inaccurate filling of the missing value, larger error and the like.
The method overcomes the defect that the existing missing value filling can only depend on single-channel data for filling, and aims at marine multi-channel observation time sequence scalar data, and fills the missing value existing in the chlorophyll sequence through the correlation between chlorophyll and marine observation time sequence scalar data such as depth, temperature, conductivity, salinity, oxygen content, dissolved oxygen concentration, chlorophyll (including the missing value), turbidity, PH value, wind speed and the like. Because ocean data is abundant and diverse, in most scenes, a target sequence usually does not exist independently but exists with numerous time sequences, a specific scene data set is formed together, missing value filling is carried out on a multichannel ocean observation time sequence scalar data set, and the actual situation of the data set acquired by an ocean acquisition system is more similar.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
FIG. 1 is a flow chart of a method for predicting missing values of multi-channel marine observation time series scalar data according to an embodiment of the present invention;
FIG. 2 is a flow diagram illustrating missing value padding according to an embodiment of the present invention;
FIG. 3 is a diagram of a recursive neural network model framework based on three-stage attention according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a convolution attention module (CBAM) shown in an embodiment of the present invention;
FIG. 5 is a schematic diagram of a channel attention module shown in an embodiment of the present invention;
FIG. 6 is a schematic diagram of a spatial attention module shown in an embodiment of the present invention;
FIG. 7 is a diagram showing chlorophyll sequence with deletion values according to an embodiment of the present invention;
FIG. 8 is a graph showing the effect of chlorophyll sequence prediction on a sample set without missing values according to an embodiment of the present invention;
FIG. 9 is a filled chlorophyll sequence diagram according to an embodiment of the present invention;
FIG. 10 is a diagram showing a chlorophyll sequence containing deletion values, a part of which has a length of 50 according to an embodiment of the present invention;
FIG. 11 is a chlorophyll sequence diagram after linear interpolation processing according to an embodiment of the present invention;
fig. 12 is a diagram illustrating the effect of filling chlorophyll deletion after model prediction according to an embodiment of the present invention.
Detailed Description
The invention is further described with reference to the following figures and examples.
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
It is noted that the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and systems according to various embodiments of the present disclosure. It should be noted that each block in the flowchart or block diagrams may represent a module, a segment, or a portion of code, which may comprise one or more executable instructions for implementing the logical function specified in the respective embodiment. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
As introduced in the background art, most of the currently common methods for filling missing values of marine multichannel observation scalar data adopt fixed values, medians and modes to fill the missing values, but the problems that the filled missing values are inaccurate, large errors exist and the like can occur. According to the three-stage attention-based recurrent neural network (TA-RNN) model, in the first stage, a convolution attention module is adopted to carry out thinning operation on an input sequence, so that the new input sequence has stronger representation capability; in the second stage, a space attention module is adopted to enable the model to selectively capture the dynamic correlation among different input sequences; and in the third stage, a time attention module is adopted. Enabling the model to adaptively capture the dynamic time correlation between different time intervals in the input sequence. The method and the device can accurately fill the missing value, thereby avoiding the problems of inaccurate filling of the missing value, larger error and the like.
Lack of current deep learningThe value filling algorithm has the defect that missing value filling cannot be carried out on missing values of multi-channel ocean observation time sequence scalar data. The main current way to fill in missing values is E2GAN, but for sensor input to E2When the GAN is used for missing value filling, most of the GAN only has two columns of data, time and detection values. The method is basically absent in an actual ocean scene, and a plurality of sensors are integrated on an ocean buoy to work simultaneously, so that the data acquired by the ocean sensors are basically multi-channel data. The method aims at ocean multi-channel observation time sequence scalar data, adopts a three-stage attention-based recurrent neural network model, predicts the current value of a target sequence by using the past value of the target sequence and the current value and the past value of other sequences related to the target sequence, and fills the predicted value in the missing value position of the current data set.
The invention provides a three-stage attention-based recurrent neural network model for accurately predicting a multichannel ocean data missing value, wherein the three-stage attention-based recurrent neural network model is shown in figure 3, and the three-stage attention modules are respectively as follows:
(1) and the convolution attention module refines the original input sequence and increases the characterization capability of the original input sequence. The convolution attention module is proposed in 2018, mixes the space attention and the channel attention in the convolution module, is a lightweight and general module with good portability, and is used for processing a multi-channel input sequence.
(2) A spatial attention module that enables the model to selectively capture dynamic spatial correlations between different input sequences.
(3) A time attention module that enables the model to adaptively capture dynamic time correlations between different time intervals in the input sequence.
As shown in FIG. 3, a convolution attention module that takes the original input sequence
Refining to generate new input sequence
After convolution attention operation, the characterization capability of the original input sequence is increased; a spatial attention module capable of selectively capturing dynamic correlations between different input sequences; the gate control cycle unit can learn the hidden layer representation of the input sequence and update the hidden state at the current moment according to the input sequence and the hidden state at the previous moment; time attention module. It can adaptively capture the dynamic time correlation between different time intervals in the sequence.
Specific embodiments of the invention are described below from various embodiments:
example one
As shown in fig. 1, the present embodiment provides a method for predicting missing values of multi-channel ocean observation time series scalar data.
Here we use a multi-channel marine observation time series scalar dataset with chlorophyll deficiency values for the canadian ocean network, the multi-channel marine observation time series scalar dataset comprising: marine observation time sequence scalar data such as depth, temperature, conductivity, salinity, oxygen content, dissolved oxygen concentration, chlorophyll (containing a deletion value), turbidity, a pH value, wind speed and the like, wherein a chlorophyll sequence with the deletion value is shown in figure 7, wherein an x axis represents the length of the chlorophyll sequence, a y axis represents the value of the chlorophyll, and the deletion value is filled in a data set by using a fixed value 999 as shown in a circle. With reference to this data set, the technical solution of this embodiment is: the multi-channel ocean observation time series scalar data missing value prediction based on the three-stage attention recurrent neural network prediction model, as shown in FIG. 2, comprises the following steps:
(1) the data set is used as the input of the model, and data preprocessing is firstly carried out on the data set to obtain an initial sequence. The pretreatment stage comprises:
(1-1) processing chlorophyll data to be filled in a linear interpolation mode to obtain initial data;
(1-2) constructing a sample set without missing values, inputting the sample set without missing values into a model for training, and calculating corresponding numerical values by adopting a loss function.
(2) Taking chlorophyll sequence as the target sequence to be predicted, and measuring the correlation between other sequences and the target sequence through Pearson correlation coefficient. By calculating the quotient of covariance and standard deviation between the target sequence and the sequences such as depth, temperature, conductivity, salinity, oxygen content, dissolved oxygen concentration, chlorophyll (containing deletion value), turbidity, PH value, wind speed and the like, seven sequences of the depth, the wind speed, the oxygen content, the dissolved oxygen, the turbidity, the temperature and the salinity are selected to form an input sequence together with the chlorophyll sequence
Wherein n represents the number of different types of sequences,
which represents the size of the length of the input sequence,
the multichannel sequence composed of seven sequences including depth, wind speed, oxygen content, dissolved oxygen, turbidity, temperature and salinity is shown.
(3) Decomposing the input sequence after (2) into chlorophyll sequences
And a multichannel sequence consisting of seven sequences including depth, wind speed, oxygen content, dissolved oxygen, turbidity, temperature and salinity
。
(4) The multi-channel sequence
Input to the CBAM module, which is shown in fig. 4. First polymerizing signatures by average pooling and maximum pooling operationsAnd generating two different spatial context descriptors to respectively represent the average pool characteristic and the maximum pool characteristic according to the spatial information of the rays:
and
。
(5) as shown in FIG. 5, the two descriptors are input into a shared network composed of a multi-layer perceptron and a hidden layer to generate a channel attention map
Namely:
in the formula
A sigmoid function is represented as a function,
and
representing multi-layer perceptron weights.
(6) Inputting the original sequence
Element-by-element multiplication operations are performed with the sequence subject to the channel attention mapping. Obtaining a new input sequence
Namely:
in the formula,
representing element-by-element multiplication.
(7) As shown in fig. 6, the newly generated sequence is
Applying the average pooling and maximum pooling operations along the channel axis, aggregating the feature mapped channel information by two pooling operations, generating two spatial context descriptors:
and
. And concatenate them to generate the efficient feature descriptor, on which we apply the convolutional layer to generate the spatial attention map
Namely:
in the formula,
a sigmiod activation function is represented,
representing a filter size of
The convolution operation of (1).
(8) Carrying out element-by-element multiplication operation on the new input sequence obtained in the step (6) and the sequence subjected to space attention mapping to obtain final refined output
Namely:
(9) output after thinning
Generating a new input sequence by a spatial attention mechanism as an input to a spatial attention module
Namely:
in the formula,
representing the kth input sequence
，
Attention weight representing hidden state of encoder at time t, attention weight
Performing SoftMax function standardization to obtain
Is the encoder hidden state at time t-1,
and
is a matrix of parameters that needs to be learned,
is an attention weight that measures the importance of the kth input feature at time t.
(10) We take attention to the weight, we can update the input sequence and the encoder hidden state at time t, i.e.:
(11) the hidden states of the decoder and the encoder at the t-1 moment and the hidden states of the encoder at the t moment are input into a time attention module, and a context vector is obtained through a time attention mechanism
Namely:
in the formula,
is a matrix of parameters that needs to be learned,
is that
The hidden state of the decoder is at the moment,
is the hidden state of the encoder at time t-1,
is hidden form of encoder at time tThe state of the optical disk is changed into a state,
indicating attention weight of decoder at time t
Performing SoftMax function standardization to obtain
Attention weights that measure the importance of the ith input feature at time t,
is a context vector.
(12) When obtaining the context vector at the time t
Combining them with the target time series and updating the decoder hidden state at time t
Namely:
in the formula,
and b is a parameter matrix mapping the connection to the decoder input,
is the input to the decoder at time t-1,
is the calculated context vector and the context vector,
indicating a connection operation，
Is a new input after a linear transformation,
is the hidden state of the decoder at time t-1.
(13) Finally, the context vector is added
Implicit with time-T decoder
Concatenate to become the hidden state of the new decoder from which the final prediction is made:
in the form of matrix
Sum vector
Mapping connections
Finally we use a linear variation (
And
) Generating the final chlorophyll prediction result. The predicted effect graph is shown in fig. 8:
(14) the final filling result is obtained by filling the predicted chlorophyll data into the data set with the chlorophyll missing value, and the result is shown in fig. 9, in which the x axis represents the length of the chlorophyll sequence, the y axis represents the value of the chlorophyll concentration, and the circled portion represents the value after filling the missing value.
Here we take a portion of the length 50 chlorophyll sequence with missing values, as shown in FIG. 10, where the x-axis represents the length of the chlorophyll sequence and the y-axis represents the value of chlorophyll. The circled portion indicates the deletion value of the chlorophyll sequence, where the deletion value is indicated by 999 constant.
The linear interpolation of the chlorophyll sequence is shown in fig. 11, in which the x-axis represents the length of the chlorophyll sequence and the y-axis represents the value of chlorophyll. The circled portion indicates the result of filling the missing values of the chlorophyll sequence after linear interpolation.
The results of model prediction of chlorophyll sequence are shown in fig. 12, in which the x-axis represents the length of chlorophyll sequence and the y-axis represents the value of chlorophyll. The circled portion represents the result of filling in the deletion values of chlorophyll sequences as predicted by the model.
Comparing fig. 10, 11, and 12, we can see that the accuracy of the recursive neural network model based on three-stage attention for missing value padding is higher than that of linear interpolation.
The present embodiment includes the following advantages:
(1) the current value of the chlorophyll sequence is predicted based on the previous value, the current value and the past value of the depth, the wind speed, the oxygen content, the dissolved oxygen, the turbidity, the temperature and the salinity sequence of the chlorophyll sequence, and the defect that the existing missing value filling technology can only fill data in a marine single-channel observation time sequence scale data set is overcome.
(2) In the embodiment, the space attention module is used for replacing the original input attention module, and the dynamic space correlation among different input sequences can be selectively captured, so that the model can focus on the characteristics related to the prediction task, the prediction accuracy of the model is improved, the training cost of the model is reduced, and the accuracy of the model for filling missing values is improved.
(3) The embodiment uses the convolution attention module to refine the input sequence, compared with the original input attention module of the DA-RNN, the convolution attention module can refine the input sequence, and the characterization capability of the input sequence is enhanced. The problem of gradient decline of the model in training mass data is solved, the prediction performance is not reduced due to the increase of data volume, and the method has good stability. The model can effectively fill large batches of data sets with missing values.
Example two
The embodiment provides a multi-channel ocean observation time sequence scalar data missing value prediction system.
The technical scheme of the embodiment comprises the following modules:
1. acquisition and preprocessing module
Acquiring a multichannel ocean observation time sequence scalar data set with a chlorophyll deficiency value, and preprocessing the data set, wherein the preprocessing process comprises the following steps:
(1) and processing the part caused by chlorophyll sequence deletion in a linear interpolation mode, constructing a sample set without chlorophyll deletion values, inputting the sample set without the deletion values into the model of the invention for training, and calculating corresponding values by adopting a loss function.
(2) Taking the chlorophyll sequence as a target sequence to be predicted, and measuring the correlation between the depth, temperature, conductivity, salinity, oxygen content, dissolved oxygen concentration, chlorophyll (including a deletion value), turbidity, PH value, wind speed and other sequences in the marine multichannel data set and the chlorophyll sequence through Pearson correlation coefficients. By calculating the quotient of covariance and standard deviation between the target sequence and other sequences, we selected seven sequences of depth, wind speed, oxygen content, dissolved oxygen, turbidity, temperature, salinity, which are most related to chlorophyll sequence, and the target sequence together to form the input sequence:
(3) decomposing the data after (2) into chlorophyll sequences
And is composed ofNew input sequence composed of depth, wind speed, oxygen content, dissolved oxygen, turbidity, temperature and salt
Where n represents the number of different types of sequences in the new input sequence,
indicating the input sequence length.
2. Convolution attention module
Will be provided with
As an input, the convolution attention module (CBAM) infers a one-dimensional channel attention map in turn
And two-dimensional spatial attention mapping
. The overall process can be expressed as follows:
wherein,
representing element-by-element multiplication, in which the channel attention value is propagated along the spatial dimension,
is the output of the final refinement.
The specific calculation process is that firstly, the spatial information of the feature mapping is aggregated by average pooling and maximum pooling operation to generate twoThe different spatial context descriptors represent the average pool characteristic and the maximum pool characteristic, respectively:
and
the two descriptors are then sent to a shared network to generate a channel attention map
The shared network consists of a multilayer perceptron and a hidden layer, and after the shared layer is applied to each descriptor, the shared layer outputs a feature vector by using element summation and combination, and the channel attention calculation formula is as follows:
wherein,
a sigmoid function is represented as a function,
and
representing multi-layer perceptron weights.
Computation space note that we first apply the average pooling and max pooling operations along the channel axis and concatenate them to generate the efficient feature descriptors. Applying pool operations along the channel axis can effectively highlight the information region. On concatenated feature descriptors, we apply convolutional layers to generate spatial attention maps
And generating two spatial context descriptors by aggregating the channel information of the feature mapping through two pool operations:
and
the spatial attention is calculated as follows:
wherein,
a sigmiod activation function is represented,
representing a filter size of
The convolution operation of (1). The input features are preprocessed through a convolution attention mechanism, so that the input features are refined, and the characterization capability of the input features is enhanced.
3. Space attention module
Output after thinning
Generating a new input sequence by a spatial attention mechanism as an input to a spatial attention module
Namely:
in the formula,
representing the kth input sequence
，
Attention weight representing hidden state of encoder at time t, attention weight
Performing SoftMax function standardization to obtain
Is the encoder hidden state at time t-1,
and
is a matrix of parameters that needs to be learned,
is an attention weight that measures the importance of the kth input feature at time t. Through a spatial attention mechanism, the model is enabled to selectively capture dynamic spatial correlations between different input features.
4. Encoder for encoding a video signal
The encoder is essentially an RNN, which encodes an input sequence into a feature representation in machine translation. For the input sequence after the space attention operation
The encoder is used for learning from
To
Mapping (at time t):
wherein,
representing the hidden state of the encoder at time t, m representing the size of the hidden state,
representing a non-linear mapping function, here we use gated round-robin units (GRUs) as
To capture long term dependencies in the sequence. The GRU consists of 2 gates: reset door
Updating door
. The update process of the GRU is as follows:
wherein,
encoder hidden state for time t-1
And input of the current time t
The connection of (a) to (b),
are parameters that need to be learned.
A sigmoid activation function is represented,
representing element-by-element multiplication.
5. Time attention module
In the decoding stage, a time attention mechanism is used for modeling dynamic time correlation among different time intervals in an input sequence, the hidden states of a decoder and an encoder at the t-1 moment and the hidden states of the encoder at the t moment are input into a time attention module, and a context vector is obtained through the time attention mechanism
The attention weight of each decoder hidden state at time t is defined as follows:
wherein,
is a matrix of parameters that needs to be learned,
is that
The hidden state of the decoder is at the moment,
is the hidden state of the encoder at time t-1,
is the hidden state of the encoder at time t,
indicating attention weight of decoder at time t
Performing SoftMax function scalingStandardized processing to obtain
Attention weights that measure the importance of the ith input feature at time t,
is a context vector.
6. Decoder
When obtaining the context vector at the time t
We combine them with the target time series and update the new hidden states of the decoder at time t
。
wherein, the matrix
Sum vector
Mapping connections
Finally we use a linear variation (
And
) Generating the final chlorophyll prediction result.
7. Model validation
As shown in fig. 11, after the prediction result is obtained, the loss value between the prediction result and the true value of the interpolated multi-channel data set is calculated by using the mean square error, and the network parameter of the model is adjusted to obtain the final chlorophyll prediction result.
8. Missing value filling
And filling the final chlorophyll prediction result into a missing value unit of the multi-channel data set to obtain a filling result.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A multi-channel ocean observation time sequence scalar data missing value prediction method is characterized by comprising the following steps:
obtaining ocean observation time sequence scalar data with ocean deficiency values;
obtaining a marine missing value prediction result by adopting a TA-RNN model based on the marine observation time sequence scalar data;
the TA-RNN model comprises a convolution attention module, a space attention module and a time attention module, wherein the convolution attention module is used for refining the ocean observation time series scalar data; the space attention module is used for capturing dynamic space correlation of the refined ocean observation time sequence scalar data; the temporal attention module is configured to capture dynamic temporal correlations between different time intervals in the spatial attention module output data.
2. The multi-channel ocean observation time series scalar data missing value prediction method of claim 1, comprising, after the obtaining the ocean observation time series scalar data with ocean missing values: and preprocessing the ocean observation time sequence scalar data with the ocean deficiency value to obtain an initial sequence.
3. The method of multi-channel ocean observation time series scalar data missing value prediction according to claim 1, comprising, before the employing the TA-RNN model: and if the marine observation time sequence scalar data with the marine deficiency value is a chlorophyll sequence, selecting a depth sequence, a wind speed sequence, an oxygen content sequence, a dissolved oxygen sequence, a turbidity sequence, a temperature sequence and a salt sequence, and constructing a multichannel sequence according to the depth sequence, the wind speed sequence, the oxygen content sequence, the dissolved oxygen sequence, the turbidity sequence, the temperature sequence, the salt sequence and the chlorophyll sequence.
4. The multi-channel ocean observation time series scalar data missing value prediction method according to claim 3, characterized in that a convolution attention module is adopted to obtain a channel attention map and a space attention map according to the multi-channel sequence; multiplying the sequence of the channel attention mapping of the multichannel sequence by the multichannel sequence element by element to obtain an initial refined sequence; and multiplying the initial refined sequence by the spatial attention mapping sequence of the initial refined sequence element by element to obtain the final refined sequence.
5. The method for predicting missing values of multi-channel ocean observation time series scalar data according to claim 4, wherein based on the final refined sequence, a spatial attention module is adopted to capture dynamic spatial correlation between different input features in the final refined sequence to obtain an input sequence.
6. The method of claim 5, wherein the encoder is adapted to learn the mapping from the input sequence to the hidden state of the encoder at time t based on the input sequence to obtain the hidden state of the encoder at time t.
7. The method for predicting missing values of multi-channel ocean observation time sequence scalar data according to claim 6, characterized in that according to the hidden state of the encoder at the time t, a time attention module is adopted to capture the dynamic time correlation among different time intervals in the sequence of the hidden state of the encoder at the time t; the specific process adopting the time attention module comprises the following steps:
determining attention weight of each input feature at the time t according to the hidden state of the encoder at the time t and the hidden state of the decoder at the time t-1; determining an attention weight of a certain input feature to a predicted value at time t based on the attention weight of each input feature at time t; and obtaining a weighted sum of all encoder hidden states, namely a context vector, based on the attention weights of all input features to the predicted values at the time t and the hidden states of the encoder at the time t.
8. The method of claim 7, wherein the context vector at time t is determined and the target sequence at time t-1 is combined to update the decoder's hidden state at time t.
9. The method of claim 8, wherein the context vector at time T is concatenated with the updated decoder's hidden state at time T to form a new decoder's hidden state, and a missing chlorophyll sequence is predicted.
10. A multi-channel ocean observation time series scalar data missing value prediction system, comprising:
a data acquisition module configured to: obtaining ocean observation time sequence scalar data with ocean deficiency values;
a prediction module configured to: obtaining a marine missing value prediction result by adopting a TA-RNN model based on the marine observation time sequence scalar data;
the TA-RNN model comprises a convolution attention module, a space attention module and a time attention module, wherein the convolution attention module is used for refining the ocean observation time series scalar data; the space attention module is used for capturing dynamic space correlation of the refined ocean observation time sequence scalar data; the temporal attention module is configured to capture dynamic temporal correlations between different time intervals in the spatial attention module output data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210285171.8A CN114385619B (en) | 2022-03-23 | 2022-03-23 | Multi-channel ocean observation time sequence scalar data missing value prediction method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210285171.8A CN114385619B (en) | 2022-03-23 | 2022-03-23 | Multi-channel ocean observation time sequence scalar data missing value prediction method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114385619A true CN114385619A (en) | 2022-04-22 |
| CN114385619B CN114385619B (en) | 2022-07-15 |
Family
ID=81205385
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210285171.8A Active CN114385619B (en) | 2022-03-23 | 2022-03-23 | Multi-channel ocean observation time sequence scalar data missing value prediction method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114385619B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115935139A (en) * | 2023-01-09 | 2023-04-07 | 吉林大学 | Space field interpolation method for ocean observation data |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107577649A (en) * | 2017-09-26 | 2018-01-12 | 广州供电局有限公司 | The interpolation processing method and device of missing data |
| US20200089650A1 (en) * | 2018-09-14 | 2020-03-19 | Software Ag | Techniques for automated data cleansing for machine learning algorithms |
| CN111339234A (en) * | 2020-02-26 | 2020-06-26 | 上海澜启信息科技有限公司 | Method, device and storage medium for repairing missing positioning data samples |
| US20200349169A1 (en) * | 2019-05-03 | 2020-11-05 | Accenture Global Solutions Limited | Artificial intelligence (ai) based automatic data remediation |
| CN112232600A (en) * | 2020-11-17 | 2021-01-15 | 烟台海颐软件股份有限公司 | Short-term power load prediction method and system based on variational modal decomposition |
| CN112527788A (en) * | 2020-12-17 | 2021-03-19 | 北京中恒博瑞数字电力科技有限公司 | Method and device for detecting and cleaning abnormal value of transformer monitoring data |
| CN112685950A (en) * | 2020-12-02 | 2021-04-20 | 山东省计算中心（国家超级计算济南中心） | Method, system and equipment for detecting abnormality of ocean time sequence observation data |
| CN113094357A (en) * | 2021-04-23 | 2021-07-09 | 大连理工大学 | Traffic missing data completion method based on space-time attention mechanism |
| US11223543B1 (en) * | 2020-09-29 | 2022-01-11 | Dell Products L.P. | Reconstructing time series datasets with missing values utilizing machine learning |
| CN113962432A (en) * | 2021-09-10 | 2022-01-21 | 国网江苏省电力有限公司电力科学研究院 | Wind power prediction method and system integrating three-dimensional convolution and light-weight convolution threshold unit |
| CN113988951A (en) * | 2021-11-19 | 2022-01-28 | 中核第四研究设计工程有限公司 | Commodity recommendation learning model construction method based on tensor decomposition and collaborative filtering |
-
2022
- 2022-03-23 CN CN202210285171.8A patent/CN114385619B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107577649A (en) * | 2017-09-26 | 2018-01-12 | 广州供电局有限公司 | The interpolation processing method and device of missing data |
| US20200089650A1 (en) * | 2018-09-14 | 2020-03-19 | Software Ag | Techniques for automated data cleansing for machine learning algorithms |
| US20200349169A1 (en) * | 2019-05-03 | 2020-11-05 | Accenture Global Solutions Limited | Artificial intelligence (ai) based automatic data remediation |
| CN111339234A (en) * | 2020-02-26 | 2020-06-26 | 上海澜启信息科技有限公司 | Method, device and storage medium for repairing missing positioning data samples |
| US11223543B1 (en) * | 2020-09-29 | 2022-01-11 | Dell Products L.P. | Reconstructing time series datasets with missing values utilizing machine learning |
| CN112232600A (en) * | 2020-11-17 | 2021-01-15 | 烟台海颐软件股份有限公司 | Short-term power load prediction method and system based on variational modal decomposition |
| CN112685950A (en) * | 2020-12-02 | 2021-04-20 | 山东省计算中心（国家超级计算济南中心） | Method, system and equipment for detecting abnormality of ocean time sequence observation data |
| CN112527788A (en) * | 2020-12-17 | 2021-03-19 | 北京中恒博瑞数字电力科技有限公司 | Method and device for detecting and cleaning abnormal value of transformer monitoring data |
| CN113094357A (en) * | 2021-04-23 | 2021-07-09 | 大连理工大学 | Traffic missing data completion method based on space-time attention mechanism |
| CN113962432A (en) * | 2021-09-10 | 2022-01-21 | 国网江苏省电力有限公司电力科学研究院 | Wind power prediction method and system integrating three-dimensional convolution and light-weight convolution threshold unit |
| CN113988951A (en) * | 2021-11-19 | 2022-01-28 | 中核第四研究设计工程有限公司 | Commodity recommendation learning model construction method based on tensor decomposition and collaborative filtering |
Non-Patent Citations (2)
| Title |
|---|
| XIANG LI等: ""ECNN: One Online Deep Learning Model for Streaming Ocean Data Prediction"", 《ACM ICEA "21: PROCEEDINGS OF THE 2021 ACM INTERNATIONAL CONFERENCE ON INTELLIGENT COMPUTING AND ITS EMERGING APPLICATIONS》, 31 December 2021 (2021-12-31), pages 170 - 175, XP058872467, DOI: 10.1145/3491396.3506519 * |
| 孙晓丽 等: ""基于改进神经过程的缺失数据填充算法"", 《中国科学院大学学报》, 15 March 2021 (2021-03-15), pages 280 - 287 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115935139A (en) * | 2023-01-09 | 2023-04-07 | 吉林大学 | Space field interpolation method for ocean observation data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114385619B (en) | 2022-07-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112347859B (en) | Method for detecting significance target of optical remote sensing image | |
| CN111428789A (en) | Network traffic anomaly detection method based on deep learning | |
| CN110427654B (en) | Landslide prediction model construction method and system based on sensitive state | |
| CN110298387A (en) | Incorporate the deep neural network object detection method of Pixel-level attention mechanism | |
| CN113128558B (en) | Target detection method based on shallow space feature fusion and adaptive channel screening | |
| CN110210513B (en) | Data classification method and device and terminal equipment | |
| CN115885289A (en) | Modeling dependency with global self-attention neural networks | |
| CN113627093B (en) | Underwater mechanism trans-scale flow field characteristic prediction method based on improved Unet network | |
| CN110765854A (en) | Video motion recognition method | |
| CN115390164B (en) | Radar echo extrapolation forecasting method and system | |
| CN110930378B (en) | Emphysema image processing method and system based on low data demand | |
| CN112347531B (en) | Brittle marble Dan Sanwei crack propagation path prediction method and system | |
| CN109272497A (en) | Method for detecting surface defects of products, device and computer equipment | |
| CN114385619B (en) | Multi-channel ocean observation time sequence scalar data missing value prediction method and system | |
| CN112949944A (en) | Underground water level intelligent prediction method and system based on space-time characteristics | |
| CN116861262A (en) | Perception model training method and device, electronic equipment and storage medium | |
| CN115953902B (en) | Traffic flow prediction method based on multi-view space-time diagram convolutional network | |
| CN117197632A (en) | Transformer-based electron microscope pollen image target detection method | |
| CN115438575B (en) | Analysis method for high-precision airfoil flow field prediction | |
| CN114239934A (en) | Power grid load sequence prediction method and system based on generative countermeasure network | |
| CN111382761B (en) | CNN-based detector, image detection method and terminal | |
| Wang et al. | Research on Semantic Segmentation Algorithm for Multiscale Feature Images Based on Improved DeepLab v3+ | |
| CN118395385B (en) | Ocean environment prediction method and system based on remote sensing and hydrologic sampling data fusion | |
| CN118378178B (en) | Transformer fault identification method and system based on residual map convolution neural network | |
| CN116393188B (en) | Microfluidic chip and method suitable for capturing circulating tumor cells |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |