CN113298131A - Attention mechanism-based time sequence data missing value interpolation method - Google Patents

Attention mechanism-based time sequence data missing value interpolation method Download PDF

Info

Publication number
CN113298131A
CN113298131A CN202110533285.5A CN202110533285A CN113298131A CN 113298131 A CN113298131 A CN 113298131A CN 202110533285 A CN202110533285 A CN 202110533285A CN 113298131 A CN113298131 A CN 113298131A
Authority
CN
China
Prior art keywords
time sequence
sequence data
data
value
complete
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110533285.5A
Other languages
Chinese (zh)
Other versions
CN113298131B (en
Inventor
季微
金博斌
李云
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Posts and Telecommunications
Original Assignee
Nanjing University of Posts and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Posts and Telecommunications filed Critical Nanjing University of Posts and Telecommunications
Priority to CN202110533285.5A priority Critical patent/CN113298131B/en
Publication of CN113298131A publication Critical patent/CN113298131A/en
Application granted granted Critical
Publication of CN113298131B publication Critical patent/CN113298131B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/10—Pre-processing; Data cleansing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Complex Calculations (AREA)

Abstract

The invention discloses an attention mechanism-based interpolation method for missing values of time sequence data, which comprises the following steps: acquiring time sequence data with missing values; inputting the time sequence data with the missing value into a generator after training, and acquiring the time sequence data after interpolation; wherein the training of the generator comprises: inputting the time sequence data with missing values into a generator, and acquiring complete time sequence data based on an attention mechanism; and inputting the time sequence data with the missing values and the complete time sequence data into a discriminator, and carrying out antagonistic training on the discriminator and the generator based on the loss function. The invention can generate new time series data which are in accordance with the distribution of the original data set. The expression ability of important features in the features can be enhanced and the expression ability of unimportant features can be reduced by trial attention mechanism, and the processing efficiency can be improved. By the method, the accuracy of the interpolation of the missing values of the time series can be improved, and the interpolation efficiency can be improved.

Description

Attention mechanism-based time sequence data missing value interpolation method
Technical Field
The invention relates to a time sequence data missing value interpolation method based on an attention mechanism, and belongs to the technical field of computer science.
Background
In recent years, with the development of artificial intelligence technology, the frequency of occurrence of time series data in human life is also increasing. The time sequence data is a sequence formed by arranging numerical values of the same statistical index according to the occurrence time sequence, and reflects the state change and the development rule of objects and behaviors along with the change of time. The common time series includes some medical data, such as the change of blood sugar value of the diabetic patient with time in one day, and also includes the change of website access amount, road traffic amount and the like in different times.
The acquired data is often lost due to instability or interference of the data acquisition equipment. The absence of time series data can cause certain difficulties in the analytical modeling and practical application of the data. For example, when predicting future weather conditions based on historical weather conditions, if historical data is missing, the accuracy of the prediction will be affected. Therefore, an accurate and effective method for interpolating missing values of an incomplete data set is needed to obtain a complete data set that can be infinitely close to the real data.
In recent years, deep learning has been a great success in the field of artificial intelligence, and is rapidly becoming the leading technology in the field of artificial intelligence. The deep learning prediction model needs to perform network training and parameter optimization based on a complete data set to learn the historical change rule of data. The missing part of the data set often implies the historical change rule of the data, and as the missing part of the data set causes that the model cannot be completely driven, the finally trained parameters have larger difference with the optimal parameters. Therefore, the technology for effectively processing the missing value of the time series data has great research significance and practical application.
Disclosure of Invention
The invention aims to overcome the defects in the prior art, provides a time sequence data missing value interpolation method based on an attention mechanism, and solves the technical problem that in the traditional generation countermeasure network structure, the input of a generator is a random vector, the random vector is directly used for filling the time sequence data missing value, a large amount of time is consumed for searching the optimal input vector for each piece of time sequence data, and therefore the interpolation efficiency of the time sequence data missing value is greatly reduced.
In order to achieve the purpose, the invention is realized by adopting the following technical scheme:
the invention provides a time sequence data missing value interpolation method based on an attention mechanism, which comprises the following steps of:
acquiring time sequence data with missing values;
inputting the time sequence data with the missing value into a generator after training, and acquiring the time sequence data after interpolation;
wherein the training of the generator comprises:
inputting the time sequence data with missing values into a generator, and acquiring complete time sequence data based on an attention mechanism;
and inputting the time sequence data with the missing values and the complete time sequence data into a discriminator, and carrying out antagonistic training on the discriminator and the generator based on the loss function.
Further, the generator is a noise reduction self-encoder comprising an encoder unit and a decoder unit.
Further, the acquiring the complete time series data includes:
the encoder unit outputs a hidden vector H of original time sequence data x according to the input original time sequence data x with a preset length of m and a missing value and a random noise vector eta;
the decoder unit interpolates the missing value of the original time sequence data x according to the original time sequence data x and the hidden vector H in combination with an attention mechanism to acquire complete time sequence data
Figure BDA0003068725750000021
Further, the missing value of the original time sequence data x is interpolated to obtain complete time sequence data
Figure BDA0003068725750000031
The method comprises the following steps:
according to the original time sequence data S of the nth timen-1Combining the hidden vector H at each moment with an attention mechanism to obtain a weight factor alpha at each moment;
performing weighted average on all hidden vectors H according to the weight factor alpha at each moment;
substituting the result of the weighted average into a hyperbolic tangent function to obtain a complete value S of the original time sequence data at the nth momentn；
The complete value S of the original time sequence data at the nth momentnSubstituting the original time sequence data as the n +1 th time sequence data into the step, and circularly calculating to obtain the complete value S of the original time sequence data at all times;
acquiring complete time sequence data according to the complete value S of the original time sequence data at all the moments
Figure BDA0003068725750000034
Wherein H ═ { H ═ H1，H2，H3，…，Hi，…，Hm}，HiAs a hidden vector at time i, α ═ α1，α2，α3，…，αi，…，αm}，αiAs weighting factor at time i, S ═ S1，S2，S3，…，Sn，…，Sm}，SnIs the complete value of the original time series data at the nth time, S0＝HmI.e. the initial input vector of the decoder is S0。
Further, the obtaining the weighting factor α at each time instant includes:
Ki＝Wk·Hi
Qn-1＝Wq·Sn-1
wherein, KiFor the ith key value, H in the attention mechanismiA hidden vector at the ith moment; qn-1Is the (n-1) th query value, S in the attention mechanismn-1Is the original time sequence data of the n-1 th time; wkAnd WqIs a parameter matrix, W, learned from training datakAnd WqThe initial value of the network is obtained by a random initialization parameter matrix and is updated by generating a loss function of a countermeasure network and a back propagation algorithm;
order:
Figure BDA0003068725750000032
wherein,
Figure BDA0003068725750000033
is a matrix KiThe transposed matrix of (2);
Figure BDA0003068725750000041
wherein alpha isiFor the weighting factor at the ith time, the softmax function is a normalized exponential function for the input
Figure BDA0003068725750000042
The values are mapped to positive outputs between 0-1 and the sum of their all outputs is 1.
Further, the weighted average result is substituted into the hyperbolic tangent function to obtain the complete value S of the original time sequence data at the nth momentnThe method comprises the following steps:
the result of the weighted average is:
Cn-1＝α1H1+α2H2+…+αiHi+…+αmHm
order:
Figure BDA0003068725750000043
wherein, the tanh function is a hyperbolic tangent function, and the expression is as follows:
Figure BDA0003068725750000044
ωnand bnIs a parameter, ω, learned from training datanAnd bnIs obtained from random initialization parameters and updated by generating a loss function against the network and a back propagation algorithm.
Further, the performing antagonistic training on the arbiter and the generator comprises:
training the discriminator with a loss function:
Figure BDA0003068725750000045
wherein D (x) is a probability value that the discriminator determines the input original time sequence data x as true,
Figure BDA0003068725750000046
for complete timing data of the discriminator pair input
Figure BDA0003068725750000047
A probability value determined to be true;
training the generator with the loss function:
Figure BDA0003068725750000048
and repeating for multiple times until the probability output by the discriminator is close to 0.5, and stopping training.
Compared with the prior art, the invention has the following beneficial effects:
the invention provides a time sequence data missing value interpolation method based on an attention mechanism, which utilizes a generation countermeasure network and combines the attention mechanism to provide a time sequence data missing value filling method based on the attention mechanism. Through antagonistic training, the method can generate new time sequence data which are consistent with the distribution of the original data set. The expression ability of important features in the features can be enhanced and the expression ability of unimportant features can be reduced by trial attention mechanism, and the processing efficiency can be improved. By the method, the accuracy of the interpolation of the missing values of the time series can be improved, and the interpolation efficiency can be improved.
Drawings
FIG. 1 is a detailed block diagram of a generation countermeasure network generator in an embodiment of the invention;
FIG. 2 is a detailed block diagram of a generation countermeasure network arbiter in an embodiment of the invention;
fig. 3 is a flowchart illustrating a missing value interpolation process of time series data according to an embodiment of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present invention is not limited thereby.
The invention takes a data set KDD CUP 2018Dataset (KDD for short) as an example, and the steps of the method for performing missing value interpolation are described in detail. The dataset is a meteorological dataset from a KDD cup 2018 challenge. The data set comprises historical meteorological data of Beijing and consists of data of a plurality of meteorological observation stations respectively located in different places of Beijing. Data from 11 of the meteorological stations are selected, each comprising a record of weather and air quality data every other hour from 1/2017 to 30/12/2017. Specifically, 12 attributes including PM2.5, PM10, carbon monoxide and temperature were recorded.
As shown in fig. 3:
step 1: a generator for generating the countermeasure network is constructed according to the figure 1, wherein the generator is a noise reduction self-encoder and comprises two parts of an encoder and a decoder. Inputting an original time sequence X (with missing data, X in figure 1) with length m and a random noise vector eta into a noise reduction self-encoder part of an encoder to output a hidden vector HmI.e. by
Hm＝Encoder(x+η)
Both the encoder and decoder are constructed of Recurrent Neural Network (RNN) units. In order to have a certain difference with original data, random noise eta is added in the process of reconstructing data by a noise reduction self-encoder unit so as to reduce the occurrence of an overfitting phenomenon and reconstruct more accurate data.
Step 1.1: the original time series with the deletions are input into the encoder shown in fig. 1 in chronological order.
Step 1.2: random noise η is added to the original time series data x (with a deletion).
Step 1.3: the encoding process is done by the encoder portion of the noise reduction self-encoder,
the length of the original time series is m. The encoder has m total RNN hidden layer outputs, which are respectively H1,H2,H3,...,Hm. In fig. 1, a time sequence with a time length of 4 and an attribute number of 4 is illustrated, each row represents an attribute, each column represents a time, so that the time is in a 4 × 4 matrix form, and the last hidden layer output of RNN is H4。
Step 2: by step 1 we get the output of each hidden layer of the encoder part RNN unit, but finally we input into the decoder part only the output H of the last hidden layerm. Order S0＝HmI.e. the initial input vector of the decoder is S0It contains the information of the original time column entered. The decoder of the noise reduction self-encoder outputs complete interpolated time series data step by step according to the time sequence. The following explains the decomposition of data interpolation into sub-steps at each time.
Step 2.1: first we want to make an initial time t1For data interpolation, we need to calculate the initial input vector S of the decoder0And the implicit vector H output by the encoder at each moment1,H2,H3,...,HmThe correlation of the values, the weight obtained is denoted as alphai,1≤i≤m。α1To alphamAre real numbers between 0 and 1. The following is a detailed description1To alphamThe calculation procedure of (1).
Step 2.1.1: calculating parameters according to the output of each hidden layer in the step 1
Ki＝Wk·Hi，i＝1,2,3,...,m
Q0＝Wq·S0
Wherein, KiI.e. the key value, Q, in the attention mechanism0I.e. the query value inside the attention mechanism, the subscript representing the number one. "." denotes a multiplication operation. WkAnd WqIs a parameter matrix obtained by learning from training data, and a parameter matrix W is initialized randomlykAnd WqThen, an update is performed, and the parameter matrix is updated by generating a loss function of the countermeasure network and a back propagation algorithm, which is introduced in the subsequent step 3.
Order to
Figure BDA0003068725750000071
Where "T" denotes a transpose operation,
Figure BDA0003068725750000072
i.e. the matrix KiThe transposed matrix of (2).
S0The weight of the hidden state m-1 times before the RNN unit of the encoder is
Figure BDA0003068725750000073
Wherein, the softmax function is also called as normalization index function and is input
Figure BDA0003068725750000074
To
Figure BDA0003068725750000075
The values are mapped to positive numbers between 0 and 1 and the sum of the output result values adds up to 1.
Step 2.1.2: using the weight factor alpha obtained in step 2.1.1iAnd (3) performing weighted average on all historical hidden vectors H of the encoder in the step 1 to obtain:
C0＝α1H1+…+αmHm
further, let
Figure BDA0003068725750000081
the tanh function is a hyperbolic tangent function, which is defined by the formula:
Figure BDA0003068725750000082
ω1and b1Is a parameter obtained by learning from training data, and a parameter omega is initialized randomly1And b1And then updating, namely updating the parameters by generating a loss function of the countermeasure network and a back propagation algorithm in the third step.
This results in the output S of the first unit of the decoder RNN1I.e. initial time t1The complete data value of. Up to this point, the initial time t1Has been completed, and then proceeds to the next time t2The interpolation of (3).
Step 2.2: recalculating the Current State S1Weights to m H states of the encoder, denoted as α1，α2，…,αmIt is worth noting that although the weight α has been previously calculated, this time since the S state is from S0Update to S1Therefore, the new weight α representing the current state S cannot be directly used1And m H encoders. The new calculation method of alpha differs from the previous one in that here the parameter Q is Q1Namely:
Ki＝Wk·Hi，i＝1,2,3,...,m
Q1＝Wq·S1
Figure BDA0003068725750000083
Figure BDA0003068725750000084
calculating vector C by formula1：
C1＝α1H1+…+αmHm
By the formula:
Figure BDA0003068725750000091
obtaining an output S of the second unit of the decoder RNN2Instant t2The complete data value of. At this point, time t2The data interpolation of (2) is completed, and then the data interpolation at the subsequent time is performed.
Step 2.3, the steps are carried out in sequence according to the method, and the parameter Q is updated at each moment according to the formula in the step 2.2i，αi，CiCalculating the time t by the updated parameteriIs output SiInstant tiAt the last time tmStop, tmThe output of the time is SmI.e., the time series data value of time m. Finally, complete interpolated time series data can be obtained.
And 3, inputting the complete time sequence and the original time sequence generated by the generator into a discriminator, wherein the structure of the discriminator is shown in figure 2, and a probability value is output by the discriminator and represents the probability that the generated sequence is a real sequence. The interpolated complete time data can be obtained by the antagonistic training of the discriminator and the generator. The specific training method is to fix the generator first, and utilize the loss function:
Figure BDA0003068725750000092
training the arbiter, D (x) is the probability that the arbiter determines true for the input true original missing timing dataThe value (P true in figure 2),
Figure BDA0003068725750000093
the complete time series data representing the generation of a false,
Figure BDA0003068725750000094
is a probability value at which the discriminator determines that the input false complete time series data is true. Then using the loss function:
Figure BDA0003068725750000095
and a training generator, wherein lambda is a hyper-parameter, and the training is repeated for a plurality of times until the probability output by the discriminator is close to 0.5.
In summary, the time-series missing value interpolation method provided by the present invention uses the generation of the countermeasure network as a basic framework. In the conventional generation countermeasure network structure, the input of the generator is a random vector, the random vector is directly used for filling missing values of time sequence data, and a large amount of time is consumed for searching an optimal input vector for each piece of time sequence data, so that the interpolation efficiency of the missing values of the time sequence data is greatly reduced.
The time sequence missing value interpolation method provided by the invention abandons the step of searching the optimal input vector for each piece of time sequence data in the traditional generation countermeasure network frame, and obtains the low-dimensional feature expression vector of the time sequence data by adopting the noise reduction self-encoder, and then reconstructs the complete time sequence data by utilizing the low-dimensional feature expression vector, thereby saving a large amount of training time.
The generator portion used in the present invention to generate the countermeasure network employs a noise reduction autoencoder. A noise-reducing self-encoder is a neural network that uses a back-propagation algorithm to make the output value equal to the input value. A noise-reducing self-encoder includes an encoder and a decoder that compress an input into a latent spatial representation and then reconstruct an output from the representation. The noise reduction self-encoder is similar to a perception mechanism of human eyes, when the human eyes observe an object, even if partial outline of the object does not enter a visual field, the human eyes can identify the type of the object, so that the noise reduction self-encoder can learn low-dimensional feature expression vectors of input data and can also be used for repairing lost data. It is because its input data is not complete in nature and can be applied naturally to the missing value padding algorithm.
The encoder part of the noise reduction self-encoder is used for automatically generating corresponding low-dimensional vectors for each piece of original time sequence data with missing values, then the generated low-dimensional vectors are used as input of a decoder part of the noise reduction self-encoder, the decoder performs interpolation on the time sequence data according to the time sequence of the time sequence data, the next time is entered after the interpolation at the current time is completed, weights are automatically distributed to the whole network based on an attention mechanism, the weighted average of all hidden vectors of the encoder can be obtained by utilizing the weights, the low-dimensional feature vectors output by the encoder can be more suitable for the data interpolation at the current time, the problem of long-time sequence information forgetting is solved, and the interpolation accuracy is improved.
The complete time sequence data generated by the generator and the time sequence data with missing in the original data set are input into the discriminator together, and after the antagonistic training of the generator and the discriminator is completed, the generator can be considered to be capable of generating a new sample which is in accordance with the distribution of the original data set according to the original time sequence data x, namely the original time sequence data after missing value interpolation.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.

Claims (7)

1. A time series data missing value interpolation method based on an attention mechanism is characterized by comprising the following steps:
acquiring time sequence data with missing values;
inputting the time sequence data with the missing value into a generator after training, and acquiring the time sequence data after interpolation;
wherein the training of the generator comprises:
inputting the time sequence data with missing values into a generator, and acquiring complete time sequence data based on an attention mechanism;
and inputting the time sequence data with the missing values and the complete time sequence data into a discriminator, and carrying out antagonistic training on the discriminator and the generator based on the loss function.
2. The method of claim 1, wherein the generator is a de-noising self-encoder comprising an encoder unit and a decoder unit.
3. The method of claim 2, wherein the obtaining complete time series data comprises:
the encoder unit outputs a hidden vector H of original time sequence data x according to the input original time sequence data x with a preset length of m and a missing value and a random noise vector eta;
the decoder unit interpolates the missing value of the original time sequence data x according to the original time sequence data x and the hidden vector H in combination with an attention mechanism to acquire complete time sequence data
Figure FDA0003068725740000011
4. An attention mechanism based time series data deficit according to claim 3The method for interpolating the missing value is characterized in that the missing value of the original time sequence data x is interpolated to obtain complete time sequence data
Figure FDA0003068725740000012
The method comprises the following steps:
according to the original time sequence data S of the nth timen-1Combining the hidden vector H at each moment with an attention mechanism to obtain a weight factor alpha at each moment;
performing weighted average on all hidden vectors H according to the weight factor alpha at each moment;
substituting the result of the weighted average into a hyperbolic tangent function to obtain a complete value S of the original time sequence data at the nth momentn；
The complete value S of the original time sequence data at the nth momentnSubstituting the original time sequence data as the n +1 th time sequence data into the step, and circularly calculating to obtain the complete value S of the original time sequence data at all times;
acquiring complete time sequence data according to the complete value S of the original time sequence data at all the moments
Figure FDA0003068725740000021
Wherein H ═ { H ═ H1，H2，H3，…，Hi，…，Hm}，HiAs a hidden vector at time i, α ═ α1，α2，α3，…，αi，…，αm}，αiAs weighting factor at time i, S ═ S1，S2，S3，…，St，…，Sm}，StIs the complete value of the original time series data at the t-th time, S0＝HmI.e. the initial input vector of the decoder is S0。
5. The method as claimed in claim 4, wherein the obtaining the weighting factor α for each time instant comprises:
Ki＝Wk·Hi
Qn-1＝Wq·Sn-1
wherein, KiFor the ith key value, H in the attention mechanismiA hidden vector at the ith moment; qn-1Is the (n-1) th query value, S in the attention mechanismn-1Is the original time sequence data of the n-1 th time; wkAnd WqIs a parameter matrix, W, learned from training datakAnd WqThe initial value of the network is obtained by a random initialization parameter matrix and is updated by generating a loss function of a countermeasure network and a back propagation algorithm;
order:
Figure FDA0003068725740000022
wherein,
Figure FDA0003068725740000023
is a matrix KiThe transposed matrix of (2);
Figure FDA0003068725740000024
wherein alpha isiFor the weighting factor at the ith time, the softmax function is a normalized exponential function for the input
Figure FDA0003068725740000031
The values are mapped to positive outputs between 0-1 and the sum of their all outputs is 1.
6. The method as claimed in claim 4, wherein the weighted average is substituted into the hyperbolic tangent function to obtain the complete value S of the original time series data at the nth timenThe method comprises the following steps:
the result of the weighted average is:
Cn-1＝α1H1+α2H2+…+αiHi+…+αmHm
order:
Figure FDA0003068725740000032
wherein, the tanh function is a hyperbolic tangent function, and the expression is as follows:
Figure FDA0003068725740000033
ωnand bnIs a parameter, ω, learned from training datanAnd bnIs obtained from random initialization parameters and updated by generating a loss function against the network and a back propagation algorithm.
7. The method of claim 3, wherein the training the discriminator and the generator in a confrontational manner comprises:
training the discriminator with a loss function:
Figure FDA0003068725740000034
wherein D (x) is a probability value that the discriminator determines the input original time sequence data x as true,
Figure FDA0003068725740000035
for complete timing data of the discriminator pair input
Figure FDA0003068725740000036
A probability value determined to be true;
training the generator with the loss function:
Figure FDA0003068725740000037
and repeating for multiple times until the probability output by the discriminator is close to 0.5, and stopping training.
CN202110533285.5A 2021-05-17 2021-05-17 Attention mechanism-based time sequence data missing value interpolation method Active CN113298131B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110533285.5A CN113298131B (en) 2021-05-17 2021-05-17 Attention mechanism-based time sequence data missing value interpolation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110533285.5A CN113298131B (en) 2021-05-17 2021-05-17 Attention mechanism-based time sequence data missing value interpolation method

Publications (2)

Publication Number Publication Date
CN113298131A true CN113298131A (en) 2021-08-24
CN113298131B CN113298131B (en) 2022-08-05

Family

ID=77322360

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110533285.5A Active CN113298131B (en) 2021-05-17 2021-05-17 Attention mechanism-based time sequence data missing value interpolation method

Country Status (1)

Country Link
CN (1) CN113298131B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113469189A (en) * 2021-09-02 2021-10-01 国网江西省电力有限公司供电服务管理中心 Method, system and device for filling missing values of power utilization acquisition data
CN117373375A (en) * 2023-10-25 2024-01-09 昊天智显科技有限公司 On-board LED interaction system and control method thereof
WO2024087129A1 (en) * 2022-10-24 2024-05-02 大连理工大学 Generative adversarial multi-head attention neural network self-learning method for aero-engine data reconstruction

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108090558A (en) * 2018-01-03 2018-05-29 华南理工大学 A kind of automatic complementing method of time series missing values based on shot and long term memory network
CN112465150A (en) * 2020-12-02 2021-03-09 南开大学 Real data enhancement-based multi-element time sequence data filling method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108090558A (en) * 2018-01-03 2018-05-29 华南理工大学 A kind of automatic complementing method of time series missing values based on shot and long term memory network
CN112465150A (en) * 2020-12-02 2021-03-09 南开大学 Real data enhancement-based multi-element time sequence data filling method

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113469189A (en) * 2021-09-02 2021-10-01 国网江西省电力有限公司供电服务管理中心 Method, system and device for filling missing values of power utilization acquisition data
WO2024087129A1 (en) * 2022-10-24 2024-05-02 大连理工大学 Generative adversarial multi-head attention neural network self-learning method for aero-engine data reconstruction
CN117373375A (en) * 2023-10-25 2024-01-09 昊天智显科技有限公司 On-board LED interaction system and control method thereof

Also Published As

Publication number Publication date
CN113298131B (en) 2022-08-05

Similar Documents

Publication Publication Date Title
CN113298131B (en) Attention mechanism-based time sequence data missing value interpolation method
CN108763377B (en) Multi-source telemetering big data feature extraction preprocessing method based on satellite fault diagnosis
CN111582610A (en) Prediction method for family energy decomposition based on convolutional neural network
CN112884236B (en) Short-term load prediction method and system based on VDM decomposition and LSTM improvement
CN112365033B (en) Wind power interval prediction method, system and storage medium
CN114694379B (en) Traffic flow prediction method and system based on self-adaptive dynamic graph convolution
CN111461455A (en) Behavior prediction method based on correlation period attention mechanism
US20240115170A1 (en) Two-stage blood glucose prediction method based on pre-training and data decomposition
CN117786602A (en) Long-period multi-element time sequence prediction method based on multi-element information interaction
CN114037143A (en) Short-term wind power combination prediction method
CN115456044A (en) Equipment health state assessment method based on knowledge graph multi-set pooling
CN113128666A (en) Mo-S-LSTMs model-based time series multi-step prediction method
CN115766125A (en) Network flow prediction method based on LSTM and generation countermeasure network
CN116702831A (en) Hybrid short-term wind power prediction method considering massive loss of data
CN114139783A (en) Wind power short-term power prediction method and device based on nonlinear weighted combination
CN115687860A (en) Antagonistic self-encoder incomplete data prediction method fusing neural differential equation
CN116737943A (en) News field-oriented time sequence knowledge graph link prediction method
CN117094431A (en) DWTfar meteorological data time sequence prediction method and equipment for multi-scale entropy gating
CN116050478A (en) Time sequence filling method based on attention mechanism
CN116563524A (en) Glance path prediction method based on multi-vision memory unit
CN114611134A (en) Multivariate time series data privacy protection method based on negative database and deep learning
CN113537573B (en) Wind power operation trend prediction method based on double space-time feature extraction
AL-ASADI Interval-valued data analysis: a review
CN118194139B (en) Spatio-temporal data prediction method based on adaptive graph learning and nerve controlled differential equation
Kunqiao et al. Short-term load forecasting based on ELM combined model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB03 Change of inventor or designer information
CB03 Change of inventor or designer information

Inventor after: Ji Wei

Inventor after: Jin Bobin

Inventor after: Li Yun

Inventor before: Ji Wei

Inventor before: Jin Bobin

Inventor before: Li Yun

GR01 Patent grant
GR01 Patent grant