CN113591954A - Filling method of missing time sequence data in industrial system - Google Patents

Filling method of missing time sequence data in industrial system Download PDF

Info

Publication number
CN113591954A
CN113591954A CN202110818499.7A CN202110818499A CN113591954A CN 113591954 A CN113591954 A CN 113591954A CN 202110818499 A CN202110818499 A CN 202110818499A CN 113591954 A CN113591954 A CN 113591954A
Authority
CN
China
Prior art keywords
data
missing
generator
original
loss
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110818499.7A
Other languages
Chinese (zh)
Other versions
CN113591954B (en
Inventor
戴运桃
梁源
彭立章
王淑娟
曾占魁
沈继红
谭思超
赵富龙
关昊夫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin Engineering University
Original Assignee
Harbin Engineering University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin Engineering University filed Critical Harbin Engineering University
Priority to CN202110818499.7A priority Critical patent/CN113591954B/en
Publication of CN113591954A publication Critical patent/CN113591954A/en
Application granted granted Critical
Publication of CN113591954B publication Critical patent/CN113591954B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2132—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on discrimination criteria, e.g. discriminant analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Abstract

The invention discloses a method for filling missing time sequence data in an industrial system, which comprises the following steps: preprocessing data; step two: aiming at missing multivariate time sequence data, constructing and generating a confrontation network model BiGRU-BEGAN based on a border balance generation confrontation network model; step three: training a BiGRU-BEGAN network model; step four: and generating complete artificial data by using the trained BiGRU-BEGAN model and filling the original missing data. According to the method, a model combining the countermeasure network and the bidirectional cyclic neural network is generated, the real existing data information is utilized to the maximum extent, complete artificial data which accords with the characteristic distribution rule of the original missing data is generated, and then the missing data is filled completely. And performing subsequent fault classification tasks by using the filled complete data, and increasing the classification accuracy of the missing data.

Description

Filling method of missing time sequence data in industrial system
Technical Field
The invention belongs to the field of missing data filling, and relates to a method for filling missing time sequence data in an industrial system, in particular to an algorithm based on a generation countermeasure network and a bidirectional cyclic neural network.
Background
In a complex large-scale industrial system, system device failure and various fault conditions cannot be completely avoided. In the operation process, the operation state needs to be monitored by observing the parameters of the instrument, and various transient working conditions are judged. The effective system fault diagnosis technology can help an operator to master the operation condition in real time, find instrument faults in time and make effective response, thereby improving the safety of system operation.
In the process of extracting historical data of the industrial system, the condition that data acquired by partial measuring instruments are missing may influence subsequent fault diagnosis tasks. In particular, when the missing time is long and the amount of missing data is large, a large diagnosis error is caused. Therefore, before fault diagnosis, it is necessary to establish a filling model for multi-element time sequence missing data. Based on the current research situation at home and abroad, the existing missing value filling method mainly considers some specific data missing types and rarely considers the time sequence information of data, and in an industrial system, the obtained time sequence data usually has numerous data types. Therefore, the existing generation countermeasure network model is improved in consideration of complex influence factors and time sequence generated by the missing values of the monitored data in the industrial system, and a filling model for the missing time sequence data is established by fully utilizing the front and back information of the missing positions of the data in combination with the bidirectional cyclic neural network.
Disclosure of Invention
In view of the above prior art, the technical problem to be solved by the present invention is to provide a method for filling missing time series data in an industrial system, so as to solve the missing problem of multivariate time series data in fault diagnosis of the industrial system.
In order to solve the above technical problem, the method for filling missing time series data in an industrial system of the present invention comprises the following steps:
the method comprises the following steps: data preprocessing: acquiring complete data in the historical operation process of an industrial system as an original data set X for fault diagnosis, wherein n types of fault data exist in the X, and performing random loss processing on the complete data, namely, if the probability of occurrence of a loss event is specified to be p0, the probability of non-occurrence is 1-p 0;
step two: aiming at missing multivariate time sequence data, constructing and generating a confrontation network model BiGRU-BEGAN based on a boundary balance generation confrontation network model, comprising a discriminator D and a generator G, wherein the discriminator D is a self-Encoder model and comprises an Encoder and a Decoder, the generator G adopts a bidirectional cyclic neural network BiGRU, the BiGRU network model is divided into a forward cyclic neural network layer and a backward cyclic neural network layer, each training sequence is respectively provided with two gated cyclic neural networks forward and backward, the hidden state in the network depends on the hidden state and the input value at the previous moment, and the two neural networks are connected with an output layer;
step three: training a BiGRU-BEGAN network model;
step four: generating complete artificial data by using a trained BiGRU-BEGAN model and filling original missing data: inputting a characteristic vector z | t extracted from original missing data in a trained model to generate complete time sequence data, and interpolating the generated data to the corresponding missing position of the original missing data.
The invention also includes:
1. step three, the training of the BiGRU-BEGAN network model specifically comprises the following steps:
s3.1: setting parameters: batch blocksize, iteration number epoch and hyper-parameter learning for set training dataRate α, Loss1Loss function coefficient theta, updated learning rate lambda, and reconstructed loss weight coefficient k of generated datat；
S3.2: training a discriminator D: extracting the characteristic z | t of the original missing data as a low-dimensional vector input by a generator, generating data G (z | t) by the generator, and calculating a loss function L of the discriminator by using the original missing data x | t and the generated data G (z | t) as input of the discriminatorD＝L(x|t)-ktL (G (z | t)), updating the weight parameters of the arbiter according to Adam optimization algorithm:
dw←▽w[L(x|t)-ktL(G(z|t))]
wd←wd-α*Adam(wd,dw)
kt+1←kt+λ(rL(x|t)-L(G(z|t)))
kt＝min(max(k,0),1)
wherein ,ktIs a weight coefficient for generating data reconstruction loss, and λ is ktAn updated learning rate; x | t represents the original missing data, and G (z | t) represents the generated data; l (x | t), L (G (z | t)) represent the reconstruction loss of the original data and the generated data, wdWeight parameter representing the discriminator, dwIs parameter w to the arbiterdThe gradient sought;
s3.3: training generator G: generating data G (z | t) by using a low-dimensional vector z | t generated by original data characteristics as an input of a generator, and calculating a loss function L of the generatorG＝L(G(z|t))+θLoss1Updating the weight parameters of the generator according to an Adam optimization algorithm:
gw←▽w[L(G(z|t))+θLoss1]
wg←wg-α*Adam(wg,gw)
where L (G (z | t)) represents the reconstruction loss of the generated data, θ is the weight coefficient, wgRepresenting the weight parameter of the generator, gwRepresenting the gradient sought for the parameters of the generator. Loss1Representing L between real data and generated data1Norm, mathematics ofThe expression is as follows:
Loss1＝||G(z|t)-x|t||1
s3.4: training the arbiter and generator alternately to MgloableThe loss function value tends to be stable and does not decrease any more, MgloableThe loss function is as follows:
Mgloable＝L(x|t)+||rL(x|t)-L(G(z|t))||1
where r represents a diversity ratio for adjusting the balance between the generator and the arbiter, the formula is as follows:
Figure BDA0003171124030000031
when E (L (x | t)) -E (L (G (z | t))), the arbiter and generator reach equilibrium.
2. Step two, the gated recurrent neural network comprises an update gate and a reset gate, wherein the update gate ztDefining the amount of the information memorized before to the current time, and controlling the output state h of the current timetHow many history states h to keep int-1And how many candidate states at the current time are retained
Figure BDA0003171124030000032
Reset gate rtDetermining candidate states at a current time t
Figure BDA0003171124030000033
Whether it needs to rely on the network status h of the last momentt-1And the degree of dependence, the update formula of which is:
Figure BDA0003171124030000034
Figure BDA0003171124030000035
zt＝σ(Wzxt+Uzht-1+bz)
rt＝σ(Wrxt+Urht-1+br)
wherein ,
Figure BDA0003171124030000036
is current candidate state information, xtFor input at the current time, htIs the current implicit status information, ht-1Is the implicit status information of the last moment, ztIndicating an update gate, rtDenotes a reset gate, Wr,Ur,brIs the weight and bias parameter of the reset gate, Wz,Uz,bzIs to update the weight and bias parameters of the gate, Wc,Uc,bcThe method comprises the following steps of updating the weight and the bias parameter of current candidate state information, wherein sigma is a sigmoid function, a tanh function is used when a memory cell is updated, and the specific function formula is as follows:
Figure BDA0003171124030000037
the invention has the beneficial effects that: compared with the prior art, the method aims at the missing condition of the multivariate time sequence data in the industrial fault diagnosis, adopts the mode of combining the generation countermeasure network and the bidirectional cyclic neural network, establishes the BiGRU-BEGAN model, generates complete artificial data, and completely fills the original missing data. The advantages are that: (1) the model includes an encoder and a decoder, wherein the encoder performs feature extraction on missing original data, and the extracted feature vectors are used as input when generating data, thereby eliminating the step of searching for suitable low-dimensional vectors. (2) When time sequence data are generated, the bidirectional cyclic neural network BiGRU is used, when the time sequence data are processed, the relation between current output and information at the previous moment can be considered, the relation between the current output and the information at the later moment can be considered, and the generation effect and the performance in subsequent classification tasks are superior to those of the gated neural network GRU. (3) The L1 norm between the original missing data and the generated data is added into the loss function, so that the generated data can be effectively close to the original data in the aspects of numerical distance and distribution trend, and the generated complete artificial data is more consistent with the expectation.
Drawings
FIG. 1 is a diagram of a BiGRU-BEGAN generation countermeasure network framework structure according to the present invention;
FIG. 2 is a schematic diagram of a self-encoder model in a boundary balanced generation countermeasure network BEGAN;
FIG. 3 is a schematic diagram of a bi-directional recurrent neural network BiGRU expanded in time;
FIG. 4 is a diagram of a missing timing data population and fault diagnosis model architecture according to the present invention;
FIG. 5(a) is a comparison of data populated by a generator using a GRU network and a BiGRU network, respectively, with the original data on average;
FIG. 5(b) is a comparison of data populated by the generator using a GRU network and a BiGRU network, respectively, with the original data in terms of variance;
FIG. 6 is an MRE comparison between data populated by a generator using a GRU network and a BiGRU network, respectively, and the original data;
FIG. 7 is a comparison of training set classification accuracy for 2500 iterations of the data-filled data set and the original complete data set, respectively;
FIG. 8 is a comparison of the classification accuracy of the validation set of 2500 iterations of the data filled data set and the original complete data set, respectively.
Detailed Description
The invention is further described with reference to the drawings and the detailed description.
According to the method, a model combining the countermeasure network and the bidirectional cyclic neural network is generated, the real existing data information is utilized to the maximum extent, complete artificial data which accords with the characteristic distribution rule of the original missing data is generated, and then the missing data is filled completely. And performing subsequent fault classification tasks by using the filled complete data, and increasing the classification accuracy of the missing data.
The invention is realized by the following steps:
the method comprises the following steps: and (4) preprocessing data. Acquiring complete data in the historical operation process of the industrial system as an original data set X for fault diagnosis, wherein n types of fault data exist in the X, and performing random loss processing on the complete data by using an algorithm, namely, the probability of occurrence of a loss event is specified to be p0, and the probability of non-occurrence is 1-p 0.
Step two: and (3) constructing a generated confrontation network model BiGRU-BEGAN based on the boundary balance generated confrontation network BEGAN model aiming at the missing multivariate time sequence data, wherein the network structure is shown in figure 1. The main framework of the BiGRU-BEGAN model is the boundary balance generation confrontation network BEGAN, which comprises two parts, namely a discriminator D and a generator G, wherein the discriminator D is a self-encoder model. As shown in fig. 2, where Encoder denotes an Encoder and Decoder denotes a Decoder, it is essentially a mapping process. And (3) calculating the error between the real data and the reconstruction loss of the generated data by using a self-coding model as a discriminator by the boundary balance generation countermeasure network BEGAN, representing that the real data and the generated data are distributed similarly if the reconstruction loss of the real data and the generated data are similar, and optimizing the lower bound of the distance between pixel-wise losses of the real data and the generated data by the Wassertein distance to train the model. The generator G is composed of a bidirectional cyclic neural network (BiGRU) which is divided into a forward cyclic neural network layer and a backward cyclic neural network layer, the basic idea of the network is that each training sequence is respectively two Gated cyclic neural networks (GRUs) forward and backward, the hidden states in the network both depend on the hidden state and the input value at the previous moment, and the two neural networks are both connected to an output layer, fig. 3 shows a schematic diagram of the bidirectional cyclic neural network expanded in time steps.
The gated recurrent neural network (GRU) in the bidirectional recurrent neural network BiGRU model consists of an update gate (update gate) and a reset gate (reset gate), and the update gate ztDefining the amount of the information memorized before to the current time, and controlling the output state h of the current timetHow many history states h to keep int-1And how many candidate states at the current time are retained
Figure BDA0003171124030000051
Reset gate rtDetermines how to combine the new input information with the previous memory information, and determines the candidate state at the current time t
Figure BDA0003171124030000052
Whether it needs to rely on the network status h of the last momentt-1And how much, r, it needs to depend ontThe value of (A) determines the candidate state
Figure BDA0003171124030000053
For the state h at the previous momentt-1The degree of dependence of (c). It updates the formula as
Figure BDA0003171124030000054
Figure BDA0003171124030000055
zt＝σ(Wzxt+Uzht-1+bz)
rt＝σ(Wrxt+Urht-1+br)
wherein ,
Figure BDA0003171124030000056
is current candidate state information, xtFor input at the current time, htIs the current implicit status information, ht-1Is the implicit status information of the last moment, ztIndicating an update gate, rtDenotes a reset gate, Wr,Ur,brIs the weight and bias parameter of the reset gate, Wz,Uz,bzThe weights and bias parameters of the gates are updated. Wc,Uc,bcIs the weight and bias parameters for updating the current candidate state information, and σ is the sigmoid function. In order to overcome the problem of gradient disappearance, the tanh function is used when updating the memory cell, and the specific function formula is
Figure BDA0003171124030000061
Step three: training a BiGRU-BEGAN network model, comprising the following steps:
(1) and setting parameters. Setting batch blocksize, iteration times epoch and hyper-parameter learning rates alpha, Loss of training data1Loss function coefficient theta, updated learning rate lambda, and reconstructed loss weight coefficient k of generated datat。
(2) And training a discriminator D. Extracting the characteristic z | t of the original missing data as a low-dimensional vector input by a generator, generating data G (z | t) by the generator, and calculating a loss function L of the discriminator by using the original missing data x | t and the generated data G (z | t) as input of the discriminatorD＝L(x|t)-ktL (G (z | t)), updating the weight parameters of the arbiter according to Adam optimization algorithm:
Figure BDA0003171124030000062
wd←wd-α*Adam(wd,dw)
kt+1←kt+λ(rL(x|t)-L(G(z|t)))
kt＝min(max(k,0),1)
wherein ,ktIs a weight coefficient for generating data reconstruction loss, and λ is ktAn updated learning rate; x | t represents the original missing data, and G (z | t) represents the generated data; l (x | t), L (G (z | t)) represent reconstruction loss of the original data and the generated data, respectively. w is adWeight parameter representing the discriminator, dwIs parameter w to the arbiterdThe gradient is determined.
(3) A training generator G. Generating data G (z | t) by using a low-dimensional vector z | t generated by original data characteristics as an input of a generator, and calculating a loss function L of the generatorG＝L(G(z|t))+θLoss1And updating the weight parameters of the generator according to an Adam optimization algorithm:
Figure BDA0003171124030000063
wg←wg-α*Adam(wg,gw)
where L (G (z | t)) represents the reconstruction loss of the generated data, θ is the weight coefficient, wgRepresenting the weight parameter of the generator, gwRepresenting the gradient sought for the parameters of the generator. Loss1Representing L between real data and generated data1Norm whose mathematical expression is
Loss1＝||G(z|t)-x|t||1
(4) Training the arbiter and generator alternately to MgloableThe loss function value tends to be stable and does not decrease any more, MgloableThe loss function is as follows:
Mgloable＝L(x|t)+||rL(x|t)-L(G(z|t))||1
where r represents a diversity ratio for adjusting the balance between the generator and the arbiter, the formula is as follows:
Figure BDA0003171124030000071
when E (L (x | t)) -E (L (G (z | t))), the arbiter and generator reach equilibrium.
Step four: and generating complete artificial data by using the trained BiGRU-BEGAN model and filling the original missing data. Inputting a characteristic vector z | t extracted from original missing data in a trained model to generate complete time sequence data, and interpolating the generated data to the corresponding missing position of the original missing data.
Specific examples are given below with reference to specific parameters:
the invention provides a model BiGRU-BEGAN for generating an antagonistic network and a bidirectional recurrent neural network by combining boundary balance based on the generation of the antagonistic network BEGAN by the boundary balance and fully considering the upper and lower time information of time sequence data, and the overall structure is shown in figure 1. Boundary equilibrium generating countermeasureThe discriminator D used by the network bgan is a self-coder model, as shown in fig. 2. Based on the characteristic that the self-encoder can extract the data features, the original missing data x | t is input into the encoder, the output real data features p are used as implicit vectors and led to the input of a generator, and the distribution (Fc) of the guiding variable z | t can approximate to the distribution of the real data, so that the distribution trend of the generated data is close to the real data. The generator G uses a bidirectional recurrent neural network BiGRU, which is shown in fig. 3 as a bidirectional recurrent neural network that is spread along time. On the loss function, the L between the original missing data and the generated data1The norm is part of the loss function of the generator in order to approximate the generated data to the original data in terms of value and overall distribution trend. And finally, training missing data by using an end-to-end BiGRU-BEGAN model to obtain generated complete artificial data, filling original missing data, performing a classification task by using the completely filled data, and comparing the classification task with the classification effect of an original complete data set, wherein FIG. 4 is a model structure for missing time sequence data filling and fault diagnosis.
The invention relates to a filling method for industrial fault diagnosis time sequence data deletion, which comprises the following steps:
the method comprises the following steps: and (4) preprocessing data. Acquiring complete data in the operation process of an industrial system as an original data set X for Fault diagnosis, wherein the X has 6 data types, a Fault1 data set is taken as a missing processing, and an algorithm is used for carrying out random missing processing on original complete Fault1 data, namely, the probability of occurrence of a loss event is specified to be p0, and the probability of non-occurrence is 1-p0 (the random loss is set to be p0 to be 50 percent). In order to fully consider the situation of long-term missing, three groups of data with the lengths of 5s continuously, 10s continuously and 15s continuously and a group of data with five rows, five columns and a group of ten rows and ten columns are manually removed, and the missing part is occupied by 0 to obtain a required training data set;
step two: aiming at missing multivariate time sequence data, constructing a BiGRU-BEGAN model on the basis of a border balance generation countermeasure network BEGAN model, wherein the border balance generation countermeasure network BEGAN comprises a discriminator D and a generator G, the discriminator D is a self-encoder model, the generator uses a Bi-directional Gated Recurrent Unit (BiGRU), and the BiGRU network model consists of a full connection layer and a GRU network layer. Input _ size, i.e., the number of feature values of input data, hidden _ size, i.e., the number of features of hidden layers, and num _ layers, i.e., the number of layers of a recurrent neural network, of the GRU network layers are set to 1 according to the input data. Setting the Bidirectional direction in the GRU network as True, namely the Bidirectional recurrent neural network BiGRU used by the invention.
Step three: training a BiGRU-BEGAN network model, comprising the following steps:
(1) and setting parameters. Setting a path of training data, training a batch blocksize (120), an iteration number epoch (50000) and a hyper-parameter learning rate alpha (0.0001), Loss1Loss function coefficient θ (1), updated learning rate λ (0.001), and reconstructed loss weight coefficient k of generated datat(initial value is 0). During training, batch normalization processing is adopted for input data.
(2) And (5) training a discriminator. Extracting low-dimensional characteristic vector z | t of original missing data, inputting the low-dimensional characteristic vector z | t into a generator, generating data G (z | t) through the generator, taking original missing data x and generated data G (z | t) as the input of a discriminator, and calculating the loss function L of the discriminatorD＝L(x|t)-ktL (G (z | t)), updating the weight parameters of the arbiter according to Adam optimization algorithm:
Figure BDA0003171124030000081
wd←wd-α*Adam(wd,dw)
kt+1←kt+λ(rL(x|t)-L(G(z|t)))
kt＝min(max(k,0),1)
wherein ,ktIs a weight coefficient for generating data reconstruction loss, and λ is ktThe updated learning rate, x represents the original missing data, G (z | t) represents the generated data, and L (x | t) and L (G (z | t)) represent the reconstruction loss of the original data and the generated data, respectively.wdWeight parameter representing the discriminator, dwIs parameter w to the arbiterdThe gradient is determined.
(3) A training generator G. Generating data G (z | t) by using a low-dimensional vector z | t generated by original data characteristics as an input of a generator, and calculating a loss function L of the generatorG＝L(G(z|t))+θLoss1And updating the weight parameters of the generator according to an Adam optimization algorithm:
Figure BDA0003171124030000082
wg←wg-α*Adam(wg,gw)
where L (G (z | t)) represents the reconstruction loss of the generated data, θ is the weight coefficient, wgRepresenting the weight parameter of the generator, gwParameter g representing pair generatorwThe gradient is determined. Loss1Representing L between real data and generated data1Norm whose mathematical expression is
Loss1＝||G(z|t)-x|t||1
(4) Training the arbiter and generator alternately to MgloableThe loss function value tends to be stable and does not decrease any more, MgloableThe loss function is as follows:
Mgloable＝L(x|t)+||rL(x|t)-L(G(z|t))||1
where r represents a diversity ratio for adjusting the balance between the generator and the arbiter, the formula is as follows:
Figure BDA0003171124030000091
when E (L (x | t)) -E (L (G (z | t))), the arbiter and generator reach equilibrium.
Step four: and generating complete data by using the trained BiGRU-BEGAN model and filling the original missing data into the complete data. Inputting a characteristic vector z | t extracted from original missing data in a trained model to generate complete artificial time sequence data, filling the generated complete data in the corresponding missing position of the original data, establishing a 1D/2D-CNN fault diagnosis model according to the filled data set, and observing the classification effect.
Analyzing the fault diagnosis experiment result of the thermal hydraulic system of the nuclear power station:
according to the specific parameter embodiment, data come from a simulation data set for Fault diagnosis of a thermal hydraulic system of a nuclear power station, as shown in table 1, the simulation data set comprises a time sequence working condition data set and a normal state data set of 5 Fault states, a Fault1 data set is used as a filling experiment data set, and training is carried out according to a constructed BEGAN-BiGRU network model and a training mode. FIGS. 5(a) and 5(b) show the comparison of the mean and variance of the data after the generator has filled the original missing data with the BiGRU network and the GRU network, respectively, with the original complete data, and FIG. 6 shows the comparison of the MRE (mean relative error) between the original missing data and the original complete data after the generator has filled the original missing data with the BiGRU network and the GRU network, respectively, the calculation formula of the MRE is as follows
Figure BDA0003171124030000092
Wherein x' is the generated data and x is the original data. Table 2 shows the test set classification accuracy of the interpolated data and raw data of the generator using the BiGRU network and the GRU network, respectively. As can be seen from the above indexes, the data filled by the present invention is very close to the original data in mean and variance, and the MRE value between the data filled by using the BiGRU network as a generator and the original data is smaller, which reflects that the error between the filled data and the original data is smaller. In conclusion, the BiGRU-BEGAN model constructed by the invention has better effect on the data quality for the generation and filling of the time series data.
And replacing the filled Fault1 data into the original data set based on the BiGRU-BEGAN model to perform six classification tasks, and observing the Fault classification accuracy. Table 3 shows the number of data samples in the training set, test set, and validation set used in the classification process, and fig. 7 and 8 are graphs comparing the classification accuracy of the training set and validation set using the original full data and replacing the Fault1 data with the filled data of the present invention in six classifications for 2500 iterations, respectively, where the dotted line represents the classification accuracy of the original full data and the solid line represents the classification accuracy of the filled data using the model. It can be seen from the graph that the classification accuracy tends to be stable when the iteration is performed for 2500 times, and the classification accuracy of the interpolation data is very close to that of the original data, which shows that in the case that the data loss rate is 50% and the interpolation data has long-term loss, the classification accuracy of the data filled by using the model is very close to that of the original complete data, and the data filled by using the method can be used for fault classification tasks.
TABLE 1 nuclear power plant data types and data sample quantities
Figure BDA0003171124030000101
TABLE 2 test set Classification accuracy comparison
Figure BDA0003171124030000102
TABLE 3 data sample number
Figure BDA0003171124030000103

Claims (3)

1. A method for filling missing time series data in an industrial system is characterized by comprising the following steps:
the method comprises the following steps: data preprocessing: acquiring complete data in the historical operation process of an industrial system as an original data set X for fault diagnosis, wherein n types of fault data exist in the X, and performing random loss processing on the complete data, namely, if the probability of occurrence of a loss event is specified to be p0, the probability of non-occurrence is 1-p 0;
step two: aiming at missing multivariate time sequence data, constructing and generating a confrontation network model BiGRU-BEGAN based on a boundary balance generation confrontation network model, comprising a discriminator D and a generator G, wherein the discriminator D is a self-Encoder model and comprises an Encoder and a Decoder, the generator G adopts a bidirectional cyclic neural network BiGRU, the BiGRU network model is divided into a forward cyclic neural network layer and a backward cyclic neural network layer, each training sequence is respectively provided with two gated cyclic neural networks forward and backward, the hidden state in the network depends on the hidden state and the input value at the previous moment, and the two neural networks are connected with an output layer;
step three: training a BiGRU-BEGAN network model;
step four: generating complete artificial data by using a trained BiGRU-BEGAN model and filling original missing data: inputting a characteristic vector z | t extracted from original missing data in a trained model to generate complete time sequence data, and interpolating the generated data to the corresponding missing position of the original missing data.
2. The method for filling missing time series data in the industrial system according to claim 1, wherein: step three, the training of the BiGRU-BEGAN network model specifically comprises the following steps:
s3.1: setting parameters: setting batch blocksize, iteration times epoch and hyper-parameter learning rates alpha, Loss of training data1Loss function coefficient theta, updated learning rate lambda, and reconstructed loss weight coefficient k of generated datat；
S3.2: training a discriminator D: extracting the characteristic z | t of the original missing data as a low-dimensional vector input by a generator, generating data G (z | t) by the generator, and calculating a loss function L of the discriminator by using the original missing data x | t and the generated data G (z | t) as input of the discriminatorD＝L(x|t)-ktL (G (z | t)), updating the weight parameters of the arbiter according to Adam optimization algorithm:
Figure FDA0003171124020000011
wd←wd-α*Adam(wd,dw)
kt+1←kt+λ(rL(x|t)-L(G(z|t)))
kt＝min(max(k,0),1)
wherein ,ktIs a weight coefficient for generating data reconstruction loss, and λ is ktAn updated learning rate; x | t represents the original missing data, and G (z | t) represents the generated data; l (x | t), L (G (z | t)) represent the reconstruction loss of the original data and the generated data, wdWeight parameter representing the discriminator, dwIs parameter w to the arbiterdThe gradient sought;
s3.3: training generator G: generating data G (z | t) by using a low-dimensional vector z | t generated by original data characteristics as an input of a generator, and calculating a loss function L of the generatorG＝L(G(z|t))+θLoss1Updating the weight parameters of the generator according to an Adam optimization algorithm:
Figure FDA0003171124020000021
wg←wg-α*Adam(wg,gw)
where L (G (z | t)) represents the reconstruction loss of the generated data, θ is the weight coefficient, wgRepresenting the weight parameter of the generator, gwRepresenting the gradient sought for the parameters of the generator. Loss1Representing L between real data and generated data1Norm, whose mathematical expression is:
Loss1＝||G(z|t)-x|t||1
s3.4: training the arbiter and generator alternately to MgloableThe loss function value tends to be stable and does not decrease any more, MgloableThe loss function is as follows:
Mgloable＝L(x|t)+||rL(x|t)-L(G(z|t))||1
where r represents a diversity ratio for adjusting the balance between the generator and the arbiter, the formula is as follows:
Figure FDA0003171124020000022
when E (L (x | t)) -E (L (G (z | t))), the arbiter and generator reach equilibrium.
3. The method for filling missing time series data in the industrial system according to claim 1 or 2, characterized in that: step two, the gated recurrent neural network comprises an update gate and a reset gate, wherein the update gate ztDefining the amount of the information memorized before to the current time, and controlling the output state h of the current timetHow many history states h to keep int-1And how many candidate states at the current time are retained
Figure FDA0003171124020000023
Reset gate rtDetermining candidate states at a current time t
Figure FDA0003171124020000024
Whether it needs to rely on the network status h of the last momentt-1And the degree of dependence, the update formula of which is:
Figure FDA0003171124020000025
Figure FDA0003171124020000026
zt＝σ(Wzxt+Uzht-1+bz)
rt＝σ(Wrxt+Urht-1+br)
wherein ,
Figure FDA0003171124020000027
is current candidate state information, xtFor input at the current time, htIs the current implicit status information, ht-1Is the implicit status information of the last moment, ztIndicating an update gate, rtDenotes a reset gate, Wr,Ur,brIs the weight and bias parameter of the reset gate, Wz,Uz,bzIs to update the weight and bias parameters of the gate, Wc,Uc,bcThe method comprises the following steps of updating the weight and the bias parameter of current candidate state information, wherein sigma is a sigmoid function, a tanh function is used when a memory cell is updated, and the specific function formula is as follows:
Figure FDA0003171124020000031
CN202110818499.7A 2021-07-20 2021-07-20 Filling method of missing time sequence data in industrial system Active CN113591954B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110818499.7A CN113591954B (en) 2021-07-20 2021-07-20 Filling method of missing time sequence data in industrial system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110818499.7A CN113591954B (en) 2021-07-20 2021-07-20 Filling method of missing time sequence data in industrial system

Publications (2)

Publication Number Publication Date
CN113591954A true CN113591954A (en) 2021-11-02
CN113591954B CN113591954B (en) 2023-10-27

Family

ID=78248342

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110818499.7A Active CN113591954B (en) 2021-07-20 2021-07-20 Filling method of missing time sequence data in industrial system

Country Status (1)

Country Link
CN (1) CN113591954B (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113935252A (en) * 2021-12-17 2022-01-14 清华大学 Rotary machine missing fault feature recovery method and system
CN114579546A (en) * 2022-02-22 2022-06-03 南京信息工程大学 Geographical big data default value supplementing method based on multivariable countermeasure generation network
CN116543603A (en) * 2023-07-07 2023-08-04 四川大学 Flight path completion prediction method and device considering airspace situation and local optimization
CN117034142A (en) * 2023-10-07 2023-11-10 之江实验室 Unbalanced medical data missing value filling method and system
CN117034143A (en) * 2023-10-10 2023-11-10 南京邮电大学 Distributed system fault diagnosis method and device based on machine learning
CN117524318A (en) * 2024-01-05 2024-02-06 深圳新合睿恩生物医疗科技有限公司 New antigen heterogeneous data integration method and device, equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110597799A (en) * 2019-09-17 2019-12-20 上海仪电（集团）有限公司中央研究院 Automatic filling method, system and equipment for missing value of time sequence data
CN111259393A (en) * 2020-01-14 2020-06-09 河南信息安全研究院有限公司 Anti-concept drift method of malicious software detector based on generation countermeasure network
CN112328588A (en) * 2020-11-27 2021-02-05 哈尔滨工程大学 Industrial fault diagnosis unbalanced time sequence data expansion method
CN112381303A (en) * 2020-11-19 2021-02-19 北京嘀嘀无限科技发展有限公司 Task index data prediction method and system
US20210197335A1 (en) * 2019-12-26 2021-07-01 Dalian University Of Technology Data Augmentation Method Based On Generative Adversarial Networks In Tool Condition Monitoring

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110597799A (en) * 2019-09-17 2019-12-20 上海仪电（集团）有限公司中央研究院 Automatic filling method, system and equipment for missing value of time sequence data
US20210197335A1 (en) * 2019-12-26 2021-07-01 Dalian University Of Technology Data Augmentation Method Based On Generative Adversarial Networks In Tool Condition Monitoring
CN111259393A (en) * 2020-01-14 2020-06-09 河南信息安全研究院有限公司 Anti-concept drift method of malicious software detector based on generation countermeasure network
CN112381303A (en) * 2020-11-19 2021-02-19 北京嘀嘀无限科技发展有限公司 Task index data prediction method and system
CN112328588A (en) * 2020-11-27 2021-02-05 哈尔滨工程大学 Industrial fault diagnosis unbalanced time sequence data expansion method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
宋维;高超;赵?;赵燕东;: "基于LSTM的活立木茎干水分缺失数据填补方法", 林业科学, no. 02 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113935252A (en) * 2021-12-17 2022-01-14 清华大学 Rotary machine missing fault feature recovery method and system
CN114579546A (en) * 2022-02-22 2022-06-03 南京信息工程大学 Geographical big data default value supplementing method based on multivariable countermeasure generation network
CN116543603A (en) * 2023-07-07 2023-08-04 四川大学 Flight path completion prediction method and device considering airspace situation and local optimization
CN116543603B (en) * 2023-07-07 2023-09-29 四川大学 Flight path completion prediction method and device considering airspace situation and local optimization
CN117034142A (en) * 2023-10-07 2023-11-10 之江实验室 Unbalanced medical data missing value filling method and system
CN117034142B (en) * 2023-10-07 2024-02-09 之江实验室 Unbalanced medical data missing value filling method and system
CN117034143A (en) * 2023-10-10 2023-11-10 南京邮电大学 Distributed system fault diagnosis method and device based on machine learning
CN117034143B (en) * 2023-10-10 2023-12-15 南京邮电大学 Distributed system fault diagnosis method and device based on machine learning
CN117524318A (en) * 2024-01-05 2024-02-06 深圳新合睿恩生物医疗科技有限公司 New antigen heterogeneous data integration method and device, equipment and storage medium
CN117524318B (en) * 2024-01-05 2024-03-22 深圳新合睿恩生物医疗科技有限公司 New antigen heterogeneous data integration method and device, equipment and storage medium

Also Published As

Publication number Publication date
CN113591954B (en) 2023-10-27

Similar Documents

Publication Publication Date Title
CN113591954A (en) Filling method of missing time sequence data in industrial system
CN109784480B (en) Power system state estimation method based on convolutional neural network
CN110119854A (en) Voltage-stablizer water level prediction method based on cost-sensitive LSTM Recognition with Recurrent Neural Network
CN113887787B (en) Flood forecast model parameter multi-objective optimization method based on long-short-term memory network and NSGA-II algorithm
CN110689171A (en) Turbine health state prediction method based on E-LSTM
CN110942194A (en) Wind power prediction error interval evaluation method based on TCN
CN111292525A (en) Traffic flow prediction method based on neural network
CN113688869B (en) Photovoltaic data missing reconstruction method based on generation countermeasure network
CN109242200B (en) Wind power interval prediction method of Bayesian network prediction model
CN112733417A (en) Abnormal load data detection and correction method and system based on model optimization
CN112215279B (en) Power grid fault diagnosis method based on immune RBF neural network
CN114548509A (en) Multi-type load joint prediction method and system for multi-energy system
CN116840720A (en) Fuel cell remaining life prediction method
CN116431966A (en) Reactor core temperature anomaly detection method of incremental characteristic decoupling self-encoder
CN115755219A (en) Flood forecast error real-time correction method and system based on STGCN
CN115345222A (en) Fault classification method based on TimeGAN model
CN114282646B (en) Optical power prediction method and system based on two-stage feature extraction and BiLSTM improvement
CN114971090A (en) Electric heating load prediction method, system, equipment and medium
CN114118401A (en) Neural network-based power distribution network flow prediction method, system, device and storage medium
CN112149896A (en) Attention mechanism-based mechanical equipment multi-working-condition fault prediction method
CN112836876A (en) Power distribution network line load prediction method based on deep learning
CN116316629A (en) Data-driven optimal power flow calculation method considering topology feature learning
CN115759343A (en) E-LSTM-based user electric quantity prediction method and device
CN114638421A (en) Method for predicting requirement of generator set spare parts
CN115048857A (en) CNN-based transient stability limit transmission power determination method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant