CN111859798A - Process industry fault diagnosis method based on bidirectional long and short time neural network - Google Patents

Abstract

The invention discloses a process industry fault diagnosis method based on a bidirectional long-short-term neural network, comprising the following steps: S1: data set preparation: TE model starts to introduce faults from 160 sets of data, and the data set is used to establish a monitoring model; S2: feature extraction: Feature extraction is based on the gradient lifting machine to extract data features. In the direction of gradient descent, an additivity model for multiple classifiers is found; S3: Establish an experimental platform; S4: Experiment: use Keras framework to build a two-way Long and short-term neural network model, the research object is the Tennessee Eastman model; S5: experimental results. The invention utilizes the strong generalization ability of the bidirectional long and short-term neural network, can avoid the shortcomings of gradient disappearance and gradient explosion in long sequences, and solves the problem of low accuracy in process industry fault diagnosis, frequent occurrences of missed and false positives, and low generalization ability. The problem.

Description

Process industry fault diagnosis method based on bidirectional long and short time neural network

technical field

The invention relates to the technical field of fault diagnosis, in particular to a process industry fault diagnosis method based on a bidirectional long-short-term neural network.

Background technique

With the rapid development of computer technology and modern industry, the industry is becoming more and more intelligent and complex. Higher requirements are placed on the safety of the production process. The occurrence of failure will cause a large number of casualties and property losses. Therefore, it is particularly important to detect or diagnose system failures in time. The development of fault diagnosis has gone through three processes. The first stage mainly relies on the experience, senses and simple data of experts and maintenance personnel. Because the production equipment in this stage is relatively simple, the fault diagnosis and monitoring are relatively simple. In the second stage, with the development of sensor and signal technology, fault diagnosis and detection focus on instrumentation, and it has been widely used in maintenance. The third stage With the development of computer technology and artificial intelligence technology, fault diagnosis and detection have entered the intelligent stage.

According to different modeling methods, fault diagnosis technology can be divided into quantitative model, qualitative model and data-driven model. Among them, quantitative models include state estimation method, parameter estimation method and analytical redundancy method. The characteristics of these methods all require accurate mechanism models, but in industrial processes due to nonlinear, time-varying, variable coupling, time correlation, multi-mode The characteristics such as state and intermittent make it difficult to establish an accurate model. Qualitative models mainly use expert knowledge, and causality is used to diagnose and locate faults through deductive reasoning. With the development of the industry, there will be more and more unknown failures, and this method cannot obtain complete knowledge, so this method has limitations. The data-driven model mainly uses the data of normal working conditions to establish a discrete model, and uses a large amount of data to refine the model to better adapt to the model. It does not require an accurate mechanism model and is very suitable for the current complex process industry. With the development of sensor technology and data real-time storage technology, a large amount of data has been saved, which has also laid the foundation for data-driven models. Data-driven methods are mainly divided into multivariate statistical analysis, machine learning, signal processing, and information fusion.

The shortcomings of traditional process industry fault diagnosis methods are low accuracy, frequent false negatives and false positives, and low generalization ability. With the improvement of technical means, a large amount of fault data can be saved. Process industry fault diagnosis method based on neural network.

SUMMARY OF THE INVENTION

The process industry fault diagnosis method based on the bidirectional long-short-term neural network proposed by the present invention solves the problems raised in the above-mentioned background art.

In order to achieve the above object, the present invention adopts the following technical solutions:

The process industry fault diagnosis method based on bidirectional long-short-time neural network includes the following steps:

S1: Data set preparation: The TE model starts to introduce faults from 160 sets of data. The data set is used to build the monitoring model. The faults are a data set of 21 predefined faults and 1 normal working condition. The test set under normal conditions is stored in d00_te.txt, the training set is d00.txt The test set of fault 1 is saved in d01_te.txt The training set is d01.txt, ..., the test set of fault 21 is d21_te.txt, and the training set d21.txt is selected under normal conditions The 520 items in the training data set d00_txt are from the 1st to the 520th item for modeling. Here, fault 8, fault 12, fault 13, fault 17, and fault 20 are selected to verify the accuracy of the model;

S2: Feature extraction: Feature extraction uses a gradient boosting machine to extract data features, and in the direction of gradient descent, finds an additivity model composed of multiple classifiers;

S3: Establish an experimental platform: the computer model is Lenovo thinkpad, the operating system is Windows 10 Home Chinese Edition (64-bit), the CPU is (Intel) Intel(R) Core(TM) i5-8250UCPU@1.60GHz(8CPU), and the memory is 8192MBRAM establishes a test platform, and the failure experiment is carried out in the python3.7 language environment;

S4: The experiment is carried out: the Keras framework is used to build a bidirectional long-short-term neural network model, and the research object is the Tennessee Eastman model;

The fault 8, fault 12, fault 13, fault 17, and fault 20 are used for verification, and the fault data set and the normal working condition data set are extracted by gradient boosting machine;

Then, the extracted features are used as the input of the bidirectional long-short-term neural network, and the bidirectional long-short-term neural network is used for binary classification. The experimental judgment is based on the accuracy rate, and the experimental results are displayed by box plots;

S5: Experimental results: fault 8, fault 12, fault 17, and fault 20. The upper and lower limits of the accuracy rate are almost 100%, and the relatively few outliers indicate that the accuracy rate is relatively stable and fluctuates very small, and the accuracy is high. Fault 13 can be known by the fault type The type is the slow drift related fault in the reaction dynamics, which tends to fluctuate greatly, while in the bidirectional LSTM the upper limit is 100% accuracy, the median is 0.89 and the lower limit is 0.55, the experimental results show that the effect is better. .

Preferably, in the selection of the sample data in the step S4, fault 8, fault 12, fault 13, fault 17, and fault 20 in the Tennessee Eastman data set are selected as the fault set, and the fault samples are divided according to the ratio of 7:3 into training set and test set.

Preferably, the method of modeling the sample data is online modeling, the Time-Step step is 350, and the corresponding accuracy rate is obtained each time it is run in pycharm, and the obtained accuracy rate is input into the box plot code That is, the box plot of BILSTM is obtained, and the box plot can stably depict the accuracy distribution without being affected by outliers.

The beneficial effects of the invention are as follows: the use of the bidirectional long-short-term neural network has strong generalization ability, can avoid the shortcomings of gradient disappearance and gradient explosion in long sequences, and solves the problem of low accuracy in process industry fault diagnosis, and the phenomenon of false alarms and false alarms often occur. And the problem of low generalization ability; the application of bidirectional long-short-term neural network in the field of process engineering meets the requirements of technological innovation; the faults in the process industry can be diagnosed in advance through the bidirectional long-short-term neural network and corresponding measures can be formulated, which can greatly reduce the number of personnel. Casualties and property damage.

Description of drawings

Fig. 1 is the operation process schematic diagram of the forget gate of the present invention;

Fig. 2 is the operation process schematic diagram of the input gate of the present invention;

Fig. 3 is the operation process schematic diagram of the output gate of the present invention;

FIG. 4 is a schematic diagram of the bidirectional LSTM operation model of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments.

1-4, the process industry fault diagnosis method based on bidirectional long-short-term neural network includes the following steps:

S1: Data set preparation: The TE model starts to introduce faults from 160 sets of data. The data set is used to build the monitoring model. The faults are a data set of 21 predefined faults and 1 normal working condition. The test set under normal conditions is stored in d00_te.txt, the training set is d00.txt The test set of fault 1 is saved in d01_te.txt The training set is d01.txt, ..., the test set of fault 21 is d21_te.txt, and the training set d21.txt is selected under normal conditions The 520 items in the training data set d00_txt are from the 1st to the 520th item for modeling. Here, fault 8, fault 12, fault 13, fault 17, and fault 20 are selected to verify the accuracy of the model;

S2: Feature extraction: Feature extraction uses a gradient boosting machine to extract data features, and in the direction of gradient descent, finds an additivity model composed of multiple classifiers;

S3: Establish an experimental platform: the computer model is Lenovo thinkpad, the operating system is Windows 10 Home Chinese Edition (64-bit), the CPU is (Intel) Intel(R) Core(TM) i5-8250UCPU@1.60GHz(8CPU), and the memory is 8192MBRAM establishes a test platform, and the failure experiment is carried out in the python3.7 language environment;

S4: The experiment is carried out: the Keras framework is used to build a bidirectional long-short-term neural network model, and the research object is the Tennessee Eastman model;

The fault 8, fault 12, fault 13, fault 17, and fault 20 are used for verification, and the fault data set and the normal working condition data set are extracted by gradient boosting machine;

Then, the extracted features are used as the input of the bidirectional long-short-term neural network, and the bidirectional long-short-term neural network is used for binary classification. The experimental judgment is based on the accuracy rate, and the experimental results are displayed by box plots;

S5: Experimental results: fault 8, fault 12, fault 17, and fault 20. The upper and lower limits of the accuracy rate are almost 100%, and the relatively few outliers indicate that the accuracy rate is relatively stable and fluctuates very small, and the accuracy is high. Fault 13 can be known by the fault type The type is the slow drift related fault in the reaction dynamics, which tends to fluctuate greatly, while in the bidirectional LSTM the upper limit is 100% accuracy, the median is 0.89 and the lower limit is 0.55, the experimental results show that the effect is better. .

Specifically, in the experiment of step S4, the construction of the bidirectional long-short-term neural network model is based on the long-short-term neural network (LSTM). The network has the shortcomings of gradient disappearance and gradient explosion in a long sequence. On the basis of the hidden layer (hidden), the cell state (cellstate) is introduced, and the gate is mainly used to select the data. The information is passed to the next neuron, and some irrelevant information is discarded. LSTM is mainly divided into forget gate, input gate, and output gate. The three gates generally take values through the sigmoid or tanh network layer. is any value in (0,1), when the output value is 1, it means that any information of the current sample can be used for the next sample, and when the output value is 0, it means that the next sample has nothing to do with the current sample;

The forget gate is to control whether to forget the hidden cell state of the previous layer with a certain probability. Get a f(t), the output f(t) is between 0 and 1. The mapping between 0 and 1 is the memory attenuation coefficient. The closer to zero, the more information should be discarded. The closer to 1, the more should be saved and forgotten. The presence of gates is also a guarantee that LSTMs can hold long-term memory. The operation process of the forget gate is shown in Figure 1;

The output of the forget gate is:

f _t =σ(W _f h _t-1 +U _f X _t +b _f ) (1)

σ represents the activation function sigmoid, W _f , U _f , b _f represents the weight of the neural network, the forget gate will first read the output information h _t-1 of the hidden layer of the previous sample and the X _t of the current sample, and then output a specific The value of , as part of the cell state C _t-1 .

Input gate: LSTM has three inputs, the input value X(t) of the network at the current moment, the output value of the LSTM at the previous moment, and the cell state of the previous moment. The input gate is responsible for processing the current sequence position. The input gate is mainly composed of two parts. 1) The sigmoid activation function is used to output i(t). This layer determines which values we will update. 2) The tanh activation function is used to output a _t , the results of the two are multiplied to update the state of the cell. The operation process of the input gate is shown in Figure 2.

The output of the input gate is:

t=σ(W _i h _t-1 +U _i X _t +b _i ) (2)

a _t =tanh(W _m h _t-1 +U _m X _t +b _m ) (3)

Output gate: There are two outputs of LSTM, one is the output of the cellstate cell state, and the other is the output of the hidden state.

The update of the cell state is the result of the combined action of the forget gate and the output gate. It consists of two parts. The first part is the product of c _{t -1} and the forget gate f _t , and the second part is the product of the input gate it and a _t . .

c _t = c _{t - 1} f _t + i _t a _t (4)

The output of the hidden state is output by two parts, the first part is output by O _t , which is obtained by the hidden state h _t-1 of the previous sequence, the data X _t of this sequence and the activation function sigmoid, and the second part is activated by the hidden state c _t and tanh function composition. The operation process of the output gate is shown in Figure 3.

和

相关，对于与因果无关的时间序列，双向长短时神经网络利用已知的时间序列和反向的位置序列，通过正反两个方向对原序列特征提取层次，显著地提升了模型的精度，因此在处理时序问题上双向长短时神经网络往往比单向长短时神经网络效果要好，在处理复杂任务上潜力更大，双向LSTM运算模型如图4。Bi-directional LSTM (bi-directionLSTM, BiLSTM) is based on LSTM, imitating the human's associative consciousness before and after understanding the text, and introduces the deformation structure of the concept of positive and negative time directions. The BiLSTM neural network structure is shown in Figure 4. A hidden layer unit saves two pieces of information, namely A and A ^* , A participates in forward operation, A ^* participates in reverse operation, forward LSTM processing from X ₁ to X _n , and backward LSTM processing from X _n to X ₁ , The two jointly determine the output y. In this way, the output comes from both the past and the future. When the forward operation is performed, the hidden layer unit S _t and S _t-1 are correlated, and the hidden layer unit is in the reverse operation.

and

Correlation, for time series unrelated to causality, the bidirectional long-short-term neural network uses the known time series and the reverse position sequence to extract the features of the original sequence in both positive and negative directions, which significantly improves the accuracy of the model. Therefore, The bidirectional long-short-term neural network is often better than the unidirectional long-short-term neural network in dealing with timing problems, and has greater potential in dealing with complex tasks. The bidirectional LSTM operation model is shown in Figure 4.

Specifically, in the data feature extraction in step S2, the gradient boosting algorithm (Gradient Boosting) used is a combination model, and the specific algorithm process is as follows:

F ^* (x)=araminE _y,x (y,F(x)) (5)

The problem of training the model according to the existing data can be regarded as an optimization problem. The goal of optimization is to satisfy the model F ^* (x) of formula 5. At this time, the solution space of the optimization problem is in the function space instead of the numerical space. If F ^* (x) changes the form and is defined as a model with parameters, namely F(x, p), then the function optimization problem becomes a parameter optimization problem, and the formula is:

F ^* (x)=F ^* (x,p ^* )(7)

代表基于参数p的损失函数，参数优化的问题，当要估计最优参数p^*时,通常采用步进寻优的方式，即首先猜测一个参数初始值p₀,然后通过M步的迭代提升求得最优参数，最优参数p^*表示为累加形式：

其中p₀为参数初始值，{p₁,p_2,…p_m}为每步迭代的提升量，梯度下降法又称为最速下降法，是一种典型的数值优化求解方法，其思想是如果实值函数为

在a处是可微的而且有定义，那么

在梯度相反的方向下降最快即更容易找到函数的最小值，如果采用梯度下降法对最优参数p^*求解，其步骤如下：where p is the parameter of the model,

Represents the loss function based on the parameter p, the problem of parameter optimization. When the optimal parameter p ^* is to be estimated, the step-by-step optimization method is usually adopted, that is, the initial value of a parameter p ₀ is first guessed, and then it is obtained through the iterative improvement of M steps. The optimal parameter is obtained, and the optimal parameter p ^* is expressed in the cumulative form:

Among them, p ₀ is the initial value of the parameter, {p ₁ , p _2, …p _m } is the amount of improvement in each step of iteration. The gradient descent method, also known as the steepest descent method, is a typical numerical optimization solution method. The idea is to If the real-valued function is

is differentiable and defined at a, then

It is easier to find the minimum value of the function in the opposite direction of the gradient. If the gradient descent method is used to solve the optimal parameter p ^* , the steps are as follows:

For models that can be represented by parameters, the parameter optimization process of the gradient descent method is a very good solution, but it is not feasible for non-parametric models. An alternative idea is to regard F(x) as a parameter as a whole, Find the optimal function in the function space, but because the sample data is limited, the value range of the function F(x) is limited by the sample set, so the data points on the sample may not be accurately estimated. In order to solve the limitation of the function space The problem of , Friedman proposed the gradient boosting algorithm (GradientBoosting), the algorithm flow is as follows: The boosting algorithm is essentially to establish an accumulation model composed of multiple basis functions, which is expressed as:

where β _m is the expansion coefficient, b(X,γ _m ) is the basis function b ₀ (X,γ _m ) is the initialized basis function, and the optimal estimated value of F(x) F ^* (x) is obtained through M steps of improvement ), which requires

Therefore, the problem of solving (8) can be transformed into obtaining the following parameters, that is, for m=1, 2...,

It is obtained from (9) and (10) that for the boosted model F _m-1 (x), β _m b(X,γ _m ) must satisfy the constraints of (8), that is, the loss function is minimized, in other words, the basis added by each iteration The function is the basis function that reduces (8) the most. In this sense, the newly added basis function is equivalent to p _m = -ρ _m k _m in the gradient descent method, and the square error is used to measure the basis function and negative gradient. i.e. formula

Equation (11) means using βb(x _i , γ) to fit the negative gradient -km ( _xi ), using the squared error to measure the closeness, and obtaining the basis function parameter γ _m when the fitting error is the _smallest ;

其中θ＝(R_v,γ_v)为树的参数，R_v为树建立过程在输入变量x空间划分的区域，γ_v为每个端节点的赋值，θ_m对应式(12)中γ_m，以式(8)的模型表示方式，梯度提升算法可以表示为M棵树累加的形式The gradient boosting algorithm uses the decision tree as the basis function, and the tree represented as V end nodes can be expressed as

where θ=(R _v ,γ _v ) is the parameter of the tree, R _v is the area divided in the input variable x space during the tree building process, γ _v is the assignment of each end node, and θ _m corresponds to γ _m in formula (12) , in the model representation of formula (8), the gradient boosting algorithm can be expressed as the accumulation of M trees

For each base tree T _m (x, θ _m ) added to the model, θ _m is required to satisfy

From the formula (12), it can be obtained in the same way:

Use the base tree to fit the current loss function, use the current loss function gradient value and the input variable x as a new sample, and then update the gradient boosting model:

where η is a coefficient used as a regularization measure to prevent overfitting.

Specifically, in step S4, the selection of sample data and the selection of sample data: select fault 8, fault 12, fault 13, fault 17, and fault 20 in the Tennessee Eastman data set as the fault set, and divide the fault samples according to 7:3 The proportion of the fault is divided into training set and test set, and the fault number and fault type are shown in Table 1:

Table 1 Fault number and fault type

Specifically, the modeling method of the sample data is online modeling, the Time-Step step is 350, and the corresponding accuracy rate is obtained by running it once in pycharm, and the obtained accuracy rate is entered into the box plot code to get The box plot of BILSTM, the box plot can stably depict the accuracy distribution without being affected by outliers.

In summary, the use of bidirectional long and short-term neural networks has strong generalization ability, can avoid the shortcomings of gradient disappearance and gradient explosion in long sequences, and solves the problem of low accuracy in process industry fault diagnosis, frequent false negatives and false positives. The two-way long-short-term neural network has achieved remarkable results in many fields, but it is rarely used in the domestic process industry. Therefore, the application of the two-way long-short-term neural network in the process industry meets the requirements of technological innovation. ; Diagnose faults in the process industry in advance and formulate corresponding measures through the bidirectional long-short-term neural network, which can greatly reduce casualties and property losses.

The above description is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited to this. The equivalent replacement or change of the inventive concept thereof shall be included within the protection scope of the present invention.

Claims (3)

1. the process industry fault diagnosis method based on bidirectional long-short-time neural network, is characterized in that, comprises the steps: S1: Data set preparation: The TE model starts to introduce faults from 160 sets of data. The data set is used to build the monitoring model. The faults are a data set of 21 predefined faults and 1 normal working condition. The test set under normal conditions is stored in d00_te.txt, the training set is d00.txt The test set of fault 1 is saved in d01_te.txt The training set is d01.txt, ..., the test set of fault 21 is d21_te.txt, and the training set d21.txt is selected under normal conditions The 520 items in the training data set d00_txt are from the 1st to the 520th item for modeling. Here, fault 8, fault 12, fault 13, fault 17, and fault 20 are selected to verify the accuracy of the model; S2: Feature extraction: Feature extraction uses a gradient boosting machine to extract data features, and in the direction of gradient descent, finds an additivity model composed of multiple classifiers; S3: Establish an experimental platform: the computer model is Lenovo thinkpad, the operating system is Windows 10 Home Chinese version (64-bit), the CPU is (Intel) Intel(R) Core(TM) i5-8250U CPU@1.60GHz(8CPU), the memory is Build a test platform for 8192MB RAM, and conduct fault experiments in python3.7 language environment; S4: The experiment is carried out: the Keras framework is used to build a bidirectional long-short-term neural network model, and the research object is the Tennessee Eastman model; The fault 8, fault 12, fault 13, fault 17, and fault 20 are used for verification, and the fault data set and the normal working condition data set are extracted by gradient boosting machine; Then, the extracted features are used as the input of the bidirectional long-short-term neural network, and the bidirectional long-short-term neural network is used for binary classification. The experimental judgment is based on the accuracy rate, and the experimental results are displayed by box plots; S5: Experimental results: fault 8, fault 12, fault 17, and fault 20. The upper and lower limits of the accuracy rate are almost 100%, and the relatively few outliers indicate that the accuracy rate is relatively stable and fluctuates very small, and the accuracy is high. Fault 13 can be known by the fault type The type is the slow drift related fault in the reaction dynamics, which tends to fluctuate greatly, while in the bidirectional LSTM the upper limit is 100% accuracy, the median is 0.89 and the lower limit is 0.55, the experimental results show that the effect is better. . 2. the process industry fault diagnosis method based on bidirectional long-short-time neural network according to claim 1, is characterized in that, the selection of sample data in described step S4, selects fault 8 in Tennessee Eastman data set, fault 12, Fault 13, fault 17, and fault 20 are used as fault sets, and the fault samples are divided into training set and test set according to the ratio of 7:3. 3. the process industry fault diagnosis method based on bidirectional long-short-time neural network according to claim 2, is characterized in that, the modeling mode of described sample data is online modeling, Time-Step step size is 350, every time in Running it once in pycharm will get its corresponding accuracy rate. Enter the obtained accuracy rate into the box plot code to get the BILSTM box plot. The box plot can be unaffected by outliers and can stably depict the accuracy distribution. Happening.

Images