CN110941928B - Rolling bearing residual life prediction method based on dropout-SAE and Bi-LSTM - Google Patents

Abstract

A rolling bearing RUL prediction method based on dropout-SAE and Bi-LSTM belongs to the field of prediction of bearing running states. The invention aims to solve the problems of long model training time and low prediction accuracy in the conventional rolling bearing RUL prediction method. The invention provides an improved SAE, namely dropout-SAE performs unsupervised deep characteristic self-adaptive extraction on a vibration signal of a rolling bearing, the network applies a new Tan activation function to replace the original sigmoid activation function, and the dropout method is adopted to realize the sparsity of the network; meanwhile, considering that the method for predicting the residual service life of the rolling bearing generally only considers the past information and ignores the future information, a bidirectional long-time memory network is introduced to serve as a prediction model of the rolling bearing RUL. The experimental results on 2 bearing data sets show that the prediction method not only can improve the convergence rate of the model, but also has higher accuracy.

Description

Rolling bearing residual life prediction method based on dropout-SAE and Bi-LSTM

Technical Field

The invention relates to a method for predicting the residual life of a rolling bearing, belonging to the field of prediction of the running state of a bearing.

Background

The rolling bearing is the most commonly used and vulnerable key part of the rotating equipment, and the performance of the whole equipment is often directly influenced by the operating state of the rolling bearing^[1]. Therefore, the method has very important practical significance for predicting the remaining service life (RUL) of the rolling bearing.

Feature extraction is an important prerequisite for rolling bearing RUL prediction. In recent years, deep learning has gained wide attention with strong adaptive feature extraction capability and nonlinear function characterization capability, and provides a new solution for the feature extraction of the vibration signal of the rolling bearing^[2]. Document [3]An improved deep belief network is provided, which directly uses the original vibration signal of a rolling bearing as a networkAnd (4) network input, namely abstract representation from a low layer to a high layer, so that the purpose of deeply mining essential characteristics of data is achieved. Documents [4 to 6]The data local abstract information is directly and automatically extracted from the rolling bearing vibration signal by using the special structural characteristics of local convolution, weight sharing, down sampling and the like of the convolutional neural network, and the deep mining of the vibration signal characteristics is realized. Although the above research utilizes a deep learning method to simplify the complex feature extraction process and excavate the deep essential features of the vibration signal, the network structure parameters still need a large amount of labeled data to perform supervised fine tuning, and the labeled data is difficult to obtain in practice.

Unsupervised feature learning can automatically extract data intrinsic features from unlabeled data^[7]The method has great advantages under the condition that the label data are few and difficult to obtain. Sparse auto-encoder (SAE), a typical unsupervised feature learning model, can realize efficient learning of concise intrinsic feature expressions from a large amount of unlabeled data^[8]At present, the method is successfully popularized to various application occasions with limited marking data^[9]. The problem of gradient disappearance is easily caused by the fact that sigmoid is adopted as an activation function in the traditional SAE, and limitation exists in the aspect of rolling bearing feature extraction by adopting KL divergence to carry out sparsity constraint.

On the basis of the feature extraction, it is the final target to perform rolling bearing RUL prediction. On the basis of obtaining a performance degradation characteristic value of a bearing, considering the advantages of a cyclic neural network in a deep learning algorithm in the aspect of processing time series data, and taking a long-short-term memory (LSTM) as a bearing performance degradation curve construction method. The method of constructing the bearing performance degradation curve using LSTM is to integrate the "past" information and then aid in processing the current information. However, the conventional LSTM does not consider that the fading process of the rolling bearing is actually a continuous process with a time dependency, and the processing of the current information is necessary to integrate the "future" information ^[10]。

In conclusion, the gradient vanishing problem is easily caused by adopting sigmoid as an activation function in the traditional SAE, and the limitation on the aspect of rolling bearing feature extraction is realized by adopting KL divergence to carry out sparsity constraint; the conventional rolling bearing RUL prediction method considers only past information and ignores future information, and these problems are not solved yet.

Disclosure of Invention

The invention aims to solve the problems of long model training time and low prediction accuracy of the conventional rolling bearing RUL prediction method, and further provides a rolling bearing RUL prediction method based on dropout-SAE and Bi-LSTM.

The technical scheme adopted by the invention for solving the technical problems is as follows:

a rolling bearing RUL prediction method based on dropout-SAE and Bi-LSTM is realized by the following steps:

step one, data preprocessing: firstly, Fourier transform is carried out on the original time domain vibration data of the rolling bearing, and the original time domain vibration data of the rolling bearing is converted into a frequency domain; then, carrying out linear function normalization processing on the obtained product; finally, dividing a training set and a test set;

step two, deep layer feature extraction: training network parameters by taking the training set as the input of dropout-SAE, and extracting the characteristics capable of representing the bearing degradation trend;

step three, constructing a Bi-LSTM model: taking a plurality of features extracted by dropout-SAE on a training set as the input of a Bi-LSTM network, taking the ratio p of the current service life feature point number to the full-life feature point number, namely the service life percentage as the label output of the current service life, and training after setting related network parameters to obtain a Bi-LSTM model;

Step four, model optimization: respectively adopting Adam, RMSProp and SGDM optimization algorithms to optimize the Bi-LSTM model, respectively calculating the Mean Square Error (MSE), the Mean Absolute Error (MAE), the Mean Absolute Percentage Error (MAPE), the Mean square percentage Error (MAPE) and the Root Mean Square Error (RMSE) of the Bi-LSTM model under each optimization algorithm, selecting the optimization algorithm with the minimum sum of the 5 errors to optimize the Bi-LSTM model to obtain optimized Bi-LSTM model parameters, and applying Dropout technology to prevent overfitting;

step five, verifying a test set: processing the test set by adopting a data preprocessing and feature extraction method which is the same as that of the training set, inputting the extracted features into the optimized Bi-LSTM network model, and predicting the p value of the known data;

step six, predicting RUL: linear fitting is carried out on the predicted p-value curve of the known data through a linear function to obtain the p-value trend of each future point, the setting of the p-value in the step (3) shows that when the p is 1, the bearing fails, namely the full life is achieved, and the full life L is utilized _qMinus the current life L_dThe RUL of the ith bearing can be obtained, as shown in equation (22):

RUL_i＝L_q-L_d (22)

RUL by predicted remaining Life_iAnd true lifetime actriul_iError between Er_iTo reflect the performance of the model for predicting the remaining life, as shown in equation (23):

further, in step two, the dropout-SAE is an improved SAE, and sparsity constraint is performed on the network by replacing the original sigmoid activation function with a Tan activation function and replacing the original KL divergence with a dropout mechanism.

Further, in step two, the network structure of dropout-SAE is: the dropout-SAE network structure is 2048-200-2048, wherein the number of nodes of the input layer corresponds to 2048 of normalized bearing frequency domain amplitude signals, the number of nodes of the hidden layer 200 corresponds to the finally extracted feature number, and the output layer and the input layer have the same dimension.

Further, in the second step, the specific process of deep feature extraction is as follows: the deep feature extraction mainly comprises two stages of pre-training and global parameter fine adjustment: in the pre-training stage, network parameters are initialized through unsupervised layer-by-layer pre-training; in the global parameter fine-tuning stage, the original input is used as a label, and the network parameters are fine-tuned through a BP back propagation algorithm and a gradient descent method so as to optimize dropout-SAE.

Further, the Bi-LSTM network consists of one hidden layer, the number of hidden states of the network is chosen to be 150, the initial learning rate is set to 0.01 and the weight matrix W and the bias b are initialized randomly, using the Root Mean Square Error (RMSE) as its loss function.

Further, in step three, a plurality of features extracted from the training set by dropout-SAE are used as the input of the Bi-LSTM network, and the specific process is as follows:

and the forward LSTM and the backward LSTM respectively carry out sequence and reverse processing on the characteristic sequence to obtain hidden layer states with opposite time sequences, and the hidden layer states are connected to obtain the same output.

Furthermore, in the third step, a Bi-LSTM model is adopted to consider past and future information of data at the same time, two hidden layer states with opposite time sequences are obtained through a forward LSTM and a backward LSTM, and then the hidden layer states are connected to obtain the same output; the forward LSTM and the backward LSTM can respectively acquire the past information and the future information of the input sequence, and the hidden state H of the Bi-LSTM at the time t_tInvolving the forward direction

And backward

Wherein: t is the sequence length, ct represents a memory unit, xt represents an input vector at the T moment, and represents the deep characteristics of the rolling bearing vibration signal obtained by SAE extraction; ht is the hidden state at time t.

The invention has the following beneficial technical effects:

the gradient vanishing problem is easily caused by the fact that sigmoid is adopted as an activation function in the traditional SAE, and the regression problem that the sparsity constraint of the network is not suitable for processing the rolling bearing feature extraction is solved by adopting the KL divergence. Based on the method, the activation function of SAE is improved, a new Tan function is used for replacing an original sigmoid function, and a dropout mechanism is used for replacing KL divergence to realize the sparsity of the network, so that an improved SAE, namely dropout-SAE is formed. And unsupervised deep feature self-adaptive extraction is carried out on the vibration signal of the rolling bearing by utilizing dropout-SAE, and supervised fine adjustment is not needed to be carried out by manually designing a label. On the basis of the feature extraction, it is the final target to perform rolling bearing RUL prediction. The invention considers that the fading process of the rolling bearing is actually a continuous change process with a front-back dependency relationship in time, and the processing of the current information is necessary to integrate the future information^[10]. Document [10 ]]A Bi-directional long-short-term memory (Bi-LSTM) network is used for short-term load prediction and good experimental effect is achieved. Document [11]The Bi-LSTM is applied to the video description to fully preserve global time and visual information. Therefore, the feasibility and superiority of Bi-LSTM in time series treatment can be proved. Therefore, the invention obtains more accurate RUL prediction results by introducing Bi-LSTM to realize the full utilization of past and future information.

The invention improves SAE, provides a dropout-SAE network, and uses the deep structure thereof to perform unsupervised self-adaptive feature extraction on the original bearing vibration signal, and uses the extracted deep feature as the performance degradation feature of the rolling bearing. In order to further solve the problem that the traditional rolling bearing RUL prediction method only considers the past information and ignores the future information, Bi-LSTM is introduced to complete the current life prediction of the rolling bearing. And finally, fitting the current service life by utilizing a linear function to realize the RUL prediction of the rolling bearing.

Aiming at the problem that the gradient is easy to disappear when a Sparse Auto Encoder (SAE) adopts a sigmoid activation function, a new Tan function is used for replacing the original sigmoid function; aiming at the limitation of the SAE in the regression prediction aspect of the sparsity constraint by adopting KL divergence, a dropout mechanism is used for replacing KL divergence to realize the sparsity of the network, so that an improved SAE (dropout-SAE) is formed. Unsupervised deep-layer feature self-adaptive extraction is carried out on the rolling bearing vibration signals by utilizing dropout-SAE, and supervised fine adjustment is not needed to be carried out by manually designing tags. Meanwhile, considering that the remaining service life (RUL) prediction method of the rolling bearing generally only considers the past information and ignores the future information, a Bi-directional long-short-term memory network (Bi-LSTM) is introduced to construct a prediction model of the RUL of the rolling bearing. Experimental results on 2 bearing data sets show that the rolling bearing RUL prediction method based on dropout-SAE and Bi-LSTM can improve the convergence rate of the model and has high accuracy.

Drawings

FIG. 1 is a schematic diagram of the AE structure;

FIG. 2 is a graph of the sigmoid function and its derivative function;

FIG. 3 is a plot of the Tan function and its derivative function;

FIG. 4 is a schematic diagram of the internal structure of an LSTM cell;

FIG. 5 is an expanded view of a Bi-LSTM network;

fig. 6 is a flowchart of rolling bearing RUL prediction;

fig. 7 is a time domain vibration signal of the bearing 1_1 and a frequency domain amplitude spectrogram after normalization, wherein: (a) time domain vibration signals, (b) normalized frequency domain amplitude spectra;

FIG. 8 is a characteristic trend graph of a part of the bearing 1_ 1;

fig. 9 is a graph of the fitting result and the fitting error of the method of the present invention for predicting the current p-value of the bearing 1_7, wherein: (a) representing the fitting result, (b) representing the fitting error;

FIG. 10 is a graph showing the RUL prediction result of the bearing 1_7 according to the method of the present invention,

figure 11 is a comparison of the time spent in feature extraction (PHM2012 bearing data set),

FIG. 12 is a graph of the predicted results for bearings 1_7RUL for three scenarios, where: (a) scheme one, (b) is scheme two (c), scheme three;

FIG. 13 is a comparison of the time spent in feature extraction (XJTU-SY bearing dataset).

Detailed Description

The implementation of the method for predicting the residual life of the rolling bearing based on dropout-SAE and Bi-LSTM according to the present invention is described as follows with reference to the accompanying drawings 1 to 13:

1dropout-SAE model

An Auto Encoder (AE) is a three-layer neural network that attempts to learn a function by an unsupervised learning algorithm so that an output value is close to an input value, and has a structure as shown in fig. 1, including an input layer, a hidden layer, and an output layer^[12]。

The input layer and the hidden layer form an encoding network, and the encoding process is to convert input x containing n data into { x ═ x₁,x₂,…,x_nConverting to a hidden layer expression h ═ h with high-level features₁,h₂,…,h_n}; the hidden layer and the output layer form a decoding network, and the decoding process is that a hidden layer vector set is reversely transformed into a reconstructed data set y which has the same dimension as the input data, wherein the reconstructed data set y is { y ═ y₁,y₂,…,y_n}。

The encoding process and the decoding process can be expressed as:

h＝S_f(b₁+W₁x) (1)

y＝S_g(b₂+W₂h) (2)

wherein: s_fActivating a function for encoding; s_gActivating a function for decoding; b₁、b₂Is an offset; w₁、W₂Is a weight matrix, W₂＝W₁ ^T。

AE is determined by optimizing parameter set θ ═ W₁,W₂,b₁,b₂-to minimize the reconstruction error,the loss function is:

activation function S in formula (1) and formula (2)_fAnd S_gGenerally, a sigmoid function is adopted, and the sigmoid function and the derivative function form thereof are as follows:

as can be seen from the sigmoid function and its derivative function image of fig. 2, when the output of the neural network is large, the sigmoid derivative becomes very small, resulting in slow model convergence, i.e. gradient vanishing.

To solve this problem, a new activation function is used herein, called Tan function, which is in the form of:

as can be seen from the Tan function and its derivative image in fig. 3, the minimum value of the Tan derivative is about 0.64, and does not appear to be 0, so that the gradient disappears, and the network model converges more quickly.

SAE adds sparse penalty term on the basis of loss function of AE, can better express input data structure, avoids network overfitting. The conventional SAE adopts KL divergence as a sparsity penalty term, which is defined as:

wherein: beta represents the activation parameter of the weight value, m represents the number of neurons in the hidden layer,

represents the average activation degree of the jth neuron of the hidden layer, rho is a sparse parameter, a_j(x) Representing the degree of activation of the jth neuron of the hidden layer given an input x.

The loss function of SAE after adding the sparse penalty term is:

J(θ)＝J_MSE(θ)+J_sparse(θ) (11)

however, the above classification problem that the real value is 0 or 1 is only applicable to the above classification problem that the KL divergence is adopted as the sparse constraint term of SAE, and the deep layer feature to be extracted for the rolling bearing is [0,1 ]]A regression problem of a certain value therebetween, cannot be found

And punishing the network as a basis. Thus, the dropout mechanism is employed herein to achieve sparsity for SAE.

The specific method is to introduce a dropout layer before an activation function in the encoding and decoding processes, and perform mask processing during encoding and decoding, so that the activation values of part of neurons in AE are set to be 0 with a certain probability q (usually 0.5)^[13]The formula is as follows:

wherein: z represents the input of the original activation function, and z' represents the input of the activation function after the dropout layer is thinned.

Once a neuron is set to 0, it means that the corresponding neuron is removed from the network, and its connection weight and bias are not updated (in a dormant state) in the learning, that is, learning using an own network smaller in scale than the original network. At this time, equations (1) and (2) are still satisfied for the encoding and decoding processes.

2Bi-LSTM model

The LSTM model consists of three gates (input gate i)_tForgetting door f_tOutput gate o_t) And a memory cell (c)_t) The three gates are used for selectively inputting, outputting and forgetting the internal memory, so that the problem of gradient explosion or gradient disappearance can be effectively solved. The internal structure of the LSTM cell is shown in fig. 4.

A complete LSTM may be represented as:

f_t＝σ(W_f·X+b_f) (14)

i_t＝σ(W_i·X+b_i) (15)

o_t＝σ(W_o·X+b_o) (16)

wherein: x is the number of_tAn input vector representing time t; h is_tIs a hidden state at time t; w and b are the weight and the bias of the LSTM respectively, and are model training parameters; σ is an activation function sigmoid;

Is a point-by-point product.

While LSTM can solve the long term dependency problem, it does not take advantage of future information. The present invention therefore exploits the Bi-LSTM model while considering both past and future information of the data, developed as shown in fig. 5. The working principle is as follows: two hidden layer states with opposite time sequences are obtained through the forward LSTM and the backward LSTM, and then the hidden layer states are connected to obtain the same output. The forward LSTM and backward LSTM can obtain the past information and future information of the input sequence respectively^[10]. Hidden state H of Bi-LSTM at time t_tInvolving a forward direction

And backward

Wherein: t is the sequence length.

3 rolling bearing RUL prediction method and process

A flow chart of the prediction method of the RUL based on the dropout-SAE and Bi-LSTM rolling bearing is shown in FIG. 6. The method comprises the following specific steps:

(1) data preprocessing: firstly, Fourier transform is carried out on the original time domain vibration data of the rolling bearing, and the original time domain vibration data of the rolling bearing is converted into a frequency domain; then, carrying out linear function normalization processing on the obtained product; and finally, dividing the training set and the test set.

(2) Deep layer characteristic extraction: the method comprises the following steps that a training set is used as input of dropout-SAE to carry out unsupervised deep feature extraction, and the method mainly comprises two stages of pre-training and global parameter fine adjustment: in the pre-training stage, network parameters are initialized through unsupervised layer-by-layer pre-training; in the global parameter fine-tuning stage, the original input is used as a label, and network parameters are fine-tuned through a BP back propagation algorithm and a gradient descent method, so that the whole model is optimized, the nonlinear function mapping capability of the network is further improved, and better characteristic expression is obtained ^[2]And finally extracting the characteristics capable of representing the degradation trend of the bearing.

(3) Constructing a Bi-LSTM model: a plurality of features extracted from a training set by dropout-SAE are used as input of a Bi-LSTM network, the ratio p of the current service life feature point number to the full-life feature point number, namely the service life percentage, is used as the label output of the current service life, and relevant network parameters are set for training.

(4) Model optimization: by calculating the Mean Square Error (MSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), Mean square percentage Error (MAPE), root Mean square percentage Error (RMSE) and the sum of the above 5 errors of the training model as evaluation criteria, 3 common optimization algorithms Adam, RMSProp and SGDM proposed in the document [14] were compared to obtain the optimal Bi-LSTM model parameters, and Dropout technique was applied to prevent overfitting.

(5) And (3) test set verification: and (3) processing the test set by adopting a data preprocessing and feature extraction method which is the same as that of the training set, inputting the extracted features into a trained Bi-LSTM network model, and predicting the p value of the known data.

(6) RUL prediction: and performing linear fitting on the predicted p-value curve of the known data by a linear function to obtain the p-value trend of each future point. As can be seen from the setting of the value of p in step (3), when p is 1, the bearing fails, i.e., the full life is reached. By total life L_qMinus the current life L_dThe RUL of the ith bearing can be obtained, as shown in formula (22):

RUL_i＝L_q-L_d (22)

(7) RUL by predicted remaining Life_iAnd true lifetime actriul_iError between Er_iTo reflect how good the model predicts the remaining life. As shown in equation (23):

4, application and analysis, and verification of the technical effects of the invention:

in order to verify the rolling bearing RUL prediction method based on dropout-SAE and Bi-LSTM, which is provided by the invention, a PHM2012 bearing data set is selected^[15]Verification was performed as experimental data. The data set is acquired by two acceleration sensors in the horizontal direction and the vertical direction, and is recorded every 10s, the recording time is 0.1s, and the sampling frequency is 25.6 kHz. The present invention uses vibration data in the horizontal direction.

As shown in Table 1, the invention selects the total life data (all data from the start of operation to complete failure of the rolling bearing) of 6 bearings of the bearings 1_1, 1_2, 2_1, 2_2, 3_1 and 3_2 as a training set to train, and selects the non-total life data (data from the start of operation to a certain time point of the rolling bearing) of 11 bearings of the rest bearings 1_3, 1_4, 1_5, 1_6, 1_7, 2_3, 2_4, 2_5, 2_6, 2_7 and 3_3 as a test set to carry out the RUL prediction experiment.

TABLE 1 Experimental data (PHM2012 bearing data set)

The experiment preprocesses the original time domain signals of 17 bearings in the training set and the test set. Taking the bearing 1_1 as an example, a sample time domain vibration signal and a corresponding normalized frequency domain amplitude signal within a 0.1s acquisition time period are shown in fig. 7.

Inputting the normalized bearing frequency domain amplitude signal into dropout-SAE for unsupervised self-monitoringAnd (4) adapting to feature extraction. Through a large number of experiments, the dropout-SAE network structure is selected to be 2048-200-2048, wherein the number of input layer nodes corresponds to 2048 points of the normalized bearing frequency domain amplitude signal, and the number of hidden layer nodes 200 corresponds to the finally extracted feature number. In order to eliminate the influence of oscillation on health indexes and ensure the unchanged characteristic of the original characteristic curve, the obtained characteristic curve is subjected to smooth filtering treatment^[16]. Any 5 features are selected from the 200-dimensional features extracted from the bearing 1_1, and the trend curve of the 5 features is shown in fig. 8.

As can be seen from fig. 8, the deep features extracted by SAE have a good monotonous trend as a whole (some features are in an upward trend in the whole life cycle of the bearing, and the other features are in a downward trend), and can well characterize the fading process of the whole life cycle of the bearing.

A training stage: deep features of the bearings 1_2, 2_1, 2_2, 3_1 and 3_2 extracted by SAE are input into a Bi-LSTM network model, and a Bi-LSTM prediction model is trained by taking a real p value as the output of the model. The Bi-LSTM network consists of one hidden layer, and the number of hidden states of the network was chosen to be 150 by iterative experiments. Using Root Mean Square Error (RMSE) as its loss function, the initial learning rate is set to 0.01 and the weight matrix W and bias b are initialized randomly. The errors and the sum of the errors of the training models under the three optimization algorithms Adam, RMSProp and SGDM are calculated as shown in table 2. Therefore, Adam serving as a self-adaptive optimization algorithm can minimize the extracted model error, and meanwhile, the Adam algorithm can iteratively update the neural network weights based on training data, so that the Adam optimizer is used for gradient optimization. In addition, the present invention utilizes Dropout technology^[19]To prevent overfitting and improve the performance of the model, the Dropout value was set to 0.1 experimentally.

TABLE 2 training errors for the three optimization algorithms

And (3) a testing stage: taking the test bearing 1_7 as an example, as in the training stage, inputting the deep features of the bearing 1_7 extracted by dropout-SAE into a trained Bi-LSTM prediction model, and predicting the current p value. The fitting result between the predicted value and the actual value is shown in fig. 9(a), and fig. 9(b) shows the corresponding fitting error.

And fitting the predicted current p value of the bearing 1_7 by using a linear function to obtain the trend of the future p value, so as to obtain the RUL prediction result of the rolling bearing 1_7, as shown in FIG. 10.

Each characteristic point of each bearing represents a life time of 10s according to the actual sampling data characteristics of the bearing. Given that 1502 points are not available for the bearing 1_7 and 2259 points are available for the full life data, it can be seen from fig. 10 that 2282 points are available for the predicted full life data points when the bearing reaches the failure threshold, i.e. p is 1. The predicted RUL was calculated from the formula (22) as (2282-.

In order to evaluate the uncertainty of the RUL prediction, interval estimation was performed on the RUL using the method of document [17], a confidence interval of 95% confidence level was set around the predicted value, and the upper and lower limits were extracted. Similar to the RUL prediction described above, these values can also be extrapolated to failure thresholds to obtain upper and lower confidence intervals for RUL prediction [7530s, 8070s ].

In order to verify the advantages obtained by dropout-SAE over SAE in convergence speed, the time consumed for deep feature extraction of rolling bearings using SAE and dropout-SAE, respectively, is shown in FIG. 11.

As can be seen from fig. 11, in 17 bearing feature extraction experiments, the time consumed for dropout-SAE feature extraction is shorter than that consumed for SAE feature extraction, which proves that dropout-SAE has a faster convergence rate than SAE.

In order to verify the effectiveness of the proposed prediction methods based on dropout-SAE and Bi-LSTM, three other schemes were set up to perform comparative experiments with the prediction method proposed by the present invention, as shown in Table 3.

The prediction method provided in Table 3 and other 3 schemes

According to the experimental process of carrying out the RUL prediction on the bearing 1_7 by the method provided by the invention, the RUL prediction on the bearing 1_7 can be obtained by other three schemes in the same way, the result is shown in FIG. 12, and the specific prediction error is shown in Table 4.

To further demonstrate the effectiveness of the method of the present invention, rolling bearing RUL predictions were evaluated using the RUL prediction accuracy scoring formula (24) for the PHM2012 bearing data set, with the average scoring results shown in table 4.

Wherein, A_iIs defined as follows:

similarly, Table 4 shows the RUL prediction error and average score for the other 10 bearings in the database, and gives the comparison results with references [18] and [19 ].

TABLE 4 comparison of RUL prediction results for different bearings (PHM2012 bearing data set)

The results of comparative experiments based on the dropout-SAE and Bi-LSTM prediction methods and other 3 schemes provided by the invention show that: under the same LSTM and Bi-LSTM prediction models, the average prediction errors of the dropout-SAE feature extraction model are respectively reduced by 5.56% and 1.25% compared with the average prediction errors of the SAE feature extraction model, and the average scores are respectively improved by 0.052 and 0.054, so that the dropout-SAE feature extraction model is more superior. Under the condition of the same dropout-SAE feature extraction model, the average error of the Bi-LSTM prediction model is reduced by 3.32% compared with the average error of the LSTM prediction model, the average score is improved by 0.099, and the Bi-LSTM prediction model can be proved to have greater superiority. Overall, the methods presented herein have lower error and higher score than all of scenario one, scenario two, and scenario three. Meanwhile, compared with the average prediction error of the document [18] and the document [19], the method disclosed by the invention reduces 25.99 percent and 46.75 percent respectively, and improves the average score by 0.313 and 0.511 respectively. The effectiveness of the method proposed herein in the prediction of rolling bearing RUL is thus further demonstrated.

To validate the generalization ability of the proposed dropout-SAE and Bi-LSTM based models, the Sigan university of transportation XJTU-SY bearing dataset was used ^[20]As new experimental data. The data set is acquired by two acceleration sensors in the horizontal direction and the vertical direction, the data set is recorded every 1min, the recording time is 1.28s each time, the sampling frequency is 25.6kHz, and vibration data in the horizontal direction are utilized. Emulating PHM2012 bearing data set^[17]The bearing was divided into non-life and life cycle data as shown in table 5. The total life data of 6 bearings 1_1, 1_2, 2_1, 2_2, 3_1 and 3_2 are selected as a training set to be trained, and the non-total life data of 9 bearings of the rest bearings 1_3, 1_4, 1_5, 2_3, 2_4, 2_5, 3_3, 3_4 and 3_5 are selected as a test set.

TABLE 5 Experimental data (XJTU-SY bearing data set)

Meanwhile, in order to simplify the experimental process, the middle 4096 points of each 1.28s collected data are selected as data samples, and SAE deep feature extraction, Bi-LSTM model construction, RUL prediction and the like are carried out according to the same experimental method of the PHM2012 bearing data set. The results of the specific experiments are shown in fig. 13 and table 6.

TABLE 6 comparison of RUL predicted results for different bearings (XJTU-SY bearing data set)

The same conclusion as that of the PHM2012 bearing data set can be drawn from the comparison of the experimental results of the above 4 methods, so that it can be further illustrated that the proposed method has better generalization capability.

5 conclusion

(1) Aiming at the problem that the gradient disappears easily caused by adopting sigmoid as an activation function in the traditional SAE, a new Tan function is used for replacing the original sigmoid function; aiming at the limitation of the SAE in the aspect of extracting the rolling bearing features by adopting KL divergence for sparsity constraint, a dropout mechanism is used for replacing the KL divergence to realize the sparsity, and an improved SAE, namely dropout-SAE is formed. And carrying out unsupervised feature self-adaptive extraction on the vibration signal of the rolling bearing by utilizing dropout-SAE, thereby obtaining deep features with certain trend and capable of representing the degradation trend of the bearing.

(2) Aiming at the problem that the standard LSTM processes the sequence in time sequence and only considers the past information and ignores the future information, a Bi-LSTM network is introduced, the same output of the Bi-LSTM network is connected with two LSTM networks with opposite time, and the past data information and the future data information of the input sequence are respectively acquired. Meanwhile, in order to obtain a better prediction result, the Bi-LSTM prediction model is optimized by using an Adam algorithm and a Dropout technology.

(3) The two data set experiments prove that compared with the traditional SAE model, the dropout-SAE model has higher convergence rate, the extracted deep characteristics are more superior in prediction of the RUL of the rolling bearing by combining with the Bi-LSTM model, and meanwhile, compared with other two documents, the prediction error is reduced by more than 25%, and the score is improved by more than 0.313.

The rolling bearing RUL prediction has advanced prediction (Er)_i>0) And lag prediction (Er)_i<0) Both results, lead predictions of equipment in industrial production life carry less risk than lag predictions. Thus, "leading prediction" has more practical significance than "lagging prediction". Although the invention improves the prediction accuracy to a certain extent,but also exacerbates the problem of "lag prediction" and therefore, optimization of the RUL prediction model will be the focus of the next research effort.

The references cited in the present invention are as follows:

[1] the performance stability and reliability of the rolling bearing are analyzed [ J ] in the study of instruments and meters, 2017,38(06): 1421-.

XIAX T,CHANG Z,YE L,et al.Stability and reliability analysis of rolling bearing performance[J].Chinese Journal of Scientific Instrument,2017,38(06):1421-1431.

[2] Zhao Guang Ye, Liu Xiao Yong, Jiang Zedong, etc. deep learning-based unsupervised construction method of bearing health factors [ J ] Instrument and Meter bulletin, 2018,39(06):82-88.

ZHAO G Q，LIL X Y，JIANG Z D，et al.Unsupervised health indicator of bearing based on deep learning[J].Chinese Journal of Scientific Instrument，2018，39(06)：82-88.

[3] The application of the Yangyu, Zhana, Cheng Jun.Total parameter dynamic learning deep belief network in the life prediction of the rolling bearing [ J ] vibration and impact, 2019,38(10): 199-.

YANG Y,ZHANG N,CHENG J S.Global parameters dynamic learning deep belief networks and its application in rolling bearing life prediction[J].Journal of Vibration and Shock,2019,38(10):199-205+249.

[4]HINCHI AZ,TKIOUAT M.Rolling element bearing remaining useful life estimation based on a convolutional long-short-term memory network[J].Procedia Computer Science,2018,127:123-132.

[5]LI X,ZHANG W,DING Q.Deep learning-based remaining useful life estimation of bearings using multi-scale feature extraction[J].Reliability Engineering and System Safety,2019,182:208-218.

[6]MAO W T,HE J L,TANG J M,et al.Predicting remaining useful life of rolling bearings based on deep feature representation and long short-term memory neural network[J].Mechanical Engineering,2018,10(12):1-18.

[7]CHERIYADAT A M.Unsupervised feature learning for aerial scene classification[J].IEEE Transactions on Geoscience&Remote Sensing,2014,52(1):439-451.

[8] Sunworjade, Shao Si Yu, as strong as it is, sparse automatic coding deep neural network-based induction motor fault diagnosis [ J ]. report of mechanical engineering, 2016,52(09):65-71.

SUN W J,SHAO S Y,YAN R Q.Induction motor fault diagnosis based on deep neural network of sparse auto-encoder[J].Journal of Mechanical Engineering,2016,52(09):65-71.

[9] Chenyu, Wenxing, Liu Mega yoga, and the like.study on the multi-bullet classification algorithm of sparse automatic encoder visual characteristic fusion [ J ] infrared and laser engineering, 2018,47(08):386-393.

CHEN Y,WEN X L,LIU Z Y,et al.Research of multi-missile classification algorithm based on sparse auto-encoder visual feature fusion[J].Infrared and Laser Engineering,2018,47(08):386-393.

[10]WANG S X,WANG X,WANG S M,et al.Bi-directional long short-term memory method based on attention mechanism and rolling update for short-term load forecasting[J].International Journal of Electrical Power&Energy System,2019,109:470-4797.

[11]BIN Y,YANG Y,SHEN F M,et al.Describing video with attention-based bidirectional LSTM[J].IEEE Transactions on Cybernetics,2019,49(7):2631-2641.

[12] Jade, Wang Hua Wei general aviation risk prediction based on sparse denoising self-coding neural network [ J ] systematic engineering and electronic technology, 2019,41(01): 112-.

YU S X,WANG H W.Risk forecasting in general aviation based on sparse de-noising auto-encoder neural network[J].Systems Engineering and Electronics,2019,41(01):112-117.

[13]SRIVASTAVA N,HINTON G,KRIZHEVSKY A,et al.Dropout:a simple way to prevent neural networks from overfitting[J].Journal of Machine Learning Research,2014,15(1):1929-1958.

[14] Shi Jia Rong, Wang Dan, Shang Van, etc. random gradient descent algorithm research progress [ J ] automated academic newspaper, 2019:1-17.

SHI J R,WANG D,SHANG F Y,et al.Research advances on stochastic gradient descent algorithms[J].Acta Automatica Sinica,2019:1-17.

[15]PATRICK N,RAFAEL G,KAMAL M,et al.PRONOSTIA:an experimental platform for bearing accelerated life test[C].IEEE International Conference on Prognostics and Health Management,Denver,CO,USA,2012:1-8.

[16] The application of the improved morphological difference filter in the fault diagnosis of the rolling bearing [ J ]. Chongqing university of science and technology (Nature science), 2018,32(01):1-6.

SHI X H,YANG X H,ZHANG X K,et al.Application of improved morphological difference filter in fault diagnosis of rolling bearings[J].Journal of Chongqing University of Technology(Natural Science),2018,32(01):1-6.

[17]SINGLETON R K,STRANGAS E G,AVIYENTEv S.Extended kalman filtering for remaining-useful-life estimation of bearings[J].Industrial Electronics IEEE Transactions on,2015,62(3):1781-1790.

[18]GUO L,LI N P,JIA F,et al.A recurrent neural network based health indicator for remaining useful life prediction of bearings[J].Neurocomputing,2017,240:98-109.

[19]HONG S,ZHOU Z,ZIO E,et al.Condition assessment for the performance degradation of bearing based on a combinatorial feature extraction method[J],Digital Signal Processing,2014,27:159-166.

[20] XJTU-SY rolling bearing accelerated life test data set interpretation [ J ] is reported in Reasia, Korea, King Biao, et al, mechanical engineering, 2019:1-6.

LEI Y G,HAN T Y,WANG B,et al.XJTU-SY rolling element bearing accelerated life test datasets:a tutorial[J].Journal of Mechanical Engineering,2019:1-6.

Claims (5)

1. A rolling bearing RUL prediction method based on dropout-SAE and Bi-LSTM is characterized in that the method is realized by the following steps:

step one, data preprocessing: firstly, Fourier transform is carried out on the original time domain vibration data of the rolling bearing, and the original time domain vibration data of the rolling bearing is converted into a frequency domain; then, carrying out linear function normalization processing on the obtained product; finally, dividing a training set and a test set;

step two, deep layer feature extraction: training network parameters by taking the training set as the input of dropout-SAE, and extracting the characteristics capable of representing the bearing degradation trend;

Step three, constructing a Bi-LSTM model: taking a plurality of features extracted by dropout-SAE on a training set as the input of a Bi-LSTM network, taking the ratio p of the current service life feature point number to the full life feature point number, namely the service life percentage as the label output of the current service life, setting relevant network parameters and then training to obtain a Bi-LSTM model;

step four, model optimization: respectively adopting Adam, RMSProp and SGDM optimization algorithms to optimize the Bi-LSTM model, respectively calculating the Mean Square Error (MSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), Mean square percentage Error (MAPE) and Root Mean Square Error (RMSE) of the Bi-LSTM model under each optimization algorithm, selecting the optimization algorithm with the minimum sum of the 5 errors to optimize the Bi-LSTM model to obtain optimized Bi-LSTM model parameters, and applying Dropout technology to prevent overfitting;

step five, verifying a test set: processing the test set by adopting a data preprocessing and feature extraction method which is the same as that of the training set, inputting the extracted features into the optimized Bi-LSTM network model, and predicting the p value of the known data;

Step six, predicting RUL: linear fitting is carried out on the predicted p-value curve of the known data through a linear function to obtain the p-value trend of each future point, and the setting of the p-value in the step (3) shows that when the p is 1, the bearing fails, namely the full life is achieved, and the full life L is utilized_qMinus the current life L_dCan obtain the ith bearingThe RUL of (2) is represented by formula (22):

RUL_i＝L_q-L_d (22)

RUL by predicted remaining Life_iAnd true lifetime actriul_iError between Er_iTo reflect the performance of the model for predicting the remaining life, as shown in equation (23):

in the second step, dropout-SAE is an improved SAE, a Tan activation function is used for replacing an original sigmoid activation function, and a dropout mechanism is used for replacing an original KL divergence to carry out sparsity constraint on the network;

the network structure of dropout-SAE is as follows: the dropout-SAE network structure is 2048-200-2048, wherein the number of nodes of the input layer corresponds to 2048 of normalized bearing frequency domain amplitude signals, the number of nodes of the hidden layer 200 corresponds to the finally extracted feature number, and the output layer and the input layer have the same dimension.

2. The rolling bearing RUL prediction method based on dropout-SAE and Bi-LSTM as claimed in claim 1, wherein in the second step, the specific process of deep feature extraction is as follows: the deep feature extraction mainly comprises two stages of pre-training and global parameter fine adjustment: in the pre-training stage, network parameters are initialized through unsupervised layer-by-layer pre-training; in the global parameter fine-tuning stage, the original input is used as a label, and the network parameters are fine-tuned through a BP back propagation algorithm and a gradient descent method so as to optimize dropout-SAE.

3. The method of predicting RUL for rolling bearings based on dropout-SAE and Bi-LSTM of claim 1, wherein the Bi-LSTM network consists of a hidden layer, the number of hidden states of the network is selected to be 150, Root Mean Square Error (RMSE) is used as its loss function, the initial learning rate is set to 0.01 and weight matrix W and bias b are initialized randomly.

4. The method for predicting RUL of rolling bearing based on dropout-SAE and Bi-LSTM according to claim 1, wherein in the third step, a plurality of features extracted from the training set by dropout-SAE are used as the input of Bi-LSTM network, and the specific process is as follows: the forward LSTM and the backward LSTM respectively carry out sequence and reverse processing on the characteristic sequence to obtain hidden layer states with opposite time sequences, and the hidden layer states are connected to obtain the same output.

5. The method for predicting RUL of rolling bearings based on dropout-SAE and Bi-LSTM according to claim 1, wherein in the third step,

a Bi-LSTM model is adopted to consider past and future information of data at the same time, two hidden layer states with opposite time sequences are obtained through a forward LSTM and a backward LSTM, and then the hidden layer states are connected to obtain the same output; the forward LSTM and the backward LSTM can respectively acquire the past information and the future information of the input sequence, and the hidden state H of the Bi-LSTM at the time t _tInvolving the forward direction

And backward

Wherein: t is the sequence length, ct represents a memory unit, xt represents an input vector at the T moment, and represents the deep characteristics of the rolling bearing vibration signal obtained by SAE extraction; ht is the hidden state at time t.

Images