CN110941928B - A method for predicting residual life of rolling bearings based on dropout-SAE and Bi-LSTM - Google Patents

Abstract

A rolling bearing RUL prediction method based on dropout-SAE and Bi-LSTM belongs to the field of prediction of bearing running states. The invention aims to solve the problems of long model training time and low prediction accuracy in the conventional rolling bearing RUL prediction method. The invention provides an improved SAE, namely dropout-SAE performs unsupervised deep characteristic self-adaptive extraction on a vibration signal of a rolling bearing, the network applies a new Tan activation function to replace the original sigmoid activation function, and the dropout method is adopted to realize the sparsity of the network; meanwhile, considering that the method for predicting the residual service life of the rolling bearing generally only considers the past information and ignores the future information, a bidirectional long-time memory network is introduced to serve as a prediction model of the rolling bearing RUL. The experimental results on 2 bearing data sets show that the prediction method not only can improve the convergence rate of the model, but also has higher accuracy.

Description

A method for predicting residual life of rolling bearings based on dropout-SAE and Bi-LSTM

technical field

The invention relates to a method for predicting the remaining life of a rolling bearing, which belongs to the field of predicting the running state of the bearing.

Background technique

As the most commonly used and easily damaged key components of rotating equipment, rolling bearings often directly affect the performance of the entire equipment ^[1] . Therefore, it is of great practical significance to predict the remaining useful life (RUL) of rolling bearings.

Feature extraction is an important prerequisite for rolling bearing RUL prediction. In recent years, deep learning has attracted widespread attention due to its powerful adaptive feature extraction capability and nonlinear function representation capability, and has provided a new solution for feature extraction of rolling bearing vibration signals ^[2] . Reference [3] proposes an improved deep belief network, which directly uses the original vibration signal of rolling bearing as the network input, and abstracts it layer by layer from the low level to the high level, so as to achieve the purpose of deeply mining the essential characteristics of the data. References [4-6] utilize the unique local convolution, weight sharing and downsampling of convolutional neural networks to automatically extract local abstract information from rolling bearing vibration signals and realize deep mining of vibration signal features. Although the above studies use deep learning methods to simplify the complex feature extraction process and excavate the deep essential characteristics of vibration signals, the network structure parameters still require a large amount of labeled data for supervised fine-tuning, and the labeled data is difficult to obtain in practice.

Unsupervised feature learning can automatically extract data intrinsic features from unlabeled data ^[7] . This method has great advantages when labeled data is scarce and difficult to obtain. As a typical unsupervised feature learning model, the sparse auto encoder (SAE) can effectively learn the concise intrinsic feature representation of the data from a large amount of unlabeled data ^[8] . It has been successfully extended to various applications. It is suitable for applications with limited labeled data ^[9] . The traditional SAE uses sigmoid as the activation function, which is easy to cause the problem of gradient disappearance, and the use of KL divergence for sparsity constraints has limitations in the feature extraction of rolling bearings.

On the basis of feature extraction, rolling bearing RUL prediction is the ultimate goal. On the basis of obtaining the eigenvalues of bearing performance degradation, considering the advantages of the recurrent neural network in the deep learning algorithm in processing time series data, the long short-term memory network (LSTM) is used as the bearing performance degradation curve. build method. The way to build a bearing performance degradation curve using LSTM is to integrate information from the "past" and then assist in processing the current information. However, the traditional LSTM does not take into account that the decay process of rolling bearings is actually a continuous change process with a front and back dependency in time, and the processing of current information is also necessary to integrate "future" information ^[10] .

To sum up, the traditional SAE uses sigmoid as the activation function, which is easy to cause the problem of gradient disappearance, and the use of KL divergence for sparsity constraints has limitations in the extraction of rolling bearing features; the traditional rolling bearing RUL prediction method only considers the past information and ignores the future information. The problem is still not resolved.

SUMMARY OF THE INVENTION

In order to solve the problems of long model training time and low prediction accuracy in the existing rolling bearing RUL prediction method, the present invention further proposes a rolling bearing RUL prediction method based on dropout-SAE and Bi-LSTM.

The technical scheme adopted by the present invention to solve the above-mentioned technical problems is:

A rolling bearing RUL prediction method based on dropout-SAE and Bi-LSTM, the implementation process of the method:

Step 1. Data preprocessing: First, perform Fourier transform on the original time-domain vibration data of the rolling bearing and convert it to the frequency domain; then normalize it with a linear function; finally, divide the training set and the test set;

Step 2. Deep feature extraction: The training set is used as the input of dropout-SAE to train network parameters, and the features that can characterize the bearing degradation trend are extracted;

Step 3. Build the Bi-LSTM model: use multiple features extracted from the training set by dropout-SAE as the input of the Bi-LSTM network, the ratio p of the current service life feature points to the full life feature points, that is, the life percentage as the current Lifetime label output, set the relevant network parameters for training, and get the Bi-LSTM model;

Step 4: Model optimization: The Bi-LSTM model is optimized by the Adam, RMSProp, and SGDM optimization algorithms, respectively, and the mean squared error (MSE) and mean absolute error of the Bi-LSTM model under each optimization algorithm are calculated respectively. (mean absolute error, MAE), mean absolute percentage error (MAPE), mean square percentage error (MAPE), root mean square error (root mean square error, RMSE), select the above five types The optimization algorithm with the smallest sum of errors is used to optimize the Bi-LSTM model, and the optimized Bi-LSTM model parameters are obtained, and the Dropout technology is applied to prevent over-fitting;

Step 5. Test set verification: use the same data preprocessing and feature extraction methods as the training set to process the test set, and input the extracted features into the optimized Bi-LSTM network model to predict the p of the known data. value;

Step 6. RUL prediction: perform linear fitting on the p-value curve of the predicted known data to obtain the p-value trend of each point in the future. From the setting of the p-value in step (3), it can be seen that when p=1 When , the bearing fails, that is, the full life is reached. By subtracting the current life L _d from the full life L _q , the RUL of the ith bearing can be obtained, as shown in formula (22):

RUL _i =L _q -L _d (22)

The prediction performance of the remaining life of the model is reflected by the error Er _i between the predicted remaining life RUL _i and the real life ActRUL _i , as shown in Equation (23):

Further, in step 2, the dropout-SAE is an improved SAE, which replaces the original sigmoid activation function with the Tan activation function, and uses the dropout mechanism to replace the original KL divergence to constrain the network sparsity.

Further, in step 2, the network structure of the dropout-SAE is: the dropout-SAE network structure is 2048-200-2048, wherein the number of input layer nodes corresponds to 2048 of the normalized bearing frequency domain amplitude signals Point, the number of hidden layer nodes 200 corresponds to the number of finally extracted features, and the output layer has the same dimension as the input layer.

Further, in step 2, the specific process of deep feature extraction is as follows: deep feature extraction mainly includes two stages: pre-training and global parameter fine-tuning: the pre-training stage initializes network parameters through unsupervised layer-by-layer pre-training; the global parameter fine-tuning stage Using the original input as a label, the network parameters are fine-tuned by BP back-propagation algorithm and gradient descent method to optimize dropout-SAE.

Further, the Bi-LSTM network consists of one hidden layer, the number of hidden states of the network is selected as 150, the root mean square error (RMSE) is used as its loss function, the initial learning rate is set to 0.01 and the weight matrix W is randomly initialized and bias b.

Further, in step 3, multiple features extracted from the training set by dropout-SAE are used as the input of the Bi-LSTM network. The specific process is as follows:

The forward LSTM and the backward LSTM process the feature sequences sequentially and in reverse, respectively, to obtain the opposite hidden layer states of the time series, and connect them to obtain the same output.

和后向的

Further, in step 3, the Bi-LSTM model is used to consider the past and future information of the data at the same time, and the two hidden layer states with opposite time series are obtained through the forward LSTM and the backward LSTM, and then they are connected to obtain the same output; Forward LSTM and backward LSTM can obtain the past and future information of the input sequence, respectively, and the hidden state Ht of Bi-LSTM at time _t contains forward

and backward

Among them: T is the sequence length, ct is the memory unit, xt is the input vector at time t, which is the deep feature of the rolling bearing vibration signal extracted by SAE; ht is the hidden state at time t.

The present invention has the following beneficial technical effects:

Because the traditional SAE uses sigmoid as the activation function, it is easy to cause the problem of gradient disappearance, and the KL divergence used to constrain the network sparsity is not suitable for dealing with regression problems such as rolling bearing feature extraction. Based on this, the present invention improves the activation function of SAE, replaces the original sigmoid function with a new Tan function, and replaces the KL divergence with the dropout mechanism to achieve network sparsity, thereby forming an improved SAE, that is, dropout-SAE. And use dropout-SAE for unsupervised deep feature adaptive extraction of rolling bearing vibration signals, without manual design labels for supervised fine-tuning. On the basis of feature extraction, rolling bearing RUL prediction is the ultimate goal. The present invention considers that the decline process of rolling bearing is actually a continuous change process with time dependent relationship, and it is also necessary to integrate "future" information in the processing of current information ^[10] . Reference [10] used bi-directional long short-term memory (Bi-LSTM) network for short-term prediction of load and achieved good experimental results. Reference [11] applies Bi-LSTM to video description to comprehensively preserve global temporal and visual information. It can be confirmed that Bi-LSTM has feasibility and superiority in time series processing. Therefore, the present invention obtains more accurate RUL prediction results by introducing Bi-LSTM to fully utilize past and future information.

The invention improves SAE, proposes a dropout-SAE network, and uses its deep structure to perform unsupervised adaptive feature extraction on the original bearing vibration signal, and uses the extracted deep feature as the performance degradation feature of the rolling bearing. In order to further solve the problem that the traditional rolling bearing RUL prediction method only considers the past information and ignores the future information, Bi-LSTM is introduced to complete the current life prediction of the rolling bearing. Finally, a first-order function is used to fit the current life to realize the RUL prediction of the rolling bearing.

Aiming at the problem that the sigmoid activation function is easy to cause the gradient to disappear in the sparse auto encoder (SAE), a new Tan function is used to replace the original sigmoid function; for SAE, the KL divergence is used to constrain the sparsity in the regression prediction. Due to the limitations of this aspect, the sparseness of the network is achieved by replacing the KL divergence with the dropout mechanism, thereby forming an improved SAE, namely dropout-SAE. And use dropout-SAE for unsupervised deep feature adaptive extraction of rolling bearing vibration signals, without manual design labels for supervised fine-tuning. At the same time, considering that the remaining useful life (RUL) prediction method of rolling bearings generally only considers past information and ignores future information, a bi-directional long short-term memory (Bi-LSTM) network is introduced to construct rolling bearing RUL prediction model. The experimental results on the two bearing datasets all show that the RUL prediction method for rolling bearings based on dropout-SAE and Bi-LSTM proposed in the present invention can not only improve the convergence speed of the model but also have a higher accuracy.

Description of drawings

Fig. 1 is a schematic diagram of AE structure;

Fig. 2 is the sigmoid function and its derivative function curve;

Fig. 3 is Tan function and its derivative function curve;

Figure 4 is a schematic diagram of the internal structure of the LSTM unit;

Figure 5 is an expanded view of the Bi-LSTM network;

Fig. 6 is the flow chart of RUL prediction of rolling bearing;

Fig. 7 is the time-domain vibration signal of bearing 1_1 and the normalized frequency-domain amplitude spectrum, wherein: (a) is the time-domain vibration signal, and (b) is the normalized frequency-domain amplitude spectrum;

Fig. 8 is the characteristic trend curve diagram of bearing 1_1 part;

9 is a graph of the fitting result and fitting error of the current p-value predicted by the method of the present invention, wherein: (a) represents the fitting result, and (b) represents the fitting error;

Fig. 10 is the RUL prediction result diagram of bearing 1-7 by the method of the present invention,

Figure 11 is a comparison of the time consumed by feature extraction (PHM2012 bearing data set),

Figure 12 is a diagram showing the prediction results of bearing 1_7 RUL for three schemes, in the figure: (a) is scheme one, (b) is scheme two (c), scheme three;

Figure 13 is a comparison of the time consumed by feature extraction (XJTU-SY bearing dataset).

Detailed ways

The implementation of the method for predicting the remaining life of a rolling bearing based on dropout-SAE and Bi-LSTM according to the present invention is described below with reference to Figures 1 to 13:

1dropout-SAE model

Auto encoder (AE) is a three-layer neural network that tries to learn a function through an unsupervised learning algorithm, so that the output value is close to the input value. Its structure is shown in Figure 1, including the input layer and the hidden layer. and the output layer ^[12] .

The input layer and the hidden layer form an encoding network, and the encoding process is to convert the input x={x ₁ ,x ₂ ,...,x _n } containing n data into a hidden layer expression h={h ₁ ,h ₂ with advanced features ,...,h _n }; the hidden layer and the output layer form a decoding network. The decoding process is that the hidden layer vector set is reversely transformed into a reconstructed data set with the same dimension as the input data y={y ₁ ,y ₂ ,...,y _n }.

The encoding process and decoding process can be expressed as:

h=S _f (b ₁ +W ₁ x) (1)

y=S _g (b ₂ +W ₂ h) (2)

Wherein: S _f is an encoding activation function; S _g is a decoding activation function; b ₁ , b ₂ are biases; W ₁ , W ₂ are weight matrices, W ₂ =W ₁ ^T .

AE minimizes the reconstruction error by optimizing the parameter set θ={W ₁ ,W ₂ ,b ₁ ,b ₂ }, and the loss function is:

The activation functions S _f and S _g in equations (1) and (2) generally use the sigmoid function, and the sigmoid function and its derivative function are in the form:

It can be seen from the image of the sigmoid function and its derivative function in Figure 2 that when the output of the neural network is large, the sigmoid derivative will become very small, causing the model to converge very slowly, that is, the gradient disappears.

In order to solve this problem, this paper adopts a new activation function called Tan function. The Tan function and its derivative function are in the form:

From the Tan function and its derivative image in Figure 3, it can be seen that the minimum value of the Tan derivative is about 0.64, which will not appear to be 0 and cause the gradient to disappear, making the network model converge faster.

SAE adds a sparse penalty term to the loss function of AE, which can better express the input data structure and avoid network overfitting. Traditional SAE uses KL divergence as a sparse penalty term, which is defined as:

表示隐藏层第j个神经元的平均激活度，ρ为稀疏参数，a_j(x)表示给定输入x的情况下隐藏层第j个神经元的激活度。Among them: β represents the activation parameter of the weight, m represents the number of neurons in the hidden layer,

represents the average activation of the jth neuron in the hidden layer, ρ is the sparsity parameter, and a _j (x) represents the activation of the jth neuron in the hidden layer given the input x.

The loss function of SAE after adding the sparse penalty term is:

J(θ)=J _MSE (θ)+J _sparse (θ) (11)

作为依据，对网络进行惩罚。因此，本文采用dropout机制实现SAE的稀疏性。However, the above use of KL divergence as the sparse constraint of SAE is only applicable to the classification problem with the true value of 0 or 1. For the regression problem that the deep feature to be extracted for rolling bearings is a certain value between [0, 1], it is impossible to Will

As a basis, the network is penalized. Therefore, this paper adopts the dropout mechanism to achieve the sparsity of SAE.

The specific method is to introduce a dropout layer before the activation function in the encoding and decoding process, and perform mask processing during encoding and decoding, so that the activation values of some neurons in AE are set to a certain probability q (usually 0.5) as 0 ^[13] , the formula is:

Where: z represents the input of the original activation function, and z′ represents the input of the activation function sparsed by the dropout layer.

Once a neuron is set to 0, it means that the corresponding neuron is removed from the network, and its connection weights and biases will not be updated in this learning (in a dormant state), that is to say, using a scale Learning from a self-network smaller than the original network. At this time, for the encoding and decoding processes, equations (1) and (2) are still satisfied.

2Bi-LSTM model

The LSTM model consists of three gates (input gate i _t , forget gate ft , output gate o _t ) and a memory unit (c _{t )} , through which the internal memory is selectively input, output and forgetting operations , which can effectively overcome the gradient explosion or gradient disappearance problem. The internal structure of the LSTM unit is shown in Figure 4.

A complete LSTM can be expressed as:

f _t =σ(W _f ·X+b _f ) (14)

i _t =σ(W _i ·X+ _bi ) (15)

o _t =σ(W _o ·X+b _o ) (16)

为逐点乘积。Where: x _t represents the input vector at time t; h _t is the hidden state at time t; W and b are the weights and biases of the LSTM, which are both model training parameters; σ is the activation function sigmoid;

is the pointwise product.

和后向的

While LSTM is able to address long-term dependencies, it does not exploit future information. Therefore, the present invention adopts the Bi-LSTM model to consider the past and future information of the data at the same time, and expands it as shown in FIG. 5 . Its working principle is: through forward LSTM and backward LSTM, two hidden layer states with opposite time series are obtained, and then they are connected to obtain the same output. Forward LSTM and backward LSTM can obtain the past and future information of the input sequence respectively ^[10] . The hidden state Ht of the Bi-LSTM at time _t contains the forward

and backward

Where: T is the sequence length.

3. Rolling bearing RUL prediction method and process

The flow chart of the RUL prediction method for rolling bearings based on dropout-SAE and Bi-LSTM is shown in Figure 6. The specific steps are:

(1) Data preprocessing: First, perform Fourier transform on the original time-domain vibration data of the rolling bearing and convert it to the frequency domain; then normalize it with a linear function; finally, divide the training set and the test set.

(2) Deep feature extraction: The training set is used as the input of dropout-SAE for unsupervised deep feature extraction, which mainly includes two stages: pre-training and global parameter fine-tuning: the pre-training stage initializes network parameters through unsupervised layer-by-layer pre-training; In the parameter fine-tuning stage, the original input is used as the label, and the network parameters are fine-tuned by the BP back-propagation algorithm and the gradient descent method, so as to optimize the whole model, further improve the nonlinear function mapping ability of the network, and obtain better feature expression ^[2] , Finally, the features that can characterize the bearing degradation trend are extracted.

(3) Constructing the Bi-LSTM model: using multiple features extracted from the training set by dropout-SAE as the input of the Bi-LSTM network, the ratio p of the current service life feature points to the full life feature points, that is, the percentage of life as the current The label output of the lifespan is trained after setting the relevant network parameters.

(4) Model optimization: by calculating the mean squared error (MSE), mean absolute error (MAE), mean absolute percentage error (MAPE), mean square percentage error of the training model (Mean square percentage error, MAPE), root mean square error (root mean square error, RMSE) and the sum of the above five errors as evaluation criteria, compare the three commonly used optimization algorithms Adam, RMSProp and SGDM proposed in the literature [14] , train to obtain the optimal Bi-LSTM model parameters, and apply Dropout technology to prevent overfitting.

(5) Test set verification: The test set is processed with the same data preprocessing and feature extraction methods as the training set, and the extracted features are input into the trained Bi-LSTM network model to predict the p of the known data. value.

(6) RUL prediction: perform a linear function on the p-value curve of the predicted known data to obtain the p-value trend of each point in the future. It can be known from the setting of the p value in step (3) that when p=1, the bearing fails, that is, the full life is reached. The RUL of the ith bearing can be obtained by subtracting the current life L _d from the full life L _q , as shown in formula (22):

RUL _i =L _q -L _d (22)

(7) The error Er _i between the predicted remaining life RUL _i and the real life ActRUL _i reflects the quality of the model remaining life prediction performance. As shown in formula (23):

4 application and analysis, the technical effect of the present invention is verified:

In order to verify the rolling bearing RUL prediction method based on dropout-SAE and Bi-LSTM proposed by the present invention, the PHM2012 bearing dataset ^[15] was selected as the experimental data for verification. The data set is collected by two acceleration sensors in the horizontal and vertical directions, and is recorded every 10s, the time of each recording is 0.1s, and the sampling frequency is 25.6kHz. The present invention uses vibration data in the horizontal direction.

As shown in Table 1, the present invention selects the full life data of 6 bearings (all data from the start of operation to complete failure of the rolling bearing) of bearings 1_1, 1_2, 2_1, 2_2, 3_1 and 3_2 as the training set for training, and the remaining bearings 1_3 , 1_4, 1_5, 1_6, 1_7, 2_3, 2_4, 2_5, 2_6, 2_7, and 3_3, a total of 11 non-full-life data of bearings (the data of rolling bearings from the start of operation to a certain time point) are used as the test set for RUL prediction experiments.

Table 1 Experimental data (PHM2012 bearing dataset)

In the experiment, the original time domain signals of 17 bearings in training set and test set are preprocessed. Taking bearing 1_1 as an example, a sample time-domain vibration signal and the corresponding normalized frequency-domain amplitude signal within the 0.1s acquisition period are shown in Figure 7.

The normalized bearing frequency domain amplitude signal is input into dropout-SAE for unsupervised adaptive feature extraction. After a lot of experiments, the dropout-SAE network structure is selected as 2048-200-2048, in which the number of input layer nodes corresponds to 2048 points of the normalized bearing frequency domain amplitude signal, and the number of hidden layer nodes 200 corresponds to the final extracted features. number. In order to eliminate the influence of oscillation on the health index and keep the original characteristic curve unchanged, the obtained characteristic curve is smoothed and filtered ^[16] . Some 5 features are arbitrarily selected from the 200-dimensional features extracted from bearing 1_1, and the trend curve is shown in Figure 8.

As can be seen from Figure 8, the deep features extracted by SAE have a good monotonic trend as a whole (part of the features show an upward trend in the entire life cycle of the bearing, and other features show a downward trend), which can well represent the entire life of the bearing. cycle of decline.

Training stage: Input the deep features extracted by SAE of bearings 1_2, 2_1, 2_2, 3_1 and 3_2 into the Bi-LSTM network model, and use the real p value as the output of the model to train the Bi-LSTM prediction model. The Bi-LSTM network consists of one hidden layer, and the number of hidden states of the network is chosen to be 150 after iterative experiments. Using root mean square error (RMSE) as its loss function, the initial learning rate is set to 0.01 and the weight matrix W and bias b are randomly initialized. Calculate the errors and the sum of the errors of the training models under the three optimization algorithms Adam, RMSProp and SGDM, as shown in Table 2. It can be seen that Adam, as an adaptive optimization algorithm, can minimize the error of the proposed model, and at the same time, the Adam algorithm can iteratively update the weight of the neural network based on the training data. Therefore, the present invention uses the Adam optimizer to perform gradient optimization. Besides, the present invention also utilizes the Dropout technique ^[19] to prevent overfitting and improve the performance of the model. After experiments, the Dropout value is set to 0.1.

Table 2 Training errors of three optimization algorithms

Test phase: Take the test bearing 1_7 as an example, the same as the training phase, input the deep features of bearing 1_7 extracted by dropout-SAE into the trained Bi-LSTM prediction model, and predict the current p value. The fitting result of the predicted value and the actual value is shown in Fig. 9(a), and Fig. 9(b) is the corresponding fitting error.

The predicted current p value of bearing 1_7 is fitted with a linear function to obtain the trend of the future p value, so as to obtain the RUL prediction result of rolling bearing 1_7, as shown in Figure 10.

According to the actual sampling data characteristics of the bearing, the life time represented by each feature point of each bearing is 10s. It is known that bearing 1_7 has a total of 1502 points of incomplete life data, and a total of 2259 points of full life data. It can be seen from Figure 10 that when the bearing reaches the failure threshold, that is, when p=1, the corresponding predicted full life data points have a total of 2282 points. The predicted RUL calculated by the formula (22) is (2282-1502)×10s=7800s, and the actual ActRUL is (2259-1502)×10s=7570s, and then the prediction error from the formula (23) is ((7570-7800)/ 7570)×100%=-3.04%, as shown in Table 4.

In order to evaluate the uncertainty of RUL prediction, the method of literature [17] was used to estimate the interval of RUL, and a confidence interval of 95% confidence level was set near the predicted value, and the upper and lower bounds were extracted. Similar to the RUL prediction above, these values can also be extrapolated to the failure threshold to obtain upper and lower confidence intervals for the RUL prediction [7530s, 8070s].

In order to verify the advantage of dropout-SAE compared with SAE in terms of convergence speed, SAE and dropout-SAE are used to extract deep features of rolling bearings respectively, and the time consumed is shown in Figure 11.

As can be seen from Figure 11, in the 17 bearing feature extraction experiments, the time consumed by dropout-SAE feature extraction is shorter than the time consumed by SAE feature extraction, which proves that dropout-SAE is faster than SAE. convergence speed.

In order to verify the effectiveness of the proposed prediction method based on dropout-SAE and Bi-LSTM, three other schemes are set up for comparison experiments with the prediction method proposed in the present invention, as shown in Table 3.

The forecasting method proposed in Table 3 and the composition of the other three schemes

According to the experimental process of RUL prediction of bearing 1_7 according to the method proposed in the present invention, the RUL prediction of bearing 1_7 of the other three schemes can be obtained in the same way. The results are shown in Figure 12, and the specific prediction errors are shown in Table 4.

In order to further prove the effectiveness of the method of the present invention, the RUL prediction accuracy score formula (24) of the PHM2012 bearing data set is used to evaluate the RUL prediction of rolling bearings, and the average score results are shown in Table 4.

where A _i is defined as:

Similarly, Table 4 gives the RUL prediction errors and average scores of the other 10 bearings in the database, and gives the comparison results with the literature [18] and [19].

Table 4 Comparison of RUL prediction results for different bearings (PHM2012 bearing data set)

It can be seen from the comparative experimental results of the prediction method based on dropout-SAE and Bi-LSTM proposed by the present invention and the other three schemes: in the case of the same LSTM and Bi-LSTM prediction models, the dropout-SAE feature extraction model is better than the SAE feature extraction model. The average prediction error obtained by the model is reduced by 5.56% and 1.25%, and the average score is increased by 0.052 and 0.054, respectively, which can prove that the dropout-SAE feature extraction model is more superior. In the case of the same dropout-SAE feature extraction model, the Bi-LSTM prediction model reduces the average error by 3.32% and the average score increases by 0.099 compared with the LSTM prediction model, which proves that the Bi-LSTM prediction model has great advantages. Overall, the method proposed in this paper has lower errors and higher scores than Scheme 1, Scheme 2 and Scheme 3. At the same time, the average prediction error of the method proposed in this paper is reduced by 25.99% and 46.75%, and the average score is increased by 0.313 and 0.511, respectively, compared with the literature [18] and literature [19]. This further proves the effectiveness of the method proposed in this paper in RUL prediction of rolling bearings.

To verify the generalization ability of the proposed model based on dropout-SAE and Bi-LSTM, the XJTU-SY bearing dataset ^[20] of Xi'an Jiaotong University is used as new experimental data. The data set is collected by two acceleration sensors in the horizontal direction and the vertical direction. It is recorded every 1 min, the time of each recording is 1.28s, the sampling frequency is 25.6kHz, and the vibration data in the horizontal direction is used. According to the PHM2012 bearing dataset ^[17] , the bearing data is divided into partial life and full life, as shown in Table 5. Select the full life data of bearings 1_1, 1_2, 2_1, 2_2, 3_1 and 3_2 as the training set for training, and the remaining bearings 1_3, 1_4, 1_5, 2_3, 2_4, 2_5, 3_3, 3_4, 3_5 have a total of 9 bearings The non-full-life data is used as the test set.

Table 5 Experimental data (XJTU-SY bearing dataset)

At the same time, in order to simplify the experimental process, the middle 4096 points of the data collected in each 1.28s were selected as data samples, and the SAE deep feature extraction, Bi-LSTM model construction, RUL prediction, etc. were carried out according to the same experimental method of the PHM2012 bearing data set. The specific experimental results are shown in Figure 13 and Table 6.

Table 6 Comparison of RUL prediction results for different bearings (XJTU-SY bearing data set)

From the comparison of the experimental results of the above four methods, the same conclusion as the PHM2012 bearing data set can be drawn, so it can be further demonstrated that the proposed method has good generalization ability.

5 Conclusion

(1) For the traditional SAE using sigmoid as the activation function, it is easy to cause the problem of gradient disappearance, and a new Tan function is used to replace the original sigmoid function; for SAE, the use of KL divergence for sparsity constraints is limited in the feature extraction of rolling bearings , replacing KL divergence with dropout mechanism to achieve its sparsity, forming an improved SAE, namely dropout-SAE. And use dropout-SAE to perform unsupervised feature self-adaptive extraction of rolling bearing vibration signals, so as to obtain deep features with certain trends that can represent the bearing degradation trend.

(2) Aiming at the problem that the standard LSTM processes sequences in time order and only considers the past information and ignores the future information, the Bi-LSTM network is introduced, and the same output is connected to two LSTM networks with opposite times to obtain the past data information of the input sequence respectively. and future data information. At the same time, in order to get better prediction results, the Bi-LSTM prediction model is optimized by Adam algorithm and Dropout technology.

(3) The proposed method is verified by experiments on two datasets. The results show that, compared with the traditional SAE model, the dropout-SAE model has a higher convergence rate and the extracted deep features combined with the Bi-LSTM model are more effective in RUL prediction of rolling bearings. Compared with the other two literatures, the prediction error is reduced by more than 25%, and the score is improved by more than 0.313.

For rolling bearing RUL prediction, there are two kinds of results: advance prediction (Er _i >0) and lag prediction (Er _i <0). The risk of advance prediction of equipment in industrial production and life is lower than that of lag prediction. Therefore, "forward forecasting" is more practical than "lagging forecasting". Although the present invention improves the prediction accuracy to a certain extent, it also exacerbates the problem of "lag prediction". Therefore, the optimization of the RUL prediction model will be the focus of the next research work.

The references cited in the present invention are as follows:

[1] Xia Xintao, Chang Zhen, Ye Liang, et al. Performance stability and reliability analysis of rolling bearings [J]. Journal of Instrumentation, 2017, 38(06): 1421-1431.

XIAX T,CHANG Z,YE L,et al.Stability and reliability analysis of rolling bearing performance[J].Chinese Journal of Scientific Instrument,2017,38(06):1421-1431.

[2] Zhao Guangquan, Liu Xiaoyong, Jiang Zedong, et al. Unsupervised construction of bearing health factor based on deep learning [J]. Journal of Instrumentation, 2018, 39(06): 82-88.

ZHAO G Q, LIL X Y, JIANG Z D, et al. Unsupervised health indicator of bearing based on deep learning[J]. Chinese Journal of Scientific Instrument, 2018, 39(06): 82-88.

[3] Yang Yu, Zhang Na, Cheng Junsheng. Application of full-parameter dynamic learning deep belief network in rolling bearing life prediction [J]. Vibration and Shock, 2019, 38(10): 199-205+249.

YANG Y,ZHANG N,CHENG J S.Global parameters dynamic learning deepbelief networks and its application in rolling bearing life prediction[J].Journal of Vibration and Shock,2019,38(10):199-205+249.

[4]HINCHI AZ,TKIOUAT M.Rolling element bearing remaining useful lifeestimation based on a convolutional long-short-term memory network[J].Procedia Computer Science,2018,127:123-132.

[5]LI X,ZHANG W,DING Q.Deep learning-based remaining useful lifeestimation of bearings using multi-scale feature extraction[J].ReliabilityEngineering and System Safety,2019,182:208-218.

[6]MAO W T,HE J L,TANG J M,et al.Predicting remaining useful life of rolling bearings based on deep feature representation and long short-termmemory neural network[J].Mechanical Engineering,2018,10(12):1-18 .

[7]CHERIYADAT A M.Unsupervised feature learning for aerial sceneclassification[J].IEEE Transactions on Geoscience&Remote Sensing,2014,52(1):439-451.

[8] Sun Wenjun, Shao Siyu, Yan Ruqiang. Fault diagnosis of induction motor based on sparse auto-encoding deep neural network [J]. Chinese Journal of Mechanical Engineering, 2016, 52(09): 65-71.

SUN W J,SHAO S Y,YAN R Q.Induction motor fault diagnosis based on deep neural network of sparse auto-encoder[J].Journal of Mechanical Engineering,2016,52(09):65-71.

[9] Chen Yu, Wen Xinling, Liu Zhaoyu, et al. Research on multi-bomb classification algorithm based on visual feature fusion of sparse autoencoders [J]. Infrared and Laser Engineering, 2018, 47(08): 386-393.

CHEN Y, WEN X L, LIU Z Y, et al. Research of multi-missile classification algorithm based on sparse auto-encoder visual feature fusion[J]. Infrared and Laser Engineering, 2018, 47(08): 386-393.

[10] WANG S X, WANG X, WANG S M, et al.Bi-directional long short-termmemory method based on attention mechanism and rolling update for short-termload forecasting[J].International Journal of Electrical Power&Energy System,2019,109: 470-4797.

[11]BIN Y,YANG Y,SHEN F M,et al.Describing video with attention-basedbidirectional LSTM[J].IEEE Transactions on Cybernetics,2019,49(7):2631-2641.

[12] Yu Sixuan, Wang Huawei. General aviation risk prediction based on sparse noise reduction autoencoding neural network [J]. Systems Engineering and Electronic Technology, 2019, 41(01): 112-117.

YU S X,WANG H W.Risk forecasting in general aviation based on sparsede-noising auto-encoder neural network[J].Systems Engineering and Electronics,2019,41(01):112-117.

[13] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: a simple way to prevent neural networks from overfitting [J]. Journal of Machine Learning Research, 2014, 15(1): 1929-1958.

[14] Shi Jiarong, Wang Dan, Shang Fanhua, et al. Research progress of stochastic gradient descent algorithm [J]. Chinese Journal of Automation, 2019: 1-17.

SHI J R, WANG D, SHANG F Y, et al. Research advances on stochasticgradient descent algorithms[J]. Acta Automatica Sinica, 2019: 1-17.

[15] PATRICK N, RAFAEL G, KAMAL M, et al. PRONOSTIA: an experimental platform for bearing accelerated life test[C]. IEEE International Conferenceon Prognostics and Health Management, Denver, CO, USA, 2012: 1-8.

[16] Shi Xiaohui, Yang Xinhua, Zhang Xiangkui, et al. Application of improved morphological difference filter in fault diagnosis of rolling bearings [J]. Journal of Chongqing University of Technology (Natural Science), 2018, 32(01): 1-6.

SHI X H,YANG X H,ZHANG X K,et al.Application of improvedmorphological difference filter in fault diagnosis of rolling bearings[J].Journal of Chongqing University of Technology(Natural Science),2018,32(01):1-6.

[17]SINGLETON R K,STRANGAS E G,AVIYENTEv S.Extended kalman filtering for remaining-useful-life estimation of bearings[J].Industrial ElectronicsIEEE Transactions on,2015,62(3):1781-1790.

[18] GUO L, LI N P, JIA F, et al. A recurrent neural network based health indicator for remaining useful life prediction of bearings[J]. Neurocomputing, 2017, 240: 98-109.

[19] HONG S, ZHOU Z, ZIO E, et al. Condition assessment for the performance degradation of bearing based on a combinatorial featureextraction method[J], Digital Signal Processing, 2014, 27: 159-166.

[20] Lei Yaguo, Han Tianyu, Wang Biao, et al. Interpretation of XJTU-SY Rolling Bearing Accelerated Life Test Data Set [J]. Chinese Journal of Mechanical Engineering, 2019: 1-6.

LEI Y G,HAN T Y,WANG B,et al.XJTU-SY rolling element bearingaccelerated life test datasets:a tutorial[J].Journal of Mechanical Engineering,2019:1-6.

Claims (5)

1. a rolling bearing RUL prediction method based on dropout-SAE and Bi-LSTM, is characterized in that, the realization process of described method: Step 1. Data preprocessing: First, perform Fourier transform on the original time-domain vibration data of the rolling bearing and convert it to the frequency domain; then normalize it with a linear function; finally, divide the training set and the test set; Step 2. Deep feature extraction: The training set is used as the input of dropout-SAE to train network parameters, and the features that can characterize the bearing degradation trend are extracted; Step 3. Build the Bi-LSTM model: use multiple features extracted from the training set by dropout-SAE as the input of the Bi-LSTM network, the ratio p of the current service life feature points to the full life feature points, that is, the life percentage as the current Lifetime label output, set the relevant network parameters for training, and obtain the Bi-LSTM model; Step 4. Model optimization: The Bi-LSTM model is optimized by the Adam, RMSProp, and SGDM optimization algorithms, respectively, and the mean squared error (MSE) and mean absolute error of the Bi-LSTM model under each optimization algorithm are calculated respectively. (mean absolute error, MAE), mean absolute percentage error (MAPE), mean square percentage error (MAPE), root mean square error (root mean square error, RMSE), select the above five types The optimization algorithm with the smallest sum of errors is used to optimize the Bi-LSTM model, and the optimized Bi-LSTM model parameters are obtained, and the Dropout technology is applied to prevent overfitting; Step 5. Test set verification: Use the same data preprocessing and feature extraction methods as the training set to process the test set, and input the extracted features into the optimized Bi-LSTM network model to predict the p of the known data. value; Step 6. RUL prediction: perform linear fitting on the p-value curve of the predicted known data to obtain the p-value trend of each point in the future. From the setting of the p-value in step (3), it can be seen that when p=1 When , the bearing fails, that is, the full life is reached. By subtracting the current life L _d from the full life L _q , the RUL of the ith bearing can be obtained, as shown in formula (22): RUL _i =L _q -L _d (22) The prediction performance of the remaining life of the model is reflected by the error Er _i between the predicted remaining life RUL _i and the real life ActRUL _i , as shown in Equation (23):

In step 2, the dropout-SAE is an improved SAE, which replaces the original sigmoid activation function with the Tan activation function, and uses the dropout mechanism to replace the original KL divergence to constrain the network sparsity; The dropout-SAE network structure is: the dropout-SAE network structure is 2048-200-2048, wherein the number of input layer nodes corresponds to 2048 points of the normalized bearing frequency domain amplitude signal, and the number of hidden layer nodes corresponds to 200 The number of features finally extracted, the output layer has the same dimension as the input layer. 2. a kind of rolling bearing RUL prediction method based on dropout-SAE and Bi-LSTM according to claim 1 is characterized in that, in step 2, the concrete process of deep feature extraction is: deep feature extraction mainly comprises pre-training and There are two stages of global parameter fine-tuning: the pre-training stage initializes network parameters through unsupervised layer-by-layer pre-training; the global parameter fine-tuning stage takes the original input as a label, and fine-tunes the network parameters through BP back-propagation algorithm and gradient descent method to optimize dropout-SAE. 3. A kind of rolling bearing RUL prediction method based on dropout-SAE and Bi-LSTM according to claim 1, is characterized in that, described Bi-LSTM network is made up of one hidden layer, and the number of hidden states of network is selected as 150 , using the root mean square error (RMSE) as its loss function, setting the initial learning rate to 0.01 and randomly initializing the weight matrix W and bias b. 4. A kind of rolling bearing RUL prediction method based on dropout-SAE and Bi-LSTM according to claim 1, it is characterized in that, in step 3, the multiple features extracted by dropout-SAE on the training set are used as Bi - The input of the LSTM network, the specific process is: the forward LSTM and the backward LSTM respectively process the feature sequence sequentially and in reverse order to obtain the hidden layer states of the opposite time series, and connect them to obtain the same output. 5. a kind of rolling bearing RUL prediction method based on dropout-SAE and Bi-LSTM according to claim 1, is characterized in that, in step 3, The Bi-LSTM model is used to consider the past and future information of the data at the same time, and two hidden layer states with opposite time series are obtained through the forward LSTM and the backward LSTM, and then they are connected to obtain the same output; the forward LSTM and the backward LSTM can be Obtain the past information and future information of the input sequence respectively, and the hidden state H t of Bi-LSTM at time _t contains the forward

and backward

Among them: T is the sequence length, ct is the memory unit, xt is the input vector at time t, which is the deep feature of the rolling bearing vibration signal extracted by SAE; ht is the hidden state at time t.

Images