WO2024045377A1

WO2024045377A1 - Deep-learning-based method for predicting residual life of aero-engine

Info

Publication number: WO2024045377A1
Application number: PCT/CN2022/135013
Authority: WO
Inventors: 乔非; 慕涵铄
Original assignee: 同济大学
Priority date: 2022-09-02
Filing date: 2022-11-29
Publication date: 2024-03-07
Also published as: CN115510740A

Abstract

The present invention relates to a deep-learning-based method for predicting the residual life of an aero-engine. The method comprises: acquiring data that reflects a full life cycle of an aero-engine, and obtaining the predicted residual life of the engine by means of a trained residual-life prediction model, wherein the life prediction model is constructed on the basis of deep learning. The training process of the prediction model comprises the following steps: S1, acquiring data that reflects a full life cycle of an aero-engine; S2, preprocessing the data; S3, on the basis of a random forest model, performing feature selection on the data; S4, on the basis of a Transformer model, performing feature extraction on data which has been subjected to feature selection; and S5, training an LSTM model by using data which has been subjected to feature extraction. Compared with the prior art, the present invention has the advantages of increasing the training speed, enhancing the stability of an algorithm, etc.

Description

A method for predicting the remaining life of aeroengines based on deep learning

Technical field

The present invention relates to the field of aviation engine life prediction, and in particular to a method for predicting the remaining life of an aerospace engine based on deep learning.

Background technique

As a key component for the normal operation of aircraft, aeroengines’ real-time monitoring of their health status and accurate prediction of remaining life are beneficial to ensuring the safe use of aircraft and formulating appropriate maintenance and replacement strategies. Practice has shown that prediction and health management technology is an emerging system state prediction and health management technology. Its application can achieve early warning of faults, prevent catastrophic accidents, and reduce maintenance costs. Therefore, it has been widely used in the industrial fields of various countries and achieved remarkable results.

As the most challenging direction in PHM, aeroengine RUL prediction has received widespread attention from industry scholars. With the development of sensor technology, it has become easier for people to obtain information about the performance degradation of engine components. This has driven the development of data-driven forecasting methods. This method uses monitoring data to extract the inherent laws of performance degradation to predict the performance degradation trend of components in a certain period of time in the future. The prediction cost is low and the accuracy is high, so it has become a current research hotspot.

With the development of sensor technology and the increase in the complexity of aeroengines, time series data has exploded. At the same time, a large amount of time series data has problems such as low unit data value, high timeliness and difficulty in feature extraction, which greatly limits the use of data. Use value. How to filter out the most relevant and effective features from massive data has become a difficult problem in aeroengine RUL prediction. Therefore, choosing appropriate methods for feature selection and feature extraction from high-dimensional, multi-parameter, large-scale data has a significant impact on the prediction effect of aeroengine RUL.

Chinese patent CN202210447896.2 discloses a method for predicting the remaining life of an aerospace engine based on deep learning. This method uses a remaining life prediction model composed of a self-attention mechanism and a bidirectional long short-term memory network. This model is used to select key features in time series data and assign corresponding weights, and then input them into the bidirectional long short-term memory network layer. The internal connections are mined, and finally the remaining life prediction results of the aircraft engine are obtained through the mapping relationship formed by the two fully connected layers.

The existing technology does not filter parameters, and redundant features and noisy data will increase model training time and reduce prediction accuracy. The self-attention mechanism feature extraction model in the existing technology simply uses the self-attention mechanism. As the network deepens, problems such as gradient disappearance and the decrease in the stability of the data feature distribution will occur, and the convergence speed is slow.

Contents of the invention

The purpose of the present invention is to provide a method for predicting the remaining life of an aerospace engine based on deep learning in order to overcome the shortcomings of the above-mentioned existing technologies.

The object of the present invention can be achieved through the following technical solutions:

As an aspect of the present invention, a method for predicting the remaining life of an aeroengine based on deep learning is provided. The method includes: obtaining data reflecting the entire life cycle of the aeroengine, and obtaining the predicted remaining life of the engine through a trained remaining life prediction model;

The life prediction model is built based on LSTM, and the training process of the prediction model includes the following steps:

S1 obtains data reflecting the entire life cycle of aeroengines;

S2 preprocesses the data;

S3 performs feature selection on data based on the random forest model;

S4 performs feature extraction on the data after feature selection based on the Transformer model;

S5 uses the feature extracted data to train the LSTM model.

As a preferred solution, the data reflecting the entire life cycle of the aeroengine includes engine ID number, operating cycle time, operating settings and sensor data.

As a preferred solution, the preprocessing of data includes:

Perform filtering and noise reduction processing on the data;

Normalize the data;

Eliminate data that has no impact on life prediction.

As a preferred solution, the steps of feature selection based on the random forest model are:

S31 selects the corresponding out-of-bag data for each decision tree and calculates the first out-of-bag data error;

S32 randomly adds noise interference to the characteristics of all samples of the out-of-bag data, and calculates the second out-of-bag data error;

S33 calculates the importance of features through the data error outside the first bag and the data error outside the second bag;

Based on the importance of each feature, a recursive feature elimination method is used to find the optimal feature subset.

As a preferred solution, the specific process of feature screening using recursive feature elimination is as follows:

S34 sorts the features according to their importance;

S35 determines the proportion of elimination, eliminates the corresponding proportion of features, and obtains a new feature set;

S36 determines whether the new feature set has a set number of features remaining. If not, return to step S34. If yes, proceed to step S37;

S37 outputs the feature set.

As a preferred solution, the Transformer model includes a multi-head attention mechanism layer based on a self-attention mechanism. The multi-head attention mechanism layer calculates the attention value of each data through the self-attention mechanism, and performs data processing based on the attention value of the data. Feature extraction, the attention value calculation process includes the following steps:

Three initialization matrices Query, Key and Value are respectively generated according to the input data. The Query matrix represents the vector that will be queried and calculated later, the Key matrix represents the vector that will be queried and calculated later, and the Value matrix represents the current actual Feature weight;

Calculate the similarity of the input data to the matrix Query and matrix Key;

Perform Softmax operation on the obtained similarity and normalize it;

Multiply the matrix Value and the value obtained by performing the Softmax operation, and sum the products to obtain the attention value under the current data.

As a preferred solution, the Transformer model also includes a residual connection layer. The residual connection layer solves the problem of gradient disappearance as the network deepens by establishing a residual network, and avoids the problem of data characteristics as the network deepens. The problem of reduced stability of distribution;

The formula of the residual block in the residual network is expressed as:

xi ₊₁ =h( _xi )+F( _xi )

Among them, x _i is the input of the i-th multi-head attention layer, x _i+1 is the output of the i-th multi-head attention layer, F( _xi ) is the residual part, and h( _xi ) is the 1*1 convolutional neural network. network.

As a preferred solution, the LSTM model training specifically includes the following steps:

The data extracted by the Transformer model features and the LSTM training labels are input into the LSTM model for model training. The training labels are the remaining life, that is, the time series of each engine is in reverse order.

As a preferred solution, the LSTM model training improves the model prediction effect by adjusting the number of memory units and the number of LSTM layers.

As a preferred solution, the LSTM model training uses the Dropout mechanism and dynamically adjusts the learning rate to improve the model's generalization ability and training speed.

Compared with the prior art, the present invention has the following beneficial effects:

1) The present invention uses random forest for feature extraction, uses the out-of-bag error rate as a metric to calculate the importance of features, and then eliminates corresponding proportions of features based on feature importance sorting, which can effectively eliminate redundant features and noise data. In addition to retaining features with strong resolving power and improving training speed, it also ensures the stability of the algorithm and avoids overfitting.

2) The present invention uses a transformer model with a residual layer for feature extraction, uses a multi-head attention mechanism layer based on the attention mechanism for feature extraction, extracts different features through multiple sets of matrices, and then splices the extracted feature vectors The purpose of getting the final output feature vector is to extract features from multiple sets of sensors at the same time to improve feature extraction efficiency. Then use the residual layer to solve the problem of gradient disappearance as the network deepens, and avoid the problem of decreased stability of data feature distribution as the network deepens, improve the convergence speed of the model, and improve the efficiency of model feature extraction.

Description of drawings

Figure 1 is a flow chart of LSTM model training of the present invention;

Figure 2 is a schematic diagram of data changes during the entire life cycle of the engine of the present invention;

Figure 3 is a flow chart of feature selection based on the random forest model in this invention;

Figure 4 is a diagram of the feature importance ranking results of the data set of the present invention; a) is the feature importance ranking result of the FD001 data set, b) is the feature importance ranking result of the FD002 data set, c) is the feature importance ranking result of the FD003 data set , d) is the ranking result of the feature importance of the FD004 data set;

Figure 5 is a diagram of the life prediction results of the data set of the present invention; a) is the prediction result of the FD001 data set, b) is the prediction result of the FD002 data set, c) is the prediction result of the FD003 data set, d) is the prediction result of the FD004 data set forecast result.

Detailed ways

The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention and provides detailed implementation modes and specific operating procedures. However, the protection scope of the present invention is not limited to the following embodiments.

This embodiment is an implementation of a method for predicting the remaining life of an aerospace engine based on deep learning. Its operation process includes the following parts:

S1. First, select several sets of multiple sensor data from the aero-engine data set that reflect the entire life cycle of the aero-engine, including engine ID number, operating cycle time, 3 operating settings and 21 sensor data.

Table 1 C-MAPSS data set introduction

S2. Perform data preprocessing for the selected data set;

In this embodiment, data selection is processed on the python platform. First, since the original data contains a large amount of random noise, a filter function is used for noise reduction, and the window width is set to 10 to improve data smoothness.

Since the monitoring data returned by multiple sensors of the engine represent different physical characteristics, that is, different dimensions and orders of magnitude, in order to eliminate the impact of data irregularities on the prediction effect and improve prediction accuracy, the original data is normalized.

Data normalization processing limits the original data to the range [0,1]. The specific formula is:

In the formula, x _i,j (t) represents the data value monitored by the j-th sensor of the i-th engine at time t, max(x _:,j ) represents the maximum value among all samples of the j-th sensor, min( x _:,j ) represents the minimum value among all samples of the j-th sensor, and x′ _i,j (t) represents the normalized value of x _i,j (t).

Considering that the data set has too many data dimensions, in order to eliminate irrelevant data attributes, reduce the number of dimensions, reduce model training time, and improve prediction accuracy, a preliminary analysis was conducted on 3 operation settings and 21 sensor data in the data set.

As shown in Figure 2, there are 7 attribute data that do not change during the entire engine life cycle, have no impact on RUL prediction, and are eliminated.

S3 performs feature selection on the data based on the RF feature selection model established on the Python platform, and sets parameters for the model.

We can think of RF feature selection as a process of ranking multiple features according to the importance of each feature. This paper uses the out-of-bag error rate as the metric for feature importance. The out-of-bag error rate represents the impact of this feature on the model accuracy when it participates in training and when it does not participate in training. This article’s method for calculating the importance of a feature x in a random forest is:

S31 selects the corresponding out-of-bag data (OOB) for each decision tree to calculate the out-of-bag data error, recorded as errOOB1.

S32 randomly adds noise interference to the feature X of all samples of out-of-bag data OOB, and calculates the out-of-bag data error again, which is recorded as errOOB2.

S33 Suppose there are N trees in the forest, then the importance of feature x is

On the basis of obtaining the importance of each feature through random forest, the method of recursive feature elimination (Recursive Feature Elimination) is used to find the optimal feature subset. The specific process of feature screening by RFE is:

S34 sorts the features according to their importance;

S36 determines whether the new feature set has a set number of features left. If not, return to step S34. If yes, go to step S37;

S37 outputs the feature set.

Use the sklearn library through the python platform to establish an RF feature selection model. The established RF feature selection model is in the shape of RandomForestRegressor (n_estimators=20, max_features=2). Among them, n_estimators is the number of decision trees, and max_features is the number of features that can be divided when selecting the most suitable attributes. This value cannot be exceeded. In this embodiment, the initial parameters are set to n_estimators=20 and max_features=2.

As shown in Figure 4, the features of the data set F001-FD004 are sorted by importance, and the sensor data is filtered and selected based on the importance sorting results.

S4. Feature extraction of data after feature selection based on the Transformer model specifically includes:

The sensor data after RF feature selection is input into the Transformer model for feature extraction, and the extracted features are used as input for subsequent LSTM model training.

Compared with traditional deep learning networks, the Transformer model introduces the Self-Attention mechanism. The introduction of this mechanism makes it easier for the Transformer model to extract long-distance interdependent features in the data, and can better extract global information. Therefore, considering the high-dimensional, multi-parameter, and large-scale data characteristics of aeroengines, this embodiment uses the Transformer model for feature extraction. The Transformer feature extraction process mainly uses the Self-Attention mechanism to calculate the correlation between each data, and extract data features based on the attention score of the data.

The Transformer model in this embodiment mainly includes a multi-head attention mechanism layer and a residual connection layer based on the self-attention mechanism.

The multi-head attention mechanism layer is composed of multiple sets of Query, Key, and Value matrices. Different features are extracted through multiple sets of Query, Key, and Value matrices, and then the extracted feature vectors are spliced together to obtain the final output feature vector. The purpose is to extract features of multiple sets of sensors at the same time to improve feature extraction efficiency. The main process is Included are:

Three matrices are generated based on the input data, namely Query, Key, and Value. The Query vector represents the vector that will be queried and calculated later, the Key vector represents the vector that will be queried and calculated later, and the Value vector represents the current actual feature weight. The three matrices are obtained by multiplying the input data with randomly initialized matrices.

For the input data, calculate the similarity of the data relative to other data, represented by f(Q,K _i ), i=1,2,3,.,..m.

(1) Perform Softmax operation on the obtained similarity and normalize it:

(2) Multiply and add the Value matrix and the value obtained by Softmax. The result is the self-attention mechanism. The attention value under the current data is the degree of attention to the data to achieve feature extraction. the goal of.

The residual connection layer solves the problem of gradient disappearance as the network deepens by establishing a residual network, and at the same time avoids the problem of declining stability of the data feature distribution as the network deepens, thereby improving the performance of the model. Convergence speed and improve model feature extraction efficiency.

The formula of the residual block in the residual network is expressed as:

xi ₊₁ =h( _xi )+F( _xi )

This embodiment establishes a Transformer model based on the python platform. The main body of the model is shaped like Attention (multiheads=20, head_dim=10), where multiheads is the number of multi-head attention heads, and head_dim is the dimension of the head. In this embodiment, the parameters are set to multiheads=20, head_dim=10

S5. Based on the LSTM model, use the data after feature extraction to predict the remaining life of the aeroengine, including:

Input the sensor data extracted by the Transformer model features and the set LSTM training labels into the LSTM model for model training. The training label is the remaining life of the engine, that is, the time series of each engine is taken in reverse order. Then the test set data is input into the trained model for RUL prediction. By adjusting the number of memory units and the number of LSTM layers to improve the model prediction effect, the Dropout mechanism and dynamic adjustment of the learning rate are introduced to improve the model's generalization ability and training speed. Set the number of LSTM network layers to one layer, the number of memory units to 20, and Dropout to 0.5. The effectiveness of the model is judged by comparing the accuracy of remaining life prediction under the test set.

This embodiment selects root mean square error (RMSE), mean absolute percentage error (MAPE) and mean absolute error (MAE) as evaluation indicators to measure the accuracy of model predictions. sex. The mathematical expression of the evaluation index is:

In the formula: N is the number of engines,

are the actual remaining life and predicted remaining life of the i-th engine respectively. As a comprehensive indicator of error analysis, RMSE reflects the accuracy of prediction, MAPE evaluates the fluctuation degree of model prediction error and reflects the robustness and stability of the model, and MAE can reflect the actual situation of model prediction error.

As shown in Figure 5, the FD001-FD004 test sets are input into the model respectively for RUL prediction. As can be seen from the figure, the model used in this embodiment has good performance in four data sets, indicating that the model has good versatility. In order to further compare the model performance of this embodiment, this embodiment compares this model with the CNN-LSTM model, LSTM model, and SVR model under four data sets FD001-FD004. The parameters of LSTM in other models are consistent with the parameters of the model in this article. Compare model performance through evaluation indicators RMSE, MAPE, and MAE. The RMSE value can reflect the accuracy of the prediction. The smaller the value, the higher the accuracy of the prediction; MAPE evaluates the fluctuation degree of the model prediction error, reflecting the robustness and stability of the model. The smaller the value, the more robust the model is. The better the stickiness and stability; the size of the MAE value can reflect the actual situation of the model's prediction error. The smaller the value, the smaller the prediction error of the model.

Table 2 Comparison of evaluation indicators under FD001

Table 3 Comparison of evaluation indicators under FD002

Table 4 Comparison of evaluation indicators under FD003

Table 5 Comparison of evaluation indicators under FD004

From Table 2 to Table 5, it can be seen from the horizontal comparison that under the four sets of data sets, the RMSE, MAPE, and MAE of the model in this embodiment are smaller than those of other models. Therefore, the prediction accuracy, stability, and The robustness is better than other comparison models. And in comparison with the SVR traditional machine learning model, it was found that the other three deep learning models performed well in RMSE, MAPE, and MAE and had more advantages. When comparing deep learning models, it can be found that the prediction performance of the LSTM model after feature extraction, whether it is CNN-LSTM or Transformer-LSTM, especially the latter, is further improved than that of the LSTM model.

From Table 2 to Table 5, it can be seen from the longitudinal comparison that as the number of engine states and failure modes increase in the data set, from FD001 to FD004, the RMSE, MAPE, and MAE performance of each model declines. However, this paper The prediction performance of the model used in the embodiment is relatively stable and the range of changes is small. Therefore, it can be seen that the model used in this embodiment is more suitable for aeroengine RUL prediction under complex conditions, and still has excellent performance under multiple operating conditions and multiple fault modes.

Considering that the aero-engine working environment is complex, there are many failure modes, and the data is high-dimensional, multi-parameter, and large-scale, this embodiment proposes an aero-engine RUL prediction method based on the RF-Transformer-LSTM model. This method is based on the RF model, Transformer model, and LSTM model. In view of the characteristics of the prediction data set such as high dimensionality, large scale, and many parameters, a good mapping relationship from high dimension to low dimension was established through the RF model and Transformer model to extract key features for RUL prediction. The extracted features are then input into the LSTM model, and the LSTM model is used to reflect the overall logical characteristics of the time series for RUL prediction. Experimental results show that the method proposed in this embodiment is feasible and effective, and has better prediction accuracy, stability, and robustness than the other three methods.

The preferred embodiments of the present invention are described in detail above. It should be understood that those skilled in the art can make many modifications and changes based on the concept of the present invention without creative efforts. Therefore, any technical solutions that can be obtained by those skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention and on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

A method for predicting the remaining life of an aeroengine based on deep learning, which is characterized in that the method includes: obtaining data reflecting the entire life cycle of the aeroengine, and obtaining the predicted remaining life of the engine through a trained remaining life prediction model;

The life prediction model is built based on LSTM, and the training process of the prediction model includes the following steps:

S1 obtains data reflecting the entire life cycle of aeroengines;

S2 preprocesses the data;

S3 performs feature selection on data based on the random forest model;

S4 performs feature extraction on the data after feature selection based on the Transformer model;

S5 uses the feature extracted data to train the LSTM model.
A method for predicting the remaining life of an aeroengine based on deep learning according to claim 1, characterized in that the data reflecting the entire life cycle of the aeroengine includes engine ID number, operating cycle time, operating settings and the parameters of each part of the engine. sensor data.
A method for predicting the remaining life of an aircraft engine based on deep learning according to claim 1, characterized in that the preprocessing of data includes:

Perform filtering and noise reduction processing on the data;

Normalize the data;

Eliminate data that has no impact on life prediction.
A deep learning-based aircraft engine remaining life prediction method according to claim 1, characterized in that the step of feature selection based on a random forest model is:

S31 selects the corresponding out-of-bag data for each decision tree and calculates the first out-of-bag data error;

S32 randomly adds noise interference to the characteristics of all samples of the out-of-bag data, and calculates the second out-of-bag data error;

S33 calculates the importance of features through the data error outside the first bag and the data error outside the second bag;

Based on the importance of each feature, a recursive feature elimination method is used to find the optimal feature subset.
A method for predicting the remaining life of an aeroengine based on deep learning according to claim 4, characterized in that the specific process of feature screening using the method of recursive feature elimination is:

S34 sorts the features according to their importance;

S35 determines the proportion of elimination, eliminates the corresponding proportion of features, and obtains a new feature set;

S36 determines whether the new feature set has a set number of features remaining. If not, return to step S34. If yes, proceed to step S37;

S37 outputs the feature set.
A method for predicting the remaining life of an aeroengine based on deep learning according to claim 1, characterized in that the Transformer model includes a multi-head attention mechanism layer based on a self-attention mechanism, and the multi-head attention mechanism layer uses a self-attention mechanism to The attention mechanism calculates the attention value of each data, and extracts data features based on the attention value of the data. The attention value calculation process includes the following steps:

Three initialization matrices Query, Key and Value are respectively generated according to the input data. The Query matrix represents the vector that will be queried and calculated later, the Key matrix represents the vector that will be queried and calculated later, and the Value matrix represents the current actual Feature weight;

Calculate the similarity of the input data to the matrix Query and matrix Key;

Perform Softmax operation on the obtained similarity and normalize it;

Multiply the matrix Value and the value obtained by performing the Softmax operation, and sum the products to obtain the attention value under the current data.
A method for predicting the remaining life of an aeroengine based on deep learning according to claim 6, characterized in that the Transformer model also includes a residual connection layer, and the residual connection layer solves the problem of network changes by establishing a residual network. The problem of gradient disappearance caused by deepening and avoiding the problem of decreasing stability of data feature distribution as the network deepens;

The formula of the residual block in the residual network is expressed as:

xi +1 =h( xi )+F( xi )

Among them, x i is the input of the i-th multi-head attention layer, x i+1 is the output of the i-th multi-head attention layer, F( xi ) is the residual part, and h( xi ) is the 1*1 convolutional neural network. network.
A deep learning-based aircraft engine remaining life prediction method according to claim 1, characterized in that the LSTM model training specifically includes the following steps:

The data extracted by the Transformer model features and the LSTM training labels are input into the LSTM model for model training. The training labels are the remaining life, that is, the time series of each engine is in reverse order.
A deep learning-based aircraft engine remaining life prediction method according to claim 8, characterized in that the LSTM model training improves the model prediction effect by adjusting the number of memory units and the number of LSTM layers.
A method for predicting the remaining life of an aeroengine based on deep learning according to claim 8, characterized in that the LSTM model training adopts a Dropout mechanism and a dynamic adjustment of the learning rate to improve the generalization ability and training speed of the model.