CN113222045B - Semi-supervised fault classification method based on weighted feature alignment self-encoder - Google Patents

Abstract

The invention discloses a semi-supervised fault classification method based on weighted feature alignment self-encoder. The method firstly uses labeled data to reconstruct and pre-train the stacked self-encoder, and estimates the probability density distribution of reconstruction errors. Then, the probability density function of the error is reconstructed according to the training data, and the weight of the unlabeled samples is calculated. Further, a semi-supervised classification model based on weighted feature alignment autoencoder is constructed by using the labeled sample set, the unlabeled sample set and the corresponding weights. The weighted feature alignment autoencoder classification model designs a cross-entropy training loss function based on the weighted Sinkhorn distance. This function enables the model to use both labeled data and unlabeled data in the fine-tuning stage, which can not only achieve in-depth mining of data information, but also improve The generalization ability of the network model. At the same time, due to the introduction of the weighting strategy, the robustness of the model is significantly improved.

Description

A Semi-Supervised Fault Classification Method Based on Weighted Feature Alignment Autoencoder

technical field

The invention belongs to the field of industrial process control, in particular to a semi-supervised fault classification method based on a weighted feature alignment self-encoder.

Background technique

Modern industrial processes are moving towards large-scale and complex development. How to ensure the safety of the production process is one of the key issues that the industrial process control field focuses on and needs to solve. Fault diagnosis is a key technology to ensure the safe operation of industrial processes, and it is of great significance to improve product quality and production efficiency. Fault classification is a part of fault diagnosis. By learning from historical fault information, automatic identification and judgment of fault types can be realized, thereby helping production personnel to quickly locate and repair faults and avoid further losses caused by faults. With the continuous development and progress of modern measurement methods, a large amount of data has been accumulated in the industrial production process. Data describes the real situation of each production stage of manufacturing, provides valuable data resources for understanding, analyzing and optimizing the manufacturing process, and is an intelligent source for realizing intelligent manufacturing. Therefore, how to reasonably use the data information accumulated in the manufacturing process to establish a data-driven intelligent analysis model to better serve the intelligent decision-making and quality control of the manufacturing process is a hot issue that the industry pays more attention to. The data-driven fault classification method uses intelligent analysis technologies such as machine learning and deep learning to deeply mine, model and analyze industrial data, and provide data-driven fault diagnosis models for users and industries. Most of the existing data-driven fault classification methods belong to supervised learning methods, and when sufficient labeled data is available, the model can achieve excellent performance. However, it is difficult to obtain large and sufficient labeled data in some industrial scenarios. Therefore, there is often a large amount of unlabeled data and a small amount of labeled data. In order to effectively utilize unlabeled data to improve the classification performance of the model, fault classification methods based on semi-supervised learning have gradually attracted attention. However, most of the existing semi-supervised fault classification methods mostly rely on certain data assumptions, such as statistical learning-based semi-supervised learning methods, graph-based semi-supervised learning methods, and others based on co-training, self-training, etc. Data labeling methods rely on an assumption that labeled samples and unlabeled samples belong to the same distribution. However, this assumption has its limitations. The data collected in the industrial process often contains a lot of noise and abnormal points, and the drift of the working conditions may occur. The labeled data is often manually screened and labeled by experts in the process field. , while the unlabeled samples are not filtered, so there is a high probability that there will be abnormal data in the unlabeled data distribution that is different from the labeled data. When the distribution of unlabeled data and labeled data is inconsistent, semi-supervised algorithms will suffer from performance degradation, even lower than supervised algorithms trained only with labeled data. Therefore, there is an urgent need to provide a robust semi-supervised learning method, so that the model can still accurately implement fault classification when there is distribution inconsistency between labeled data and unlabeled data.

SUMMARY OF THE INVENTION

The purpose of the present invention is to aim at the deficiencies of the prior art, to provide a semi-supervised fault classification method based on the weighted feature alignment autoencoder, the method comprises the following steps:

A semi-supervised fault classification method based on weighted feature alignment autoencoder, the method includes the following steps:

和无标签样本集

其中，x代表输入样本，y代表样本标签，m表示有标签样本个数，n表示无标签样本个数；Step 1: Collect the normal working condition data of the industrial process and various fault data, and obtain the training data set for modeling: the labeled sample set

and unlabeled sample set

Among them, x represents the input sample, y represents the sample label, m represents the number of labeled samples, and n represents the number of unlabeled samples;

Step 2: Build a stacked autoencoder model for reconstruction, and use the labeled sample set to train the stacked autoencoder model;

Step 3: Estimate the probability density distribution of the reconstruction error of the training data, calculate the weight of unlabeled samples, and further construct a weighted feature alignment autoencoder classification model;

Step 4: Collect field work data and input the weighted feature alignment autoencoder classification model, and output the corresponding fault category.

Further, the step 2 is specifically divided into the following sub-steps:

(2.1) Build a stacked autoencoder model for reconstruction, including multi-layer encoders and decoders, the model output is the reconstruction of the input, and the calculation formula is as follows:

和

分别表示编码器和解码器的权重向量和偏差向量，

代表模型对输入的重构；where x represents the input, z _k represents the extracted k-th layer features, k represents the k-th layer of the stacked autoencoder,

and

represent the weight vector and bias vector of the encoder and decoder, respectively,

Represents the reconstruction of the input by the model;

(2.2) Using the labeled samples constructed in step 1, the stacked autoencoder model is trained by the stochastic gradient descent algorithm, and the model training loss function is defined as the reconstruction error of the input, and the reconstruction error is represented by the following formula:

代表第i个有标签输入样本，

代表堆叠自编码器对它的重构；in,

represents the ith labeled input sample,

represents its reconstruction by stacked autoencoders;

其中，单个样本的重构误差参照如下公式计算：(2.3) Use the trained stacked autoencoder model to calculate the reconstruction error of labeled samples

Among them, the reconstruction error of a single sample is calculated with reference to the following formula:

Further, the step 3 is specifically divided into the following sub-steps:

的分布参数g和h(3.1) Calculating the reconstruction error E _l of the labeled samples obeys the χ ² distribution

The distribution parameters g and h of

g·h=mean(E _l ) (5)

2g ² ·h=variance(E _l ) (6)

单个样本的重构误差计算公式和公式(4)相同；(3.2) Calculate the reconstruction error of unlabeled samples

The calculation formula of the reconstruction error of a single sample is the same as formula (4);

对P_u进行归一化，得到无标签样本的权重

(3.3) Calculate the probability that the reconstruction error E _u of unlabeled samples occurs under the distribution E _l

Normalize P _u to get the weight of unlabeled samples

(3.4) Constructing a weighted feature alignment autoencoder classification model, using labeled sample sets, unlabeled sample sets and corresponding weights to train the weighted feature alignment autoencoder classification model. The training process is divided into: unsupervised pre-training and supervised fine-tuning. In the unsupervised pre-training stage, a stacked autoencoder is trained with labeled samples and unlabeled samples together. The unsupervised pre-training method is the same as steps (2.1)~(2.3). The supervised fine-tuning is to add a fully connected neural network layer to the stacked autoencoder obtained by unsupervised pre-training and use it as the output of the category, so as to obtain the deep extracted features and category labels of the labeled samples, and the unsupervised pre-training. The deep extraction features of the label samples and the predicted category label output, the specific calculation formula is as follows:

代表第i个有标签样本的深层提取特征，

代表预测的第i个有标签样本的类别标签，{w_c，b_c}表示全连接神经网络层的权重向量和偏差向量；

代表无标签样本的深层提取特征，

代表预测的类别标签输出；in,

represents the deep extracted features of the i-th labeled sample,

represents the class label of the predicted i-th labeled sample, {w _c , b _c } represents the weight vector and bias vector of the fully connected neural network layer;

represents the deep extracted features of unlabeled samples,

represents the predicted class label output;

和

以及无标签样本的权重

(3.7) Assuming that the number of classes is F, obtain the deep extraction features of labeled samples and unlabeled samples corresponding to each class f∈F

and

and the weights of unlabeled samples

(3.8) The training loss function of the weighted feature alignment autoencoder classification model is calculated by the following formula:

代表加权Sinkhorn距离函数，用于度量属于同一类别的有标签数据特征分布和无标签数据特征分布的距离，同时实现了对重构误差较大的异常无标签样本进行降权；α为Sinkhorn距离的权重，

为网络参数的L₂正则化惩罚项，β是它的权重，p_ij代表对应于类别f的有标签样本i的特征

到无标签样本j的特征

的转移概率，d_ij代表对应于类别f的有标签样本i的特征

到无标签样本j的特征

的距离，

代表对应于类别f的无标签样本j的权重，mf和nf分别代表对应于类别f的有标签样本和无标签样本的数量。where crossentropy represents the cross entropy loss function,

Represents the weighted Sinkhorn distance function, which is used to measure the distance between the feature distribution of labeled data and the feature distribution of unlabeled data belonging to the same category, and realizes the weight reduction of abnormal unlabeled samples with large reconstruction error; α is the Sinkhorn distance Weights,

is the L2 _{regularization} penalty term for the network parameters, β is its weight, and _pij represents the feature of the labeled sample i corresponding to the class f

to the features of unlabeled sample j

The transition probability of d _ij represents the feature of the labeled sample i corresponding to the class f

to the features of unlabeled sample j

the distance,

represents the weight of unlabeled sample j corresponding to class f, and mf and nf represent the number of labeled samples and unlabeled samples corresponding to class f, respectively.

The beneficial effects of the present invention are as follows:

Aiming at the problem that the performance of the traditional semi-supervised classification model is degraded when the distribution of labeled data and unlabeled data is inconsistent, the invention proposes a robust semi-supervised fault classification method based on a weighted feature alignment autoencoder. This method designs a model training loss function based on weighting and feature alignment strategy. The introduction of the weighting strategy improves the robustness of the semi-supervised classification model and reduces the performance degradation of the classification model caused by inconsistent sample distribution. The introduction of the feature alignment strategy enables the model to use both labeled data and unlabeled data in the fine-tuning stage, which can not only realize the deep mining of data information, but also improve the generalization ability and classification performance of the network model.

Description of drawings

Figure 1 is a schematic diagram of a stacked autoencoder;

Figure 2 is a flow chart of the TE process;

Fig. 3 is the schematic diagram of data logarithm reconstruction error;

Figure 4 is a schematic diagram of the classification accuracy of different algorithms.

Detailed ways

The present invention will be described in detail below according to the accompanying drawings and preferred embodiments, and the purpose and effects of the present invention will become clearer.

In the semi-supervised fault classification method based on the weighted feature alignment autoencoder of the present invention, the stacked autoencoder is reconstructed and pre-trained by using labeled data, and the probability density distribution of the reconstruction error is estimated. Then, the probability density function of the error is reconstructed according to the training data, and the weight of the unlabeled samples is calculated. Further, a semi-supervised classification model based on weighted feature alignment autoencoder is constructed by using the labeled sample set, the unlabeled sample set and the corresponding weights. The weighted feature alignment autoencoder classification model designs a cross-entropy training loss function based on the weighted Sinkhorn distance. This function enables the model to use both labeled data and unlabeled data in the fine-tuning stage, which can not only achieve in-depth mining of data information, but also improve The generalization ability of the network model. At the same time, due to the introduction of the weighting strategy, the robustness of the model is significantly improved.

The specific steps of the method of the present invention are as follows:

和无标签样本集

其中，x代表输入样本，y代表样本标签，m表示有标签样本个数，n表示无标签样本个数；Step 1: Collect the normal working condition data of the industrial process and various fault data, and obtain the training data set for modeling: the labeled sample set

and unlabeled sample set

Among them, x represents the input sample, y represents the sample label, m represents the number of labeled samples, and n represents the number of unlabeled samples;

Step 2: Build a stacked autoencoder model for reconstruction, and use the labeled sample set to train the stacked autoencoder model; it is divided into the following sub-steps:

(2.1) Build a stacked autoencoder model for reconstruction, including multi-layer encoders and decoders, the model output is the reconstruction of the input, and the calculation formula is as follows:

和

分别表示编码器和解码器的权重向量和偏差向量，

代表模型对输入的重构；where x represents the input, z _k represents the extracted k-th layer features, k represents the k-th layer of the stacked autoencoder,

and

represent the weight vector and bias vector of the encoder and decoder, respectively,

Represents the reconstruction of the input by the model;

(2.2) Using the labeled sample set constructed in step 1, the stacked autoencoder model is trained by the stochastic gradient descent algorithm, and the model training loss function is defined as the reconstruction error of the input, and the reconstruction error is represented by the following formula :

代表第i个有标签输入样本，

代表堆叠自编码器对它的重构；in,

represents the ith labeled input sample,

represents its reconstruction by stacked autoencoders;

其中，单个样本的重构误差参照如下公式计算：(2.3) Use the trained stacked autoencoder model to calculate the reconstruction error of labeled samples

Among them, the reconstruction error of a single sample is calculated with reference to the following formula:

Step 3: Estimate the probability density distribution of the reconstruction error of the training data, calculate the weight of unlabeled samples, and further construct a weighted feature alignment autoencoder classification model;

The third step is specifically divided into the following sub-steps:

的分布参数g和h(3.1) Calculating the reconstruction error E _l of the labeled samples obeys the χ ² distribution

The distribution parameters g and h of

g·h=mean(E _l ) (5)

2g ² ·h=variance(E _l ) (6)

单个样本的重构误差计算公式和公式(4)相同；(3.2) Calculate the reconstruction error of unlabeled samples

The calculation formula of the reconstruction error of a single sample is the same as formula (4);

对P_u进行归一化，得到无标签样本的权重

(3.3) Calculate the probability that the reconstruction error E _u of unlabeled samples occurs under the distribution E _l

Normalize P _u to get the weight of unlabeled samples

(3.4) Constructing a weighted feature alignment autoencoder classification model, using labeled sample sets, unlabeled sample sets and corresponding weights to train the weighted feature alignment autoencoder classification model. The training process can be divided into: unsupervised pre-training and supervised fine-tuning.

In the unsupervised pre-training stage, a stacked autoencoder is trained with labeled samples and unlabeled samples together. The unsupervised pre-training method is the same as steps (2.1)~(2.3), that is, firstly construct a stacked autoencoder model for reconstruction, and then train the stacked autoencoder with labeled samples and unlabeled samples;

The supervised fine-tuning is to add a fully connected neural network layer to the stacked autoencoder obtained by unsupervised pre-training and use it as the output of the category, so as to obtain the deep extracted features and category labels of the labeled samples, and the unsupervised pre-training. The deep extraction features of the label samples and the predicted category label output, the specific calculation formula is as follows:

代表第i个有标签样本的深层提取特征，

代表预测的第i个有标签样本的类别标签，{w_c，b_c}表示全连接神经网络层的权重向量和偏差向量；

代表无标签样本的深层提取特征，

代表预测的类别标签输出；in,

represents the deep extracted features of the i-th labeled sample,

represents the class label of the predicted i-th labeled sample, {w _c , b _c } represents the weight vector and bias vector of the fully connected neural network layer;

represents the deep extracted features of unlabeled samples,

represents the predicted class label output;

和

以及无标签样本的权重

(3.7) Assuming that the number of categories is F, the deep extraction features of labeled samples and unlabeled samples corresponding to each category f∈F are obtained according to the following formula

and

and the weights of unlabeled samples

(3.8) The training loss function of the weighted feature alignment autoencoder classification model is calculated by the following formula:

代表加权Sinkhorn距离函数，α为Sinkhorn距离的权重，

为网络参数的L₂正则化惩罚项，β是它的权重，p_ij代表对应于类别f的有标签样本i的特征

到无标签样本j的特征

的转移概率，d_ij代表对应于类别f的有标签样本i的特征

到无标签样本j的特征

的距离，

代表对应于类别f的无标签样本j的权重，mf和nf分别代表对应于类别f的有标签样本和无标签样本的数量。新设计的基于加权Sinkhorn距离的训练损失函数主要目的有两个。一个是在微调阶段对属于同一类别的有标签数据和无标签数据通过堆叠自编码器提取的特征对齐，使它们的分布接近。另一个是通过该带无标签样本权重的加权Sinkhorn特征距离，实现了对重构误差较大的异常无标签样本进行降权。where crossentropy represents the cross entropy loss function,

represents the weighted Sinkhorn distance function, α is the weight of the Sinkhorn distance,

is the L2 _{regularization} penalty term for the network parameters, β is its weight, and _pij represents the feature of the labeled sample i corresponding to the class f

to the features of unlabeled sample j

The transition probability of d _ij represents the feature of the labeled sample i corresponding to the class f

to the features of unlabeled sample j

the distance,

represents the weight of unlabeled sample j corresponding to class f, and mf and nf represent the number of labeled samples and unlabeled samples corresponding to class f, respectively. The newly designed training loss function based on weighted Sinkhorn distance has two main purposes. One is to align the features extracted by stacking autoencoders on labeled data and unlabeled data belonging to the same class in the fine-tuning stage to make their distributions close. The other is that the weighted Sinkhorn feature distance with the weight of the unlabeled sample is used to reduce the weight of the abnormal unlabeled sample with large reconstruction error.

Step 4: Collect field work data and input the weighted feature alignment autoencoder classification model, and output the corresponding fault category.

The effectiveness of the method of the present invention is verified below with a specific industrial process example. All data are collected on the Tennessee Eastman (TE) chemical experiment simulation platform, which is widely used in the field of fault diagnosis and fault classification as a typical chemical process research object. The flow chart of the TE process is shown in Figure 2, and its main equipment includes a continuous stirring reactor, a gas-liquid separation tower, a centrifugal compressor, a partial condenser and a reboiler. The modeling process data contains 16 process variables and 10 fault categories. The detailed process variables and fault information descriptions are shown in Table 1 and Table 2, respectively.

Table 1

Table 2

fault number describe Fault type 1 A/C describes the feed flow ratio change (stream 4) step 5 Condenser cooling water inlet temperature change step 7 Material C pressure loss (stream 4) step 10 Temperature change of material C (stream 4) Random Variables 14 Reactor cooling water valve sticky

The collected data contains a total of 3600 samples, which come from 6 categories, and 600 samples are collected in each category. The collected data is divided into training data (containing 300 labeled data and 3000 unlabeled data) and test data (containing 300 labeled data). In order to simulate the inconsistency between the unlabeled data and the labeled data distribution, we add Gaussian noise to the original unlabeled data according to a certain proportion.

Figure 3 presents the logarithmic reconstruction errors of labeled data, normal unlabeled data, and anomalous unlabeled data that are inconsistent with the distribution of labeled data under the stacked autoencoder reconstruction model. It can be clearly seen from Figure 3 that the reconstruction errors of labeled data and normal unlabeled data are relatively close, while the reconstruction error of abnormal unlabeled data is significantly larger than that of labeled data and normal unlabeled data. This is the basis for detecting abnormally distributed unlabeled data based on weighted feature alignment autoencoders.

Figure 4 shows the classification accuracy of the three algorithms under different proportions of inconsistent labeled and unlabeled data distributions. Among them, the MLP method is a supervised neural network classification model, the Tri-training method is a neural network classification model obtained based on collaborative training, and the Weighted FA-SAE method is a weighted feature alignment-based autoencoder classification model proposed by the present invention. Tri-traing and Weighted FA-SAE are semi-supervised deep learning network models. As can be seen from the figure, the classification performance of most semi-supervised learning algorithms is better than that of supervised algorithms; in addition, with the gradual expansion of the proportion of inconsistent distribution of labeled data and unlabeled data, the performance of semi-supervised algorithms has declined. Among them, when the distribution inconsistency rate reaches 90%, the classification accuracy of the Tri-training method is even lower than that of the supervised MLP method. In contrast, the Weighted FA-SAE method proposed in the present invention has better classification performance than MLP and Tri-training methods under different degrees of distribution inconsistency rates.

Those of ordinary skill in the art can understand that the above are only preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still Modifications are made to the technical solutions described in the foregoing examples, or equivalent replacements are made to some of the technical features. All modifications and equivalent replacements made within the spirit and principle of the invention shall be included within the protection scope of the invention.

Claims (2)

1. A semi-supervised fault classification method based on a weighted feature alignment self-encoder is characterized by comprising the following steps:

the method comprises the following steps: collecting normal working condition data and various fault data of an industrial process to obtain a training data set for modeling: sample set with labels

And unlabeled sample set

Wherein x represents an input sample, y represents a sample label, m represents the number of labeled samples, and n represents the number of unlabeled samples;

step two: constructing a stacking self-encoder model for reconstruction, and training the stacking self-encoder model by utilizing a labeled sample set;

step three: estimating the probability density distribution of the reconstruction error of the training data, calculating the weight of the label-free sample, and further constructing a weighted feature alignment self-encoder classification model;

the third step is specifically divided into the following substeps:

(3.1) calculating the reconstruction error E of the labeled exemplars_lCompliance chi²Distribution of

Distribution parameters g and h of

g·h＝mean(E_l)

2g²·h＝variance(E_l)

(3.2) calculating reconstruction error of unlabeled exemplar

The reconstruction error calculation formula for a single sample is as follows:

wherein,

representing the reconstruction of the model to the input;

(3.3) calculating the reconstruction error E of the unlabeled exemplars_uIn distribution E_lProbability of occurrence of

To P_uNormalizing to obtain the weight of the unlabeled sample

(3.4) constructing a weighted feature alignment self-encoder classification model, and training the weighted feature alignment self-encoder classification model by adopting a labeled sample set, an unlabeled sample set and corresponding weights; the training process comprises the following steps: unsupervised pre-training and supervised fine tuning; in an unsupervised pre-training stage, a stack self-encoder is trained by adopting a labeled sample and an unlabeled sample together; the supervised fine tuning is formed by adding a fully-connected neural network layer on a stacked self-encoder obtained by unsupervised pre-training and using the fully-connected neural network layer as output of categories, so as to obtain deep extraction features and category labels of the labeled samples and deep extraction features and predicted category label output of the unlabeled samples, and a specific calculation formula is as follows:

wherein,

represents the deep-extracted features of the ith labeled sample,

class label representing predicted ith labeled sample, { w_c，b_cRepresenting weight vectors and deviation vectors of the fully connected neural network layer;

represents a deep extraction feature of the unlabeled exemplar,

a class label output representing a prediction;

(3.5) the number of classes is F, and deep extraction features of labeled samples and unlabeled samples corresponding to each class F epsilon F are obtained

And

and weight of unlabeled exemplars

(3.6) calculating a training loss function of the weighted feature alignment self-encoder classification model using the following formula:

wherein, crossentropy represents a cross entropy loss function,

the representative weighted Sinkhorn distance function is used for measuring the distance between the characteristic distribution of the labeled data and the characteristic distribution of the unlabeled data belonging to the same category, and meanwhile, the weight reduction of the abnormal unlabeled sample with larger reconstruction error is realized; alpha is the weight of the Sinkhorn distance,

l being a network parameter₂Regularization penalty term, β is its weight, p_ijRepresenting features of labeled exemplars i corresponding to category f

Features to unlabeled sample j

Transition probability of d_ijRepresenting features of labeled exemplars i corresponding to class f

Features to unlabeled sample j

The distance of (a) to (b),

represents the weight of the unlabeled exemplar j corresponding to the class f, and mf and nf represent the number of labeled and unlabeled exemplars corresponding to the class f, respectively;

step four: and acquiring field working data, inputting the weighted features to align the self-encoder classification model, and outputting corresponding fault categories.

2. The semi-supervised fault classification method based on weighted feature alignment self-encoder according to claim 1, wherein the second step is specifically divided into the following sub-steps:

(2.1) constructing a stacked self-encoder model for reconstruction, comprising a multi-layer encoder and a decoder, wherein the output of the model is the reconstruction of the input, and the calculation formula is as follows:

wherein x represents the input, z_kRepresenting the extracted k-th layer features, k representing the k-th layer of the stacked self-encoder,

and

representing the weight vector and the disparity vector of the encoder and decoder respectively,

reconstruction of the input by the representative model;

(2.2) training the stacked self-encoder model by adopting the labeled samples constructed in the step one and adopting a random gradient descent algorithm, wherein a model training loss function is defined as a reconstruction error of an input, and the reconstruction error is represented by the following formula:

wherein,

representing the ith labeled input sample,

representing the reconstruction of the stacked auto-encoder;

(2.3) calculating the reconstruction error of the labeled sample by using the trained stacked self-encoder model

Wherein the reconstruction error of a single sample is calculated with reference to the following formula:

Images