CN113222045A - Semi-supervised fault classification method based on weighted feature alignment self-encoder - Google Patents

Semi-supervised fault classification method based on weighted feature alignment self-encoder Download PDF

Info

Publication number
CN113222045A
CN113222045A CN202110575307.4A CN202110575307A CN113222045A CN 113222045 A CN113222045 A CN 113222045A CN 202110575307 A CN202110575307 A CN 202110575307A CN 113222045 A CN113222045 A CN 113222045A
Authority
CN
China
Prior art keywords
encoder
unlabeled
labeled
self
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110575307.4A
Other languages
Chinese (zh)
Other versions
CN113222045B (en
Inventor
张新民
张宏毅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN202110575307.4A priority Critical patent/CN113222045B/en
Publication of CN113222045A publication Critical patent/CN113222045A/en
Application granted granted Critical
Publication of CN113222045B publication Critical patent/CN113222045B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Probability & Statistics with Applications (AREA)
  • Image Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a semi-supervised fault classification method based on a weighted feature alignment self-encoder. Then, the weight of the unlabeled sample is calculated according to the probability density function of the error reconstructed by the training data. Further, a semi-supervised classification model based on the weighted feature alignment self-encoder is constructed by utilizing the labeled sample set, the unlabeled sample set and the corresponding weights. The weighted feature alignment self-encoder classification model designs a cross entropy training loss function based on weighted Sinkhorn distance, and the function enables the model to use labeled data and unlabeled data at the fine tuning stage, so that not only can deep mining of data information be realized, but also the generalization capability of a network model can be improved. Meanwhile, due to the introduction of a weighting strategy, the robustness of the model is obviously improved.

Description

Semi-supervised fault classification method based on weighted feature alignment self-encoder
Technical Field
The invention belongs to the field of industrial process control, and particularly relates to a semi-supervised fault classification method based on a weighted feature alignment self-encoder.
Background
Modern industrial processes are moving towards large scale, complex processes. How to ensure the safety of the production process is one of key problems which are focused on and need to be solved in the field of industrial process control. The fault diagnosis is a key technology for guaranteeing the safe operation of the industrial process, and has important significance for improving the product quality and the production efficiency. The fault classification belongs to a link in fault diagnosis, and automatic identification and judgment of fault types are realized by learning from historical fault information, so that production personnel are helped to quickly locate and repair the faults, and further loss caused by the faults is avoided. With the continuous development and progress of modern measurement means, a great deal of data is accumulated in the industrial production process. The data describes the actual conditions of each production stage of the manufacturing, provides valuable data resources for reading, analyzing and optimizing the manufacturing process, and is an intelligent source for realizing intelligent manufacturing. Therefore, how to reasonably utilize the data information accumulated in the manufacturing process to establish a data-driven intelligent analysis model to better serve the intelligent decision and quality control of the manufacturing process is a hot point of great concern in the industry. The data-driven fault classification method utilizes intelligent analysis technologies such as machine learning and deep learning to deeply mine, model and analyze industrial data and provide a data-driven fault diagnosis mode for users and industries. Most of the existing data-driven fault classification methods belong to supervised learning methods, and when sufficient labeled data can be obtained, the model can obtain excellent performance. However, it is difficult to obtain large, sufficient tagged data in certain industrial scenarios. Thus, there is often a large amount of unlabeled data and a small amount of labeled data. In order to effectively utilize the unlabeled data to improve the classification performance of the model, a fault classification method based on semi-supervised learning is gradually receiving attention. However, most existing semi-supervised fault classification methods mostly rely on certain data assumptions, such as semi-supervised learning methods based on statistical learning, semi-supervised learning methods based on graphs, and other methods for labeling unlabeled data based on cooperative training, self-training, etc., which all rely on one assumption, namely: the labeled and unlabeled swatches belong to the same distribution. However, this assumption has its limitation, data collected by an industrial process often include a large amount of noise and abnormal points, and may drift working conditions, labeled data is often manually screened and labeled by experts in the process field, while unlabeled samples are not screened, so that there is a high possibility that abnormal data different from the labeled data may occur in the unlabeled data. When the distribution of the non-labeled data is inconsistent with that of the labeled data, the performance of the semi-supervised algorithm is reduced and is even lower than that of the supervised algorithm which only uses the labeled data for training. Therefore, it is desirable to provide a robust semi-supervised learning method, so that the model can still accurately implement fault classification when the labeled data and the unlabeled data have inconsistent distribution.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a semi-supervised fault classification method based on a weighted feature alignment self-encoder, which comprises the following steps:
a semi-supervised fault classification method based on weighted feature alignment self-encoder includes the following steps:
the method comprises the following steps: collecting normal working condition data and various fault data of an industrial process to obtain a training data set for modeling: sample set with labels
Figure BDA0003084132720000021
And unlabeled sample set
Figure BDA0003084132720000022
Wherein x represents an input sample, y represents a sample label, m represents the number of labeled samples, and n represents the number of unlabeled samples;
step two: constructing a stacking self-encoder model for reconstruction, and training the stacking self-encoder model by utilizing a labeled sample set;
step three: estimating the probability density distribution of the reconstruction error of the training data, calculating the weight of the label-free sample, and further constructing a weighted feature alignment self-encoder classification model;
step four: and acquiring field working data, inputting the weighted features to align the self-encoder classification model, and outputting corresponding fault categories.
Further, the second step is specifically divided into the following sub-steps:
(2.1) constructing a stacked self-encoder model for reconstruction, comprising a multi-layer encoder and a decoder, wherein the output of the model is the reconstruction of the input, and the calculation formula is as follows:
Figure BDA0003084132720000023
Figure BDA0003084132720000024
wherein x represents the input, zkRepresenting the extracted k-th layer features, k representing the k-th layer of the stacked self-encoder,
Figure BDA0003084132720000025
and
Figure BDA0003084132720000026
representing the weight vector and the disparity vector of the encoder and decoder respectively,
Figure BDA0003084132720000027
reconstruction of the input by the representative model;
(2.2) training the stacked self-encoder model by adopting the labeled samples constructed in the step one and adopting a random gradient descent algorithm, wherein a model training loss function is defined as a reconstruction error of an input, and the reconstruction error is represented by the following formula:
Figure BDA0003084132720000028
wherein the content of the first and second substances,
Figure BDA0003084132720000029
representing the ith labeled input sample,
Figure BDA00030841327200000210
representing the reconstruction of the stacked auto-encoder;
(2.3) calculating the reconstruction error of the labeled sample by using the trained stacked self-encoder model
Figure BDA00030841327200000211
Wherein the reconstruction error of a single sample is calculated with reference to the following formula:
Figure BDA00030841327200000212
further, the third step is specifically divided into the following sub-steps:
(3.1) calculating the reconstruction error E of the labeled exemplarslCompliance chi2Distribution of
Figure BDA0003084132720000031
Distribution parameters g and h of
g·h＝mean(El) (5)
2g2·h＝variance(El) (6)
(3.2) calculating reconstruction error of unlabeled exemplar
Figure BDA0003084132720000032
The reconstruction error calculation formula of a single sample is the same as the formula (4);
(3.3) calculating the reconstruction error E of the unlabeled exemplarsuIn distribution ElProbability of occurrence of
Figure BDA0003084132720000033
To PuNormalizing to obtain the weight of the unlabeled sample
Figure BDA0003084132720000034
And (3.4) constructing a weighted feature alignment self-encoder classification model, and training the weighted feature alignment self-encoder classification model by adopting a labeled sample set, an unlabeled sample set and corresponding weights. The training process comprises the following steps: unsupervised pre-training and supervised fine tuning. In the unsupervised pre-training phase, labeled samples and unlabeled samples are used together to train a stacked self-encoder. The unsupervised pre-training method is the same as the steps (2.1) - (2.3). The supervised fine tuning is formed by adding a fully-connected neural network layer on a stacked self-encoder obtained by unsupervised pre-training and using the fully-connected neural network layer as output of categories, so as to obtain deep extraction features and category labels of the labeled samples and deep extraction features and predicted category label output of the unlabeled samples, and a specific calculation formula is as follows:
Figure BDA0003084132720000035
Figure BDA0003084132720000036
Figure BDA0003084132720000037
Figure BDA0003084132720000038
wherein the content of the first and second substances,
Figure BDA0003084132720000039
represents the deep-extracted features of the ith labeled sample,
Figure BDA00030841327200000310
class labels representing the predicted ith labeled exemplar, { wc，bcRepresenting weight vectors and deviation vectors of the fully connected neural network layer;
Figure BDA00030841327200000311
represents a deep extraction feature of the unlabeled exemplar,
Figure BDA00030841327200000312
a class label output representing a prediction;
(3.7) assuming the number of classes as F, obtaining deep extraction features of labeled exemplars and unlabeled exemplars corresponding to each class F e F
Figure BDA00030841327200000313
And
Figure BDA00030841327200000314
and weight of unlabeled exemplars
Figure BDA00030841327200000315
(3.8) calculating a training loss function of the weighted feature alignment self-encoder classification model using the following formula:
Figure BDA0003084132720000041
Figure BDA0003084132720000042
Figure BDA0003084132720000043
wherein, crossentropy represents a cross entropy loss function,
Figure BDA0003084132720000044
the representative weighted Sinkhorn distance function is used for measuring the distance between the characteristic distribution of the labeled data and the characteristic distribution of the unlabeled data belonging to the same category, and meanwhile, the weight reduction of the abnormal unlabeled sample with larger reconstruction error is realized; alpha is the weight of the Sinkhorn distance,
Figure BDA0003084132720000045
l being a network parameter2Regularization penalty term, β is its weight, pijRepresents corresponds toFeatures of labeled exemplars i of class f
Figure BDA0003084132720000046
Features to unlabeled sample j
Figure BDA0003084132720000047
Transition probability of dijRepresenting features of labeled exemplars i corresponding to class f
Figure BDA0003084132720000048
Features to unlabeled sample j
Figure BDA0003084132720000049
The distance of (a) to (b),
Figure BDA00030841327200000410
represents the weight of the unlabeled exemplar j corresponding to the class f, and mf and nf represent the number of labeled and unlabeled exemplars corresponding to the class f, respectively.
The invention has the following beneficial effects:
the invention provides a robust semi-supervised fault classification method based on a weighted feature alignment self-encoder, aiming at the problem of performance degradation of a traditional semi-supervised classification model when labeled data and unlabelled data are not distributed uniformly. The method designs a model training loss function based on a weighting and feature alignment strategy. The introduction of the weighting strategy improves the robustness of the semi-supervised classification model and reduces the problem of performance reduction of the classification model caused by inconsistent sample distribution. And the introduction of the characteristic alignment strategy enables the model to use the labeled data and the unlabeled data at the same time in the fine tuning stage, so that the deep mining of data information can be realized, and the generalization capability and classification performance of the network model can be improved.
Drawings
FIG. 1 is a schematic diagram of a stacked self-encoder;
FIG. 2 is a TE process flow diagram;
FIG. 3 is a schematic diagram of data log reconstruction errors;
FIG. 4 is a graph illustrating classification accuracy of different algorithms.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments, and the objects and effects of the present invention will become more apparent, it being understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.
The semi-supervised fault classification method based on the weighted feature alignment self-encoder comprises the steps of firstly using labeled data to carry out reconstruction pre-training on a stacked self-encoder, and estimating probability density distribution of reconstruction errors. Then, the weight of the unlabeled sample is calculated according to the probability density function of the error reconstructed by the training data. Further, a semi-supervised classification model based on the weighted feature alignment self-encoder is constructed by utilizing the labeled sample set, the unlabeled sample set and the corresponding weights. The weighted feature alignment self-encoder classification model designs a cross entropy training loss function based on weighted Sinkhorn distance, and the function enables the model to use labeled data and unlabeled data at the fine tuning stage, so that not only can deep mining of data information be realized, but also the generalization capability of a network model can be improved. Meanwhile, due to the introduction of a weighting strategy, the robustness of the model is obviously improved.
The method comprises the following specific steps:
the method comprises the following steps: collecting normal working condition data and various fault data of an industrial process to obtain a training data set for modeling: sample set with labels
Figure BDA0003084132720000051
And unlabeled sample set
Figure BDA0003084132720000052
Wherein x represents an input sample, y represents a sample label, m represents the number of labeled samples, and n represents the number of unlabeled samples;
step two: constructing a stacking self-encoder model for reconstruction, and training the stacking self-encoder model by utilizing a labeled sample set; the method is specifically divided into the following substeps:
(2.1) constructing a stacked self-encoder model for reconstruction, comprising a multi-layer encoder and a decoder, wherein the output of the model is the reconstruction of the input, and the calculation formula is as follows:
Figure BDA0003084132720000053
Figure BDA0003084132720000054
wherein x represents the input, zkRepresenting the extracted k-th layer features, k representing the k-th layer of the stacked self-encoder,
Figure BDA0003084132720000055
and
Figure BDA0003084132720000056
representing the weight vector and the disparity vector of the encoder and decoder respectively,
Figure BDA0003084132720000057
reconstruction of the input by the representative model;
(2.2) training the stacked self-encoder model by adopting the labeled sample set constructed in the step one and adopting a random gradient descent algorithm, wherein a model training loss function is defined as an input reconstruction error, and the reconstruction error is represented by the following formula:
Figure BDA0003084132720000058
wherein the content of the first and second substances,
Figure BDA0003084132720000059
representing the ith labeled input sample,
Figure BDA00030841327200000510
representing the reconstruction of the stacked auto-encoder;
(2.3) good training by using trainingComputing reconstruction errors for labeled samples
Figure BDA00030841327200000511
Wherein the reconstruction error of a single sample is calculated with reference to the following formula:
Figure BDA00030841327200000512
step three: estimating the probability density distribution of the reconstruction error of the training data, calculating the weight of the label-free sample, and further constructing a weighted feature alignment self-encoder classification model;
the third step is specifically divided into the following substeps:
(3.1) calculating the reconstruction error E of the labeled exemplarslCompliance chi2Distribution of
Figure BDA00030841327200000513
Distribution parameters g and h of
g·h＝mean(El) (5)
2g2·h＝variance(El) (6)
(3.2) calculating reconstruction error of unlabeled exemplar
Figure BDA0003084132720000061
The reconstruction error calculation formula of a single sample is the same as the formula (4);
(3.3) calculating the reconstruction error E of the unlabeled exemplarsuIn distribution ElProbability of occurrence of
Figure BDA0003084132720000062
To PuNormalizing to obtain the weight of the unlabeled sample
Figure BDA0003084132720000063
And (3.4) constructing a weighted feature alignment self-encoder classification model, and training the weighted feature alignment self-encoder classification model by adopting a labeled sample set, an unlabeled sample set and corresponding weights. The training process can be divided into: unsupervised pre-training and supervised fine tuning.
In the unsupervised pre-training phase, labeled samples and unlabeled samples are used together to train a stacked self-encoder. The unsupervised pre-training method is the same as the steps (2.1) - (2.3), namely, a stacking self-encoder model for reconstruction is constructed firstly, and then the stacking self-encoder is trained by using the labeled sample and the unlabeled sample;
the supervised fine tuning is formed by adding a fully-connected neural network layer on a stacked self-encoder obtained by unsupervised pre-training and using the fully-connected neural network layer as output of categories, so as to obtain deep extraction features and category labels of the labeled samples and deep extraction features and predicted category label output of the unlabeled samples, and a specific calculation formula is as follows:
Figure BDA0003084132720000064
Figure BDA0003084132720000065
Figure BDA0003084132720000066
Figure BDA0003084132720000067
wherein the content of the first and second substances,
Figure BDA0003084132720000068
represents the deep-extracted features of the ith labeled sample,
Figure BDA0003084132720000069
class labels representing the predicted ith labeled exemplar, { wc，bcRepresenting weight vectors and deviation vectors of the fully connected neural network layer;
Figure BDA00030841327200000610
represents a deep extraction feature of the unlabeled exemplar,
Figure BDA00030841327200000611
a class label output representing a prediction;
(3.7) assuming the number of classes as F, deep-extraction features of labeled and unlabeled exemplars corresponding to each class F E F are obtained according to the following formula
Figure BDA00030841327200000612
And
Figure BDA00030841327200000613
and weight of unlabeled exemplars
Figure BDA00030841327200000614
(3.8) calculating a training loss function of the weighted feature alignment self-encoder classification model using the following formula:
Figure BDA0003084132720000071
Figure BDA0003084132720000072
Figure BDA0003084132720000073
wherein, crossentropy represents a cross entropy loss function,
Figure BDA0003084132720000074
representing a weighted Sinkhorn distance function, alpha is the weight of the Sinkhorn distance,
Figure BDA0003084132720000075
l being a network parameter2Regularization penalty term, β is its weight, pijRepresenting features of labeled exemplars i corresponding to class f
Figure BDA0003084132720000076
Features to unlabeled sample j
Figure BDA0003084132720000077
Transition probability of dijRepresenting features of labeled exemplars i corresponding to class f
Figure BDA0003084132720000078
Features to unlabeled sample j
Figure BDA0003084132720000079
The distance of (a) to (b),
Figure BDA00030841327200000710
represents the weight of the unlabeled exemplar j corresponding to the class f, and mf and nf represent the number of labeled and unlabeled exemplars corresponding to the class f, respectively. The main purpose of the newly designed training loss function based on the weighted Sinkhorn distance is two. One is to align the labeled data and unlabeled data belonging to the same class in the fine tuning stage by stacking the features extracted from the encoder so that their distributions are close. And the other is that the weight reduction of the abnormal unlabeled sample with larger reconstruction error is realized through the weighted Sinkhorn characteristic distance with the unlabeled sample weight.
Step four: and acquiring field working data, inputting the weighted features to align the self-encoder classification model, and outputting corresponding fault categories.
The validity of the method of the invention is verified below with a specific industrial process example. All data are collected on a Tennessee-Eastman (TE) chemical engineering experiment simulation platform in the United states, and the platform is widely applied to the field of fault diagnosis and fault classification as a typical chemical process research object. The TE process is schematically shown in FIG. 2, and its main equipment includes a continuous stirred tank reactor, a gas-liquid separation column, a centrifugal compressor, a partial condenser and a reboiler. The modeled process data contained 16 process variables and 10 fault categories, and the detailed process variable and fault information descriptions are shown in tables 1 and 2, respectively.
TABLE 1
Figure BDA00030841327200000711
Figure BDA0003084132720000081
TABLE 2
Fault numbering Description of the invention Type of failure
1 A/C describes the feed flow ratio variation (stream 4) Step change
5 Condenser cooling water inlet temperature change Step change
7 Material C pressure loss (stream 4) Step change
10 Temperature Change of Material C (stream 4) Random variable
14 Cooling water valve of reactor Viscous glue
The collected data contains a total of 3600 samples from 6 classes, 600 samples for each class. The collected data was divided into training data (containing 300 labeled data and 3000 unlabeled data) and test data (containing 300 labeled data). In order to simulate the situation that the distribution of the non-tag data is inconsistent with that of the tag data, Gaussian noise is added into the original non-tag data according to a certain proportion.
Fig. 3 shows log reconstruction errors of labeled data, normal unlabeled data, and abnormal unlabeled data that are not in accordance with the distribution of the labeled data under the stacked self-encoder reconstruction model. As is apparent from fig. 3, the reconstruction errors of the labeled data and the normal unlabeled data are relatively close, while the reconstruction error of the abnormal unlabeled data is significantly larger than the reconstruction errors of the labeled data and the normal unlabeled data. This is the basis for detecting abnormally distributed unlabeled data from the encoder based on weighted feature alignment.
Fig. 4 shows the classification accuracy of the three algorithms under different labeled and unlabeled data distribution inconsistent ratios. The MLP method is a supervised neural network classification model, the Tri-tracking method is a neural network classification model obtained based on cooperative training, and the Weighted FA-SAE method is a Weighted feature alignment-based self-encoder classification model provided by the invention. Tri-tracking and Weighted FA-SAE belong to a semi-supervised deep learning network model. As can be seen from the figure, the classification performance of most semi-supervised learning algorithms is superior to that of supervised algorithms; in addition, with the gradual expansion of the distribution inconsistency ratio of the labeled data and the unlabeled data, the performance of the semi-supervised algorithm is reduced, wherein when the distribution inconsistency reaches 90%, the classification precision of the Tri-tracking method is even lower than that of the supervised MLP method. In contrast, the Weighted FA-SAE method provided by the invention has better classification performance than MLP and Tri-tracking methods under different degrees of distribution inconsistency rate.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and although the invention has been described in detail with reference to the foregoing examples, it will be apparent to those skilled in the art that various changes in the form and details of the embodiments may be made and equivalents may be substituted for elements thereof. All modifications, equivalents and the like which come within the spirit and principle of the invention are intended to be included within the scope of the invention.

Claims (3)

1. A semi-supervised fault classification method based on a weighted feature alignment self-encoder is characterized by comprising the following steps:
the method comprises the following steps: collecting normal working condition data and various fault data of an industrial process to obtain a training data set for modeling: sample set with labels
Figure FDA0003084132710000011
And unlabeled sample set
Figure FDA0003084132710000012
Wherein x represents an input sample, y represents a sample label, m represents the number of labeled samples, and n represents the number of unlabeled samples.
Step two: constructing a stacking self-encoder model for reconstruction, and training the stacking self-encoder model by utilizing a labeled sample set;
step three: estimating the probability density distribution of the reconstruction error of the training data, calculating the weight of the label-free sample, and further constructing a weighted feature alignment self-encoder classification model;
step four: and acquiring field working data, inputting the weighted features to align the self-encoder classification model, and outputting corresponding fault categories.
2. The semi-supervised fault classification method based on weighted feature alignment self-encoder according to claim 1, wherein the second step is specifically divided into the following sub-steps:
(2.1) constructing a stacked self-encoder model for reconstruction, comprising a multi-layer encoder and a decoder, wherein the output of the model is the reconstruction of the input, and the calculation formula is as follows:
Figure FDA0003084132710000013
Figure FDA0003084132710000014
wherein x represents the input, zkRepresenting the extracted k-th layer features, k representing the k-th layer of the stacked self-encoder,
Figure FDA0003084132710000015
and
Figure FDA0003084132710000016
representing the weight vector and the disparity vector of the encoder and decoder respectively,
Figure FDA0003084132710000017
reconstruction of the input by the representative model;
(2.2) training the stacked self-encoder model by adopting the labeled samples constructed in the step one and adopting a random gradient descent algorithm, wherein a model training loss function is defined as a reconstruction error of an input, and the reconstruction error is represented by the following formula:
Figure FDA0003084132710000018
wherein the content of the first and second substances,
Figure FDA0003084132710000019
representing the ith labeled input sample,
Figure FDA00030841327100000110
representing the reconstruction of the stacked auto-encoder;
(2.3) calculating the reconstruction error of the labeled sample by using the trained stacked self-encoder model
Figure FDA00030841327100000111
Wherein the reconstruction error of a single sample is calculated with reference to the following formula:
Figure FDA00030841327100000112
3. the semi-supervised fault classification method based on weighted feature alignment self-encoder according to claim 2, wherein the step three is particularly divided into the following sub-steps:
(3.1) calculating the reconstruction error E of the labeled exemplarslCompliance chi2Distribution of
Figure FDA0003084132710000021
Distribution parameters g and h of
g·h＝mean(El) (5)
2g2·h＝variance(El) (6)
(3.2) calculating reconstruction error of unlabeled exemplar
Figure FDA0003084132710000022
The reconstruction error calculation formula of a single sample is the same as the formula (4);
(3.3) calculating the reconstruction error E of the unlabeled exemplarsuIn distribution ElTake place ofProbability of (2)
Figure FDA0003084132710000023
To PuNormalizing to obtain the weight of the unlabeled sample
Figure FDA0003084132710000024
And (3.4) constructing a weighted feature alignment self-encoder classification model, and training the weighted feature alignment self-encoder classification model by adopting a labeled sample set, an unlabeled sample set and corresponding weights. The training process comprises the following steps: unsupervised pre-training and supervised fine tuning. In the unsupervised pre-training phase, labeled samples and unlabeled samples are used together to train a stacked self-encoder. The unsupervised pre-training method is the same as the steps (2.1) - (2.3). The supervised fine tuning is formed by adding a fully-connected neural network layer on a stacked self-encoder obtained by unsupervised pre-training and using the fully-connected neural network layer as output of categories, so as to obtain deep extraction features and category labels of the labeled samples and deep extraction features and predicted category label output of the unlabeled samples, and a specific calculation formula is as follows:
Figure FDA0003084132710000025
Figure FDA0003084132710000026
Figure FDA0003084132710000027
Figure FDA0003084132710000028
wherein the content of the first and second substances,
Figure FDA0003084132710000029
represents the deep-extracted features of the ith labeled sample,
Figure FDA00030841327100000210
class labels representing the predicted ith labeled exemplar, { wc，bcRepresenting weight vectors and deviation vectors of the fully connected neural network layer;
Figure FDA00030841327100000211
represents a deep extraction feature of the unlabeled exemplar,
Figure FDA00030841327100000212
a class label output representing a prediction;
(3.7) assuming the number of classes as F, obtaining deep extraction features of labeled exemplars and unlabeled exemplars corresponding to each class F e F
Figure FDA00030841327100000213
And
Figure FDA00030841327100000214
and weight of unlabeled exemplars
Figure FDA00030841327100000215
(3.8) calculating a training loss function of the weighted feature alignment self-encoder classification model using the following formula:
Figure FDA0003084132710000031
Figure FDA0003084132710000032
Figure FDA0003084132710000033
wherein, crossentropy represents a cross entropy loss function,
Figure FDA0003084132710000034
the representative weighted Sinkhorn distance function is used for measuring the distance between the characteristic distribution of the labeled data and the characteristic distribution of the unlabeled data belonging to the same category, and meanwhile, the weight reduction of the abnormal unlabeled sample with larger reconstruction error is realized; alpha is the weight of the Sinkhorn distance,
Figure FDA0003084132710000035
l being a network parameter2Regularization penalty term, β is its weight, pijRepresenting features of labeled exemplars i corresponding to class f
Figure FDA0003084132710000036
Features to unlabeled sample j
Figure FDA0003084132710000037
Transition probability of dijRepresenting features of labeled exemplars i corresponding to class f
Figure FDA0003084132710000038
Features to unlabeled sample j
Figure FDA0003084132710000039
The distance of (a) to (b),
Figure FDA00030841327100000310
represents the weight of the unlabeled exemplar j corresponding to the class f, and mf and nf represent the number of labeled and unlabeled exemplars corresponding to the class f, respectively.
CN202110575307.4A 2021-05-26 2021-05-26 Semi-supervised fault classification method based on weighted feature alignment self-encoder Active CN113222045B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110575307.4A CN113222045B (en) 2021-05-26 2021-05-26 Semi-supervised fault classification method based on weighted feature alignment self-encoder

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110575307.4A CN113222045B (en) 2021-05-26 2021-05-26 Semi-supervised fault classification method based on weighted feature alignment self-encoder

Publications (2)

Publication Number Publication Date
CN113222045A true CN113222045A (en) 2021-08-06
CN113222045B CN113222045B (en) 2022-06-24

Family

ID=77098569

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110575307.4A Active CN113222045B (en) 2021-05-26 2021-05-26 Semi-supervised fault classification method based on weighted feature alignment self-encoder

Country Status (1)

Country Link
CN (1) CN113222045B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113705729A (en) * 2021-09-27 2021-11-26 中原动力智能机器人有限公司 Garbage classification model modeling method, garbage classification device and garbage classification medium
CN114819108A (en) * 2022-06-22 2022-07-29 中国电力科学研究院有限公司 Fault identification method and device for comprehensive energy system
CN115184054A (en) * 2022-05-30 2022-10-14 深圳技术大学 Mechanical equipment semi-supervised fault detection and analysis method, device, terminal and medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111026058A (en) * 2019-12-16 2020-04-17 浙江大学 Semi-supervised deep learning fault diagnosis method based on Watherstein distance and self-encoder
CN112183581A (en) * 2020-09-07 2021-01-05 华南理工大学 Semi-supervised mechanical fault diagnosis method based on self-adaptive migration neural network

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111026058A (en) * 2019-12-16 2020-04-17 浙江大学 Semi-supervised deep learning fault diagnosis method based on Watherstein distance and self-encoder
CN112183581A (en) * 2020-09-07 2021-01-05 华南理工大学 Semi-supervised mechanical fault diagnosis method based on self-adaptive migration neural network

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
SHEN ZHANG ET AL.: "Semi-Supervised Bearing Fault Diagnosis and Classification Using Variational Autoencoder-Based Deep Generative Models", 《IEEE SENSORS JOURNAL》 *
邵伟明等: "基于循环神经网络的半监督动态软测量建模方法", 《电子测量与仪器学报》 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113705729A (en) * 2021-09-27 2021-11-26 中原动力智能机器人有限公司 Garbage classification model modeling method, garbage classification device and garbage classification medium
CN115184054A (en) * 2022-05-30 2022-10-14 深圳技术大学 Mechanical equipment semi-supervised fault detection and analysis method, device, terminal and medium
CN115184054B (en) * 2022-05-30 2022-12-27 深圳技术大学 Mechanical equipment semi-supervised fault detection and analysis method, device, terminal and medium
WO2023231374A1 (en) * 2022-05-30 2023-12-07 深圳技术大学 Semi-supervised fault detection and analysis method and apparatus for mechanical device, terminal, and medium
CN114819108A (en) * 2022-06-22 2022-07-29 中国电力科学研究院有限公司 Fault identification method and device for comprehensive energy system

Also Published As

Publication number Publication date
CN113222045B (en) 2022-06-24

Similar Documents

Publication Publication Date Title
CN113222045B (en) Semi-supervised fault classification method based on weighted feature alignment self-encoder
CN109146246B (en) Fault detection method based on automatic encoder and Bayesian network
CN106649789B (en) It is a kind of based on the industrial process Fault Classification for integrating semi-supervised Fei Sheer and differentiating
CN106843195B (en) The Fault Classification differentiated based on adaptive set at semi-supervised Fei Sheer
CN113642754B (en) Complex industrial process fault prediction method based on RF noise reduction self-coding information reconstruction and time convolution network
CN101169623B (en) Non-linear procedure fault identification method based on kernel principal component analysis contribution plot
CN109297689B (en) Large-scale hydraulic machinery intelligent diagnosis method introducing weight factors
CN113222046B (en) Feature alignment self-encoder fault classification method based on filtering strategy
CN108875771A (en) A kind of failure modes model and method being limited Boltzmann machine and Recognition with Recurrent Neural Network based on sparse Gauss Bernoulli Jacob
CN111026058B (en) Semi-supervised deep learning fault diagnosis method based on Watherstein distance and self-encoder
CN107153409A (en) A kind of nongausian process monitoring method based on missing variable modeling thinking
CN108345284A (en) A kind of quality dependent failure detection method becoming gauge block based on two
CN108375965A (en) A kind of nongausian process monitoring method rejected based on changeable gauge block crossing dependency
CN110765587A (en) Complex petrochemical process fault diagnosis method based on dynamic regularization judgment local retention projection
WO2023273249A1 (en) Tsvm-model-based abnormality detection method for automatic verification system of smart electricity meter
CN108830006B (en) Linear-nonlinear industrial process fault detection method based on linear evaluation factor
CN112904810A (en) Process industry nonlinear process monitoring method based on effective feature selection
CN114757269A (en) Complex process refined fault detection method based on local subspace-neighborhood preserving embedding
CN106022352A (en) Submersible piston pump fault diagnosis method based on support vector machine
CN111639304A (en) CSTR fault positioning method based on Xgboost regression model
CN111259523A (en) Process monitoring method based on KPCA-CVA model and random algorithm
CN114492614A (en) Method and device for classifying faults in hot rolling process of strip steel based on ensemble learning
CN116204825A (en) Production line equipment fault detection method based on data driving
CN115564021A (en) Fault root cause sequencing method in polyester fiber polymerization process
CN115618708A (en) Equipment health state prediction method based on incremental inform algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant