CN113283631A - Industrial equipment fault prediction method based on self-attention mechanism and time sequence convolution network - Google Patents

Industrial equipment fault prediction method based on self-attention mechanism and time sequence convolution network Download PDF

Info

Publication number
CN113283631A
CN113283631A CN202110393245.5A CN202110393245A CN113283631A CN 113283631 A CN113283631 A CN 113283631A CN 202110393245 A CN202110393245 A CN 202110393245A CN 113283631 A CN113283631 A CN 113283631A
Authority
CN
China
Prior art keywords
industrial equipment
model
data
time sequence
time
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110393245.5A
Other languages
Chinese (zh)
Inventor
包致成
张卫山
王涛
于泽沛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China University of Petroleum East China
Original Assignee
China University of Petroleum East China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China University of Petroleum East China filed Critical China University of Petroleum East China
Priority to CN202110393245.5A priority Critical patent/CN113283631A/en
Publication of CN113283631A publication Critical patent/CN113283631A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/04Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/215Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2458Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2474Sequence data queries, e.g. querying versioned data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/049Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/20Administration of product repair or maintenance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/04Manufacturing
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Human Resources & Organizations (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Strategic Management (AREA)
  • Economics (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Databases & Information Systems (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Business, Economics & Management (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Tourism & Hospitality (AREA)
  • Marketing (AREA)
  • Quality & Reliability (AREA)
  • Operations Research (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Game Theory and Decision Science (AREA)
  • Fuzzy Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Development Economics (AREA)
  • Manufacturing & Machinery (AREA)
  • Primary Health Care (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention provides an industrial equipment fault prediction method based on a self-attention mechanism and a time sequence convolution network, which is characterized by preprocessing data, performing time sequence processing and label translation, inputting the data into a multi-head self-attention multi-dimensional time sequence convolution network for training, and inputting an industrial equipment operation data matrix into a model for industrial equipment state prediction.

Description

Industrial equipment fault prediction method based on self-attention mechanism and time sequence convolution network
Technical Field
The invention relates to fault diagnosis of a neural network, industrial big data and industrial equipment, in particular to a fault prediction method of the industrial equipment based on a self-attention mechanism and a time sequence convolution network.
Background
The industry determines the speed, scale and level of the modernization of the national economy and plays a leading role in the national economy of all countries in the world of the present generation. With the rapid development of domestic industrial intelligence, the number of industrial equipment is continuously increased, and the internal structure is more and more complex, so that the method has important significance for predicting the faults of the industrial equipment. The closest techniques to the present invention in recent years are:
(1) the industrial equipment fault prediction method based on the CNN network comprises the following steps: CNN is a short name of a convolutional neural network, is one of deep learning algorithms, is widely applied to the image field and the data mining field at present, but cannot take the time sequence of data into consideration when the CNN is used for predicting the fault of industrial equipment.
(2) The LightGBM-based industrial equipment fault prediction method comprises the following steps: LightGBM uses a Histogram algorithm, occupies less memory, and has lower complexity of data separation, and the idea is to discretize continuous floating point features into k discrete values, construct a Histogram with a width of k, traverse training data, and count cumulative statistics of each discrete value in the Histogram. When the characteristics are selected, only the optimal segmentation point needs to be searched in a traversing mode according to the discrete value of the histogram, but the time sequence of the fault data of the industrial equipment cannot be effectively processed when the LightGBM is used for processing.
In order to make up for the fact that the traditional method cannot mine the time sequence of data, the method makes full use of the advantages of a time sequence convolution network and a self-attention mechanism, and further improvement of the accuracy of industrial equipment fault prediction is achieved.
Disclosure of Invention
In order to solve the defects and shortcomings in the prior art, the invention provides an industrial equipment fault prediction method based on a self-attention mechanism and a time sequence convolution network. By using a time sequence convolution network, the fault occurrence characteristics of the industrial equipment are centrally mined from the fault data of the industrial equipment, and the time sequence characteristics of the data are fully utilized under the condition of ensuring the sufficient receptive field of the convolution network by using causal convolution and expansion convolution; by applying a multi-head self attention mechanism, the time node and the characteristic information with the maximum correlation with the fault are found out from the time characteristic and the dimension characteristic respectively, and the accuracy of fault prediction is improved.
The technical scheme of the invention is as follows:
step (1): preprocessing industrial equipment data, removing zero values and abnormal values, filling missing values in a fitting mode, and cutting a data set in a time sequence mode to form an industrial equipment time sequence data set;
step (2): translating equipment fault state conditions in the industrial equipment time sequence data set according to the prediction requirement and the data acquisition interval;
and (3): building a model, taking the first 80% of a time sequence data set of industrial equipment as a training set, taking the last 20% of the time sequence data set as a test set, cutting, inputting the training set into a multi-head self-attention multi-dimensional time sequence convolution network for training for a certain number of times, testing the network by using the test set, completing model training, and storing the model;
and (4): when the equipment runs, accumulating the industrial equipment running data to form an industrial equipment running data matrix, and inputting the industrial equipment running data matrix into the model;
and (5): displaying the equipment fault prediction condition given by the model to a user, completing the equipment fault prediction at the current moment, and repeatedly performing the step (4) to continue the industrial equipment fault prediction at the next moment;
the invention has the beneficial effects that:
(1) mining the fault occurrence characteristics of the industrial equipment from the fault data of the industrial equipment in a centralized manner by using a time sequence convolution network, and fully utilizing the time sequence characteristics of the data under the condition of ensuring the sufficient receptive field of the convolution network by utilizing causal convolution and expansion convolution;
(2) and by applying a multi-head self attention mechanism, time nodes and characteristic information with the maximum fault correlation are found from time and dimensional characteristics respectively, so that the accuracy of fault prediction is improved.
Drawings
To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following sections will briefly introduce the drawings that are needed to be used in the description of the embodiments or the prior art. The drawings in the description represent only some embodiments of the invention, and other drawings may be derived from those drawings by a person skilled in the art without inventive effort.
FIG. 1 is a flow chart of a method for predicting a failure of an industrial device using a self-attention mechanism and a time-series convolutional network according to the present invention;
FIG. 2 is a block diagram of a self-attention based and timing based convolutional network used in the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The following describes a detailed flow of the method for predicting the fault of the industrial equipment based on the self-attention mechanism and the time-series convolutional network with reference to fig. 1 and fig. 2:
step (1): preprocessing industrial equipment data, removing zero values and abnormal values, filling missing values in a fitting mode, and cutting a data set in a time sequence mode to form an industrial equipment time sequence data set;
step (2): translating equipment fault state conditions in the industrial equipment time sequence data set according to the prediction requirement and the data acquisition interval;
and (3): building a model, taking the first 80% of a time sequence data set of industrial equipment as a training set, taking the last 20% of the time sequence data set as a test set, cutting, inputting the training set into a multi-head self-attention multi-dimensional time sequence convolution network for training for a certain number of times, testing the network by using the test set, completing model training, and storing the model;
and (4): when the equipment runs, accumulating the industrial equipment running data to form an industrial equipment running data matrix, and inputting the industrial equipment running data matrix into the model;
and (5): displaying the equipment fault prediction condition given by the model to a user, completing the equipment fault prediction at the current moment, and repeatedly performing the step (4) to continue the industrial equipment fault prediction at the next moment;
according to the method for predicting the fault of the industrial equipment based on the self-attention mechanism and the time sequence convolution network, the time sequence convolution network is used, the fault occurrence characteristics of the industrial equipment are centrally mined from the fault data of the industrial equipment, the causal convolution and the expansion convolution are used, the time sequence characteristics of the data are fully utilized under the condition that the sufficient receptive field of the convolution network is guaranteed, meanwhile, the same convolution kernel is shared among all convolution layers, and the consumption of computing resources during training and model operation is reduced; by applying a multi-head self attention mechanism, the time node and the characteristic information with the maximum correlation with the fault are found out from the time characteristic and the dimension characteristic respectively, and the accuracy of fault prediction is improved.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims (7)

1. A method for predicting industrial equipment faults based on a self-attention mechanism and a time sequence convolution network is characterized in that time sequence processing and label translation are carried out on data, then the data are input into a multi-head self-attention multi-dimensional time sequence convolution network for training, and an industrial equipment operation data matrix is input into a model for predicting industrial equipment states, and specifically comprises the following steps:
step (1): preprocessing industrial equipment data, removing zero values and abnormal values, filling missing values in a fitting mode, and cutting a data set in a time-sequencing mode to form an industrial equipment time-sequence data set;
step (2): performing time sequence translation on equipment fault state conditions in the industrial equipment time sequence data set according to the prediction requirement and the data acquisition interval;
and (3): building a model, taking the first 80% of data in the industrial equipment time sequence data set as a training set, taking the second 20% of data as a test set for cutting, inputting the training set into a multi-head self-attention multi-dimensional time sequence convolution network for training for a certain number of times, testing the network by using the test set, completing model training, and storing the model;
and (4): when the equipment runs, accumulating the industrial equipment running data to form an industrial equipment running data matrix, and inputting the industrial equipment running data matrix into the model;
and (5): displaying the equipment fault prediction condition given by the model to a user, completing the equipment fault prediction at the current moment, and repeatedly performing the step (4) to continue the industrial equipment fault prediction at the next moment;
2. the method for predicting the fault of the industrial equipment based on the self-attention mechanism and the time sequence convolutional network as claimed in claim 1, wherein when the fault of the industrial equipment is preprocessed, each time step is sequentially traversed in a time window mode, and according to the predicted time requirement, the state of the equipment at a certain time after the current time is taken as a label of the current time window for thermal coding to form a fault data set of the industrial equipment.
3. The method of claim 1, wherein the multi-headed self-attention multidimensional time series convolutional network comprises the following modules: the self-attention-driven multi-head model comprises a multi-head self-attention mechanism layer, a time sequence convolution residual block and a full connection layer, wherein the time sequence convolution residual block comprises causal convolution, expansion convolution and residual connection, the whole model is formed in a mode that an input layer is connected with a plurality of stacked time sequence convolution residual blocks, a tail end block of the plurality of stacked time sequence convolution residual blocks is connected with a multi-head attention layer and the full connection layer, and the full connection layer is connected with a full connection layer, wherein the number of neurons is the sum of all the fault numbers of equipment.
4. The method for predicting the fault of the industrial equipment based on the self-attention mechanism and the time sequence convolution network as claimed in claim 1 or 3, characterized in that a multi-head self-attention mechanism is used so that a model finds out the association relationship between different feature dimensions and a plurality of time steps before the current time and the association relationship between the equipment fault and a plurality of equipment features so as to improve the fault accuracy of the model diagnosis equipment, and meanwhile, the model uses a plurality of time sequence convolution residual blocks so that the model can process a plurality of dimension information of the industrial equipment at the same time.
5. The method of claim 1, wherein when the model is tested, the model predicts that the equipment state is an imminent failure, and the model is also a failure at a specified time after a current time window, the prediction is correct; when the model predicts that the equipment state is normal and the model is normal at the specified time after the current time window, the prediction is correct; the other prediction cases are prediction errors. Therefore, the accuracy of the model is calculated by dividing the prediction accuracy by the total number of predictions.
6. The method for predicting the fault of the industrial equipment based on the self-attention mechanism and the time sequence convolution network is characterized in that when a model is used for actual prediction, a data matrix is constructed in a mode of accumulating industrial equipment data, when the number of rows of the current industrial equipment data matrix is less than the minimum required number of rows predicted by the model, prediction is not performed, the current industrial equipment state is used as one row of the data matrix to be added to the last row of the industrial equipment data matrix, when the number of rows of the current industrial equipment data matrix is enough to be predicted, the current equipment state is used as one row of the data matrix to be added to the last row of the industrial equipment data matrix, the first row of the industrial equipment data matrix is removed, and then the data matrix is input into the model to perform prediction on the equipment state.
7. The method of claim 1 or 6, wherein the number of rows of the data window and the data matrix of the industrial equipment used in prediction for constructing the data set is determined by the acquisition frequency and the prediction time requirement, and the number is calculated as the product of the prediction time and the one-minute data acquisition frequency.
CN202110393245.5A 2021-04-13 2021-04-13 Industrial equipment fault prediction method based on self-attention mechanism and time sequence convolution network Pending CN113283631A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110393245.5A CN113283631A (en) 2021-04-13 2021-04-13 Industrial equipment fault prediction method based on self-attention mechanism and time sequence convolution network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110393245.5A CN113283631A (en) 2021-04-13 2021-04-13 Industrial equipment fault prediction method based on self-attention mechanism and time sequence convolution network

Publications (1)

Publication Number Publication Date
CN113283631A true CN113283631A (en) 2021-08-20

Family

ID=77276541

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110393245.5A Pending CN113283631A (en) 2021-04-13 2021-04-13 Industrial equipment fault prediction method based on self-attention mechanism and time sequence convolution network

Country Status (1)

Country Link
CN (1) CN113283631A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114444663A (en) * 2022-01-28 2022-05-06 吉林大学 Water supply pipe network leakage detection and positioning method based on time convolution network
CN114580472A (en) * 2022-02-28 2022-06-03 西北大学 Large-scale equipment fault prediction method with repeated cause and effect and attention in industrial internet
CN115309736A (en) * 2022-10-10 2022-11-08 北京航空航天大学 Time sequence data anomaly detection method based on self-supervision learning multi-head attention network

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114444663A (en) * 2022-01-28 2022-05-06 吉林大学 Water supply pipe network leakage detection and positioning method based on time convolution network
CN114580472A (en) * 2022-02-28 2022-06-03 西北大学 Large-scale equipment fault prediction method with repeated cause and effect and attention in industrial internet
CN114580472B (en) * 2022-02-28 2022-12-23 西北大学 Large-scale equipment fault prediction method with repeated cause and effect and attention in industrial internet
CN115309736A (en) * 2022-10-10 2022-11-08 北京航空航天大学 Time sequence data anomaly detection method based on self-supervision learning multi-head attention network

Similar Documents

Publication Publication Date Title
CN113283631A (en) Industrial equipment fault prediction method based on self-attention mechanism and time sequence convolution network
Wu et al. Data-driven remaining useful life prediction via multiple sensor signals and deep long short-term memory neural network
CN111460728B (en) Method and device for predicting residual life of industrial equipment, storage medium and equipment
CN111273623B (en) Fault diagnosis method based on Stacked LSTM
CN110766143A (en) Equipment fault intelligent diagnosis method based on artificial neural network
CN113516226A (en) Hybrid model multivariate time sequence anomaly detection method based on graph neural network
CN113887616A (en) Real-time abnormity detection system and method for EPG (electronic program guide) connection number
CN112434390B (en) PCA-LSTM bearing residual life prediction method based on multi-layer grid search
CN110334948B (en) Power equipment partial discharge severity evaluation method and system based on characteristic quantity prediction
CN115017826B (en) Method for predicting residual service life of equipment
CN116821783A (en) Fault detection method, system, equipment and medium
CN115269314A (en) Transaction abnormity detection method based on log
US20230350402A1 (en) Multi-task learning based rul predication method under sensor fault condition
Jin et al. Data-driven resiliency solutions for boards and systems
CN115859090A (en) Turnout fault diagnosis method and system based on 1DCNN-LSTM
CN116562120A (en) RVE-based turbine engine system health condition assessment method and RVE-based turbine engine system health condition assessment device
CN115600116A (en) Dynamic detection method, system, storage medium and terminal for time series abnormity
CN112416920B (en) MES-oriented data cleaning method and system
CN113191556A (en) Nuclear power Loca event fault prediction and diagnosis method
CN117743939A (en) Wind power equipment residual service life prediction method and system and electronic equipment
Xiaolong et al. Research on Fault Diagnosis Method of Rolling Bearing Based on Improved Convolutional Neural Network
CN116090897A (en) Method for predicting printing quality of solder paste printer
Jia et al. A Dual-Stage Attention-Based LSTM Neural Network for Tool Remaining Useful Life Prediction
Sun et al. Remaining Useful Life Prediction Based on CNN-BGRU-SA
CN117744495A (en) Method for predicting service life of extra-large bearing driven by multiple models in different degradation stages

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication