CN111721535A - Bearing fault detection method based on convolution multi-head self-attention mechanism - Google Patents

Bearing fault detection method based on convolution multi-head self-attention mechanism Download PDF

Info

Publication number
CN111721535A
CN111721535A CN202010583027.3A CN202010583027A CN111721535A CN 111721535 A CN111721535 A CN 111721535A CN 202010583027 A CN202010583027 A CN 202010583027A CN 111721535 A CN111721535 A CN 111721535A
Authority
CN
China
Prior art keywords
bearing
convolution
attention
head self
attention mechanism
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010583027.3A
Other languages
Chinese (zh)
Other versions
CN111721535B (en
Inventor
王卫杰
叶瑞达
任元
何亮
樊亚洪
张克明
傅百恒
耿梦梦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peoples Liberation Army Strategic Support Force Aerospace Engineering University
Original Assignee
Peoples Liberation Army Strategic Support Force Aerospace Engineering University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peoples Liberation Army Strategic Support Force Aerospace Engineering University filed Critical Peoples Liberation Army Strategic Support Force Aerospace Engineering University
Priority to CN202010583027.3A priority Critical patent/CN111721535B/en
Publication of CN111721535A publication Critical patent/CN111721535A/en
Application granted granted Critical
Publication of CN111721535B publication Critical patent/CN111721535B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
    • G01M13/00—Testing of machine parts
    • G01M13/04—Bearings
    • G01M13/045—Acoustic or vibration analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/12—Classification; Matching

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Acoustics & Sound (AREA)
  • Testing Of Devices, Machine Parts, Or Other Structures Thereof (AREA)

Abstract

A bearing fault detection method based on a convolution multi-head self-attention mechanism is disclosed. The detection method comprises the following steps: and acquiring and preprocessing a vibration signal of the fault bearing to generate a bearing fault data set, constructing a convolution multi-head self-attention mechanism network and training to obtain a bearing fault detection result. The convolutional multi-head self-attention mechanism network comprises: the system comprises a convolutional layer, a position encoder, a multi-head self-attention module, a global average pooling layer and a full-connection layer; extracting initial characteristics of a bearing signal by the convolution layer; the position encoder carries out position encoding on the initial characteristics of the bearing; the multi-head self-attention module learns the initial features; the global average pooling layer regularizes the network to prevent overfitting; different failure types of the fully-connected layer output bearing. The invention provides an efficient and accurate method for detecting the bearing fault, thereby effectively maintaining the normal operation of mechanical equipment.

Description

Bearing fault detection method based on convolution multi-head self-attention mechanism
Technical Field
The invention relates to the field of equipment health management, in particular to a bearing fault diagnosis method.
Background
The bearing is used as the core of the rotating mechanical component and is concerned with the normal operation of the whole mechanical equipment. During the operation of the equipment, the stator, the rotor and other parts of the bearing are easily damaged due to overload, friction, corrosion, gluing and the like. These failures can cause the entire mechanical equipment to fail, affect the performance of the production equipment, and can cause personnel injury. In order to maintain high performance of the machine while avoiding casualties and economic losses due to bearing failure, the best solution is to perform fault detection and health monitoring of the bearings.
With the rapid development of sensor technology, computer technology and information processing technology, the device health management method based on data driving and deep learning is becoming a new research hotspot and development trend. The method comprehensively utilizes sensor data and a machine learning theory, learns the failure characteristics of the machine by establishing deep learning model training data, can effectively solve the problem that diagnosis experts are rare relative to mechanical equipment, and can effectively monitor massive data and quickly predict the occurrence of accidents. The bearing fault can be effectively detected by utilizing the deep learning model.
The patent publication of invention with application number 201910728620.X discloses a rolling bearing obstacle diagnosis based on a self-attention neural network, which learns a vibration signal through a self-attention mechanism; the method attempts to learn bearing fault signatures using a single self-attentive approach, failing to learn rich bearing fault signatures. Meanwhile, a self-attention mechanism is directly used, and local characteristics of the bearing cannot be effectively learned. The method provided by the invention effectively solves the problems, and simultaneously uses the position encoder to provide position information when the multi-head self-attention module learns the bearing characteristics, thereby effectively solving the position information problem in the global characteristic learning.
Disclosure of Invention
Objects of the invention
The invention aims to provide a bearing fault detection method based on convolution multi-head attention. The bearing fault detection method is high in bearing fault detection precision and suitable for practical engineering projects.
(II) technical scheme
The technical scheme of the invention is that a bearing fault detection method based on convolution multi-head attention comprises the following steps: the method comprises the following steps of collecting vibration signals of a fault bearing, preprocessing the vibration signals to generate a bearing fault data set, constructing a convolution multi-head self-attention mechanism network, and further training to obtain a bearing fault detection result.
The method comprises the steps of collecting fault signals of the bearing, collecting different types of fault bearing vibration signal parameter information through a sensor, and recording a bearing fault label.
The bearing signal preprocessing operation is to carry out standard normalization processing on the bearing signal and then cut the bearing signal in equal length, wherein the standard normalization function is as follows:
Figure BDA0002553145630000021
where x represents the sample signal, μ represents the sample signal mean, and σ represents the sample signal standard deviation.
And generating a bearing fault data set, carrying out equal-quantity random selection on different types of fault bearing signals, and randomly dividing the fault bearing signals into a training set, a verification set and a test set according to the ratio of 7:2: 1.
Constructing a convolution multi-head self-attention mechanism network, wherein the structure sequentially comprises the following steps: convolutional layer → position encoder → multi-headed self-attention module → global averaging pooling layer → fully-connected layer output layer. The convolution layer is a one-dimensional convolution neural network, the number of convolution kernels is set to be 32, the size of the convolution kernels is set to be 8 multiplied by 8, and the step length is set to be 8; the position encoder uses positionencoding:
Figure BDA0002553145630000031
PE is a two-dimensional matrix, sin variables are added at even positions, cos variables are added at odd positions, the whole PE matrix is filled, the initial characteristics of the bearing signals are extracted by using a convolutional neural network, position coding is completed, and when multi-head self-attention is used, the characteristic position coding is facilitated to learn associated characteristics.
The multi-head self-attention mechanism allows the model to jointly pay attention to information from different representation subspaces at different positions, and the self-attention mechanism is independently used, so that abundant characteristic information cannot be obtained, and the bearing characteristic learning process comprises the following steps:
Figure BDA0002553145630000032
wherein a is a bearing feature matrix, WqA weight matrix consisting of query vectors q (query), WkA weight matrix consisting of key vectors k (key), WvA weight matrix consisting of a vector of values v (value); using the scaled dot product as the attention mechanism:
Figure BDA0002553145630000033
dkis the square root of the key vector dimension, in the self-attention mechanism the output of self-attention is a weighted sum of value vectors v, the weight assigned to each value vector is calculated by the degree of correlation of the query vector q and the current key vector k, the multi-headed self-attention mechanism:
Figure BDA0002553145630000034
wherein Wi Q，Wi K，Wi VAnd WoAre learnable parameters.
The global average pooling layer can reduce the network parameter quantity and prevent the overfitting phenomenon.
The number of neurons in the fully connected layer equals the total number of bearing fault classes.
Training a convolution multi-head self-attention mechanism network: inputting the bearing training set and the verification set into a convolution multi-head self-attention machine network, setting a learning rate learning _ rate of the network to be 0.0055 by using a cross entropy loss function, training the network by using a gradient descent method, and updating the weight values and the learning rates of the training set and the verification set until the network loss function is converged to obtain the trained convolution multi-head self-attention machine network; classifying fault bearing signals: and inputting the bearing test set into the trained convolution multi-head self-attention mechanism network to obtain a bearing fault detection result, and calculating the precision of the convolution multi-head self-attention mechanism network model for bearing fault detection by comparing the bearing fault detection result with a correct label.
The invention realizes the intelligent detection of automatic extraction of various fault characteristics of the bearing. The invention identifies the accurate identification of various fault characteristics of the bearing.
(III) advantageous effects
The technical scheme of the invention has the following beneficial technical effects: the method is used for identifying vibration signals of various fault bearings, the experimental result is shown in FIG. 4, the identification result reaches 98.8%, and the result proves that the bearing fault detection method based on the convolution multi-head self-attention mechanism is effective.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a graph of loss function during training according to an embodiment of the present invention;
FIG. 3 is a graph of an accuracy function during training according to an embodiment of the present invention;
FIG. 4 is a recognition result confusion diagram according to an embodiment of the present invention;
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings in conjunction with the following detailed description. It should be understood that the description is intended to be exemplary only, and is not intended to limit the scope of the present invention. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present invention.
Embodiment 1 of the present invention, a bearing fault detection method based on convolution multi-head attention, which is shown in fig. 1, is performed according to the following steps:
the method comprises the steps of collecting fault signals of the bearing, collecting 10 different types of fault bearing vibration signal parameter information through a sensor, and recording a bearing fault label.
The bearing signal preprocessing operation is to carry out standard normalization processing on the bearing signal, then cut the bearing signal according to 2048 sampling points in equal length, and the standard normalization function is as follows:
Figure BDA0002553145630000051
where x represents the sample signal, μ represents the sample signal mean, and σ represents the sample signal standard deviation.
And (3) generating a bearing fault data set, randomly selecting 1000 parts of 10 different types of fault bearing signals, and randomly dividing the signals into a training set, a verification set and a test set according to the ratio of 7:2: 1.
Constructing a convolution multi-head self-attention mechanism network, wherein the structure sequentially comprises the following steps: convolutional layer → position encoder → multi-headed self-attention module → global averaging pooling layer → fully-connected layer output layer. The convolution layer is a one-dimensional convolution neural network, the number of convolution kernels is set to be 32, the size of the convolution kernels is set to be 8 multiplied by 8, and the step length is set to be 8; the position encoder uses positionencoding:
Figure BDA0002553145630000052
PE is a two-dimensional matrix, sin variables are added at even positions, cos variables are added at odd positions, the whole PE matrix is filled, the initial characteristics of the bearing signals are extracted by using a convolutional neural network, position coding is completed, and when multi-head self-attention is used, the characteristic position coding is facilitated to learn associated characteristics.
The multi-head self-attention mechanism allows the model to jointly pay attention to information from different representation subspaces at different positions, and the self-attention mechanism is independently used, so that abundant characteristic information cannot be obtained, and the bearing characteristic learning process comprises the following steps:
Figure BDA0002553145630000061
wherein a is a bearing feature matrix, WqA weight matrix consisting of query vectors q (query), WkA weight matrix consisting of key vectors k (key), WvA weight matrix consisting of a vector of values v (value); using the scaled dot product as the attention mechanism:
Figure BDA0002553145630000062
dkis the square root of the key vector dimension, in the self-attention mechanism the output of self-attention is a weighted sum of value vectors v, the weight assigned to each value vector is calculated by the degree of correlation of the query vector q and the current key vector k, the multi-headed self-attention mechanism:
Figure BDA0002553145630000063
wherein Wi Q，Wi K，Wi VAnd WoAre learnable parameters.
The global average pooling layer can reduce the network parameter quantity and prevent the overfitting phenomenon.
The full-connection layer outputs different types of faults respectively in a model training stage, the number of the neurons in the full-connection layer is equal to the number of the bearing fault types, namely the number of the neurons is set to be 10.
Training a convolution multi-head self-attention mechanism network: inputting the bearing training set and the verification set into a convolution multi-head self-attention mechanism network, setting a learning rate learning _ rate of the network to be 0.0055 by using a cross entropy loss function, training the network by using a gradient descent method, and updating the weight values and the learning rates of the training set and the verification set until the network loss function is converged to obtain the trained convolution multi-head self-attention mechanism network. The loss functions of the training set and the validation set during the training process are shown in fig. 2. The accuracy of the training set and the validation set during the training process is shown in fig. 3. Inputting the bearing test set into the trained convolution multi-head self-attention mechanism network to obtain a bearing fault detection result, comparing the bearing fault detection result with a correct label, calculating the bearing fault detection accuracy of the convolution multi-head self-attention mechanism network model, wherein the bearing fault detection accuracy reaches 98.8%, and the identification result confusion graph is shown in fig. 4.
Those skilled in the art will appreciate that the invention may be practiced without these specific details.

Claims (1)

1. A bearing fault detection method based on a convolution multi-head self-attention mechanism is characterized by comprising the following steps: collecting a vibration signal of a fault bearing and preprocessing the vibration signal to generate a bearing fault data set, constructing a convolution multi-head self-attention mechanism network, and further training to obtain a bearing fault detection result, wherein the method comprises the following steps;
(1) collecting and preprocessing a fault bearing vibration signal:
acquiring different kinds of fault bearing vibration signal parameter information through a sensor, and recording a bearing fault label; carrying out standard normalization processing on the bearing signals, and then cutting the bearing signals in equal length, wherein the standard normalization function is as follows:
Figure FDA0002553145620000011
wherein x represents a sample signal, μ represents a sample signal average value, and σ represents a sample signal standard deviation;
(2) generating a bearing fault data set:
carrying out equal-quantity random selection on different types of fault bearing signals, and randomly dividing the fault bearing signals into a training set, a verification set and a test set according to the ratio of 7:2: 1;
(3) constructing a convolution multi-head self-attention mechanism network, wherein the structure sequentially comprises the following steps: convolutional layer → position encoder → multi-headed self-attention module → global average pooling layer → fully-connected layer output layer;
(3a) the convolution layer is a one-dimensional convolution neural network, the number of convolution kernels is set to be 32, the size of the convolution kernels is set to be 8 multiplied by 8, and the step length is set to be 8;
(3b) the position encoder uses positional encoding:
Figure FDA0002553145620000021
PE is a two-dimensional matrix, sin variables are added at even positions, cos variables are added at odd positions, the whole PE matrix is filled up, the initial characteristics of the bearing signals are extracted by using a convolutional neural network, position coding is completed, and when multi-head self-attention is used, the characteristic position coding is facilitated to learn associated characteristics;
(3c) the multi-head self-attention mechanism allows the model to jointly pay attention to information from different representation subspaces at different positions, and the self-attention mechanism is independently used, so that abundant characteristic information cannot be obtained, and the bearing characteristic learning process comprises the following steps:
Figure FDA0002553145620000022
wherein a is a bearing feature matrix, WqA weight matrix consisting of query vectors q (query), WkA weight matrix consisting of key vectors k (key), WvA weight matrix consisting of a vector of values v (value);
self-attention mechanism using scaled dot products:
Figure FDA0002553145620000023
dkis the square root of the key vector dimension, the output from attention is the weighted sum of the value vectors V, the weight assigned to each value vector is calculated by the degree of correlation of the query vector Q and the current key vector K, Q, K, V is the multi-headed self-attention mechanism:
Figure FDA0002553145620000024
the input of the multi-head self-attention mechanism is changed from Q, K and V into QWi Q,KWi K,VWi VSelecting an 8-head self-attention mechanism, changing the dimensionality of Q, K and V from the original 8n dimensionality into n dimensionality in dimensionality, calculating one head each time, then splicing the 8 times of scaling dot product self-attention results, and performing W-time scaling on the resultsoAnd carrying out linear transformation to obtain a final multi-head self-attention value.
(3d) The global average pooling layer can reduce the number of network parameters and prevent an overfitting phenomenon;
(3e) the number of neurons in the fully connected layer is equal to the total number of bearing fault categories;
(4) training a convolution multi-head self-attention mechanism network:
inputting the bearing training set and the verification set into a convolution multi-head self-attention machine network, setting a learning rate learning _ rate of the network to be 0.0055 by using a cross entropy loss function, training the network by using a gradient descent method, and updating the weight values and the learning rates of the training set and the verification set until the network loss function is converged to obtain the trained convolution multi-head self-attention machine network;
(5) classifying fault bearing signals:
and inputting the bearing test set into the trained convolution multi-head self-attention mechanism network to obtain a bearing fault detection result, and calculating the precision of the convolution multi-head self-attention mechanism network model for bearing fault detection by comparing the bearing fault detection result with a correct label.
CN202010583027.3A 2020-06-23 2020-06-23 Bearing fault detection method based on convolution multi-head self-attention mechanism Active CN111721535B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010583027.3A CN111721535B (en) 2020-06-23 2020-06-23 Bearing fault detection method based on convolution multi-head self-attention mechanism

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010583027.3A CN111721535B (en) 2020-06-23 2020-06-23 Bearing fault detection method based on convolution multi-head self-attention mechanism

Publications (2)

Publication Number Publication Date
CN111721535A true CN111721535A (en) 2020-09-29
CN111721535B CN111721535B (en) 2021-11-30

Family

ID=72568428

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010583027.3A Active CN111721535B (en) 2020-06-23 2020-06-23 Bearing fault detection method based on convolution multi-head self-attention mechanism

Country Status (1)

Country Link
CN (1) CN111721535B (en)

Cited By (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112183453A (en) * 2020-10-15 2021-01-05 哈尔滨市科佳通用机电股份有限公司 Deep learning-based water injection port cover plate unlocking-in-place fault detection method and system
CN112528548A (en) * 2020-11-27 2021-03-19 东莞市汇林包装有限公司 Self-adaptive depth coupling convolution self-coding multi-mode data fusion method
CN112629854A (en) * 2020-11-25 2021-04-09 西安交通大学 Bearing fault classification method based on neural network attention mechanism
CN112801069A (en) * 2021-04-14 2021-05-14 四川翼飞视科技有限公司 Face key feature point detection device, method and storage medium
CN112819037A (en) * 2021-01-12 2021-05-18 广东石油化工学院 Fault diagnosis method based on cross attention and self attention classification parameter distribution
CN112906739A (en) * 2021-01-18 2021-06-04 河南工业大学 Fault diagnosis method based on multi-head attention and shafting equipment periodicity
CN113158445A (en) * 2021-04-06 2021-07-23 中国人民解放军战略支援部队航天工程大学 Prediction algorithm for residual service life of aero-engine with convolution memory residual self-attention mechanism
CN113205177A (en) * 2021-04-25 2021-08-03 广西大学 Electric power terminal identification method based on incremental collaborative attention mobile convolution
CN113255780A (en) * 2021-05-28 2021-08-13 润联软件系统(深圳)有限公司 Reduction gearbox fault prediction method and device, computer equipment and storage medium
CN113343591A (en) * 2021-07-16 2021-09-03 浙江大学 Product key part life end-to-end prediction method based on self-attention network
CN113344070A (en) * 2021-06-01 2021-09-03 南京林业大学 Remote sensing image classification system and method based on multi-head self-attention module
CN113392214A (en) * 2021-06-03 2021-09-14 齐鲁工业大学 K selection strategy-based sparse self-attention text classification method and system
CN113469263A (en) * 2021-07-13 2021-10-01 润联软件系统(深圳)有限公司 Prediction model training method and device suitable for small samples and related equipment
CN113674225A (en) * 2021-07-30 2021-11-19 南京信息工程大学 Power equipment fault detection method based on convolutional neural network
CN113865868A (en) * 2021-08-24 2021-12-31 东南大学 Rolling bearing fault diagnosis method based on time-frequency domain expression
CN114332825A (en) * 2022-03-10 2022-04-12 中汽信息科技(天津)有限公司 Road terrain distribution identification method and device based on deep learning and storage medium
WO2022077901A1 (en) * 2020-10-13 2022-04-21 同济大学 Bearing failure mode diagnosis method using small sample data sets, and system
WO2022083335A1 (en) * 2020-10-20 2022-04-28 神思电子技术股份有限公司 Self-attention mechanism-based behavior recognition method
CN114638256A (en) * 2022-02-22 2022-06-17 合肥华威自动化有限公司 Transformer fault detection method and system based on sound wave signals and attention network
CN114913396A (en) * 2022-07-15 2022-08-16 西北工业大学 Motor bearing fault diagnosis method
CN114937021A (en) * 2022-05-31 2022-08-23 哈尔滨工业大学 Swin-Transformer-based crop disease fine-granularity classification method
CN114993677A (en) * 2022-05-11 2022-09-02 山东大学 Rolling bearing fault diagnosis method and system based on unbalanced small sample data
CN115659283A (en) * 2022-12-12 2023-01-31 陕西金元新能源有限公司 Wind power equipment damage prediction method based on attention mechanism of multi-task learning
CN116243683A (en) * 2023-03-15 2023-06-09 青岛澎湃海洋探索技术有限公司 Method for diagnosing faults of propulsion system based on torque and multi-head self-encoder
CN116428129A (en) * 2023-06-13 2023-07-14 山东大学 Fan blade impact positioning method and system based on attention mixing neural network
CN117153144A (en) * 2023-10-31 2023-12-01 杭州宇谷科技股份有限公司 Battery information voice broadcasting method and device based on terminal calculation

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107421741A (en) * 2017-08-25 2017-12-01 南京信息工程大学 A kind of Fault Diagnosis of Roller Bearings based on convolutional neural networks
CN108681539A (en) * 2018-05-07 2018-10-19 内蒙古工业大学 A kind of illiteracy Chinese nerve interpretation method based on convolutional neural networks
CN108734290A (en) * 2018-05-16 2018-11-02 湖北工业大学 It is a kind of based on the convolutional neural networks construction method of attention mechanism and application
CN109165667A (en) * 2018-07-06 2019-01-08 中国科学院自动化研究所 Based on the cerebral disease categorizing system from attention mechanism
CN109726524A (en) * 2019-03-01 2019-05-07 哈尔滨理工大学 A kind of rolling bearing remaining life prediction technique based on CNN and LSTM
CN109902399A (en) * 2019-03-01 2019-06-18 哈尔滨理工大学 Rolling bearing fault recognition methods under a kind of variable working condition based on ATT-CNN
CN109919205A (en) * 2019-02-25 2019-06-21 华南理工大学 Based on bull from the convolution echo state network timing classification method of attention mechanism
CN110196946A (en) * 2019-05-29 2019-09-03 华南理工大学 A kind of personalized recommendation method based on deep learning
CN110355608A (en) * 2019-07-18 2019-10-22 浙江大学 Based on the tool abrasion prediction technique from attention mechanism and deep learning
CN110608884A (en) * 2019-08-08 2019-12-24 桂林电子科技大学 Rolling bearing state diagnosis method based on self-attention neural network
CN110619045A (en) * 2019-08-27 2019-12-27 四川大学 Text classification model based on convolutional neural network and self-attention
CN110672343A (en) * 2019-09-29 2020-01-10 电子科技大学 Rotary machine fault diagnosis method based on multi-attention convolutional neural network
WO2020068831A1 (en) * 2018-09-26 2020-04-02 Visa International Service Association Dynamic graph representation learning via attention networks
CN111046907A (en) * 2019-11-02 2020-04-21 国网天津市电力公司 Semi-supervised convolutional network embedding method based on multi-head attention mechanism

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107421741A (en) * 2017-08-25 2017-12-01 南京信息工程大学 A kind of Fault Diagnosis of Roller Bearings based on convolutional neural networks
CN108681539A (en) * 2018-05-07 2018-10-19 内蒙古工业大学 A kind of illiteracy Chinese nerve interpretation method based on convolutional neural networks
CN108734290A (en) * 2018-05-16 2018-11-02 湖北工业大学 It is a kind of based on the convolutional neural networks construction method of attention mechanism and application
CN109165667A (en) * 2018-07-06 2019-01-08 中国科学院自动化研究所 Based on the cerebral disease categorizing system from attention mechanism
WO2020068831A1 (en) * 2018-09-26 2020-04-02 Visa International Service Association Dynamic graph representation learning via attention networks
CN109919205A (en) * 2019-02-25 2019-06-21 华南理工大学 Based on bull from the convolution echo state network timing classification method of attention mechanism
CN109902399A (en) * 2019-03-01 2019-06-18 哈尔滨理工大学 Rolling bearing fault recognition methods under a kind of variable working condition based on ATT-CNN
CN109726524A (en) * 2019-03-01 2019-05-07 哈尔滨理工大学 A kind of rolling bearing remaining life prediction technique based on CNN and LSTM
CN110196946A (en) * 2019-05-29 2019-09-03 华南理工大学 A kind of personalized recommendation method based on deep learning
CN110355608A (en) * 2019-07-18 2019-10-22 浙江大学 Based on the tool abrasion prediction technique from attention mechanism and deep learning
CN110608884A (en) * 2019-08-08 2019-12-24 桂林电子科技大学 Rolling bearing state diagnosis method based on self-attention neural network
CN110619045A (en) * 2019-08-27 2019-12-27 四川大学 Text classification model based on convolutional neural network and self-attention
CN110672343A (en) * 2019-09-29 2020-01-10 电子科技大学 Rotary machine fault diagnosis method based on multi-attention convolutional neural network
CN111046907A (en) * 2019-11-02 2020-04-21 国网天津市电力公司 Semi-supervised convolutional network embedding method based on multi-head attention mechanism

Non-Patent Citations (8)

* Cited by examiner, † Cited by third party
Title
I. BELLO等: "Attention Augmented Convolutional Networks", 《2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV)》 *
J. LI等: "Global-Local Temporal Representations for Video Person Re-Identification", 《2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV)》 *
R. LI等: "Dilated Residual Network with Multi-head Self-attention for Speech Emotion Recognition", 《ICASSP 2019 - 2019 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS》 *
S. CHEN等: "Self-Attention based Network For Medical Query Expansion", 《2019 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN)》 *
Y. LIU等: "Exploring a Unified Attention-Based Pooling Framework for Speaker Verification", 《018 11TH INTERNATIONAL SYMPOSIUM ON CHINESE SPOKEN LANGUAGE PROCESSING (ISCSLP)》 *
孟恒宇: "基于Transformer重建的时序数据异常检测与关系提取", 《计算机工程》 *
熊漩等: "融合多头自注意力机制的中文分类方法", 《电子测量技术》 *
王翔等: "基于多注意力机制的深度神经网络故障诊断算法", 《浙江理工大学学报(自然科学版)》 *

Cited By (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022077901A1 (en) * 2020-10-13 2022-04-21 同济大学 Bearing failure mode diagnosis method using small sample data sets, and system
CN112183453A (en) * 2020-10-15 2021-01-05 哈尔滨市科佳通用机电股份有限公司 Deep learning-based water injection port cover plate unlocking-in-place fault detection method and system
WO2022083335A1 (en) * 2020-10-20 2022-04-28 神思电子技术股份有限公司 Self-attention mechanism-based behavior recognition method
CN112629854A (en) * 2020-11-25 2021-04-09 西安交通大学 Bearing fault classification method based on neural network attention mechanism
CN112528548A (en) * 2020-11-27 2021-03-19 东莞市汇林包装有限公司 Self-adaptive depth coupling convolution self-coding multi-mode data fusion method
CN112819037B (en) * 2021-01-12 2024-01-30 广东石油化工学院 Fault diagnosis method based on classification parameter distribution of cross attention and self attention
CN112819037A (en) * 2021-01-12 2021-05-18 广东石油化工学院 Fault diagnosis method based on cross attention and self attention classification parameter distribution
CN112906739A (en) * 2021-01-18 2021-06-04 河南工业大学 Fault diagnosis method based on multi-head attention and shafting equipment periodicity
CN113158445A (en) * 2021-04-06 2021-07-23 中国人民解放军战略支援部队航天工程大学 Prediction algorithm for residual service life of aero-engine with convolution memory residual self-attention mechanism
CN112801069A (en) * 2021-04-14 2021-05-14 四川翼飞视科技有限公司 Face key feature point detection device, method and storage medium
CN113205177A (en) * 2021-04-25 2021-08-03 广西大学 Electric power terminal identification method based on incremental collaborative attention mobile convolution
CN113205177B (en) * 2021-04-25 2022-03-25 广西大学 Electric power terminal identification method based on incremental collaborative attention mobile convolution
CN113255780A (en) * 2021-05-28 2021-08-13 润联软件系统(深圳)有限公司 Reduction gearbox fault prediction method and device, computer equipment and storage medium
CN113255780B (en) * 2021-05-28 2024-05-03 润联智能科技股份有限公司 Reduction gearbox fault prediction method and device, computer equipment and storage medium
CN113344070A (en) * 2021-06-01 2021-09-03 南京林业大学 Remote sensing image classification system and method based on multi-head self-attention module
CN113392214A (en) * 2021-06-03 2021-09-14 齐鲁工业大学 K selection strategy-based sparse self-attention text classification method and system
CN113392214B (en) * 2021-06-03 2022-09-06 齐鲁工业大学 K selection strategy-based sparse self-attention text classification method and system
CN113469263A (en) * 2021-07-13 2021-10-01 润联软件系统(深圳)有限公司 Prediction model training method and device suitable for small samples and related equipment
CN113343591A (en) * 2021-07-16 2021-09-03 浙江大学 Product key part life end-to-end prediction method based on self-attention network
CN113343591B (en) * 2021-07-16 2022-05-03 浙江大学 Product key part life end-to-end prediction method based on self-attention network
CN113674225A (en) * 2021-07-30 2021-11-19 南京信息工程大学 Power equipment fault detection method based on convolutional neural network
CN113865868B (en) * 2021-08-24 2023-12-22 东南大学 Rolling bearing fault diagnosis method based on time-frequency domain expression
CN113865868A (en) * 2021-08-24 2021-12-31 东南大学 Rolling bearing fault diagnosis method based on time-frequency domain expression
CN114638256A (en) * 2022-02-22 2022-06-17 合肥华威自动化有限公司 Transformer fault detection method and system based on sound wave signals and attention network
CN114638256B (en) * 2022-02-22 2024-05-31 合肥华威自动化有限公司 Transformer fault detection method and system based on acoustic wave signals and attention network
CN114332825A (en) * 2022-03-10 2022-04-12 中汽信息科技(天津)有限公司 Road terrain distribution identification method and device based on deep learning and storage medium
CN114993677A (en) * 2022-05-11 2022-09-02 山东大学 Rolling bearing fault diagnosis method and system based on unbalanced small sample data
CN114937021A (en) * 2022-05-31 2022-08-23 哈尔滨工业大学 Swin-Transformer-based crop disease fine-granularity classification method
CN114913396A (en) * 2022-07-15 2022-08-16 西北工业大学 Motor bearing fault diagnosis method
CN115659283A (en) * 2022-12-12 2023-01-31 陕西金元新能源有限公司 Wind power equipment damage prediction method based on attention mechanism of multi-task learning
CN116243683A (en) * 2023-03-15 2023-06-09 青岛澎湃海洋探索技术有限公司 Method for diagnosing faults of propulsion system based on torque and multi-head self-encoder
CN116243683B (en) * 2023-03-15 2024-02-13 青岛澎湃海洋探索技术有限公司 Method for diagnosing faults of propulsion system based on torque and multi-head self-encoder
CN116428129A (en) * 2023-06-13 2023-07-14 山东大学 Fan blade impact positioning method and system based on attention mixing neural network
CN116428129B (en) * 2023-06-13 2023-09-01 山东大学 Fan blade impact positioning method and system based on attention mixing neural network
CN117153144A (en) * 2023-10-31 2023-12-01 杭州宇谷科技股份有限公司 Battery information voice broadcasting method and device based on terminal calculation
CN117153144B (en) * 2023-10-31 2024-02-06 杭州宇谷科技股份有限公司 Battery information voice broadcasting method and device based on terminal calculation

Also Published As

Publication number Publication date
CN111721535B (en) 2021-11-30

Similar Documents

Publication Publication Date Title
CN111721535B (en) Bearing fault detection method based on convolution multi-head self-attention mechanism
Li et al. A deep learning approach for anomaly detection based on SAE and LSTM in mechanical equipment
Sun et al. Convolutional discriminative feature learning for induction motor fault diagnosis
CN111914873A (en) Two-stage cloud server unsupervised anomaly prediction method
Ko et al. Fault classification in high-dimensional complex processes using semi-supervised deep convolutional generative models
CN111562108A (en) Rolling bearing intelligent fault diagnosis method based on CNN and FCMC
CN113865868B (en) Rolling bearing fault diagnosis method based on time-frequency domain expression
CN110991471B (en) Fault diagnosis method for high-speed train traction system
CN112305388B (en) On-line monitoring and diagnosing method for insulation partial discharge faults of generator stator winding
CN116593157A (en) Complex working condition gear fault diagnosis method based on matching element learning under small sample
CN115859077A (en) Multi-feature fusion motor small sample fault diagnosis method under variable working conditions
CN110263767A (en) In conjunction with the intelligent Rotating Shaft Fault method of compressed data acquisition and deep learning
Tao et al. Fault diagnosis of rolling bearing using deep belief networks
CN116150901A (en) Rolling bearing residual life prediction method based on attention-enhanced time-frequency converter
CN116735170A (en) Intelligent fault diagnosis method based on self-attention multi-scale feature extraction
CN111428788A (en) Deep learning-based multi-fault diagnosis method and system for steam turbine generator set rotor
CN111539381B (en) Construction method of wind turbine bearing fault classification diagnosis model
Techane et al. Rotating machinery prognostics and application of machine learning algorithms: Use of deep learning with similarity index measure for health status prediction
CN114926702B (en) Small sample image classification method based on depth attention measurement
CN116167008A (en) Abnormal positioning method for internet of things sensing cloud data center based on data enhancement
Ahmed et al. Three-stage method for rotating machine health condition monitoring using vibration signals
CN114417704A (en) Wind turbine generator health assessment method based on improved stack type self-coding
Li et al. Health indicator by CAE feature extractor
CN110750876A (en) Bearing data model training and using method
Ahsan et al. Advanced Fault Diagnosis in Rotating Machines Using 2D Grayscale Images with Improved Deep Convolutional Neural Networks

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant