CN111721535B - Bearing fault detection method based on convolution multi-head self-attention mechanism - Google Patents

Bearing fault detection method based on convolution multi-head self-attention mechanism Download PDF

Info

Publication number
CN111721535B
CN111721535B CN202010583027.3A CN202010583027A CN111721535B CN 111721535 B CN111721535 B CN 111721535B CN 202010583027 A CN202010583027 A CN 202010583027A CN 111721535 B CN111721535 B CN 111721535B
Authority
CN
China
Prior art keywords
bearing
head self
convolution
attention
attention mechanism
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010583027.3A
Other languages
Chinese (zh)
Other versions
CN111721535A (en
Inventor
王卫杰
叶瑞达
任元
何亮
樊亚洪
张克明
傅百恒
耿梦梦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peoples Liberation Army Strategic Support Force Aerospace Engineering University
Original Assignee
Peoples Liberation Army Strategic Support Force Aerospace Engineering University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peoples Liberation Army Strategic Support Force Aerospace Engineering University filed Critical Peoples Liberation Army Strategic Support Force Aerospace Engineering University
Priority to CN202010583027.3A priority Critical patent/CN111721535B/en
Publication of CN111721535A publication Critical patent/CN111721535A/en
Application granted granted Critical
Publication of CN111721535B publication Critical patent/CN111721535B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01MTESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
    • G01M13/00Testing of machine parts
    • G01M13/04Bearings
    • G01M13/045Acoustic or vibration analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/12Classification; Matching

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Acoustics & Sound (AREA)
  • Testing Of Devices, Machine Parts, Or Other Structures Thereof (AREA)

Abstract

A bearing fault detection method based on a convolution multi-head self-attention mechanism is disclosed. The detection method comprises the following steps: and acquiring and preprocessing a vibration signal of the fault bearing to generate a bearing fault data set, constructing a convolution multi-head self-attention mechanism network and training to obtain a bearing fault detection result. The convolutional multi-head self-attention mechanism network comprises: the system comprises a convolutional layer, a position encoder, a multi-head self-attention module, a global average pooling layer and a full-connection layer; extracting initial characteristics of a bearing signal by the convolution layer; the position encoder carries out position encoding on the initial characteristics of the bearing; the multi-head self-attention module learns the initial features; the global average pooling layer regularizes the network to prevent overfitting; different failure types of the fully-connected layer output bearing. The invention provides an efficient and accurate method for detecting the bearing fault, thereby effectively maintaining the normal operation of mechanical equipment.

Description

Bearing fault detection method based on convolution multi-head self-attention mechanism
Technical Field
The invention relates to the field of equipment health management, in particular to a bearing fault diagnosis method.
Background
The bearing is used as the core of the rotating mechanical component and is concerned with the normal operation of the whole mechanical equipment. During the operation of the equipment, the stator, the rotor and other parts of the bearing are easily damaged due to overload, friction, corrosion, gluing and the like. These failures can cause the entire mechanical equipment to fail, affect the performance of the production equipment, and can cause personnel injury. In order to maintain high performance of the machine while avoiding casualties and economic losses due to bearing failure, the best solution is to perform fault detection and health monitoring of the bearings.
With the rapid development of sensor technology, computer technology and information processing technology, the device health management method based on data driving and deep learning is becoming a new research hotspot and development trend. The method comprehensively utilizes sensor data and a machine learning theory, learns the failure characteristics of the machine by establishing deep learning model training data, can effectively solve the problem that diagnosis experts are rare relative to mechanical equipment, and can effectively monitor massive data and quickly predict the occurrence of accidents. The bearing fault can be effectively detected by utilizing the deep learning model.
The patent publication of invention with application number 201910728620.X discloses a rolling bearing obstacle diagnosis based on a self-attention neural network, which learns a vibration signal through a self-attention mechanism; the method attempts to learn bearing fault signatures using a single self-attentive approach, failing to learn rich bearing fault signatures. Meanwhile, a self-attention mechanism is directly used, and local characteristics of the bearing cannot be effectively learned. The method provided by the invention effectively solves the problems, and simultaneously uses the position encoder to provide position information when the multi-head self-attention module learns the bearing characteristics, thereby effectively solving the position information problem in the global characteristic learning.
Disclosure of Invention
Objects of the invention
The invention aims to provide a bearing fault detection method based on convolution multi-head attention. The bearing fault detection method is high in bearing fault detection precision and suitable for practical engineering projects.
(II) technical scheme
The technical scheme of the invention is that a bearing fault detection method based on convolution multi-head attention comprises the following steps: the method comprises the following steps of collecting vibration signals of a fault bearing, preprocessing the vibration signals to generate a bearing fault data set, constructing a convolution multi-head self-attention mechanism network, and further training to obtain a bearing fault detection result.
The method comprises the steps of collecting fault signals of the bearing, collecting different types of fault bearing vibration signal parameter information through a sensor, and recording a bearing fault label.
The bearing signal preprocessing operation is to carry out standard normalization processing on the bearing signal and then cut the bearing signal in equal length, wherein the standard normalization function is as follows:
Figure BDA0002553145630000021
where x represents the sample signal, μ represents the sample signal mean, and σ represents the sample signal standard deviation.
And generating a bearing fault data set, carrying out equal-quantity random selection on different types of fault bearing signals, and randomly dividing the fault bearing signals into a training set, a verification set and a test set according to the ratio of 7:2: 1.
Constructing a convolution multi-head self-attention mechanism network, wherein the structure sequentially comprises the following steps: convolutional layer → position encoder → multi-headed self-attention module → global averaging pooling layer → fully-connected layer output layer. The convolution layer is a one-dimensional convolution neural network, the number of convolution kernels is set to be 32, the size of the convolution kernels is set to be 8 multiplied by 8, and the step length is set to be 8; the position encoder uses positional encoding:
Figure BDA0002553145630000031
PE is a two-dimensional matrix, sin variables are added at even positions, cos variables are added at odd positions, the whole PE matrix is filled, the initial characteristics of the bearing signals are extracted by using a convolutional neural network, position coding is completed, and when multi-head self-attention is used, the characteristic position coding is facilitated to learn associated characteristics.
The multi-head self-attention mechanism allows the model to jointly pay attention to information from different representation subspaces at different positions, and the self-attention mechanism is independently used, so that abundant characteristic information cannot be obtained, and the bearing characteristic learning process comprises the following steps:
Figure BDA0002553145630000032
wherein a is a bearing feature matrix, WqA weight matrix consisting of query vectors q (query), WkA weight matrix consisting of key vectors k (key), WvA weight matrix consisting of a vector of values v (value); using scaled dot productsAs an attention mechanism:
Figure BDA0002553145630000033
dkis the square root of the key vector dimension, in the self-attention mechanism the output of self-attention is a weighted sum of value vectors v, the weight assigned to each value vector is calculated by the degree of correlation of the query vector q and the current key vector k, the multi-headed self-attention mechanism:
Figure BDA0002553145630000034
wherein Wi Q,Wi K,Wi VAnd WoAre learnable parameters.
The global average pooling layer can reduce the network parameter quantity and prevent the overfitting phenomenon.
The number of neurons in the fully connected layer equals the total number of bearing fault classes.
Training a convolution multi-head self-attention mechanism network: inputting the bearing training set and the verification set into a convolution multi-head self-attention machine network, setting a learning rate learning _ rate of the network to be 0.0055 by using a cross entropy loss function, training the network by using a gradient descent method, and updating the weight values and the learning rates of the training set and the verification set until the network loss function is converged to obtain the trained convolution multi-head self-attention machine network; classifying fault bearing signals: and inputting the bearing test set into the trained convolution multi-head self-attention mechanism network to obtain a bearing fault detection result, and calculating the precision of the convolution multi-head self-attention mechanism network model for bearing fault detection by comparing the bearing fault detection result with a correct label.
The invention realizes the intelligent detection of automatic extraction of various fault characteristics of the bearing. The invention identifies the accurate identification of various fault characteristics of the bearing.
(III) advantageous effects
The technical scheme of the invention has the following beneficial technical effects: the method is used for identifying vibration signals of various fault bearings, the experimental result is shown in FIG. 4, the identification result reaches 98.8%, and the result proves that the bearing fault detection method based on the convolution multi-head self-attention mechanism is effective.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a graph of loss function during training according to an embodiment of the present invention;
FIG. 3 is a graph of an accuracy function during training according to an embodiment of the present invention;
FIG. 4 is a recognition result confusion diagram according to an embodiment of the present invention;
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings in conjunction with the following detailed description. It should be understood that the description is intended to be exemplary only, and is not intended to limit the scope of the present invention. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present invention.
Embodiment 1 of the present invention, a bearing fault detection method based on convolution multi-head attention, which is shown in fig. 1, is performed according to the following steps:
the method comprises the steps of collecting fault signals of the bearing, collecting 10 different types of fault bearing vibration signal parameter information through a sensor, and recording a bearing fault label.
The bearing signal preprocessing operation is to carry out standard normalization processing on the bearing signal, then cut the bearing signal according to 2048 sampling points in equal length, and the standard normalization function is as follows:
Figure BDA0002553145630000051
where x represents the sample signal, μ represents the sample signal mean, and σ represents the sample signal standard deviation.
And (3) generating a bearing fault data set, randomly selecting 1000 parts of 10 different types of fault bearing signals, and randomly dividing the signals into a training set, a verification set and a test set according to the ratio of 7:2: 1.
Constructing a convolution multi-head self-attention mechanism network, wherein the structure sequentially comprises the following steps: convolutional layer → position encoder → multi-headed self-attention module → global averaging pooling layer → fully-connected layer output layer. The convolution layer is a one-dimensional convolution neural network, the number of convolution kernels is set to be 32, the size of the convolution kernels is set to be 8 multiplied by 8, and the step length is set to be 8; the position encoder uses positional encoding:
Figure BDA0002553145630000052
PE is a two-dimensional matrix, sin variables are added at even positions, cos variables are added at odd positions, the whole PE matrix is filled, the initial characteristics of the bearing signals are extracted by using a convolutional neural network, position coding is completed, and when multi-head self-attention is used, the characteristic position coding is facilitated to learn associated characteristics.
The multi-head self-attention mechanism allows the model to jointly pay attention to information from different representation subspaces at different positions, and the self-attention mechanism is independently used, so that abundant characteristic information cannot be obtained, and the bearing characteristic learning process comprises the following steps:
Figure BDA0002553145630000061
wherein a is a bearing feature matrix, WqA weight matrix consisting of query vectors q (query), WkA weight matrix consisting of key vectors k (key), WvA weight matrix consisting of a vector of values v (value); using the scaled dot product as the attention mechanism:
Figure BDA0002553145630000062
dkis the square root of the key vector dimension, and in the self-attention mechanism the output from attention is a weighted sum of value vectors v, assigned to each value vectorThe weight of the key vector k is calculated through the correlation degree of the query vector q and the current key vector k, and the multi-head self-attention mechanism is as follows:
Figure BDA0002553145630000063
wherein Wi Q,Wi K,Wi VAnd WoAre learnable parameters.
The global average pooling layer can reduce the network parameter quantity and prevent the overfitting phenomenon.
The full-connection layer outputs different types of faults respectively in a model training stage, the number of the neurons in the full-connection layer is equal to the number of the bearing fault types, namely the number of the neurons is set to be 10.
Training a convolution multi-head self-attention mechanism network: inputting the bearing training set and the verification set into a convolution multi-head self-attention mechanism network, setting a learning rate learning _ rate of the network to be 0.0055 by using a cross entropy loss function, training the network by using a gradient descent method, and updating the weight values and the learning rates of the training set and the verification set until the network loss function is converged to obtain the trained convolution multi-head self-attention mechanism network. The loss functions of the training set and the validation set during the training process are shown in fig. 2. The accuracy of the training set and the validation set during the training process is shown in fig. 3. Inputting the bearing test set into the trained convolution multi-head self-attention mechanism network to obtain a bearing fault detection result, comparing the bearing fault detection result with a correct label, calculating the bearing fault detection accuracy of the convolution multi-head self-attention mechanism network model, wherein the bearing fault detection accuracy reaches 98.8%, and the identification result confusion graph is shown in fig. 4.
Those skilled in the art will appreciate that the invention may be practiced without these specific details.

Claims (1)

1. A bearing fault detection method based on a convolution multi-head self-attention mechanism is characterized by comprising the following steps: collecting a vibration signal of a fault bearing and preprocessing the vibration signal to generate a bearing fault data set, constructing a convolution multi-head self-attention mechanism network, and further training to obtain a bearing fault detection result, wherein the method comprises the following steps;
(1) collecting and preprocessing a fault bearing vibration signal:
acquiring different kinds of fault bearing vibration signal parameter information through a sensor, and recording a bearing fault label; carrying out standard normalization processing on the bearing signals, and then cutting the bearing signals in equal length, wherein the standard normalization function is as follows:
Figure FDA0002553145620000011
wherein x represents a sample signal, μ represents a sample signal average value, and σ represents a sample signal standard deviation;
(2) generating a bearing fault data set:
carrying out equal-quantity random selection on different types of fault bearing signals, and randomly dividing the fault bearing signals into a training set, a verification set and a test set according to the ratio of 7:2: 1;
(3) constructing a convolution multi-head self-attention mechanism network, wherein the structure sequentially comprises the following steps: convolutional layer → position encoder → multi-headed self-attention module → global average pooling layer → fully-connected layer output layer;
(3a) the convolution layer is a one-dimensional convolution neural network, the number of convolution kernels is set to be 32, the size of the convolution kernels is set to be 8 multiplied by 8, and the step length is set to be 8;
(3b) the position encoder uses positional encoding:
Figure FDA0002553145620000021
PE is a two-dimensional matrix, sin variables are added at even positions, cos variables are added at odd positions, the whole PE matrix is filled up, the initial characteristics of the bearing signals are extracted by using a convolutional neural network, position coding is completed, and when multi-head self-attention is used, the characteristic position coding is facilitated to learn associated characteristics;
(3c) the multi-head self-attention mechanism allows the model to jointly pay attention to information from different representation subspaces at different positions, and the self-attention mechanism is independently used, so that abundant characteristic information cannot be obtained, and the bearing characteristic learning process comprises the following steps:
Figure FDA0002553145620000022
wherein a is a bearing feature matrix, WqA weight matrix consisting of query vectors q (query), WkA weight matrix consisting of key vectors k (key), WvA weight matrix consisting of a vector of values v (value);
self-attention mechanism using scaled dot products:
Figure FDA0002553145620000023
dkis the square root of the key vector dimension, the output from attention is the weighted sum of the value vectors V, the weight assigned to each value vector is calculated by the degree of correlation of the query vector Q and the current key vector K, Q, K, V is the multi-headed self-attention mechanism:
Figure FDA0002553145620000024
the input of the multi-head self-attention mechanism is changed from Q, K and V into QWi Q,KWi K,VWi VSelecting an 8-head self-attention mechanism, changing the dimensionality of Q, K and V from the original 8n dimensionality into n dimensionality in dimensionality, calculating one head each time, then splicing the 8 times of scaling dot product self-attention results, and performing W-time scaling on the resultsoPerforming linear transformation to obtain a final multi-head self-attention value;
(3d) the global average pooling layer can reduce the number of network parameters and prevent an overfitting phenomenon;
(3e) the number of neurons in the fully connected layer is equal to the total number of bearing fault categories;
(4) training a convolution multi-head self-attention mechanism network:
inputting the bearing training set and the verification set into a convolution multi-head self-attention machine network, setting a learning rate learning _ rate of the network to be 0.0055 by using a cross entropy loss function, training the network by using a gradient descent method, and updating the weight values and the learning rates of the training set and the verification set until the network loss function is converged to obtain the trained convolution multi-head self-attention machine network;
(5) classifying fault bearing signals:
and inputting the bearing test set into the trained convolution multi-head self-attention mechanism network to obtain a bearing fault detection result, and calculating the precision of the convolution multi-head self-attention mechanism network model for bearing fault detection by comparing the bearing fault detection result with a correct label.
CN202010583027.3A 2020-06-23 2020-06-23 Bearing fault detection method based on convolution multi-head self-attention mechanism Active CN111721535B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010583027.3A CN111721535B (en) 2020-06-23 2020-06-23 Bearing fault detection method based on convolution multi-head self-attention mechanism

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010583027.3A CN111721535B (en) 2020-06-23 2020-06-23 Bearing fault detection method based on convolution multi-head self-attention mechanism

Publications (2)

Publication Number Publication Date
CN111721535A CN111721535A (en) 2020-09-29
CN111721535B true CN111721535B (en) 2021-11-30

Family

ID=72568428

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010583027.3A Active CN111721535B (en) 2020-06-23 2020-06-23 Bearing fault detection method based on convolution multi-head self-attention mechanism

Country Status (1)

Country Link
CN (1) CN111721535B (en)

Families Citing this family (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112417954B (en) * 2020-10-13 2022-12-16 同济大学 Bearing fault mode diagnosis method and system for small sample data set
CN112183453B (en) * 2020-10-15 2021-05-11 哈尔滨市科佳通用机电股份有限公司 Deep learning-based water injection port cover plate unlocking-in-place fault detection method and system
CN112257572B (en) * 2020-10-20 2022-02-01 神思电子技术股份有限公司 Behavior identification method based on self-attention mechanism
CN112629854B (en) * 2020-11-25 2022-08-05 西安交通大学 Bearing fault classification method based on neural network attention mechanism
CN112528548A (en) * 2020-11-27 2021-03-19 东莞市汇林包装有限公司 Self-adaptive depth coupling convolution self-coding multi-mode data fusion method
CN112819037B (en) * 2021-01-12 2024-01-30 广东石油化工学院 Fault diagnosis method based on classification parameter distribution of cross attention and self attention
CN112906739B (en) * 2021-01-18 2021-11-05 河南工业大学 Fault diagnosis method based on multi-head attention and shafting equipment periodicity
CN113158445B (en) * 2021-04-06 2022-10-21 中国人民解放军战略支援部队航天工程大学 Prediction algorithm for residual service life of aero-engine with convolution memory residual error self-attention mechanism
CN112801069B (en) * 2021-04-14 2021-06-29 四川翼飞视科技有限公司 Face key feature point detection device, method and storage medium
CN113205177B (en) * 2021-04-25 2022-03-25 广西大学 Electric power terminal identification method based on incremental collaborative attention mobile convolution
CN113255780B (en) * 2021-05-28 2024-05-03 润联智能科技股份有限公司 Reduction gearbox fault prediction method and device, computer equipment and storage medium
CN113344070A (en) * 2021-06-01 2021-09-03 南京林业大学 Remote sensing image classification system and method based on multi-head self-attention module
CN113392214B (en) * 2021-06-03 2022-09-06 齐鲁工业大学 K selection strategy-based sparse self-attention text classification method and system
CN113469263B (en) * 2021-07-13 2024-06-14 润联智能科技股份有限公司 Prediction model training method and device suitable for small samples and related equipment
CN113343591B (en) * 2021-07-16 2022-05-03 浙江大学 Product key part life end-to-end prediction method based on self-attention network
CN113674225A (en) * 2021-07-30 2021-11-19 南京信息工程大学 Power equipment fault detection method based on convolutional neural network
CN113865868B (en) * 2021-08-24 2023-12-22 东南大学 Rolling bearing fault diagnosis method based on time-frequency domain expression
CN114638256B (en) * 2022-02-22 2024-05-31 合肥华威自动化有限公司 Transformer fault detection method and system based on acoustic wave signals and attention network
CN114332825B (en) * 2022-03-10 2022-06-17 中汽信息科技(天津)有限公司 Road terrain distribution identification method and device based on deep learning and storage medium
CN114993677B (en) * 2022-05-11 2023-05-02 山东大学 Rolling bearing fault diagnosis method and system for unbalanced small sample data
CN114937021A (en) * 2022-05-31 2022-08-23 哈尔滨工业大学 Swin-Transformer-based crop disease fine-granularity classification method
CN114913396A (en) * 2022-07-15 2022-08-16 西北工业大学 Motor bearing fault diagnosis method
CN115659283A (en) * 2022-12-12 2023-01-31 陕西金元新能源有限公司 Wind power equipment damage prediction method based on attention mechanism of multi-task learning
CN116243683B (en) * 2023-03-15 2024-02-13 青岛澎湃海洋探索技术有限公司 Method for diagnosing faults of propulsion system based on torque and multi-head self-encoder
CN116428129B (en) * 2023-06-13 2023-09-01 山东大学 Fan blade impact positioning method and system based on attention mixing neural network
CN116773544B (en) * 2023-06-15 2024-07-26 上海工程技术大学 Steel structure plate fault detection method and system based on deep learning
CN117153144B (en) * 2023-10-31 2024-02-06 杭州宇谷科技股份有限公司 Battery information voice broadcasting method and device based on terminal calculation

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107421741A (en) * 2017-08-25 2017-12-01 南京信息工程大学 A kind of Fault Diagnosis of Roller Bearings based on convolutional neural networks
CN108681539A (en) * 2018-05-07 2018-10-19 内蒙古工业大学 A kind of illiteracy Chinese nerve interpretation method based on convolutional neural networks
CN108734290A (en) * 2018-05-16 2018-11-02 湖北工业大学 It is a kind of based on the convolutional neural networks construction method of attention mechanism and application
CN109165667A (en) * 2018-07-06 2019-01-08 中国科学院自动化研究所 Based on the cerebral disease categorizing system from attention mechanism
CN109726524A (en) * 2019-03-01 2019-05-07 哈尔滨理工大学 A kind of rolling bearing remaining life prediction technique based on CNN and LSTM
CN109902399A (en) * 2019-03-01 2019-06-18 哈尔滨理工大学 Rolling bearing fault recognition methods under a kind of variable working condition based on ATT-CNN
CN109919205A (en) * 2019-02-25 2019-06-21 华南理工大学 Based on bull from the convolution echo state network timing classification method of attention mechanism
CN110196946A (en) * 2019-05-29 2019-09-03 华南理工大学 A kind of personalized recommendation method based on deep learning
CN110355608A (en) * 2019-07-18 2019-10-22 浙江大学 Based on the tool abrasion prediction technique from attention mechanism and deep learning
CN110608884A (en) * 2019-08-08 2019-12-24 桂林电子科技大学 Rolling bearing state diagnosis method based on self-attention neural network
CN110619045A (en) * 2019-08-27 2019-12-27 四川大学 Text classification model based on convolutional neural network and self-attention
CN110672343A (en) * 2019-09-29 2020-01-10 电子科技大学 Rotary machine fault diagnosis method based on multi-attention convolutional neural network
WO2020068831A1 (en) * 2018-09-26 2020-04-02 Visa International Service Association Dynamic graph representation learning via attention networks
CN111046907A (en) * 2019-11-02 2020-04-21 国网天津市电力公司 Semi-supervised convolutional network embedding method based on multi-head attention mechanism

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107421741A (en) * 2017-08-25 2017-12-01 南京信息工程大学 A kind of Fault Diagnosis of Roller Bearings based on convolutional neural networks
CN108681539A (en) * 2018-05-07 2018-10-19 内蒙古工业大学 A kind of illiteracy Chinese nerve interpretation method based on convolutional neural networks
CN108734290A (en) * 2018-05-16 2018-11-02 湖北工业大学 It is a kind of based on the convolutional neural networks construction method of attention mechanism and application
CN109165667A (en) * 2018-07-06 2019-01-08 中国科学院自动化研究所 Based on the cerebral disease categorizing system from attention mechanism
WO2020068831A1 (en) * 2018-09-26 2020-04-02 Visa International Service Association Dynamic graph representation learning via attention networks
CN109919205A (en) * 2019-02-25 2019-06-21 华南理工大学 Based on bull from the convolution echo state network timing classification method of attention mechanism
CN109902399A (en) * 2019-03-01 2019-06-18 哈尔滨理工大学 Rolling bearing fault recognition methods under a kind of variable working condition based on ATT-CNN
CN109726524A (en) * 2019-03-01 2019-05-07 哈尔滨理工大学 A kind of rolling bearing remaining life prediction technique based on CNN and LSTM
CN110196946A (en) * 2019-05-29 2019-09-03 华南理工大学 A kind of personalized recommendation method based on deep learning
CN110355608A (en) * 2019-07-18 2019-10-22 浙江大学 Based on the tool abrasion prediction technique from attention mechanism and deep learning
CN110608884A (en) * 2019-08-08 2019-12-24 桂林电子科技大学 Rolling bearing state diagnosis method based on self-attention neural network
CN110619045A (en) * 2019-08-27 2019-12-27 四川大学 Text classification model based on convolutional neural network and self-attention
CN110672343A (en) * 2019-09-29 2020-01-10 电子科技大学 Rotary machine fault diagnosis method based on multi-attention convolutional neural network
CN111046907A (en) * 2019-11-02 2020-04-21 国网天津市电力公司 Semi-supervised convolutional network embedding method based on multi-head attention mechanism

Non-Patent Citations (8)

* Cited by examiner, † Cited by third party
Title
Attention Augmented Convolutional Networks;I. Bello等;《2019 IEEE/CVF International Conference on Computer Vision (ICCV)》;20191231;第3285-3294页 *
Dilated Residual Network with Multi-head Self-attention for Speech Emotion Recognition;R. Li等;《ICASSP 2019 - 2019 IEEE International Conference on Acoustics》;20191231;第6675-6679页 *
Exploring a Unified Attention-Based Pooling Framework for Speaker Verification;Y. Liu等;《018 11th International Symposium on Chinese Spoken Language Processing (ISCSLP)》;20181231;第200-204页 *
Global-Local Temporal Representations for Video Person Re-Identification;J. Li等;《2019 IEEE/CVF International Conference on Computer Vision (ICCV)》;20191231;第3957-3966页 *
Self-Attention based Network For Medical Query Expansion;S. Chen等;《2019 International Joint Conference on Neural Networks (IJCNN)》;20191231;第1-9页 *
基于Transformer重建的时序数据异常检测与关系提取;孟恒宇;《计算机工程》;20200226;第69-76页 *
基于多注意力机制的深度神经网络故障诊断算法;王翔等;《浙江理工大学学报(自然科学版)》;20191202;第43卷(第2期);第224-231页 *
融合多头自注意力机制的中文分类方法;熊漩等;《电子测量技术》;20200531;第125-130页 *

Also Published As

Publication number Publication date
CN111721535A (en) 2020-09-29

Similar Documents

Publication Publication Date Title
CN111721535B (en) Bearing fault detection method based on convolution multi-head self-attention mechanism
CN111914873B (en) Two-stage cloud server unsupervised anomaly prediction method
Ko et al. Fault classification in high-dimensional complex processes using semi-supervised deep convolutional generative models
CN111562108A (en) Rolling bearing intelligent fault diagnosis method based on CNN and FCMC
CN113865868B (en) Rolling bearing fault diagnosis method based on time-frequency domain expression
Shajihan et al. CNN based data anomaly detection using multi-channel imagery for structural health monitoring
CN110261116A (en) A kind of Bearing Fault Detection Method and device
CN116593157A (en) Complex working condition gear fault diagnosis method based on matching element learning under small sample
CN115859077A (en) Multi-feature fusion motor small sample fault diagnosis method under variable working conditions
CN110263767A (en) Intelligent rotating shaft fault diagnosis method combining compressed data acquisition and deep learning
CN110991471B (en) Fault diagnosis method for high-speed train traction system
CN109034076A (en) A kind of automatic clustering method and automatic cluster system of mechanical fault signals
CN112305388B (en) On-line monitoring and diagnosing method for insulation partial discharge faults of generator stator winding
CN115018012B (en) Internet of things time sequence anomaly detection method and system under high dimensionality characteristics
Tao et al. Fault diagnosis of rolling bearing using deep belief networks
CN117473411A (en) Bearing life prediction method based on improved transducer model
CN116735170A (en) Intelligent fault diagnosis method based on self-attention multi-scale feature extraction
CN116796224A (en) Gear box fault diagnosis method based on time-frequency domain image and convolutional neural network
CN111428788A (en) Deep learning-based multi-fault diagnosis method and system for steam turbine generator set rotor
Techane et al. Rotating machinery prognostics and application of machine learning algorithms: Use of deep learning with similarity index measure for health status prediction
CN111539381B (en) Construction method of wind turbine bearing fault classification diagnosis model
CN117664558A (en) Generator gear box abnormality detection method, device, equipment and storage medium
CN112861275A (en) Rotary machine fault diagnosis method based on minimum information entropy feature learning model
CN114926702B (en) Small sample image classification method based on depth attention measurement
CN116167008A (en) Abnormal positioning method for internet of things sensing cloud data center based on data enhancement

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant