CN112926641A - Three-stage feature fusion rotating machine fault diagnosis method based on multi-modal data - Google Patents

Three-stage feature fusion rotating machine fault diagnosis method based on multi-modal data Download PDF

Info

Publication number
CN112926641A
CN112926641A CN202110196388.7A CN202110196388A CN112926641A CN 112926641 A CN112926641 A CN 112926641A CN 202110196388 A CN202110196388 A CN 202110196388A CN 112926641 A CN112926641 A CN 112926641A
Authority
CN
China
Prior art keywords
stage
feature fusion
feature
data
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110196388.7A
Other languages
Chinese (zh)
Other versions
CN112926641B (en
Inventor
李沂滨
王代超
贾磊
宋艳
高晟耀
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University
Original Assignee
Shandong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University filed Critical Shandong University
Priority to CN202110196388.7A priority Critical patent/CN112926641B/en
Publication of CN112926641A publication Critical patent/CN112926641A/en
Application granted granted Critical
Publication of CN112926641B publication Critical patent/CN112926641B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
    • G01M13/00—Testing of machine parts
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a three-stage feature fusion rotating machine fault diagnosis method based on multi-modal data, which is used for acquiring parameter data of a machine running state to obtain data of at least two modes; respectively inputting the acquired modal data into a preset neural network model to obtain a final fault classification result; the preset neural network model at least sequentially comprises a first self-feature fusion stage, a mutual-feature fusion stage and a second self-feature fusion stage; the method adopts a three-stage feature fusion method comprising a one-dimensional convolutional neural network and a two-dimensional convolutional neural network to fuse multi-scale features and perform fault diagnosis, wherein the two-dimensional convolutional neural network extracts the correlation between feature mapping maps, and an attention mechanism can perform different weight distribution on the feature maps, so that important information is highlighted, redundant information is reduced, and the performance of fault diagnosis is greatly improved.

Description

Three-stage feature fusion rotating machine fault diagnosis method based on multi-modal data
Technical Field
The disclosure relates to the technical field of mechanical fault diagnosis, in particular to a three-stage feature fusion rotating mechanical fault diagnosis method based on multi-mode data.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
With the continuous development of smart manufacturing, industrial systems become more and more complex and nonlinear, and the loss caused by equipment damage is greater and greater. Early fault detection can not only eliminate faults before the faults cause huge economic loss, but also avoid occurrence of major safety accidents. However, due to the complexity and non-linearity of industrial systems, it is difficult to build an accurate model. Due to the continuous development of information science and technology, industrial systems generate a large amount of operational data, which contains a large amount of valuable information. Therefore, in a complex system with high integration, the fault diagnosis method based on data driving is more effective than the model building method based on expert knowledge.
The traditional fault diagnosis and state monitoring method depends on indexes such as current imbalance, overvoltage and the like, and faults can not be accurately judged and positioned at an early stage with weak fault signals and short duration. In the past decades, conventional Machine Learning (ML) algorithms have been widely used for fault diagnosis, such as Support Vector Machines (SVMs), decision trees, and Artificial Neural Networks (ANNs). However, despite the success of these intelligent approaches, there are two drawbacks: (1) these intelligent fault diagnosis methods need to be used in combination with feature extraction methods, resulting in strong dependence on feature selection. In addition, the feature extraction and classification are separately designed, the time consumption is long, and the global optimization cannot be carried out; (2) most of the methods belong to shallow structures, and for complex systems, effective feature representation and nonlinear mapping relations are difficult to learn.
Deep Learning (DL) has been studied by researchers in various fields in recent years because it is possible to learn a Deep feature representation and a nonlinear mapping relationship. Further, unlike the conventional fault diagnosis method, DL can adaptively extract features and perform global optimization. And a Convolutional Neural Network (CNN) is the most commonly used deep learning method in image recognition, and also shows better performance in fault diagnosis.
Although DL has good performance in feature extraction, most fault diagnosis methods only use single-modality data. To date, most fault diagnosis studies have employed only a single source of signal, such as vibration and current. However, in complex industrial systems, it is difficult to obtain high quality fault information using a single signal source. Studies have shown that multi-sensor signals typically contain complementary fault information compared to single-sensor data. Therefore, the fault diagnosis method based on data fusion can effectively improve the accuracy and reliability of fault diagnosis. Data fusion can be divided into a data level, a feature level and a decision level according to different fusion degrees. However, data-level fusion requires a highly consistent data structure, and in decision-level fusion, when there is a large decision conflict between classifiers, the final classification result will be very unstable.
Disclosure of Invention
In order to solve the defects of the prior art, the invention provides a three-stage feature fusion rotary machine fault diagnosis method based on multi-mode data, which adopts a three-stage feature fusion method comprising a one-dimensional convolution neural network and a two-dimensional convolution neural network to fuse multi-scale features and carry out fault diagnosis, the two-dimensional convolution neural network extracts the correlation among feature mapping maps, and an attention mechanism can carry out different weight distribution on the feature maps, thereby highlighting important information, reducing redundant information and greatly improving the performance of fault diagnosis.
In order to achieve the purpose, the following technical scheme is adopted in the disclosure:
the first aspect of the disclosure provides a three-stage feature fusion rotary machine fault diagnosis method based on multi-modal data.
A three-stage feature fusion rotating machine fault diagnosis method based on multi-modal data is characterized by comprising the following steps: the method comprises the following steps:
acquiring mechanical operation state parameter data to obtain data of at least two modes;
respectively inputting the acquired modal data into a preset neural network model to obtain a final fault classification result;
the preset neural network model at least sequentially comprises a first self-feature fusion stage, a mutual-feature fusion stage and a second self-feature fusion stage.
A second aspect of the present disclosure provides a three-stage feature fusion rotary machine fault diagnosis system based on multi-modal data.
A three-stage feature fusion rotary machine fault diagnosis system based on multi-modal data, comprising:
a data acquisition module configured to: acquiring mechanical operation state parameter data to obtain data of at least two modes;
a fault classification module configured to: respectively inputting the acquired modal data into a preset neural network model to obtain a final fault classification result;
the preset neural network model at least sequentially comprises a first self-feature fusion stage, a mutual-feature fusion stage and a second self-feature fusion stage.
A third aspect of the present disclosure provides a computer-readable storage medium on which a program is stored, the program, when executed by a processor, implementing the steps in the three-stage feature fusion rotational mechanical fault diagnosis method based on multi-modal data according to the first aspect of the present disclosure.
A fourth aspect of the present disclosure provides an electronic device, including a memory, a processor, and a program stored on the memory and executable on the processor, wherein the processor implements the steps of the method for diagnosing a fault of a three-stage feature fusion rotating machine based on multi-modal data according to the first aspect of the present disclosure when executing the program.
Compared with the prior art, the beneficial effect of this disclosure is:
the method, the system, the medium or the electronic equipment disclosed by the disclosure researches the fault diagnosis performance of different modal data and different layer feature fusion, explores the influence rule of different layer feature fusion on the diagnosis performance, adopts a three-stage feature fusion method comprising 1D CNN (one-dimensional convolutional neural network) and 2D CNN (two-dimensional convolutional neural network) to fuse multi-scale features and perform fault diagnosis, wherein the 2D CNN extracts the correlation between feature mapping maps, and an attention mechanism can perform different weight distribution on the feature maps, so that important information is highlighted, redundant information is reduced, and the fault diagnosis performance is greatly improved.
Advantages of additional aspects of the disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the disclosure.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure and are not to limit the disclosure.
Fig. 1 is a schematic diagram of a convolution-pooling layer in a 1D CNN provided in embodiment 1 of the present disclosure.
Fig. 2 is a schematic diagram of a convolution-pooling layer in a 2D CNN provided in embodiment 1 of the present disclosure.
Fig. 3 is a schematic diagram of an AMDC-CNN network structure provided in embodiment 1 of the present disclosure.
Fig. 4 is a schematic diagram of a 1D CNN module based on an attention mechanism provided in embodiment 1 of the present disclosure.
Fig. 5 is a schematic diagram of a stacking process of feature images provided in embodiment 1 of the present disclosure.
Detailed Description
The present disclosure is further described with reference to the following drawings and examples.
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
The embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict.
Example 1:
the embodiment 1 of the present disclosure provides a three-stage feature fusion rotating machine fault diagnosis method based on multi-modal data, which is characterized in that: the method comprises the following steps:
acquiring mechanical operation state parameter data to obtain data of at least two modes;
respectively inputting the acquired modal data into a preset neural network model to obtain a final fault classification result;
the preset neural network model at least sequentially comprises a first self-feature fusion stage, a mutual-feature fusion stage and a second self-feature fusion stage.
Current research is focused primarily on fusing the last layer of features of multimodal data. As the feature layer deepens, part of fault information is lost, and the fusion of features of the last layer may not be the best choice. For the problem, in the embodiment, vibration and torque signal data of a Paderborn bearing data set are adopted, fourier transform is performed on the two data to obtain four modal data, the four modal data are classified by adopting 1D CNN, and the two modal data with the best diagnostic performance are searched; and then, carrying out feature layer fusion on the two kinds of modal data, respectively verifying the diagnostic performance when carrying out different layer feature fusion by adopting the two kinds of modal data, and researching the influence rule of the different layer feature fusion on the diagnostic performance.
In the fault diagnosis of mechanical equipment, the detection quantity and the fault characteristics and the fault source are always in a nonlinear relation, and most of CNN-based characteristic fusion methods only adopt simple convolution layer and pooling layer superposition and connect the characteristics on a single dimension, so that the nonlinear mapping relation is difficult to extract. The second problem solved is therefore the data fusion network model building problem. To solve this problem, the present embodiment adopts a new fault diagnosis method (Attention based Multi-dimensional localization conditioner CNN, AMDC-CNN) with Multi-modal data three-stage feature fusion. Three-stage fault feature fusion is carried out from two aspects of self-feature fusion and mutual feature fusion, complementary fault information in multi-mode data is fully fused, and the accuracy of fault diagnosis is improved. And different weight distributions are carried out on the feature map by applying an attention mechanism, so that important information is highlighted, and redundant information is reduced. The fused features are then used for final fault classification to improve final classification accuracy.
Specifically, the method comprises the following steps:
similar to other computational techniques, CNN's inspiration comes from image recognition mechanisms of the mammalian visual cortex. Unlike the global image processing method, the image it obtains from the retina is processed in a hierarchical and distributed manner. A group of nerve cells acts directly on the input to extract basic features, such as edge features. Convolution is a common spatial linear filtering method in image processing, and the three most important features using convolution kernel are: local/sparse connections, weight/parameter sharing, and translation invariance representation, which makes CNN require less pre-processing. Unlike other DNNs, CNNs use smaller convolution kernels to act on local regions of the input image to extract subtle and critical features.
The 1D CNN generally includes a convolutional layer, a pooling layer, and a fully-connected layer, and its working mechanism can be summarized as: the convolution kernel slides in the whole sequence by proper steps to extract local features, and the extracted feature value changes with different convolution kernel weight vectors in the convolution layer. The convolutional layer is always connected to the sub-sampling layer (e.g., Max-Pooling) through a non-linear mapping function (e.g., ReLu), and the appropriate sub-sampling layer can effectively reduce the dimensionality of the input without losing information. And after connecting the plurality of convolutions and the pools, extracting effective characteristic vectors, and classifying results through a full connection layer. In the training process, all the weights such as convolution kernels of different convolution layers are updated by effective learning algorithms such as random gradient descent and the like. A one-dimensional convolution-pooling module is shown in fig. 1, where the red dashed line and the green line represent convolution and sub-sampling operations, respectively.
In the figure, the output of the first convolutional layer can be calculated as follows:
Figure BDA0002946852950000071
wherein N isl-1Representing the number of (l-1) layer pooling layer outputs,
Figure BDA0002946852950000072
a bias scalar representing the kth neuron in the l-th convolutional layer,
Figure BDA0002946852950000073
represents the weight of the kth neuron in the first convolutional layer,
Figure BDA0002946852950000074
represents the output of the i-th neuron in the (l-1) -th pooling layer, cov1D represents a one-dimensional convolution operation, and f (-) represents the activation function of the convolutional layer.
The output of the l-th pooling layer can be calculated as follows:
Sl＝ss(Yl) (2)
where ss denotes the down-sampling operation.
2D CNN works much the same as 1D CNN, except that a two-dimensional convolution kernel and pooling window are used to extract features. A two-dimensional convolution-pooling module is shown in fig. 2.
The schematic diagram of AMDC-CNN proposed in this embodiment is shown in fig. 3. The AMDC-CNN is composed of a plurality of data input channels, the number of the channels can be set according to actual conditions, and the data input channels have expandability. While the present embodiment sets AMDC-CNN to dual channels. And then, performing feature extraction and fusion on the input data by adopting three stages of self-feature fusion, mutual feature fusion and self-feature fusion, and finally inputting the input data to a full-connection layer for fault classification. The purpose of AMDC-CNN is to find complementary information and related information between different features from different input data by connecting features of different dimensions.
Specifically, the AMDC-CNN is composed of four processes, namely a first feature fusion stage, a second feature fusion stage, a third feature fusion stage, and a fault classification stage.
S1: first feature fusion phase
The first feature fusion stage belongs to a self-feature fusion stage and is used for independently processing data of two channels. Each channel consists of two attention-based 1D CNN modules as shown in fig. 4.
In the first attention-based 1D CNN module, the input data is first convolved. The parameters of the convolution operation are (16,7,1), wherein the three parameters respectively represent the number of convolution kernels, the size of the kernels and the convolution step size. Then applying the maximum dimensionality reduction of (3,3), which respectively represent the pooling window size and the pooling step number, and then performing standardization processing by Batch Normalization (BN) to obtain C116 characteristic graphs.
Then, use this C1The feature maps generate evaluation vectors to evaluate the importance of the feature maps. Handling this C with global average pooling1Feature maps and generate C corresponding to the feature maps1Original evaluation vector composed of elements. The vector is then transformed by two Fully Connected (FC) layers, whose activation functions are ReLU and softmax, respectively. In fig. 4, r is a dimensionality reduction ratio, and the dimensionality reduction ratio in all models is set to 8. Based on these two FCs, a final evaluation vector can be obtained, and C is calculated1Multiplying the features with the vector recalibrates the features.
But as the feature layer deepens, part of the failure information is lost. Therefore, in order to ensure the integrity of the module input data as much as possible, the convolution operation of (1,1,1) is adopted to carry out convolution on the input data, and then the feature mapping is connectedTo C before1On the feature map, form a new C1+1 feature maps as the output of this module. In the second attention-based 1D CNN block, the convolution operation of (32,5,1) is performed first, and other parameters are the same as those of the first block. In addition, the network structure of the two channels is the same.
S2: second feature fusion phase
The second feature fusion stage belongs to the inter-feature fusion stage, i.e. features extracted from two channels are connected in a different dimension than in the first stage.
Suppose that the profiles of the two channels are respectively F1And F2The output of this stage can be expressed as:
Figure BDA0002946852950000091
where, Conc and Trans denote splicing operation and transposition operation, respectively, W1And W2Characteristic diagram lengths, C, of two channels, respectively2The + 1-33 is the number of feature maps for each channel, and the feature maps for two channels are merged into one.
S3: third feature fusion phase
In the third feature fusion stage, the multi-scale features of the connected feature maps are further extracted by using 1D CNN and 2D CNN in this embodiment. Specifically, a stack of two attention-based 1D CNN modules is used in the first channel, the parameters of the convolution operation are (64,3,1) and (128,3,1), respectively, and the other parameters are the same as those in the first feature fusion stage, because of C2The +1 feature maps are extracted from the same input, so there is some correlation between them.
To extract the relevant information, the feature maps are stacked into a two-dimensional image, and feature extraction is performed in the second channel using 2D CNN, the stacking process is shown in fig. 5. After the feature image is obtained, the first module convolves with a (64, [3,3], [1,1])2D CNN, these three parameters have the same meaning as the one-dimensional CNN, and then uses maximal pooling of ([3,3], [3,3]) followed by a BN. In addition, as in the first feature fusion stage, to ensure the integrity of the input information, a (1, [1,1], [1,1]) convolution operation is also used. In the second module, the parameters are the same as the first module except that (128, [3,3], [1,1])2D CNN is used first.
S4: fault classification phase
And fault classification is carried out by adopting Softmax regression at the last full connection layer, and the Softmax cross entropy loss function is minimized in the back propagation process. In the classification process, the softmax function maps the extracted features to all fault categories, so that each fault category corresponds to a probability, which can be expressed as:
Figure BDA0002946852950000101
in the formula
Figure BDA0002946852950000102
Is the jth input, N, of the softmax functionclassAs total number of fault classes, pj(x) The j-th fault category corresponds to a probability, the sum of all the probabilities is 1, and the category with the highest probability is the classification result. Then calculating the error between the output of the softmax function and the real label through a cross entropy loss function:
Figure BDA0002946852950000103
where y is the true fault tag of the input, yjAnd the output probability of the softmax function corresponding to the jth fault category. The weights and biases are updated by calculating the partial derivatives of E (y, p (x)). For example, the update process of the weights can be expressed as:
Figure BDA0002946852950000104
wherein l is the learning rate, and controls the update speed of the weight w.
Example 2:
the embodiment 2 of the present disclosure provides a three-stage feature fusion rotating machine fault diagnosis system based on multi-modal data, including:
a data acquisition module configured to: acquiring mechanical operation state parameter data to obtain data of at least two modes;
a fault classification module configured to: respectively inputting the acquired modal data into a preset neural network model to obtain a final fault classification result;
the preset neural network model at least sequentially comprises a first self-feature fusion stage, a mutual-feature fusion stage and a second self-feature fusion stage.
The working method of the system is the same as the three-stage feature fusion rotating machine fault diagnosis method based on multi-modal data provided in embodiment 1, and details are not repeated here.
Example 3:
the embodiment 3 of the present disclosure provides a computer-readable storage medium, on which a program is stored, which when executed by a processor, implements the steps in the method for diagnosing a fault of a three-stage feature fusion rotating machine based on multi-modal data according to embodiment 1 of the present disclosure, where the steps are:
acquiring mechanical operation state parameter data to obtain data of at least two modes;
respectively inputting the acquired modal data into a preset neural network model to obtain a final fault classification result;
the preset neural network model at least sequentially comprises a first self-feature fusion stage, a mutual-feature fusion stage and a second self-feature fusion stage.
The detailed steps are the same as those of the three-stage feature fusion rotary machine fault diagnosis method based on multi-modal data provided in embodiment 1, and are not described again here.
Example 4:
the embodiment 4 of the present disclosure provides an electronic device, which includes a memory, a processor, and a program stored in the memory and executable on the processor, where the processor executes the program to implement the steps in the method for diagnosing a fault of a three-stage feature fusion rotating machine based on multi-modal data according to the embodiment 1 of the present disclosure, where the steps are as follows:
acquiring mechanical operation state parameter data to obtain data of at least two modes;
respectively inputting the acquired modal data into a preset neural network model to obtain a final fault classification result;
the preset neural network model at least sequentially comprises a first self-feature fusion stage, a mutual-feature fusion stage and a second self-feature fusion stage.
The detailed steps are the same as those of the three-stage feature fusion rotary machine fault diagnosis method based on multi-modal data provided in embodiment 1, and are not described again here.
As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure, and various modifications and changes may be made to the present disclosure by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.

Claims (10)

1. A three-stage feature fusion rotating machine fault diagnosis method based on multi-modal data is characterized by comprising the following steps: the method comprises the following steps:
acquiring mechanical operation state parameter data to obtain data of at least two modes;
respectively inputting the acquired modal data into a preset neural network model to obtain a final fault classification result;
the preset neural network model at least sequentially comprises a first self-feature fusion stage, a mutual-feature fusion stage and a second self-feature fusion stage.
2. A three-stage feature fusion rotary machine fault diagnosis method based on multi-modal data as claimed in claim 1, characterized in that:
in the first self-feature fusion stage, data of each mode corresponds to one channel, and each channel consists of two one-dimensional convolution neural network modules with the same structure and based on an attention mechanism.
3. A three-stage feature fusion rotary machine fault diagnosis method based on multi-modal data as claimed in claim 2, characterized in that:
in each one-dimensional convolution neural network module, performing first convolution operation, dimensionality reduction and batch normalization processing on input data to obtain C feature maps;
processing the obtained C feature maps by adopting global average pooling, generating an original evaluation vector consisting of C elements corresponding to the feature maps, and transforming the original evaluation vector by two full-connection layers to obtain a final evaluation vector;
multiplying the obtained C characteristic graphs by the evaluation vector to recalibrate the characteristic graphs;
the input data is convolved with a second convolution operation and the feature map is then concatenated to the previous feature map to form a new C +1 feature map as the output of the module.
4. A three-stage feature fusion rotary machine fault diagnosis method based on multi-modal data as claimed in claim 2, characterized in that:
and in the mutual feature fusion stage, the feature graphs output by each channel are connected in a dimension different from that of the first self-feature fusion stage, and the feature maps of the two channels are merged into one.
5. A three-stage feature fusion rotary machine fault diagnosis method based on multi-modal data as claimed in claim 2, characterized in that:
and in the second self-feature fusion stage, a one-dimensional convolution neural network module and a two-dimensional convolution neural network module are adopted to extract multi-scale features of the connected feature map.
6. A three-stage feature fusion rotary machine fault diagnosis method based on multi-modal data as claimed in claim 5, characterized in that:
and stacking two one-dimensional convolution neural network modules in at least one channel, stacking the characteristic diagram into a two-dimensional image in at least one channel, and then extracting the characteristics by using the two-dimensional convolution neural network modules.
7. A three-stage feature fusion rotary machine fault diagnosis method based on multi-modal data as claimed in claim 6, characterized in that:
and inputting the data of each channel output in the second self-feature fusion stage into the full connection layer, performing fault classification on the full connection layer by adopting Softmax regression, and minimizing a Softmax cross entropy loss function in the back propagation process.
8. A three-stage feature fusion rotating machinery fault diagnosis system based on multi-modal data is characterized in that: the method comprises the following steps:
a data acquisition module configured to: acquiring mechanical operation state parameter data to obtain data of at least two modes;
a fault classification module configured to: respectively inputting the acquired modal data into a preset neural network model to obtain a final fault classification result;
the preset neural network model at least sequentially comprises a first self-feature fusion stage, a mutual-feature fusion stage and a second self-feature fusion stage.
9. A computer-readable storage medium, on which a program is stored, which, when being executed by a processor, carries out the steps of the three-stage feature fusion rotational mechanical fault diagnosis method based on multi-modal data according to any one of claims 1 to 7.
10. An electronic device comprising a memory, a processor, and a program stored on the memory and executable on the processor, wherein the processor implements the steps of the method for three-stage feature fusion rotational mechanical fault diagnosis based on multi-modal data according to any one of claims 1-7 when executing the program.
CN202110196388.7A 2021-02-22 2021-02-22 Three-stage feature fusion rotating machine fault diagnosis method based on multi-mode data Active CN112926641B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110196388.7A CN112926641B (en) 2021-02-22 2021-02-22 Three-stage feature fusion rotating machine fault diagnosis method based on multi-mode data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110196388.7A CN112926641B (en) 2021-02-22 2021-02-22 Three-stage feature fusion rotating machine fault diagnosis method based on multi-mode data

Publications (2)

Publication Number Publication Date
CN112926641A true CN112926641A (en) 2021-06-08
CN112926641B CN112926641B (en) 2022-05-17

Family

ID=76170056

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110196388.7A Active CN112926641B (en) 2021-02-22 2021-02-22 Three-stage feature fusion rotating machine fault diagnosis method based on multi-mode data

Country Status (1)

Country Link
CN (1) CN112926641B (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113281029A (en) * 2021-06-09 2021-08-20 重庆大学 Rotating machinery fault diagnosis method and system based on multi-scale network structure
CN113392912A (en) * 2021-06-18 2021-09-14 大唐环境产业集团股份有限公司 Multi-mode operation fault diagnosis and early warning method, system and equipment for slurry circulating pump
CN113608063A (en) * 2021-06-25 2021-11-05 北京智芯微电子科技有限公司 Power line fault identification method and device and electronic equipment
CN113639993A (en) * 2021-08-17 2021-11-12 燕山大学 Gearbox fault diagnosis method of multi-mode multi-task convolutional neural network
CN113720605A (en) * 2021-07-26 2021-11-30 中国中材国际工程股份有限公司 Cement production rotating equipment fault diagnosis method based on machine learning
CN113780151A (en) * 2021-09-07 2021-12-10 山东大学 Bearing fault diagnosis method and system based on bilinear feature fusion
CN113988163A (en) * 2021-10-20 2022-01-28 向前 Radar high-resolution range profile identification method based on multi-scale grouping fusion convolution
CN114660993A (en) * 2022-05-25 2022-06-24 中科航迈数控软件(深圳)有限公司 Numerical control machine tool fault prediction method based on multi-source heterogeneous data feature dimension reduction
CN115143128A (en) * 2022-06-28 2022-10-04 浙江石水泵业科技有限公司 Fault diagnosis method and system for small submersible electric pump
CN117273085A (en) * 2023-11-17 2023-12-22 山东智慧译百信息技术有限公司 Information classification method based on convolutional neural network
CN117540190A (en) * 2023-11-22 2024-02-09 广东粤电科试验检测技术有限公司 Transformer fault diagnosis method and system based on CNN self-feature fusion
CN118275824A (en) * 2024-05-30 2024-07-02 深圳友讯达科技股份有限公司 Intelligent detection early warning method based on ring main unit and related equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102054179A (en) * 2010-12-14 2011-05-11 广州大学 Online state monitoring and fault diagnosis device and method for rotary machine
US20180060719A1 (en) * 2016-08-29 2018-03-01 International Business Machines Corporation Scale-space label fusion using two-stage deep neural net
CN108614548A (en) * 2018-04-03 2018-10-02 北京理工大学 A kind of intelligent failure diagnosis method based on multi-modal fusion deep learning
CN108681747A (en) * 2018-05-11 2018-10-19 武汉理工大学 Rotary machinery fault diagnosis based on deep learning and condition monitoring system and method
CN111275007A (en) * 2020-02-24 2020-06-12 山东大学 Bearing fault diagnosis method and system based on multi-scale information fusion

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102054179A (en) * 2010-12-14 2011-05-11 广州大学 Online state monitoring and fault diagnosis device and method for rotary machine
US20180060719A1 (en) * 2016-08-29 2018-03-01 International Business Machines Corporation Scale-space label fusion using two-stage deep neural net
CN108614548A (en) * 2018-04-03 2018-10-02 北京理工大学 A kind of intelligent failure diagnosis method based on multi-modal fusion deep learning
CN108681747A (en) * 2018-05-11 2018-10-19 武汉理工大学 Rotary machinery fault diagnosis based on deep learning and condition monitoring system and method
CN111275007A (en) * 2020-02-24 2020-06-12 山东大学 Bearing fault diagnosis method and system based on multi-scale information fusion

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
吴静然 等: "采用多尺度注意力机制的旋转机械故障诊断方法", 《西安交通大学学报》 *

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113281029A (en) * 2021-06-09 2021-08-20 重庆大学 Rotating machinery fault diagnosis method and system based on multi-scale network structure
CN113392912A (en) * 2021-06-18 2021-09-14 大唐环境产业集团股份有限公司 Multi-mode operation fault diagnosis and early warning method, system and equipment for slurry circulating pump
CN113608063A (en) * 2021-06-25 2021-11-05 北京智芯微电子科技有限公司 Power line fault identification method and device and electronic equipment
CN113720605A (en) * 2021-07-26 2021-11-30 中国中材国际工程股份有限公司 Cement production rotating equipment fault diagnosis method based on machine learning
CN113639993B (en) * 2021-08-17 2022-06-07 燕山大学 Gearbox fault diagnosis method of multi-mode multi-task convolutional neural network
CN113639993A (en) * 2021-08-17 2021-11-12 燕山大学 Gearbox fault diagnosis method of multi-mode multi-task convolutional neural network
CN113780151A (en) * 2021-09-07 2021-12-10 山东大学 Bearing fault diagnosis method and system based on bilinear feature fusion
CN113988163A (en) * 2021-10-20 2022-01-28 向前 Radar high-resolution range profile identification method based on multi-scale grouping fusion convolution
CN114660993A (en) * 2022-05-25 2022-06-24 中科航迈数控软件(深圳)有限公司 Numerical control machine tool fault prediction method based on multi-source heterogeneous data feature dimension reduction
CN115143128A (en) * 2022-06-28 2022-10-04 浙江石水泵业科技有限公司 Fault diagnosis method and system for small submersible electric pump
CN117273085A (en) * 2023-11-17 2023-12-22 山东智慧译百信息技术有限公司 Information classification method based on convolutional neural network
CN117273085B (en) * 2023-11-17 2024-03-01 山东智慧译百信息技术有限公司 Information classification method based on convolutional neural network
CN117540190A (en) * 2023-11-22 2024-02-09 广东粤电科试验检测技术有限公司 Transformer fault diagnosis method and system based on CNN self-feature fusion
CN118275824A (en) * 2024-05-30 2024-07-02 深圳友讯达科技股份有限公司 Intelligent detection early warning method based on ring main unit and related equipment

Also Published As

Publication number Publication date
CN112926641B (en) 2022-05-17

Similar Documents

Publication Publication Date Title
CN112926641B (en) Three-stage feature fusion rotating machine fault diagnosis method based on multi-mode data
CN111898095B (en) Deep migration learning intelligent fault diagnosis method, device, storage medium and equipment
CN111275007B (en) Bearing fault diagnosis method and system based on multi-scale information fusion
CN110657984B (en) Planetary gearbox fault diagnosis method based on reinforced capsule network
CN113191390B (en) Image classification model construction method, image classification method and storage medium
CN111612051A (en) Weak supervision target detection method based on graph convolution neural network
CN113887342A (en) Equipment fault diagnosis method based on multi-source signals and deep learning
CN109785409B (en) Image-text data fusion method and system based on attention mechanism
CN115980560A (en) CNN-GRU-based high-voltage circuit breaker mechanical fault diagnosis system, method and equipment
CN111524140A (en) Medical image semantic segmentation method based on CNN and random forest method
CN116385773A (en) Small target detection method, storage medium and electronic equipment
CN115797629A (en) Example segmentation method based on detection enhancement and multi-stage bounding box feature refinement
CN113962262B (en) Continuous learning-based intelligent radar signal sorting method
CN117593514B (en) Image target detection method and system based on deep principal component analysis assistance
Ma Summary of research on application of deep learning in image recognition
CN108898157B (en) Classification method for radar chart representation of numerical data based on convolutional neural network
CN116206151A (en) Image classification residual neural network training realization method
CN117235490A (en) Fault self-adaptive diagnosis method integrating deep volume and self-attention network
CN115578325A (en) Image anomaly detection method based on channel attention registration network
Liu et al. A deep neural network pruning method based on gradient L1-norm
CN114267062A (en) Model training method, electronic device, and computer-readable storage medium
Prystavka et al. RECOGNITION OF AERIAL PHOTOGRAPHY OBJECTS BASED ON DATA SETS WITH DIFFERENT AGGREGATION OF CLASSES.
CN117523549B (en) Three-dimensional point cloud object identification method based on deep and wide knowledge distillation
Zhang et al. Efficient Bearing Fault Diagnosis by Fast-Residual Network With 2-D Representation of Vibration Signals
CN116756483B (en) Mechanical fault diagnosis method, device and equipment under condition that target working condition data are unavailable

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant