CN114764575A - Multi-modal data classification method based on deep learning and time sequence attention mechanism - Google Patents
Multi-modal data classification method based on deep learning and time sequence attention mechanism Download PDFInfo
- Publication number
- CN114764575A CN114764575A CN202210376944.3A CN202210376944A CN114764575A CN 114764575 A CN114764575 A CN 114764575A CN 202210376944 A CN202210376944 A CN 202210376944A CN 114764575 A CN114764575 A CN 114764575A
- Authority
- CN
- China
- Prior art keywords
- signal
- layer
- convolution
- characteristic signal
- inputting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
A multi-mode data classification method based on deep learning and time sequence attention mechanisms comprises the steps of firstly utilizing PC-TBG-ECG and PC-TBG-PCG models to respectively realize feature extraction of electrocardiosignals and heart sound signals, and then adopting an XGboost integration classification algorithm to perform feature selection and classification on the extracted features. While the operation efficiency is increased, the regularization is added, and overfitting is effectively prevented. The method is suitable for classification detection of data in different modes, and can analyze signals from various angles, so that the classification accuracy is improved.
Description
Technical Field
The invention relates to the field of multi-modal data classification, in particular to a multi-modal data classification method based on deep learning and time sequence attention mechanism.
Background
Electrocardiogram (ECG) and Phonocardiogram (PCG) are non-invasive and cost-effective signal acquisition tools, and potential features of two signals can be mined and analyzed from various angles according to complementarity between the two, so that the classification effect is improved. In the conventional research, related researchers mainly use single-mode data or a single classifier to classify signals, but the classification research using the method cannot classify the signals from the comprehensive point of view, so the classification method fusing multi-mode data proposed by the research is extremely suitable for the practical requirement.
Disclosure of Invention
In order to overcome the defects of the technology, the invention provides a method which is suitable for classification detection of data in different modes, can analyze signals from various angles and further improves the accuracy of classification.
The technical scheme adopted by the invention for overcoming the technical problems is as follows:
a multi-modal data classification method based on deep learning and time sequence attention mechanism comprises the following steps:
a) selecting training-a in PhysioNet/CinC Challenge 2016 as a data set, expanding the data set, and dividing the expanded data set into a training set and a test set;
b) establishing an electrocardiosignal model, wherein the electrocardiosignal model is sequentially composed of a PC module, a TBG module and a classification module;
c) resampling the electrocardiosignals in the training set and the testing set to 2048 sampling points, and then carrying out z-score normalization processing to obtain normalized electrocardiosignals x'ecg;
d) Normalizing the electrocardiosignals x 'in the training set'ecgInputting the signal into a PC module of the electrocardiosignal model, and outputting the signal to obtain a characteristic signal X1The PC module is composed of four convolution branches and a 1 multiplied by 1 convolution block in sequence;
e) the characteristic signal X1Inputting the signal into a TBG module of the electrocardiosignal model, and outputting the signal to obtain a characteristic signal X2The TBG module consists of 3 convolutional coding modules and a bidirectional GRU layer with a TPA mechanism;
f) the characteristic signal X2Inputting the prediction data into a classification module of the electrocardiosignal model, and outputting the prediction data to obtain a prediction category fecgThe classification module is sequentially composed of a full connection layer and a Softmax activation layer;
g) repeating the steps d) to f) for N times, and obtaining an optimal electrocardiosignal model after training by using an SGD optimizer and minimizing a cross entropy loss function;
h) establishing a heart sound signal model which sequentially consists of a PC module, a TBG module and a classification module;
i) resampling the heart sound signals in the training set and the test set to 8000 sampling points, and then carrying out z-score normalization processing to obtain normalized heart sound signals x'pcg;
j) The heart sound signals x 'after normalization in the training set'pcgInputting the signal into PC module of heart sound signal model, outputting to obtain characteristic signal Y1The PC module is composed of four convolution branches and a 1 multiplied by 1 convolution block in sequence;
k) the characteristic signal Y1Inputting the signal into TBG module of heart sound signal model, and outputting to obtain characteristic signal Y2The TBG module consists of 4 convolutional coding modules and a bidirectional GRU layer with a TPA mechanism;
l) applying the characteristic signal Y2Inputting the prediction into a classification module of a heart sound signal model, and outputting the prediction category fpcgThe classification module is sequentially composed of a full connection layer and a Softmax activation layer;
m) repeating the steps j) to l) M times, and obtaining an optimal heart sound signal model after training by using an SGD optimizer and minimizing a cross entropy loss function;
n) manually dividing the data set into a new training set and a new testing set according to the proportion of 4:1, inputting the new training set into the optimal electrocardiosignal model, and outputting a 64-dimensional characteristic signal X through a TBG (tunnel boring generator) module of the optimal electrocardiosignal model3Inputting the new training set into the optimal heart sound signal model, and passing through the optimal heart sound signal modelThe TBG module outputs a 64-dimensional characteristic signal Y3By the formula PPx=[X3,Y3]Calculating to obtain spliced 128-dimensional feature fusion signal PPx;
o) fusing features into a signal PPxInputting the signals into an XGboost classifier to obtain a feature fusion signal PPxThe importance score ranking of (2) and selecting the signal of the top 64 of the importance score ranking as the characteristic signal PP1 xSelecting an optimal hyper-parameter by adopting 5-fold cross validation, and training the XGboost classifier by utilizing the optimal hyper-parameter to obtain an optimized XGboost classifier;
p) inputting the new test set into the optimal electrocardiosignal model, and outputting a 64-dimensional characteristic signal X through a TBG (tunnel boring generator) module of the optimal electrocardiosignal model4Inputting the new test set into the optimal heart sound signal model, and outputting a 64-dimensional characteristic signal Y through a TBG (tunnel boring generator) module of the optimal heart sound signal model4By the formula PPc=[X4,Y4]Calculating to obtain spliced 128-dimensional feature fusion signal PPc;
q) feature fusion signal PPcInputting the signals into an XGboost classifier to obtain a feature fusion signal PPcThe importance score ranking of (2) and selecting the signal of the top 64 of the importance score ranking as the characteristic signal PP1 c。
Preferably, the data set is expanded in step a) by using a sliding window segmentation method, and the data set is divided into 5 different training sets and test sets by using a five-fold cross validation method.
Further, in step c), the formula is usedCalculating to obtain a normalized electrocardiosignal x'ecgIn the formula xecgFor training and testing the concentrated ECG signal, uecgIs the mean value, σ, of the electrocardiosignalecgIs the variance of the electrocardiosignals.
Further, step d) comprises the following steps:
d-1) the firstOne convolution branch is composed of a convolution layer with 32 channel numbers, convolution kernel size of 1 × 15 and step length of 1, a batch normalization layer and a ReLU activation layer in sequence, and electrocardiosignals x 'after normalization in a training set'ecgInputting the signal into the first convolution branch, and outputting to obtain a 32-dimensional characteristic signal E1;
d-2) the second convolution branch comprises a convolution layer with 32 channels, convolution kernel size of 1 × 13 and step length of 1, a batch normalization layer and a ReLU activation layer in sequence, and the electrocardiosignal x 'after normalization in the training set'ecgInputting the signal into the second convolution branch, and outputting to obtain a 32-dimensional characteristic signal E2;
d-3) the third convolution branch comprises a convolution layer with 32 channels, convolution kernel size of 1 × 9 and step size of 1, a batch normalization layer and a ReLU activation layer in sequence, and the electrocardiosignal x 'after normalization in the training set'ecgInputting the signal into a third convolution branch, and outputting to obtain a 32-dimensional characteristic signal E3;
d-4) the fourth convolution branch comprises a convolution layer with a channel number of 32, a convolution kernel size of 1 x 5 and a step length of 1, a batch normalization layer and a ReLU activation layer in sequence, and the electrocardiosignals x 'after normalization in the training set'ecgInputting the signal into the fourth convolution branch, and outputting to obtain a 32-dimensional characteristic signal E4;
d-5) converting the characteristic signal E1Characteristic signal E2Characteristic signal E3Characteristic signal E4Performing characteristic cascade to obtain a 128-dimensional characteristic signal E ═ E after cascade1,E2,E3,E4];
d-6)1 × 1 convolution block is composed of convolution layers with 16 channels, 1 × 1 convolution kernel size, and 1 step size, and ReLU active layer, and the 128-dimensional characteristic signal E is set to [ E ═ E1,E2,E3,E4]Inputting the signal into a 1 × 1 convolution block, and outputting to obtain a 16-dimensional characteristic signal X1。
Further, step e) comprises the steps of:
e-1) the first convolutional encoding module sequentially comprises convolutional layers with the number of channels being 32 and the size of convolutional kernel being 1 multiplied by 11, and batch normalizationLayer, ReLU active layer, and pooling layer of size 4, and applying the characteristic signal X1Inputting the signal into a first convolution coding module, and outputting to obtain a 32-dimensional characteristic signal E5;
E-2) the second convolutional coding module sequentially comprises a convolutional layer with the channel number of 64 and the convolutional kernel size of 1 multiplied by 7, a batch normalization layer, a ReLU activation layer and a pooling layer with the size of 2, and a characteristic signal E is obtained5Inputting the signal into a second convolutional coding module, and outputting to obtain a 64-dimensional characteristic signal E6;
E-3) the third convolutional coding module consists of a convolutional layer with the channel number of 128 and the convolutional kernel size of 3, a batch normalization layer, a ReLU activation layer and a pooling layer with the size of 2 in sequence, and a characteristic signal E is generated6Inputting the signal into a third convolutional coding module, and outputting to obtain a 128-dimensional characteristic signal E7;
E-4) converting the characteristic signal E7Inputting into 32-unit bidirectional GRU layer with TPA mechanism, and outputting to obtain 64-dimensional characteristic signal X2In the bidirectional GRU layer of TPA mechanism by formulaCalculating to obtain a characteristic signal X2Where i ═ 1, 2.., n }, n ═ 128, T is transposition, τ isiFor the attention weight of the ith row vector,σ (-) is a sigmoid function,is a time pattern matrix GCRow i of (1), GCConv1d (G), Conv1d (·) is a one-dimensional convolution operation, G is a hidden state matrix,gifor the hidden state vector of the ith bidirectional GRU, i ═ 1,2, akIs a weight coefficient, gtIs the hidden state vector of the bi-directional GRU at time t.
Further, in the step g), the value of N is 150, the learning rate of the SGD optimizer is 0.001, the learning rate is attenuated to be 0.1 at every 80 periods, and the formula is usedCalculating to obtain a cross entropy loss function cc (x), wherein L is the number of categories, L is 2, fi(x) To predict class fecgThe predictive label of the ith category of (c),as a prediction class fecgThe real category of the corresponding ith category; in the step m), the value of N is 180, the learning rate of the SGD optimizer is 0.001, the learning rate is attenuated to be 0.1 at every 90 periods, and the N is calculated according to a formulaCalculating to obtain a cross entropy loss function cc (y), wherein L is the number of categories, L is 2, fi(y) is prediction class fpcgThe predictive label of the ith category of (c),as a prediction class fpcgTrue category of the ith category of (1).
Further, in step i), the formula is usedCalculating to obtain a normalized heart sound signal x'pcgIn the formula xpcgFor the heart sound signals in the training set and test set, upcgIs the mean value, σ, of the heart sound signalpcgIs the variance of the heart sound signal.
Further, step j) comprises the following steps:
j-1) the first convolution branch is composed of convolution layer with 32 channels, convolution kernel size of 1 × 15 and step size of 2, batch normalization layer and ReLU activation layer in sequence, and the heart sound signal x 'after normalization in training set'pcgInputting the signal into the first convolution branch, and outputting to obtain a 32-dimensional characteristic signal P1;
j-2) the second convolution branch comprises a convolution layer with 32 channels, convolution kernel size of 1 × 11 and step size of 2, a batch normalization layer and a ReLU activation layer in sequence, and the heart sound signal x 'after normalization in the training set'pcgInputting the signal into the second convolution branch, and outputting to obtain 32-dimensional characteristic signal P2;
j-3) the third convolution branch comprises a convolution layer with a channel number of 32, a convolution kernel size of 1 × 9 and a step size of 2, a batch normalization layer and a ReLU activation layer in sequence, and the heart sound signal x 'after normalization in the training set'pcgInputting the signal into a third convolution branch, and outputting to obtain a 32-dimensional characteristic signal P3;
j-4) the fourth convolution branch comprises a convolution layer with a channel number of 32, a convolution kernel size of 1 × 5 and a step size of 2, a batch normalization layer and a ReLU activation layer in sequence, and the heart sound signal x 'after normalization in the training set'pcgInputting the signal into the fourth convolution branch, and outputting to obtain a 32-dimensional characteristic signal P4;
j-5) converting the characteristic signal P1Characteristic signal P2Characteristic signal P3Characteristic signal P4Performing characteristic cascade to obtain a 128-dimensional characteristic signal P ═ P after cascade1,P2,P3,P4];
j-6)1 × 1 convolution block is composed of convolution layer with 32 channel number, convolution kernel size 1 × 1 and step size 1 and ReLU active layer, and 128-dimensional characteristic signal P is [ P ═ P1,P2,P3,P4]Inputting into a 1 × 1 convolution block, and outputting to obtain a 32-dimensional characteristic signal Y1。
Further, step k) comprises the steps of:
k-1) the first convolutional coding module sequentially comprises a convolutional layer with the number of channels being 16 and the size of a convolutional kernel being 1 multiplied by 1, a batch normalization layer, a ReLU active layer and a pooling layer with the size being 4, and a characteristic signal Y is obtained1Inputting the data into a first convolutional coding module, and outputting to obtain a 16-dimensional characteristic signal P5;
k-2) the second convolutional encoding module sequentially comprises 32 channels and convolutional cores1 × 11 convolution layer, batch normalization layer, ReLU active layer, and pooling layer of size 2, and applying characteristic signal P5Inputting the signal into a second convolutional coding module, and outputting to obtain a 32-dimensional characteristic signal P6;
k-3) the third convolutional coding module sequentially comprises a convolutional layer with the channel number of 64 and the convolutional kernel size of 1 multiplied by 7, a batch normalization layer, a ReLU active layer and a pooling layer with the size of 2, and a characteristic signal P is obtained6Inputting the signal into a third convolutional coding module, and outputting to obtain a 64-dimensional characteristic signal P7;
k-4) the fourth convolutional coding module sequentially comprises a convolutional layer with the channel number of 128 and the convolutional kernel size of 1 multiplied by 3, a batch normalization layer, a ReLU active layer and a pooling layer with the size of 2, and a characteristic signal P is obtained7Inputting the signal into a fourth convolutional coding module, and outputting to obtain a 128-dimensional characteristic signal P8;
k-5) converting the characteristic signal P8Inputting into 32-unit bidirectional GRU layer with TPA mechanism, and outputting to obtain 64-dimensional characteristic signal Y2In the bidirectional GRU layer of TPA mechanism by formulaCalculating to obtain a characteristic signal Y2。
The invention has the beneficial effects that: firstly, the characteristics of electrocardiosignals and heart sound signals are respectively extracted by utilizing PC-TBG-ECG and PC-TBG-PCG models, and then the XGboost integrated classification algorithm is adopted to select and classify the extracted characteristics. While the operation efficiency is increased, the regularization is added, and overfitting is effectively prevented. The method is suitable for classification detection of data in different modes, and can analyze signals from various angles, so that the classification accuracy is improved.
Drawings
FIG. 1 is a flow chart of a method of the present invention;
fig. 2 is a network configuration diagram of the PC module of the present invention.
Detailed Description
The present invention is further described with reference to fig. 1 and 2.
A multi-modal data classification method based on deep learning and time sequence attention mechanism comprises the following steps:
a) selecting training-a in PhysioNet/CinC Challenge 2016 as a data set, expanding the data set, and dividing the expanded data set into a training set and a test set.
b) And establishing an electrocardiosignal model (PC-TBG-ECG), wherein the electrocardiosignal model is sequentially composed of a PC module, a TBG module and a classification module.
c) Resampling the electrocardiosignals in the training set and the testing set to 2048 sampling points, and then carrying out z-score normalization processing to obtain a normalized electrocardiosignal x'ecg。
d) Normalizing the electrocardiosignals x 'in the training set'ecgInputting the signal into a PC module of the electrocardiosignal model, and outputting the signal to obtain a characteristic signal X1The PC module, in turn, is made up of four convolution branches and a 1 x 1 convolution block.
e) The characteristic signal X1Inputting the signal into a TBG module of the electrocardiosignal model, and outputting the signal to obtain a characteristic signal X2The TBG module consists of 3 convolutional coding modules and a Bi-directional GRU layer (TPA-Bi-GRU) with TPA mechanism. f) The characteristic signal X2Inputting the prediction data into a classification module of the electrocardiosignal model, and outputting the prediction data to obtain a prediction category fecgThe classification module is composed of a full connection layer and a Softmax activation layer in sequence.
g) And (f) repeating the steps d) to f) N times, and obtaining the trained optimal electrocardiosignal model by using an SGD optimizer and minimizing a cross entropy loss function.
h) And establishing a heart sound signal model (PC-TBG-PCG), wherein the heart sound signal model is composed of a PC module, a TBG module and a classification module in sequence.
i) Resampling the heart sound signals in the training set and the test set to 8000 sampling points, and then carrying out z-score normalization processing to obtain normalized heart sound signals x'pcg。
j) The heart sound signals x 'after normalization in the training set'pcgInputting the signal into PC module of heart sound signal model, outputting to obtain characteristic signal Y1The PC module is sequentially composed of four volumesThe product branch is formed with a 1 x 1 convolution block.
k) The characteristic signal Y1Inputting the signal into a TBG module of the heart sound signal model, and outputting to obtain a characteristic signal Y2The TBG module consists of 4 convolutional coding modules and a Bi-directional GRU layer with TPA mechanism (TPA-Bi-GRU). l) applying the characteristic signal Y2Inputting the prediction into a classification module of a heart sound signal model, and outputting the prediction category fpcgThe classification module is composed of a full connection layer and a Softmax activation layer in sequence.
M) repeating the steps j) to l) M times, and obtaining the trained optimal heart sound signal model by minimizing a cross entropy loss function by using an SGD optimizer.
n) manually dividing the data set into a new training set and a new testing set according to the proportion of 4:1, inputting the new training set into the optimal electrocardiosignal model, and outputting a 64-dimensional characteristic signal X through a TBG (tunnel boring generator) module of the optimal electrocardiosignal model3Inputting the new training set into the optimal heart sound signal model, and outputting a 64-dimensional characteristic signal Y through a TBG module of the optimal heart sound signal model3By the formula PPx=[X3,Y3]Calculating to obtain spliced 128-dimensional feature fusion signal PPx。
o) fusing features into a signal PPxInputting the signals into an XGboost classifier to obtain a feature fusion signal PPxThe importance score ranking of (2) and selecting the signal of the top 64 of the importance score ranking as the characteristic signal PP1 xAnd selecting an optimal hyper-parameter by adopting 5-fold cross validation, and training the XGboost classifier by utilizing the optimal hyper-parameter to obtain the optimized XGboost classifier.
p) inputting the new test set into the optimal electrocardiosignal model, and outputting a 64-dimensional characteristic signal X through a TBG (tunnel boring generator) module of the optimal electrocardiosignal model4Inputting the new test set into the optimal heart sound signal model, and outputting a 64-dimensional characteristic signal Y through a TBG (tunnel boring generator) module of the optimal heart sound signal model4By the formula PPc=[X4,Y4]Calculating to obtain spliced 128-dimensional feature fusion signal PPc。
q) feature fusion signal PPcInputting the signals into an XGboost classifier to obtain a feature fusion signal PPcThe importance score ranking of (2) and selecting the signal of the top 64 of the importance score ranking as the characteristic signal PP1 c。
The signals do not need to be subjected to noise reduction, filtering and other processing, the problems of low classification accuracy rate or low practicability and the like caused by unreasonable signal preprocessing in the prior art are solved, and the robustness of the model is ensured. Firstly, the characteristics of electrocardiosignals and heart sound signals are respectively extracted by utilizing PC-TBG-ECG and PC-TBG-PCG models, and then the XGboost integrated classification algorithm is adopted to select and classify the extracted characteristics. While the operation efficiency is increased, the regularization is added, and overfitting is effectively prevented. The method is suitable for classification detection of different modal data, and can analyze signals from various angles, thereby improving the accuracy of classification.
Example 1:
in the step a), the data set is expanded by using a sliding window segmentation method, and the data set is divided into 5 different training sets and test sets by using a five-fold cross validation method.
Example 2:
in step c) by the formulaCalculating to obtain a normalized electrocardiosignal x'ecgIn the formula xecgFor training and testing the concentrated ECG signal, uecgIs the mean value, σ, of the electrocardiosignalecgIs the variance of the electrocardiosignals.
Example 3:
the step d) comprises the following steps:
d-1) the first convolution branch comprises a convolution layer with 32 channels, convolution kernel size of 1 × 15 and step size of 1, a batch normalization layer and a ReLU activation layer in sequence, and the electrocardiosignal x 'after normalization in the training set'ecgInputting the signal into the first convolution branch, and outputting to obtain a 32-dimensional characteristic signal E1;
d-2) The second convolution branch comprises convolution layer with 32 channels, convolution kernel size of 1 × 13 and step size of 1, batch normalization layer and ReLU activation layer in sequence, and the electrocardiosignal x 'after normalization in training set'ecgInputting the signal into the second convolution branch, and outputting to obtain a 32-dimensional characteristic signal E2;
d-3) the third convolution branch comprises a convolution layer with a channel number of 32, a convolution kernel size of 1 x 9 and a step length of 1, a batch normalization layer and a ReLU activation layer in sequence, and the electrocardiosignals x 'after normalization in the training set'ecgInputting the signal into a third convolution branch, and outputting to obtain a 32-dimensional characteristic signal E3;
d-4) the fourth convolution branch comprises a convolution layer with a channel number of 32, a convolution kernel size of 1 x 5 and a step length of 1, a batch normalization layer and a ReLU activation layer in sequence, and the electrocardiosignals x 'after normalization in the training set'ecgInputting the signal into the fourth convolution branch, and outputting to obtain a 32-dimensional characteristic signal E4;
d-5) converting the characteristic signal E1Characteristic signal E2Characteristic signal E3Characteristic signal E4Performing feature cascade to obtain 128-dimensional feature signal E ═ E after cascade1,E2,E3,E4];
d-6)1 × 1 convolution block is composed of convolution layers with 16 channels, 1 × 1 convolution kernel size, and 1 step size, and ReLU active layer, and the 128-dimensional characteristic signal E is set to [ E ═ E1,E2,E3,E4]Inputting the signal into a 1 × 1 convolution block, and outputting to obtain a 16-dimensional characteristic signal X1。
Example 4:
step e) comprises the following steps:
e-1) the first convolutional coding module consists of a convolutional layer with the number of channels of 32 and the convolutional kernel size of 1 multiplied by 11, a batch normalization layer, a ReLU activation layer and a pooling layer with the size of 4 in sequence, and the characteristic signal X is converted into a characteristic signal1Inputting the data into a first convolution coding module, and outputting to obtain a 32-dimensional characteristic signal E5;
e-2) the second convolutional encoding module sequentially consists of 64 channels and 1 × 7 convolutional kernel sizeThe convolution layer, the batch normalization layer, the ReLU activation layer, and the pooling layer with size of 2, and the characteristic signal E5Inputting the signal into a second convolutional coding module, and outputting to obtain a 64-dimensional characteristic signal E6;
E-3) the third convolution coding module consists of a convolution layer with the channel number of 128 and the convolution kernel size of 3, a batch normalization layer, a ReLU activation layer and a pooling layer with the size of 2 in sequence, and a characteristic signal E is generated6Inputting the signal into a third convolutional coding module, and outputting to obtain a 128-dimensional characteristic signal E7;
E-4) converting the characteristic signal E7Inputting into 32-unit bidirectional GRU layer with TPA mechanism, and outputting to obtain 64-dimensional characteristic signal X2In the bidirectional GRU layer of TPA mechanism by formulaCalculating to obtain a characteristic signal X2Where i ═ 1, 2.., n }, n ═ 128, T is transposition, τ isiFor the attention weight of the ith row vector,σ (-) is a sigmoid function,is a time pattern matrix GCRow i of (1), GCConv1d (G), Conv1d (·) is a one-dimensional convolution operation, G is a hidden state matrix,gifor the hidden state vector of the ith bi-directional GRU, i ═ 1,2, t-1, t is time, wkIs a weight coefficient, gtIs the hidden state vector of the bi-directional GRU at time t.
Example 5:
in the step g), the value of N is 150, the learning rate of the SGD optimizer is 0.001, the learning rate of every 80 periods is attenuated to be 0.1 currently, and the method is carried out by a formulaCalculating to obtain a cross entropy loss function cc (x), wherein L is the number of categories, L is 2, fi(x) As a prediction class fecgThe predictive tag of the ith category of (a),as a prediction class fecgThe real category of the corresponding ith category; in the step m), the value of N is 180, the learning rate of the SGD optimizer is 0.001, the learning rate is attenuated to be 0.1 at every 90 periods, and the N is calculated according to a formulaCalculating to obtain a cross entropy loss function cc (y), wherein L is the number of categories, L is 2, fi(y) is prediction class fpcgThe predictive label of the ith category of (c),as a prediction class fpcgTrue category of the ith category of (1).
Example 6:
in step i) by formulaCalculating to obtain a normalized heart sound signal x'pcgIn the formula xpcgFor the heart sound signals of the training set and the test set, upcgIs the mean value, σ, of the heart sound signalpcgIs the variance of the heart sound signal.
Example 7:
the step j) comprises the following steps:
j-1) the first convolution branch is composed of convolution layer with 32 channels, convolution kernel size of 1 × 15 and step size of 2, batch normalization layer and ReLU activation layer in sequence, and the heart sound signal x 'after normalization in training set'pcgInputting the signal into the first convolution branch, and outputting to obtain a 32-dimensional characteristic signal P1;
j-2) the second convolution branch consists of convolution layer with 32 channel number, convolution kernel size of 1 × 11 and step size of 2, batch normalization layer and ReLU activation layer in sequenceNormalized heart sound signal x 'in training set'pcgInputting the signal into the second convolution branch, and outputting to obtain a 32-dimensional characteristic signal P2;
j-3) the third convolution branch comprises a convolution layer with a channel number of 32, a convolution kernel size of 1 × 9 and a step size of 2, a batch normalization layer and a ReLU activation layer in sequence, and the heart sound signal x 'after normalization in the training set'pcgInputting the signal into a third convolution branch, and outputting to obtain a 32-dimensional characteristic signal P3;
j-4) the fourth convolution branch comprises a convolution layer with a channel number of 32, a convolution kernel size of 1 × 5 and a step size of 2, a batch normalization layer and a ReLU activation layer in sequence, and the heart sound signal x 'after normalization in the training set'pcgInputting the signal into the fourth convolution branch, and outputting to obtain a 32-dimensional characteristic signal P4;
j-5) converting the characteristic signal P1Characteristic signal P2Characteristic signal P3Characteristic signal P4Performing characteristic cascade to obtain a 128-dimensional characteristic signal P ═ P after cascade1,P2,P3,P4];
j-6)1 × 1 convolution block is composed of convolution layer with 32 channel number, convolution kernel size 1 × 1 and step size 1 and ReLU active layer, and 128-dimensional characteristic signal P is [ P ═ P1,P2,P3,P4]Inputting into a 1 × 1 convolution block, and outputting to obtain a 32-dimensional characteristic signal Y1。
Example 8:
step k) comprises the following steps:
k-1) the first convolutional coding module sequentially comprises a convolutional layer with the number of channels being 16 and the convolutional kernel size being 1 multiplied by 1, a batch normalization layer, a ReLU activation layer and a pooling layer with the size being 4, and the feature signal Y is processed1Inputting the signals into a first convolutional encoding module, and outputting to obtain a 16-dimensional characteristic signal P5;
k-2) the second convolutional coding module sequentially comprises a convolutional layer with the number of channels of 32 and the convolutional kernel size of 1 multiplied by 11, a batch normalization layer, a ReLU activation layer and a pooling layer with the size of 2, and the feature signal P is converted into a linear convolution function5Input to a second convolutional encoding moduleIn the method, 32-dimensional characteristic signal P is obtained by output6;
k-3) the third convolutional coding module sequentially comprises a convolutional layer with the channel number of 64 and the convolutional kernel size of 1 multiplied by 7, a batch normalization layer, a ReLU active layer and a pooling layer with the size of 2, and a characteristic signal P is obtained6Inputting the signal into a third convolutional coding module, and outputting to obtain a 64-dimensional characteristic signal P7;
k-4) the fourth convolutional coding module sequentially comprises a convolutional layer with the channel number of 128 and the convolutional kernel size of 1 multiplied by 3, a batch normalization layer, a ReLU active layer and a pooling layer with the size of 2, and a characteristic signal P is obtained7Inputting the data into a fourth convolutional coding module, and outputting to obtain a 128-dimensional characteristic signal P8;
k-5) converting the characteristic signal P8Inputting into 32-unit bidirectional GRU layer with TPA mechanism, and outputting to obtain 64-dimensional characteristic signal Y2In the bidirectional GRU layer of TPA mechanism by formulaCalculating to obtain a characteristic signal Y2。
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (9)
1. A multi-modal data classification method based on deep learning and time-series attention mechanism is characterized by comprising the following steps:
a) selecting training-a in PhysioNet/CinC Challenge 2016 as a data set, expanding the data set, and dividing the expanded data set into a training set and a test set;
b) establishing an electrocardiosignal model, wherein the electrocardiosignal model consists of a PC module, a TBG module and a classification module in sequence;
c) resampling the electrocardiosignals in the training set and the testing set to 2048 sampling points, and then carrying out z-score normalization processing to obtain a normalized electrocardiosignal x'ecg;
d) Normalizing the electrocardiosignals x 'in the training set'ecgInputting the signal into a PC module of the electrocardiosignal model, and outputting the signal to obtain a characteristic signal X1The PC module is composed of four convolution branches and a 1 multiplied by 1 convolution block in sequence;
e) the characteristic signal X1Inputting the signal into a TBG module of the electrocardiosignal model, and outputting the signal to obtain a characteristic signal X2The TBG module consists of 3 convolutional coding modules and a bidirectional GRU layer with a TPA mechanism;
f) the characteristic signal X2Inputting the signals into a classification module of the electrocardiosignal model, and outputting to obtain a prediction category fecgThe classification module is sequentially composed of a full connection layer and a Softmax activation layer;
g) repeating the steps d) to f) N times, and obtaining an optimal electrocardiosignal model after training by using an SGD optimizer through a minimized cross entropy loss function;
h) establishing a heart sound signal model which sequentially consists of a PC module, a TBG module and a classification module;
i) resampling the heart sound signals in the training set and the test set to 8000 sampling points, and then carrying out z-score normalization processing to obtain normalized heart sound signals x'pcg;
j) Normalizing the heart sound signals x 'in the training set'pcgInputting the signal into PC module of heart sound signal model, outputting to obtain characteristic signal Y1The PC module is composed of four convolution branches and a 1 multiplied by 1 convolution block in sequence;
k) the characteristic signal Y1Inputting the signal into a TBG module of the heart sound signal model, and outputting to obtain a characteristic signal Y2The TBG module consists of 4 convolutional coding modules and a bidirectional GRU layer with a TPA mechanism;
l) applying the characteristic signal Y2Input into a classification module of the heart sound signal model,output derived prediction class fpcgThe classification module is sequentially composed of a full connection layer and a Softmax activation layer;
m) repeating the steps j) to l) M times, and obtaining an optimal heart sound signal model after training by minimizing a cross entropy loss function by using an SGD optimizer;
n) manually dividing the data set into a new training set and a new testing set according to the ratio of 4:1, inputting the new training set into the optimal electrocardiosignal model, and outputting through a TBG (tunnel boring gate) module of the optimal electrocardiosignal model to obtain a 64-dimensional characteristic signal X3Inputting the new training set into the optimal heart sound signal model, and outputting a 64-dimensional characteristic signal Y through a TBG module of the optimal heart sound signal model3By the formula PPx=[X3,Y3]Calculating to obtain spliced 128-dimensional feature fusion signal PPx;
o) fusing features into a signal PPxInputting the signals into an XGboost classifier to obtain a feature fusion signal PPxRank the importance score of (2), and select the signal with the top 64 of the importance score rank as the characteristic signal PP1 xSelecting an optimal hyper-parameter by adopting 5-fold cross validation, and training the XGboost classifier by utilizing the optimal hyper-parameter to obtain an optimized XGboost classifier;
p) inputting the new test set into the optimal electrocardiosignal model, and outputting a 64-dimensional characteristic signal X through a TBG (tunnel boring generator) module of the optimal electrocardiosignal model4Inputting the new test set into the optimal heart sound signal model, and outputting a 64-dimensional characteristic signal Y through a TBG module of the optimal heart sound signal model4By the formula PPc=[X4,Y4]Calculating to obtain spliced 128-dimensional feature fusion signal PPc;
q) feature fusion signal PPcInputting the signals into an XGboost classifier to obtain a feature fusion signal PPcRank the importance score of (2), and select the signal with the top 64 of the importance score rank as the characteristic signal PP1 c。
2. The multi-modal data classification method based on deep learning and time series attention mechanism as claimed in claim 1, wherein: in the step a), a sliding window segmentation method is used for expanding the data set, and a five-fold cross validation method is used for dividing the data set into 5 different training sets and test sets.
3. The multi-modal data classification method based on deep learning and time series attention mechanism as claimed in claim 1, wherein: in step c) by the formulaCalculating to obtain a normalized electrocardiosignal x'ecgIn the formula xecgFor training and testing the concentrated ECG signal uecgIs the mean value, σ, of the electrocardiosignalecgIs the variance of the electrocardiosignals.
4. The multi-modal data classification method based on deep learning and time series attention mechanism as claimed in claim 1, wherein the step d) comprises the following steps:
d-1) the first convolution branch comprises a convolution layer with 32 channels, convolution kernel size of 1 × 15 and step size of 1, a batch normalization layer and a ReLU activation layer in sequence, and the electrocardiosignal x 'after normalization in the training set'ecgInputting the signal into the first convolution branch, and outputting to obtain a 32-dimensional characteristic signal E1;
d-2) the second convolution branch comprises a convolution layer with 32 channels, convolution kernel size of 1 × 13 and step length of 1, a batch normalization layer and a ReLU activation layer in sequence, and the electrocardiosignal x 'after normalization in the training set'ecgInputting the signal into a second convolution branch, and outputting to obtain a 32-dimensional characteristic signal E2;
d-3) the third convolution branch comprises a convolution layer with a channel number of 32, a convolution kernel size of 1 x 9 and a step length of 1, a batch normalization layer and a ReLU activation layer in sequence, and the electrocardiosignals x 'after normalization in the training set'ecgInputting the signal into a third convolution branch, and outputting to obtain a 32-dimensional characteristic signal E3;
d-4) the fourth convolution branch comprises a convolution layer with a channel number of 32, a convolution kernel size of 1 x 5 and a step length of 1, a batch normalization layer and a ReLU activation layer in sequence, and the electrocardiosignals x 'after normalization in the training set'ecgInputting the signal into the fourth convolution branch, and outputting to obtain a 32-dimensional characteristic signal E4;
d-5) converting the characteristic signal E1Characteristic signal E2Characteristic signal E3Characteristic signal E4Performing characteristic cascade to obtain a 128-dimensional characteristic signal E ═ E after cascade1,E2,E3,E4];
d-6)1 × 1 convolution block is composed of convolution layers with 16 channels, 1 × 1 convolution kernel size, and 1 step size, and ReLU active layer, and the 128-dimensional characteristic signal E is set to [ E ═ E1,E2,E3,E4]Inputting the signal into a 1 × 1 convolution block, and outputting to obtain a 16-dimensional characteristic signal X1。
5. The method for multi-modal data classification based on deep learning and temporal attention mechanism according to claim 1, wherein step e) comprises the following steps:
e-1) the first convolutional coding module consists of a convolutional layer with the number of channels of 32 and the convolutional kernel size of 1 multiplied by 11, a batch normalization layer, a ReLU activation layer and a pooling layer with the size of 4 in sequence, and the characteristic signal X is converted into a characteristic signal1Inputting the signal into a first convolution coding module, and outputting to obtain a 32-dimensional characteristic signal E5;
E-2) the second convolutional coding module sequentially comprises a convolutional layer with the channel number of 64 and the convolutional kernel size of 1 multiplied by 7, a batch normalization layer, a ReLU activation layer and a pooling layer with the size of 2, and a characteristic signal E is obtained5Inputting the signal into a second convolutional coding module, and outputting to obtain a 64-dimensional characteristic signal E6;
E-3) the third convolution coding module consists of a convolution layer with the channel number of 128 and the convolution kernel size of 3, a batch normalization layer, a ReLU activation layer and a pooling layer with the size of 2 in sequence, and a characteristic signal E is generated6Inputting the signal into a third convolutional coding module, and outputting to obtain a 128-dimensional characteristic signal E7;
E-4) converting the characteristic signal E7Inputting into 32-unit bidirectional GRU layer with TPA mechanism, and outputting to obtain 64-dimensional characteristic signal X2In the bidirectional GRU layer of TPA mechanism by formulaCalculating to obtain a characteristic signal X2Where i ═ 1, 2.., n }, n ═ 128, T is transposition, τ isiFor the attention weight of the ith row vector,sigma (-) is a sigmoid function,is a time pattern matrix GCLine i of (1), GCConv1d (G), Conv1d (·) is a one-dimensional convolution operation, G is a hidden state matrix,gifor the hidden state vector of the ith bidirectional GRU, i ═ 1,2, akIs a weight coefficient, gtIs the hidden state vector of the bi-directional GRU at time t.
6. The method for multi-modal data classification based on deep learning and temporal attention mechanism according to claim 1, characterized in that: in the step g), the value of N is 150, the learning rate of the SGD optimizer is 0.001, the learning rate is attenuated to be 0.1 at every 80 periods, and the formula is usedCalculating to obtain a cross entropy loss function cc (x), wherein L is the number of categories, L is 2, fi(x) To predict class fecgThe predictive tag of the ith category of (a),to predict class fecgThe real category of the corresponding ith category; in the step m), the value of N is 180, the learning rate of the SGD optimizer is 0.001, the learning rate is attenuated to be 0.1 at every 90 periods, and the N is calculated according to a formulaCalculating to obtain a cross entropy loss function cc (y), wherein L is the number of categories, L is 2, fi(y) is prediction class fpcgThe predictive label of the ith category of (c),as a prediction class fpcgTrue category of the ith category of (1).
7. The method for multi-modal data classification based on deep learning and temporal attention mechanism according to claim 1, characterized in that: in step i) by the formulaCalculating to obtain a normalized heart sound signal x'pcgIn the formula xpcgFor the heart sound signals in the training set and test set, upcgIs the mean value, σ, of the heart sound signalpcgIs the variance of the heart sound signal.
8. The method for multi-modal data classification based on deep learning and time series attention mechanism as claimed in claim 1, wherein step j) comprises the following steps:
j-1) the first convolution branch is composed of convolution layer with 32 channels, convolution kernel size of 1 × 15 and step size of 2, batch normalization layer and ReLU activation layer in sequence, and the heart sound signal x 'after normalization in training set'pcgInputting the signal into the first convolution branch, and outputting to obtain a 32-dimensional characteristic signal P1;
j-2) the second convolution branch is composed of convolution layer with 32 channel number, convolution kernel size of 1 × 11 and step length of 2, batch normalization layer and ReLU activation layer in sequence, and the training is performedCentralizing normalized heart sound signal x'pcgInputting the signal into the second convolution branch, and outputting to obtain a 32-dimensional characteristic signal P2;
j-3) the third convolution branch comprises a convolution layer with a channel number of 32, a convolution kernel size of 1 × 9 and a step size of 2, a batch normalization layer and a ReLU activation layer in sequence, and the heart sound signal x 'after normalization in the training set'pcgInputting the signal into a third convolution branch, and outputting to obtain a 32-dimensional characteristic signal P3;
j-4) the fourth convolution branch comprises a convolution layer with 32 channels, convolution kernel size of 1 × 5 and step size of 2, a batch normalization layer and a ReLU active layer in sequence, and the heart sound signal x 'after normalization in the training set'pcgInputting the signal into the fourth convolution branch, and outputting to obtain a 32-dimensional characteristic signal P4;
j-5) converting the characteristic signal P1Characteristic signal P2Characteristic signal P3Characteristic signal P4Performing feature cascade to obtain a cascaded 128-dimensional feature signal P ═ P1,P2,P3,P4];
j-6)1 × 1 convolution block is composed of convolution layer with 32 channel number, convolution kernel size 1 × 1 and step size 1 and ReLU active layer, and 128-dimensional characteristic signal P is [ P ═ P1,P2,P3,P4]Inputting into a 1 × 1 convolution block, and outputting to obtain a 32-dimensional characteristic signal Y1。
9. The multi-modal data classification method based on deep learning and time series attention mechanism as claimed in claim 1, wherein step k) comprises the following steps:
k-1) the first convolutional coding module sequentially comprises a convolutional layer with the number of channels being 16 and the convolutional kernel size being 1 multiplied by 1, a batch normalization layer, a ReLU activation layer and a pooling layer with the size being 4, and the feature signal Y is processed1Inputting the signals into a first convolutional encoding module, and outputting to obtain a 16-dimensional characteristic signal P5;
k-2) the second convolutional encoding module sequentially comprises a convolutional layer with the number of channels being 32 and the size of convolutional kernel being 1 × 11, a batch normalization layer and a ReLUAn active layer, a pooling layer of size 2, and a feature signal P5Inputting the signal into a second convolutional coding module, and outputting to obtain a 32-dimensional characteristic signal P6;
k-3) the third convolutional coding module sequentially comprises a convolutional layer with the channel number of 64 and the convolutional kernel size of 1 multiplied by 7, a batch normalization layer, a ReLU activation layer and a pooling layer with the size of 2, and a characteristic signal P is obtained6Inputting the signal into a third convolutional coding module, and outputting to obtain a 64-dimensional characteristic signal P7;
k-4) the fourth convolutional coding module sequentially comprises a convolutional layer with the channel number of 128 and the convolutional kernel size of 1 multiplied by 3, a batch normalization layer, a ReLU active layer and a pooling layer with the size of 2, and a characteristic signal P is obtained7Inputting the signal into a fourth convolutional coding module, and outputting to obtain a 128-dimensional characteristic signal P8;
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210376944.3A CN114764575B (en) | 2022-04-11 | 2022-04-11 | Multi-modal data classification method based on deep learning and time sequence attention mechanism |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210376944.3A CN114764575B (en) | 2022-04-11 | 2022-04-11 | Multi-modal data classification method based on deep learning and time sequence attention mechanism |
Publications (2)
Publication Number | Publication Date |
---|---|
CN114764575A true CN114764575A (en) | 2022-07-19 |
CN114764575B CN114764575B (en) | 2023-02-28 |
Family
ID=82364741
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210376944.3A Active CN114764575B (en) | 2022-04-11 | 2022-04-11 | Multi-modal data classification method based on deep learning and time sequence attention mechanism |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114764575B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116186593A (en) * | 2023-03-10 | 2023-05-30 | 山东省人工智能研究院 | Electrocardiosignal detection method based on separable convolution and attention mechanism |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2018130541A (en) * | 2017-02-16 | 2018-08-23 | タタ コンサルタンシー サービシズ リミテッドTATA Consultancy Services Limited | Method and system for detection of coronary artery disease in person using fusion approach |
CN110236518A (en) * | 2019-04-02 | 2019-09-17 | 武汉大学 | The method and device of electrocardio and the heart shake combined signal classification neural network based |
CN110537910A (en) * | 2019-09-18 | 2019-12-06 | 山东大学 | coronary heart disease noninvasive screening system based on electrocardio and heart sound signal joint analysis |
CN113288163A (en) * | 2021-06-04 | 2021-08-24 | 浙江理工大学 | Multi-feature fusion electrocardiosignal classification model modeling method based on attention mechanism |
CN113855063A (en) * | 2021-10-21 | 2021-12-31 | 华中科技大学 | Heart sound automatic diagnosis system based on deep learning |
CN114190952A (en) * | 2021-12-01 | 2022-03-18 | 山东省人工智能研究院 | 12-lead electrocardiosignal multi-label classification method based on lead grouping |
-
2022
- 2022-04-11 CN CN202210376944.3A patent/CN114764575B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2018130541A (en) * | 2017-02-16 | 2018-08-23 | タタ コンサルタンシー サービシズ リミテッドTATA Consultancy Services Limited | Method and system for detection of coronary artery disease in person using fusion approach |
CN110236518A (en) * | 2019-04-02 | 2019-09-17 | 武汉大学 | The method and device of electrocardio and the heart shake combined signal classification neural network based |
CN110537910A (en) * | 2019-09-18 | 2019-12-06 | 山东大学 | coronary heart disease noninvasive screening system based on electrocardio and heart sound signal joint analysis |
CN113288163A (en) * | 2021-06-04 | 2021-08-24 | 浙江理工大学 | Multi-feature fusion electrocardiosignal classification model modeling method based on attention mechanism |
CN113855063A (en) * | 2021-10-21 | 2021-12-31 | 华中科技大学 | Heart sound automatic diagnosis system based on deep learning |
CN114190952A (en) * | 2021-12-01 | 2022-03-18 | 山东省人工智能研究院 | 12-lead electrocardiosignal multi-label classification method based on lead grouping |
Non-Patent Citations (2)
Title |
---|
HANLI, ET AL.: "Integrating multi-domain deep features of electrocardiogram and phonocardiogram for coronary artery disease detection", 《COMPUTERS IN BIOLOGY AND MEDICINE》 * |
李俊杰 等: "基于心音心电信号的心衰分析系统", 《SOFTWARE ENGINEERING AND APPLICATIONS》 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116186593A (en) * | 2023-03-10 | 2023-05-30 | 山东省人工智能研究院 | Electrocardiosignal detection method based on separable convolution and attention mechanism |
CN116186593B (en) * | 2023-03-10 | 2023-10-03 | 山东省人工智能研究院 | Electrocardiosignal detection method based on separable convolution and attention mechanism |
Also Published As
Publication number | Publication date |
---|---|
CN114764575B (en) | 2023-02-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN112766355B (en) | Electroencephalogram signal emotion recognition method under label noise | |
CN112784730B (en) | Multi-modal emotion recognition method based on time domain convolutional network | |
CN113033309A (en) | Fault diagnosis method based on signal downsampling and one-dimensional convolution neural network | |
CN110390952A (en) | City sound event classification method based on bicharacteristic 2-DenseNet parallel connection | |
CN112294341B (en) | Sleep electroencephalogram spindle wave identification method and system based on light convolutional neural network | |
CN114176607B (en) | Electroencephalogram signal classification method based on vision transducer | |
CN112732921B (en) | False user comment detection method and system | |
CN105095863A (en) | Similarity-weight-semi-supervised-dictionary-learning-based human behavior identification method | |
CN111582041A (en) | Electroencephalogram identification method based on CWT and MLMSFFCNN | |
CN114564990A (en) | Electroencephalogram signal classification method based on multi-channel feedback capsule network | |
CN110909928A (en) | Energy load short-term prediction method and device, computer equipment and storage medium | |
CN114764575B (en) | Multi-modal data classification method based on deep learning and time sequence attention mechanism | |
CN113069117A (en) | Electroencephalogram emotion recognition method and system based on time convolution neural network | |
CN114595725B (en) | Electroencephalogram signal classification method based on addition network and supervised contrast learning | |
CN114722950B (en) | Multi-mode multi-variable time sequence automatic classification method and device | |
Tripathi et al. | Multivariate time series classification with an attention-based multivariate convolutional neural network | |
CN115965864A (en) | Lightweight attention mechanism network for crop disease identification | |
CN116186593A (en) | Electrocardiosignal detection method based on separable convolution and attention mechanism | |
CN114863572A (en) | Myoelectric gesture recognition method of multi-channel heterogeneous sensor | |
Ayache et al. | Speech command recognition using deep learning | |
Cho et al. | Fruit ripeness prediction based on DNN feature induction from sparse dataset | |
Tlaisun et al. | Efficient deep learning approach for modern skin cancer detection | |
CN117216522A (en) | Bearing fault type diagnosis method, device and equipment based on machine learning | |
CN116258914B (en) | Remote Sensing Image Classification Method Based on Machine Learning and Local and Global Feature Fusion | |
CN111368686B (en) | Electroencephalogram emotion classification method based on deep learning |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |