CN112329609A - Feature fusion transfer learning arrhythmia classification system based on 2D heart beat - Google Patents

Feature fusion transfer learning arrhythmia classification system based on 2D heart beat Download PDF

Info

Publication number
CN112329609A
CN112329609A CN202011210145.6A CN202011210145A CN112329609A CN 112329609 A CN112329609 A CN 112329609A CN 202011210145 A CN202011210145 A CN 202011210145A CN 112329609 A CN112329609 A CN 112329609A
Authority
CN
China
Prior art keywords
dimensional
layer
arrhythmia classification
frequency
feature fusion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011210145.6A
Other languages
Chinese (zh)
Inventor
张亚涛
张锋
李向宇
鲍喆
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University
Original Assignee
Shandong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University filed Critical Shandong University
Priority to CN202011210145.6A priority Critical patent/CN112329609A/en
Publication of CN112329609A publication Critical patent/CN112329609A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08Feature extraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/25Fusion techniques
    • G06F18/253Fusion techniques of extracted features
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/02Preprocessing
    • G06F2218/04Denoising
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/12Classification; Matching

Abstract

The application discloses characteristic fusion transfer learning arrhythmia classification system based on 2D heart beat includes: an acquisition module configured to: acquiring a target electrocardiosignal; a reconstruction module configured to: reconstructing a target electrocardiosignal from one dimension into a two-dimensional electrocardiosignal; a feature extraction module configured to: respectively extracting a first characteristic and a second characteristic from the two-dimensional electrocardiosignals; a feature fusion module configured to: fusing the first characteristic and the second characteristic; a classification module configured to: and inputting the fused features into a trained classifier, and outputting arrhythmia classification results corresponding to the current target electrocardiosignals.

Description

Feature fusion transfer learning arrhythmia classification system based on 2D heart beat
Technical Field
The application relates to the technical field of a heart rate duration classification system, in particular to a feature fusion transfer learning arrhythmia classification system based on 2D heart beat.
Background
The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.
The traditional arrhythmia classification methods mainly comprise two types, one is a feature extraction and machine learning method, and the other is a deep learning method without feature extraction.
The machine learning method based on the feature extraction comprises the following steps: the traditional machine learning algorithm needs to artificially extract various features including time domain, frequency domain and the like according to prior experience, and then trains a proper machine learning classifier, so that the quality of the extracted features affects the final classification result. Thaweesak et al, by using two-stage classification, study the performance of SVM with fixed input feature dimension and free selection, and the method has the defect that the feature dimension must be fixed, which limits the feature expansion, resulting in less extracted features and affecting the final classification precision. Nasiri j.a. et al extracted 22 features from the electrocardiographic signals, combined the genetic algorithm with the support vector machine, enhanced the generalization performance of the model to a certain extent, the features extracted by the method are too few to reflect the comprehensive information in the signals, making the accuracy difficult to further improve. Each of these two methods can identify three and four types of arrhythmias, but extract fewer features. Ozcift et al propose a method for training RFs based on random forests using a data resampling strategy. In addition, variants of random forests are also used for electrocardiogram classification. Such as oblique random forests (ORAF), are also used for quality assessment of electrocardiograms. Random forests have some interpretability in the machine learning classification problem, but their accuracy and generalization ability are still limited by the quality of the proposed features.
It can be seen that the traditional machine learning algorithm relies on prior knowledge in the early stage of feature extraction. The method has strict requirements on the extracted features, the design of the classifier is very limited by the reflecting capacity of the feature extraction on the intrinsic properties and contained information of the electrocardiosignals, the performance of the heart beat classification also depends on the correctness and the authenticity of feature selection, and the machine learning classification method of artificial feature selection has limited generalization capacity. Therefore, the traditional manual feature extraction and machine learning classification method is difficult to be applied to complex and changeable clinical data and application.
Based on a deep learning method: at present, the advantage of deep learning is that the original electrocardiosignals after pretreatment are directly classified to obtain results without manually extracting prior characteristics. At present, most of the classical deep learning-based methods in arrhythmia classification are based on original one-dimensional electrocardiogram data. Kiranyaz et al obtained excellent performance with a one-dimensional deep convolutional neural network (1-D CNN) in classifying and identifying ventricular ectopic beats and supraventricular ectopic beats in the arrhythmia reference database MIT-BIH arrhythmia benchmark database. Rajendra Acharya et al also used a one-dimensional deep convolutional neural network to identify 15 arrhythmias in MIT arrhythmia database MIT-BIH arrhythmia database, and compared with other 13 arrhythmia classification methods by machine learning, the method has higher relative classification precision. Sayantan G et al classify electrocardiogram beats using deep confidence networks and active learning, but this method only learns the feature representation of the ECG using a one-dimensional gaussian-bernoulli deep belief network. Shi et al propose a new multi-input deep learning network and use it for atrial fibrillation detection, however this active and migratory learning based approach still uses one-dimensional electrocardiographic data. In fact, because the acquired electrocardiographic data is in a one-dimensional form, the 1-D CNN method has the advantages of high operation speed, simple structure and the like when being used for classifying the arrhythmia of the electrocardiographic signal, but the classification precision of the 1-D CNN method is difficult to further improve because the information contained in the one-dimensional data is limited and the whole information of the physiological change of the heart contained in the data cannot be comprehensively reflected. Mashrur et al convert one-dimensional electrocardiosignals into a two-dimensional time-frequency graph by wavelet transform and learn and identify atrial fibrillation by using an AlexNet convolutional neural network, however, the signals obtained by wavelet transform are still narrow-band signals, and single frequency information cannot be accurately obtained.
Disclosure of Invention
In order to overcome the defects of the prior art, the application provides a 2D heartbeat-based feature fusion transfer learning arrhythmia classification system;
in a first aspect, the application provides a 2D heartbeat-based feature fusion and migration learning arrhythmia classification system;
feature fusion transfer learning arrhythmia classification system based on 2D heartbeat includes:
an acquisition module configured to: acquiring a target electrocardiosignal;
a reconstruction module configured to: reconstructing a target electrocardiosignal from one dimension into a two-dimensional electrocardiosignal;
a feature extraction module configured to: respectively extracting a first characteristic and a second characteristic from the two-dimensional electrocardiosignals;
a feature fusion module configured to: fusing the first characteristic and the second characteristic;
a classification module configured to: and inputting the fused features into a trained classifier, and outputting arrhythmia classification results corresponding to the current target electrocardiosignals.
In a second aspect, the present application further provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, the one or more computer programs are stored in the memory, and when the electronic device runs, the processor executes the one or more computer programs stored in the memory, so that the electronic device performs the functions of the system according to the first aspect.
In a third aspect, the present application further provides a computer-readable storage medium for storing computer instructions, which when executed by a processor, perform the functions of the system of the first aspect.
Compared with the prior art, the beneficial effects of this application are:
(1) a migration learning network architecture based on feature fusion of a 101-layer residual error network ResNet-101 is provided, the network architecture adopts two feature extraction layer channels, one is a ResNet-101 feature extraction layer pre-trained by ImageNet, the network architecture has the function of acquiring general image features in a two-dimensional heart beating time-frequency diagram, the other is an 8-layer CNN feature extraction layer trained by a two-dimensional ECG time-frequency analysis diagram, the network architecture has the function of acquiring special domain features of ECG in the two-dimensional ECG time-frequency diagram, the special domain features are general time-frequency features common to image textures, shapes, colors and the like compared with general image features extracted by ResNet-101 trained by an ImageNet image library, due to the limitation of the ImageNet image library (images in the library are natural scene images, such as plants, animals and the like), and the two-dimensional ECG analysis diagram is a spectrum image different from natural scenes, therefore, the features extracted by the 8CNN feature extraction layers are different from the common image features, and can reflect the electrocardio time-frequency features of electrocardio energy and frequency distribution. Through the two network feature extraction layers, the general image features and special domain features of the electrocardio are obtained, so that the essence of the signal is comprehensively reflected, and the precision is improved;
(2) the invention discloses a time-frequency analysis method combining Hilbert-Huang Transform (HHT) and Wigner-Ville Distribution (WVD), namely HHT-WVD time-frequency analysis. According to the method, the one-dimensional electrocardiosignals are reconstructed into the two-dimensional electrocardiosignals, the two-dimensional data form containing more information than the one-dimensional data can be obtained, the change of the dynamic state of the heart is reflected more comprehensively, and the method is beneficial to mining the relation contained in the data in the electrocardio data, so that the information is deeply mined, and the purpose of improving the classification precision is achieved. Compared with other time-frequency analysis methods, the method combines the advantages of HHT and WVD time-frequency analysis methods, is more suitable for analyzing non-stationary random signals, reduces cross item interference to a certain extent, ensures time-frequency focusing, and reflects the instantaneous frequency of the signals.
Advantages of additional aspects of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application.
FIG. 1 is a system architecture diagram of a first embodiment;
FIG. 2 is a flowchart of a wavelet denoising transformation algorithm according to a first embodiment;
FIG. 3(a) is an example of triple median filtering baseline wander removal prior to wavelet de-noising;
FIG. 3(b) is an example of triple median filtering baseline wander removal before denoising after wavelet denoising;
FIG. 4(a) is an example of cubic median filtering baseline wander removal before wavelet de-noising;
FIG. 4(b) is an example of triple median filtering baseline wander removal before denoising after wavelet denoising;
FIG. 5 is R-wave positioning;
6(a) -6 (e) are examples of extracted AAMI standard five-type heartbeats;
7(a) -7 (e) are two-dimensional time-frequency diagrams of one-dimensional heart beat data in five types of heart beat categories reconstructed by adopting a HHT-WVD method;
FIG. 8 is a ResNet-101+ CNN feature fusion network model;
FIG. 9 is a model of a feature fusion and migration learning network of ResNet-101+ CNN;
FIG. 10 is a HHT-WVD signal reconstruction scheme.
Detailed Description
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise, and it should be understood that the terms "comprises" and "comprising", and any variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The embodiments and features of the embodiments in the present application may be combined with each other without conflict.
Interpretation of terms: one-hot encoding: i.e., one-bit-efficient encoding, essentially employs an N-bit status register to encode N states, each state being represented by its own independent register bit and only one bit being active at any one time. Confusion matrix: each column of the confusion matrix represents a prediction category, the total number of each column representing the number of data predicted for that category; each row represents a true attribution category of data, and the total number of data in each row represents the number of data instances for that category. The values in each column represent the number of classes for which real data is predicted.
According to the method, the one-dimensional electrocardiosignals are converted into the heart beat two-dimensional time-frequency diagram by the HHT-WVD, and a new ResNet-101+8CNN fusion characteristic transfer learning network structure is designed on the basis of the ResNet-101 model, so that the accuracy of electrocardiosignal classification is improved. In order to achieve the above purpose, the detailed technical solution of the present application is shown in fig. 1.
Example one
The embodiment provides a 2D heartbeat-based feature fusion transfer learning arrhythmia classification system;
feature fusion transfer learning arrhythmia classification system based on 2D heartbeat includes:
an acquisition module configured to: acquiring a target electrocardiosignal;
a reconstruction module configured to: reconstructing a target electrocardiosignal from one dimension into a two-dimensional electrocardiosignal;
a feature extraction module configured to: respectively extracting a first characteristic and a second characteristic from the two-dimensional electrocardiosignals;
a feature fusion module configured to: fusing the first characteristic and the second characteristic;
a classification module configured to: and inputting the fused features into a trained classifier, and outputting arrhythmia classification results corresponding to the current target electrocardiosignals.
As one or more embodiments, after the obtaining module and before the reconstructing module, the method further includes:
a preprocessing module; the preprocessing module comprises a denoising unit, an R wave detection unit and a heartbeat extraction unit which are sequentially connected;
the denoising unit configured to: denoising the acquired target electrocardiosignal;
the R-wave detection unit configured to: performing R wave positioning on the denoised target electrocardiosignals;
the heartbeat extraction unit configured to: based on the R wave positioning point, intercepting M points to obtain a heart beat; and obtaining a plurality of heart beats based on a plurality of R wave positioning points.
Further, the denoising unit specifically includes:
a wavelet decomposition subunit configured to perform wavelet decomposition on the target electrocardiographic signal based on the set number of decomposition layers;
a threshold quantization subunit configured to: selecting a soft threshold or a hard threshold to carry out quantization processing on the high-frequency coefficients from the first layer to the P layer; and P is a positive integer greater than 1.
A one-dimensional wavelet reconstruction subunit configured to: and performing one-dimensional wavelet reconstruction according to the P-th low-frequency coefficient of the wavelet decomposition and the first-layer to P-th high-frequency coefficients.
Illustratively, the denoising unit includes: and filtering and denoising the electrocardio data of the MIH-BIH arrhythmia database by using wavelet transformation. The wavelet transform denoising algorithm flow is shown in fig. 2.
The wavelet denoising method comprises the following specific steps:
(1) preprocessing the electrocardiosignals containing the noise and performing wavelet decomposition. The wavelet is selected and the number of decomposed layers N is determined, and then the signal s is subjected to N-layer decomposition.
(2) Threshold quantization of the high frequency coefficients of the wavelet decomposition. And selecting soft threshold or hard threshold quantization processing for the high-frequency coefficients of the first layer to the Nth layer.
(3) And (5) one-dimensional wavelet reconstruction. And performing one-dimensional reconstruction according to the low-frequency coefficient of the Nth layer of the wavelet decomposition and the high-frequency coefficients from the first layer to the Nth layer.
The key point of the wavelet transformation denoising method lies in how to select a threshold and determine a wavelet basis, db5 is adopted as a wavelet function to carry out three-layer decomposition, and a Stein unbiased likelihood is used for estimating the threshold.
The invention also adopts three times of median filtering to correct the electrocardio baseline and further removes low-frequency noise such as baseline drift and the like.
FIG. 3(a) is an example of triple median filtering baseline wander removal prior to wavelet de-noising; fig. 3(b) is an example of removing baseline wander by cubic median filtering before denoising after wavelet denoising.
FIG. 4(a) is an example of cubic median filtering baseline wander removal before wavelet de-noising; fig. 4(b) is an example of removing baseline wander by cubic median filtering before denoising after wavelet denoising.
Furthermore, the R wave detection unit positions the R wave by adopting a Pan-Tompkins algorithm.
Illustratively, the R wave detection unit has more successful algorithms for positioning and detecting the electrocardio R wave, such as a classic Pan-Tompkins algorithm, a wavelet transform method and the like. The method is used for detecting and positioning the R wave by using a classical Pan-Tompkins algorithm. The R-wave localization is shown in fig. 5.
Further, the heartbeat extraction unit respectively takes M1 points to the left and M2 points to the right based on the R wave positioning points, and the total M points are taken.
For example, the heartbeat extracting unit needs to intercept all heartbeats in each record after the R-wave positioning is completed. In the application, 100 and 150 points are respectively taken to the left and the right by taking the position of the R wave as a center, and the total number is 250 points. The basis for this is that a complete heartbeat lasts about 0.70s, and the sampling rate of the electrocardiosignals of the MIT-BIH arrhythmia database is 360Hz, so the application takes the R wave as the center to intercept 250 points as a heartbeat. The data may have different dimensions, larger range, or self-variation, which may increase the error in classification, so the invention also centers each heartbeat. Each heartbeat after the centering process is a single sample for training and testing the CNN model.
As one or more embodiments, the reconstruction module is specifically configured to:
firstly, reconstructing a target electrocardiosignal into a new analytic signal by adopting Hilbert-Huang transform (HHT);
and processing the new analytic signal by adopting WVD conversion to obtain a two-dimensional time-frequency diagram.
Illustratively, the reconstruction module reconstructs the one-dimensional electrocardio sequence into a two-dimensional time-frequency graph by using a HHT-WVD-based time-frequency analysis method so as to facilitate deep learning development analysis. Firstly, reconstructing the one-dimensional electrocardiosignal into a new analytic signal by HHT (Hilbert-Huang transform), and then analyzing the new analytic signal by WVD on the basis, thereby obtaining a two-dimensional time-frequency diagram.
The WVD method and HHT principles are as follows: the WV distribution is bilinear time-frequency distribution, can be used for nonlinear signal analysis, and can reflect the energy distribution of signals in a time-frequency domain. It is defined as follows
Figure BDA0002758556060000091
Where x (t) is a one-dimensional signal and x is a complex conjugate.
The HHT transform is a newer time-frequency analysis tool based on empirical mode decomposition, which is independent of the fourier transform. The HHT decomposes an original signal into a plurality of eigenmode functions by empirical mode decomposition, wherein each component of the eigenmode functions reflects the internal structure of the signal. The components are then reconstructed into an analytic signal using a hilbert transform. HHT is described as follows:
Figure BDA0002758556060000092
where x (t) is the original signal, k is the number of decomposed eigenmodes, IMFi(t) is the i-th component of the empirical mode, rkAre residual terms.
The component of each natural mode can be converted into the following formula
Figure BDA0002758556060000101
Where p and v are fractional principal values. Then, an analytic signal Z (t) can be reconstructed by Hilbert transform, as shown in the following formula
Figure BDA0002758556060000102
Where p and v are fractional principal values.
7(a) -7 (e) are two-dimensional time-frequency graphs of one-dimensional heart beat data in the five types of heart beat categories reconstructed by adopting the HHT-WVD method.
As one or more embodiments, the feature extraction module specifically includes a first network and a second network that are parallel to each other; the first network is ResNet-101, and the second network is CNN;
the ResNet-101 is obtained by pre-training an image database ImageNet;
the CNN is obtained by two-dimensional time-frequency graph training.
The ResNet-101 is used for extracting image characteristics of the two-dimensional electrocardio time-frequency diagram;
the CNN is used for extracting the electrocardio characteristics of the two-dimensional electrocardio.
Further, the specific structure of the CNN includes:
the device comprises an input layer, a first coiling layer, a first pooling layer, a second coiling layer, a second pooling layer, a third coiling layer, a third pooling layer, a fourth coiling layer, a fourth pooling layer, a full-connection layer and an output layer which are connected in sequence.
As one or more embodiments, the feature fusion module is specifically configured to: and performing splicing fusion processing or weighted summation fusion processing on the first characteristic and the second characteristic.
As one or more embodiments, a trained classifier of the classification module; the specific training process comprises the following steps:
constructing an arrhythmia classification model;
the arrhythmia classification model comprises: the system comprises a feature extraction module, a feature fusion module, a full connection layer and a Softmax classifier which are connected in sequence;
constructing a training set and a testing set, wherein the training set and the testing set are two-dimensional heart beat time-frequency graphs of known arrhythmia classification results;
and inputting the training set and the test set into the arrhythmia classification model for training to obtain the trained arrhythmia classification model.
The training set and test set acquisition process includes:
denoising the electrocardiosignals of the known arrhythmia classification result;
performing R-wave positioning processing on the result after denoising processing;
extracting a heart beat based on the R wave positioning result;
performing Hilbert-Huang transform and WVD transform based on the heart beat; obtaining a two-dimensional heart beat time-frequency diagram obtained after transformation;
and performing data equalization processing on the two-dimensional heart beat time-frequency diagram by adopting an up-sampling mode to obtain a training set and a test set.
All heartbeats of the MIT-BIH arrhythmia database were classified into N, V, S, Q and F, according to the AAMI Standard Classification method. The number of the five types of heartbeats is shown in table 1. Obviously, the difference of the number of heart beats is very different. In the experiment, 80% of various data are used as training data, and 20% are used as test data. The application adopts an up-sampling method to expand the V, S, Q, F four types of heart beat quantity to be the same as the N types of heart beat quantity.
TABLE 1 AAMI five-class cardiac beat number, training set, test set and before and after balance
Figure BDA0002758556060000121
Further, the arrhythmia classification result includes: n, V, S, Q, F class.
In the present application, according to the Association for the advancement of medical instrumentation (AAMI) standard classification method, all extracted heartbeats of the MIT-BIH arrhythmia database are classified into N-type (normal or bundle branch block), V-type (ventricular abnormality), S-type (supraventricular abnormality), Q-type (unknown), and F-type (fusion), as shown in fig. 6(a) to 6 (e).
The method converts one-dimensional electrocardiosignals into a two-dimensional heart beat time-frequency diagram as input, is used for training and testing a ResNet-101+ CNN characteristic fusion network model, and expresses 5 types of heart beats by one-hot codes.
1. ResNet-101+ CNN feature fusion network structure design
The feature fusion network model based on the transfer learning is designed based on a residual error network ResNet-101. The network adopts double-channel acquisition characteristics, wherein one characteristic is that a classical residual error network ResNet-101 is used for transfer learning to acquire general universal image characteristics of a two-dimensional heart beat time-frequency diagram, and the second channel is used for acquiring special field characteristics of the two-dimensional heart beat by using 8-layer CNN.
Because the residual error network ResNet-101 model is obtained by pre-training the image database ImageNet, and meanwhile, the parameter migration of the transfer learning is adopted, the ResNet-101 network can obtain the general image characteristics of the two-dimensional electrocardio time-frequency image, and the 8-layer CNN model parameters are obtained by directly training the two-dimensional electrocardio time-frequency image, so that the network can obtain the special field characteristics of the two-dimensional electrocardio. According to the method, a feature fusion layer is added, the feature graphs generated by the two channels are unified and normalized, and then are input into a unified full-connection layer and a SoftMax classification layer, and finally N, V, S, Q, F five types of output are obtained. FIG. 8 shows a ResNet-101+ CNN feature fusion network model of the present application. The structural parameters of the two-dimensional CNN and the parameter settings of the gradient descent method are listed in table 2.
TABLE 2 CNN structural parameters
Figure BDA0002758556060000131
In the application, a migration learning part uses an ImageNet pre-trained ResNet-101 network model, adopts a parameter migration mode, namely randomly initializes a few convolution layers of the network model, uses weight parameters of the pre-trained network model for other convolution layers, and trains network parameters through a data set to be trained.
The feature fusion layer design function in this application is to re-normalize the generic image features generated by ImageNet pre-trained ResNet-101 to a 12 x 12 feature map of the same size as the CNN derived feature map.
Evaluation indexes are as follows:
the performance of the application is evaluated by adopting four indexes of accuracy, positive prediction rate, sensitivity and specificity, which are respectively as follows: acc (curative), ppv (positive predictive value), sen (sensitivity), spec (sensitivity), are defined as
Figure BDA0002758556060000141
Figure BDA0002758556060000142
Figure BDA0002758556060000143
Figure BDA0002758556060000144
Wherein, tp (true positive): actually, the prediction result of the class is the number of samples of the class;
FP (false Positive): actually, the prediction results of other categories are the number of samples of the category;
tn (true newtive): the number of samples that the other class predicts as the other class;
FN (false New terminal): it is the number of samples that this class predicts as the other class.
Meanwhile, the confusion matrix of the classification result of the 5-type heart beat is calculated to evaluate the performance.
TABLE 3 construction of confusion matrix
Figure BDA0002758556060000145
For any element in the matrix
Figure BDA0002758556060000151
Represents the number of samples whose actual class is O (original) and whose predicted result is P (predicted). The diagonal is the number of correct predictions, and the other elements are the number of misjudgments.
Results of the experiment
Transfer learning classification result based on ResNet-101
Table 4 shows the classification results of the invention under 10-fold cross validation on the training set, including the confusion matrix of the prediction results of five types of heartbeats and various indexes. It can be seen that the scheme of the invention shows good performance on acc, ppv, sen and spec of each type of heart beat.
TABLE 4 Heart beat Classification results for 10-fold on training set
Figure BDA0002758556060000152
On the 21715 test set of heart beat, the classification accuracy of the scheme of the application is 96.42%, and 21097 heart beats are correctly classified.
The ResNet-101+ CNN feature fusion transfer learning network architecture comprises the following steps: the neural network model of the feature fusion structure capable of fusing the general image features and the features of the special electrocardio field is provided on the basis of the ResNet-101 residual error network, and the identification accuracy rate in the arrhythmia heartbeat classification reaches 97.15%. The feature fusion network model of the present application is shown in fig. 9.
The ResNet-101+ CNN feature fusion migration learning network model mainly comprises two branches, wherein one branch is a feature extraction layer obtained by parameter migration of a ResNet-101 network pre-trained by ImageNet, and the function of the feature extraction layer is to obtain general image features of two-dimensional heartbeat data; the second branch is a feature extraction layer of the CNN network obtained by directly training the two-dimensional heartbeat data, and aims to obtain special heartbeat features of the two-dimensional heartbeat data. The features obtained by the two branches are normalized by the feature fusion layer to form FeatureMap feature maps with uniform sizes, and then the features are classified by the full connection layer and the SoftMax to finally obtain a classification result.
8-layer CNN feature extraction layer structure: aiming at the input of two-dimensional electrocardio time-frequency data 224 multiplied by 224, a convolution kernel used by a convolution layer of the first layer is 3 multiplied by 3, the depth is 4, all 0 filling is not used, the step length is 1, an activation function uses ReLu, and an output characteristic diagram 4@222 multiplied by 222 is obtained. The second pooling layer uses average pooling, with length and width steps of 2, and an output profile of 4@111 × 111. The third layer of convolutional layers uses the convolutional kernel size of 4 multiplied by 4, does not use all 0 padding, has the step size of 1, outputs the characteristic diagram 8@108 multiplied by 108, and activates the function ReLu. The fourth pooling layer uses average pooling, with length and width steps of 2, and an output profile of 8@54 × 54. The fifth convolutional layer uses a convolutional kernel size of 3 × 3 without all 0 padding, the step size is 1, the output characteristic diagram 4@52 × 52, the activation function ReLu. The sixth pooling layer uses average pooling, the length and width step length are both 2, and a characteristic graph 4@26 × 26 is output; the seventh convolutional layer uses convolution kernel size 3 × 3, does not use all 0 padding, has step size of 1, outputs feature map 4@24 × 24, and activates function ReLu. The eighth pooling layer uses average pooling, with length and width steps of 2, and an output profile of 4@12 × 12.
The signal reconstruction method based on HHT-WVD comprises the following steps: the HHT is used for reconstructing the one-dimensional electrocardiosignal into a new analysis signal, and then the analysis signal is subjected to WVD time-frequency analysis to obtain a two-dimensional time-frequency distribution graph, so that the purposes that the one-dimensional electrocardiosignal is reconstructed into a two-dimensional image and the energy distribution of the two-dimensional image can be reflected are achieved. The HHT-WVD signal reconstruction scheme is shown in fig. 10.
HHT transform is a time-frequency analysis method independent of Fourier transform, which decomposes an original signal into a plurality of empirical mode components through empirical mode decomposition, and reconstructs the components into analytic signals by using Hilbert transform. The HHT transduction allows for a better analysis of the signal, which allows for a clearer time and frequency relationship of the ECG signal, suppressing cross terms to some extent. The WVD method has good time-frequency focusing property, and the time-frequency focusing property is suitable for analyzing ECG signals with energy mainly concentrated near QRS waves. The original signal is firstly reconstructed into an analytic signal by HHT conversion, and then two-dimensional time-frequency distribution of the analytic signal is obtained by WV distribution, so that cross terms influencing precision are restrained, the clearness of the time and frequency change relation of the signal is maintained, the clearness of the relation of signal energy and time and frequency is also ensured, and the time-frequency focusing property is maintained. The finally obtained two-dimensional time-frequency distribution can clearly and accurately reflect the information contained in the signal.
The migration learning architecture can extract general and special characteristics: ResNet-101 is obtained by ImageNet database pre-training, because this database contains many kinds of pictures, therefore the general characteristic of the picture can be extracted in the migration learning by Resnet-101 network after ImageNet data pre-training, and the network structure of this application is on the basis of ResNet-101, have added CNN convolution neural network of 8 layers to carry on the characteristic extraction, this CNN network parameter is trained with the two-dimentional heart beat training set that the arrhythmia database produces completely, therefore what this deep learning network extracted is the specialized domain characteristic that can reflect the information that the heart beat contains. The method and the device fuse general and special image features extracted by the two networks, and then input the common full connection layer, the softmax layer and the output layer to finally obtain a result. The migration learning framework for extracting general and special features can acquire general information of an image and special field information of a two-dimensional heart beat time-frequency image, so that the aim of acquiring comprehensive information is fulfilled. Compared with traditional transfer learning and deep learning, the transfer learning network architecture can combine general and special field characteristics of images, has the advantages of more comprehensive information acquisition, especially acquisition of special field characteristics of electrocardio, and achieves higher classification accuracy.
Technique for reconstructing one-dimensional data into two-dimensional images: the electrocardiosignal is a one-dimensional nonlinear physiological signal, so most deep learning arrhythmia classification based on the neural network at the present stage uses a 1-dimensional convolution kernel. Aiming at the problem that one-dimensional signals cannot reflect the time-frequency relation and energy distribution contained in the signals, the method reconstructs the one-dimensional ECG into a two-dimensional time-frequency diagram by using a HHT-WVD method. Compared with a one-dimensional deep learning analysis method, the method can extract more essential and comprehensive information in the electrocardiosignals. The traditional WVD can reflect the time-frequency characteristics of non-stationary signals and has good time-frequency focusing performance, however, the WVD method is quadratic time-frequency distribution, so that cross terms are inevitably generated, and the change relation of the time and the frequency of the signals is blurred due to the existence of the cross terms. The HHT transform can be frequency-cleaning of the signal, which suppresses cross terms, cleaning the signal energy from the time frequency. The method combines HHT and WVD, fully utilizes the suppression cross terms of the HHT, makes the number-limiting energy and the time frequency become clear, fully utilizes the good time-frequency focusing characteristic of the WVD, and finally obtains the two-dimensional time-frequency graph capable of accurately and clearly reflecting the time-frequency relation and the energy distribution in the signal.
Example two
The present embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; the processor is connected to the memory, the one or more computer programs are stored in the memory, and when the electronic device runs, the processor executes the one or more computer programs stored in the memory, so that the electronic device performs the functions of the system according to the first embodiment.
EXAMPLE III
The present embodiments also provide a computer-readable storage medium for storing computer instructions, which when executed by a processor, perform the functions of the system of the first embodiment.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (10)

1. Feature fusion transfer learning arrhythmia classification system based on 2D heart beat, characterized by includes:
an acquisition module configured to: acquiring a target electrocardiosignal;
a reconstruction module configured to: reconstructing a target electrocardiosignal from one dimension into a two-dimensional electrocardiosignal;
a feature extraction module configured to: respectively extracting a first characteristic and a second characteristic from the two-dimensional electrocardiosignals;
a feature fusion module configured to: fusing the first characteristic and the second characteristic;
a classification module configured to: and inputting the fused features into a trained classifier, and outputting arrhythmia classification results corresponding to the current target electrocardiosignals.
2. The system of claim 1, wherein the acquisition module is followed by the reconstruction module and further comprising:
a preprocessing module; the preprocessing module comprises a denoising unit, an R wave detection unit and a heartbeat extraction unit which are sequentially connected;
the denoising unit configured to: denoising the acquired target electrocardiosignal;
the R-wave detection unit configured to: performing R wave positioning on the denoised target electrocardiosignals;
the heartbeat extraction unit configured to: based on the R wave positioning point, intercepting M points to obtain a heart beat; and obtaining a plurality of heart beats based on a plurality of R wave positioning points.
3. The 2D beat-based feature fusion transfer learning arrhythmia classification system of claim 2,
the denoising unit specifically includes:
a wavelet decomposition subunit configured to perform wavelet decomposition on the target electrocardiographic signal based on the set number of decomposition layers;
a threshold quantization subunit configured to: selecting a soft threshold or a hard threshold to carry out quantization processing on the high-frequency coefficients from the first layer to the P layer; p is a positive integer greater than 1;
a one-dimensional wavelet reconstruction subunit configured to: and performing one-dimensional wavelet reconstruction according to the P-th low-frequency coefficient of the wavelet decomposition and the first-layer to P-th high-frequency coefficients.
4. The 2D beat-based feature fusion transfer learning arrhythmia classification system of claim 2,
the R wave detection unit positions the R wave by adopting a Pan-Tompkins algorithm;
the heart beat extraction unit respectively takes M1 points to the left and M2 points to the right based on R wave positioning points, and the total M points are taken.
5. The 2D beat-based feature fusion transfer learning arrhythmia classification system of claim 1,
the reconstruction module is specifically configured to:
firstly, reconstructing a target electrocardiosignal into a new analytic signal by adopting Hilbert-Huang transform (HHT);
and processing the new analytic signal by adopting WVD conversion to obtain a two-dimensional time-frequency diagram.
6. The 2D beat-based feature fusion transfer learning arrhythmia classification system of claim 1,
the feature extraction module specifically comprises a first network and a second network which are parallel; the first network is ResNet-101, and the second network is CNN;
the ResNet-101 is obtained by pre-training an image database ImageNet;
the CNN is obtained by training a two-dimensional time-frequency diagram;
the ResNet-101 is used for extracting image characteristics of the two-dimensional electrocardio time-frequency diagram;
the CNN is used for extracting the electrocardio characteristics of the two-dimensional electrocardio;
alternatively, the first and second electrodes may be,
the specific structure of the CNN includes:
the device comprises an input layer, a first coiling layer, a first pooling layer, a second coiling layer, a second pooling layer, a third coiling layer, a third pooling layer, a fourth coiling layer, a fourth pooling layer, a full-connection layer and an output layer which are connected in sequence.
7. The 2D beat-based feature fusion transfer learning arrhythmia classification system of claim 1,
a trained classifier of the classification module; the specific training process comprises the following steps:
constructing an arrhythmia classification model;
the arrhythmia classification model comprises: the system comprises a feature extraction module, a feature fusion module, a full connection layer and a Softmax classifier which are connected in sequence;
constructing a training set and a testing set, wherein the training set and the testing set are two-dimensional heart beat time-frequency graphs of known arrhythmia classification results;
and inputting the training set and the test set into the arrhythmia classification model for training to obtain the trained arrhythmia classification model.
8. The 2D beat-based feature fusion transfer learning arrhythmia classification system as claimed in claim 1 wherein the training and test set acquisition process comprises:
denoising the electrocardiosignals of the known arrhythmia classification result;
performing R-wave positioning processing on the result after denoising processing;
extracting a heart beat based on the R wave positioning result;
performing Hilbert-Huang transform and WVD transform based on the heart beat; obtaining a two-dimensional heart beat time-frequency diagram obtained after transformation;
and performing data equalization processing on the two-dimensional heart beat time-frequency diagram by adopting an up-sampling mode to obtain a training set and a test set.
9. An electronic device, comprising: one or more processors, one or more memories, and one or more computer programs; wherein a processor is connected to the memory, the one or more computer programs being stored in the memory, the processor executing the one or more computer programs stored in the memory when the electronic device is running, to cause the electronic device to perform the functions of the system of any one of claims 1-8.
10. A computer readable storage medium storing computer instructions which, when executed by a processor, perform the functions of the system of any one of claims 1 to 8.
CN202011210145.6A 2020-11-03 2020-11-03 Feature fusion transfer learning arrhythmia classification system based on 2D heart beat Pending CN112329609A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011210145.6A CN112329609A (en) 2020-11-03 2020-11-03 Feature fusion transfer learning arrhythmia classification system based on 2D heart beat

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011210145.6A CN112329609A (en) 2020-11-03 2020-11-03 Feature fusion transfer learning arrhythmia classification system based on 2D heart beat

Publications (1)

Publication Number Publication Date
CN112329609A true CN112329609A (en) 2021-02-05

Family

ID=74323556

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011210145.6A Pending CN112329609A (en) 2020-11-03 2020-11-03 Feature fusion transfer learning arrhythmia classification system based on 2D heart beat

Country Status (1)

Country Link
CN (1) CN112329609A (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113057647A (en) * 2021-03-25 2021-07-02 山东省人工智能研究院 Quality evaluation method of electrocardiosignal
CN113080996A (en) * 2021-04-08 2021-07-09 大同千烯科技有限公司 Electrocardiogram analysis method and device based on target detection
CN113171106A (en) * 2021-04-25 2021-07-27 安徽十锎信息科技有限公司 Electrocardio abnormality detection method based on VQ-VAE2 and deep neural network method
CN113495974A (en) * 2021-07-23 2021-10-12 山东新一代信息产业技术研究院有限公司 Sound classification processing method, device, equipment and medium
CN113499081A (en) * 2021-07-06 2021-10-15 中山大学 Method and system for detecting atrial fibrillation of driver based on two-channel deep neural network
CN113520362A (en) * 2021-05-27 2021-10-22 五邑大学 Arrhythmia detection method and device and storage medium
CN113598784A (en) * 2021-08-25 2021-11-05 济南汇医融工科技有限公司 Arrhythmia detection method and system
CN113729723A (en) * 2021-09-18 2021-12-03 浙江工业大学 Electrocardiosignal quality analysis method and device based on ResNet-50 and transfer learning
CN113743259A (en) * 2021-08-23 2021-12-03 吉林大学 Arrhythmia classification method and system based on ResOHEM-net

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107811626A (en) * 2017-09-10 2018-03-20 天津大学 A kind of arrhythmia classification method based on one-dimensional convolutional neural networks and S-transformation
CN108647614A (en) * 2018-04-28 2018-10-12 吉林大学 The recognition methods of electrocardiogram beat classification and system
CN108932452A (en) * 2017-05-22 2018-12-04 中国科学院半导体研究所 Arrhythmia cordis beat classification method based on multiple dimensioned convolutional neural networks
CN108962393A (en) * 2018-05-12 2018-12-07 鲁东大学 Automatic arrhythmia analysis method based on compression figure neural network
CN109303560A (en) * 2018-11-01 2019-02-05 杭州质子科技有限公司 A kind of atrial fibrillation recognition methods of electrocardiosignal in short-term based on convolution residual error network and transfer learning
CN109620211A (en) * 2018-11-01 2019-04-16 吉林大学珠海学院 A kind of intelligent abnormal electrocardiogram aided diagnosis method based on deep learning
CN111329469A (en) * 2020-03-05 2020-06-26 广州天嵌计算机科技有限公司 Arrhythmia prediction method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108932452A (en) * 2017-05-22 2018-12-04 中国科学院半导体研究所 Arrhythmia cordis beat classification method based on multiple dimensioned convolutional neural networks
CN107811626A (en) * 2017-09-10 2018-03-20 天津大学 A kind of arrhythmia classification method based on one-dimensional convolutional neural networks and S-transformation
CN108647614A (en) * 2018-04-28 2018-10-12 吉林大学 The recognition methods of electrocardiogram beat classification and system
CN108962393A (en) * 2018-05-12 2018-12-07 鲁东大学 Automatic arrhythmia analysis method based on compression figure neural network
CN109303560A (en) * 2018-11-01 2019-02-05 杭州质子科技有限公司 A kind of atrial fibrillation recognition methods of electrocardiosignal in short-term based on convolution residual error network and transfer learning
CN109620211A (en) * 2018-11-01 2019-04-16 吉林大学珠海学院 A kind of intelligent abnormal electrocardiogram aided diagnosis method based on deep learning
CN111329469A (en) * 2020-03-05 2020-06-26 广州天嵌计算机科技有限公司 Arrhythmia prediction method

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113057647A (en) * 2021-03-25 2021-07-02 山东省人工智能研究院 Quality evaluation method of electrocardiosignal
CN113080996A (en) * 2021-04-08 2021-07-09 大同千烯科技有限公司 Electrocardiogram analysis method and device based on target detection
CN113171106A (en) * 2021-04-25 2021-07-27 安徽十锎信息科技有限公司 Electrocardio abnormality detection method based on VQ-VAE2 and deep neural network method
CN113520362A (en) * 2021-05-27 2021-10-22 五邑大学 Arrhythmia detection method and device and storage medium
CN113499081A (en) * 2021-07-06 2021-10-15 中山大学 Method and system for detecting atrial fibrillation of driver based on two-channel deep neural network
CN113495974A (en) * 2021-07-23 2021-10-12 山东新一代信息产业技术研究院有限公司 Sound classification processing method, device, equipment and medium
CN113495974B (en) * 2021-07-23 2024-02-02 山东新一代信息产业技术研究院有限公司 Sound classification processing method, device, equipment and medium
CN113743259A (en) * 2021-08-23 2021-12-03 吉林大学 Arrhythmia classification method and system based on ResOHEM-net
CN113598784A (en) * 2021-08-25 2021-11-05 济南汇医融工科技有限公司 Arrhythmia detection method and system
CN113598784B (en) * 2021-08-25 2024-04-09 济南汇医融工科技有限公司 Arrhythmia detection method and system
CN113729723A (en) * 2021-09-18 2021-12-03 浙江工业大学 Electrocardiosignal quality analysis method and device based on ResNet-50 and transfer learning
CN113729723B (en) * 2021-09-18 2024-02-02 浙江工业大学 Electrocardiogram signal quality analysis method and device based on ResNet-50 and transfer learning

Similar Documents

Publication Publication Date Title
CN112329609A (en) Feature fusion transfer learning arrhythmia classification system based on 2D heart beat
CN111657935B (en) Epilepsia electroencephalogram recognition system based on hierarchical graph convolutional neural network, terminal and storage medium
US10869610B2 (en) System and method for identifying cardiac arrhythmias with deep neural networks
CN110840402B (en) Atrial fibrillation signal identification method and system based on machine learning
US11564612B2 (en) Automatic recognition and classification method for electrocardiogram heartbeat based on artificial intelligence
Singh et al. Attention-based convolutional denoising autoencoder for two-lead ECG denoising and arrhythmia classification
CN109498037B (en) Brain cognition measurement method based on deep learning extraction features and multiple dimension reduction algorithm
Farag A self-contained STFT CNN for ECG classification and arrhythmia detection at the edge
CN111370120B (en) Heart diastole dysfunction detection method based on heart sound signals
CN113349793A (en) Arrhythmia classification method based on layered attention model
CN116361688A (en) Multi-mode feature fusion model construction method for automatic classification of electrocardiographic rhythms
Saba-Sadiya et al. EEG channel interpolation using deep encoder-decoder networks
CN117503057A (en) Epileptic seizure detection device and medium for constructing brain network based on high-order tensor decomposition
CN114648048A (en) Electrocardiosignal noise reduction method based on variational self-coding and PixelCNN model
Gopika et al. Performance improvement of deep learning architectures for phonocardiogram signal classification using fast fourier transform
CN117193537A (en) Double-branch convolutional neural network motor imagery intention decoding method based on self-adaptive transfer learning
CN116898451A (en) Method for realizing atrial fibrillation prediction by using neural network with multi-scale attention mechanism
CN115017960B (en) Electroencephalogram signal classification method based on space-time combined MLP network and application
CN116172573A (en) Arrhythmia image classification method based on improved acceptance-ResNet-v 2
JP2023104885A (en) Electrocardiographic heart rate multi-type prediction method based on graph convolution
CN114224354B (en) Arrhythmia classification method, arrhythmia classification device, and readable storage medium
CN116028858A (en) Pre-labeled self-supervision neural network learning heart beat classification method and system
CN113171102B (en) ECG data classification method based on continuous deep learning
CN114041800B (en) Electrocardiosignal real-time classification method and device and readable storage medium
CN114266738A (en) Longitudinal analysis method and system for mild brain injury magnetic resonance image data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20210205

RJ01 Rejection of invention patent application after publication