CN117349600B - Heart sound and heart electricity combined diagnosis method and system based on dual-mode dual input - Google Patents
Heart sound and heart electricity combined diagnosis method and system based on dual-mode dual input Download PDFInfo
- Publication number
- CN117349600B CN117349600B CN202311657353.4A CN202311657353A CN117349600B CN 117349600 B CN117349600 B CN 117349600B CN 202311657353 A CN202311657353 A CN 202311657353A CN 117349600 B CN117349600 B CN 117349600B
- Authority
- CN
- China
- Prior art keywords
- heart sound
- sound signal
- signal segment
- segment
- heart
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000003745 diagnosis Methods 0.000 title claims abstract description 50
- 230000005611 electricity Effects 0.000 title claims description 13
- 230000009977 dual effect Effects 0.000 title abstract description 9
- 230000005236 sound signal Effects 0.000 claims abstract description 230
- 239000013598 vector Substances 0.000 claims abstract description 82
- 230000009466 transformation Effects 0.000 claims abstract description 49
- 230000007246 mechanism Effects 0.000 claims abstract description 39
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 27
- 230000004927 fusion Effects 0.000 claims abstract description 18
- 239000012634 fragment Substances 0.000 claims description 42
- 230000006870 function Effects 0.000 claims description 40
- 238000001914 filtration Methods 0.000 claims description 35
- 238000011176 pooling Methods 0.000 claims description 23
- 238000012545 processing Methods 0.000 claims description 23
- 230000003044 adaptive effect Effects 0.000 claims description 19
- 230000004913 activation Effects 0.000 claims description 17
- 230000000747 cardiac effect Effects 0.000 claims description 16
- 239000011159 matrix material Substances 0.000 claims description 16
- 238000010606 normalization Methods 0.000 claims description 13
- 230000006835 compression Effects 0.000 claims description 12
- 238000007906 compression Methods 0.000 claims description 12
- 230000002159 abnormal effect Effects 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 8
- 239000000284 extract Substances 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 7
- 230000008034 disappearance Effects 0.000 claims description 7
- 238000012937 correction Methods 0.000 claims description 6
- 238000002156 mixing Methods 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 6
- 238000013139 quantization Methods 0.000 claims description 6
- 230000001629 suppression Effects 0.000 claims description 6
- 230000009191 jumping Effects 0.000 claims description 5
- 238000013135 deep learning Methods 0.000 abstract description 5
- 208000002330 Congenital Heart Defects Diseases 0.000 abstract description 3
- 230000005831 heart abnormality Effects 0.000 abstract description 3
- 239000010410 layer Substances 0.000 description 69
- 238000012549 training Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 8
- 238000003062 neural network model Methods 0.000 description 8
- 230000002776 aggregation Effects 0.000 description 7
- 238000004220 aggregation Methods 0.000 description 7
- 238000004590 computer program Methods 0.000 description 7
- 208000029078 coronary artery disease Diseases 0.000 description 7
- 238000009826 distribution Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 7
- 238000012544 monitoring process Methods 0.000 description 7
- 230000001360 synchronised effect Effects 0.000 description 7
- 230000002902 bimodal effect Effects 0.000 description 6
- 230000008859 change Effects 0.000 description 6
- 208000019622 heart disease Diseases 0.000 description 6
- 230000011218 segmentation Effects 0.000 description 6
- 238000007781 pre-processing Methods 0.000 description 5
- 238000012216 screening Methods 0.000 description 5
- 206010044642 Tricuspid valve stenosis Diseases 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000005070 sampling Methods 0.000 description 4
- 208000037656 Respiratory Sounds Diseases 0.000 description 3
- HCHKCACWOHOZIP-UHFFFAOYSA-N Zinc Chemical compound [Zn] HCHKCACWOHOZIP-UHFFFAOYSA-N 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 230000035945 sensitivity Effects 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 206010000891 acute myocardial infarction Diseases 0.000 description 2
- 230000004075 alteration Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 210000001367 artery Anatomy 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000002457 bidirectional effect Effects 0.000 description 2
- HVYWMOMLDIMFJA-DPAQBDIFSA-N cholesterol Chemical compound C1C=C2C[C@@H](O)CC[C@]2(C)[C@@H]2[C@@H]1[C@@H]1CC[C@H]([C@H](C)CCCC(C)C)[C@@]1(C)CC2 HVYWMOMLDIMFJA-DPAQBDIFSA-N 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000035487 diastolic blood pressure Effects 0.000 description 2
- 239000002355 dual-layer Substances 0.000 description 2
- 238000011049 filling Methods 0.000 description 2
- 210000002837 heart atrium Anatomy 0.000 description 2
- 230000035488 systolic blood pressure Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 208000001778 Coronary Occlusion Diseases 0.000 description 1
- 201000000057 Coronary Stenosis Diseases 0.000 description 1
- 206010011086 Coronary artery occlusion Diseases 0.000 description 1
- 206010044565 Tremor Diseases 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 230000004931 aggregating effect Effects 0.000 description 1
- 230000003143 atherosclerotic effect Effects 0.000 description 1
- 230000001746 atrial effect Effects 0.000 description 1
- 238000010009 beating Methods 0.000 description 1
- 230000008033 biological extinction Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 235000012000 cholesterol Nutrition 0.000 description 1
- 238000002586 coronary angiography Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000003595 mist Substances 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 210000004165 myocardium Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 239000002356 single layer Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 210000000591 tricuspid valve Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/318—Heart-related electrical modalities, e.g. electrocardiography [ECG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7253—Details of waveform analysis characterised by using transforms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B7/00—Instruments for auscultation
- A61B7/02—Stethoscopes
- A61B7/04—Electric stethoscopes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
- G06N3/0455—Auto-encoder networks; Encoder-decoder networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0475—Generative networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/02—Preprocessing
- G06F2218/04—Denoising
- G06F2218/06—Denoising by applying a scale-space analysis, e.g. using wavelet analysis
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Public Health (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Veterinary Medicine (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Signal Processing (AREA)
- Pathology (AREA)
- Physiology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Psychiatry (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Acoustics & Sound (AREA)
- Cardiology (AREA)
- Fuzzy Systems (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
- Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
Abstract
The invention discloses a heart sound and electrocardio combined diagnosis method and system based on dual-mode dual input, which are used for synchronously collecting heart sound and electrocardio signals, wherein the heart sound signals are filtered, fused with wavelet transformation of a self-adaptive threshold value and denoising of an countermeasure network based on self-constraint conditions to obtain a second heart sound signal segment, and the electrocardio signals are filtered to obtain a second electrocardio signal segment; the artificial heart sound and electrocardio characteristics utilize a double-layer long-short-term memory network to obtain a second characteristic vector; after the multi-scale wavelet decomposition of the second heart sound signal segment, the second heart sound signal segment and the second heart sound signal segment form a multi-channel heart sound image electrocardiogram segment, and a residual error network of a fusion channel attention mechanism and a spatial attention mechanism is utilized to obtain a third feature vector; and splicing the feature vectors to carry out classified diagnosis. According to the invention, the extracted artificial characteristics of heart sounds and electrocardio multi-domain are comprehensively utilized, the deep learning characteristics are effectively fused, the capability of resisting noise in uncontrolled complex environments is improved, and the robustness and accuracy of heart abnormality diagnosis are improved.
Description
Technical Field
The invention belongs to the field of heart sound and electrocardio monitoring, and particularly relates to a heart sound and electrocardio combined diagnosis system and method based on dual-mode dual input.
Background
The main cause of heart diseases, such as coronary heart disease, is the accumulation of coronary wall plaque, which consists of cholesterol and other deposits of substances in the arteries, which can lead to coronary stenosis or occlusion. In severe cases, the ruptured plaque can completely occlude the artery, thereby causing acute myocardial infarction. At present, coronary angiography is a common and effective method for diagnosing coronary atherosclerotic heart disease (coronary heart disease), is a safer and more reliable invasive diagnosis technology, and is used for diagnosing the golden standard of the coronary heart disease, but requires a professional operation procedure, quite long time and cost, and is difficult to popularize in daily home monitoring. The electrocardio and heart sound respectively reflect the electrical and mechanical activities of the heart, and the electrocardio and heart sound signals are respectively formed by the waveforms of the total potential change and the beating sound of the heart in the processes of removing and repolarizing the myocardium in the heart. Heart sound and electrocardio monitoring and diagnosis are important technical means for effectively screening and tracking follow-up of coronary heart disease. Practice has shown that the use of only one of the electrocardiographic or heart sound signals to detect coronary heart disease is far from adequate, as some patients do not show abnormalities in the electrocardiographic information number but do not show normal heart sound, and vice versa.
Chinese publication No. CN 115177260A, publication No. 2022-10-14, entitled "Artificial neural network-based Intelligent electrocardiograph heart sound diagnosis method and apparatus", discloses an artificial neural network-based method for acquiring electrocardiograph data and heart sound data synchronously acquired by a user to be diagnosed; processing the electrocardio data and the heart sound data to obtain an electrocardio heart sound feature vector, namely the electrocardio heart sound feature vector comprises a first heart sound feature vector, a second heart sound feature vector, a third heart sound feature vector and an electrocardio feature vector; and inputting the electrocardio heart sound feature vector into a diagnosis model to obtain a diagnosis type. The above method fails to clarify the technical connotation of the first heart sound feature vector, the second heart sound feature vector and the third heart sound feature vector, and does not consider the noise influence of the internal physiological change and the external environment change of the monitoring environment.
Chinese publication No. CN 115177262A, publication No. 2022-10-14, entitled "deep learning-based heart sound and electrocardio combined diagnosis device and System", uses CRDNet network model to calculate the classification result corresponding to each equal-length segment according to each equal-length segment obtained by preprocessing heart sound and electrocardio original signals; and forming a diagnosis analysis report according to each classification result. Wherein the crdnaet network model comprises: the system comprises a PCG specific mode encoder, an ECG specific mode encoder, a dense fusion encoder and a collaborative decision-making module, wherein the PCG specific mode encoder and the ECG specific mode encoder have the same structure; each specific modal encoder is used for carrying out step-by-step extraction on the characteristics of the input equal-length fragments, the zeroth level is the space-time characteristics, finally deep characteristics are obtained, classification is carried out based on the deep characteristics, and a modal internal classification result under the current equal-length fragment input is obtained; the dense fusion encoder is used for carrying out step-by-step fusion on the feature correspondence extracted step by two specific mode encoders, in each step fusion process, the contribution of different modes and different areas of the current equal-length fragments of each mode to the classification result is adaptively evaluated, a pixel-level weight graph is generated for the multi-level aggregation feature of each mode, and the multi-level aggregation feature of each mode and the pixel-level weight graph corresponding to the multi-level aggregation feature of each mode are subjected to pixel-level multiplication to obtain the weighted multi-level aggregation feature of the mode; the weighted multi-stage aggregation features of all modes and the fusion features obtained by the previous stage fusion are fused through convolution operation, so that the fusion features of the current stage are generated, and the combined classification result under the input of the current equal-length fragments is obtained; the multi-level aggregation feature of each mode adopted for generating the zero-level fusion feature is the space-time feature corresponding to the mode, and the multi-level aggregation feature of each mode adopted for generating the fusion features of other levels is obtained by aggregating the current-level feature and the previous-level feature extracted by a specific mode encoder of the mode; and the collaborative decision module is used for weighting and adding the intra-modal classification results of the PCG and the ECG and the joint classification result to obtain a final classification result under the input of the current equal-length fragments. The sensitivity (TP/(FP+TN)), the specificity (TN/(FP+TN)), and the average accuracy (average of sensitivity and specificity) of the heart sound and heart electricity combined diagnosis method respectively reach 94.85%,93.97% and 94.41%, wherein TP is the number of true positive bars, TN is the number of true negative bars, FP is the number of false positive bars, and FN is the number of false negative bars by adopting a "Training-a" subset of PhysioNet/CinC Change 2016.
Chinese publication No. CN115640507a, publication No. 2023-1-24, entitled "abnormal constant screening method based on electrocardiographic and heart sound combined analysis", includes: respectively constructing an electrocardiographic segmentation model and a heart sound segmentation model, and judging the state of each frame of electrocardiographic data and heart sound data by using the segmentation models; respectively inputting the synchronous electrocardiograph data and heart sound data segmentation results into corresponding confidence modules, and taking the synchronous electrocardiograph data and heart sound data with the highest confidence as effective signal fragments; decoding the effective signal segment, and calculating the electrocardio-heart sound combined discrete feature by using the decoded effective signal segment; and fusing synchronous electrocardio data, heart sound data and electrocardio and heart sound combined discrete characteristics in the decoded effective signal segment, inputting the fused data into a priori distribution network, inputting output data of the priori distribution network into a screening module, and screening abnormal electrocardio and heart sound data.
Chinese publication No. CN116019480A, publication No. 2023-4-28, entitled "method and apparatus for identifying tricuspid valve stenosis by fusion of time sequence characteristics of heart sounds and electrocardiosignals", mainly comprises the following steps: step 1, synchronously collecting a heart sound image and an electrocardiogram of a detected person, wherein the sampling frequency is set to be 1000Hz; step 2, carrying out characteristic positioning on the electrocardiosignals based on the electrocardiograms acquired in the step 1, and determining peak tip positions of the electrocardiosignals, namely R waves, T waves and P waves; step 3, performing characteristic positioning on the heart sound signals based on the heart sound images acquired in the step 1, and determining peak tip positions of the first heart sound S1 and the second heart sound S2; step 4, obtaining a time difference T1 between the electrocardio characteristic R wave and the first heart sound S1 and a time difference T2 between the electrocardio characteristic T wave and the second heart sound S2 in the same cardiac cycle, if the time interval between T1 and T2 is abnormal, continuing to diagnose tricuspid valve stenosis, otherwise, returning to a normal result; step 5, if the tricuspid valve stenosis is diagnosed, sampling the amplitude values of the diastole and the systole of the heart sound signal, and if the murmur appears in the diastole of the heart sound signal, judging that the murmur is caused by the tricuspid valve stenosis; if the heart sound signal appears in systole, the tricuspid valve is judged to be closed incompletely.
Chinese publication No. CN115062763A, publication No. 2023-4-28, entitled "end-to-end heart sound segmentation method based on Residual Bi-LSTM network", comprises the following main steps: synchronously collecting heart sounds and electrocardiosignals of healthy and sick volunteers, and ensuring that the duration of each collection comprises a plurality of cardiac cycles; the acquired heart sound signals are divided into a plurality of segments according to a certain length L, the certain length is ensured to at least comprise more than one cardiac cycle, and filling is carried out for the length L or a threshold value is set for discarding; marking the collected heart sound signals according to gold standards corresponding to heart sounds and electrocardiograms, and arranging the heart sound signals into a data set; training a Residual Bidirectional LSTM initial model by data; and forward propagation calculation is carried out on the brand-new heart sound signals which are not in the data set by utilizing the Residual Bidirectional LSTM network model trained by the data set, and finally, the segmentation task of the heart sound signals is realized.
Based on the above description, taking coronary heart disease as an example, the system and method for heart sound and heart electricity combined diagnosis still belong to the weaker field, most of the systems and methods only limit time sequence and frequency domain characteristics, and particularly under uncontrolled complex environments, the performance is rapidly reduced due to complex noise of heart sound, and great improvement space is provided in the aspects of accuracy, robustness and the like. On the premise of effective heart sound and electrocardio preprocessing, if multi-domain artificial features can be comprehensively extracted and deep learning features of heart sound and electrocardio are effectively fused, more potential modes in the heart sound and electrocardio signals can be identified, the diagnosis performance is improved, and therefore the method is more beneficial to reducing potential life hazards of heart diseases such as coronary heart diseases.
Disclosure of Invention
The invention aims to provide a heart sound and electrocardio combined diagnosis method and system based on dual-mode dual input, which comprehensively utilizes the extracted heart sound and electrocardio multi-domain artificial characteristics by using an Internet of things, an artificial intelligence technology and a cloud computing technology, effectively fuses the deep learning characteristics of heart sound and electrocardio, recognizes more potential modes in heart sound electrocardiosignals, reduces the influence of noise such as breathing sound, muscle shake and environmental noise in uncontrolled complex environments, improves the accuracy of diagnosing heart abnormalities, reduces the potential life risk of heart diseases and avoids acute myocardial infarction.
In order to achieve the above object, the solution of the present invention is:
a heart sound and heart electricity combined diagnosis method based on dual modes and dual inputs comprises,
synchronously acquiring heart sound signals and electrocardiosignals, wherein the acquired time length comprises a plurality of cardiac cycles;
the heart sound signal is divided into a plurality of first heart sound signal fragments with equal length according to the length L, and the electrocardiosignal is divided into a plurality of first electrocardiosignal fragments with equal length according to the length L; wherein the length L comprises at least one cardiac cycle;
sequentially carrying out band-pass filtering, wavelet transformation fusing an adaptive threshold value and denoising normalization of an countermeasure network based on self-constraint conditions on the first heart sound signal segment to obtain a second heart sound signal segment; sequentially carrying out band-pass filtering, notch filtering and normalization on the first electrocardiosignal segment to obtain a second electrocardiosignal segment;
Extracting artificial heart sound characteristics from the second heart sound signal segment, extracting artificial electrocardio characteristics from the second heart sound signal segment, wherein the artificial heart sound characteristics and the artificial electrocardio characteristics form a first characteristic vector, the first characteristic vector is taken as input, and a double-layer long-short-term memory network is utilized to obtain a second characteristic vector; performing multi-scale wavelet decomposition on the second heart sound signal segment to obtain a third heart sound signal segment, wherein the third heart sound signal segment and the second heart sound signal segment form a multi-channel heart sound electrocardiograph segment, the multi-channel heart sound electrocardiograph segment is taken as input, and a residual error network fusing a channel attention mechanism and a space attention mechanism is utilized to obtain a third feature vector;
splicing the second feature vector and the third feature vector, and performing classification diagnosis by adopting sigmoid;
wherein the wavelet transformation fusing the adaptive threshold to the first heart sound signal segment and the denoising generating the countermeasure network based on the self-constraint condition comprises,
performing band-pass filtering on the first heart sound signal segment, and then performing wavelet transformation of a self-adaptive threshold value to obtain a fourth heart sound signal segment;
The fourth heart sound signal segment is sent to a self-constraint condition-based generation countermeasure network to obtain a denoised heart sound signal segment;
wherein the wavelet transformation fusing the adaptive threshold and the generation of the countermeasure network based on the self-constraint condition are obtained according to the following method,
dividing the clean heart sound signal according to the length L to obtain a plurality of fifth heart sound signal fragments with equal length; mixing the clean heart sound signals and noise data, dividing the clean heart sound signals and the noise data according to the length L to obtain a plurality of equal-length zeroth heart sound signal fragments, and carrying out band-pass filtering on the zeroth heart sound signal fragments;
performing band-pass filtering on the zeroth heart sound signal segment, performing wavelet transformation of the self-adaptive threshold value, and repeating for a plurality of times to obtain a plurality of zeroth fourth heart sound signal segments;
the self-constraint-based condition generation countermeasure network comprises a generator and a discriminator, wherein the generator comprises an encoding stage and a decoding stage, and the encoder consists of k jumping convolution layers and a parameter correction linear unit PReLUs; the structure of the decoder is reversely symmetrical to the encoder;
the zeroth four heart sound signal segment and the fifth heart sound signal segment are parallelly fed into a generator, and each layer of output of a decoding stage and corresponding characteristics of an encoding stage are spliced and input into the next layer of a decoder; when the k-th layer signal compression result n_k of the zeroth fourth heart sound signal segment and the L-th layer signal compression result c_k of the fifth heart sound signal segment 1 The smaller the norm distance is, the more effective noise suppression can be achieved; defining a loss function of the generator as a self-constrained linear activation function, and avoiding the disappearance of the induced gradient;
and obtaining the optimal wavelet transformation fused with the self-adaptive threshold value through the self-constrained linear activation function by adopting an RMSprop optimization algorithm, and generating an countermeasure network based on the self-constrained condition.
Particularly, the heart sound signal is divided into a plurality of first heart sound signal fragments with equal length according to the length L, wherein the heart sound signal is divided according to the length L, and the rest part is filled or discarded under the length L;
the electrocardiosignal is divided into a plurality of first electrocardiosignal fragments with equal length according to the length L, wherein the electrocardiosignal is divided according to the length L, and the rest part is not filled or discarded with the length L.
In particular, the wavelet transformation of the self-adaptive threshold value is carried out on the first heart sound signal segment after the band-pass filtering to obtain a fourth heart sound signal segment, which comprises,
performing wavelet transformation on the first heart sound signal segment after band-pass filtering to obtain a high-frequency wavelet coefficient and a low-frequency wavelet coefficient;
respectively carrying out self-adaptive threshold quantization processing on the high-frequency wavelet coefficient and the low-frequency wavelet coefficient;
And carrying out wavelet inverse transformation on the high-frequency wavelet coefficient and the low-frequency wavelet coefficient which are subjected to the self-adaptive threshold quantization processing, and reconstructing the signal according to the wavelet inverse transformation to obtain a fourth heart sound signal segment.
In particular, the first characteristic vector is used as input, a double-layer long-short-period memory network is utilized to obtain a second characteristic vector, the structure of the double-layer long-short-period memory network is that,wherein, the method comprises the steps of, wherein,representing the current time step->First->Hidden state of layer->Indicate->Hidden state of one time step on the layer, < ->Indicating the hidden state of the layer above the current time step,/->、Representing a weight matrix, +.>Representing the bias.
In particular, the second heart sound signal segment is subjected to wavelet decomposition to obtain a third heart sound signal segment, the third heart sound signal segment and the second heart sound signal segment form a multi-channel heart sound graph electrocardiogram segment, which comprises,
adopting multi-scale wavelet decomposition to the second heart sound signal segment to obtain a third heart sound signal segment, wherein the function and the order N of the multi-scale wavelet decomposition 0 And number of decomposition layers L 0 Determining the standard mean square error between the wavelet result approximation and the real value of the heart sound signal;
the third heart sound signal segment and the second heart sound signal segment form L 0 +1-channel multichannel phonocardiogram electrocardiogram segment.
In particular, taking the multi-channel electrocardiographic segment as input, obtaining a third feature vector by utilizing a residual network of a fusion channel attention mechanism and a spatial attention mechanism, comprising,
the calculation formula of the channel attention mechanism is that,
，
the calculation formula of the spatial attention mechanism is that,
，
wherein,representing channel attention matrix,/->Representing a spatial attention matrix;A feature map representing an input;Representing a weight matrix of the multi-layer perceptron;Representing a multi-layer perceptron @, @>Represents an average pooling of the data in the pool,representing maximum pooling, ++>And->Feature vectors after pooling operation, < + >, respectively>In order for the convolution operation to be performed,and->Representing the feature matrix after the channel pooling operation, < + >>Representing an activation function.
In particular, the second feature vector and the third feature vector are spliced, and sigmoid is adopted for classified diagnosis, which comprises,
splicing the second feature vector and the third feature vector to obtain a fourth feature vector;
and adopting sigmoid to map the fourth feature vector into a prediction probability, and judging whether the result is normal or abnormal according to a preset threshold value so as to realize classification diagnosis.
A heart sound and heart electricity combined diagnosis system based on dual modes and dual inputs comprises,
The signal acquisition module is configured to synchronously acquire heart sound signals and electrocardiosignals, and the acquired duration comprises a plurality of cardiac cycles;
the signal segment dividing module is configured to divide the heart sound signal into a plurality of first heart sound signal segments with equal length according to the length L, and divide the electrocardiosignal into a plurality of first electrocardiosignal segments with equal length according to the length L; wherein the length L comprises at least one cardiac cycle;
the heart sound signal processing module is configured to sequentially perform band-pass filtering, wavelet transformation fused with a self-adaptive threshold value and denoising and normalization based on a self-constraint condition generation countermeasure network on the first heart sound signal segment to obtain a second heart sound signal segment;
the electrocardiosignal processing module is configured to sequentially carry out band-pass filtering, notch filtering and normalization on the first electrocardiosignal segment to obtain a second electrocardiosignal segment;
the first feature vector acquisition module is configured to extract artificial heart sound features from the second heart sound signal segment, extract artificial electrocardio features from the second heart sound signal segment, and form a first feature vector by the artificial heart sound features and the artificial electrocardio features;
The second characteristic vector acquisition module is configured to acquire a second characteristic vector by using the double-layer long-short-term memory network by taking the first characteristic vector as input;
the third feature vector acquisition module is configured to perform multi-scale wavelet decomposition on the second heart sound signal segment to obtain a third heart sound signal segment, wherein the third heart sound signal segment and the second heart sound signal segment form a multi-channel heart sound electrocardiograph segment, the multi-channel heart sound electrocardiograph segment is taken as input, and a residual network integrating a channel attention mechanism and a space attention mechanism is utilized to obtain a third feature vector; the classification module is configured to splice the second feature vector and the third feature vector and perform classification diagnosis by adopting sigmoid;
wherein the heart sound signal processing module is configured to sequentially bandpass filter the first heart sound signal segment, fuse wavelet transform of the adaptive threshold and generate denoising of the countermeasure network based on the self-constraint condition, including,
performing band-pass filtering on the first heart sound signal segment, and performing wavelet transformation of a self-adaptive threshold value to obtain a fourth heart sound signal segment;
The fourth heart sound signal segment is sent to a self-constraint condition-based generation countermeasure network to obtain a denoised heart sound signal segment;
wherein the wavelet transformation fusing the adaptive threshold and the generation of the countermeasure network based on the self-constraint condition are obtained according to the following method,
dividing the clean heart sound signal according to the length L to obtain a plurality of fifth heart sound signal fragments with equal length; mixing the clean heart sound signals and noise data, dividing the clean heart sound signals and the noise data according to the length L to obtain a plurality of equal-length zeroth heart sound signal fragments, and carrying out band-pass filtering on the zeroth heart sound signal fragments;
performing band-pass filtering on the zeroth heart sound signal segment, performing wavelet transformation of the self-adaptive threshold value, and repeating for a plurality of times to obtain a plurality of zeroth fourth heart sound signal segments;
the self-constraint-based condition generation countermeasure network comprises a generator and a discriminator, wherein the generator comprises an encoding stage and a decoding stage, and the encoder consists of k jumping convolution layers and a parameter correction linear unit PReLUs; the structure of the decoder is reversely symmetrical to the encoder;
the zeroth fourth heart sound signal segment and the fifth heart sound signal segment are sent into a generator in parallel, and each layer of decoding stage outputs Splicing the corresponding characteristics of the encoding stage to input the next layer of the decoder; when the k-th layer signal compression result n_k of the zeroth fourth heart sound signal segment and the L-th layer signal compression result c_k of the fifth heart sound signal segment 1 The smaller the norm distance is, the more effective noise suppression can be achieved; defining a loss function of the generator as a self-constrained linear activation function, and avoiding the disappearance of the induced gradient;
and obtaining the optimal wavelet transformation fused with the self-adaptive threshold value through the self-constrained linear activation function by adopting an RMSprop optimization algorithm, and generating an countermeasure network based on the self-constrained condition.
After the scheme is adopted, the invention adopts artificial intelligence and heart sound and heart electricity combined diagnosis, and has the beneficial effects that:
(1) The wavelet transformation of the self-adaptive threshold and the method for denoising heart sounds based on the self-constrained condition generation countermeasure network are fused, useful heart sound components can be prevented from being deleted by mistake to the greatest extent through the wavelet transformation of the self-adaptive threshold, noise can be restrained by the self-constrained condition generation countermeasure network, heart sound quality is enhanced, signal to noise ratio is improved, heart sound distortion is avoided, noise resistance in uncontrolled complex environments is improved, and a foundation is laid for robustness and accuracy of diagnosis of heart abnormalities;
(2) The method has the advantages that the noisy heart sound signals and the clean heart sound signals are input into the self-constraint condition-based generation countermeasure network in a parallel mode, the clean heart sound signals provide references for the noisy heart sound signals, the problem that the traditional generation countermeasure network only depends on the slow convergence of the noise signal input is solved, and the gradient disappearance is avoided while noise is effectively suppressed and useful heart sound loss is avoided by defining a self-constraint linear activation function;
(3) The dual-mode dual-input neural network model not only learns the heart sound electrocardiograph as the characteristic of a one-dimensional signal, but also learns the electrocardiogram characteristic of the heart sound chart, overcomes the bias of the heart sound and electrocardiograph single-mode characteristic, overcomes the defect of relying on artificial characteristics or deep learning characteristics, and improves the accuracy and stability of diagnosis;
(4) Introducing a residual structure, fusing a channel attention mechanism and a space attention mechanism, realizing the refinement of a residual feature map, emphasizing or suppressing partial features, and avoiding the problems of gradient explosion, gradient disappearance and degradation;
the invention has the following advantages compared with the traditional heart disease diagnosis system (or method):
(1) The heart sound and electrocardio combined diagnosis is more comprehensive, and misdiagnosis and missed diagnosis caused by heart sound or electrocardio can be effectively avoided, for example, the atrial electric activity completely disappears when the atrium flutters and the atrium tremors;
(2) The traditional method often has the ideal assumption premise that heart sounds are noiseless, the influence of noise of lung sounds and surrounding environments on the practice center sound collection is difficult to avoid, the performance is quickly reduced or the system is not applicable at all, and the system has stronger robustness and accuracy;
(3) The dual-mode dual-input neural network model effectively learns and fuses the characteristics of one-dimensional heart sounds and electrocardiosignals, and the characteristics of a heart sound map and an electrocardiogram, so that information complementation is realized, the information coverage is widened, the one-sidedness and the limitation of a single mode are overcome, and the accuracy and the stability are higher;
(4) The advantages of non-invasive data acquisition of the heart sound sensor and the heart electric sensor are fully exerted, the shortage of medical resources and uneven distribution can be relieved, and the daily home monitoring is facilitated; the cloud-side-end system structure deployment is utilized, large-scale user acquisition can be supported, early screening and tracking follow-up visit of heart diseases are facilitated, and the practical problems of lack of quantity of specialized doctors, uneven level and the like are overcome.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a block diagram of the system of the present invention;
FIG. 3 is a graph showing the comparison of wavelet transform fused with adaptive threshold to denoising of an countermeasure network based on self-constrained condition generation with conventional denoising effects;
Wherein, (a) is an original signal, (b) is a noise signal, (c) is a denoising effect generated by combining wavelet transformation of an adaptive threshold value and a self-constraint-based condition of an countermeasure network, and (d) is a traditional denoising effect based on wavelet transformation.
Detailed Description
The technical scheme and beneficial effects of the present invention will be described in detail below with reference to the accompanying drawings.
The invention provides a heart sound and heart electricity combined diagnosis method based on double modes and double inputs, which comprises the following steps,
synchronously acquiring heart sound signals and electrocardiosignals, wherein the acquired time length comprises a plurality of cardiac cycles;
the heart sound signal is divided into a plurality of first heart sound signal fragments with equal length according to the length L, and the electrocardiosignal is divided into a plurality of first electrocardiosignal fragments with equal length according to the length L; wherein the length L comprises at least one cardiac cycle;
sequentially carrying out band-pass filtering, wavelet transformation fusing an adaptive threshold value and denoising normalization of an countermeasure network based on self-constraint conditions on the first heart sound signal segment to obtain a second heart sound signal segment; sequentially carrying out band-pass filtering, notch filtering and normalization on the first electrocardiosignal segment to obtain a second electrocardiosignal segment;
Extracting artificial heart sound characteristics from the second heart sound signal segment, extracting artificial electrocardio characteristics from the second heart sound signal segment, wherein the artificial heart sound characteristics and the artificial electrocardio characteristics form a first characteristic vector, the first characteristic vector is taken as input, and a double-layer long-short-term memory network is utilized to obtain a second characteristic vector; performing multi-scale wavelet decomposition on the second heart sound signal segment to obtain a third heart sound signal segment, wherein the third heart sound signal segment and the second heart sound signal segment form a multi-channel heart sound electrocardiograph segment, the multi-channel heart sound electrocardiograph segment is taken as input, and a residual error network fusing a channel attention mechanism and a space attention mechanism is utilized to obtain a third feature vector;
splicing the second feature vector and the third feature vector, and performing classification diagnosis by adopting sigmoid;
wherein the wavelet transformation fusing the adaptive threshold to the first heart sound signal segment and the denoising generating the countermeasure network based on the self-constraint condition comprises,
performing band-pass filtering on the first heart sound signal segment, and then performing wavelet transformation of a self-adaptive threshold value to obtain a fourth heart sound signal segment;
The fourth heart sound signal segment is sent to a self-constraint condition-based generation countermeasure network to obtain a denoised heart sound signal segment;
wherein the wavelet transformation fusing the adaptive threshold and the generation of the countermeasure network based on the self-constraint condition are obtained according to the following method,
dividing the clean heart sound signal according to the length L to obtain a plurality of fifth heart sound signal fragments with equal length; mixing the clean heart sound signals and noise data, dividing the clean heart sound signals and the noise data according to the length L to obtain a plurality of equal-length zeroth heart sound signal fragments, and carrying out band-pass filtering on the zeroth heart sound signal fragments;
performing band-pass filtering on the zeroth heart sound signal segment, performing wavelet transformation of the self-adaptive threshold value, and repeating for a plurality of times to obtain a plurality of zeroth fourth heart sound signal segments;
the self-constraint-based condition generation countermeasure network comprises a generator and a discriminator, wherein the generator comprises an encoding stage and a decoding stage, and the encoder consists of k jumping convolution layers and a parameter correction linear unit PReLUs; the structure of the decoder is reversely symmetrical to the encoder;
the zeroth four heart sound signal segment and the fifth heart sound signal segment are parallelly fed into a generator, and each layer of output of a decoding stage and corresponding characteristics of an encoding stage are spliced and input into the next layer of a decoder; l of a k-th layer signal compression result n_k of a zeroth fourth heart sound signal segment and a k-th layer signal compression result c_k of a fifth heart sound signal segment 1 The smaller the norm distance is, the more effective noise suppression can be achieved; defining a loss function of the generator as a self-constrained linear activation function, and avoiding the disappearance of the induced gradient;
and obtaining the optimal wavelet transformation fused with the self-adaptive threshold value through the self-constrained linear activation function by adopting an RMSprop optimization algorithm, and generating an countermeasure network based on the self-constrained condition.
The invention also provides a heart sound and heart electricity combined diagnosis system based on the double-mode double input, which comprises,
the signal acquisition module is configured to synchronously acquire heart sound signals and electrocardiosignals, and the acquired duration comprises a plurality of cardiac cycles;
the signal segment dividing module is configured to divide the heart sound signal into a plurality of first heart sound signal segments with equal length according to the length L, and divide the electrocardiosignal into a plurality of first electrocardiosignal segments with equal length according to the length L; wherein the length L comprises at least one cardiac cycle;
the heart sound signal processing module is configured to sequentially perform band-pass filtering, wavelet transformation fused with a self-adaptive threshold value and denoising and normalization based on a self-constraint condition generation countermeasure network on the first heart sound signal segment to obtain a second heart sound signal segment;
The electrocardiosignal processing module is configured to sequentially carry out band-pass filtering, notch filtering and normalization on the first electrocardiosignal segment to obtain a second electrocardiosignal segment;
the first feature vector acquisition module is configured to extract artificial heart sound features from the second heart sound signal segment, extract artificial electrocardio features from the second heart sound signal segment, and form a first feature vector by the artificial heart sound features and the artificial electrocardio features;
the second characteristic vector acquisition module is configured to acquire a second characteristic vector by using the double-layer long-short-term memory network by taking the first characteristic vector as input;
the third feature vector acquisition module is configured to perform multi-scale wavelet decomposition on the second heart sound signal segment to obtain a third heart sound signal segment, wherein the third heart sound signal segment and the second heart sound signal segment form a multi-channel heart sound electrocardiograph segment, the multi-channel heart sound electrocardiograph segment is taken as input, and a residual network integrating a channel attention mechanism and a space attention mechanism is utilized to obtain a third feature vector; the classification module is configured to splice the second feature vector and the third feature vector and perform classification diagnosis by adopting sigmoid;
Wherein the heart sound signal processing module is configured to sequentially bandpass filter the first heart sound signal segment, fuse wavelet transform of the adaptive threshold and generate denoising of the countermeasure network based on the self-constraint condition, including,
performing band-pass filtering on the first heart sound signal segment, and performing wavelet transformation of a self-adaptive threshold value to obtain a fourth heart sound signal segment;
the fourth heart sound signal segment is sent to a self-constraint condition-based generation countermeasure network to obtain a denoised heart sound signal segment;
wherein the wavelet transformation fusing the adaptive threshold and the generation of the countermeasure network based on the self-constraint condition are obtained according to the following method,
dividing the clean heart sound signal according to the length L to obtain a plurality of fifth heart sound signal fragments with equal length; mixing the clean heart sound signals and noise data, dividing the clean heart sound signals and the noise data according to the length L to obtain a plurality of equal-length zeroth heart sound signal fragments, and carrying out band-pass filtering on the zeroth heart sound signal fragments;
performing band-pass filtering on the zeroth heart sound signal segment, performing wavelet transformation of the self-adaptive threshold value, and repeating for a plurality of times to obtain a plurality of zeroth fourth heart sound signal segments;
the self-constraint-based condition generation countermeasure network comprises a generator and a discriminator, wherein the generator comprises an encoding stage and a decoding stage, and the encoder consists of k jumping convolution layers and a parameter correction linear unit PReLUs; the structure of the decoder is reversely symmetrical to the encoder;
The zeroth four heart sound signal segment and the fifth heart sound signal segment are parallelly fed into a generator, and each layer of output of a decoding stage and corresponding characteristics of an encoding stage are spliced and input into the next layer of a decoder; l of a k-th layer signal compression result n_k of a zeroth fourth heart sound signal segment and a k-th layer signal compression result c_k of a fifth heart sound signal segment 1 The smaller the norm distance is, the more effective noise suppression can be achieved; defining a loss function of the generator as a self-constrained linear activation function, and avoiding the disappearance of the induced gradient;
and obtaining the optimal wavelet transformation fused with the self-adaptive threshold value through the self-constrained linear activation function by adopting an RMSprop optimization algorithm, and generating an countermeasure network based on the self-constrained condition.
As a preferred embodiment of the present invention, as shown in FIG. 1, a dual-mode dual-input based heart sound and heart electricity combined diagnosis method comprises the following steps:
step 1, synchronously acquiring heart sounds and electrocardiosignals by adopting a signal acquisition module, and ensuring that the duration of each acquisition comprises a plurality of cardiac cycles;
step 2, the fog server receives heart sounds and electrocardiosignals sent by the signal acquisition module and divides the heart sounds and the electrocardiosignals into heart sound signal fragments and electrocardiosignal fragments with equal length according to a certain length L, wherein the length L ensures that at least one cardiac cycle is contained, and filling or discarding is carried out under the length L;
Step 3, removing high-frequency noise and low-frequency noise in the heart sound signal fragments and the electrocardiosignal fragments with equal length in the step 2 by adopting a Butterworth band-pass filter, generating an countermeasure network by adopting wavelet transformation fused with an adaptive threshold and based on a self-constraint condition, reducing noise overlapped with effective heart sound, improving signal-to-noise ratio and avoiding heart sound distortion, and carrying out normalization processing to obtain a preprocessed heart sound signal fragment; the fog server also adopts a Butterworth band-pass filter to remove baseline drift and denoising in the electrocardiosignal segments, and utilizes a notch filter to remove power frequency interference in the electrocardiosignal segments, and then performs normalization processing to obtain preprocessed electrocardiosignal segments;
step 4, the cloud platform extracts artificial heart sound features and artificial electrocardio features from the preprocessed heart sound signal segments and preprocessed electrocardio signal segments; decomposing the preprocessed heart sound signal segment into a multi-scale wavelet and forming a multi-channel heart sound map electrocardiogram segment with the preprocessed electrocardiosignal segment;
step 5, the cloud platform adopts a dual-mode dual-input neural network model to carry out joint diagnosis, and the artificial heart sound characteristics and the artificial electrocardio characteristics are input into a double-layer long-short-term memory network; the multichannel heart sound electrocardiogram fragments are input to a residual error network for fusing a channel attention mechanism and a space attention mechanism;
And 6, splicing the output of the residual error network of the double-layer long-short-period memory network, the fusion channel attention mechanism and the spatial attention mechanism in the step 5, and performing classification diagnosis by adopting sigmoid.
Examples
A heart sound and heart electricity combined diagnosis method based on a bimodal double-input neural network model comprises a training stage and an application stage. The training stage comprises mist server side training and cloud platform training.
The fog server side (preprocessing) training phase comprises:
(A) Constructing a data set, selecting a clean (noiseless) synchronous heart sound signal (PhysioNet/CinC change 2016), taking a hospital environment noise data set (https:// www.kaggle.com/datasets/nafin 59/hospital-current-noise) and an ICBHI 2017 lung sound data set as noise data, mixing the noise data with a sampling rate of 2000Hz, dividing heart sounds and electrocardiosignal fragments by 3s, and dividing the noise data into a training set and a test set according to a ratio of 4:1;
(B) Using a Butt Wo Sigao pass filter on noisy heart sound signalsFiltering high frequency noise above 400Hz and low frequency noise below 25Hz to obtain heart sound signal +.>；
(C) The method adopts wavelet transformation fused with an adaptive threshold and generates overlapping part noise of 25Hz-400Hz and effective heart sounds by an countermeasure network based on self-constraint conditions, and specifically comprises the following steps:
(C-1) versus heart sound SignalPerforming wavelet transformation, and obtaining a high-frequency wavelet coefficient and a low-frequency wavelet coefficient by adopting a wavelet basis function (Daubechies-5) and decomposing and calculating;
and (C-2) respectively carrying out threshold quantization treatment on the decomposed high-frequency wavelet coefficient and low-frequency wavelet coefficient, wherein the threshold adopts an adaptive threshold, and the useful heart sound component is prevented from being deleted by mistake to the greatest extent, wherein the adaptive threshold is defined as:
，
wherein,for the initial threshold value->Is the firstiThe threshold value of the layer is set,Ithe number of layers for decomposition.
(C-3) performing inverse wavelet transform on the high-frequency wavelet coefficient and the low-frequency wavelet coefficient after the threshold quantization processing, and reconstructing the signals to obtain heart sound signals after wavelet threshold denoising。
(C-4) to enhance the quality of the heart sound signal,the input to the self-constraint based condition generation countermeasure network further suppresses noise, improves the signal to noise ratio and avoids heart sound distortion.
The generating countermeasure network based on the self-constraint condition is composed of a generator and a discriminator.
The noisy heart sound signal samples and the clean heart sound signal are input in a parallel manner to a generator network, the output of which is a de-noised heart sound signal. The generator includes an encoding stage and a decoding stage. The encoder consists of k jump convolution layers and a parameter correction linear unit PReLUs, wherein the convolution kernel is of the size The step size is 4. The decoder is constructed in reverse symmetry with the encoder. Each layer of output of the decoding stage and the corresponding feature of the encoding stage are spliced and input to the next layer of the decoder. Sample of heart sound signal with noise->From the wavelet threshold denoised heart sound signal set, and clean heart sound signal +.>Parallel input generator network training, both using the same network parameters. When->Intermediate k-layer signal compression result n_k and clean heart sound signal +.>L of the intermediate k-layer signal compression result c_k 1 The smaller the norm distance, the more effective the noise suppression and the avoidance of loss of useful heart sounds, but at the same time the tendency for gradient extinction to occur. To this end, a self-constrained linear activation function is defined and acts as a loss function for the generator: />
，
Wherein, the heart sound sample with noiseTaking the heart sound signal data distribution after denoising from wavelet threshold +.>Noise samples z obey the noise data distribution +.>，Representing denoised heart sounds->And clean heart sounds->L of feature vector of (2) 1 A norm distance;L representing c_k and n_k 1 A norm distance; the generator and the arbiter are G and D, respectively;，Representing the weight coefficient.
The discriminator comprises k convolution layers and 1 full connection layer, wherein each convolution layer is followed by 1 BN layer and 1 PReLU activation function layer, and the output dimension is that . Input pair of signals of the discriminator>Or->The output value is 0 or 1. Loss function of the arbiter:
，
wherein, the heart sound sample with noiseTaking the heart sound signal data distribution after denoising from wavelet threshold +.>Noise samples z obey the noise data distribution +.>Clean heart sounds->The generator and arbiter are G and D, respectively.
Repeating the steps (C-2) to (C-3), adopting an RMSprop optimization algorithm, setting the batch size to be 32, and setting the learning rate to beThe parameter k is designed to be 7, the maximum iteration period is set to be 160, and the optimal wavelet transformation fused with the self-adaptive threshold value is obtained through the self-constrained linear activation function, and the countermeasure network is generated based on the self-constrained condition.
Cloud platform training, comprising:
(a) Clean synchronous heart sounds and electrocardiosignals, wherein the sampling rate is 2000Hz, heart sounds and electrocardiosignal fragments are divided into training sets and testing sets according to the ratio of 4:1;
(b) Normalizing by Z-score method to obtain normalized heart sound signal segment；
(c) Normalizing by Z-score method to obtain normalized electrocardiosignal fragment；
(d) From heart sound signal segments、The electrocardiosignal segments extract artificial features respectively, 5-domain heart sound features (time domain, frequency domain, energy domain, entropy domain and kurtosis domain features), 3-domain electrocardiosignal features (time domain, frequency domain and time-frequency domain features) and 256-dimensional fused heart sound electrocardiosignal feature vectors- >；
(e) Heart sound and electrocardio characteristic vectorInputting the depth feature vector into a double-layer long-short-term memory network to obtain a 128-dimensional depth feature vector>The method comprises the steps of carrying out a first treatment on the surface of the The double-layer long-short-period memory networkThe structure is as follows:Wherein->Representing the current time step->First->Hidden state of layer->Indicate->Hidden state of one time step on the layer, < ->Indicating the hidden state of the layer above the current time step,/->、Representing a weight matrix, +.>Representing the bias;
(f) Based on a multiscale wavelet decomposition (Daubechies function, order N 0 And number of decomposition layers L 0 ) Heart sound signal segmentAnd electrocardio signal segment->Composition->Channel phonocardiogram electrocardiograph fragment->Dimension is->The residual error is input into a residual error network of a fusion channel attention mechanism and a spatial attention mechanism, the refinement of a residual error characteristic diagram is realized through the fusion channel attention mechanism and the spatial attention mechanism, partial characteristics are emphasized or suppressed, and a 128-dimensional depth characteristic vector is obtained>The method comprises the steps of carrying out a first treatment on the surface of the The channel attention calculation formula is as follows:
，
the spatial attention matrix is obtained by carrying out average pooling and maximum pooling on channel dimensions to compress the channel size and then carrying out convolution learning spatial feature calculation, and the spatial attention calculation formula is as follows:
，
wherein,representing channel attention matrix,/- >Representing a spatial attention matrix;A feature map representing an input;Representing a weight matrix of the multi-layer perceptron;Representing a multi-layer perceptron @, @>Represents an average pooling of the data in the pool,representing maximum pooling, ++>And->Feature vectors after pooling operation, < + >, respectively>In order for the convolution operation to be performed,and->Representing the feature matrix after the channel pooling operation, < + >>Representing an activation function.
The (f) is based on multi-scale wavelet decomposition (wavelet function, orderAnd number of decomposition layers->) Heart sound signal segmentAnd electrocardio signal segment->Composition->Channel phonocardiogram electrocardiograph fragment->Dimension isInput to the fusion channel attention machineThe residual network of the system and the spatial attention mechanism comprises the following specific steps:
(f-1) determining a wavelet function, an optimal order, based on decomposing the heart sound signal segment by the multi-scale waveletAnd optimal number of decomposition layers->The specific process comprises the following steps: setting the wavelet function range as Daubechies function, symlets function, the order range as 2-8 and the decomposition layer number range as 1-7; calculating the standard mean square error of wavelet result approximation and heart sound signal true value under different orders and different decomposition layers by adopting a multi-scale wavelet decomposition method, and determining the optimal order +.>And the optimal number of decomposition layers 。
(f-2) the product obtained from (f-1)The layer heart sound signal segment is converted by Matlab GUI kit to obtain +.>A heart sound map segment; the electrocardiosignal segments are converted into 1 electrocardiograph segment by Matlab GUI to form +.>A channel phonocardiogram electrocardiogram segment; />
(f-3) combining severalChannel phonocardiogram electrocardiograph fragment->The residual network is input, the convolution kernel size is +.>The step length is 2;
(f-4) passing through a maximum pooling layer, wherein the pooling window is 3, and the step length is 2;
(f-5) inputting a continuous Bottleneck structure, wherein a main line of the Bottleneck structure comprises a first 2 convolution layers with a convolution kernel size of 1 and a middle one convolution layer with a convolution kernel size of 3, wherein the steps are all 1;
(f-6) branching of bottleneck structures in series using a channel attention module and a spatial attention module; the channel attention module comprises 2 parallel pooling layers, namely space maximum pooling and space tie pooling, and passes through 1 multi-layer perceptron; the spatial attention module comprises 2 parallel pooling layers, namely channel maximum pooling and channel tie pooling respectively, and then passes through 1 convolution layer;
(f-7) obtaining 128-dimensional depth feature vectors through a max pooling layer and full connection layer 。
(g) Stitching 128-dimensional depth feature vector E 2 128-dimensional depth feature vector E 3 Obtaining feature vectorsAnd the sigmoid is adopted to map the characteristics into the prediction probability, so that the classification diagnosis is finally realized, the diagnosis result is divided into two categories of normal and abnormal, the normal is represented by 0, and the abnormal is represented by 1:
，
，
(h) The parameters of a double-layer long-short-term memory network, a residual error network integrating a channel attention mechanism and a space attention mechanism are updated by adopting a gradient descent method, so that a loss function is minimized, and an optimal bimodal double-input neural network model after training is obtained, wherein the loss function is defined as follows:
，
wherein,probability of being true for the ith sample, +.>The probability is output for the network for which the i-th sample is true.
An application phase comprising the steps of:
step S1, a user synchronously collects heart sounds and electrocardiosignals by adopting a signal collection module;
step S2, the fog server receives heart sounds and electrocardiosignals sent by the signal acquisition module, divides the heart sounds and the electrocardiosignals into equal-length heart sound signal fragments and electrocardiosignal fragments according to the length of 3S, and ensures that the fog server at least contains more than one cardiac cycle, and fills or discards the heart sounds and the electrocardiosignals with the length of less than 3S;
step S3, preprocessing the heart sound signal segment and the electrocardiosignal segment by the fog server;
Step S4, the cloud platform extracts artificial features from the heart sound signal segment and the electrocardio signal segment preprocessed in the step S3, wherein the artificial features comprise 5-domain heart sound features (time domain, frequency domain, energy domain, entropy domain and kurtosis domain features) and 3-domain electrocardio features (time domain, frequency domain and time-frequency domain features); the multi-scale wavelet decomposes heart sounds and forms a multi-channel electrocardiograph segment of the heart sounds and the electrocardio;
s5, the cloud platform adopts an optimal bimodal double-input neural network model, namely, the artificial characteristics are input into a double-layer long-short-term memory network, and the multi-channel electrocardiograph segments are input into a residual network integrating a channel attention mechanism and a space attention mechanism; splicing the double output results and performing classification diagnosis by adopting sigmoid;
and S6, if the cloud platform is classified and diagnosed as abnormal, sending alarm information to a specific mobile intelligent terminal.
In the step S3, the fog server preprocesses the heart sound signal segment and the electrocardiosignal segment, and the specific process is as follows:
step S31a, using a Bart Wo Sigao pass filter to filter high-frequency noise above 400Hz and low-frequency noise below 25Hz from heart sounds;
step S31b, generating a denoising method of the countermeasure network by utilizing the optimal wavelet transformation and self-constraint condition fused with the self-adaptive threshold value to reduce the noise overlapped with 25Hz-400Hz effective heart sounds;
Step S31c, normalizing by using a Z-score method;
step S32a, removing baseline drift and noise of electrocardiosignals by using a Butterworth band-pass filter (1-60 Hz);
step S32b, removing 50Hz power frequency interference of electrocardiosignals by using a notch filter;
and step S32c, performing normalization processing on the filtered electrocardiosignals by using a Z-score method.
The signal acquisition module comprises a main control MCU unit, a heart sound sensor unit, a heart electric sensor unit, an analog-to-digital converter, a power supply module and a wireless communication unit. THE heart sound sensor unit adopts THE THE ONE sensor of THE ThinkLabs company, THE heart electric sensor unit adopts a Neurosky BMD 101, THE main control MCU unit adopts an STM32F103ZET6 chip, THE AD7606 of Analog Devices company is adopted as an Analog-to-digital converter, and THE wireless communication unit adopts a Realtek RTL8723BU to support Bluetooth and WIFI protocols.
When the method is applied, the cloud platform can send the diagnosis result to the mobile intelligent terminal, and the mobile intelligent terminal collects the conventional attribute data of the user, including age, gender, height, weight, body mass index, systolic pressure and diastolic pressure.
The mobile intelligent terminal registers user data including age, gender, height, weight, body mass index, systolic pressure and diastolic pressure;
The fog server is positioned on the edge network, receives heart sounds and electrocardiosignals sent by the heart sounds and electrocardiosignals synchronous acquisition module by utilizing an integrated Bluetooth and WIFI fusion gateway and performs pretreatment, wherein the pretreatment comprises removing baseline drift and denoising of the electrocardiosignals by adopting a Butterworth band-pass filter; and removing high-frequency noise and low-frequency noise of heart sounds by adopting a Butt Wo Sigao pass filter, and generating a denoising method of an countermeasure network by adopting a wavelet transformation and self-constraint condition fused with a self-adaptive threshold value in order to reduce noise overlapped with effective heart sounds. In the implementation, the fog server adopts a Cortex A8 processor chip, a 250G Byte solid state disk, a Linux operating system Xen VMM, a fusion gateway (Bluetooth and WIFI) and a 10M/100M network card, and is connected with the cloud platform through the 10M/100M network card.
The cloud platform adopts a virtualization technology, and the received data is input into a bimodal dual-input neural network model for diagnosis, and the cloud platform comprises a feature extraction module, a bimodal dual-input network module and a classification module. The characteristic extraction module extracts artificial characteristics including 5-domain heart sound characteristics (time domain, frequency domain, energy domain, entropy domain and kurtosis domain characteristics) and 3-domain electrocardio characteristics (time domain, frequency domain and time-frequency domain characteristics); the dual-mode dual-input network module comprises a dual-layer long-short-period memory network, a residual error network integrating a channel attention mechanism and a space attention mechanism, 5-domain heart sound features and 3-domain heart electric features are input into the dual-layer long-short-period memory network, heart sounds and electrocardio are decomposed through wavelets to form 7-channel heart sound map electrocardiogram fragments, and the 7-channel heart sound map electrocardiogram fragments are input into the residual error network integrating the channel attention mechanism and the space attention mechanism; the classification module is responsible for splicing the double output results and adopts a Sigmoid algorithm to carry out classification diagnosis.
Fig. 2 shows the architecture of a dual-mode dual-input-based heart sound and heart electricity combined diagnosis system, which comprises a data acquisition end (comprising a plurality of signal acquisition modules), a fog server (preprocessing) and a cloud platform. Fig. 3 shows a heart sound original signal and a noise signal, and for the heart sound original signal and the noise signal, fig. 3 also compares the wavelet transformation fused with the adaptive threshold with the self-constraint-based condition generation countermeasure network denoising effect and the traditional heart sound signal denoising effect based on the wavelet transformation.
The cloud platform Training uses a "Training-a" subset of PhysioNet/CinC Change 2016, including 338 noiseless synchronized heart sounds, electrocardiographic recordings (normal 116, abnormal 272). And (3) determining a multi-scale wavelet decomposition using a Daubechies function, an optimal order 5 and an optimal decomposition layer number 6 according to the step (f-1), and forming a 7-channel electrocardiographic segment of the electrocardiograph through the multi-scale wavelet decomposition and the electrocardiograph. The training results adopt sensitivity (TP/(FP+TN)), specificity (TN/(FP+TN)) and accuracy ((TP+TN)/(TP+FP+TN+FN)) as indexes to reach 95.83%,96.74% and 96.89% respectively, wherein TP is the number of true positive bars, TN is the number of true negative bars, FP is the number of false positive bars, and FN is the number of false negative bars.
Further, since the conventional wavelet transform requires manual parameter setting, it is easily affected by subjective factors and subjective experiences, and useful heart sound components are often deleted by mistake. The heart sound denoising method of the countermeasure network is generated by combining wavelet transformation of the self-adaptive threshold and self-constraint conditions, so that effective denoising can be ensured, the heart sound quality can be improved, heart sound distortion can be avoided, and a data foundation is laid for the diagnosis accuracy of the bimodal double-input neural network model.
Further, the features extract artificial features, i.e. features designed and defined artificially, including 5-domain heart sound features (time domain, frequency domain, energy domain, entropy domain and kurtosis domain features), 3-domain electrocardiographic features (time domain, frequency domain, time-frequency domain features). In order to improve efficiency, the artificial characteristics of heart sounds and electrocardiograms can be selected and then input into a double-layer long-short-term memory network, so that the situation that the accuracy suddenly drops easily although the single-layer long-short-term memory network is suitable for time series is avoided.
Furthermore, the invention supports collection of heart sounds and electrocardio data with false alarm as samples to update network parameters of the model, thereby gradually enhancing the prediction accuracy of the model.
Furthermore, the invention supports a double-input data processing mode, namely, separately-controlled input artificial features and depth features, can fully utilize the artificial features, improves the identification degree of various heart sounds and electrocardio abnormal signals, and is beneficial to improving the accuracy of heart sounds and electrocardio monitoring and diagnosis results; the advantages of the heart sound sensor and the electrocardio sensor in non-invasive data acquisition are fully exerted, and the daily home monitoring is facilitated.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The scheme in the embodiment of the invention can be realized by adopting various computer languages, such as object-oriented programming language Java, an transliteration script language JavaScript and the like.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (7)
1. Heart sound and heart electricity combined diagnosis system based on dual-mode double input is characterized in that: comprising the steps of (a) a step of,
the signal acquisition module is configured to synchronously acquire heart sound signals and electrocardiosignals, and the acquired duration comprises a plurality of cardiac cycles;
the signal segment dividing module is configured to divide the heart sound signal into a plurality of first heart sound signal segments with equal length according to the length L, and divide the electrocardiosignal into a plurality of first electrocardiosignal segments with equal length according to the length L; wherein the length L comprises at least one cardiac cycle;
the heart sound signal processing module is configured to sequentially perform band-pass filtering, wavelet transformation fused with a self-adaptive threshold value and denoising and normalization based on a self-constraint condition generation countermeasure network on the first heart sound signal segment to obtain a second heart sound signal segment;
the electrocardiosignal processing module is configured to sequentially carry out band-pass filtering, notch filtering and normalization on the first electrocardiosignal segment to obtain a second electrocardiosignal segment;
The first feature vector acquisition module is configured to extract artificial heart sound features from the second heart sound signal segment, extract artificial electrocardio features from the second heart sound signal segment, and form a first feature vector by the artificial heart sound features and the artificial electrocardio features; wherein: the artificial heart sound features comprise 5-domain heart sound features, namely time domain, frequency domain, energy domain, entropy domain and kurtosis domain features, and the artificial heart sound features comprise 3-domain heart sound features, namely time domain, frequency domain and time-frequency domain features;
the second characteristic vector acquisition module is configured to acquire a second characteristic vector by using the double-layer long-short-term memory network by taking the first characteristic vector as input;
the third feature vector acquisition module is configured to perform multi-scale wavelet decomposition on the second heart sound signal segment to obtain a third heart sound signal segment, wherein the third heart sound signal segment and the second heart sound signal segment form a multi-channel heart sound electrocardiograph segment, the multi-channel heart sound electrocardiograph segment is taken as input, and a residual network integrating a channel attention mechanism and a space attention mechanism is utilized to obtain a third feature vector; the classification module is configured to splice the second feature vector and the third feature vector and perform classification diagnosis by adopting sigmoid;
Wherein the heart sound signal processing module is configured to sequentially bandpass filter the first heart sound signal segment, fuse wavelet transform of the adaptive threshold and generate denoising of the countermeasure network based on the self-constraint condition, including,
performing band-pass filtering on the first heart sound signal segment, and performing wavelet transformation of a self-adaptive threshold value to obtain a fourth heart sound signal segment;
the fourth heart sound signal segment is sent to a self-constraint condition-based generation countermeasure network to obtain a denoised heart sound signal segment;
wherein the wavelet transformation fusing the adaptive threshold and the generation of the countermeasure network based on the self-constraint condition are obtained according to the following method,
dividing the clean heart sound signal according to the length L to obtain a plurality of fifth heart sound signal fragments with equal length; mixing the clean heart sound signals and noise data, dividing the clean heart sound signals and the noise data according to the length L to obtain a plurality of equal-length zeroth heart sound signal fragments, and carrying out band-pass filtering on the zeroth heart sound signal fragments; the clean heart sound signal is a noise-free heart sound signal;
performing band-pass filtering on the zeroth heart sound signal segment, performing wavelet transformation of the self-adaptive threshold value, and repeating for a plurality of times to obtain a plurality of zeroth fourth heart sound signal segments;
The self-constraint-based condition generation countermeasure network comprises a generator and a discriminator, wherein the generator comprises an encoding stage and a decoding stage, and the encoder consists of k jumping convolution layers and a parameter correction linear unit PReLUs; the structure of the decoder is reversely symmetrical to the encoder;
the zeroth four heart sound signal segment and the fifth heart sound signal segment are parallelly fed into a generator, and each layer of output of a decoding stage and corresponding characteristics of an encoding stage are spliced and input into the next layer of a decoder; when the k-th layer signal compression result n_k of the zeroth fourth heart sound signal segment and the L-th layer signal compression result c_k of the fifth heart sound signal segment 1 The smaller the norm distance is, the more effective noise suppression can be achieved; defining a loss function of the generator as a self-constrained linear activation function, and avoiding the disappearance of the induced gradient;
and obtaining the optimal wavelet transformation fused with the self-adaptive threshold value through the self-constrained linear activation function by adopting an RMSprop optimization algorithm, and generating an countermeasure network based on the self-constrained condition.
2. The system of claim 1, wherein: the heart sound signal is divided into a plurality of first heart sound signal fragments with equal length according to the length L, wherein the heart sound signal is divided according to the length L, and the rest part is filled or discarded under the length L;
The electrocardiosignal is divided into a plurality of first electrocardiosignal fragments with equal length according to the length L, wherein the electrocardiosignal is divided according to the length L, and the rest part is not filled or discarded with the length L.
3. The system of claim 1, wherein: performing wavelet transformation of the self-adaptive threshold on the first heart sound signal segment after band-pass filtering to obtain a fourth heart sound signal segment, including,
performing wavelet transformation on the first heart sound signal segment after band-pass filtering to obtain a high-frequency wavelet coefficient and a low-frequency wavelet coefficient;
respectively carrying out self-adaptive threshold quantization processing on the high-frequency wavelet coefficient and the low-frequency wavelet coefficient;
and carrying out wavelet inverse transformation on the high-frequency wavelet coefficient and the low-frequency wavelet coefficient which are subjected to the self-adaptive threshold quantization processing, and reconstructing the signal according to the wavelet inverse transformation to obtain a fourth heart sound signal segment.
4. The system of claim 1, wherein: the first characteristic vector is used as input, a double-layer long-short-term memory network is utilized to obtain a second characteristic vector, the structure of the double-layer long-short-term memory network is that,wherein->Representing the current time step->First->Hidden state of layer- >Indicate->Hidden state of one time step on the layer, < ->Indicating the hidden state of the layer above the current time step,/->、Representing a weight matrix, +.>Representing the bias.
5. The system of claim 1, wherein: performing multi-scale wavelet decomposition on the second heart sound signal segment to obtain a third heart sound signal segment, wherein the third heart sound signal segment and the second heart sound signal segment form a multi-channel heart sound graph electrocardiogram segment, and the multi-channel heart sound graph electrocardiogram segment comprises,
adopting multi-scale wavelet decomposition to the second heart sound signal segment to obtain a third heart sound signal segment, wherein the function and the order N of the multi-scale wavelet decomposition 0 And number of decomposition layers L 0 Determining the standard mean square error between the wavelet result approximation and the real value of the heart sound signal;
the third heart sound signal segment and the second heart sound signal segment form L 0 +1-channel multichannel phonocardiogram electrocardiogram segment.
6. The system of claim 1, wherein: the multi-channel electrocardiograph segment is taken as input, a residual network of a fusion channel attention mechanism and a spatial attention mechanism is utilized to obtain a third feature vector, which comprises,
the calculation formula of the channel attention mechanism is that,
，
the calculation formula of the spatial attention mechanism is that,
，
Wherein,representation ofChannel attention matrix, < >>Representing a spatial attention matrix;Fa feature map representing an input;W 1 ，W 0 representing a weight matrix of the multi-layer perceptron;MLPrepresenting a multi-layer perceptron @, @>Representing average pooling>Representing maximum pooling, ++>And->Feature vectors after pooling operation, < + >, respectively>For convolution operation, ++>And->Representing the feature matrix after the channel pooling operation, < + >>Representing an activation function.
7. The system of claim 1, wherein: splicing the second characteristic vector and the third characteristic vector, and performing classified diagnosis by adopting sigmoid, wherein the method comprises the steps of,
splicing the second feature vector and the third feature vector to obtain a fourth feature vector;
and adopting sigmoid to map the fourth feature vector into a prediction probability, and judging whether the result is normal or abnormal according to a preset threshold value so as to realize classification diagnosis.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311657353.4A CN117349600B (en) | 2023-12-06 | 2023-12-06 | Heart sound and heart electricity combined diagnosis method and system based on dual-mode dual input |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311657353.4A CN117349600B (en) | 2023-12-06 | 2023-12-06 | Heart sound and heart electricity combined diagnosis method and system based on dual-mode dual input |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117349600A CN117349600A (en) | 2024-01-05 |
| CN117349600B true CN117349600B (en) | 2024-01-30 |
Family
ID=89371442
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311657353.4A Active CN117349600B (en) | 2023-12-06 | 2023-12-06 | Heart sound and heart electricity combined diagnosis method and system based on dual-mode dual input |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117349600B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118332425B (en) * | 2024-06-11 | 2024-10-18 | 华南师范大学 | Method, device, computer equipment and storage medium for analyzing heart sound signals |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20100128083A (en) * | 2009-05-27 | 2010-12-07 | 충북대학교 산학협력단 | Apparatus and method for diagnosing cardiac disorder |
| CN110537910A (en) * | 2019-09-18 | 2019-12-06 | 山东大学 | coronary heart disease noninvasive screening system based on electrocardio and heart sound signal joint analysis |
| CN115177262A (en) * | 2022-06-13 | 2022-10-14 | 华中科技大学 | Heart sound and electrocardiogram combined diagnosis device and system based on deep learning |
| CN115177260A (en) * | 2022-07-07 | 2022-10-14 | 张福伟 | Intelligent electrocardio-heart sound diagnosis method and device based on artificial neural network |
| CN115640507A (en) * | 2022-12-09 | 2023-01-24 | 南京备力医疗科技有限公司 | Abnormal data screening method based on electrocardio-heart sound joint analysis |
| CN116584902A (en) * | 2023-07-18 | 2023-08-15 | 山东大学 | Heart sound classification device based on feature optimization and visualization |
-
2023
- 2023-12-06 CN CN202311657353.4A patent/CN117349600B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20100128083A (en) * | 2009-05-27 | 2010-12-07 | 충북대학교 산학협력단 | Apparatus and method for diagnosing cardiac disorder |
| CN110537910A (en) * | 2019-09-18 | 2019-12-06 | 山东大学 | coronary heart disease noninvasive screening system based on electrocardio and heart sound signal joint analysis |
| CN115177262A (en) * | 2022-06-13 | 2022-10-14 | 华中科技大学 | Heart sound and electrocardiogram combined diagnosis device and system based on deep learning |
| CN115177260A (en) * | 2022-07-07 | 2022-10-14 | 张福伟 | Intelligent electrocardio-heart sound diagnosis method and device based on artificial neural network |
| CN115640507A (en) * | 2022-12-09 | 2023-01-24 | 南京备力医疗科技有限公司 | Abnormal data screening method based on electrocardio-heart sound joint analysis |
| CN116584902A (en) * | 2023-07-18 | 2023-08-15 | 山东大学 | Heart sound classification device based on feature optimization and visualization |
Non-Patent Citations (3)
| Title |
|---|
| 《Detection of coronary artery disease using multi-modal feature fusion and hybrid feature selection》;Zhang, H 等;《PHYSIOLOGICAL MEASUREMENT》;第41卷(第11期);全文 * |
| 冠心病无创检测新方法及系统设计;高宏建;吴水才;侯立亚;白燕萍;杨春兰;;北京工业大学学报(第05期);全文 * |
| 基于双模态信号集成深度学习的冠心病检测研究;李晗;《中国博士学位论文全文数据库（电子期刊）》医药卫生科技辑(2021年第01期);E062-41 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117349600A (en) | 2024-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220093216A1 (en) | Discovering novel features to use in machine learning techniques, such as machine learning techniques for diagnosing medical conditions | |
| US10869610B2 (en) | System and method for identifying cardiac arrhythmias with deep neural networks | |
| CN110236518B (en) | Electrocardio and heart-shock signal combined classification method and device based on neural network | |
| US20230143594A1 (en) | Systems and methods for reduced lead electrocardiogram diagnosis using deep neural networks and rule-based systems | |
| US20220093215A1 (en) | Discovering genomes to use in machine learning techniques | |
| Li et al. | Application of an EMG interference filtering method to dynamic ECGs based on an adaptive wavelet-Wiener filter and adaptive moving average filter | |
| CN117349600B (en) | Heart sound and heart electricity combined diagnosis method and system based on dual-mode dual input | |
| CN115337018B (en) | Electrocardiogram signal classification method and system based on overall dynamic characteristics | |
| CN111370120B (en) | Heart diastole dysfunction detection method based on heart sound signals | |
| Deng et al. | ST-Net: Synthetic ECG tracings for diagnosing various cardiovascular diseases | |
| CN116049674A (en) | Method and system for generating invasive blood pressure waveform estimation based on countermeasure network | |
| JP2023104885A (en) | Electrocardiographic heart rate multi-type prediction method based on graph convolution | |
| Zhang et al. | Classification of pulmonary arterial pressure using photoplethysmography and bi-directional LSTM | |
| Gopika et al. | Performance improvement of deep learning architectures for phonocardiogram signal classification using fast fourier transform | |
| Liu et al. | Semantic segmentation of qrs complex in single channel ecg with bidirectional lstm networks | |
| Alla et al. | A robust ECG signal enhancement technique through optimally designed adaptive filters | |
| He et al. | Dual attention convolutional neural network based on adaptive parametric ReLU for denoising ECG signals with strong noise | |
| Liu et al. | Energy-efficient intelligent pulmonary auscultation for post COVID-19 era wearable monitoring enabled by two-stage hybrid neural network | |
| Lomoio et al. | DCAE-SR: Design of a Denoising Convolutional Autoencoder for reconstructing Electrocardiograms signals at Super Resolution. | |
| Rodriguez et al. | Embedded system based on an ARM microcontroller to analyze heart rate variability in real time using wavelets | |
| US20220414181A1 (en) | Method and device for translating between stochastic signals | |
| RU2823433C1 (en) | Method of processing and analysing electrocardiogram (ecg) data | |
| Martorano | A hybrid machine learning framework for single-lead ECG signal quality assessment | |
| ROMAISSA | Automated Heartbeat Classification and Cardiovascular Disease Detection Using Deep Learning | |
| Ramesh et al. | Analysis of ECG Noise Cancellation and Abnormality Classification using Various Adaptive Algorithms |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |