CN114820573A - Atrial fibrillation auxiliary analysis method based on semi-supervised learning - Google Patents
Atrial fibrillation auxiliary analysis method based on semi-supervised learning Download PDFInfo
- Publication number
- CN114820573A CN114820573A CN202210563074.0A CN202210563074A CN114820573A CN 114820573 A CN114820573 A CN 114820573A CN 202210563074 A CN202210563074 A CN 202210563074A CN 114820573 A CN114820573 A CN 114820573A
- Authority
- CN
- China
- Prior art keywords
- left atrium
- convolution
- image
- atrial fibrillation
- modules
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 206010003658 Atrial Fibrillation Diseases 0.000 title claims abstract description 32
- 238000004458 analytical method Methods 0.000 title claims abstract description 14
- 210000005246 left atrium Anatomy 0.000 claims abstract description 60
- 230000006870 function Effects 0.000 claims abstract description 41
- 230000011218 segmentation Effects 0.000 claims abstract description 39
- 238000003709 image segmentation Methods 0.000 claims abstract description 31
- 230000000747 cardiac effect Effects 0.000 claims abstract description 27
- 238000000034 method Methods 0.000 claims abstract description 25
- 230000009191 jumping Effects 0.000 claims abstract description 4
- 230000001746 atrial effect Effects 0.000 claims description 22
- 238000010586 diagram Methods 0.000 claims description 18
- 238000000926 separation method Methods 0.000 claims description 16
- 230000004913 activation Effects 0.000 claims description 11
- 238000010606 normalization Methods 0.000 claims description 11
- 238000013507 mapping Methods 0.000 claims description 7
- 238000012549 training Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000014509 gene expression Effects 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 238000002156 mixing Methods 0.000 claims description 5
- 230000002779 inactivation Effects 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 3
- 230000008602 contraction Effects 0.000 claims description 3
- 230000003205 diastolic effect Effects 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 230000006872 improvement Effects 0.000 claims description 3
- 210000004115 mitral valve Anatomy 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 2
- 239000000126 substance Substances 0.000 claims description 2
- 238000012546 transfer Methods 0.000 claims description 2
- 238000013527 convolutional neural network Methods 0.000 description 5
- 238000003745 diagnosis Methods 0.000 description 5
- 210000003484 anatomy Anatomy 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000001575 pathological effect Effects 0.000 description 2
- 208000020446 Cardiac disease Diseases 0.000 description 1
- 206010003119 arrhythmia Diseases 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000036772 blood pressure Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000003759 clinical diagnosis Methods 0.000 description 1
- 230000009849 deactivation Effects 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000002405 diagnostic procedure Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 210000002837 heart atrium Anatomy 0.000 description 1
- 208000019622 heart disease Diseases 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000002559 palpation Methods 0.000 description 1
- 238000013186 photoplethysmography Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000013517 stratification Methods 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 230000036962 time dependent Effects 0.000 description 1
- 230000017105 transposition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/60—Rotation of a whole image or part thereof
-
- G06T5/90—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10088—Magnetic resonance imaging [MRI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30048—Heart; Cardiac
Abstract
The invention relates to an atrial fibrillation auxiliary analysis method based on semi-supervised learning, which comprises the steps of firstly, building an image segmentation model, wherein the image segmentation model comprises an encoder and a decoder, the encoder and the decoder respectively comprise three convolution modules and two multilayer perceptron modules, the corresponding modules of the encoder and the decoder are in jumping connection, and the multilayer perceptron modules move a feature map in three directions of width, height and depth so as to expand the multilayer perceptrons to a three-dimensional direction; designing a loss function to train the image segmentation model, and inputting the cardiac magnetic resonance image into the trained image segmentation model to obtain a left atrium segmentation prediction map; based on the left atrium segmentation prediction graph, four clinical indexes of left atrium volume, strain rate and ejection fraction are calculated, and according to whether the clinical indexes are in a normal reference range or not, a doctor is assisted in analyzing atrial fibrillation. The characteristics that CNN is good at capturing local information and MLP is good at capturing global information are fully utilized, the segmentation precision is improved, and the method has important significance for clinical practice.
Description
Technical Field
The invention relates to the technical field of medical image segmentation, in particular to an atrial fibrillation auxiliary analysis method based on semi-supervised learning.
Background
Atrial fibrillation is the most common cardiac arrhythmia and is a cardiac disorder originating in the atria, and it is estimated that over 3000 thousands of people are affected worldwide, and although the risk of disease can be reduced by appropriate treatment, it is often insidious and difficult to diagnose and intervene in a timely manner. The current diagnostic methods of atrial fibrillation mainly comprise heart palpation, photoplethysmography, blood pressure monitoring vibrometry and electrocardiogram. As most patients mainly suffer from paroxysmal atrial fibrillation, the four diagnosis methods can not accurately capture the attack time of the atrial fibrillation, and have long diagnosis period, high cost, low accuracy and easy subjective influence of doctors.
The anatomical structure of the left atrium provides important information for the pathological research of atrial fibrillation, the volume, strain rate and ejection fraction of the left atrium are independent factors for predicting atrial fibrillation, compared with a healthy group, the left atrium of an atrial fibrillation patient is remarkably increased in volume, strain and strain rate are reduced, and ejection fraction is reduced. The accuracy of these clinical index calculations depends on the accurate delineation of the left atrial contour. With the development of deep learning, researchers propose a plurality of methods for segmenting the left atrium, but most of clinical data are three-dimensional by adopting a two-dimensional segmentation method, and the two-dimensional method training model cannot utilize information among slices, so that part of pathological information is lost. Three-dimensional segmentation methods can utilize not only intra-slice information, but also inter-slice anatomical structure information, which is often a prerequisite for medical diagnosis, patient stratification, and clinical treatment.
The segmentation method of the Convolutional Neural Network (CNN) mainly uses a convolutional kernel or a filter to continuously extract features, and many studies show that the actual receptive field of the features is far smaller than the theoretical receptive field, which is not favorable for fully utilizing context information to capture the features, but the CNN has an advantage in processing local features. The fully connected layers of multi-layer perceptrons (MLPs) are generally better at capturing global contextual information and spatial anatomical relationships, but are not good at capturing local features. The MLP model has better parameter and calculation efficiency than the CNN in the same period, and does not need to have excessive trade-off between performance and efficiency.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to solve the technical problem of providing an atrial fibrillation auxiliary analysis method based on semi-supervised learning; firstly, obtaining a left atrium segmentation prediction image by using an image segmentation model, calculating four clinical indexes of left atrium volume, strain rate and ejection fraction based on the left atrium segmentation prediction image, and comparing the four clinical indexes with a normal reference range to assist a doctor in diagnosing whether atrial fibrillation exists.
The technical scheme adopted by the invention for solving the technical problems is as follows:
an atrial fibrillation aided analysis method based on semi-supervised learning is characterized by comprising the following steps of:
1) the method comprises the steps of building an image segmentation model, wherein the image segmentation model comprises an encoder and a decoder, the encoder and the decoder respectively comprise three convolution modules and two multilayer perceptron modules, and the corresponding modules of the encoder and the decoder are in jumping connection; the convolution module comprises a convolution layer, a normalization layer and an activation function layer, and the multilayer sensor module comprises a multilayer sensor with a moving window in the width direction, a multilayer sensor with a moving window in the height direction, a multilayer sensor with a moving window in the depth direction, a normalization layer, an activation function layer and a random inactivation layer; the multilayer perceptron module moves the feature map in three directions of width, height and depth, and then the multilayer perceptron is expanded to a three-dimensional direction;
2) designing a loss function to train the image segmentation model, wherein the loss function comprises supervision loss and unsupervised uncertainty loss, and inputting a cardiac magnetic resonance image into the trained image segmentation model to obtain a left atrium segmentation prediction map;
3) calculating four clinical indexes of left atrium volume, strain rate and ejection fraction based on the left atrium segmentation prediction graph, and assisting a doctor to analyze atrial fibrillation according to whether the clinical indexes are in a normal reference range;
the formula for the left atrial volume is:
wherein L represents the distance between the posterior wall of the left atrium and the attachment point of the mitral valve, A 1 、A 2 Showing the left atrium in a two-chambered long-axis view and a four-chambered view, respectivelyThe slice area is obtained by multiplying the number of pixel points by the spatial resolution;
drawing a curve of the change of the left atrium volume along with time, and calculating the gradient of the curve, wherein the time period when the gradient is less than 0 indicates that the left atrium continuously contracts, so that the starting time point when the gradient is less than 0 is taken as the end diastole, and the ending time point when the gradient is less than 0 is taken as the end systole;
calculating strain, strain rate and ejection fraction of the left atrium, wherein the strain comprises circumferential strain, total strain, passive strain and active strain, and the strain rate is a derivative of the strain to time; the calculation formula is as follows:
wherein L is t Contour perimeter L representing the left atrial segmentation prediction map at time t 0 Contour perimeter, LA, representing the left atrial segmentation prediction map at the initial time EDV Representing the left atrial end diastolic volume, LA ESV Representing the left atrial end systolic volume, LA Vpre Representing the volume before left atrial contraction;
the four clinical indexes of the left atrium volume, the strain rate and the ejection fraction are compared with a normal reference range, so that the aim of assisting in analyzing atrial fibrillation is fulfilled.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention extracts the left atrium segmentation prediction map through the image segmentation model, calculates four clinical indexes of left atrium volume, strain rate and ejection fraction based on the left atrium segmentation prediction map, compares the indexes with a normal value range, assists a doctor in atrial fibrillation diagnosis, performs intervention treatment on a patient, improves the understanding of atrial fibrillation symptoms and clinical diagnosis, has important significance in clinical practice, and is beneficial to the development of a medical automatic diagnosis technology from a laboratory to clinical application.
2. The image segmentation model consists of a convolution module and a multilayer perceptron module, and fully utilizes the characteristics that CNN is good at capturing local information and MLP is good at capturing global information; the multilayer perceptron module comprises multilayer perceptrons with moving windows in the width direction, the height direction and the depth direction, the characteristic diagram moves in the width direction, the height direction and the depth direction, the multilayer perceptron is further expanded to the three-dimensional direction, the MLP can better learn the global semantic relationship, the number and the calculation complexity of parameters are reduced, better representation is generated, and a lightweight image segmentation model is constructed to help segmentation.
3. The method comprises the following steps of preprocessing an original cardiac magnetic resonance image including traditional data enhancement and image hybrid enhancement to enhance data, wherein the data enhancement plays a crucial role in preventing model overfitting and enhancing the generalization capability of a neural network; the traditional data enhancement method has limited improvement on the model performance, and due to the problems of low contrast, fuzzy boundary, complex left atrium structure and the like of the cardiac magnetic resonance image, the traditional data enhancement method cannot be completely suitable for the cardiac magnetic resonance image, so that the image after the traditional data enhancement is subjected to mixed enhancement, and more information can be provided by the mixed image to promote the training of the model.
4. In the model training process, the uncertainty of the left atrium segmentation prediction graph is calculated, the quality of the left atrium segmentation prediction graph is estimated by using the uncertainty, unreliable left atrium segmentation prediction graphs are filtered out, and the model is retrained by using the screened left atrium segmentation prediction graph, so that the model learns the most reliable information, and the overall uncertainty of the model is reduced.
5. Semi-supervised learning is a method combining supervised learning and unsupervised learning, the semi-supervised learning utilizes a small quantity of marked images and a large quantity of marked images, the dependence of a model on the marked images is reduced, the difficulty of data annotation is relieved to a great extent, and the semi-supervised segmentation performance is equivalent to that of supervised segmentation. In order to supervise the marked image and the unmarked image simultaneously, a loss function combining supervision loss and unsupervised uncertainty loss is provided, the segmentation quality of the network model is better evaluated, and the model is optimized.
Drawings
FIG. 1 is an overall flow chart of the present invention;
FIG. 2 is a flow chart of image pre-processing according to the present invention;
FIG. 3 is a flow chart of image blending enhancement of the present invention;
FIG. 4 is a block diagram of an image segmentation model of the present invention;
FIG. 5 is a block diagram of the convolution module of the present invention;
FIG. 6 is a block diagram of a multi-layer sensor module of the present invention;
FIG. 7 is a block diagram of a multilayer sensor of the present invention having a moving window;
FIG. 8 is a flow chart of an atrial fibrillation analysis of the present invention.
Detailed Description
The technical solutions of the present invention are described in detail below with reference to the accompanying drawings and the detailed description, but the scope of the present invention is not limited thereto.
The invention relates to an atrial fibrillation auxiliary analysis method (a method for short, see figures 1-8) based on semi-supervised learning, which comprises the following steps:
s1, making a data set, wherein the data set consists of a plurality of original cardiac magnetic resonance images and preprocessed cardiac magnetic resonance images;
acquiring an original cardiac magnetic resonance image, and preprocessing the original cardiac magnetic resonance image to realize data expansion; the preprocessing comprises traditional data enhancement and image mixing enhancement, wherein the traditional data enhancement comprises random turning, center cutting and contrast improvement;
the image mixing enhancement is to mix two cardiac magnetic resonance images subjected to traditional data enhancement to obtain a mixed cardiac magnetic resonance image and a corresponding label; firstly, randomly selecting an original cardiac magnetic resonance Image1 for traditional data enhancement, wherein the Label of the original cardiac magnetic resonance Image1 is Label1, and obtaining an enhanced cardiac magnetic resonance Image1 'and a Label Label 1'; then, randomly selecting another original cardiac magnetic resonance Image2 for traditional data enhancement, wherein the Label of the original cardiac magnetic resonance Image2 is Label2, and obtaining an enhanced cardiac magnetic resonance Image2 'and a Label Label 2'; finally, the enhanced cardiac magnetic resonance images Imagel 'and Image2' are Mixed, namely the pixel values of the enhanced cardiac magnetic resonance images Image1 'and Image2' are averaged to obtain a Mixed cardiac magnetic resonance Image and a corresponding Label Mixed Label;
s2, constructing an image segmentation model for left atrium segmentation;
as shown in fig. 4, the image segmentation model is based on a VNet network and includes an encoder and a decoder, where the encoder includes three convolution modules and two multilayer perceptron modules, and the decoder also includes three convolution modules and two multilayer perceptron modules, and the corresponding modules of the encoder and the decoder are connected in a jumping manner, the encoder is a compression path, and the decoder is a decompression path; the encoder performs downsampling by convolution, and reduces the resolution of the feature map to one half of the resolution of the input image; the decoder block adopts transposition convolution to carry out up-sampling, and the resolution of the compressed feature image is increased by 2 times; the convolution module comprises a convolution layer, a normalization layer and an activation function layer, and the multilayer sensor module comprises a multilayer sensor with a moving window, a normalization layer, an activation function layer and a random deactivation (dropout) layer;
the method comprises the steps of performing downsampling on all modules of an encoder by adopting convolution with the step length stride being 2 and the convolution kernel filters being 2 multiplied by 2, halving the resolution of a feature map, and replacing pooling by utilizing the convolution, so that the memory occupied by model training is smaller; the method comprises the following steps of performing up-sampling by adopting transposed convolution with the step length stride being 2 and the filters being 2 multiplied by 2 among all modules of a decoder, and doubling the resolution of a compressed feature map; transferring modules with the same resolution in the encoding path to the decoding path through a jump connection to provide an original high-resolution feature map for the corresponding modules in the decoder; the three convolution modules of the encoder sequentially comprise 1, 2 and 3 convolution layers, the convolution kernel filters of each convolution layer are 5 multiplied by 5, an input image is subjected to forward propagation along with different stages of a compression path, the semantic distinguishing characteristics of feature mapping are enhanced layer by layer, and the resolution ratio is gradually reduced; the three convolution modules of the decoder sequentially comprise 3, 2 and 1 convolution layers; the normalization layer uses a Group Norm (GN) function instead of a Batchnorm function, so that the problem that the Batchnorm function has poor effect on smaller batch sizes (batch sizes) is solved; the activation function layer uses the GELU function to replace the ReLU function, and the GELU function introduces a random regularization idea on the basis of the ReLU function, so that the image segmentation model has better generalization capability;
the multilayer perceptron module comprises a multilayer perceptron with a moving window in the width direction, a multilayer perceptron with a moving window in the height direction and a multilayer perceptron with a moving window in the depth direction, the feature map is moved in the width direction, the height direction and the depth direction, and then the multilayer perceptron (MLP) is expanded to the three-dimensional direction, so that the MLP can better learn the global semantic relationship;
first, the feature map X is moved in the width direction, and the moved feature map X is used shift Mapping to a separation profile T W And then the separation characteristic diagram T of the multilayer perceptron pair is utilized W Processing, and then transferring the processed feature map to a multilayer perceptron with a moving window in the height direction through a depth separable convolution (DWConv) to obtain a feature map Y; moving the feature diagram Y in the height direction, and moving the moved feature diagram Y shift Mapping to a separation profile T H Separation profile T H The characteristic image Z is obtained after being processed by the multilayer perceptron and transmitted to the multilayer perceptron with a moving window in the depth direction; the feature map Z is arranged in depthMoving in the direction, and mapping the moved characteristic diagram Z into a separation characteristic diagram T D Separation profile T D After being processed by the multilayer perceptron, the data sequentially pass through a normalization layer, an activation function layer and a random inactivation layer, and then residual connection is carried out to obtain an output characteristic diagram of the multilayer perceptron module, and the output characteristic diagram is transmitted to the next module; the expression of the multi-layer perceptron module is:
Y＝f(DWConv(MLP(T W ))) (2)
Z＝f(MLP(T H )) (4)
Y＝f(T+dropout(GELU(GN(MLP(T D ))))) (6)
wherein T represents a separation characteristic diagram, Shift W (·)、Shift H (. and Shift) D (. cndot.) represents a movement function in the width, height, and depth directions, respectively, W represents the width, H represents the height, D represents the depth, Tokenize (. cndot.) represents a function mapped to a separation profile, MLP (. cndot.) represents a multi-layer perceptron process function, f (. cndot.) represents a transfer function, GN (. cndot.) represents a normalization function, GELU (. cndot.) represents a GELU activation function, and dropout (. cndot.) represents an overfitting process function;
s3, designing a loss function to train the image segmentation model, and inputting the cardiac magnetic resonance image into the trained image segmentation model to obtain a left atrium segmentation prediction map; calculating the uncertainty of the left atrium segmentation prediction graph, wherein the lower the uncertainty is, the better the effect of the left atrium segmentation prediction graph is, filtering the left atrium segmentation prediction graph with the uncertainty higher than an uncertainty threshold value, retraining the image segmentation model by using the left atrium segmentation prediction graph with the uncertainty lower than the uncertainty threshold value to obtain a trained image segmentation model, and obtaining the left atrium segmentation prediction graph by using the trained image segmentation model;
selecting prediction entropy as a measure, and approximately estimating uncertainty, wherein formula expressions are shown as formulas (7) and (8);
wherein the content of the first and second substances,
denotes the probability that the kth prediction is of class c, P c Expressing the probability of predicting as a class c, and K expressing the prediction times; p represents the uncertainty, which is estimated at the voxel level, { P }. belonging to R H*W*D ；
The loss function comprises supervision loss and unsupervised uncertainty loss, and is shown as formula (9);
L＝L S +λL C (9)
wherein L is S Denotes loss of supervision, L C Represents an unsupervised uncertainty loss, and λ represents a coefficient controlling the balance between the supervised loss and the unsupervised uncertainty loss;
the expression for the loss of supervision is:
wherein cross _ entropy represents cross entropy loss, and Dice _ loss represents Dice loss;
the expression for unsupervised uncertainty loss is:
wherein g (-) represents an indicator function, f v 、f‘ v Respectively representing the predictions at the v-th voxel of the pre-and post-perturbation-added image segmentation models, P v Representing an uncertainty at the v-th voxel, the slice representing an uncertainty threshold;
the balance between supervision loss and unsupervised uncertainty loss is controlled by using a time-dependent Gaussian function lambda (t'), the design can ensure that the loss function is dominated by the supervision loss at the beginning, and the degradation of an image segmentation model is avoided, wherein the expression is as follows:
where t' denotes the current training step, t max Representing a maximum training step;
s4, calculating four clinical indexes of left atrium volume, strain rate and ejection fraction based on the left atrium segmentation prediction graph, and assisting a doctor to analyze atrial fibrillation according to whether the clinical indexes are in a normal reference range;
the time from the diastole to the systole of the left atrium is defined as a complete cardiac cycle in medicine, and the correct identification of the time point of the diastole end and the systole end of the left atrium is the key of atrial fibrillation analysis; the common knowledge can know that the volume of the left atrium of the human body reaches the maximum value at the end diastole and reaches the minimum value at the end systole, so that the time points of the end diastole and the end systole of the left atrium can be accurately determined by calculating the volume of the left atrium and analyzing the trend of the left atrium along with the time;
the formula for the left atrial volume is:
wherein L represents the distance between the posterior wall of the left atrium and the attachment point of the mitral valve, A 1 、A 2 Respectively show the long axis view and the four of the two chambersThe slicing area of the left atrium in the chamber view is obtained by multiplying the number of pixel points by the spatial resolution;
drawing a curve of the left atrial volume with time, smoothing the curve by using Gaussian filtering, and removing local minimum and maximum values in the curve; calculating the gradient of the curve, wherein the time period of the gradient less than 0 indicates that the left atrium continues to contract, so that the starting time point of the gradient less than 0 is taken as the end diastole, and the ending time point of the gradient less than 0 is taken as the end systole;
after determining the time points of the left atrial end diastole and the end systole, the left atrial end diastole volume LA is respectively read from the change curve of the left atrial volume along with the time EDV Left atrial end systolic volume LA ESV And the volume LA before contraction of the left atrium Vpre For calculating the left atrial strain, strain rate and ejection fraction; strain includes circumferential strain, total strain, passive strain and active strain, strain rate is the derivative of strain with time; ejection fraction is a clinical indicator related to the left atrial end systolic volume and end diastolic volume; the calculation formula is as follows:
wherein L is t Contour perimeter L representing the left atrial segmentation prediction map at time t 0 The contour perimeter of the left atrium segmentation prediction map at the initial moment is shown;
the doctor compares the four clinical indexes of the left atrium volume, the strain rate and the ejection fraction with a normal reference range, and combines the experience to diagnose whether the atrial fibrillation exists or not, so that the aim of assisting in analyzing the atrial fibrillation is fulfilled.
Nothing in this specification is said to apply to the prior art.
Claims (5)
1. An atrial fibrillation aided analysis method based on semi-supervised learning is characterized by comprising the following steps of:
1) the method comprises the steps of building an image segmentation model, wherein the image segmentation model comprises an encoder and a decoder, the encoder and the decoder respectively comprise three convolution modules and two multilayer perceptron modules, and the corresponding modules of the encoder and the decoder are in jumping connection; the convolution module comprises a convolution layer, a normalization layer and an activation function layer, and the multilayer sensor module comprises a multilayer sensor with a moving window in the width direction, a multilayer sensor with a moving window in the height direction, a multilayer sensor with a moving window in the depth direction, a normalization layer, an activation function layer and a random inactivation layer; the multilayer perceptron module moves the feature map in three directions of width, height and depth, and then the multilayer perceptron is expanded to a three-dimensional direction;
2) designing a loss function to train the image segmentation model, wherein the loss function comprises supervision loss and unsupervised uncertainty loss, and inputting a cardiac magnetic resonance image into the trained image segmentation model to obtain a left atrium segmentation prediction map;
3) calculating four clinical indexes of left atrium volume, strain rate and ejection fraction based on the left atrium segmentation prediction graph, and assisting a doctor to analyze atrial fibrillation according to whether the clinical indexes are in a normal reference range;
the formula for the left atrial volume is:
wherein L represents the distance between the posterior wall of the left atrium and the attachment point of the mitral valve, A 1 、A 2 Slice areas of the left atrium in a two-chamber long axis view and a four-chamber view, respectively;
drawing a curve of the change of the left atrium volume along with time, and calculating the gradient of the curve, wherein the time period when the gradient is less than 0 indicates that the left atrium continuously contracts, so that the starting time point when the gradient is less than 0 is taken as the end diastole, and the ending time point when the gradient is less than 0 is taken as the end systole;
calculating strain, strain rate and ejection fraction of the left atrium, wherein the strain comprises circumferential strain, total strain, passive strain and active strain, and the strain rate is a derivative of the strain to time; the calculation formula is as follows:
wherein L is t Contour perimeter L representing the left atrial segmentation prediction map at time t 0 Contour perimeter, LA, representing the left atrial segmentation prediction map at the initial time EDV Representing the left atrial end diastolic volume, LA ESV Indicating the left atrial end systolic volume, LAV pre Representing the volume before left atrial contraction;
the four clinical indexes of the left atrium volume, the strain rate and the ejection fraction are compared with a normal reference range, so that the aim of assisting in analyzing atrial fibrillation is fulfilled.
2. The atrial fibrillation aided analysis method based on semi-supervised learning of claim 1, wherein the processing procedure of the multi-layer perceptron module is as follows:
first, the feature map X is moved in the width direction, and the moved feature map X is used shift Mapping to a separation profile T W And then the separation characteristic diagram T of the multilayer perceptron pair is utilized W Processing, and then transferring the processed feature map to a multilayer perceptron with a moving window in the height direction through a depth separable convolution (DWConv) to obtain a feature map Y; moving the feature diagram Y in the height direction, and moving the moved feature diagram Y shift Mapping to a separation profile T H Separation profile T H The characteristic image Z is obtained after being processed by the multilayer perceptron and transmitted to the multilayer perceptron with a moving window in the depth direction; moving the feature map Z in the depth direction, and mapping the moved feature map Z into a separation feature map T D Separation profile T D After being processed by the multilayer perceptron, the data sequentially pass through a normalization layer, an activation function layer and a random inactivation layer, and then residual connection is carried out to obtain an output characteristic diagram of the multilayer perceptron module, and the output characteristic diagram is transmitted to the next module; the expression of the multi-layer perceptron module is:
Y＝f(DWConv(MLP(T W ))) (2)
Z＝f(MLP(T H )) (4)
Y＝f(T+dropout(GELU(GN(MLP(T D ))))) (6)
wherein, Shift W (·)、Shift H (. and Shift) D (. cndot.) represents a movement function in the width, height, and depth directions, respectively, W represents the width, H represents the height, D represents the depth, Tokenize (. cndot.) represents a function mapped to a separation feature map, MLP (. cndot.) represents a multi-layer perceptron process function, f (. cndot.) represents a transfer function, GN (. cndot.) represents a normalization function, GELU (. cndot.) represents a GELU activation function, dropout (. cndot.) represents an over-fit process function, and T represents a separation feature map.
3. The atrial fibrillation aided analysis method based on semi-supervised learning of claim 1 or 2, wherein the downsampling is carried out among the modules of the encoder by adopting convolution with the step size of 2 and the convolution kernel of 2 x 2; the method comprises the following steps that (1) transposed convolution with the step length of 2 and the convolution kernel of 2 multiplied by 2 is adopted among all modules of a decoder for up-sampling; the three convolution modules of the encoder sequentially comprise 1, 2 and 3 convolution layers, the three convolution modules of the decoder sequentially comprise 3, 2 and 1 convolution layers, and the convolution kernels of the convolution layers of the encoder and the decoder are 5 multiplied by 5.
4. The atrial fibrillation aided analysis method based on semi-supervised learning of claim 1, wherein the image segmentation model training process further comprises calculating the uncertainty of a left atrium segmentation prediction map, filtering the left atrium segmentation prediction map with the uncertainty higher than an uncertainty threshold, and retraining the image segmentation model by using the left atrium segmentation prediction map with the uncertainty lower than the uncertainty threshold to obtain the trained image segmentation model;
selecting the prediction entropy as a measure, and approximately estimating uncertainty, as shown in formulas (7) and (8);
wherein the content of the first and second substances,
denotes the probability that the kth prediction is of class c, P c Expressing the probability of predicting as a class c, and K expressing the prediction times; p represents uncertainty.
5. The atrial fibrillation aided analysis method based on semi-supervised learning of claim 1, wherein the method further comprises the steps of acquiring original cardiac magnetic resonance images, preprocessing the original cardiac magnetic resonance images and realizing data expansion; the preprocessing comprises traditional data enhancement and image mixing enhancement, wherein the traditional data enhancement comprises random turning, center cutting and contrast improvement; and the image mixing and enhancing is to average and mix the pixel values of the two cardiac magnetic resonance images subjected to the traditional data enhancement to obtain a mixed cardiac magnetic resonance image and a corresponding label.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210563074.0A CN114820573A (en) | 2022-05-18 | 2022-05-18 | Atrial fibrillation auxiliary analysis method based on semi-supervised learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210563074.0A CN114820573A (en) | 2022-05-18 | 2022-05-18 | Atrial fibrillation auxiliary analysis method based on semi-supervised learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114820573A true CN114820573A (en) | 2022-07-29 |
Family
ID=82517836
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210563074.0A Pending CN114820573A (en) | 2022-05-18 | 2022-05-18 | Atrial fibrillation auxiliary analysis method based on semi-supervised learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114820573A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116977330A (en) * | 2023-09-21 | 2023-10-31 | 天津医科大学总医院 | Atrial fibrillation auxiliary analysis method based on pulse neural network and context awareness |
| CN117011315A (en) * | 2023-10-07 | 2023-11-07 | 中国人民解放军总医院第二医学中心 | LGE MRI left atrial scar segmentation system and method |
-
2022
- 2022-05-18 CN CN202210563074.0A patent/CN114820573A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116977330A (en) * | 2023-09-21 | 2023-10-31 | 天津医科大学总医院 | Atrial fibrillation auxiliary analysis method based on pulse neural network and context awareness |
| CN116977330B (en) * | 2023-09-21 | 2023-12-08 | 天津医科大学总医院 | Atrial fibrillation auxiliary analysis method based on pulse neural network and context awareness |
| CN117011315A (en) * | 2023-10-07 | 2023-11-07 | 中国人民解放军总医院第二医学中心 | LGE MRI left atrial scar segmentation system and method |
| CN117011315B (en) * | 2023-10-07 | 2023-12-29 | 中国人民解放军总医院第二医学中心 | LGE MRI left atrial scar segmentation system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220148191A1 (en) | Image segmentation method and apparatus and storage medium | |
| CN114820573A (en) | Atrial fibrillation auxiliary analysis method based on semi-supervised learning | |
| CN110619322A (en) | Multi-lead electrocardio abnormal signal identification method and system based on multi-flow convolution cyclic neural network | |
| WO2010117576A2 (en) | Image processing and machine learning for diagnostic analysis of microcirculation | |
| CN111248882B (en) | Method and device for predicting blood pressure | |
| CN111134662A (en) | Electrocardio abnormal signal identification method and device based on transfer learning and confidence degree selection | |
| Militante et al. | Pneumonia detection through adaptive deep learning models of convolutional neural networks | |
| WO2022121100A1 (en) | Darts network-based multi-modal medical image fusion method | |
| EP4108164A1 (en) | Blood pressure prediction method and device | |
| CN109192305A (en) | A kind of heart function automatic analysis method based on deep-cycle neural network | |
| CN112001122A (en) | Non-contact physiological signal measuring method based on end-to-end generation countermeasure network | |
| Rath et al. | Improved heart disease detection from ECG signal using deep learning based ensemble model | |
| CN115908463A (en) | 3D coronary artery image segmentation method based on semi-supervised consistency learning | |
| CN111584072A (en) | Neural network model training method suitable for small samples | |
| CN116977330B (en) | Atrial fibrillation auxiliary analysis method based on pulse neural network and context awareness | |
| CN110827255A (en) | Plaque stability prediction method and system based on coronary artery CT image | |
| CN113436128B (en) | Dual-discriminator multi-mode MR image fusion method, system and terminal | |
| CN112614112B (en) | Segmentation method for stripe damage in MCSLI image | |
| CN111145901B (en) | Deep venous thrombosis thrombolytic curative effect prediction method and system, storage medium and terminal | |
| CN114938952A (en) | Hemoglobin concentration prediction method based on prior causal model | |
| Xia et al. | AwCPM-Net: a collaborative constraint GAN for 3D coronary artery reconstruction in intravascular ultrasound sequences | |
| CN115472258A (en) | Method for generating MRI (magnetic resonance imaging) image and predicting curative effect after breast cancer neoadjuvant chemotherapy | |
| Zhu | Estimating left ventricular volume with ROI-based convolutional neural network | |
| CN114767163A (en) | Intelligent diagnosis system for congenital heart disease of children based on echocardiogram | |
| CN114431873A (en) | Method for constructing electrocardiogram heart rate multi-type prediction model based on graph convolution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |