WO2023082416A1 - 一种基于深度学习的心房颤动评估方法和装置 - Google Patents

一种基于深度学习的心房颤动评估方法和装置 Download PDF

Info

Publication number
WO2023082416A1
WO2023082416A1 PCT/CN2021/138551 CN2021138551W WO2023082416A1 WO 2023082416 A1 WO2023082416 A1 WO 2023082416A1 CN 2021138551 W CN2021138551 W CN 2021138551W WO 2023082416 A1 WO2023082416 A1 WO 2023082416A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
atrial
atrium
images
dimensional
Prior art date
Application number
PCT/CN2021/138551
Other languages
English (en)
French (fr)
Inventor
黄建龙
吴剑煌
杨可
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Publication of WO2023082416A1 publication Critical patent/WO2023082416A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30048—Heart; Cardiac

Definitions

  • the present invention relates to the technical field of medical image processing, and more specifically, to a method and device for evaluating atrial fibrillation based on deep learning.
  • Atrial fibrillation is the most common sustained cardiac arrhythmia and has many complications, such as hypertension, heart failure and coronary heart disease.
  • the function of atrial remodeling is an important pathophysiological mechanism for exploring AF.
  • the main mechanisms of atrial remodeling include electrical remodeling, structural remodeling, metabolic remodeling, and autonomic remodeling. These remodeling initially compensate and maintain cardiac function but may develop into maladaptive changes leading to progressive pump failure and malignant arrhythmias, among others. Therefore, an in-depth understanding of the size and function of the atria and their remodeling mechanisms could provide new insights into the impact of the atria, therapeutic strategies for AF, and important information on cardiovascular disease prognosis and risk class. Since the left atrium is an important part of the heart structure, this article mainly uses the left atrium as an example for illustration.
  • Non-invasive medical imaging is an indispensable technology in the cardiovascular field.
  • MRI magnetic resonance imaging
  • improved algorithms for processing medical imaging make MRI an excellent visualization tool for assessing atrial disease.
  • feature extractors such as AlexNet, VGGNet, and GoogLeNet.
  • the purpose of the present invention is to overcome the defects of the above-mentioned prior art, and provide a method and device for evaluating atrial fibrillation based on deep learning.
  • a method for evaluating atrial fibrillation based on deep learning includes the following steps:
  • the basic feature map is input to the trained U-Net network for image segmentation
  • a device for assessing atrial fibrillation based on deep learning includes:
  • Image acquisition unit of different scales used to obtain multiple atrium images with different ambiguities by using Gaussian convolution operation for the acquired original atrium image;
  • Feature extraction unit for the atrium images with different ambiguities, use the convolutional neural network to extract feature maps to obtain corresponding multiple feature maps, and use the channel weighting module to fuse the multiple feature maps to obtain basic feature maps picture;
  • Image segmentation unit for inputting the basic feature map to the trained U-Net network for image segmentation
  • An image reconstruction unit used for reconstructing a three-dimensional atrium model in a complete cardiac cycle based on the obtained segmented images.
  • the advantage of the present invention is that a novel U-Net with Gaussian blur and channel weights is designed to automatically segment the atrial region of the atrial images of AF patients. After Gaussian blurring, images with different resolutions are obtained. The high-resolution image clearly shows the details of the atrium, while the low-resolution image clearly shows the overall outline of the atrium, thereby solving the problem of few medical image features.
  • Fig. 1 is the flowchart of the atrial fibrillation assessment method based on deep learning according to one embodiment of the present invention
  • Fig. 2 is a comparison result of the same image at different Gaussian scales according to an embodiment of the present invention
  • Fig. 3 is a schematic diagram of an image segmentation framework according to an embodiment of the present invention.
  • Figure 4 is a three-dimensional view of a whole heart according to one embodiment of the present invention.
  • Fig. 5 is a schematic diagram of comparing segmentation results of eight sample image slices with relevant ground truth values according to one embodiment of the present invention
  • Fig. 6 is a schematic diagram of reconstructing a three-dimensional left atrium model from a two-dimensional left atrium image according to an embodiment of the present invention
  • FIG. 7 is a 3D diagram of the left atrium reconstructed from different points of the cardiac cycle of an AF patient according to an embodiment of the present invention.
  • the present invention proposes a technical solution for automatically segmenting atrium regions based on convolutional neural networks, wherein the convolutional neural networks can be of various types.
  • U-Net provides a fully convolutional network model for semantic segmentation tasks, which has shown satisfactory performance in image segmentation.
  • U-Net network is used as an example to illustrate, Its input is an arbitrary image, and the output is a segmentation result image of the same size as the input, and then uses the segmentation information to obtain the semantics of the image.
  • a dynamically changing 3D left atrium model is reconstructed based on MRI and U-Net segmentation technology to explore the clinical value of artificial intelligence (AI) in cardiac diagnosis.
  • AI artificial intelligence
  • the purpose of the present invention is to reconstruct a 3D left atrium model in a complete cardiac cycle using AI-based segmentation results based on visual analysis, suitable for AF patients.
  • the present invention improves the U-Net network, and in this paper, the improved network model is called GCW-UNet.
  • the provided method for assessing atrial fibrillation based on deep learning includes the following steps.
  • Step S110 using Gaussian convolution to extract atrial MRI images with different ambiguities.
  • CNNs are capable of extracting various features from MRI images.
  • image features of different scales are extracted based on Gaussian convolution.
  • Gaussian convolution blurs the original image in a large scale and reduces corresponding detail features.
  • small-scale images retain more details.
  • CNN will extract the overall outline and detail features of the foreground.
  • Gaussian convolution small-scale images and large-scale images can be fused into a feature map, which can fuse the detailed features and global features of MRI.
  • the equation for a two-dimensional Gaussian convolution is expressed as:
  • the parameters of the Gaussian convolution kernel follow the normal distribution law, and the convolution kernel is related to the blur degree of the image.
  • the feature matrix M(x,y) obtained by Gaussian convolution is expressed as:
  • I(x, y) is the original MRI image
  • (x, y) represents the coordinates of the pixel.
  • Figure 2 is a comparison of atrial MRI images with different scales of Gaussian convolution, where Figure 2(a) represents the original image, Figure 2(b) represents the small-scale blur, and Figure 2(c) represents the large-scale image, and the corresponding Different Gaussian convolution kernel sizes. It should be understood that, by setting the size of the convolution kernel, images with different degrees of blur can be obtained, not limited to the two types of small-scale and large-scale.
  • Step S120 for images with different degrees of ambiguity, use a convolutional neural network to extract corresponding feature maps.
  • the GCW-UNet model includes two stages. In the first stage, three MRI images with different blurs are obtained by Gaussian convolution operation. For large-scale blurred images, the overall contour features of the image are preserved, and for small-scale images, the detailed features of the image are preserved. Then, three different feature maps are obtained by CNN, and these three feature maps are concatenated together.
  • the feature maps are input to the channel weight (CW) module.
  • the feature map of the input channel weight module is first computed by global average pooling (GAP).
  • GAP global average pooling
  • the one-dimensional feature vector is calculated by 1 ⁇ 1 convolution and ReLU activation function to reduce the number of channels of the one-dimensional vector, thereby reducing the calculation amount of network parameters.
  • the dimension of the original one-dimensional vector is restored by 1 ⁇ 1 convolution and ReLU activation function, which strengthens the correlation of each element of the one-dimensional vector.
  • the softmax function makes each element value between 0 and 1 of a one-dimensional vector called channel weights. The channel weights are then multiplied with the feature map to obtain the base feature map.
  • the improved U-Net is used for image segmentation.
  • the improvement is that the original copy and crop module (copy and crop) of U-Net is replaced by the channel weight module.
  • the channel weight module is a channel attention mechanism, which can multiply each channel of the feature map by a different weight, and the weight is related to the importance of the channel.
  • the left part of the network is used for feature extraction (downsampling), and the right part is used for upsampling.
  • Such structures are also known as encoders and decoders.
  • the size and resolution of the input feature matrix will be reduced.
  • U-net employs a deconvolution operation layer, as shown in Figure 3.
  • the GCW-UNet framework includes two stages. In the first stage, the basic feature map is obtained through Gaussian blur and channel weighting modules. In the second stage, the basic feature map is down-sampled and up-sampled to obtain Get prediction segmentation results
  • the U-Net network does not contain a channel weight module, but passes the layers of the same resolution in the encoding path to the decoding path through skip connections, providing it with original high-resolution features,
  • the channel weight module is used to replace the skip connection. In this way, the details of the MRI image can be preserved and the channel weight can be adapted, thereby enhancing the atrial segmentation ability of the network.
  • the traditional neural network due to the high similarity between the background (the area outside the left atrium) and the foreground (the left atrium) in the MRI image, the traditional neural network usually misclassifies the background as the foreground, and the GCW-UNet provided by the present invention can directly fuse the local Features and global features, and adaptive channel weights, solve the problem of unbalanced foreground and background pixel numbers and the need to extract detailed features, making atrial edge segmentation more accurate.
  • Step S130 training a model with a set loss function for image segmentation.
  • Atrial segmentation is a dichotomous problem, which is equivalent to separating the foreground from the background.
  • the left atrium is the foreground, and the outside of the left atrium is divided into the background. Since the number of pixels in the left atrium region is unbalanced with the number of background pixels, the CNN tends to recognize the foreground part as the background.
  • the Dice loss function is used for the training of the segmentation network of the present invention.
  • the Dice coefficient is derived from the overlap of two samples and ranges from 0 to 1. The equation for this coefficient is expressed as:
  • the training process refers to using the sample data set to train the GCW-UNet model to obtain model parameters (such as weight and bias, etc.).
  • the application process refers to using the trained The model segments the target atrium actually collected to obtain a two-dimensional (2D) segmented image.
  • the training process and the application process are basically the same, and will not be repeated here.
  • Step S140 reconstructing the three-dimensional atrium structure based on the image segmentation result.
  • FIG. 4 By reconstructing 2D cardiac images into 3D structures, cardiac data can be viewed more comprehensively.
  • the 3D image clearly shows the overall structure of the heart, as shown in Figure 4.
  • Three-dimensional atrial reconstruction can shorten examination time, allowing medical experts to accurately understand the physiological changes of great vessels and atria.
  • Three-dimensional atrial reconstruction is based on 2D image slices. For example, each MRI slice with 25 time frames is randomly selected in the test set for segmentation.
  • the left atrium is reconstructed into a 3D image structure using the segmented 2D atrial image.
  • the atrial structure is perfectly displayed.
  • Left atrium visualization technology provides an important reference for the clinical diagnosis of AF. For example, based on different time frames of a heartbeat, the dynamic changes of the left atrium structure within a cardiac cycle can be presented to better assist the assessment of atrial fibrillation in heart disease patients.
  • the reconstruction process from a 2D image to a 3D image can also be implemented using a deep learning model, that is, the deep learning model is trained using the known correspondence between 2D image slices and 3D image structures.
  • the first part of the present invention uses the convolutional neural network (GCW-UNet) to segment the left atrium area, and the second part reconstructs the two-dimensional left atrium into a three-dimensional model, and the end-to-end automatic reconstruction can be realized by integrating these two parts , which automates the process from raw MRI input to reconstructed 3D left atrium output.
  • GCW-UNet convolutional neural network
  • the present invention also provides a device for assessing atrial fibrillation based on deep learning.
  • the device includes: an image acquisition unit of different scales, which is used to obtain a plurality of atrial images with different ambiguities by using Gaussian convolution operation for the acquired original atrial images; a feature extraction unit, which is used for the atrial images with different ambiguities image, using a convolutional neural network to extract a feature map, obtaining a plurality of corresponding feature maps, and using a channel weighting module to fuse the multiple feature maps to obtain a basic feature map; an image segmentation unit, which is used to combine the basic features The map is input to the trained U-Net network for image segmentation; the image reconstruction unit is used to reconstruct a three-dimensional atrium model in a complete cardiac cycle based on the obtained segmented image.
  • MRIs of AF patients were collected using a Siemens Avanto, 1.5 Tesla, model-syngo MRB15 scanner and Numaris-4 software.
  • the MRI equipment was able to acquire multiple heart slices, which provided the data for training the neural network for automatic segmentation. Since the heart chamber area occupies a portion of the entire MRI, the size of the image is reduced to 288 x 288 (pixels).
  • Real images are manually segmented by experienced radiologists.
  • the segmented left atrium region is a binary image. The left atrium is used as the foreground, and the gray value of the foreground is 255. The area outside the left atrium is the background, and the gray value of the background is 0.
  • the rectified linear function prevents vanishing gradients. Therefore, ReLU is used as the activation function, and Stochastic Gradient Descent (SGD) is used as the optimizer.
  • the initial learning rate is set to 0.001, and the learning rate is reduced every 5 epochs.
  • data augmentation performs flipping, resizing, and warping operations to randomly match a portion of the MRI slice.
  • the batch size is 4. Dice loss is used as the loss function.
  • the training and testing of GCW-UNet are implemented on the Windows system, GPU using RTX 2070 8G and CPU using Intel(R) Core(TM) i7-5500U@2.40GHz.
  • GCW-UNet is implemented using the Keras framework. After 100 epochs, the model reaches the convergence condition.
  • Jaccard index and Dice similarity coefficient were used, where TP is true positive, FP is false positive, TN is true negative, and FN is false negative.
  • the Jaccard index is an important index to measure the results of semantic segmentation.
  • the Jaccard indicator is equivalent to the intersection over union (IOU), indicating the proportion of the intersection of the predicted value and the actual value in the union.
  • the Jaccard indicator is expressed as:
  • the Dice similarity coefficient is an evaluation index, which measures the segmentation result by the proportion of the intersection in the joint set.
  • the Dice similarity coefficient is expressed as:
  • Table 1 compares the state-of-the-art left atrium segmentation method with the GCW UNET method of the present invention.
  • the left atrium segmentation of the present invention is closer to the manual segmentation of experienced radiologists.
  • the average Dice similarity coefficient reaches 93.57%.
  • the invention improves the segmentation accuracy and obtains the best performance.
  • the method proposed by the present invention successfully reconstructs the left atrium model of AF patients in the whole cardiac cycle.
  • a 3D left atrium is reconstructed from a series of 2D left atrium images, where four slices are binary images predicted by the network, and arrows indicate the direction of blood flow in the atrium.
  • This reconstructed 3D left atrium can help medical experts observe the size changes of the left atrium from a physiological point of view, combine with electrocardiogram to diagnose AF and study the diagnosis results.
  • the dynamically reconstructed 3D left atrium can allow medical experts to better understand the complex structure of the left atrium, determine the impact of AF disease and its impact on left atrial structure, and be used to evaluate atrial myopathy.
  • AF stimulates left atrial remodeling mechanisms, including atrial structural remodeling.
  • Structural remodeling of the left atrium mainly manifests as decreased atrial contractility, atrial enlargement, ultrastructural changes of atrial myocytes, and atrial fibrosis.
  • Fibrosis is the most prominent manifestation of structural remodeling due to arrhythmia, and left atrial enlargement is the main feature of structural remodeling. Therefore, the reconstruction of the 3D structure of the left atrium provides an effective basis for diagnosis. By observing the reconstructed 3D left atrium in one cardiac cycle, it can be found that the size of the atrium in patients with atrial fibrillation changes irregularly, as shown in Figure 7.
  • ECG electrocardiogram
  • Figure 7 is a 3D reconstruction of the left atrium at different points in the cardiac cycle from an AF patient.
  • Frames 1-5 are the blood flow from the left atrium through the mitral valve into the left ventricle, which is the process of atrial contraction.
  • Frames 5-15 are the blood flow from the left and right pulmonary veins into the left atrium, which is the process of atrial diastole. Therefore, 1-15 frames of 3D atria constitute a complete cardiac cycle.
  • left atrial structures were identified and constructed. It can be proved that in the cardiac cycle of AF patients, there are obvious irregularities in the size of the dynamic 3D left atrium model, and the left atrium segmented by the present invention is closer to the left atrium segmented manually by an experienced radiologist.
  • the present invention reviews and studies the application of medical imaging and AI-based computer-aided design and reconstruction techniques in AF diagnosis.
  • a 3D left atrium model was reconstructed by GCW-UNet, which solved the problem of fewer features in MRI images, and replaced skip connections to preserve the details of MRI images.
  • the present invention also reconstructs the range of the 3D left atrium in the cardiac cycle, so that the changes in the size of the left atrium in the cardiac cycle of AF patients can be observed. It should be noted that the present invention can also be used for processing other medical images, such as CT images.
  • the present invention can be a system, method and/or computer program product.
  • a computer program product may include a computer readable storage medium having computer readable program instructions thereon for causing a processor to implement various aspects of the present invention.
  • a computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device.
  • a computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Computer-readable storage media include: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded device, such as a printer with instructions stored thereon A hole card or a raised structure in a groove, and any suitable combination of the above.
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • flash memory static random access memory
  • SRAM static random access memory
  • CD-ROM compact disc read only memory
  • DVD digital versatile disc
  • memory stick floppy disk
  • mechanically encoded device such as a printer with instructions stored thereon
  • a hole card or a raised structure in a groove and any suitable combination of the above.
  • computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.
  • Computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • a network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
  • Computer program instructions for carrying out operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or Source or object code written in any combination, including object-oriented programming languages—such as Smalltalk, C++, Python, etc., and conventional procedural programming languages—such as the “C” language or similar programming languages.
  • Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as via the Internet using an Internet service provider). connect).
  • electronic circuits such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can be customized by utilizing state information of computer-readable program instructions that can Various aspects of the invention are implemented by executing computer readable program instructions.
  • These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that when executed by the processor of the computer or other programmable data processing apparatus , producing an apparatus for realizing the functions/actions specified in one or more blocks in the flowchart and/or block diagram.
  • These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause computers, programmable data processing devices and/or other devices to work in a specific way, so that the computer-readable medium storing instructions includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks in flowcharts and/or block diagrams.
  • each block in a flowchart or block diagram may represent a module, a portion of a program segment, or an instruction that includes one or more Executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified function or action , or may be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation by means of hardware, implementation by means of software, and implementation by a combination of software and hardware are all equivalent.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Medical Informatics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

一种基于深度学习的心房颤动评估方法和装置。该方法包括：对于采集的原始心房图像，利用高斯卷积运算获得多个不同模糊度的心房图像；对于所述不同模糊度的心房图像，利用卷积神经网络提取特征图，获得对应的多个特征图，并利用通道加权模块融合所述多个特征图，获得基本特征映射图；将所述基本特征映射图输入到经训练的U-Net网络进行图像分割；基于获得的分割图像重建完整心动周期中的三维心房模型。所述方法能够精确分割心房图像，清晰地显示心房的整体轮廓和细节特征。

Description

一种基于深度学习的心房颤动评估方法和装置 技术领域
本发明涉及医学图像处理技术领域，更具体地，涉及一种基于深度学习的心房颤动评估方法和装置。
背景技术
心房颤动(AF)是最常见的持续性心律失常，并且有许多并发症，诸如高血压、心力衰竭和冠心病等。心房重塑的功能是探讨房颤的重要病理生理机制。心房重塑的主要机制包括电重塑、结构重塑、代谢重塑和自主神经重塑等。这些重塑最初补偿和维持心脏功能，但可能发展成非适应性变化，导致渐进式泵衰竭和恶性心律失常等。因此，深入了解心房的大小、功能及其重塑机制可以提供对心房影响的新见解、AF的治疗策略，以及心血管疾病预后和风险等级的重要信息。由于左心房是心脏结构的重要组成部分，因此本文主要左心房为例进行说明。
由于左心房结构的复杂性和不同患者左心房之间的差异性，使得很难准确地了解每个患者的左心房，导致误诊和治疗不理想。无创医学成像是心血管领域不可缺少的技术，例如MRI(磁共振成像)检查方式的高测量精度使其成为监测心血管疾病进展和治疗的理想方式。并且，处理医学成像的改进算法使MRI成为评估心房疾病的优秀可视化工具。目前已有许多特征提取器，例如AlexNet、VGGNet和GoogLeNet等。然而，在现有技术中，左心房图像重建效果还有待改建，并且左心房3D重建之前的手动分割效率低下且容易出错。
发明内容
本发明的目的是克服上述现有技术的缺陷，提供一种基于深度学习的心房颤动评估方法和装置。
根据本发明的第一方面，提供一种基于深度学习的心房颤动评估方法。该方法包括以下步骤：
对于采集的原始心房图像，利用高斯卷积运算获得多个不同模糊度的心房图像；
对于所述不同模糊度的心房图像，利用卷积神经网络提取特征图，获得对应的多个特征图，并利用通道加权模块融合所述多个特征图，获得基本特征映射图；
将所述基本特征映射图输入到经训练的U-Net网络进行图像分割；
基于获得的分割图像重建完整心动周期中的三维心房模型。
根据本发明的第二方面，提供一种基于深度学习的心房颤动评估装置。该装置包括：
不同尺度图像获取单元：用于对于采集的原始心房图像，利用高斯卷积运算获得多个不同模糊度的心房图像；
特征提取单元：用于对于所述不同模糊度的心房图像，利用卷积神经网络提取特征图，获得对应的多个特征图，并利用通道加权模块融合所述多个特征图，获得基本特征映射图；
图像分割单元：用于将所述基本特征映射图输入到经训练的U-Net网络进行图像分割；
图像重建单元：用于基于获得的分割图像重建完整心动周期中的三维心房模型。
与现有技术相比，本发明的优点在于，设计了一种具有高斯模糊和通道权重的新型U-Net，用于自动分割AF患者心房图像的心房区域。高斯模糊后，得到不同分辨率的图像，高分辨率图像清晰地显示了心房的细节特征，而低分辨率图像清晰地显示了心房的整体轮廓，从而解决医学图像特征少的问题。
通过以下参照附图对本发明的示例性实施例的详细描述，本发明的其它特征及其优点将会变得清楚。
附图说明
被结合在说明书中并构成说明书的一部分的附图示出了本发明的实施例，并且连同其说明一起用于解释本发明的原理。
图1是根据本发明一个实施例的基于深度学习的心房颤动评估方法的流程图；
图2是根据本发明一个实施例的相同图像在不同高斯尺度下的对比结果；
图3是根据本发明一个实施例的图像分割框架示意图；
图4是根据本发明一个实施例的整个心脏的三维视图；
图5是根据本发明一个实施例的将八个样本图像切片的分割结果与相关的真实值进行比较的示意图；
图6是根据本发明一个实施例的从二维左心房图像重建三维左心房模型的示意图；
图7是根据本发明一个实施例的从AF患者重建心动周期不同点的3D左心房示意图；
附图中，Input image-输入图像；Channel weight-通道权重；Output image-输出图像；Deconvolution-反卷积；Gaussia blur-高斯模糊；Multiply-乘；Convolution-卷积；Max pool-最大池化；Superior vena cava-上腔静脉；Right pulmonary vena-右肺静脉；Aorta-主动脉。
具体实施方式
现在将参照附图来详细描述本发明的各种示例性实施例。应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本发明的范围。
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本发明及其应用或使用的任何限制。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。
在这里示出和讨论的所有例子中，任何具体值应被解释为仅仅是示例性的，而不是作为限制。因此，示例性实施例的其它例子可以具有不同的 值。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。
本发明以心脏磁共振图像为例，提出了一种基于卷积神经网络的自动分割心房区域的技术方案，其中卷积神经网络可采用多种类型。考虑到在计算机视觉领域，U-Net为语义分割任务提供了全卷积网络模型，在图像分割方面表现出令人满意的性能，在下文的描述中，以U-Net网络为例进行说明，其输入是任意图像，输出是与输入相同大小的分割结果图像，然后利用分割信息得到图像的语义。
在一个实施例中，基于MRI和U-Net分割技术重建了动态变化的3D左心房模型，以探讨人工智能(AI)在心脏诊断中的临床价值。本发明的目的是在视觉分析的基础上，使用基于AI的分割结果重建完整心动周期中的3D左心房模型，适用于AF患者。
为了实现对左心房图像的精确分割，本发明对U-Net网络进行了改进，在本文中，将改进后的网络模型称为GCW-UNet。具体地，参见图1所示，所提供的基于深度学习的心房颤动评估方法包括以下步骤。
步骤S110，利用高斯卷积提取不同模糊度的心房MRI图像。
卷积神经网络(CNN)能够从MRI图像中提取各种特征。在一个实施例中，基于高斯卷积提取不同尺度的图像特征，高斯卷积在大尺度内模糊原始图像，并且减少相应的细节特征，另一方面，小尺度图像会保留更多细节。在本发明实施例中，CNN将提取前景的整体轮廓以及细节特征。
通过高斯卷积，可以将小尺度图像和大尺度图像融合成特征映射，该特征映射可以融合MRI的细节特征和全局特征。例如，二维高斯卷积的方程表示为：
Figure PCTCN2021138551-appb-000001
高斯卷积核的参数遵循正态分布规律，卷积核与图像的模糊程度相关。通过高斯卷积获得的特征矩阵M(x,y)表示为：
M(x,y)＝G(x,y)×I(x,y)      (2)
其中，I(x,y)是原始MRI图像，(x,y)表示像素点的坐标。
图2是具有不同尺度高斯卷积的心房MRI图像的对比图，其中图2(a)表示原始图像，图2(b)表示小尺度模糊，图2(c)表示大尺度图像，不同尺度对应不同的高斯卷积核大小。应理解的是，通过设置卷积核大小，可获得不同模糊度的图像，而不限于小尺度和大尺度两种类型。
步骤S120，对于不同模糊度的图像，利用卷积神经网络提取对应的特征图。
具体地，GCW-UNet模型包括两个阶段。在第一阶段，通过高斯卷积运算获得三幅不同模糊度的MRI图像。对于大尺度模糊图像，保留图像的整体轮廓特征，对于小尺度图像，保留图像的细节特征。然后，通过CNN获得三个不同的特征图，并将这三个特征图连接在一起。特征图输入通道权重(CW)模块。为了获得一维特征向量，在一个实施例中，输入通道权重模块的特征图首先通过全局平均池化(GAP)计算。接下来，一维特征向量通过1×1卷积和ReLU激活函数计算以减少一维向量的通道数，从而减少网络参数的计算量。然后，通过1×1卷积和ReLU激活函数恢复原始一维向量的维度，其加强一维向量各个元素的相关性。最后，softmax函数使一维向量的每个元素值在0与1之间，该一维向量称为通道权重。然后将通道权重与特征映射相乘，以获得基本特征映射。
在第二阶段，使用改进的U-Net进行图像分割，改进之处在于，使用通道权重模块取代U-Net原有的复制和裁剪模块(copy and crop)。通道权重模块是通道注意力机制，它可以将特征图的每个通道乘以不同的权重，权重与通道的重要性相关。对于U-Net网络，网络左侧部分用于特征提取(下采样)，右侧部分用于上采样。这种结构也称为编码器和解码器。在下采样部分进行网络操作后，输入特征矩阵的大小和分辨率将降低。为了将大小恢复到原始大小，U-net采用了反褶积操作层，如图3所示。由图3可以看出，GCW-UNet框架包括两个阶段，在第一阶段，通过高斯模糊和通道加权模块获得基本特征图，在第二阶段，对基本特征图进行下采样和上采样，以获得预测分割结果
需要说明的是，在现有技术中，U-Net网络不包含通道权重模块，而 是通过跳跃连接将编码路径中相同分辨率的层传递到解码路径，为其提供原始的高分辨率特征，而本发明的U-net网络部分，利用通道权重模块替代跳跃连接，通过这种方式，可以保留MRI图像的细节并自适应通道权重，进而增强网络的心房分割能力。
综上，由于MRI图像中背景(左心房外的区域)和前景(左心房)的高度相似性，传统的神经网络通常将背景误分类为前景，而本发明提供的GCW-UNet可以直接融合局部特征和全局特征，并自适应通道权重，解决了前景和背景像素数量不平衡以及需要提取细节特征的问题，使得心房边缘分割更加准确。
步骤S130，以设定的损失函数训练模型，用于图像分割。
心房分割属于二分法问题，相当于将前景与背景分开。左心房为前景，左心房外部分为背景。由于左心房区域的像素数量与背景像素数量不平衡，CNN倾向于将前景部分识别为背景。针对这种情况，在一个实施例中，采用Dice损失函数用于本发明分割网络的训练。本质上，Dice系数源自两个样本的重叠，范围从0到1。该系数的方程表示为：
Figure PCTCN2021138551-appb-000002
其中，|X|是分割网络分开的区域，|Y|是真实值，并且|X∩Y|是真实值和预测结果的交集。如果Dice＝1，则两个区域完全重合。Dice损失可以由Dice系数得出，并且损失方程表示为：
Loss＝1-Dice.     (4)
需说明的是，本发明整体上分为训练过程和应用过程，训练过程是指采用样本数据集训练GCW-UNet模型，获得模型参数(如权重和偏置等)，应用过程是指利用经训练模型对实际采集的目标心房进行分割，获得二维(2D)分割图像。训练过程和应用过程基本相同，在此不再赘述。
步骤S140，基于图像分割结果重建三维心房结构。
通过将2D心脏图像重建成3D结构，可以更全面地观察心脏数据。3D图像清楚地显示心脏的整体结构，如图4所示。三维心房重建可以缩短检查时间，从而使医学专家能够准确了解大血管和心房的生理变化。三维 心房重建基于2D图像切片。例如，在测试集中随机选择每个具有25个时间帧的MRI切片进行分割。利用分割后的二维心房图像将左心房重建成3D图像结构。最终，将心房结构完美展示出来。左心房可视化技术对AF的临床诊断提供了重要的参考意义。例如，基于一次心跳的不同时间帧，可以呈现一个心动周期内左心房结构的动态变化，更好地辅助心脏病患者房颤的评估。
需说明的是，二维图像到三维图像的重建过程也可以利用深度学习模型实现，即利用已知的二维图像切片到三维图像结构的对应关系训练深度学习模型。
综上，本发明第一部分利用卷积神经网络(GCW-UNet)对左心房区域进行分割，第二部分将二维左心房重建成三维模型，通过整合这两部分内容可以实现端到端的自动重建，即实现从原始MRI的输入到重建的3D左心房的输出的自动化过程。
相应地，本发明还提供一种基于深度学习的心房颤动评估装置。该装置包括：不同尺度图像获取单元，其用于对于采集的原始心房图像，利用高斯卷积运算获得多个不同模糊度的心房图像；特征提取单元，其用于对于所述不同模糊度的心房图像，利用卷积神经网络提取特征图，获得对应的多个特征图，并利用通道加权模块融合所述多个特征图，获得基本特征映射图；图像分割单元，其用于将所述基本特征映射图输入到经训练的U-Net网络进行图像分割；图像重建单元，其用于基于获得的分割图像重建完整心动周期中的三维心房模型。
为了进一步验证本发明的效果，进行了实验，实验内容和相关设置如下。
1)患者数据
使用的患者数据集来自皇家阿德莱德医院(Royal Adelaide Hospital)和阿得雷德大学(University of Adelaide)。所有心脏成像均经伦理委员会批准。此外，使用西门子Avanto、1.5Tesla、model-syngo MRB15扫描仪和Numaris-4软件收集了AF患者的MRI。MRI设备能够采集多个心脏切片，这为训练自动分割神经网络提供了数据。由于心脏腔面积占整个MRI的一部分，因 此图像的大小减小到288×288(像素)。真实图像由经验丰富的放射科医生手动分割。分割的左心房区域是二值图像。左心房用作前景，前景的灰度值为255。左心房外的区域为背景，背景的灰度值为0。
2)训练过程
线性整流函数(ReLU)可以防止梯度消失。因此，使用ReLU作为激活函数，并且使用随机梯度下降(SGD)作为优化器。初始学习率设置为0.001，学习率每5个阶段降低一次。为了防止过度拟合，数据增强通过翻转、调整大小和扭曲操作来随机匹配MRI切片的一部分。批量大小为4。Dice损失用作损失函数。GCW-UNet的训练和测试均在Window系统、采用RTX 2070 8G的GPU和采用Intel(R)Core(TM)i7-5500U@2.40GHz的CPU上实现。GCW-UNet采用Keras框架实现。在100个阶段后，模型达到收敛条件。
3)分割评估
为了验证本发明的有效性，采用了Jaccard指标和Dice相似系数，其中TP为真阳性，FP为假阳性，TN为真阴性，并且FN为假阴性。
Jaccard指标是衡量语义分割结果的重要指标。Jaccard指标相当于联合上的交集(IOU)，表示联合中预测值与真实值交集的比例。Jaccard指标表示为：
Figure PCTCN2021138551-appb-000003
Dice相似系数是评价指标，它以交集在联合集中的比例来衡量分割结果。例如，Dice相似系数表示为：
Figure PCTCN2021138551-appb-000004
进一步地，测试了训练模型的预测效果。具体地，随机选取了几个样本进行了预测，参见图5所示，其是将八个样本图像切片的分割结果与相关的真实值进行比较，图中两条线分别表示预测值和真实值，从两条线的拟合程度可以看出本发明能够获得精确的预测结果。
根据Jaccard指标和DICE相似系数，将本发明与传统的分割方法进行了比较。如表1比较了最先进的左心房分割方法和本发明的GCW UNET 方法。
表1 Jaccard指标和Dice相似系数的结果
Figure PCTCN2021138551-appb-000005
由表1可知，与目前最先进的左心房分割方法相比，本发明的左心房分割更接近有经验的放射科医生的手动分割。在测试数据集上，平均Dice相似系数达到93.57％，相对于现有技术，本发明提高了分割精度并且获得最佳性能。
4)左心房的3D重建
本发明提出的方法成功地重建了AF患者在整个心动周期内的左心房模型。如图6所示，从一系列2D左心房图像重建了3D左心房，其中四个切片是网络预测的二值图像，箭头表示心房的血流方向。这种重建的3D左心房可以帮助医学专家从生理角度观察左心房的大小变化，结合心电图诊断AF并且研究诊断结果。此外，动态重建的3D左心房可以使医学专家更好地理解复杂结构的左心房，确定AF疾病的影响及其对左心房结构的影响，并且用于评估心房肌病。
5)、AF患者的3D左心房模型
从生理学角度来看，AF刺激左心房重塑机制，包括心房结构重构。左心房结构重塑主要表现为心房收缩力下降、心房增大、心房肌细胞超微结构改变和心房纤维化。纤维化是心律失常所致结构重建的最突出表现，并且左心房扩大是结构重建的主要特征。因此，左心房3D结构的重建为诊断提供了有效的依据。通过观察一个心动周期内重建的3D左心房，可以发现房颤患者的心房大小变化不规则，如图7所示。在心律失常的研究中，配合心房大小观察心电图(ECG)至关重要，因为ECG可以捕捉心脏的电活动来发现和跟踪心律失常。ECG提供了有关AF治疗中心跳频率的信息，这有助于评估AF的风险。
图7是从AF患者重建心动周期不同时间点的3D左心房。第1-5帧是从左心房经二尖瓣流入左心室的血流，这是心房收缩的过程。第5-15帧是从左肺静脉和右肺静脉流入左心房的血流，这是心房舒张的过程。因此，1-15帧3D心房构成了完整的心动周期。在ECG的每个时间帧，都识别并构建了左心房结构。由此可以证明，在AF患者的心动周期中，动态3D左心房模型的大小存在明显的不规则性，而本发明分割的左心房更接近于有经验的放射科医生手工分割的左心房。
综合所述，本发明回顾和研究了医学成像以及基于AI的计算机辅助设计和重建技术在AF诊断中的应用。为了正确诊断AF疾病并研究其对左心房的影响，通过GCW-UNet重建了3D左心房模型，解决了MRI图像特征较少的问题，并且替代了跳跃连接以保留MRI图像的细节。此外，本发明还重建了心动周期中的3D左心房的范围，从而可以观察到AF患者的心动周期中的左心房大小的变化。需要说明的是，本发明也可以用于其他医学图像的处理，如CT图像等。
本发明可以是系统、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本发明的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是但不限于电存储设备、 磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本发明操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++、Python等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、 现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本发明的各个方面。
这里参照根据本发明实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本发明的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本发明的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。对于本领域技术人 员来说公知的是，通过硬件方式实现、通过软件方式实现以及通过软件和硬件结合的方式实现都是等价的。
以上已经描述了本发明的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。本发明的范围由所附权利要求来限定。

Claims (10)

  1. 一种基于深度学习的心房颤动评估方法，包括以下步骤：
    对于采集的原始心房图像，利用高斯卷积运算获得多个不同模糊度的心房图像；
    对于所述不同模糊度的心房图像，利用卷积神经网络提取特征图，获得对应的多个特征图，并利用通道加权模块融合所述多个特征图，获得基本特征映射图；
    将所述基本特征映射图输入到经训练的U-Net网络进行图像分割；
    基于获得的分割图像重建完整心动周期中的三维图像结构。
  2. 根据权利要求1所述的方法，其特征在于，所述通道加权模块包括全局池化层、卷积激活层和softmax层，其中全局池化层对输入特征图执行全局平均池化计算，获得一维特征向量；卷积激活层通过卷积运算和激活处理加强所述一维特征向量中各个元素的相关性；softmax层用于将所述一维特征向量的每个元素值处理为在0与1之间，获得通道权重。
  3. 根据权利要求2所述的方法，其特征在于，所述U-Net网络包含提取不同深度特征的U型结构，对于相同深度层之间的跳跃连接，使用所述通道权重模块取代U-Net的复制和裁剪。
  4. 根据权利要求1所述的方法，其特征在于，所述多个不同模糊度的心房图像根据下式获得：
    M(x,y)＝G(x,y)×I(x,y)
    Figure PCTCN2021138551-appb-100001
    其中，I(x,y)是原始图像，G(x,y)是二维高斯卷积的方程，σ表示方差。
  5. 根据权利要求1所述的方法，其特征在于，所述U-Net网络的训练损失函数设置为：
    Loss＝1-Dice
    Figure PCTCN2021138551-appb-100002
    其中，|X|是U-net网络分开的区域，|Y|是真实值，|X∩Y|是真实值和预测结果的交集。
  6. 根据权利要求1所述的方法，其特征在于，对获得的分割图像重建完整心动周期中的三维图像结构包括：
    对于分割后二维图像，采集一个心动周期内多个时间帧的心房图像切片；
    基于所述多个时间帧的心房图像切片，将心房重建成三维图像结构，进而分析一个心动周期内重建的三维心房变化。
  7. 根据权利要求1所述的方法，其特征在于，所述原始心房图像是磁共振成像。
  8. 一种基于深度学习的心房颤动评估装置，包括：
    不同尺度图像获取单元：用于对于采集的原始心房图像，利用高斯卷积运算获得多个不同模糊度的心房图像；
    特征提取单元：用于对于所述不同模糊度的心房图像，利用卷积神经网络提取特征图，获得对应的多个特征图，并利用通道加权模块融合所述多个特征图，获得基本特征映射图；
    图像分割单元：用于将所述基本特征映射图输入到经训练的U-Net网络进行图像分割；
    图像重建单元：用于基于获得的分割图像重建完整心动周期中的三维心房模型。
  9. 一种计算机可读存储介质，其上存储有计算机程序，其中，该程序被处理器执行时实现根据权利要求1至7中任一项所述方法的步骤。
  10. 一种计算机设备，包括存储器和处理器，在所述存储器上存储有能够在处理器上运行的计算机程序，其特征在于，所述处理器执行所述程序时实现权利要求1至7中任一项所述的方法的步骤。
PCT/CN2021/138551 2021-11-15 2021-12-15 一种基于深度学习的心房颤动评估方法和装置 WO2023082416A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111345864.3 2021-11-15
CN202111345864.3A CN114155208B (zh) 2021-11-15 2021-11-15 一种基于深度学习的心房颤动评估方法和装置

Publications (1)

Publication Number Publication Date
WO2023082416A1 true WO2023082416A1 (zh) 2023-05-19

Family

ID=80459856

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/138551 WO2023082416A1 (zh) 2021-11-15 2021-12-15 一种基于深度学习的心房颤动评估方法和装置

Country Status (2)

Country Link
CN (1) CN114155208B (zh)
WO (1) WO2023082416A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108447062A (zh) * 2018-02-01 2018-08-24 浙江大学 一种基于多尺度混合分割模型的病理切片非常规细胞的分割方法
US20190073569A1 (en) * 2017-09-07 2019-03-07 International Business Machines Corporation Classifying medical images using deep convolution neural network (cnn) architecture
CN110807362A (zh) * 2019-09-23 2020-02-18 腾讯科技（深圳）有限公司 一种图像检测方法、装置和计算机可读存储介质
CN111192245A (zh) * 2019-12-26 2020-05-22 河南工业大学 一种基于U-Net网络的脑肿瘤分割网络及分割方法
CN112712522A (zh) * 2020-10-30 2021-04-27 陕西师范大学 一种病理学图像的口腔癌上皮组织区域自动分割方法

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106203428B (zh) * 2016-07-05 2019-04-26 武汉大学 基于模糊估计融合的图像显著性检测方法
CN109801294A (zh) * 2018-12-14 2019-05-24 深圳先进技术研究院 三维左心房分割方法、装置、终端设备及存储介质
CN110458837B (zh) * 2019-08-19 2023-08-22 上海联影医疗科技股份有限公司 一种图像后处理方法、装置、电子设备及存储介质
CN111932550B (zh) * 2020-07-01 2021-04-30 浙江大学 一种基于深度学习的3d心室核磁共振视频分割系统
CN113177456B (zh) * 2021-04-23 2023-04-07 西安电子科技大学 基于单阶段全卷积网络和多特征融合的遥感目标检测方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190073569A1 (en) * 2017-09-07 2019-03-07 International Business Machines Corporation Classifying medical images using deep convolution neural network (cnn) architecture
CN108447062A (zh) * 2018-02-01 2018-08-24 浙江大学 一种基于多尺度混合分割模型的病理切片非常规细胞的分割方法
CN110807362A (zh) * 2019-09-23 2020-02-18 腾讯科技（深圳）有限公司 一种图像检测方法、装置和计算机可读存储介质
CN111192245A (zh) * 2019-12-26 2020-05-22 河南工业大学 一种基于U-Net网络的脑肿瘤分割网络及分割方法
CN112712522A (zh) * 2020-10-30 2021-04-27 陕西师范大学 一种病理学图像的口腔癌上皮组织区域自动分割方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
HOU, JINCHENG; YUAN, XUCHUN; HU, GUOYING; ZHOU, HONG; HU, GUODONG: "Research on Left Ventricle Segmentation of Heart CT Image Based on Full Convolutional Networks", MODERN MEDICAL IMAGEOLOGY, vol. 28, no. 12, 31 December 2019 (2019-12-31), CN , pages 2567 - 2571, XP009546378 *

Also Published As

Publication number Publication date
CN114155208B (zh) 2022-07-08
CN114155208A (zh) 2022-03-08

Similar Documents

Publication Publication Date Title
Litjens et al. State-of-the-art deep learning in cardiovascular image analysis
Saikumar et al. A novel implementation heart diagnosis system based on random forest machine learning technique.
Gul et al. Deep learning techniques for liver and liver tumor segmentation: A review
Singh et al. Machine learning in cardiac CT: basic concepts and contemporary data
US9968257B1 (en) Volumetric quantification of cardiovascular structures from medical imaging
Zhao et al. A novel U-Net approach to segment the cardiac chamber in magnetic resonance images with ghost artifacts
Militello et al. A semi-automatic approach for epicardial adipose tissue segmentation and quantification on cardiac CT scans
He et al. Automatic segmentation and quantification of epicardial adipose tissue from coronary computed tomography angiography
de Albuquerque et al. Fast fully automatic heart fat segmentation in computed tomography datasets
CN112184690A (zh) 冠脉血管走向的预测方法、预测模型的训练方法及装置
Bratt et al. Left atrial volume as a biomarker of atrial fibrillation at routine chest CT: deep learning approach
Azarmehr et al. Segmentation of left ventricle in 2D echocardiography using deep learning
Jafari et al. Deep Bayesian image segmentation for a more robust ejection fraction estimation
Bui et al. An automatic random walk based method for 3D segmentation of the heart in cardiac computed tomography images
Lin et al. Echocardiography-based AI detection of regional wall motion abnormalities and quantification of cardiac function in myocardial infarction
Farhad et al. Cardiac phase detection in echocardiography using convolutional neural networks
CN114565711A (zh) 基于深度学习的心脏图像重建方法及系统
Graves et al. Siamese pyramidal deep learning network for strain estimation in 3D cardiac cine-MR
Mangalagiri et al. Toward generating synthetic CT volumes using a 3D-conditional generative adversarial network
Arega et al. Using polynomial loss and uncertainty information for robust left atrial and scar quantification and segmentation
WO2023082416A1 (zh) 一种基于深度学习的心房颤动评估方法和装置
Liu et al. Multislice left ventricular ejection fraction prediction from cardiac MRIs without segmentation using shared SptDenNet
Chen et al. Spatio-temporal multi-task network cascade for accurate assessment of cardiac CT perfusion
Chernyshov et al. Automated Segmentation and Quantification of the Right Ventricle in 2-D Echocardiography
Gavirni et al. Clinically relevant myocardium segmentation in cardiac magnetic resonance images

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21963854

Country of ref document: EP

Kind code of ref document: A1