WO2022161192A1 - 一种spect三维重建图像左心室自动分割的方法 - Google Patents

一种spect三维重建图像左心室自动分割的方法 Download PDF

Info

Publication number
WO2022161192A1
WO2022161192A1 PCT/CN2022/072199 CN2022072199W WO2022161192A1 WO 2022161192 A1 WO2022161192 A1 WO 2022161192A1 CN 2022072199 W CN2022072199 W CN 2022072199W WO 2022161192 A1 WO2022161192 A1 WO 2022161192A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
spect
segmentation
left ventricle
network
Prior art date
Application number
PCT/CN2022/072199
Other languages
English (en)
French (fr)
Inventor
张铎
朱闻韬
韩璐
黄海亮
祁二钊
Original Assignee
之江实验室
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 之江实验室 filed Critical 之江实验室
Priority to JP2022564825A priority Critical patent/JP7270319B2/ja
Publication of WO2022161192A1 publication Critical patent/WO2022161192A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10108—Single photon emission computed tomography [SPECT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30048—Heart; Cardiac

Definitions

  • the invention relates to the field of medical imaging and deep learning, in particular to a method for automatic segmentation of the left ventricle of a SPECT three-dimensional reconstructed image based on a deep learning network.
  • SPECT cardiac imaging is currently the gold standard for clinical diagnosis of coronary heart disease, myocardial ischemia and other cardiovascular diseases, as well as for efficacy evaluation and prognosis judgment. lesions, providing more detailed functional activity information of myocardial tissue.
  • clinical SPECT examination a series of operations and analysis are required on the reconstructed SPECT images.
  • the calculation of the left ventricular ejection coefficient is an important indicator for evaluating cardiac function, which requires segmentation of the left ventricular cavity and ventricular wall. , calculated by extracting the left ventricular cavity volume at different heartbeat cycles.
  • the clinical standard SPECT view of the heart is in the short axis SA direction, and since the long axis of the left ventricle is not parallel to the long axis of the human body, it is usually necessary to manually rotate the reconstructed SPECT image to obtain the standard SA view, and image in this view. Segmentation was used to calculate left ventricular ejection coefficient. At the same time, left ventricular images based on standard SA views can be used to prepare cardiac polar maps for activity analysis of left ventricular myocardium.
  • Transferring images from the conventional RA view to the cardiac standard SA view for clinical analysis often requires manual operations by physicians. This subjective operation easily introduces random errors and affects the accuracy of analysis, and consumes a long manual operation time.
  • the current common clinical nuclear medicine cardiac image analysis software mostly adopts conventional image processing segmentation methods, such as segmentation method based on left ventricular wall centerline, segmentation method based on left ventricular model, segmentation method based on cardiac atlas method, segmentation method based on threshold or k-means clustering, etc. Due to the low resolution of SPECT images and the fact that cardiac images are easily affected by breathing and heartbeat motion, the image motion boundary is blurred, and the current segmentation methods often have low segmentation accuracy when performing segmentation.
  • the purpose of the present invention is to provide a method for automatic segmentation of the left ventricle of a SPECT three-dimensional reconstructed image based on a deep learning network in view of the deficiencies of the prior art.
  • a method for automatic segmentation of the left ventricle of a SPECT three-dimensional reconstructed image comprising the following steps:
  • Step 1 Perform downscaling and resampling on the conventional view RA of the SPECT 3D reconstructed image, and use the resampled downscaled conventional view RA-r as the input of the feature extraction network, which consists of a convolution module and a fully connected layer , utilize the convolution module to perform feature extraction on the reduced conventional view RA-r, and form a 6-dimensional feature vector T-r after the full connection is expanded, and the 6-dimensional feature vector T-r includes translation parameters in 3 directions and rotation parameters in 3 angles;
  • Step 2 Adjust T-r to the feature vector T by using the proportional relationship between the regular view RA and the reduced regular view RA-r, wherein the rotation parameter is unchanged and the translation parameter is proportionally enlarged;
  • Step 3 applying the feature vector T-r to the reduced conventional view RA-r to obtain a predicted reduced image SA-r' by using the spatial transformation network, and applying the feature vector T to the conventional view RA to obtain the predicted image SA';
  • Step 4 Take the center of the predicted image SA' as the center, intercept the extracted heart part in the image to form the predicted heart image SA-H', and at the same time, perform image gradient calculation on the heart image SA-H' to obtain the corresponding gradient map SA-G ;
  • Step 5 Integrate the heart image SA-H' and the gradient map SA-G into a two-channel image, extract image features through downsampling and upsampling of the 3D U-NET network, and perform segmentation processing through the softmax layer to obtain the predicted left ventricle Structural segmentation result F;
  • L-img is the image loss function between the predicted reduced image SA-r' and the reduced standard view SA-r
  • L-par is the difference between the feature vector T-r and the rigid registration parameter P-r
  • the parameter loss function, L-seg is the label loss function between the predicted segmentation result F and the SA direction segmentation label G
  • the reduced standard view SA-r is manually turned to the conventional view RA of the SPECT three-dimensional reconstructed image and reduced proportionally Obtained
  • the rigid registration parameter P-r is the registration parameter between the reduced conventional view RA-r and the reduced standard view SA-r calculated by the rigid registration algorithm, including translation parameters in 3 directions and rotation parameters in 3 angles
  • the SA direction segmentation label G includes 3 values of left ventricular cavity, ventricular wall and background, obtained by taking the heart image SA-H intercepted at the center of the standard view SA and manually delineating the ventricular cavity and ventricular wall of the left ventricle of the heart .
  • the sizes of the reduced conventional view RA-r and the reduced standard view SA-r are preferably 64*64*64 voxels; the cardiac image SA-H and the predicted cardiac image SA-H' should cover the entire cardiac image. , and at the same time, it is best to include as little other high-intensity organ information as possible, preferably 32*32*32 voxel size.
  • g x , g y , g z are the gradients in the x, y, and z directions, respectively, and the calculation formula is:
  • i, j, and k are the coordinate indices in the three directions of x, y, and z in the image.
  • the gradient image can better provide image boundary information, synchronously reading the gradient image can optimize the recognition of image edges by the network and further improve the effectiveness of the network for image segmentation.
  • each convolution module includes a convolution layer and a pooling layer; the downsampling and upsampling of the three-dimensional U-NET network.
  • the sampling is 4 times, in which downsampling includes convolutional layers and pooling layers, and upsampling includes convolutional layers and deconvolutional layers.
  • the background is set to 0
  • the left ventricular cavity is set to 1
  • the left ventricular wall is set to 2.
  • ⁇ , ⁇ and ⁇ are 1,100 and 10, respectively.
  • the image loss function L-img adopts a mean square error function.
  • parameter loss function L-par adopts an absolute value loss function L1 or a norm loss function L2.
  • the present invention uses the deep learning network of multi-task learning to synchronously extract the position feature and the semantic feature of the image, and uses the mutual supervision of the dual network features to achieve the effect of network integration training to achieve different angles to standard views.
  • the integration of automatic steering, cardiac positioning and left ventricular structure segmentation reduces the complexity and human error of manual steering and segmentation, realizes fully automatic image operation and improves accuracy, and the present invention further uses gradient images to increase the accuracy of the image.
  • the accuracy of edge segmentation of image data is a deep learning network of multi-task learning to synchronously extract the position feature and the semantic feature of the image, and uses the mutual supervision of the dual network features to achieve the effect of network integration training to achieve different angles to standard views.
  • FIG. 1 is a schematic flow chart of the automatic segmentation of the left ventricle of the SPECT three-dimensional image according to the present invention.
  • Figure 2 is a schematic diagram of the automatic steering and positioning module in the structure of the SPECT three-dimensional image left ventricle automatic segmentation model.
  • FIG. 3 is a schematic diagram of the automatic segmentation module in the structure of the automatic segmentation model of the left ventricle of the SPECT three-dimensional image.
  • a method for automatic segmentation of the left ventricle of a SPECT three-dimensional reconstructed image based on a deep learning network proposed by the present invention is specifically: scaling the original SPECT three-dimensional reconstructed image of the chest to a size of 64*64*64 voxels by linear interpolation , using the feature extraction network to extract the rigid registration parameter features from the reduced image, and using the spatial transformation network and the extracted rigid registration parameter features to automatically turn the SPECT 3D reconstructed image to obtain the predicted image of the standard view, from the predicted image of the standard view.
  • the central 32*32*32 voxel part is cut to obtain the cardiac image, and the image is automatically segmented through the U-NET network to obtain the segmentation result of the left ventricular structure under the standard view.
  • the process is shown in Figure 1.
  • the feature extraction network, the spatial transformation network and the U-NET network jointly use multi-task learning and training, and the total loss function of the training image loss function L-img, parameter loss function L-par and label loss function L-seg Joint loss function; through multi-task joint learning, the joint learning of the front and rear parts of the network and the optimization of the target can be guaranteed.
  • SPECT three-dimensional image left ventricle automatic segmentation model (structure shown in Figure 2 and Figure 3) is provided to realize the integrated automatic steering, positioning and segmentation of the method of the present invention.
  • the construction and training of the model specifically include the following step:
  • Step 1 Acquire the conventional view RA of 600 cases of SPECT 3D reconstructed images, and manually turn to the standard view SA for clinical analysis with the left ventricle at the center of the image.
  • the standard view SA take the center of the image as the center to intercept 32*32* 32-voxel size image to obtain the heart image SA-H and manually delineate the ventricular cavity and ventricular wall of the left ventricle of the heart to obtain the SA direction segmentation label G, and calculate the rigidity between the conventional view RA and the standard view SA through the rigid registration algorithm
  • Both the regular view RA and the standard view SA are resampled to the reduced regular view RA-r and the reduced standard view SA-r with a size of 64*64*64 voxels by linear interpolation, and the rigid registration parameter P is adjusted to reduce the rigidity
  • the parameter P-r is registered to form a mapping dataset of RA-r, SA-r and P-r of the SPECT image.
  • the reduction rigid registration parameter P-r includes translation parameters in 3 directions and rotation parameters in 3 angles, the rotation parameters of P and P-r are the same, and the translation parameters are proportionally reduced according to the scaling ratio;
  • the SA direction segmentation label G includes The 3 values of left ventricular cavity, ventricular wall and background, where the background value is 0, the left ventricular cavity value is 1, and the left ventricular wall value is 2.
  • Step 2 Input the reduced regular view RA-r to the feature extraction network, use the convolution module to extract the features of the reduced regular view RA-r, and fully connect and expand to form a 6-dimensional feature vector T-r, using the equal ratio of P-r and P
  • the relationship adjusts T-r to the eigenvector T of the same proportion as P, in which the rotation parameter is unchanged and the translation parameter is proportionally enlarged;
  • FIG. 2 shows the structure of an embodiment of the automatic steering module, which includes a feature extraction network and a spatial transformation network.
  • the convolution module in the feature extraction network consists of 3 ⁇ 3 ⁇ 3 convolution units and Relu activation function units.
  • Step 3 The feature vector T-r is applied to the regular view RA-r through the spatial transformation network to obtain the predicted reduced image SA-r', and the feature vector T is applied to the SPECT reconstructed image regular view RA to obtain the predicted image SA' (Fig. 2); Taking the center of the predicted image SA' as the center, intercept an image with a size of 32*32*32 voxels to extract the heart part, and form a heart image SA-H'. Calculate and generate the gradient map SA-G of the SA-r' image ( Figure 3), where the calculation formula obtained by the gradient map SA-G is:
  • g x , g y , g z are the gradients in the x, y, and z directions, respectively, and the calculation formula is:
  • i, j, and k are the coordinate indices in the three directions of x, y, and z in the image.
  • Step 4 Further integrate the cardiac image SA-H' and the gradient map SA-G into a two-channel image and use the three-dimensional U-NET network to obtain the predicted left ventricular structure segmentation result F.
  • Figure 3 shows an example structure of the image segmentation module.
  • the convolution module is composed of a 3 ⁇ 3 ⁇ 3 convolution unit (Convolution, Conv.) and a Relu activation function unit, and the upsampling module is composed of a 3 ⁇ 3 ⁇ 3 transposed volume.
  • the product unit (Transpose Convolution, Trans.Conv.) and the Relu activation function unit are composed.
  • the last module is connected to the softmax layer through a 1 ⁇ 1 ⁇ 1 convolution unit to achieve the output of the final segmentation result.
  • the dashed lines represent the bi-level information that replicates the cropping operation on the data to combine the image and features.
  • Step 5 Build the image loss function L-img between the predicted reduced image SA-r' and the reduced standard view RA-r, the parameter loss function L-par between the feature vector T-r and the rigid registration parameter P-r, and the predicted segmentation result
  • the label loss function L-seg between the segmentation labels G in the F and SA directions is used to train and optimize the network to obtain an automatic segmentation model of the left ventricle in SPECT 3D images.
  • the specific implementation is subdivided into the following sub-steps:
  • the training of the automatic segmentation model is a multi-task learning process, and the loss matrix will contain the constraint information of the RA-r to SA-r steering model and the constraint information of the segmentation model from SA-H to the segmentation result F,
  • the learning objectives of multiple tasks together form constraints on the overall network to train this automatic segmentation model.
  • the image loss function L-img of the steering model part adopts the mean square error function between the predicted image and the reference image, and the corresponding rigid registration parameter loss function L-par adopts the absolute value loss function L1 or the norm loss function L2.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Quality & Reliability (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Nuclear Medicine (AREA)

Abstract

一种SPECT三维重建图像左心室自动分割的方法，通过对原始SPECT胸部三维图像线性插值进行等比例缩小，利用特征提取网络对缩小后图像提取刚性配准参数特征，利用空间变换网络及参数特征对SPECT图像进行自动转向获得标准视图的预测图像，从预测图像中切割中心部位获取心脏图像，并通过U-NET网络进行图像自动分割获得标准视图下的左心室结构分割结果。上述方法使用多任务学习的深度学习网络同步提取图像的位置特征和语义特征，并利用双网络特征的互相监督达到网络一体化训练的效果实现不同角度到标准视图的一体化自动转向、心脏定位及左心室的结构分割，减少了手动转向、分割的复杂性和人为误差，实现了图像操作的全自动并提高了准确性。

Description

一种SPECT三维重建图像左心室自动分割的方法 技术领域
本发明涉及医学影像领域和深度学习领域，尤其涉及一种基于深度学习网络的SPECT三维重建图像左心室自动分割的方法。
背景技术
SPECT心脏成像是目前临床诊断冠心病、心肌缺血等心血管疾病和疗效评价以及预后判断的金标准，其可非介入式的提供心肌组织的功能性信息来检测到尚未导致结构上变化的潜在病变，提供心肌组织更详细的功能活性信息。临床进行SPECT检查时，需要对重建后的SPECT图像进行一系列的操作和分析，其中左心室射血系数的计算是评价心脏功能的一个重要指标，其需要对左心室心室腔及心室壁进行分割，以提取不同心跳周期时的左心室腔体积进行计算。临床标准的SPECT心脏视图为短轴图SA方向，且由于左心室长轴与人体的长轴不平行，所以通常需要手动对重建SPECT图像进行旋转以获得标准SA视图，并在此视图中进行图像分割用以计算左心室射血系数。同时，基于标准SA视图的左心室图像可用于制备心脏极坐标图进行左心室心肌的活性分析。
将图像从常规RA视图转至临床分析的心脏标准SA视图临床上常需要医师手动操作，此主观性的操作容易引入随机误差而影响分析准确性，且需消耗较长的手动操作时间。而对于左心室图像分割，目前常见的临床核医学心脏图像分析软件多采用常规图像处理的分割方法，例如基于左心室壁中心线的分割方法、基于左心室模型的分割方法、基于心脏图谱的分割方法、基于阈值或k均值聚类的分割方法等，由于SPECT图像分辨率较低，且心脏图像易受到呼吸和心跳运动影响出现图像运动边界模糊，目前的分割方法在进行分割时常出现分割精度低、对分割边缘提取不准确的问题从而进一步影响量化分析的精度。因此，对于SPECT心脏图像的临床处理和分析时，如何实现常规重建视图到特定的临床分析所用的标准SA视图的稳定、准确的图像自动转向和定位，如何在较低图像分辨率的SPECT心脏图像中实现对左心室结构的精准分割提取，是临床SPECT心脏处理中所面临的技术难题。
发明内容
本发明的目的在于针对现有技术的不足，提供一种基于深度学习网络的SPECT三维重建图像左心室自动分割的方法。
本发明的目的是通过以下技术方案来实现的：
一种SPECT三维重建图像左心室自动分割的方法，包括以下步骤：
步骤一：对SPECT三维重建图像的常规视图RA进行缩小重采样，并将重采样后的缩小常规视图RA-r作为特征提取网络的输入，所述特征提取网络由卷积模块和全连接层组成，利用卷积模块对缩小常规视图RA-r进行特征提取，并全连接展开后形成6维特征向量T-r，所述6维特征向量T-r包含3个方向的平移参量和3个角度的旋转参量；
步骤二：利用常规视图RA与缩小常规视图RA-r的等比例关系将T-r调整为特征向量T，其中旋转参量不变而平移参量等比例放大；
步骤三：利用空间变换网络将特征向量T-r应用于缩小常规视图RA-r中得到预测缩小图像SA-r’，同时将特征向量T应用于常规视图RA中得到预测图像SA’；
步骤四：以预测图像SA’的中心为中心，截取图像中的提取心脏部分，形成预测心脏图像SA-H’，同时，对心脏图像SA-H’进行图像梯度计算获得对应梯度图SA-G；
步骤五：将心脏图像SA-H’和梯度图SA-G融合为双通道图像并经过三维U-NET网络的下采样及上采样提取图像特征并经过softmax层进行分割处理即获得预测的左心室结构分割结果F；
其中，所述特征提取网络、空间变换网络和U-NET网络联合采用多任务共同学习训练，训练的总损失函数L＝δL-par+μL-img+λL-seg；
其中δ、μ和λ为权重系数；L-img为预测缩小图像SA-r’与缩小标准视图SA-r之间的图像损失函数、L-par为特征向量T-r与刚性配准参数P-r之间的参数损失函数，L-seg为预测分割结果F与SA方向分割标签G之间的标签损失函数；所述缩小标准视图SA-r通过手动转向SPECT三维重建图像的常规视图RA并按等比例缩小获得；所述刚性配准参数P-r为刚性配准算法计算的缩小常规视图RA-r与缩小标准视图SA-r之间的配准参数，包含3个方向的平移参量和3个角度的旋转参量；所述SA方向分割标签G包含左心室腔、心室壁及背景的3个数值，通过以标准视图SA中心为中心截取的心脏图像SA-H并手动勾画心脏左心室的心室腔及心室壁获得。
进一步地，所述缩小常规视图RA-r、缩小标准视图SA-r的尺寸优选为64*64*64体素大小；心脏图像SA-H和预测心脏图像SA-H’应以涵盖整个心脏图像，同时尽量少的包含其他高强度器官信息为最佳，优选为32*32*32体素大小。
进一步地，所述步骤三中，通过构建变换矩阵P＝[R M T]获取预测图像，其中，M＝[tx、ty、tz]表示位移矩阵，tx、ty、tz分别为特征向量中对应的3个方向的平移参量，R为旋转矩阵，将R从欧拉角R＝[β、α、γ]转换为世界坐标系参数：
Figure PCTCN2022072199-appb-000001
进一步地，所述梯度图SA-G获得的计算公式为：
Figure PCTCN2022072199-appb-000002
其中，g x,g y,g z分别为x、y、z方向的梯度，其计算公式为：
g x(i，j，k)＝SA-H’(i+1，j，k)-SA-H’(i，j，k)
g y(i，j，k)＝SA-H’(i，j+1，k)-SA-H’(i，j，k)
g z(i，j，k)＝SA-H’(i，j，k+1)-SA-H’(i，j，k)
其中i、j、k为图像内x、y、z三个方向的坐标索引。
由于梯度图像可以更好的提供图像边界信息，同步读入梯度图可以优化网络对图像边缘的识别从而进一步提高网络对图像分割的效力。
进一步地，所述特征提取网络的卷积模块、全连接层的个数均为3个，每个卷积模块包含卷积层和池化层；所述三维U-NET网络的下采样及上采样均为4次，其中下采样包含卷积层和池化层，上采样包含卷积层和反卷积层。
进一步地，所述SA方向分割标签G中，将背景设置为0,左心室腔设置为1，左心室壁设置为2。
进一步地，所述δ、μ和λ分别取值为1,100和10。
进一步地，所述图像损失函数L-img采用均方差函数。
进一步地，所述参数损失函数L-par采用绝对值损失函数L1或范数损失函数L2。
进一步地，所述标签损失函数L-seg采用Dice-loss损失函数，其中，分别对左心室腔及左心室壁计算标签损失函数L-seg-1及L-seg-2，最终L-seg＝L-seg-2+L-seg-1。
本发明的有益效果是：本发明使用多任务学习的深度学习网络同步提取图像的位置特征和语义特征，并利用双网络特征的互相监督达到网络一体化训练的效果以实现不同角度到标准视图的一体化自动转向、心脏定位及左心室的结构分割，减少了手动转向、分割的复杂性和人为误差，实现了图像操作的全自动并提高了准确性，并且本发明还进一步采用梯度图像增加对图像数据边缘分割的准确性。
附图说明
图1是本发明SPECT三维图像左心室自动分割的流程示意图。
图2是SPECT三维图像左心室自动分割模型结构中的自动转向和定位模块示意图。
图3是SPECT三维图像左心室自动分割模型结构中的自动分割模块示意图。
具体实施方式
下面结合附图详细说明本发明。
本发明提出的一种基于深度学习网络的SPECT三维重建图像左心室自动分割的方法，该方法具体为：对原始SPECT胸部三维重建图像通过线性插值进行等比例缩放至64*64*64体素大小，利用特征提取网络对缩小后图像提取刚性配准参数特征，利用空间变换网络及提取的刚性配准参数特征对SPECT三维重建图像进行自动转向获得标准视图的预测图像，从标准视图的预测图像中切割中心32*32*32体素部位以获取心脏图像，并通过U-NET网络进行图像自动分割以获得标准视图下的左心室结构分割结果，其流程如图1所示。其中，所述特征提取网络、空间变换网络和U-NET网络联合采用多任务共同学习训练，训练的总损失函数图像损失函数L-img、参数损失函数L-par以及标签损失函数L-seg的联合损失函数；通过多任务共同学习可以保证前后部分网络的共同学习和对目标的优化。
下面，提供一个SPECT三维图像左心室自动分割模型(结构如图2和图3所示)，以实现本发明方法的一体化的自动转向、定位及分割，该模型的构建和训练，具体包括以下步骤：
步骤一：获取600例SPECT三维重建图像的常规视图RA，并手动转向至用于临床分析的、左心室位于图像中心的标准视图SA，在标准视图SA中以图像中心为中心截取32*32*32体素大小的图像以获得心脏图像SA-H并手动勾画心脏左心室的心室腔及心室壁以获得SA方向分割标签G，通过刚性配准算法计算常规视图RA及标准视图SA之间的刚性配准参数P，刚性配准参数P的6个参数分别为3个方向的平移参量和3个角度的旋转角度参量P＝[tx、ty、tz、β、α、γ]，形成SPECT图像常规视图RA、标准视图SA、SA方向分割标签G以及刚性配准参数P的映射数据库。将常规视图RA及标准视图SA均采用线性插值等比例重采样至64*64*64体素大小的缩小常规视图RA-r及缩小标准视图SA-r，并调整刚性配准参数P为缩小刚性配准参数P-r以形成SPECT图像的RA-r、SA-r与P-r的映射数据组。同样地，缩小刚性配准参数P-r包含3个方向的平移参量和3个角度的旋转参量，P与P-r的旋转参量相同而平移参量根据缩放比例进行等比例缩减；所述SA方向分割标签G包含左心室腔、心室壁及背景的3个数值，其中，背景值为0，左心室腔值为1，左心室壁值为2。
步骤二：将缩小常规视图RA-r输入至特征提取网络，利用卷积模块对缩小常规视图 RA-r进行特征提取，并全连接展开后形成6维特征向量T-r，利用P-r与P的等比例关系将T-r调整为与P比例相同的特征向量T，其中旋转参量不变而平移参量等比例放大；T-r与T的6维向量可拆分为位移矩阵M＝[tx、ty、tz]及旋转矩阵R＝[β、α、γ]，将R从欧拉角转换为世界坐标系参数:
Figure PCTCN2022072199-appb-000003
重构变换矩阵为T’＝[R M T]以适配后续空间变换网络的参数输入。图2中为自动转向模块的实施例结构，其包含了特征提取网络及空间变换网络。特征提取网络中的卷积模块由3×3×3的卷积单元和Relu激活函数单元组成。
步骤三：经过空间变换网络将特征向量T-r应用于常规视图RA-r中以得到预测缩小图像SA-r’，将特征向量T应用于SPECT重建图像常规视图RA中以得到预测图像SA’(图2)；以预测图像SA’中心为中心，截取32*32*32体素大小的图像以提取心脏部分，形成心脏图像SA-H’。计算并生成SA-r’图像的梯度图SA-G(图3)，其中梯度图SA-G获得的计算公式为：
Figure PCTCN2022072199-appb-000004
其中，g x,g y,g z分别为x、y、z方向的梯度，其计算公式为：
g x(i，j，k)＝SA-H’(i+1，j，k)-SA-H’(i，j，k)
g y(i，j，k)＝SA-H’(i，j+1，k)-SA-H’(i，j，k)
g z(i，j，k)＝SA-H’(i，j，k+1)-SA-H’(i，j，k)
其中i、j、k为图像内x、y、z三个方向的坐标索引。
步骤四：进一步将心脏图像SA-H’及梯度图SA-G整合为双通道图像并利用三维U-NET网络以获得预测的左心室结构分割结果F。图3为图像分割模块的实施例结构，卷积模块由3×3×3的卷积单元(Convolution,Conv.)和Relu激活函数单元组成，上采样模块由3×3×3的转置卷积单元(Transpose Convolution,Trans.Conv.)和Relu激活函数单元组成。最后一个模块通过1×1×1的卷积单元与softmax层进行连接实现最终分割结果的输出。虚线表示对数据复制裁切操作以结合图像和特征的双层信息。
步骤五：构建预测缩小图像SA-r’与缩小标准视图RA-r之间的图像损失函数L-img、特征向量T-r与刚性配准参数P-r之间的参数损失函数L-par以及预测分割结果F与SA方向分割标签G之间的标签损失函数L-seg，对网络进行训练优化以获得SPECT三维图 像左心室自动分割模型。其中的具体实现细分以下子步骤：
(5.1)自动分割模型的训练是一个多任务学习的过程，其损失矩阵将包含对RA-r到SA-r转向模型的约束信息和从SA-H到分割结果F的分割模型的约束信息，多任务的学习目标共同形成对整体网络的约束以训练此自动分割模型。模型的整体损失矩阵设计为图像损失函数L-img、参数损失函数L-par以及标签损失函数L-seg的联合损失函数L＝δL-par+μL-img+λL-seg，其中δ、μ和λ为权重系数，其根据经验分别取值1，100和10。
(5.2)转向模型部分的图像损失函数L-img采用预测图像和参考图像间的均方差函数，起对应的刚性配准参数损失函数L-par采用绝对值损失函数L1或范数损失函数L2。
(5.3)分割模型部分标签损失函数L-seg采用Dice-loss损失函数，其中，分别对左心室腔及左心室壁计算标签损失函数L-seg-1及L-seg-2，最终L-seg＝L-seg-2+L-seg-1。
上述实施例用来解释说明本发明，而不是对本发明进行限制，在本发明的精神和权利要求的保护范围内，对本发明作出的任何修改和改变，都落入本发明的保护范围。

Claims (10)

  1. 一种SPECT三维重建图像左心室自动分割的方法，其特征在于，包括以下步骤：
    步骤一：对SPECT三维重建图像的常规视图RA进行缩小重采样，并将重采样后的缩小常规视图RA-r作为特征提取网络的输入，所述特征提取网络由卷积模块和全连接层组成，利用卷积模块对缩小常规视图RA-r进行特征提取，并全连接展开后形成6维特征向量T-r，所述6维特征向量T-r包含3个方向的平移参量和3个角度的旋转参量；
    步骤二：利用常规视图RA与缩小常规视图RA-r的等比例关系将T-r调整为特征向量T，其中旋转参量不变而平移参量等比例放大；
    步骤三：利用空间变换网络将特征向量T-r应用于缩小常规视图RA-r中得到预测缩小图像SA-r’，同时将特征向量T应用于常规视图RA中得到预测图像SA’；
    步骤四：以预测图像SA’的中心为中心，截取图像中的心脏部分，形成预测心脏图像SA-H’，同时，对预测心脏图像SA-H’进行图像梯度计算获得对应梯度图SA-G；
    步骤五：将预测心脏图像SA-H’和梯度图SA-G融合为双通道图像并经过三维U-NET网络的下采样及上采样提取图像特征并经过softmax层进行分割处理即获得预测的左心室结构分割结果F；
    其中，所述特征提取网络、空间变换网络和三维U-NET网络联合采用多任务共同学习训练，训练的总损失函数L＝δL-par+μL-img+λL-seg；
    其中δ、μ和λ为权重系数；L-img为预测缩小图像SA-r’与缩小标准视图SA-r之间的图像损失函数、L-par为特征向量T-r与刚性配准参数P-r之间的参数损失函数，L-seg为预测分割结果F与SA方向分割标签G之间的标签损失函数；所述缩小标准视图SA-r通过手动转向SPECT三维重建图像的常规视图RA并按等比例缩小获得；所述刚性配准参数P-r为刚性配准算法计算的缩小常规视图RA-r与缩小标准视图SA-r之间的配准参数，包含3个方向的平移参量和3个角度的旋转参量；所述SA方向分割标签G包含左心室腔、心室壁及背景的3个数值，通过以标准视图SA中心为中心截取的心脏图像SA-H并手动勾画心脏左心室的心室腔及心室壁获得。
  2. 根据权利要求1所述SPECT三维重建图像左心室自动分割的方法，其特征在于，所述缩小常规视图RA-r、缩小标准视图SA-r的尺寸为64*64*64体素大小；心脏图像SA-H和预测心脏图像SA-H’的尺寸为32*32*32体素大小。
  3. 根据权利要求1所述SPECT三维重建图像左心室自动分割的方法，其特征在于，所述步骤三中，所述空间变换网络通过构建变换矩阵P＝[R M T]获取预测图像，其中，M＝[tx、ty、tz]表示位移矩阵，tx、ty、tz分别为特征向量中对应的3个方向的平移参量，R为旋转矩 阵，采用世界坐标系表示：
    Figure PCTCN2022072199-appb-100001
    R＝[β、α、γ]为旋转矩阵的欧拉角表示形式。
  4. 根据权利要求1所述SPECT三维重建图像左心室自动分割的方法，其特征在于，所述梯度图SA-G获得的计算公式为：
    Figure PCTCN2022072199-appb-100002
    其中，g x,g y,g z分别为x、y、z方向的梯度，其计算公式为：
    g x(i，j，k)＝SA-H’(i+1，j，k)-SA-H’(i，j，k)
    g y(i，j，k)＝SA-H’(i，j+1，k)-SA-H’(i，j，k)
    g z(i，j，k)＝SA-H’(i，j，k+1)-SA-H’(i，j，k)
    其中i、j、k为图像内x、y、z三个方向的坐标索引。
  5. 根据权利要求1所述SPECT三维重建图像左心室自动分割的方法，其特征在于，所述特征提取网络的卷积模块、全连接层的个数均为3个，每个卷积模块包含卷积层和池化层；所述三维U-NET网络的下采样及上采样均为4个，其中下采样包含卷积层和池化层，上采样包含卷积层和反卷积层。
  6. 根据权利要求1所述SPECT三维重建图像左心室自动分割的方法，其特征在于，所述SA方向分割标签G中，将背景设置为0,左心室腔设置为1，左心室壁设置为2。
  7. 根据权利要求1所述SPECT三维重建图像左心室自动分割的方法，其特征在于，所述δ、μ和λ分别取值为1,100和10。
  8. 根据权利要求1所述SPECT三维重建图像左心室自动分割的方法，其特征在于，所述图像损失函数L-img采用均方差函数。
  9. 根据权利要求1所述SPECT三维重建图像左心室自动分割的方法，其特征在于，所述参数损失函数L-par采用绝对值损失函数L1或范数损失函数L2。
  10. 根据权利要求1所述SPECT三维重建图像左心室自动分割的方法，其特征在于，所述标签损失函数L-seg采用Dice-loss损失函数，其中，分别对左心室腔及左心室壁计算标签损失函数L-seg-1及L-seg-2，最终L-seg＝L-seg-2+L-seg-1。
PCT/CN2022/072199 2021-02-01 2022-01-15 一种spect三维重建图像左心室自动分割的方法 WO2022161192A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2022564825A JP7270319B2 (ja) 2021-02-01 2022-01-15 Ｓｐｅｃｔの３次元再構成画像の左心室自動分割方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110135783.4 2021-02-01
CN202110135783.4A CN112508949B (zh) 2021-02-01 2021-02-01 一种spect三维重建图像左心室自动分割的方法

Publications (1)

Publication Number Publication Date
WO2022161192A1 true WO2022161192A1 (zh) 2022-08-04

Family

ID=74953088

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/072199 WO2022161192A1 (zh) 2021-02-01 2022-01-15 一种spect三维重建图像左心室自动分割的方法

Country Status (3)

Country Link
JP (1) JP7270319B2 (zh)
CN (1) CN112508949B (zh)
WO (1) WO2022161192A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116934768A (zh) * 2023-08-16 2023-10-24 中国人民解放军总医院 用于提高cta影像模态中血管分割精度的方法及系统

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112508949B (zh) * 2021-02-01 2021-05-11 之江实验室 一种spect三维重建图像左心室自动分割的方法
CN113516658B (zh) * 2021-09-14 2021-12-17 之江实验室 一种pet三维图像左心室自动转向及分割的方法
CN113838068A (zh) * 2021-09-27 2021-12-24 深圳科亚医疗科技有限公司 心肌节段的自动分割方法、装置和存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109584254A (zh) * 2019-01-07 2019-04-05 浙江大学 一种基于深层全卷积神经网络的心脏左心室分割方法
CN110120051A (zh) * 2019-05-10 2019-08-13 上海理工大学 一种基于深度学习的右心室自动分割方法
CN110163876A (zh) * 2019-05-24 2019-08-23 山东师范大学 基于多特征融合的左心室分割方法、系统、设备及介质
WO2020019740A1 (zh) * 2018-07-24 2020-01-30 深圳先进技术研究院 左心室心肌分割方法、装置及计算机可读存储介质
CN111242956A (zh) * 2020-01-09 2020-06-05 西北工业大学 基于U-Net超声胎心和胎肺深度学习联合分割方法
CN112508949A (zh) * 2021-02-01 2021-03-16 之江实验室 一种spect三维重建图像左心室自动分割的方法

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20150099169A (ko) * 2014-02-21 2015-08-31 전남대학교병원 심근관류 측정방법 및 그를 이용한 심장 허혈 진단방법
CN107545584B (zh) * 2017-04-28 2021-05-18 上海联影医疗科技股份有限公司 医学图像中定位感兴趣区域的方法、装置及其系统
CN107220965B (zh) * 2017-05-05 2021-03-09 上海联影医疗科技股份有限公司 一种图像分割方法及系统
JP6483875B1 (ja) 2018-01-25 2019-03-13 日本メジフィジックス株式会社 心筋画像表示方法、心筋画像表示処理プログラム及び心筋画像処理装置
CN109801294A (zh) * 2018-12-14 2019-05-24 深圳先进技术研究院 三维左心房分割方法、装置、终端设备及存储介质
JP2019128358A (ja) 2019-02-13 2019-08-01 日本メジフィジックス株式会社 心筋画像表示方法、心筋画像表示処理プログラム及び心筋画像処理装置
CN111340186B (zh) * 2020-02-17 2022-10-21 之江实验室 基于张量分解的压缩表示学习方法
CN111369537A (zh) * 2020-03-05 2020-07-03 上海市肺科医院（上海市职业病防治院） 一种肺磨玻璃结节的自动分割系统及方法
CN111739161B (zh) * 2020-07-23 2020-11-20 之江实验室 一种有遮挡情况下的人体三维重建方法、装置及电子设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020019740A1 (zh) * 2018-07-24 2020-01-30 深圳先进技术研究院 左心室心肌分割方法、装置及计算机可读存储介质
CN109584254A (zh) * 2019-01-07 2019-04-05 浙江大学 一种基于深层全卷积神经网络的心脏左心室分割方法
CN110120051A (zh) * 2019-05-10 2019-08-13 上海理工大学 一种基于深度学习的右心室自动分割方法
CN110163876A (zh) * 2019-05-24 2019-08-23 山东师范大学 基于多特征融合的左心室分割方法、系统、设备及介质
CN111242956A (zh) * 2020-01-09 2020-06-05 西北工业大学 基于U-Net超声胎心和胎肺深度学习联合分割方法
CN112508949A (zh) * 2021-02-01 2021-03-16 之江实验室 一种spect三维重建图像左心室自动分割的方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116934768A (zh) * 2023-08-16 2023-10-24 中国人民解放军总医院 用于提高cta影像模态中血管分割精度的方法及系统
CN116934768B (zh) * 2023-08-16 2024-05-10 中国人民解放军总医院 用于提高cta影像模态中血管分割精度的方法及系统

Also Published As

Publication number Publication date
CN112508949B (zh) 2021-05-11
JP7270319B2 (ja) 2023-05-10
CN112508949A (zh) 2021-03-16
JP2023516227A (ja) 2023-04-18

Similar Documents

Publication Publication Date Title
WO2022161192A1 (zh) 一种spect三维重建图像左心室自动分割的方法
US20230104173A1 (en) Method and system for determining blood vessel information in an image
CN109166133B (zh) 基于关键点检测和深度学习的软组织器官图像分割方法
Lötjönen et al. Statistical shape model of atria, ventricles and epicardium from short-and long-axis MR images
Zheng et al. Four-chamber heart modeling and automatic segmentation for 3-D cardiac CT volumes using marginal space learning and steerable features
Zhuang et al. A registration-based propagation framework for automatic whole heart segmentation of cardiac MRI
US7359538B2 (en) Detection and analysis of lesions in contact with a structural boundary
CN113570627B (zh) 深度学习分割网络的训练方法及医学图像分割方法
WO2023040628A1 (zh) 一种pet三维图像左心室自动转向及分割的方法
Cai et al. A framework combining window width-level adjustment and Gaussian filter-based multi-resolution for automatic whole heart segmentation
CN115830016B (zh) 医学图像配准模型训练方法及设备
US20230394670A1 (en) Anatomically-informed deep learning on contrast-enhanced cardiac mri for scar segmentation and clinical feature extraction
CN114170150A (zh) 基于曲率损失函数的视网膜渗出液全自动分割方法
CN107909653B (zh) 一种基于稀疏主成分分析的心脏软组织三维重建方法
CN112164447B (zh) 图像处理方法、装置、设备及存储介质
CN111862320B (zh) 一种spect三维重建图像到标准视图的自动转向方法
CN112598669B (zh) 一种基于数字人技术的肺叶分割方法
Lötjönen et al. Four-chamber 3-D statistical shape model from cardiac short-axis and long-axis MR images
Li et al. 3D intersubject warping and registration of pulmonary CT images for a human lung model
Villard et al. ISACHI: integrated segmentation and alignment correction for heart images
CN111667515A (zh) 基于流形正则化的血管3d/2d弹性配准方法及装置
Cheng et al. 3-D Reconstruction of Medical Image Using Wavelet Transform and Snake Model.
CN114820754B (zh) 一种基于模板匹配的心脏四腔心自动定位方法
Saif et al. Localization of Left Ventricular Epicardium and Endocardium Using Convolutional Neural Network and Transfer Learning
CN117422684A (zh) 一种心脏磁共振左心室心肌自动化分段方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22745069

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022564825

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22745069

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 22745069

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 12-02-2024)