WO2022183851A1 - 一种基于数字人技术的肺叶分割方法 - Google Patents

一种基于数字人技术的肺叶分割方法 Download PDF

Info

Publication number
WO2022183851A1
WO2022183851A1 PCT/CN2022/072198 CN2022072198W WO2022183851A1 WO 2022183851 A1 WO2022183851 A1 WO 2022183851A1 CN 2022072198 W CN2022072198 W CN 2022072198W WO 2022183851 A1 WO2022183851 A1 WO 2022183851A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
digital human
lung
lung lobe
model
Prior art date
Application number
PCT/CN2022/072198
Other languages
English (en)
French (fr)
Inventor
朱闻韬
饶璠
张铎
Original Assignee
之江实验室
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 之江实验室 filed Critical 之江实验室
Priority to JP2022562001A priority Critical patent/JP7378694B2/ja
Publication of WO2022183851A1 publication Critical patent/WO2022183851A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30061—Lung
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/44—Morphing

Definitions

  • the invention relates to the field of image segmentation, in particular to a lung lobe segmentation method based on digital human technology.
  • Lung lobe segmentation is a method to obtain lobe boundary information through image segmentation. It is an important prerequisite for lung visualization and lung quantitative analysis, and plays a very important role in the early diagnosis and treatment of lung cancer.
  • the functions of lung lobes are relatively independent and lung diseases usually occur in a single lung lobe.
  • Accurate lobe segmentation is the premise of many lung surgeries (such as lobe volume reduction surgery). Lung lobes are separated by fissures, but in practical applications, lobe segmentation is affected by factors such as limited CT resolution, incomplete fissures, abnormal surrounding structural distribution, and abnormal lung parenchyma.
  • Lob segmentation is still an important part of lung image processing. difficult problem.
  • the existing methods are mainly divided into two categories.
  • the first type is to obtain lung fissures by the method of lung fissure detection, and then obtain the surface of the lung lobes through the lung fissures; the second type is to directly use the method of image segmentation to obtain the lung lobes.
  • Some scholars designed the VanderBurg linear operator to detect lung fissures based on the structural characteristics of lung fissures in two-dimensional space. However, this method can only be applied to lung images without lesions. Many researchers combine this method with structural characteristics and anatomical knowledge. Combined to improve the stability and accuracy of fissure detection. In addition, with the rapid development of deep learning, some people have applied the knowledge of deep learning to lung fissure detection.
  • Some scholars have proposed a multi-layer Seg3DNet-based FissureNet lung fissure detection network. The images are processed separately and adopt a coarse-to-fine strategy. This method has been verified in mainstream databases and achieved better detection results than traditional methods.
  • the digital human model technology is a method of learning the shape parameters of the atlas from a large number of sample images through statistical modeling methods, and it uses these parameters to adjust the shape of the organs in the model.
  • the digital human model generally includes the following steps: image segmentation, surface registration of standards and individuals, and construction of a digital human statistical map.
  • the digital human model has boundary point cloud or mask image information of organs.
  • the present invention proposes a lung lobe segmentation method based on digital human technology, which realizes lung lobe segmentation based on image registration.
  • the method obtains a digital human model through registration and data statistics of lung images, uses the digital human model to generate a digital human image, registers the digital human image with the lung image to be segmented, and generates a digital human image according to the shape parameters of the deformed digital human image.
  • the new digital human image is continuously iterated to obtain a digital human image that is closer to the patient's lung image to be segmented, and then the new digital human image is non-rigidly registered with the patient's to-be-segmented lung image to obtain a deformation field.
  • the lobe segmentation results obtained by adding to the boundary point cloud or mask image of the digital human lobe.
  • a lung lobe segmentation method based on digital human technology comprising the following steps:
  • Step 1 Obtain a digital human model through registration and statistical lung image construction
  • Step 2 use the digital human image in the digital human model and the lung image to be segmented to perform non-rigid registration to obtain the deformation field and the deformed digital human image;
  • Step 3 Fitting the shape parameters of the digital human to the deformed digital human image in the second step, and generating a new digital human image through the digital human model in the first step according to the shape parameters of the digital human;
  • Step 4 Iteratively execute the new digital human image according to steps 2 and 3 for several times until the threshold of the number of iterations is reached or the shape parameters of the digital human are converged;
  • Step 5 Perform non-rigid registration of the digital human image obtained in the last iteration with the lung image to be segmented and obtain the deformation field;
  • Step 6 Add the deformation field obtained in Step 5 to the boundary point cloud or mask image of the digital human image lung lobe obtained in the last iteration to obtain the lung lobe segmentation result.
  • the trained non-rigid registration model is used to perform non-rigid registration on the digital human image in the digital human model and the lung image to be segmented; the non-rigid registration model uses UNet as the registration network.
  • the input of the non-rigid registration model is the digital human image in the digital human model and the lung image to be segmented, the registration network outputs the deformation field, and the deformation network outputs the deformed digital human image.
  • the non-rigid registration model uses a reference image and a floating image pair as a training set, wherein the reference image and the lobe mask of the floating image are used as labels, and the objective function used during training is as follows:
  • CT1 represents the reference image
  • CT2 represents the floating image
  • M1 is the lobe mask of CT1
  • STN(CT2) is the deformed image of CT2
  • STN(M2) is the deformed mask image
  • D is the image similarity measure function
  • D ice is the Dice metric function
  • R(DVF) is the regularization term of the image deformation field.
  • step 3 adopt singular value decomposition, calculate generalized inverse or train the VGG network to fit the shape parameter of digital human to the digital human image after deformation in step 2.
  • the VGG network consists of a convolutional layer, a maximum pooling layer and a fully connected layer.
  • the mask image is obtained by converting the closed boundary point cloud data by using image filling or seed growth method.
  • the type of the lung image is a CT image, an MRI image, an ultrasound image, or a PET image.
  • the method of the present invention uses the digital human model to perform organ segmentation in medical images for the first time, and the digital human model can be used to obtain the lung digital human images of different postures by setting shape parameters, and the digital human image generated by the digital human model does not Distortion characteristics, iteratively register the digital human image with the lung image to be segmented to generate a new digital human image, so that the final generated digital human image is closer to the image to be segmented, thereby effectively improving the patient images with abnormalities or lesions. Accuracy and stability of lower lobe segmentation.
  • Figure 1 is the overall flow chart of lung lobe segmentation
  • Figure 2 is a structural diagram of a non-rigid registration model based on deep learning
  • Fig. 3 is the shape parameter fitting neural network structure diagram based on deep learning
  • Fig. 4 is the structure diagram of the coding network in the shape parameter fitting neural network.
  • Figure 1 is a schematic diagram of a lung lobe segmentation method based on digital human technology, and the method includes the following steps:
  • Step 1 Generate a lung digital human model including lung lobe mask information through registration and data statistics of lung CT images, which specifically includes the following sub-steps:
  • the i-th sample X i ⁇ (x 1 ,y 1 ),(x 2 ,y 2 ),(x k ,y k ),...,(x 1 ,y 1 ) ⁇ (taking the two-dimensional case as an example) Contains k vertices, then the data is counted and normalized:
  • b [b 1 , b 2 ,...b c ,] is the shape parameter, which can be calculated by the least square method or matrix orthogonalization.
  • Step 2 Use the digital human image in the digital human model and the lung CT image to be segmented to perform non-rigid registration to obtain the deformation field and the deformed digital human image.
  • the trained non-rigid registration model is used to realize non-rigid registration.
  • the structure of the non-rigid registration model is shown in Figure 2.
  • UNet is used as the registration network, followed by a deformation network (STN: Spatial Transform Network, spatial deformation network).
  • STN Spatial Transform Network, spatial deformation network.
  • the input of the non-rigid registration model is the reference image and the to-be-matched image.
  • the quasi-floating image is registered by the registration network, and the deformation field is output.
  • the deformation field and the digital human image are obtained through the deformation network to obtain the deformed image.
  • the images in the training set are divided into lobe masks as labels, and the lobe masks as floating images are input to the deformation network during training to obtain the deformed point set.
  • the training objective function includes image similarity metric (such as NCC metric), mask image similarity metric (Dice metric), and deformation field regularization term (the L2 norm of the first derivative
  • CT1 represents the reference image
  • CT2 represents the floating image
  • M1 is the lobe mask of CT1
  • STN(CT2) is the deformed image of CT2
  • STN(M2) is the deformed mask image point set
  • R(DVF) Regularization term for the image deformation field.
  • the digital human image and its lung lobe mask in step 1 and the clinically collected lung CT image and its lung lobe mask are used to train the model until the objective function converges, and the trained non-rigid registration model can be obtained.
  • the digital human image (AAM image) is input to the non-rigid registration model as a floating image and the lung CT image to be segmented as a reference image, and the deformation field and the deformed digital human image can be obtained.
  • Step 3 Fitting the shape parameters of the digital human to the deformed digital human image in the second step, and generating a new digital human image according to the shape parameters of the digital human;
  • a neural network method is also used to fit the shape parameters of the deformed digital human image.
  • the neural network for shape parameter fitting is shown in FIG. 3 and includes an encoding network and an image generation network.
  • the encoding network structure As shown in Figure 3 and Table 1, the image generation network is implemented by Equation (7).
  • the coding network outputs the fitted shape parameter
  • the fitted shape parameter outputs the fitted image through the image generation network.
  • the input image and the output fitted image be A and B, respectively
  • the label shape parameter and the fitting shape parameter are b, respectively
  • the objective function can be expressed as:
  • NCC(A, B) is the normalized correlation coefficient between A and B
  • ⁇ 1 , ⁇ 2 are the weights
  • F is the F-norm.
  • the digital human image in the digital human model constructed in step 1 is used as input, and the shape parameters are used as labels to train the network. When the objective function no longer decreases, the training is completed.
  • step 2 the deformed digital human image in step 2 is input into the encoding network to obtain the fitted shape parameters.
  • Step 4 Repeat steps 5 and 6 to perform registration and shape parameter fitting on the new digital human image for many times until the threshold of the number of iterations is reached or the shape parameters of the digital human are converged, and the digital human image generated by the final digital human model is obtained. , the digital human image is closer to the lung CT image to be segmented.
  • Step 5 Once again, the new digital human image and the lung CT image to be segmented obtained in step 4 are used as the input of the non-rigid registration model constructed in step 2 to obtain the registered deformation field.
  • Step 6 Add the deformation field obtained in Step 5 to the boundary point cloud or mask image of the lung lobe of the digital human image, and the obtained result is the lung lobe segmentation result obtained by this method.
  • the neural network algorithm is used to realize non-rigid image registration and digital human shape parameter fitting, the calculation speed is faster, and the lung lobe segmentation result can be obtained quickly.
  • the present invention can also use the elastix function in the Elastix toolkit to obtain the deformation field of the digital human image and the clinical patient's lung CT image and the deformed digital human image, specifically:
  • a lung lobe segmentation method based on digital human technology includes the following steps:
  • Step 1 Using the elastix function in the Elastix toolkit, the digital human image generated by the digital human model constructed in Example 1 and the clinical patient's lung CT image are used as input to obtain a deformation field and a deformed digital human image.
  • Step 2 Use the deformed digital human image in step 1 to fit the shape parameters of the digital human by using the trained coding network (FIG. 4), and generate a new digital human image according to the shape parameters of the digital human.
  • Step 3 Repeat steps 1 and 2 until the threshold of the number of iterations is reached or the shape parameters of the digital human are converged, and a new digital human image generated after final fitting is obtained.
  • Step 4 Using the elastix function in the Elastix toolkit, the new digital human image and the clinical patient's lung CT image generated in step 4 are used as input to obtain the deformation field.
  • Step 5 Add the deformation field obtained in Step 4 to the boundary point cloud or mask image of the digital human lung lobe, and the obtained result is the lung lobe segmentation result obtained by this method.
  • the present invention can also connect the trained non-rigid registration model with the encoding network, and output the fitted shape parameters in one step, as follows:
  • a lung lobe segmentation method based on digital human technology includes the following steps:
  • Step 1 The digital human image generated by the digital human model constructed in Example 1 and the clinical patient's lung CT image are used as the input of the joint network composed of the non-rigid registration model and the coding network connection, and the deformed digital human image is obtained. shape parameters, and generate a new digital human image according to the shape parameters of the digital human.
  • Step 2 Repeat step 1 for several times until the threshold of the number of iterations is reached or the shape parameters of the digital human are converged, and a new digital human image generated after final fitting is obtained.
  • Step 3 The new digital human image and the clinical patient's lung CT image generated in step 2 are used as the input of the non-rigid registration model to obtain the deformation field.
  • Step 4 Add the deformation field obtained in Step 3 to the boundary point cloud or mask image of the digital human lung lobe to obtain the lung lobe segmentation result.
  • the present invention is mainly used for CT images, but can also be extended to MRI, ultrasound and PET images.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Quality & Reliability (AREA)
  • Radiology & Medical Imaging (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Geometry (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Analysis (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)
  • Nuclear Medicine (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)

Abstract

本发明公开了一种基于数字人技术的肺叶分割方法，该方法通过数字人图像和临床上患者的肺部图像进行非刚性配准，得到变形场和形变后的数字人图像，采用形变后的数字人图像拟合出数字人的形状参数并根据形状参数生成新的数字人图像，再将新的数字人图像与患者肺部图像不断迭代配准和更新，得到更接近于患者肺部图像的数字人图像，最后将数字人图像与患者肺部图像进行非刚性配准并获得变形场，将变形场加到数字人肺叶的边界点云或掩模图像上，得到的结果即为本方法得到的肺叶分割结果。本发明方法首次利用数字人模型进行医学图像中器官分割，本发明方法可以有效提高患者图像存在异常或病变情形下肺叶分割的精度和稳定性。

Description

一种基于数字人技术的肺叶分割方法 技术领域
本发明涉及图像分割领域，尤其涉及一种基于数字人技术的肺叶分割方法。
背景技术
肺叶分割是通过图像分割的方法获得肺叶边界信息的方法，是肺部可视化和肺部定量分析的重要前提，在肺癌的早期诊断和治疗中发挥着非常重要的作用。在临床中，肺叶的功能相对独立且肺部疾病通常发生在单个肺叶中，精确的肺叶分割是众多肺部手术(如肺叶减容手术)的前提。肺叶是由肺裂隔开的，但在实际应用中，肺叶分割受到CT分辨率有限、肺裂不完整、周围异常结构分布以及肺实质异常等因素的影响，肺叶分割依然是肺部图像处理中的难点问题。
现有的方法主要分为两类，第一类是采用肺裂检测的方法得到肺裂，再通过肺裂得到肺叶表面；第二类是直接采用图像分割的方法得到肺叶。有学者针对二维空间内肺裂的结构特征设计VanderBurg线性算子进行了肺裂检测，但该方法只能运用于无病变的肺部图像，大量研究人员将该方法与结构特征和解剖学知识相结合以提高肺裂检测的稳定性和精度。另外，随着深度学习的迅速发展，也有人将深度学习的相关知识应用于肺裂检测，有学者提出了基于多层Seg3DNet的FissureNet肺裂检测网络，该方法将肺部左侧图像和右侧图像分开处理并采用由粗到细的策略，该方法已在主流的数据库中进行了验证，取得了比传统方法更好的检测结果。
随着计算机技术的发展，直接采用图像分割得到肺叶的方法也得到了迅速的发展，这类方法更强调对先验知识的利用。这类方法中最典型的就是基于Atlas的肺叶分割，该方法首先建立一套肺叶图像的Atlas图集，在选择图集中与病人图像最接近的图像与病人图像进行配准从而得到肺叶分割结果。另外也有学者采用深度学习的方法实现了肺叶的分割，Ferreira等提出了一个用于肺叶分割的FRV-Net，该方法基于VNet结构并在每层都计算Dice函数，仅需要少量样本数就能完成模型的训练。
数字人模型技术是通过统计建模方法从大量样本图像中学习得到图谱形状参数的一种方法，它利用这些参数对模型中器官形状进行调整。数字人模型一般包括以下几个步骤：图像分割、标准与个体的曲面配准和构建数字人统计图谱。数字人模型中具有器官的边界点云或者掩模图像信息。
发明内容
本发明针对现有技术的不足，提出了一种基于数字人技术的肺叶分割方法，该方法基于图像配准实现肺叶分割。该方法通过配准和数据统计肺部图像获得数字人模型，利用数字人模型生成数字人图像，将数字人图像与待分割肺部图像进行配准，根据形变后的数字人图像的形状参数生成新的数字人图像，不断迭代从而得到更接近于患者待分割肺部图像的数字人图像再将新的数字人图像与患者待分割肺部图像进行非刚性配准并获得变形场，将变形场加到数字人肺叶的边界点云或掩模图像上从而得到的肺叶分割结果。
本发明的目的是通过以下技术方案来实现的：一种基于数字人技术的肺叶分割方法，包括以下步骤：
步骤一：通过配准和数据统计肺部图像构建获得数字人模型；
步骤二：采用数字人模型中的数字人图像和待分割肺部图像进行非刚性配准，得到变形场和形变后的数字人图像；
步骤三：对步骤二中形变后的数字人图像拟合数字人的形状参数，并依据数字人的形状参数通过步骤一的数字人模型生成新的数字人图像；
步骤四：将新的数字人图像按照步骤二和步骤三迭代执行多次，直至达到迭代次数阈值或者数字人的形状参数收敛；
步骤五：将最后一次迭代得到的数字人图像与待分割肺部图像进行非刚性配准并获得变形场；
步骤六：将步骤五中得到的变形场加在最后一次迭代得到的数字人图像肺叶的边界点云或掩模图像上，得到肺叶分割结果。
进一步地，所述步骤二中，采用训练好的非刚性配准模型对数字人模型中的数字人图像和待分割肺部图像进行非刚性配准；非刚性配准模型采用UNet作为配准网络，其后连接形变网络，非刚性配准模型的输入为数字人模型中的数字人图像及待分割肺部图像，配准网络输出变形场，形变网络输出形变后的数字人图像。
进一步地，所述非刚性配准模型采用参考图像和浮动图像对作为训练集，其中，参考图像和浮动图像的肺叶掩模作为标签，训练时采用的目标函数如下：
min:f＝D(CT1,STN(CT2))+D ice(M1,STN(M2))+R(DVF)
式中，CT1表示参考图像，CT2代表浮动图像；M1为CT1的肺叶掩模；STN(CT2)为CT2形变后的图像，STN(M2)为形变后的掩模图像；D为图像相似性度量函数，D ice为Dice度量函数；R(DVF)为图像变形场的正则化项。
进一步地，所述步骤三中，采用奇异值分解、计算广义逆或训练好的VGG网络对步 骤二中形变后的数字人图像拟合数字人的形状参数。
进一步地，所述VGG网络由卷积层、最大池化层和全连接层组成。
进一步地，所述掩模图像采用图像填充或种子生长方法对封闭的边界点云数据进行转换获得。
进一步地，所述肺部图像的类型为CT图像MRI图像、超声图像或PET图像等。
本发明的有益效果是：本发明方法首次利用数字人模型进行医学图像中器官分割，利用数字人模型能够通过设置形状参数得到不同体态的肺部数字人图像，且数字人模型生成数字人图像不失真的特性，将数字人图像与待分割肺部图像不断迭代配准生成新的数字人图像，使最终生成的数字人图像更接近于待分割的图像，从而有效提高患者图像存在异常或病变情形下肺叶分割的精度和稳定性。
附图说明
图1是肺叶分割整体流程图；
图2是基于深度学习的非刚性配准模型的结构图；
图3是基于深度学习的形状参数拟合神经网络结构图；
图4是形状参数拟合神经网络中的编码网络结构图。
具体实施方式
下面根据实施例和附图详细说明本发明。
实施例1
如图1为基于数字人技术的肺叶分割方法的示意图，该方法包括以下步骤：
步骤一：通过配准和数据统计肺部CT图像生成包含肺叶掩模信息的肺部数字人模型，具体包括如下子步骤：
(1.1)对收集的每张肺部CT图像利用阈值划分进行肺叶分割预处理，然后再通过交互式分割软件进行精确肺叶分割。
(1.2)将每张分割后的肺部CT图像与模板进行曲面配准，得到配准后的边界点云数据，作为样本集。该过程常采用非刚性点云配准算法，如TPS-RPM以及non-rigid ICP等。
(1.3)构建数字人统计图谱，设样本集：
Ω＝{X 1,X 2,…,X N}             (1)
第i个样本X i＝{(x 1,y 1),(x 2,y 2),(x k,y k),…,(x 1,y 1)}(以二维情形为例)包含k个顶点，接着对数据进行统计并做归一化处理：
Figure PCTCN2022072198-appb-000001
Figure PCTCN2022072198-appb-000002
Figure PCTCN2022072198-appb-000003
接着根据主成分分析的流程进行计算平均向量和协方差矩阵：
Figure PCTCN2022072198-appb-000004
Figure PCTCN2022072198-appb-000005
接着计算协方差矩阵S的特征向量φ j及其对应特征值λ j，选取最大的前c个特征值及其对应的特征向量，则按照主成分分析的定义，每一个样本的形状估计可以表示为：
Figure PCTCN2022072198-appb-000006
其中b＝[b 1,b 2,…b c,]即为形状参数，可采用最小二乘法或矩阵正交化计算得到。
(1.4)将封闭的边界点云数据通过图像填充或种子生长转换成掩模图像。
步骤二：采用数字人模型中的数字人图像和待分割肺部CT图像进行非刚性配准，得到变形场和形变后的数字人图像。
本实施例中，利用训练好的非刚性配准模型实现非刚性配准。非刚性配准模型的结构如图2所示，采用UNet作为配准网络，其后连接形变网络(STN：Spatial Transform Network,空间形变网络)，非刚性配准模型的输入为参考图像和待配准的浮动图像，经配准网络配准后输出变形场，变形场加数字人图像经形变网络得到形变后的图像。模型训练时，对训练集中的图像划分肺叶掩模作为标签，在训练时同时将作为浮动图像的肺叶掩模输入至形变网络，得到形变后的点集。训练的目标函数包括图像相似性度量(如NCC度量)、掩模图像相似性度量(Dice度量)以及变形场正则化项(一阶导的L2范数)具体表示如下；
min:f＝D(CT1,STN(CT2))+D ice(M1,STN(M2))+R(DVF)
式中，CT1表示参考图像，CT2代表浮动图像；M1为CT1的肺叶掩模；STN(CT2)为CT2形变后的图像，STN(M2)为形变后的掩模图像点集；R(DVF)图像变形场的正则化项。
本实施例中采用步骤一中的数字人图像及其肺叶掩模和临床采集的肺部CT图像及其 肺叶掩模对模型进行训练直至目标函数收敛，即可得到训练后的非刚性配准模型。
最后将数字人图像(AAM图像)作为浮动图像和待分割肺部CT图像作为参考图像输入至非刚性配准模型，即可得到变形场和形变后的数字人图像。
步骤三：对步骤二中形变后的数字人图像拟合数字人的形状参数，并依据数字人的形状参数生成新的数字人图像；
本实施例中，同样采用神经网络方法对形变后的数字人图像的进行形状参数拟合，其中形状参数拟合的神经网络如图3所示，包含编码网络和图像生成网络，其中编码网络结构如图3和表1所示，图像生成网络由公式(7)实现。
表1 编码网络结构
Figure PCTCN2022072198-appb-000007
其中，编码网络输出拟合的形状参数，拟合的形状参数经图像生成网络输出拟合后的图像。设输入图像和输出的所拟合图像分别为A,B，标签形状参数和拟合形状参数分别为b，
Figure PCTCN2022072198-appb-000008
则目标函数可以表示为：
Figure PCTCN2022072198-appb-000009
其中NCC(A,B)为A,B之间的归一化相关系数，μ 1,μ 2为权值，||*|| F为F-范数。
利用步骤一构建的数字人模型中的数字人图像作为输入，形状参数作为标签对网络进行训练，当目标函数不再下降时，训练完成。
最后将步骤二中形变后的数字人图像输入至编码网络即可得到拟合的形状参数。
步骤四：重复执行步骤五和步骤六对新的数字人图像进行配准和形状参数拟合多次，直到达到迭代次数阈值或者数字人的形状参数收敛，得到最终数字人模型生成的数字人图像，该数字人图像更接近于待分割的肺部CT图像。
步骤五：再一次将步骤四中得到的新数字人图像和待分割的肺部CT图像作为步骤二中构建的非刚性配准模型的输入，得到配准的变形场。
步骤六：将步骤五中得到的变形场加在数字人图像肺叶的边界点云或掩模图像上，得到的结果即为本方法得到的肺叶分割结果。
本实施例采用神经网络算法实现了非刚性图像配准以及数字人形状参数拟合，计算速度更快，可以快速得到肺叶分割结果。
实施例2
作为一优选方案，本发明还可以采用Elastix工具包中elastix函数获取数字人图像和临床上患者的肺部CT图像的变形场和形变后的数字人图像，具体为：
一种基于数字人技术的肺叶分割方法，该方法包括以下步骤：
步骤一：采用Elastix工具包中elastix函数，将实施例1中构建的数字人模型生成的数字人图像和临床上患者的肺部CT图像作为输入，得到变形场和形变后的数字人图像。
步骤二：采用步骤一中形变后的数字人图像利用训练好的编码网络(图4)拟合数字人的形状参数，并依据数字人的形状参数生成新的数字人图像。
步骤三：重复执行步骤一和步骤二多次，直到达到迭代次数阈值或者数字人的形状参数收敛，得到最终拟合后生成的新的数字人图像。
步骤四：采用Elastix工具包中elastix函数，将步骤四中生成的新的数字人图像和临床上患者的肺部CT图像作为输入，得到变形场。
步骤五：将步骤四中得到的变形场加在数字人肺叶的边界点云或掩模图像上，得到的结果即为本方法得到的肺叶分割结果。
实施例3
作为另一优选方案，本发明还可以将训练好的非刚性配准模型和编码网络连接，一步输出拟合后的形状参数，具体如下：
一种基于数字人技术的肺叶分割方法，该方法包括以下步骤：
步骤一：将实施例1中构建的数字人模型生成的数字人图像和临床上患者的肺部CT图像作为非刚性配准模型和编码网络连接组成的联合网络的输入，获得形变后数字人的形状参数，并依据数字人的形状参数生成新的数字人图像。
步骤二：重复执行步骤一多次，直到达到迭代次数阈值或者数字人的形状参数收敛，得到最终拟合后生成的新的数字人图像。
步骤三：将步骤二中生成的新的数字人图像和临床上患者的肺部CT图像作为非刚性配准模型的输入，得到变形场。
步骤四：将步骤三中得到的变形场加在数字人肺叶的边界点云或掩模图像上，得到肺叶分割结果。
本发明主要用于CT图像，但也可扩展至MRI、超声和PET图像。
显然，上述实施例仅仅是为清楚地说明所作的举例，而并非对实施方式的限定。对于所属领域的普通技术人员来说，在上述说明的基础上还可以做出其他不同形式的变化或变动。这里无需也无法把所有的实施方式予以穷举。而由此所引申出的显而易见的变化或变动仍处于本发明的保护范围。

Claims (7)

  1. 一种基于数字人技术的肺叶分割方法，其特征在于，包括以下步骤：
    步骤一：通过配准和数据统计肺部图像构建获得数字人模型；
    步骤二：采用数字人模型中的数字人图像和待分割肺部图像进行非刚性配准，得到变形场和形变后的数字人图像；
    步骤三：对步骤二中形变后的数字人图像拟合数字人的形状参数，并依据数字人的形状参数通过步骤一的数字人模型生成新的数字人图像；
    步骤四：将新的数字人图像按照步骤二和步骤三迭代执行多次，直至达到迭代次数阈值或者数字人的形状参数收敛；
    步骤五：将最后一次迭代得到的数字人图像与待分割肺部图像进行非刚性配准并获得变形场；
    步骤六：将步骤五中得到的变形场加在最后一次迭代得到的数字人图像肺叶的边界点云或掩模图像上，得到肺叶分割结果。
  2. 根据权利要求1所述的肺叶分割方法，其特征在于，所述步骤二中，采用训练好的非刚性配准模型对数字人模型中的数字人图像和待分割肺部图像进行非刚性配准；非刚性配准模型采用UNet作为配准网络，其后连接形变网络，非刚性配准模型的输入为数字人模型中的数字人图像和待分割肺部图像，配准网络输出变形场，形变网络输出形变后的数字人图像。
  3. 根据权利要求2所述的肺叶分割方法，其特征在于，所述非刚性配准模型采用参考图像和浮动图像对作为训练集，其中，参考图像和浮动图像的肺叶掩模作为标签，训练时采用的目标函数如下：
    min:f＝D(CT1,STN(CT2))+D ice(M1,STN(M2))+R(DVF)
    式中，CT1表示参考图像，CT2代表浮动图像；M1为CT1的肺叶掩模；STN(CT2)为CT2形变后的图像，STN(M2)为形变后的掩模图像；D为图像相似性度量函数，D ice为Dice度量函数；R(DVF)为图像变形场的正则化项。
  4. 根据权利要求1所述的肺叶分割方法，其特征在于，所述步骤三中，采用奇异值分解、计算广义逆或训练好的VGG网络对步骤二中形变后的数字人图像拟合数字人的形状参数。
  5. 根据权利要求4所述的肺叶分割方法，其特征在于，所述VGG网络由卷积层、最大池化层和全连接层组成。
  6. 根据权利要求1所述的肺叶分割方法，其特征在于，所述掩模图像采用图像填充或种子生长方法对封闭的边界点云数据进行转换获得。
  7. 根据权利要求1所述的肺叶分割方法，其特征在于，所述肺部图像的类型为CT图像MRI图像、超声图像或PET图像。
PCT/CN2022/072198 2021-03-04 2022-01-15 一种基于数字人技术的肺叶分割方法 WO2022183851A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2022562001A JP7378694B2 (ja) 2021-03-04 2022-01-15 デジタルヒューマン技術に基づく肺葉分割方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110241193.X 2021-03-04
CN202110241193.XA CN112598669B (zh) 2021-03-04 2021-03-04 一种基于数字人技术的肺叶分割方法

Publications (1)

Publication Number Publication Date
WO2022183851A1 true WO2022183851A1 (zh) 2022-09-09

Family

ID=75210333

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/072198 WO2022183851A1 (zh) 2021-03-04 2022-01-15 一种基于数字人技术的肺叶分割方法

Country Status (3)

Country Link
JP (1) JP7378694B2 (zh)
CN (1) CN112598669B (zh)
WO (1) WO2022183851A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112598669B (zh) * 2021-03-04 2021-06-01 之江实验室 一种基于数字人技术的肺叶分割方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105956386A (zh) * 2016-04-27 2016-09-21 深圳市智影医疗科技有限公司 基于健康人胸片的健康指标指数分类系统和方法
CN109658425A (zh) * 2018-12-12 2019-04-19 上海联影医疗科技有限公司 一种肺叶分割方法、装置、计算机设备及存储介质
CN109903264A (zh) * 2019-01-16 2019-06-18 深圳市旭东数字医学影像技术有限公司 数字人图像与ct图像的配准方法及系统
US20190220701A1 (en) * 2018-01-16 2019-07-18 Siemens Healthcare Gmbh Trained generative network for lung segmentation in medical imaging
WO2020044840A1 (ja) * 2018-08-31 2020-03-05 富士フイルム株式会社 領域分割装置、方法およびプログラム、類似度決定装置、方法およびプログラム、並びに特徴量導出装置、方法およびプログラム
CN111583385A (zh) * 2020-04-15 2020-08-25 滨州医学院 一种可变形数字人解剖学模型的个性化变形方法及系统
CN112598669A (zh) * 2021-03-04 2021-04-02 之江实验室 一种基于数字人技术的肺叶分割方法

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10878576B2 (en) * 2018-02-14 2020-12-29 Elekta, Inc. Atlas-based segmentation using deep-learning
CN109636808B (zh) * 2018-11-27 2022-08-12 杭州健培科技有限公司 一种基于全卷积神经网络的肺叶分割方法
CN110335358B (zh) * 2019-06-18 2021-11-05 大连理工大学 可变形数字人解剖学模型的个性化变形方法
CN110610481A (zh) * 2019-08-06 2019-12-24 深圳市旭东数字医学影像技术有限公司 基于医学图像的脑干自动分割方法及系统
CN111127471B (zh) * 2019-12-27 2023-08-29 之江实验室 一种基于双标签损失的胃癌病理切片图像分割方法及系统
CN111260700B (zh) * 2020-01-09 2023-05-30 复旦大学 一种多参数磁共振图像全自动配准与分割方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105956386A (zh) * 2016-04-27 2016-09-21 深圳市智影医疗科技有限公司 基于健康人胸片的健康指标指数分类系统和方法
US20190220701A1 (en) * 2018-01-16 2019-07-18 Siemens Healthcare Gmbh Trained generative network for lung segmentation in medical imaging
WO2020044840A1 (ja) * 2018-08-31 2020-03-05 富士フイルム株式会社 領域分割装置、方法およびプログラム、類似度決定装置、方法およびプログラム、並びに特徴量導出装置、方法およびプログラム
CN109658425A (zh) * 2018-12-12 2019-04-19 上海联影医疗科技有限公司 一种肺叶分割方法、装置、计算机设备及存储介质
CN109903264A (zh) * 2019-01-16 2019-06-18 深圳市旭东数字医学影像技术有限公司 数字人图像与ct图像的配准方法及系统
CN111583385A (zh) * 2020-04-15 2020-08-25 滨州医学院 一种可变形数字人解剖学模型的个性化变形方法及系统
CN112598669A (zh) * 2021-03-04 2021-04-02 之江实验室 一种基于数字人技术的肺叶分割方法

Also Published As

Publication number Publication date
CN112598669A (zh) 2021-04-02
JP2023516219A (ja) 2023-04-18
CN112598669B (zh) 2021-06-01
JP7378694B2 (ja) 2023-11-14

Similar Documents

Publication Publication Date Title
WO2021088747A1 (zh) 基于深度学习的预测消融术后肝脏肿瘤形态变化的方法
CN106485695B (zh) 基于统计形状模型的医学图像Graph Cut分割方法
CN104504737B (zh) 一种从肺部ct图像获得三维气管树的方法
CN112258514B (zh) 一种ct影像肺血管的分割方法
Ye et al. Medical image diagnosis of prostate tumor based on PSP-Net+ VGG16 deep learning network
Heyde et al. Anatomical image registration using volume conservation to assess cardiac deformation from 3D ultrasound recordings
CN115830016B (zh) 医学图像配准模型训练方法及设备
CN110570430B (zh) 基于体配准的眼眶骨组织分割方法
Xiao et al. Deep learning-based lung image registration: A review
WO2022183851A1 (zh) 一种基于数字人技术的肺叶分割方法
CN111127488B (zh) 一种基于统计形状模型自动构建患者解剖结构模型的方法
Hu et al. Lung CT image registration through landmark-constrained learning with convolutional neural network
Guo et al. Automatic landmark detection and registration of brain cortical surfaces via quasi-conformal geometry and convolutional neural networks
CN113724320A (zh) 一种基于形状先验的肾脏图像分割方法
CN114943753B (zh) 基于局部结构矢量对齐的位姿配准方法及装置
Urschler et al. Assessing breathing motion by shape matching of lung and diaphragm surfaces
CN113658193B (zh) 基于信息融合的肝脏ct影像肿瘤分割方法
Zhang et al. Automatic implant shape design for minimally invasive repair of pectus excavatum using deep learning and shape registration
CN113160417B (zh) 一种基于泌尿系统的多器官三维重建控制方法
WO2021213053A1 (zh) 基于软组织表面形变估计组织内部目标运动的系统与方法
CN111798500B (zh) 一种基于层次邻域谱特征的微分同胚非刚性配准算法
Roy et al. MDL-IWS: multi-view deep learning with iterative watershed for pulmonary fissure segmentation
CN107808387A (zh) 一种医学图像序列中的目标跟踪方法
Lu et al. Three-dimensional lung medical image registration based on improved demons algorithm
CN110335348A (zh) 一种基于RFR-SSMs的骨盆股骨医学图像建模方法

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2022562001

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22762320

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22762320

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 22762320

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 28.02.2024)