WO2023108526A1 - 一种医学图像分割方法、系统、终端以及存储介质 - Google Patents

一种医学图像分割方法、系统、终端以及存储介质 Download PDF

Info

Publication number
WO2023108526A1
WO2023108526A1 PCT/CN2021/138686 CN2021138686W WO2023108526A1 WO 2023108526 A1 WO2023108526 A1 WO 2023108526A1 CN 2021138686 W CN2021138686 W CN 2021138686W WO 2023108526 A1 WO2023108526 A1 WO 2023108526A1
Authority
WO
WIPO (PCT)
Prior art keywords
medical image
segmentation
branch
model
data
Prior art date
Application number
PCT/CN2021/138686
Other languages
English (en)
French (fr)
Inventor
刘佳能
李志成
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Priority to PCT/CN2021/138686 priority Critical patent/WO2023108526A1/zh
Publication of WO2023108526A1 publication Critical patent/WO2023108526A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present application belongs to the technical field of medical image processing, and in particular relates to a medical image segmentation method, system, terminal and storage medium.
  • Medical image segmentation is the basis of various medical image applications. In clinical auxiliary diagnosis, image-guided surgery and radiation therapy, medical image segmentation technology shows more and more important clinical value.
  • Traditional medical image segmentation is based on manual segmentation by experienced doctors, and this purely manual segmentation method is often time-consuming and laborious, and is greatly affected by the doctor's subjective influence. Even experienced doctors may make mistakes when they are tired. of the division. Furthermore, the effectiveness of segmentation by inexperienced physicians is often difficult to measure.
  • the present application provides a medical image segmentation method, system, terminal and storage medium, aiming to solve one of the above-mentioned technical problems in the prior art at least to a certain extent.
  • a medical image segmentation method comprising:
  • the medical image sample data including multimodal medical images and clinical information of cases corresponding to the multimodal medical images;
  • the weak-semi-supervised model includes a segmentation branch for performing segmentation tasks and a survival prediction branch for performing survival prediction tasks, and input the medical image sample data into the segmentation branch and survival period prediction branch, the features extracted by the segmentation branch and lifetime prediction branch are fused and iteratively trained to obtain a trained image segmentation model;
  • the acquisition of medical image sample data includes:
  • the multimodal medical images are four modal images of FLAIR, T1, T2 and T1c of each case;
  • the clinical information includes the survival period and survival status of the case.
  • the technical solution adopted in the embodiment of the present application also includes: the acquisition of medical image sample data is specifically:
  • the medical image data set is grouped according to a set ratio to obtain a training set, a verification set and a test set.
  • the technical solution adopted in the embodiment of the present application also includes: the preprocessing of the multimodal medical image sample data and Mask data is specifically:
  • the multimodal medical image and the corresponding Mask data are clipped;
  • the clipping method is specifically as follows: obtain the center point of each multimodal medical image, expand the area of the set size outward with the center point, and cut off all the multimodal medical images. Parts other than the above-mentioned area are obtained to obtain the clipped medical image and Mask data;
  • the normalized medical images of the four modalities and the clipped Mask data are respectively spliced to obtain a preprocessed medical image data set.
  • the technical solution adopted in the embodiment of the present application also includes: the grouping of the medical image data sets according to the set ratio is specifically:
  • a 10-fold cross-validation algorithm is used, and 10% of the training set data is used as the validation set in each round.
  • described construction weak-semi-supervised model is 3D U-Net network
  • the training process of described 3D U-Net network comprises:
  • the training set data is respectively input into the segmentation branch and the lifetime prediction branch for down-sampling processing, and the features obtained by the down-sampling of the segmentation branch become one-dimensional features through the flatten operation, and then input to the Transformer module;
  • the Transformer module adopts the idea based on residual connection, adds the input feature to the data before input, reshape is the shape before input, and simultaneously leads the feature obtained by the Transformer module to the lifetime prediction branch; the lifetime prediction The branch converts the distribution of features through the Adapter module, and fuses the features output by the segmentation branch and the survival period prediction branch through the information fusion module, and then passes through the fully connected layer to obtain the risk value of the survival period prediction; the segmentation branch obtains After reshaping the feature map, the feature map is restored to the size of the initial input image by upsampling, and then binarized to obtain the output result of the segmentation task.
  • the technical solution adopted by the embodiment of the present application also includes: the training mode of the 3D U-Net network is specifically:
  • the training mode of Teacher-Student is used to train the 3D U-Net network; the pseudo labels generated by unlabeled medical images are added to the training set, and in each round of training, if the current training effect is better than the previous round, Then use the Teacher model to update the Student model, otherwise continue training. If the number of training times exceeds the set number of times and the Student model cannot be updated, the model is considered to have converged and the model training ends.
  • a medical image segmentation system comprising:
  • Data acquisition module used to acquire medical image sample data, the medical image sample data including multimodal medical images and clinical information of cases corresponding to the multimodal medical images;
  • Model training module used to construct a weak-semi-supervised model, the weak-semi-supervised model includes a segmentation branch for performing segmentation tasks and a survival period prediction branch for performing survival period prediction tasks, and the medical image sample data Inputting the segmentation branch and the lifetime prediction branch respectively, the features extracted by the segmentation branch and the lifetime prediction branch are fused and iteratively trained to obtain a trained image segmentation model;
  • Image segmentation module used to input the medical image to be segmented into the trained image segmentation model for image segmentation.
  • a terminal includes a processor and a memory coupled to the processor, wherein,
  • the memory stores program instructions for realizing the medical image segmentation method
  • the processor is configured to execute the program instructions stored in the memory to control medical image segmentation.
  • a storage medium storing program instructions executable by a processor, and the program instructions are used to execute the medical image segmentation method.
  • the beneficial effects produced by the embodiment of the present application are that the medical image segmentation method, system, terminal and storage medium of the embodiment of the present application adopt a semi-supervised segmentation method without relying on too much labeled data;
  • the weak supervision method high-level semantics such as survival period are used as the source of weak supervision, and the Transformer module is used to mine the correlation between features, so as to focus on the tumor area and further improve the segmentation accuracy.
  • Fig. 1 is the flowchart of the medical image segmentation method of the embodiment of the present application.
  • Fig. 2 is the 3D U-Net network structure schematic diagram of the embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a medical image segmentation system according to an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of a terminal according to an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a storage medium according to an embodiment of the present application.
  • FIG. 1 is a flowchart of a medical image segmentation method according to an embodiment of the present application.
  • the medical image segmentation method of the embodiment of the present application comprises the following steps:
  • S10 Obtain a certain amount of multimodal medical image sample data, and generate Mask (label) data of the multimodal medical image sample data;
  • the acquired multimodal medical image sample data includes four modal images of FLAIR, T1, T2 and T1c for each case and the clinical information of the case corresponding to the multimodal medical image.
  • the clinical information of the case includes the survival period And information such as survival status, the size of the four modal data is the same.
  • S20 Preprocess the multimodal medical image sample data and Mask data to generate a medical image dataset for model training, and group the medical image dataset according to a set ratio to obtain a training set, a verification set and a test set ;
  • the preprocessing process of multimodal medical image sample data specifically includes:
  • the cropping method specifically includes: finding the center point of each medical image, expanding a region of a set size outwards based on the center point, and cutting off parts outside the region to obtain a trimmed medical image.
  • the size of the clipping area is set to 96*128*128, which can be set according to actual operation.
  • the embodiment of the application uses the min-max algorithm to normalize the image pixel values of different modalities to 0. -1, and the clipped Mask data does not need to be normalized.
  • the medical image data size after splicing is 96*128*128*4.
  • the division ratio of the test set is 25%, and the other 75% is used as a training set and a verification set, and a 10-fold cross-validation algorithm is used, that is, 10% of the training set data is used as a verification set in each round, Model training can be performed using as much data as possible while evaluating the effect of model training.
  • S30 Construct a weak-semi-supervised model, and input the training set into the weak-semi-supervised model for iterative training to obtain a trained image segmentation model;
  • the embodiment of the present application uses the pytorch framework to construct a weak-semi-supervised model.
  • the weak-semi-supervised model is a 3D U-Net network transformed from a 2D U-Net network, that is, the 3D U-Net network in the 2D U-Net network All 2D operations such as 2D convolution and 2D pooling are replaced with 3D operations.
  • the 3D U-Net network structure is shown in Figure 2.
  • the 3D U-Net network includes two branch structures: a segmentation branch for performing segmentation tasks and a survival prediction branch for performing survival prediction tasks, and also includes residual modules, Transformer modules, Adapter modules, information fusion modules, and Survival Predict module, specifically:
  • the training set is input into the segmentation branch and the survival period prediction branch respectively.
  • the output of the segmentation branch is the feature extracted from the medical image data; the survival period prediction branch is implemented by a fully connected neural network, and the output value is the risk corresponding to each case. value. Due to the differences in the distribution of features extracted by different segmentation tasks, there may be cancellations if they are directly fused.
  • the features extracted by the segmentation branch are converted into data distribution through the Adapter module, so that the features obtained by the segmentation branch are fused.
  • the survival prediction task can also utilize the features learned in the segmentation task.
  • the data distribution conversion process of the Adapter module is as follows: firstly, the mean and std of the features in the lifetime prediction task are calculated by using the feature distribution of the lifetime prediction branch, and then the data transformation is performed on the segmentation branch, and the mean is subtracted from the features in the segmentation task Then divide by std so that the segmentation task has the same mean and std as the lifetime prediction task.
  • the features obtained by the segmentation branch are transformed into the same distribution as the features obtained by the lifetime prediction branch, which can effectively avoid the loss of information between features and provide information that cannot be provided by other tasks.
  • the loss function of the segmentation branch is Dice coefficient and BCE (Binary Cross Entropy) Loss
  • the loss function of the lifetime prediction branch is NegativeLogLikelihood (negative logarithmic likelihood).
  • the features extracted by the segmentation branch are converted into one-dimensional features through the flatten operation, and the one-dimensional features of each channel (channel) are spliced and then input into the Transformer module.
  • the Transformer module adopts the idea based on residual connection, uses the residual module to add the input features to the data before input, and then reshape to the shape before input, so as to further mine the internal relationship between different features.
  • each residual module is composed of two convolutional layers, that is, two convolution operations are performed and then the residual connection is performed. After each convolution, LeakyRelu is used for nonlinear mapping, and GroupNorm is used for normalization. ;
  • the information fusion module is used to merge the features output by the segmentation branch and the lifetime prediction branch by convolution, and perform reconvolution operation on the merged features.
  • the output result after convolution is the fused feature, so that Features learned in the survival prediction task are transferred to the segmentation task.
  • the 3D U-Net network adopts a double downsampling method, and the input data is respectively input into two encoders (that is, the segmentation branch and the lifetime prediction branch), and then the two encoders’ The output results are fused, and finally the fused image is introduced into the decoder for upsampling.
  • the 3D U-Net network training process of the embodiment of the present application includes: inputting the training set data into the segmentation branch and the lifetime prediction branch respectively for downsampling processing, performing a total of 3 downsamplings, and temporarily retaining each downsampling The result; the features obtained by the subsampling of the segmentation branch are first transformed into one-dimensional features through the flatten operation, and then input to the Transformer module; the Transformer module uses the idea based on residual connection to add the input features to the data before input, and then reshape is the shape before input, and at the same time, the features obtained by the Transformer module are exported to the lifetime prediction branch; the lifetime prediction branch converts the distribution of features through the Adapter module, and uses the information fusion module to output the segmentation branch and the lifetime prediction branch.
  • the features are fused, and then through the fully connected layer, the risk value of the survival period prediction can be obtained, and the risk value is binarized, and then the binarization result is used as an influencing factor to calculate its impact on the survival period.
  • the segmentation branch obtains the reshaped feature map, it undergoes 3 times of upsampling to restore the feature map to the size of the initial input image, and then undergoes binarization processing to obtain the output result of the segmentation task.
  • the embodiment of the present application adopts the Teacher-Student training mode to train the built 3D U-Net network.
  • the Teacher-Student training mode is specifically: add the pseudo-label generated by the image data without label to the training set.
  • update the Student model with the Teacher model otherwise continue Training, if the Student model fails to be updated after more than 20 rounds of training, it is considered that the model has converged and the training can be ended.
  • use the C-Index evaluation index and Dice coefficient to evaluate the training effect of the current model.
  • the 3D U-Net network of the embodiment of the present application adopts a semi-supervised segmentation method, which can effectively reduce the required labeled data; combine segmentation tasks with lifetime prediction tasks to achieve feature sharing and mutual promotion; combine weak In the supervision method, high-level semantics such as survival period are used as weak supervision sources, and the Transformer module is used to mine the correlation between features, so as to focus on the tumor area and further improve the segmentation accuracy.
  • the P value and the KM curve were calculated to evaluate the performance of the model. Weakly supervised or semi-supervised have better segmentation results.
  • the obtained segmentation result is compared with the real manually marked Mask, and the final model is judged by calculating the Dice loss.
  • S60 Input the medical image to be segmented into the trained image segmentation model, and output the segmentation result through the image segmentation model.
  • the medical image segmentation method of the embodiment of the present application adopts a semi-supervised segmentation method without relying on too much labeled data; combined with a weak supervision method, high-level semantics such as lifetime are used as weak supervision sources, and the Transformer module is used to mine
  • the correlation between features can focus on the tumor area and further improve the segmentation accuracy.
  • features are shared and mutually promoted.
  • FIG. 3 is a schematic structural diagram of a medical image segmentation system according to an embodiment of the present application.
  • the medical image segmentation system 40 of the embodiment of the present application includes:
  • Data acquisition module 41 used to acquire medical image sample data, the medical image sample data includes multimodal medical images and clinical information of cases corresponding to the multimodal medical images;
  • Model training module 42 used to construct a weak-semi-supervised model, the weak-semi-supervised model includes a segmentation branch for performing segmentation tasks and a survival period prediction branch for performing survival period prediction tasks, and input medical image sample data into segmentation
  • the branch and the lifetime prediction branch fuse the features extracted by the segmentation branch and the lifetime prediction branch and perform iterative training to obtain a trained image segmentation model
  • Image segmentation module 43 for inputting the medical image to be segmented into the trained image segmentation model for image segmentation.
  • FIG. 4 is a schematic diagram of a terminal structure in an embodiment of the present application.
  • the terminal 50 includes a processor 51 and a memory 52 coupled to the processor 51 .
  • the memory 52 stores program instructions for realizing the above medical image segmentation method.
  • the processor 51 is used to execute the program instructions stored in the memory 52 to control the medical image segmentation.
  • the processor 51 may also be referred to as a CPU (Central Processing Unit, central processing unit).
  • the processor 51 may be an integrated circuit chip with signal processing capabilities.
  • the processor 51 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components .
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • FIG. 5 is a schematic structural diagram of a storage medium according to an embodiment of the present application.
  • the storage medium of the embodiment of the present application stores a program file 61 capable of realizing all the above-mentioned methods, wherein the program file 61 can be stored in the above-mentioned storage medium in the form of a software product, and includes several instructions to make a computer device (which can It is a personal computer, a server, or a network device, etc.) or a processor (processor) that executes all or part of the steps of the methods in various embodiments of the present invention.
  • a computer device which can It is a personal computer, a server, or a network device, etc.
  • processor processor
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. , or terminal devices such as computers, servers, mobile phones, and tablets.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

本申请涉及一种医学图像分割方法、系统、终端以及存储介质。所述方法包括：获取医学图像样本数据，医学图像样本数据包括多模态医学图像以及所述多模态医学图像对应病例的临床信息；构建弱-半监督模型，弱-半监督模型包括用于执行分割任务的分割分支和用于执行生存期预测任务的生存期预测分支，将所述医学图像样本数据分别输入分割分支和生存期预测分支，将所述分割分支和生存期预测分支提取的特征进行融合并迭代训练，得到训练好的图像分割模型；将待分割医学图像输入训练好的图像分割模型进行图像分割。本申请采用半监督的分割方式，不用依赖于过多的有标签数据；结合弱监督方式，采用生存期等高级语义作为弱监督源，可以提高图像分割精度。

Description

一种医学图像分割方法、系统、终端以及存储介质 技术领域
本申请属于医学图像处理技术领域，特别涉及一种医学图像分割方法、系统、终端以及存储介质。
背景技术
医学图像分割是各种医学图像应用的基础，在临床辅助诊断、图像引导的外科手术和放射治疗中，医学图像分割技术显示出越来越重要的临床价值。传统的医学图像分割都是基于有经验的医生手动分割，而这种纯手动分割的方法往往费时费力，且受医生的主观影响比较大，即使有经验的医生，在疲惫状态下也可能产生错误的分割。此外，经验不足的医生分割的效果也往往难以衡量。
随着深度学习技术的快速发展，基于深度学习的全自动影像分割得到迅猛的发展，在某些领域甚至已经超越了人类，因此，基于深度学习技术的全自动分割就成为了研究的热点。然而，深度学习往往依赖于海量的高质量有标签数据，而医学影像数据往往比较稀缺，获取高质量的有标签数据通常比较困难。此外，人工标注的成本也极高，受不同标注者的影响较大。
发明内容
本申请提供了一种医学图像分割方法、系统、终端以及存储介质，旨在至少在一定程度上解决现有技术中的上述技术问题之一。
为了解决上述问题，本申请提供了如下技术方案：
一种医学图像分割方法，包括：
获取医学图像样本数据，所述医学图像样本数据包括多模态医学图像以及所述多模态医学图像对应病例的临床信息；
构建弱-半监督模型，所述弱-半监督模型包括用于执行分割任务的分割分支和用于执行生存期预测任务的生存期预测分支，将所述医学图像样本数据分别输入分割分支和生存期预测分支，将所述分割分支和生存期预测分支提取的特征进行融合并迭代训练，得到训练好的图像分割模型；
将待分割医学图像输入训练好的图像分割模型进行图像分割。
本申请实施例采取的技术方案还包括：所述获取医学图像样本数据包括：
所述多模态医学图像为每一位病例的FLAIR、T1、T2和T1c四种模态图像；
所述临床信息包括所述病例的生存期以及生存状态。
本申请实施例采取的技术方案还包括：所述获取医学图像样本数据具体为：
生成所述多模态医学图像样本数据的Mask数据；
对所述多模态医学图像样本数据以及Mask数据进行预处理，生成用于模型训练的医学图像数据集；
根据设定比例对所述医学图像数据集进行分组，得到训练集、验证集和测试集。
本申请实施例采取的技术方案还包括：所述对所述多模态医学图像样本数据以及Mask数据进行预处理具体为：
对所述多模态医学图像以及对应的Mask数据进行剪裁；剪裁方式具体为：获取每幅多模态医学图像的中心点，以所述中心点向外扩展设定大小的区域，剪掉所述区域以外的部分，得到剪裁后的医学图像和Mask数据；
采用min-max算法对所述剪裁后的医学图像进行归一化；
分别将所述归一化后的四个模态的医学图像以及剪裁后的Mask数据进行拼接操作，得到预处理完成后的医学图像数据集。
本申请实施例采取的技术方案还包括：所述根据设定比例对所述医学图像数据集进行分组具体为：
采用10折交叉验证算法，每一轮中取10％的训练集数据作为验证集。
本申请实施例采取的技术方案还包括：所述构建弱-半监督模型为3D U-Net网络，所述3D U-Net网络的训练过程包括：
将所述训练集数据分别输入分割分支和生存期预测分支中进行下采样处理，所述分割分支下采样得到的特征经过flatten操作变为一维的特征，再输入到Transformer模块；
所述Transformer模块采用基于残差连接的思想，将输入特征与输入前的数据相加，再reshape为输入前的shape，同时将Transformer模块得到的特征引出到生存期预测分支；所述生存期预测分支通过Adapter模块对特征的分布进行转换，并通过信息融合模块将分割分支和生存期预测分支输出的特征进行融合，再经过全连接层，得到生存期预测的风险值；所述分割分支在得到reshape后的特征图后，通过上采样将所述特征图还原为初始输入图像的尺寸大小，再经过二值化处理，得到分割任务的输出结果。
本申请实施例采取的技术方案还包括：所述3D U-Net网络的训练模式具体为：
采用Teacher-Student的训练模式对所述3D U-Net网络进行训练；将无标签的医学图像生成的伪标签加入到训练集中，在每轮训练中，如果当前的训练效果比上一轮好，则用Teacher模型更新Student模型，否则继续训练，如果训练次数超过设定次数都未能更新Student模型，则认为模型已经收敛，模型训练结束。
本申请实施例采取的另一技术方案为：一种医学图像分割系统，包括：
数据获取模块：用于获取医学图像样本数据，所述医学图像样本数据包括多模态医学图像以及所述多模态医学图像对应病例的临床信息；
模型训练模块：用于构建弱-半监督模型，所述弱-半监督模型包括用于执行分割任务的分割分支和用于执行生存期预测任务的生存期预测分支，将所述医学图像样本数据分别输入分割分支和生存期预测分支，将所述分割分支和生存期预测分支提取的特征进行融合并迭代训练，得到训练好的图像分割模型；
图像分割模块：用于将待分割医学图像输入训练好的图像分割模型进行图像分割。
本申请实施例采取的又一技术方案为：一种终端，所述终端包括处理器、与所述处理器耦接的存储器，其中，
所述存储器存储有用于实现所述医学图像分割方法的程序指令；
所述处理器用于执行所述存储器存储的所述程序指令以控制医学图像分割。
本申请实施例采取的又一技术方案为：一种存储介质，存储有处理器可运行的程序指令，所述程序指令用于执行所述医学图像分割方法。
相对于现有技术，本申请实施例产生的有益效果在于：本申请实施例的医学图像分割方法、系统、终端以及存储介质采用半监督的分割方式，不用依赖于过多的有标签数据；结合弱监督方式，采用生存期等高级语义作为弱监督源，并采用Transformer模块挖掘特征之间的相关性，从而聚焦肿瘤区域，进一步提高分割精度。通过将分割任务与生存期预测任务相结合，实现特征共享，相互促进。
附图说明
图1是本申请实施例的医学图像分割方法的流程图；
图2为本申请实施例的3D U-Net网络结构示意图；
图3为本申请实施例的医学图像分割系统结构示意图；
图4为本申请实施例的终端结构示意图；
图5为本申请实施例的存储介质的结构示意图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本申请，并不用于限定本申请。
请参阅图1，是本申请实施例的医学图像分割方法的流程图。本申请实施例的医学图像分割方法包括以下步骤：
S10：获取一定数量的多模态医学图像样本数据，并生成多模态医学图像样本数据的Mask(标签)数据；
本步骤中，获取的多模态医学图像样本数据包括每一位病例的FLAIR、T1、T2和T1c四种模态图像以及多模态医学图像对应病例的临床信息，病例的临床信息包括生存期以及生存状态等信息，四种模态数据的大小相同。
S20：对多模态医学图像样本数据以及Mask数据进行预处理，生成用于模型训练的医学图像数据集，并根据设定比例对医学图像数据集进行分组，得到训练集、验证集和测试集；
本步骤中，多模态医学图像样本数据的预处理过程具体包括：
S21：根据每幅医学图像的中心点对原始医学图像以及对应的Mask数据进行剪裁；
其中，由于原始的多模态医学图像太大，且尺寸不统一，存在大量的背景区域，而肿瘤区域一般位于医学图像的中间区域，因此，需要对多模态医学图像进行中心区域的裁剪，使得所有多模态医学图像在裁剪后能够一一对应。裁剪方式具体为：找出每幅医学图像的中心点，并以此中心点向外扩展设定大小的区域，剪掉该区域以外的部分，得到剪裁后的医学图像。本申请实施例中，设定剪裁区域的大小为96*128*128，具体可根据实际操作进行设定。
S22：采用min-max算法对剪裁后的医学图像进行归一化，将所有医学图像的像素值压缩到0-1之间；
其中，由于FLAIR、T1、T2、T1c四种模态数据的成像方式不同，导致图像的对比度存在差异，因此本申请实施例采用min-max算法将不同模态的图像像素值归一化到0-1之间，而剪裁后的Mask数据无需进行归一化操作。
S23：分别将归一化后的四个模态的医学图像以及剪裁后的Mask数据进行拼接操作，得到预处理完成后的医学图像数据集；
其中，拼接后的医学图像数据大小为96*128*128*4。
本申请实施例后，在对医学图像数据进行预处理完之后，还需要根据设定比例对数据集进行分组。优选地，本实施例中测试集的划分比例为25％，其他75％作为训练集和验证集，并采用10折交叉验证算法，即每一轮中取10％的训练集数据作为验证集，可以在评估模型训练效果的同时使用尽可能多的数据进行模型训练。
S30：构建弱-半监督模型，并将训练集输入弱-半监督模型进行迭代训练，得到训练好的图像分割模型；
本步骤中，本申请实施例采用pytorch框架进行弱-半监督模型的搭建，弱-半监督模型为由2D U-Net网络改造而成的3D U-Net网络，即将2D U-Net网 络中的2D卷积和2D池化等所有2D操作全部换为3D操作。3D U-Net网络结构如图2所示。3D U-Net网络包括用于执行分割任务的分割分支和用于执行生存期预测任务的生存期预测分支两个分支结构，还包括残差模块、Transformer模块、Adapter模块、信息融合模块以及Survival Predict模块，具体的：
将训练集分别输入分割分支和生存期预测分支，分割分支的输出为从医学图像数据中提取到的特征；生存期预测分支采用全连接神经网络实现，输出值为每一位病例所对应的风险值。由于不同的分割任务所提取的特征分布存在差异，如果直接融合可能会存在对消的情况，本申请实施例通过Adapter模块将分割分支提取的特征进行数据分布转换，从而将分割分支得到的特征融合到生存期预测分支中，使生存期预测任务也能够利用分割任务中学到的特征。Adapter模块的数据分布转换过程具体为：首先利用生存期预测分支的特征分布计算出生存期预测任务中特征的mean和std，然后在分割分支上进行数据变换，用分割任务中的特征减去mean再除以std，使分割任务具有与生存期预测任务相同的mean和std。通过Adapter模块将分割分支获得的特征转化为与生存期预测分支获得的特征相同的分布，可以有效避免特征之间的信息丢失，同时提供其他任务无法提供的信息。其中，分割分支的损失函数为Dice系数以及BCE(Binary Cross Entropy)Loss，生存期预测分支的损失函数为NegativeLogLikelihood(负对数似然)。
Transformer模块的输入和输出维度分别为201，内部网络层数为4层，n_head＝1。将分割分支提取的特征经过flatten操作变为一维的特征，将每个channel(通道)的一维特征进行拼接，再输入到Transformer模块中。Transformer模块采用基于残差连接的思想，采用残差模块将输入特征与输入前的数据相加， 再reshape为输入前的shape，从而进一步挖掘不同特征之间的内部联系。其中，每个残差模块分别由2个卷积层组成，即进行2次卷积操作再残差连接，在每次卷积之后，采用LeakyRelu进行非线性映射，并采用GroupNorm进行归一化操作；残差模块的残差公式为：x l+1＝x l+F(x l)。
信息融合模块用于采用卷积的方式对分割分支和生存期预测分支输出的特征进行合并，对合并后的特征进行再卷积操作，卷积后的输出结果即为融合后的特征，从而将生存期预测任务中学到的特征迁移到分割任务中。
为了能够充分学习到生存期信息和Mask信息，3D U-Net网络采用双下采样的方式，输入数据分别输入两个编码器(即分割分支和生存期预测分支)，再将两个编码器的输出结果进行融合，最后将融合后的图像引入解码器中进行上采样。具体的，本申请实施例的3D U-Net网络训练过程包括：将训练集数据分别输入分割分支和生存期预测分支中进行下采样处理，总共进行3次下采样，并暂时保留每次下采样的结果；其中分割分支下采样得到的特征先经过flatten操作变为一维的特征，再输入到Transformer模块；Transformer模块采用基于残差连接的思想，将输入特征与输入前的数据相加，再reshape为输入前的shape，同时将Transformer模块得到的特征引出到生存期预测分支；生存期预测分支通过Adapter模块对特征的分布进行转换，并通过信息融合模块将分割分支和生存期预测分支输出的特征进行融合，再经过全连接层，即可得到生存期预测的风险值，对风险值进行二值化处理，再将二值化结果作为一个影响因子，计算其对生存期的影响。分割分支在得到reshape后的特征图后，经过3次上采样，将特征图还原为初始输入图像的尺寸大小，再经过二值化处理，即得到分割任务的输出结果。
本申请实施例采用Teacher-Student的训练模式对搭建好的3D U-Net网络 进行训练。Teacher-Student训练模式具体为：将无label的图像数据生成的伪标签加入到训练集中，在每轮训练中，如果当前的训练效果比上一轮好，则用Teacher模型更新Student模型，否则继续训练，如果训练次数超过20轮都未能更新Student模型，则认为模型已经收敛，可以结束训练。并采用C-Index评价指标和Dice系数来评价当前模型的训练效果。
基于上述，本申请实施例的3D U-Net网络采用半监督的分割方式，能够有效减少所需的有标签数据；将分割任务与生存期预测任务相结合，实现特征共享，相互促进；结合弱监督方式，采用生存期等高级语义作为弱监督源，并采用Transformer模块挖掘特征之间的相关性，从而聚焦肿瘤区域，进一步提高分割精度。
S40：将验证数据集输入训练好的图像分割模型进行模型评估；
本步骤中，在构建并训练完模型后，为了进一步验证图像分割模型的分割效果，分别计算了P值和KM曲线对模型性能进行评估，结果表明，本申请实施例的图像分割模型比单独的弱监督或半监督具有更好的分割效果。
S50：将测试数据集输入图像分割模型进行模型测试；
本步骤中，通过将测试集数据输入到训练好的图像分割模型中，将得到的分割结果与真实手工标注的Mask进行比较，通过计算Dice loss来评判最终的模型好坏。
S60：将待分割医学图像输入训练好的图像分割模型，通过图像分割模型输出分割结果。
基于上述，本申请实施例的医学图像分割方法采用半监督的分割方式，不用依赖于过多的有标签数据；结合弱监督方式，采用生存期等高级语义作为弱监督源，并采用Transformer模块挖掘特征之间的相关性，从而聚焦肿瘤区域， 进一步提高分割精度。通过将分割任务与生存期预测任务相结合，实现特征共享，相互促进。
请参阅图3，为本申请实施例的医学图像分割系统结构示意图。本申请实施例的医学图像分割系统40包括：
数据获取模块41：用于获取医学图像样本数据，医学图像样本数据包括多模态医学图像以及多模态医学图像对应病例的临床信息；
模型训练模块42：用于构建弱-半监督模型，弱-半监督模型包括用于执行分割任务的分割分支和用于执行生存期预测任务的生存期预测分支，将医学图像样本数据分别输入分割分支和生存期预测分支，将分割分支和生存期预测分支提取的特征进行融合并迭代训练，得到训练好的图像分割模型；
图像分割模块43：用于将待分割医学图像输入训练好的图像分割模型进行图像分割。
请参阅图4，为本申请实施例的终端结构示意图。该终端50包括处理器51、与处理器51耦接的存储器52。
存储器52存储有用于实现上述医学图像分割方法的程序指令。
处理器51用于执行存储器52存储的程序指令以控制医学图像分割。
其中，处理器51还可以称为CPU(Central Processing Unit，中央处理单元)。处理器51可能是一种集成电路芯片，具有信号的处理能力。处理器51还可以是通用处理器、数字信号处理器(DSP)、专用集成电路(ASIC)、现成可编程门阵列(FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
请参阅图5，为本申请实施例的存储介质的结构示意图。本申请实施例的 存储介质存储有能够实现上述所有方法的程序文件61，其中，该程序文件61可以以软件产品的形式存储在上述存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器(processor)执行本发明各个实施方式方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质，或者是计算机、服务器、手机、平板等终端设备。
对所公开的实施例的上述说明，使本领域专业技术人员能够实现或使用本发明。对这些实施例的多种修改对本领域的专业技术人员来说将是显而易见的，本发明中所定义的一般原理可以在不脱离本发明的精神或范围的情况下，在其它实施例中实现。因此，本发明将不会被限制于本发明所示的这些实施例，而是要符合与本发明所公开的原理和新颖特点相一致的最宽的范围。

Claims (10)

  1. 一种医学图像分割方法，其特征在于，包括：
    获取医学图像样本数据，所述医学图像样本数据包括多模态医学图像以及所述多模态医学图像对应病例的临床信息；
    构建弱-半监督模型，所述弱-半监督模型包括用于执行分割任务的分割分支和用于执行生存期预测任务的生存期预测分支，将所述医学图像样本数据分别输入分割分支和生存期预测分支，将所述分割分支和生存期预测分支提取的特征进行融合并迭代训练，得到训练好的图像分割模型；
    将待分割医学图像输入训练好的图像分割模型进行图像分割。
  2. 根据权利要求1所述的医学图像分割方法，其特征在于，所述获取医学图像样本数据包括：
    所述多模态医学图像为每一位病例的FLAIR、T1、T2和T1c四种模态图像；
    所述临床信息包括所述病例的生存期以及生存状态。
  3. 根据权利要求2所述的医学图像分割方法，其特征在于，所述获取医学图像样本数据具体为：
    生成所述多模态医学图像样本数据的Mask数据；
    对所述多模态医学图像样本数据以及Mask数据进行预处理，生成用于模型训练的医学图像数据集；
    根据设定比例对所述医学图像数据集进行分组，得到训练集、验证集和测试集。
  4. 根据权利要求3所述的医学图像分割方法，其特征在于，所述对所述多模态医学图像样本数据以及Mask数据进行预处理具体为：
    对所述多模态医学图像以及对应的Mask数据进行剪裁；剪裁方式具体为：获取每幅多模态医学图像的中心点，以所述中心点向外扩展设定大小的区域，剪掉所述区域以外的部分，得到剪裁后的医学图像和Mask数据；
    采用min-max算法对所述剪裁后的医学图像进行归一化；
    分别将所述归一化后的四个模态的医学图像以及剪裁后的Mask数据进行拼接操作，得到预处理完成后的医学图像数据集。
  5. 根据权利要求3所述的医学图像分割方法，其特征在于，所述根据设定比例对所述医学图像数据集进行分组具体为：
    采用10折交叉验证算法，每一轮中取10％的训练集数据作为验证集。
  6. 根据权利要求1至5任一项所述的医学图像分割方法，其特征在于，所述构建弱-半监督模型为3D U-Net网络，所述3D U-Net网络的训练过程包括：
    将所述训练集数据分别输入分割分支和生存期预测分支中进行下采样处理，所述分割分支下采样得到的特征经过flatten操作变为一维的特征，再输入到Transformer模块；
    所述Transformer模块采用基于残差连接的思想，将输入特征与输入前的数据相加，再reshape为输入前的shape，同时将Transformer模块得到的特征引出到生存期预测分支；所述生存期预测分支通过Adapter模块对特征的分布进行转换，并通过信息融合模块将分割分支和生存期预测分支输出的特征进行融合，再经过全连接层，得到生存期预测的风险值；所述分割分支在得到reshape后的特征图后，通过上采样将所述特征图还原为初始输入图像的尺寸大小，再经过二值化处理，得到分割任务的输出结果。
  7. 根据权利要求6所述的医学图像分割方法，其特征在于，所述3D U-Net网络的训练模式具体为：
    采用Teacher-Student的训练模式对所述3D U-Net网络进行训练；将无标签的医学图像生成的伪标签加入到训练集中，在每轮训练中，如果当前的训练效果比上一轮好，则用Teacher模型更新Student模型，否则继续训练，如果训练次数超过设定次数都未能更新Student模型，则认为模型已经收敛，模型训练结束。
  8. 一种医学图像分割系统，其特征在于，包括：
    数据获取模块：用于获取医学图像样本数据，所述医学图像样本数据包括多模态医学图像以及所述多模态医学图像对应病例的临床信息；
    模型训练模块：用于构建弱-半监督模型，所述弱-半监督模型包括用于执行分割任务的分割分支和用于执行生存期预测任务的生存期预测分支，将所述医学图像样本数据分别输入分割分支和生存期预测分支，将所述分割分支和生存期预测分支提取的特征进行融合并迭代训练，得到训练好的图像分割模型；
    图像分割模块：用于将待分割医学图像输入训练好的图像分割模型进行图像分割。
  9. 一种终端，其特征在于，所述终端包括处理器、与所述处理器耦接的存储器，其中，
    所述存储器存储有用于实现权利要求1-7任一项所述的医学图像分割方法的程序指令；
    所述处理器用于执行所述存储器存储的所述程序指令以控制医学图像分割。
  10. 一种存储介质，其特征在于，存储有处理器可运行的程序指令，所述程序指令用于执行权利要求1至7任一项所述医学图像分割方法。
PCT/CN2021/138686 2021-12-16 2021-12-16 一种医学图像分割方法、系统、终端以及存储介质 WO2023108526A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/138686 WO2023108526A1 (zh) 2021-12-16 2021-12-16 一种医学图像分割方法、系统、终端以及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/138686 WO2023108526A1 (zh) 2021-12-16 2021-12-16 一种医学图像分割方法、系统、终端以及存储介质

Publications (1)

Publication Number Publication Date
WO2023108526A1 true WO2023108526A1 (zh) 2023-06-22

Family

ID=86775050

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/138686 WO2023108526A1 (zh) 2021-12-16 2021-12-16 一种医学图像分割方法、系统、终端以及存储介质

Country Status (1)

Country Link
WO (1) WO2023108526A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117333777A (zh) * 2023-12-01 2024-01-02 山东元明晴技术有限公司 一种坝体异常识别方法、装置及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200226368A1 (en) * 2019-01-15 2020-07-16 International Business Machines Corporation Weakly and fully labeled mammogram classification and localization with a dual branch deep neural network
CN112489050A (zh) * 2020-12-13 2021-03-12 成都易书桥科技有限公司 一种基于特征迁移的半监督实例分割算法
CN113257413A (zh) * 2021-06-22 2021-08-13 安翰科技(武汉)股份有限公司 基于深度学习的癌症预后生存预测方法、设备及存储介质
CN113706545A (zh) * 2021-08-23 2021-11-26 浙江工业大学 一种基于双分支神经判别降维的半监督图像分割方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200226368A1 (en) * 2019-01-15 2020-07-16 International Business Machines Corporation Weakly and fully labeled mammogram classification and localization with a dual branch deep neural network
CN112489050A (zh) * 2020-12-13 2021-03-12 成都易书桥科技有限公司 一种基于特征迁移的半监督实例分割算法
CN113257413A (zh) * 2021-06-22 2021-08-13 安翰科技(武汉)股份有限公司 基于深度学习的癌症预后生存预测方法、设备及存储介质
CN113706545A (zh) * 2021-08-23 2021-11-26 浙江工业大学 一种基于双分支神经判别降维的半监督图像分割方法

Non-Patent Citations (9)

* Cited by examiner, † Cited by third party
Title
"Master's Thesis", 1 June 2020, NORTHWEST UNIVERSITY, CN, article LU, DI: "The Multi-parametric Mri-based High Risk Subregion Identification and Survival Stratification for Glioblastoma Patients", pages: 1 - 56, XP009546485, DOI: 10.27405/d.cnki.gxbdu.2020.001897 *
FU, JIE ET AL.: "An automatic deep learning-based workflow for glioblastoma survival prediction using pre-operative multimodal MR images", ARXIV:2001.11155V1, 30 January 2020 (2020-01-30), XP081940607 *
GEORGE PAPANDREOU, LIANG-CHIEH CHEN, KEVIN MURPHY, ALAN L. YUILLE: "Weakly- and semi-supervised learning of a deep convolutional network for semantic image segmentation", ARXIV:1502.02734V3, 5 October 2015 (2015-10-05), XP055277518, Retrieved from the Internet <URL:https://arxiv.org/abs/1502.02734v3> [retrieved on 20160602], DOI: 10.1109/ICCV.2015.203 *
HAN, WOO-SUP ET AL.: "Neuromorphic Neural Network for Multimodal Brain Image Segmentation and Overall Survival Analysis", BRAINLESION: GLIOMA, MULTIPLE SCLEROSIS, STROKE AND TRAUMATIC BRAIN INJURIES, vol. 11384, 26 January 2019 (2019-01-26) *
MARIIA DOBKO; DANYLO-IVAN KOLINKO; OSTAP VINIAVSKYI; YURII YELISIEIEV: "Combining CNNs With Transformer for Multimodal 3D MRI Brain Tumor Segmentation With Self-Supervised Pretraining", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 15 October 2021 (2021-10-15), 201 Olin Library Cornell University Ithaca, NY 14853, XP091078901 *
QIZHU LI; ANURAG ARNAB; PHILIP H.S TORR: "Weakly- and Semi-Supervised Panoptic Segmentation", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 10 August 2018 (2018-08-10), 201 Olin Library Cornell University Ithaca, NY 14853 , XP080903329 *
SUN LI, ZHANG SONGTAO, CHEN HANG, LUO LIN: "Brain Tumor Segmentation and Survival Prediction Using Multimodal MRI Scans With Deep Learning", FRONTIERS IN NEUROSCIENCE, vol. 13, 16 August 2019 (2019-08-16), pages 810, XP093072730, DOI: 10.3389/fnins.2019.00810 *
vol. 11, 21 September 2021, SPRINGER INTERNATIONAL PUBLISHING, article WANG WENXUAN; CHEN CHEN; DING MENG; YU HONG; ZHA SEN; LI JIANGYUN: "TransBTS: Multimodal Brain Tumor Segmentation Using Transformer", pages: 109 - 119, XP047619149, DOI: 10.1007/978-3-030-87193-2_11 *
YUTONG XIE; JIANPENG ZHANG; YONG XIA; CHUNHUA SHEN: "Semi- and Weakly Supervised Directional Bootstrapping Model for Automated Skin Lesion Segmentation", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 8 March 2019 (2019-03-08), 201 Olin Library Cornell University Ithaca, NY 14853 , XP081131037 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117333777A (zh) * 2023-12-01 2024-01-02 山东元明晴技术有限公司 一种坝体异常识别方法、装置及存储介质
CN117333777B (zh) * 2023-12-01 2024-02-13 山东元明晴技术有限公司 一种坝体异常识别方法、装置及存储介质

Similar Documents

Publication Publication Date Title
CN112241766B (zh) 基于样本生成和迁移学习的肝脏ct图像多病变分类方法
WO2020019738A1 (zh) 磁共振血管壁成像的斑块处理方法、装置和计算设备
CN111291825B (zh) 病灶分类模型训练方法、装置、计算机设备和存储介质
Zhang et al. The extraction method of tooth preparation margin line based on S‐Octree CNN
CN114418946A (zh) 一种医学图像分割方法、系统、终端以及存储介质
CN111881991B (zh) 一种识别欺诈的方法、装置及电子设备
CN111951281B (zh) 图像分割方法、装置、设备及存储介质
CN110648331B (zh) 用于医学图像分割的检测方法、医学图像分割方法及装置
US20220253977A1 (en) Method and device of super-resolution reconstruction, computer device and storage medium
US20200193604A1 (en) Encoder Regularization of a Segmentation Model
WO2023108526A1 (zh) 一种医学图像分割方法、系统、终端以及存储介质
WO2021139351A1 (zh) 图像分割方法、装置、介质及电子设备
Hou et al. Af-net: A medical image segmentation network based on attention mechanism and feature fusion
Sarica et al. A dense residual U-net for multiple sclerosis lesions segmentation from multi-sequence 3D MR images
CN115631397A (zh) 一种基于双模态图像的目标检测方法及装置
Ma et al. Unsupervised deformable image registration network for 3D medical images
CN112101438B (zh) 一种左右眼分类方法、装置、服务器和存储介质
Wang et al. Densely connected swin-unet for multiscale information aggregation in medical image segmentation
WO2024087858A1 (zh) 图像处理模型的训练方法、装置、电子设备、计算机程序产品及计算机存储介质
CN113724185A (zh) 用于图像分类的模型处理方法、装置及存储介质
CN117333497A (zh) 一种基于掩码监督策略的高效建模的三维医学影像分割方法
CN116188435B (zh) 一种基于模糊逻辑的医学图像深度分割方法
WO2020215682A1 (zh) 眼底图像样本扩展方法、装置、电子设备及计算机非易失性可读存储介质
CN115908947A (zh) 一种基于反绎学习的多模态牙周图像识别方法和系统
Wang et al. GazeGNN: A Gaze-Guided Graph Neural Network for Disease Classification

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21967661

Country of ref document: EP

Kind code of ref document: A1