WO2022100034A1 - 基于深度学习的甲状腺细胞病理切片恶性区域的检测方法 - Google Patents

基于深度学习的甲状腺细胞病理切片恶性区域的检测方法 Download PDF

Info

Publication number
WO2022100034A1
WO2022100034A1 PCT/CN2021/091988 CN2021091988W WO2022100034A1 WO 2022100034 A1 WO2022100034 A1 WO 2022100034A1 CN 2021091988 W CN2021091988 W CN 2021091988W WO 2022100034 A1 WO2022100034 A1 WO 2022100034A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
malignant
pathological
slices
image
Prior art date
Application number
PCT/CN2021/091988
Other languages
English (en)
French (fr)
Inventor
魏军
沈烁
钱东东
卢旭玲
Original Assignee
广州柏视医疗科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州柏视医疗科技有限公司 filed Critical 广州柏视医疗科技有限公司
Publication of WO2022100034A1 publication Critical patent/WO2022100034A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G06T2207/10061—Microscopic image from scanning electron microscope
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20024—Filtering details
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20132—Image cropping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion

Definitions

  • the present invention relates to the field of medical image processing, in particular to a method for detecting malignant regions of pathological slices of thyroid cells based on deep learning.
  • Thyroid cell pathological screening is in a stage of popularization.
  • the pathological screening method of cell puncture has the advantages of less trauma, low risk and rapid diagnosis, and has been popularized in many tertiary hospitals.
  • due to the late start of cytopathological screening there are relatively few pathologists, and experienced doctors are even scarcer, resulting in a backlog of cases that need to be diagnosed.
  • diagnosis of cytopathological slices there is often a problem that the target cells cannot be obtained or the target cells are too few, which makes the diagnosis impossible.
  • this paper proposes a deep learning method to pre-extract high-risk malignant cell areas on pathological slices to assist doctors in diagnosis and decision-making.
  • the workload of pathologists is greatly reduced, and the It reduces the misdiagnosis rate and improves the work efficiency of the pathologist, so that the pathologist can focus on more complex cases.
  • the purpose of the present invention is to provide a method for detecting malignant regions of thyroid cell pathological slices based on deep learning, which adopts a multi-stage training method from coarse to fine, first removes obviously invalid cuts, and improves effective cuts into good ones. Malignant classification further improves detection accuracy by removing false positives.
  • the present invention provides a deep learning-based detection method for malignant regions of pathological slices of thyroid cells, including the following steps: slicing step, preprocessing step, image sampling step, dicing preliminary screening step, benign and malignant classification step. , post-processing steps for suspicious areas, and high-risk areas display steps for pathological sections; sectioning step: pathological sectioning of thyroid cells; pre-processing step: digitizing the image of the pathological section on the microscope and then smearing it with different dyes.
  • the slicing step includes converting the color pathological slice image from RGB space to gray space and smoothing the image by using Gaussian filtering.
  • an overlapping sampling method based on an overlapping area of 50% is used to cut out the complete pathological slice into a suitable size, and the resolution of the slice is 512 ⁇ 512 pixels.
  • the ineffective dicing in the dicing preliminary screening step is an area that does not meet the required resolution or does not contain enough thyroid cells.
  • the initial weights obtained by the light-weight resnet18 classification network trained on imagenet are used as the initial weights to perform preliminary screening of dicing, and the predicted probability of resnet18 is lower than the threshold. Blocks will be screened out.
  • the step of classifying benign and malignant includes adopting a weakly supervised learning method, making full use of unlabeled data to participate in the training, and the specific training process includes a first-stage training and a second-stage training; the first-stage training Including: on 140 slices diagnosed with papillary thyroid carcinoma, 1203 images of malignant PTC slices were marked by senior thyroid cytopathologists as positive classes, and 1203 slices were randomly cropped on 120 normal slices, as Negative class; allocate the training set, test set and validation set according to the ratio of 4:1:1, and participate in the training of model 1.
  • resnet18 is selected to build model 1;
  • the model with the best performance; the model with the best performance is used to make predictions on the unlabeled cytopathological slices of the thyroid, and the labels of the slices whose predicted probability is greater than a given threshold are set as positive labels;
  • the block label smaller than the given threshold is set as the negative example label; and the labeled data of the positive example label and the negative example label are added to the training set;
  • the second stage training includes: selecting the resnet34 model as the second stage training model 2, Iteratively train model 2 and update the weights to obtain the optimal model of model 2 on the test set, and use the same method as in the first stage to set pseudo-labels for unlabeled dicing to expand the training set; among them,
  • the first-stage training and the second-stage training are used to build model 3, model 4, and model 5 with resnet50, resnet101, and resnet152 respectively to gradually expand the data to further improve the performance of the model; among them,
  • the overlapping area is eliminated by using the maximum suppression technique, and among the intersecting blocks, the block with the highest malignant probability is selected as the output.
  • the model input for false positive removal in the post-processing step of the suspicious area includes: the probability of segmentation predicted by the model 5 of the image module; the feature extracted by the penultimate layer of the model 5; based on the local binary value The image features of the pathological cut of thyroid cells extracted by the pattern; and the coordinate information of the center point of the cut in the image.
  • the high-risk regions of the pathological sections show the probability of malignancy prediction for each section in the step normalized to 0-255.
  • the present invention provides a deep learning-based detection system for the malignant region of thyroid cell pathological slices, which can be used for the aforementioned detection method.
  • the detection system includes an equipment part, a hardware part and a software part; the equipment part mainly includes: Microscopes that provide image sources, etc.; the hardware part mainly includes GPU and CPU; the software part mainly includes Keras, Pytorch, tensorflow, Caffe or Paddle, etc.
  • the deep learning-based method for detecting malignant regions of pathological slices of thyroid cells of the present invention has the following beneficial effects: it adopts a multi-stage training method from coarse to fine, first removes obviously invalid cuts, It can effectively cut into benign and malignant classification, and further improve the detection accuracy by removing false positives. Pre-extract high-risk malignant cell areas on pathological slices to assist doctors in diagnosis and decision-making. Through this process, the workload of pathologists is greatly reduced, the rate of misdiagnosis is also reduced, and the work efficiency of pathologists is improved. , allowing pathologists to focus on more complex cases.
  • FIG. 1 is a flowchart of a detection method according to an embodiment of the present invention.
  • a method for detecting malignant regions of thyroid cell pathological slices based on deep learning mainly includes the following slicing steps, preprocessing steps, image sampling steps, dicing preliminary screening steps, benign Malignant classification steps, post-processing steps for suspicious areas, and high-risk areas display steps for pathological sections, etc.
  • preprocessing is to enhance the features of the image.
  • the images of the pathological sections on the microscope are digitally processed and then smeared with different dyes to obtain colored pathological sections, which are convenient for doctors to observe and diagnose.
  • different staining methods and the operator's technique will bring about differences in the staining of different sections, which will interfere with the use of machine learning and deep learning methods for diagnostic analysis.
  • the image preprocessing stage the image is first converted from RGB space to grayscale space; at the same time, due to the presence of stains on the glass slide, noise is introduced into the cytopathological section, so Gaussian filtering is used at the same time.
  • the image is smoothed to reduce the effect of noise.
  • this embodiment adopts the overlapping sampling method based on the overlapping area of 0.5 (50% overlap) (using the overlapping sampling method can ensure that all possible positions are collected, and at the same time ensure that at least one suspicious area can be completely Rendering), crop out blocks of suitable size (in this embodiment, a block with a resolution size of 512 ⁇ 512 is selected), which is used as the input of the deep neural network model.
  • This example trains a lightweight resnet18 classification network (specific model training method: manually label 1,000 blocks with insufficient resolution and insufficient thyroid cells as negative classes, the field of view is clear and the thyroid cells in the field of view satisfy the next
  • the block of the workflow is used as a positive example, and the initial weight obtained by resnet18 in imagenet training is used as the initial weight for migration training, and the model with the best effect on the validation set is selected as the initial screening model for dicing. Cuts with a predicted probability lower than the threshold (set as 0.2 in this embodiment) are eliminated), some invalid cuts are screened out, and the remaining cuts are sent to the benign and malignant classification module for further detection.
  • the benign and malignant classification module is the core part of this embodiment. This module can screen out the thyroid malignant cell blocks that doctors focus on, and use the high-risk area display module of pathological slices to draw a heat map of malignant cell clusters, which is convenient for pathologists to carry out. diagnosis.
  • this embodiment adopts a weakly supervised learning method, making full use of the unlabeled data to participate in the training.
  • this embodiment selects the resnet34 model as the second-stage training model Model (Model) 2, performs iterative training on the Model (Model) 2, updates the weights, and obtains the Model (Model) 2 in the test. the best model on the set.
  • the training set is augmented by setting pseudo-labels for unlabeled slices using the same method as in the first stage.
  • step 3 Similar to the method in step 1 and step 2, respectively use resnet50, resnet101, and resnet152 to build model (Model) 3, model (Model) 4, and model (Model) 5 to gradually expand the data to further improve the performance of the model.
  • Model 1, Model 2, Model 3, Model 4, and Model 5 on the validation set are 0.78, 0.80, 0.83, 0.87, and 0.95, respectively. . It shows that the weakly supervised learning method can effectively improve the classification performance of benign and malignant cells.
  • model (Model) 1 to model (Model) 5 a technique of maximum suppression (NMS) is used to remove the overlapping area: among the intersecting blocks, the block with the highest malignant probability is selected as the output .
  • NMS maximum suppression
  • this embodiment proposes a scheme for removing false positives, that is, constructing a random forest-based machine learning method to remove false positives from the prediction results of the model (Model) 5 of benign and malignant classification.
  • the input of the model comes from Four parts: the probability of segmentation predicted by the model (Model) 5 of the image module; the features extracted by the penultimate layer of the model (Model) 5; the thyroid cell pathology extracted based on the Local Binary Pattern (LBP) The image features of the slice; and the coordinate information of the center point of the slice in the image (normalized to [0, 1]). Train a random forest model.
  • the model removes blocks whose probability is lower than a certain threshold (the threshold in this embodiment is set to 0.2), which further eases the work of the pathologist.
  • the threshold can be adjusted dynamically. If the pathologist wants to know more about the cutting of malignant cell pathological slices for further evaluation, the threshold can be manually adjusted to a lower value.
  • the probability of malignancy prediction for each cut is normalized to [0, 255], mapped to the original image, the grayscale image is converted into a color image, and a heat map is generated, which is presented to the cytopathologist, especially to the pathologist.
  • a visual suggestion for inexperienced physicians: which areas to focus on. The method of this embodiment can transfer the experience learned from experienced pathologists to young doctors and help young doctors grow.
  • Module 4 in this embodiment can work in series with 5 (post-processing step of suspicious area), or can work independently.
  • the doctor needs to display the results faster, based on the threshold set by the doctor, the suspicious malignant regions can be directly sorted and displayed according to the probability from high to low. If the doctor wants a more accurate result, the result selected by the doctor based on the threshold value (0.5 is selected in this embodiment) and the predicted probability based on module 5 can be displayed and output in descending order.
  • a deep learning-based detection system for malignant regions of pathological slices of thyroid cells can be used for the aforementioned detection method.
  • the detection system includes an equipment part, a hardware part and a software part; the equipment part mainly includes a device that provides image sources. Microscope, etc.; the hardware part mainly includes GPU and CPU, etc.; the software part mainly includes Keras, Pytorch, tensorflow, Caffe or Paddle, etc.
  • the development and use of this embodiment is Keras: 2.1.6 version, but the present invention is not limited to this version.
  • the extensible deep learning framework can also be applied to the development system of the present invention but is not limited to Pytorch, tensorflow, Caffe or Paddle.
  • the hardware developed for the GPU in this embodiment uses a GeForce GTX1080ti, but the present invention is not limited to this model of GPU. It can be any discrete graphics card, including but not limited to, GeForce RTX 1060, GeForce RTX 2080ti, GeForce P6000. (For more graphics cards, check the NVIDIA official website).
  • the hardware developed by the CPU of this embodiment uses Intel(R) Xeon(R) CPU E5-2640 v4@2.40GHz. For more server CPUs see Intel and AMD Server CPU Types.
  • the pathological pictures in this example use HE-stained images, and other staining schemes include but are not limited to CK7, P40, CD56, silver hexamine, TTF1, PAS, and the like.
  • the deep learning-based method for detecting malignant regions of pathological slices of thyroid cells of the present invention has the following beneficial effects: it adopts a multi-stage training method from coarse to fine, and firstly removes obviously invalid cuts, so that effective Divide into benign and malignant classification, and further improve the detection accuracy by removing false positives. Pre-extract high-risk malignant cell areas on pathological slices to assist doctors in diagnosis and decision-making. Through this process, the workload of pathologists is greatly reduced, the rate of misdiagnosis is also reduced, and the work efficiency of pathologists is improved. This allows pathologists to focus on more complex cases.

Landscapes

  • Engineering & Computer Science (AREA)
  • Quality & Reliability (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Investigating Or Analysing Biological Materials (AREA)

Abstract

一种基于深度学习的甲状腺细胞病理切片恶性区域的检测方法，主要包括下列步骤：对甲状腺细胞进行病理切片；将病理切片在显微镜上的图像经过数字化处理后使用不同的染色剂进行涂抹以得到彩色的病理切片；将完整的病理切片裁剪出合适大小的切块作为深度神经网络模型的输入；筛除掉病理切片部分无效的切块；采用弱监督的学习方法对切块初步筛查后的病理切片进行良恶性分类；利用去假阳的方案构建一个基于随机森林的机器学习方法对良恶性分类的预测结果进行假阳的去除；借此，能进一步提高检测的准确率。以及病理切片的高危区域显示步骤：将每个切块的恶性预测的概率归一化并映射到原图的中，生成热力图；给病理医生更直观的可视化显示。

Description

基于深度学习的甲状腺细胞病理切片恶性区域的检测方法 技术领域
本发明是关于医学图像处理领域，特别是关于一种基于深度学习的甲状腺细胞病理切片恶性区域的检测方法。
背景技术
病理切片作为病理诊断的金标准，在临床和科研中都有十分重要的作用。甲状腺细胞病理筛查处于一个正在普及的阶段，细胞穿刺的病理筛查方法具有创伤小，风险低，快速诊断的优点，已经在许多三甲医院进行了普及。但是目前由于细胞病理筛查起步较晚，病理医生相对比较少，有经验的医生更是稀缺，从而造成了需要诊断的病例的积压。同时，细胞病理切片诊断中，经常存在的问题是获取不到目标细胞或者目标细胞过少，导致无法诊断。
基于此，本文提出了一种深度学习方法，在病理切片上预提取高风险的恶性细胞区域，协助医生进行诊断，作出决策，通过此流程，在大幅减少病理医生的工作量的同时，也降低了误诊率，提高了病理医生的工作效率，从而使病理医生可以将精力集中到更加复杂的病例上。
公开于该背景技术部分的信息仅仅旨在增加对本发明的总体背景的理解，而不应当被视为承认或以任何形式暗示该信息构成已为本领域一般技术人员所公知的现有技术。
发明内容
本发明的目的在于提供一种基于深度学习的甲状腺细胞病理切片恶性区域的检测方法，其采用由粗到细多阶段训练的方法，先剔除掉明显无效的切块，对有效的切块进良恶性分类，通过去除假阳进一步提高检测的准确率。
为实现上述目的，本发明提供了一种基于深度学习的甲状腺细胞病理切片恶性区域的检测方法，包括下列步骤切片步骤、预处理步骤、图像采样步骤、切块初步筛查步骤、良恶性分类步骤、可疑区域的后处理步骤以及病理切片的高危区域显示步骤；切片步骤：对甲状腺细胞进行病理切片；预处理步骤：将病理切片在显微镜上的图像经过数字化处理后使用不同的染色剂进行涂抹以得到彩色的病理切片；图像采样步骤：将完整的病理切片裁剪出合适大小的切块作为深度神经网络模型的输入；切块初步筛查步骤：筛除掉病理切片部分无效的切块；良恶性分类步骤：采用弱监督的学习方法对切块初步筛查后的病理切片进行良恶性分类；可疑区域的后处理步骤：利用去假阳的方案构建一个基于随机森林的机器学习方法对良恶性分类的预测结果进行假阳的去除；以及病理切片的高危区域显示步骤：将每个切块的恶性预测的概率归一化并映射到原图的中，将灰度图转化成彩色图像且生成热力图。
在一优选的实施方式中，切片步骤包括将彩色的病理切片图像由RGB空间转到灰度空间以及采用高斯滤波的方法对图像进行平滑处理。
在一优选的实施方式中，图像采样步骤中采用基于重叠面积为50％的重叠采样方法将完整的病理切片裁剪出合适大小的切块，切块的分辨率为512x512像素。
在一优选的实施方式中，切块初步筛查步骤中无效的切块为分辨率达不到要求的区域或未包含足够的甲状腺细胞的区域。
在一优选的实施方式中，切块初步筛查步骤中采用轻量化的resnet18的分类网络在imagenet训练得到的初始权重作为初始权重进行切块初步筛查，在resnet18的预测概率低于阈值的切块将被筛除。
在一优选的实施方式中，良恶性分类步骤包括采用弱监督的学习方法，充分利用未做标签的数据参与到训练中，训练具体流程包括第一阶段训练以及第二阶段训练；第一阶段训练包括：在140张诊断为甲状腺乳头状癌的切片上，由资深的甲状腺细胞病理医生标记1203张恶性的PTC切块图像作为正 类，在120张正常的切片上随机裁剪1203张切块，作为负类；按照4：1：1的比例分配训练集测试集和验证集，参与到模型1的训练中，本文选择resnet18构建模型1；对模型迭代训练，更新模型的权重，得到在测试集上性能最优的模型；以性能最优的模型对无标签的甲状腺的细胞病理切片的切块上进行预测，对预测概率大于给定阈值的切块的标签设定为正例标签；对预测概率小于给定阈值的切块标签设定为负例标签；及将正例标签和负例标签的标记的数据加入到训练集中；第二阶段训练包括：选择resnet34模型作为第二阶段训练模型2，对模型2进行迭代训练，对权重进行更新，得到模型2在测试集上最优的模型，并采用与第一阶段相同的方法对无标签的切块设定伪标签，扩充训练集；其中，第一阶段训练和第二阶段训练均分别以resnet50、resnet101、resnet152构建模型3、模型4、模型5逐步扩充数据，以进一步提高模型的性能；其中，对模型1、模型2、模型3、模型4、模型5在验证集上的准确率分别为0.78，0.80，0.83，0.87，0.95。
在一优选的实施方式中，在模型1至模型5的预测阶段，使用极大抑制技术将重叠的区域剔除掉，在相交的切块里，选择恶性概率最大的切块作为输出。
在一优选的实施方式中，可疑区域的后处理步骤中进行假阳去除的模型输入包括：图像模块的模型5预测的切块的概率；模型5倒数第二层提取的特征；基于局部二值模式提取的甲状腺细胞病理切块的图像特征；以及切块的中心点在图像中的坐标信息。
在一优选的实施方式中，病理切片的高危区域显示步骤中的每个切块的恶性预测的概率归一化到0～255。
为实现上述目的，本发明提供了一种基于深度学习的甲状腺细胞病理切片恶性区域的检测系统，其可以用于前述的检测方法，检测系统包括设备部分、硬件部分以及软件部分；设备部分主要包括提供图像来源的显微镜等；硬件部分主要包括GPU及CPU等；软件部分主要包括Keras、Pytorch、tensorflow、Caffe或Paddle等。
与现有技术相比，本发明的基于深度学习的甲状腺细胞病理切片恶性区域的检测方法具有以下有益效果：其采用由粗到细多阶段训练的方法，先剔除掉明显无效的切块，对有效的切块进良恶性分类，通过去除假阳进一步提高检测的准确率。在病理切片上预提取高风险的恶性细胞区域，协助医生进行诊断，做出决策，通过此流程，在大幅减少病理医生的工作量的同时，也降低了误诊率，提高了病理医生的工作效率，从而使病理医生可以将精力集中到更加复杂的病例上。
附图说明
图1是根据本发明一实施方式的检测方法的流程图。
具体实施方式
下面结合附图，对本发明的具体实施方式进行详细描述，但应当理解本发明的保护范围并不受具体实施方式的限制。
除非另有其它明确表示，否则在整个说明书和权利要求书中，术语“包括”或其变换如“包含”或“包括有”等等将被理解为包括所陈述的元件或组成部分，而并未排除其它元件或其它组成部分。
如图1所示，根据本发明优选实施方式的基于深度学习的甲状腺细胞病理切片恶性区域的检测方法，其主要包括下列切片步骤、预处理步骤、图像采样步骤、切块初步筛查步骤、良恶性分类步骤、可疑区域的后处理步骤以及病理切片的高危区域显示步骤等。
1：预处理
预处理的目的是为了对图像的特征进行强化，病理切片在显微镜上的图像经过数字化处理后使用不同的染色剂进行涂抹，得到彩色的病理切片，方便医生观察和诊断。但是不同的染色方法，以及操作人员的手法会带来不同的切片的染色差异，对使用机器学习和深度学习方法进行诊断分析带来干扰。 为减少这种影响，在图像预处理阶段，首先将图像由RGB空间转到灰度空间；同时由于玻片上存在污渍之类给细胞病理切片上引入了噪点，因此，同时采用高斯滤波的方法对图像进行了平滑处理，降低噪声的影响。
2：图像采样
完整的甲状腺细胞病理切片一般都非常大，将完整切片送到深度神经网络中训练，会耗费巨大的显存，成本非常高。因此本实施例采用了基于重叠面积为0.5(50％重叠)的重叠采样方法(采用重叠采样的方法，可以保证所有可能的位置都采集到，同时也保证了至少有一个可疑的区域能够完整的呈现)，裁剪出合适大小的块(在本实施例中选取的是512×512的分辨率大小的切块)，作为深度神经网络模型的输入。
3：切块初步筛查
在甲状腺的细胞病理切片中，有些区域分辨率达不到要求；有些区域没有包含足够的甲状腺细胞的区域，对诊断没有多大意义，这部分切块不需要专业医生也很容易筛查出来，因此本实施例训练了一个轻量化的resnet18的分类网络(具体的模型训练方法：人工标注1000例分辨率不足和没有足够甲状腺细胞的区块作为负类，视野清晰和视野中的甲状腺细胞满足下一个工作流的区块作为正例，基于resnet18在imagenet训练得到的初始权重作为初始权重进行迁移训练，选择在验证集上效果最优的模型作为切块初步筛查模型，在预测阶段，将resnet18的预测概率低于阈值(本实施例中设定的为0.2)的切块剔除掉)，筛除掉部分无效的切块，将剩下的切块送到良恶性分类模块，进行进一步检测。
4：良恶性分类
良恶性分类模块是本实施例的核心部分，这个模块，可以筛查出医生重点关注的甲状腺恶性细胞块，并利用病理切片高危区域显示模块绘制出恶性细胞团簇的热力图，方便病理医生进行诊断。
由于甲状腺细胞病理切片较大，有经验的病理医生较少，在病理切片上 标注恶性细胞块是一件非常耗时的事情。而无标签的甲状腺细胞病理切片则很容易获取，基于此，在良恶性分类模块，本实施例采用了一种弱监督的学习方法，充分利用未做标签的数据参与到训练中来，具体流程为：
1)在140张诊断为甲状腺乳头状癌(PTC)的切片上，由资深的甲状腺细胞病理医生标记1203张恶性的PTC切块图像作为正类，在120张正常的切片上随机裁剪1203张切块，作为负类。按照4：1：1的比例分配训练集测试集和验证集，参与到模型(Model)1的训练中(模型(Model)1选择为Resnet18)。对模型迭代训练，更新模型的权重，得到在测试集上性能最优的模型。以该模型对无标签的甲状腺的细胞病理切片的切块上进行预测，对预测概率大于给定阈值的切块的标签设定为正例标签(在本实施例中该阈值设定为0.9)。对预测概率小于给定阈值的切块标签设定为负例标签。将这些标记的数据加入到训练集中。(在本实施例中，以原始数据量的0.25倍扩充正例和负例的数量)。
2)由于训练集增大，本实施例选择了resnet34模型作为第二阶段训练模型模型(Model)2，对模型(Model)2进行迭代训练，对权重进行更新，得到模型(Model)2在测试集上最优的模型。使用和第一阶段相同的方法对无标签的切块设定伪标签，扩充训练集。
3)和步骤一和步骤二的方法类似，分别以resnet50，resnet101，resnet152构建模型(Model)3,模型(Model)4,模型(Model)5逐步扩充数据，进一步提高模型的性能。
4)对模型(Model)1,模型(Model)2,模型(Model)3,模型(Model)4,模型(Model)5在验证集上的准确率分别为0.78，0.80，0.83，0.87，0.95。说明弱监督学习的方法可以有效的提升良恶性细胞的分类性能。
在模型(Model)1～模型(Model)5的预测阶段，使用一种极大抑制的技术(NMS)将重叠的区域剔除掉：在相交的切块里，选择恶性概率最大的切块作为输出。
5：可疑区域的后处理
在本模块，本实施例提出了一种去假阳的方案，即构建一个基于随机森林的机器学习方法对良恶性分类的模型(Model)5预测结果进行假阳的去除，模型的输入来自于四个部分：图像模块的模型(Model)5预测的切块的概率；模型(Model)5倒数第二层提取的特征；基于局部二值模式(Local Binary Pattern(LBP))提取的甲状腺细胞病理切块的图像特征；以及切块的中心点在图像中的坐标信息(归一化到[0，1])。对随机森林模型进行训练。模型将概率低于一定的阈值(本实施例的阈值设定为0.2)的切块剔除掉，进一步减轻病理医生的工作。本实施例中这个阈值是可以动态调节的，如果病理医生想了解更多的恶性细胞病理切片的切块，做进一步的评估，可以手动将阈值调节的更低一点。
6：病理切片的高危区域显示
将每个切块的恶性预测的概率，归一化到[0,255]，映射到原图的中，将灰度图转化成彩色图像，生成热力图，呈现给细胞病理医生，给病理医生特别是经验不十分丰富的医生一个可视化的建议：哪部分的区域是需要重点关注的。本实施例的方法可以将从经验丰富的病理医生中学习到的经验迁移到年轻的医生中，帮助年轻的医生成长。
本实施例的模块4(良恶性分类步骤)可以和5(可疑区域的后处理步骤)可以串联工作，也可以独立工作。当医生需要更快的显示结果时，可以基于医生设定的阈值，直接将可疑恶性区域按照概率由高到低排序显示输出。如果医生想要更精确的结果，可以将医生基于阈值选择的结果(本实施例选择的是0.5)，基于模块5的预测概率按由高到低的显示输出。
根据本发明优选实施方式的基于深度学习的甲状腺细胞病理切片恶性区域的检测系统，其可以用于前述的检测方法，检测系统包括设备部分、硬件部分以及软件部分；设备部分主要包括提供图像来源的显微镜等；硬件部分主要包括GPU及CPU等；软件部分主要包括Keras、Pytorch、tensorflow、 Caffe或Paddle等。
在一些实施方式中，本实施例开发使用是Keras：2.1.6版本，但本发明并不限于此版本。同时可扩展使用的深度学习框架也可以但不限于Pytorch、tensorflow、Caffe或Paddle都可以应用于本发明的开发系统中。
在一些实施方式中，本实施例的GPU开发的硬件使用的是GeForce GTX1080ti，但本发明并不限于此型号的GPU。可以是任何独立显卡，包括且不限于，GeForce RTX 1060,GeForce RTX 2080ti，GeForce P6000.(更多显卡可查看英伟达官网)。
在一些实施方式中，本实施例的CPU开发的硬件使用的是Intel(R)Xeon(R)CPU E5-2640 v4@2.40GHz。更多服务器CPU参看Inter和AMD的服务器的CPU类型。
在一些实施方式中，本实施例的病理图片使用的是HE染色的图像，其他染色方案包括且不限于CK7，P40，CD56，六胺银，TTF1，PAS等。
综上所述，本发明的基于深度学习的甲状腺细胞病理切片恶性区域的检测方法具有以下有益效果：其采用由粗到细多阶段训练的方法，先剔除掉明显无效的切块，对有效的切块进良恶性分类，通过去除假阳进一步提高检测的准确率。在病理切片上预提取高风险的恶性细胞区域，协助医生进行诊断，作出决策，通过此流程，在大幅减少病理医生的工作量的同时，也降低了误诊率，提高了病理医生的工作效率，从而使病理医生可以将精力集中到更加复杂的病例上。
前述对本发明的具体示例性实施方案的描述是为了说明和例证的目的。这些描述并非想将本发明限定为所公开的精确形式，并且很显然，根据上述教导，可以进行很多改变和变化。对示例性实施例进行选择和描述的目的在于解释本发明的特定原理及其实际应用，从而使得本领域的技术人员能够实现并利用本发明的各种不同的示例性实施方案以及各种不同的选择和改变。本发明的范围意在由权利要求书及其等同形式所限定。

Claims (10)

  1. 一种基于深度学习的甲状腺细胞病理切片恶性区域的检测方法，其特征在于，包括下列步骤：
    切片步骤：对甲状腺细胞进行病理切片；
    预处理步骤：将病理切片在显微镜上的图像经过数字化处理后使用不同的染色剂进行涂抹以得到彩色的病理切片；
    图像采样步骤：将完整的病理切片裁剪出合适大小的切块作为深度神经网络模型的输入；
    切块初步筛查步骤：筛除掉病理切片部分无效的切块；
    良恶性分类步骤：采用弱监督的学习方法对切块初步筛查后的病理切片进行良恶性分类；
    可疑区域的后处理步骤：利用去假阳的方案构建一个基于随机森林的机器学习方法对良恶性分类的预测结果进行假阳的去除；
    病理切片的高危区域显示步骤：将每个切块的恶性预测的概率归一化并映射到原图的中，将灰度图转化成彩色图像且生成热力图。
  2. 如权利要求1所述的检测方法，其特征在于，所述切片步骤包括将彩色的病理切片图像由RGB空间转到灰度空间以及采用高斯滤波的方法对图像进行平滑处理。
  3. 如权利要求1所述的检测方法，其特征在于，所述图像采样步骤中采用基于重叠面积为50％的重叠采样方法将完整的病理切片裁剪出合适大小的切块，所述切块的分辨率为512x512像素。
  4. 如权利要求1所述的检测方法，其特征在于，所述切块初步筛查步骤中无效的切块为分辨率达不到要求的区域或未包含足够的甲状腺细胞的区域。
  5. 如权利要求4所述的检测方法，其特征在于，所述切块初步筛查步骤中采用轻量化的resnet18的分类网络在imagenet训练得到的初始权重作为初始权重进行切块初步筛查，在resnet18的预测概率低于阈值的切块将被筛除。
  6. 如权利要求1所述的检测方法，其特征在于，所述良恶性分类步骤包括采用弱监督的学习方法，充分利用未做标签的数据参与到训练中，训练具体流程包括：
    第一阶段训练包括：
    在140张诊断为甲状腺乳头状癌的切片上，由资深的甲状腺细胞病理医生标记1203张恶性的PTC切块图像作为正类，在120张正常的切片上随机裁剪1203张切块，作为负类；
    按照4：1：1的比例分配训练集测试集和验证集，参与到模型1的训练中，其中模型1的训练方法选择的为resnet18；
    对模型迭代训练，更新模型的权重，得到在测试集上性能最优的模型；以所述性能最优的模型对无标签的甲状腺的细胞病理切片的切块进行预测，对预测概率大于给定阈值的切块的标签设定为正例标签；
    对预测概率小于给定阈值的切块标签设定为负例标签；及
    将所述正例标签和所述负例标签的标记的数据加入到训练集中；以及
    第二阶段训练包括：
    选择resnet34模型作为第二阶段训练模型模型2，对模型2进行迭代训练，对权重进行更新，得到模型2在测试集上最优的模型，并采用与第一阶段相同的方法对无标签的切块设定伪标签，扩充训练集；
    其中，所述第一阶段训练和所述第二阶段训练均分别以resnet50、resnet101、resnet152构建模型3、模型4、模型5逐步扩充数据，以进一步提高模型的性能；
    其中，对模型1、模型2、模型3、模型4、模型5在验证集上的准确率 分别为0.78，0.80，0.83，0.87，0.95。
  7. 如权利要求6所述的检测方法，其特征在于，在模型1～模型5的预测阶段，使用极大抑制技术将重叠的区域剔除掉，在相交的切块里，选择恶性概率最大的切块作为输出。
  8. 如权利要求6所述的检测方法，其特征在于，所述可疑区域的后处理步骤中进行假阳去除的模型输入包括：
    图像模块的模型5预测的切块的概率；
    模型5倒数第二层提取的特征；
    基于局部二值模式提取的甲状腺细胞病理切块的图像特征；以及
    切块的中心点在图像中的坐标信息。
  9. 如权利要求1所述的检测方法，其特征在于，所述病理切片的高危区域显示步骤中的每个切块的恶性预测的概率归一化到0～255。
  10. 一种基于深度学习的甲状腺细胞病理切片恶性区域的检测系统，其应用于如权利要求1至9任一项的检测方法，其特征在于，所述检测系统包括设备部分、硬件部分以及软件部分；所述设备部分包括提供图像来源的显微镜；所述硬件部分包括GPU及CPU；所述软件部分包括Keras、Pytorch、tensorflow、Caffe或Paddle。
PCT/CN2021/091988 2020-11-10 2021-05-07 基于深度学习的甲状腺细胞病理切片恶性区域的检测方法 WO2022100034A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011243793.1A CN112508850B (zh) 2020-11-10 2020-11-10 基于深度学习的甲状腺细胞病理切片恶性区域的检测方法
CN202011243793.1 2020-11-10

Publications (1)

Publication Number Publication Date
WO2022100034A1 true WO2022100034A1 (zh) 2022-05-19

Family

ID=74955773

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/091988 WO2022100034A1 (zh) 2020-11-10 2021-05-07 基于深度学习的甲状腺细胞病理切片恶性区域的检测方法

Country Status (2)

Country Link
CN (1) CN112508850B (zh)
WO (1) WO2022100034A1 (zh)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115222734A (zh) * 2022-09-20 2022-10-21 山东大学齐鲁医院 一种用于胃黏膜肠上皮化生的图像分析方法及系统
CN115661815A (zh) * 2022-12-07 2023-01-31 赛维森(广州)医疗科技服务有限公司 基于全局特征映射的病理图像分类方法、图像分类装置
CN115880262A (zh) * 2022-12-20 2023-03-31 桂林电子科技大学 基于在线噪声抑制策略的弱监督病理图像组织分割方法
CN116313115A (zh) * 2023-05-10 2023-06-23 浙江大学 基于线粒体动态表型和深度学习的药物作用机制预测方法
CN116453114A (zh) * 2022-12-14 2023-07-18 西南医科大学附属口腔医院 一种基于深度学习的病理图像分析方法、设备及系统
CN116448760A (zh) * 2023-03-21 2023-07-18 上海华维可控农业科技集团股份有限公司 一种基于机器视觉的农业智能监测系统及方法
CN116580011A (zh) * 2023-05-19 2023-08-11 北京透彻未来科技有限公司 一种深度学习模型的子宫内膜癌全幻灯片图像检测系统
CN117557557A (zh) * 2024-01-09 2024-02-13 广州锟元方青医疗科技有限公司 一种基于细胞核分割模型的甲状腺病理切片细胞检测方法
CN117636064A (zh) * 2023-12-21 2024-03-01 浙江大学 一种基于儿童病理切片的神经母细胞瘤智能分类系统

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112508850B (zh) * 2020-11-10 2021-07-20 广州柏视医疗科技有限公司 基于深度学习的甲状腺细胞病理切片恶性区域的检测方法
CN113538435B (zh) * 2021-09-17 2022-02-18 北京航空航天大学 一种基于深度学习的胰腺癌病理图像分类方法及系统
CN114743195B (zh) * 2022-04-13 2022-12-09 赛维森(广州)医疗科技服务有限公司 甲状腺细胞病理数字图像识别器训练方法、图像处理方法
CN115909006B (zh) * 2022-10-27 2024-01-19 武汉兰丁智能医学股份有限公司 基于卷积Transformer的乳腺组织图像分类方法及系统

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7440601B1 (en) * 2003-10-10 2008-10-21 The United States Of America As Represented By The Department Of Health And Human Services Automated identification of ileocecal valve
CN103854028A (zh) * 2008-12-22 2014-06-11 赛利恩影像股份有限公司 自动检测医学图像中的病变的方法和系统
US20190015059A1 (en) * 2017-07-17 2019-01-17 Siemens Healthcare Gmbh Semantic segmentation for cancer detection in digital breast tomosynthesis
CN109670510A (zh) * 2018-12-21 2019-04-23 万达信息股份有限公司 一种基于深度学习的胃镜活检病理数据筛查系统和方法
CN110060245A (zh) * 2019-04-15 2019-07-26 青岛大学 一种基于深度神经网络的胃癌淋巴结染色病理图像自动识别方法
CN110838118A (zh) * 2019-09-24 2020-02-25 上海联影智能医疗科技有限公司 用于医疗过程中异常检测的系统和方法
CN112508850A (zh) * 2020-11-10 2021-03-16 广州柏视医疗科技有限公司 基于深度学习的甲状腺细胞病理切片恶性区域的检测方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108520518A (zh) * 2018-04-10 2018-09-11 复旦大学附属肿瘤医院 一种甲状腺肿瘤超声图像识别方法及其装置
CN110309860B (zh) * 2019-06-06 2022-11-08 昆明理工大学 基于卷积神经网络对肺结节的恶性程度进行分类的方法
CN111612752A (zh) * 2020-05-15 2020-09-01 江苏省人民医院（南京医科大学第一附属医院） 基于Faster-RCNN的超声图像甲状腺结节智能检测系统

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7440601B1 (en) * 2003-10-10 2008-10-21 The United States Of America As Represented By The Department Of Health And Human Services Automated identification of ileocecal valve
CN103854028A (zh) * 2008-12-22 2014-06-11 赛利恩影像股份有限公司 自动检测医学图像中的病变的方法和系统
US20190015059A1 (en) * 2017-07-17 2019-01-17 Siemens Healthcare Gmbh Semantic segmentation for cancer detection in digital breast tomosynthesis
CN109670510A (zh) * 2018-12-21 2019-04-23 万达信息股份有限公司 一种基于深度学习的胃镜活检病理数据筛查系统和方法
CN110060245A (zh) * 2019-04-15 2019-07-26 青岛大学 一种基于深度神经网络的胃癌淋巴结染色病理图像自动识别方法
CN110838118A (zh) * 2019-09-24 2020-02-25 上海联影智能医疗科技有限公司 用于医疗过程中异常检测的系统和方法
CN112508850A (zh) * 2020-11-10 2021-03-16 广州柏视医疗科技有限公司 基于深度学习的甲状腺细胞病理切片恶性区域的检测方法

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115222734B (zh) * 2022-09-20 2023-01-17 山东大学齐鲁医院 一种用于胃黏膜肠上皮化生的图像分析方法及系统
CN115222734A (zh) * 2022-09-20 2022-10-21 山东大学齐鲁医院 一种用于胃黏膜肠上皮化生的图像分析方法及系统
CN115661815A (zh) * 2022-12-07 2023-01-31 赛维森(广州)医疗科技服务有限公司 基于全局特征映射的病理图像分类方法、图像分类装置
CN115661815B (zh) * 2022-12-07 2023-09-12 赛维森(广州)医疗科技服务有限公司 基于全局特征映射的病理图像分类方法、图像分类装置
CN116453114A (zh) * 2022-12-14 2023-07-18 西南医科大学附属口腔医院 一种基于深度学习的病理图像分析方法、设备及系统
CN116453114B (zh) * 2022-12-14 2024-03-05 西南医科大学附属口腔医院 一种基于深度学习的病理图像分析方法、设备及系统
CN115880262A (zh) * 2022-12-20 2023-03-31 桂林电子科技大学 基于在线噪声抑制策略的弱监督病理图像组织分割方法
US11935279B1 (en) 2022-12-20 2024-03-19 Guilin University Of Electronic Technology Weakly supervised pathological image tissue segmentation method based on online noise suppression strategy
CN115880262B (zh) * 2022-12-20 2023-09-05 桂林电子科技大学 基于在线噪声抑制策略的弱监督病理图像组织分割方法
CN116448760B (zh) * 2023-03-21 2023-10-20 上海华维可控农业科技集团股份有限公司 一种基于机器视觉的农业智能监测系统及方法
CN116448760A (zh) * 2023-03-21 2023-07-18 上海华维可控农业科技集团股份有限公司 一种基于机器视觉的农业智能监测系统及方法
CN116313115B (zh) * 2023-05-10 2023-08-15 浙江大学 基于线粒体动态表型和深度学习的药物作用机制预测方法
CN116313115A (zh) * 2023-05-10 2023-06-23 浙江大学 基于线粒体动态表型和深度学习的药物作用机制预测方法
CN116580011A (zh) * 2023-05-19 2023-08-11 北京透彻未来科技有限公司 一种深度学习模型的子宫内膜癌全幻灯片图像检测系统
CN117636064A (zh) * 2023-12-21 2024-03-01 浙江大学 一种基于儿童病理切片的神经母细胞瘤智能分类系统
CN117636064B (zh) * 2023-12-21 2024-05-28 浙江大学 一种基于儿童病理切片的神经母细胞瘤智能分类系统
CN117557557A (zh) * 2024-01-09 2024-02-13 广州锟元方青医疗科技有限公司 一种基于细胞核分割模型的甲状腺病理切片细胞检测方法
CN117557557B (zh) * 2024-01-09 2024-04-26 广州锟元方青医疗科技有限公司 一种基于细胞核分割模型的甲状腺病理切片细胞检测方法

Also Published As

Publication number Publication date
CN112508850B (zh) 2021-07-20
CN112508850A (zh) 2021-03-16

Similar Documents

Publication Publication Date Title
WO2022100034A1 (zh) 基于深度学习的甲状腺细胞病理切片恶性区域的检测方法
Silva-Rodríguez et al. Going deeper through the Gleason scoring scale: An automatic end-to-end system for histology prostate grading and cribriform pattern detection
CN109154978B (zh) 用于检测植物疾病的系统和方法
CN110428432B (zh) 结肠腺体图像自动分割的深度神经网络算法
CN111985536B (zh) 一种基于弱监督学习的胃镜病理图像分类方法
CN109447998B (zh) 基于PCANet深度学习模型下的自动分割方法
CN111986150B (zh) 一种数字病理图像的交互式标注精细化方法
Wan et al. Robust nuclei segmentation in histopathology using ASPPU-Net and boundary refinement
CN110245657B (zh) 病理图像相似性检测方法及检测装置
CN109829882B (zh) 一种糖尿病视网膜病变分期预测方法
CN111062955A (zh) 一种肺部ct影像数据分割方法及系统
CN110838100A (zh) 一种基于滑动窗口的结肠镜病理切片筛查与分割系统
CN112488234B (zh) 一种基于注意力池化的端到端组织病理图像分类方法
Mathew et al. Computational methods for automated mitosis detection in histopathology images: A review
WO2020066257A1 (ja) 分類装置、分類方法、プログラム、ならびに、情報記録媒体
Bansal et al. Deep transfer learning techniques with hybrid optimization in early prediction and diagnosis of different types of oral cancer
CN112990214A (zh) 一种医学图像特征识别预测模型
Reddy et al. Multi-modal fusion of deep transfer learning based COVID-19 diagnosis and classification using chest x-ray images
CN117095180B (zh) 基于分期识别的胚胎发育阶段预测与质量评估方法
CN111062909A (zh) 乳腺肿块良恶性判断方法及设备
CN113870194B (zh) 深层特征和浅层lbp特征融合的乳腺肿瘤超声图像处理装置
Sertel et al. Computer-aided prognosis of neuroblastoma: classification of stromal development on whole-slide images
CN115311491A (zh) 一种基于课程学习及空间注意力的骨折假阳性筛除方法
Moradkhani et al. A new image mining approach for detecting micro-calcification in digital mammograms
Naik et al. Hybrid Feature Set based Mitotic Detection in Breast Histopathology Images

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21890567

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21890567

Country of ref document: EP

Kind code of ref document: A1