WO2023078041A1 - 基于自适应比例学习的肿瘤组织病理分类系统及方法 - Google Patents

基于自适应比例学习的肿瘤组织病理分类系统及方法 Download PDF

Info

Publication number
WO2023078041A1
WO2023078041A1 PCT/CN2022/124693 CN2022124693W WO2023078041A1 WO 2023078041 A1 WO2023078041 A1 WO 2023078041A1 CN 2022124693 W CN2022124693 W CN 2022124693W WO 2023078041 A1 WO2023078041 A1 WO 2023078041A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
image block
pathological
proportion
loss function
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/124693
Other languages
English (en)
French (fr)
Inventor
李劲松
田雨
周天舒
叶前呈
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University
Original Assignee
Zhejiang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University filed Critical Zhejiang University
Publication of WO2023078041A1 publication Critical patent/WO2023078041A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion

Definitions

  • the present invention relates to the field of medical image processing and machine learning, in particular to a tumor histopathological classification system and method based on adaptive proportional learning.
  • Pathological examination is the first step in cancer diagnosis and treatment, and the histopathological information provided is crucial for the judgment of cancer patients.
  • many pathological indicators cannot be fully and detailedly investigated in daily pathological examinations, and some rare tumor differentiations are only reported as moderately or poorly differentiated if they are not identified by pathologists.
  • tumors in organs other than the liver may also show hepatic differentiation, without squamous epithelial distribution but may also show squamous cell carcinoma differentiation, and more common neuroendocrine differentiation, etc., these types of differentiation Appearance may affect the patient's prognosis.
  • the lack of description of the relevant types of differentiation in the existing pathology reports is not conducive to the development of accurate personalized medicine. Therefore, additional identification methods or tools are needed to identify them and effectively supplement pathology reports.
  • Existing schemes include two types, one is manual reading, and the other is machine reading.
  • Manual film reading that is, the doctor checks the section image of the pathological sample after staining, and divides the tumor cells into high, medium and low differentiation grades according to the cell differentiation morphology, and divides the tumor cells into glandular tumor cells according to the tissue morphology between the cells. Growth patterns such as shape, nest shape and solid flake.
  • pathologists cannot complete a detailed examination without missing every region, and it is easy to miss some rare but highly aggressive differentiation subtypes.
  • Machine reading refers to the use of machine learning algorithms (including deep learning models) for model training based on pathological image features and image labels.
  • accuracy of the model test reaches a certain level, it can be applied to pathological feature recognition.
  • Most of the model training adopts the supervised learning method, that is, each minimum unit of training needs to have a class label. In order to achieve a high recognition accuracy, a large amount of training data and labels are required, and the labeling cost is too high. At present, the classification model for whether it is cancer is relatively common.
  • the use of multi-instance learning can effectively reduce the labeling cost, and at the same time, large-scale weakly supervised learning can improve the recognition accuracy.
  • Proportional label learning uses the proportion of categories in the data package for machine learning. However, in the field of pathological images, the acquisition cost of accurate proportional labels is about the same as that of supervised learning, which loses the advantage of low label cost of weakly supervised learning.
  • Prior art has following shortcoming:
  • the purpose of the present invention is to propose a tumor histopathological classification system and method based on self-adaptive ratio learning in view of the deficiencies in the prior art.
  • a tumor histopathological classification system based on adaptive ratio learning the system includes a ratio label labeling module, a foreground segmentation and image block positioning module and an adaptive multi-magnification integration ratio learning modules;
  • the proportional labeling module acquires several tumor tissue pathological slices and digitally scans them, manually labels the scanned pathological images according to the target category of the classification task, and constructs a data set;
  • the foreground segmentation and image block positioning module calculates the thumbnail size of the pathological image at different magnifications according to the set target image block overlap rate and utilizes the difference distribution of RGB channels and gray values of the thumbnail Features segment the foreground of tumor tissue, remove the background of tumor tissue, and each pixel in the thumbnail corresponds to an image block;
  • the self-adaptive multi-magnification integration ratio learning module calculates the probability that the segmented image blocks in the data set belong to each classification task target category under different magnifications through the neural network model, and obtains the proportion of each classification task target category in the pathological image, as Calculate the ratio. For each classification task target category, compare the calculated minimum value of probability under different magnifications with half of the maximum probability under different magnifications, and take the larger value as the image block in the classification task.
  • the loss function is the sum of the magnification loss function item at all levels, the integrated loss function item and the ratio fitting loss function item, where the ratio fitting loss function item is if the calculation ratio of the classification task target category is lower than the ratio threshold, then The difference between the proportion threshold and the calculated proportion is used as the proportion fitting loss function item.
  • the proportion fitting loss function item in the loss function is zero, and the proportion threshold is adjusted according to the training result to iteratively train the model.
  • the trained neural network model calculates the proportion of pathological images containing the target categories of each classification task.
  • the specific process of calculating the thumbnail size of the pathological image by the foreground segmentation and image block positioning module is as follows: mark the target image block size as Star (pixels)* Star (pixels), pixels are pixels, and the resolution of the target image block is is R tar , the overlap rate of the target image block is O tar , the target thumbnail is acquired for the pathological image B i whose original size is W ori (pixels) ⁇ H ori (pixels), and the original resolution is R ori , W ori (pixels ) and H ori (pixels) are the width and height of the original pathological image size respectively, then the width and height dimensions of the target thumbnail are:
  • the target thumbnail corresponds to the coordinates of the upper left corner of the target segmentation area of the pathological image B i as Among them, mod means to take the remainder, and perform pixel-level segmentation on the target segmentation area of the thumbnail.
  • Each pixel in the target thumbnail corresponds to an image block, and the size of the image block is
  • the foreground segmentation and image block positioning module uses the RGB channel of the thumbnail and the difference distribution feature of the gray value to segment the tumor tissue foreground.
  • the specific process is as follows: extract the red V Red , green V Green , and blue V Blue channels of the thumbnail Value and grayscale image value V Gray , calculate the segmentation thresholds of the five difference density distributions of V Red -V Gray , V Green -V Gray , V Blue -V Gray , V Red -V Green , and V Blue -V Green ; If the number of a pixel on the thumbnail that exceeds the above five segmentation thresholds is greater than or equal to 3, the image block corresponding to the pixel is regarded as the tissue foreground; otherwise, the image block corresponding to the pixel is removed as the background.
  • magnification loss function items at all levels in the loss function are as follows:
  • op represents different magnifications
  • ⁇ k is the training weight of the training image block in the k-th class calculation scale, is the training weight of the training image patch between the calculated scale and the scale threshold
  • is the set of training image blocks of the kth class is the integration probability of the jth image block in the kth class, is the label of the jth image block in the kth class
  • N i is the number of foreground image blocks in the i-th pathological image.
  • proportional fitting loss function item in the loss function is to make the training image blocks in a training batch All from the same pathological image: scale fitting loss function term defined as:
  • the specific selection process of the training image blocks is as follows: According to the k-th category in the classification task target category, the corresponding multi-magnification integration probabilities are arranged in descending order, and at this time The corresponding image block has the highest probability in the kth class, The corresponding image block has the lowest probability in the k-th class; for the image block classified as the k-th class in the pathological image B i , according to the marked scale threshold, select the image block within the scale threshold as the training image block:
  • the present invention also provides a tumor histopathological classification method based on self-adaptive ratio learning, the method comprising the following steps:
  • Step 1 Obtain several pathological slices of tumor tissue and perform digital scanning, manually label the scanned pathological images according to the target category of the classification task, and construct a data set;
  • Step 2 Under different magnifications, the thumbnail size of the pathological image is calculated according to the set target image block overlap rate, and the tumor tissue foreground is segmented using the difference distribution characteristics of the RGB channel and gray value of the thumbnail, Remove the background of the tumor tissue, perform pixel-level segmentation on the thumbnail, and each pixel corresponds to an image block;
  • Step 3 Calculate the probability that the segmented image blocks in the data set belong to each classification task target category under different magnifications through the neural network model, and obtain the proportion of the pathological image containing each classification task target category as the calculation ratio;
  • Step 4 For each classification task target category, compare the calculated minimum value of probability under different magnifications with half of the maximum probability under different magnifications, and take the larger value as the image block in the classification task Integration probability under the target category; arrange the integration probability of all image blocks under the same category in descending order, select training image blocks according to the marked ratio threshold, and assign a category label to each training image block;
  • Step 5 According to the difference between the marked ratio threshold and the calculated ratio, a higher training weight is given to the training image block between the calculated ratio and the ratio threshold in the pathological image, and the weight is proportional to the difference, and the adaptive weight assignment is performed ;
  • Step 6 Adjust the proportion threshold according to the training result to iteratively train the model, and calculate the proportion of each classification task target category contained in the pathological image through the trained neural network model.
  • the present invention uses the calibration standard of the proportional label to mark the image proportional label with a clear minimum ratio as the labeling system, reduces the cost of labeling and the requirements for labeling accuracy, and expands the data utilization rate; Extraction, complete multi-magnification image block pre-segmentation and coordinate positioning; finally, through adaptive proportional learning, improve data utilization and training efficiency; and through multi-magnification integrated training, improve recognition accuracy.
  • the present invention maximizes the detection rate on the basis of increasing the additional workload as low as possible.
  • Fig. 1 is a schematic flow chart of tumor histopathological classification based on adaptive proportional learning in the present invention
  • Fig. 2 is a schematic diagram of the calibration of the proportional label of the present invention
  • Fig. 3 is an example diagram of the image foreground segmentation result of the present invention.
  • a kind of tumor histopathological classification system based on self-adaptive scale learning provided by the present invention, this system comprises scale label labeling module, foreground segmentation and image block positioning module and self-adaptive multi-magnification integration scale learning module;
  • the proportional labeling module acquires several tumor tissue pathological slices and digitally scans them, manually labels the scanned pathological images according to the target category of the classification task, and constructs a data set; the specific process is as follows
  • B ⁇ B 1 , . . . , B i , . . . , B n ⁇ .
  • C categories in the classification task target category, and for each pathological image B i , construct a proportional label vector as
  • the ratio threshold refers to the lowest value of the ratio range when roughly estimated manually. Assuming that the proportion of the area of category k in B i is subjectively estimated to be about 50%, and it is impossible to determine whether the proportion is 48% or 54% or an objectively determined value without detailed delineation and calculation, but it is roughly between 40-60%, then the calibration It can be any value within (0, 40%], and the ratio threshold is set to 40%.
  • C 3 are non-cancerous tissue C 0 , common adenocarcinoma tissue C 1 , and hepatoid adenocarcinoma tissue C 2 .
  • Figure 2 The pathological diagnosis is hepatocellular carcinoma. During manual evaluation, it is roughly judged that C 2 is inside the dotted line box and C 0 is outside the box. If the two areas are similar, then give
  • the foreground segmentation and image block positioning module will calculate the thumbnail size of the pathological image and segment the thumbnail into the tumor tissue foreground, remove the tumor tissue background, and each pixel in the thumbnail corresponds to an image block; the specific process is as follows:
  • the image blocks of different magnifications are segmented based on the 20X magnification.
  • the magnification is 20X
  • the target image block resolution R tar 0.5 ⁇ m/pixel
  • the target image block size is S tar (pixels)*S tar (pixels)
  • pixels are pixels
  • the target image block overlap rate is O tar
  • the target thumbnail is obtained for the pathological image B i whose original size is W ori (pixels) ⁇ H ori (pixels)
  • the original resolution is R ori
  • W ori (pixels) and H ori (pixels) are the original pathological image respectively
  • the width and height of the size, then the width W tar and height H tar dimensions of the target thumbnail are respectively:
  • the coordinates (W left , H up ) of the upper left corner of the target segmented area of the pathological image B i corresponding to the thumbnail are:
  • Each pixel in the target thumbnail corresponds to a training image block under 20X magnification, and the size of the image block is:
  • each pixel corresponds to an image block; the specific process is as follows: extract the red (V) of the thumbnail Red ), green (V Green ), blue (V Blue ) channel values and grayscale value (V Gray ).
  • V Gray grayscale value
  • the segmentation thresholds of the five difference density distributions of V Red -V Gray , V Green -V Gray , V Blue -V Gray , V Red -V Green , and V Blue -V Green are calculated respectively.
  • the image block corresponding to the pixel It is regarded as the foreground of the tissue, and the coordinates of the upper left corner of the image block are (W left +a m *S 20X , H up +b m *S 20X ); otherwise, the image block corresponding to the pixel is taken as the background and removed.
  • the foreground segmentation each contour and invalid background on the image are identified, and only the foreground tissue (the non-white area in the rightmost image in Figure 3) is retained.
  • the coordinates of the upper left corner are The image size is 2*S 20X ⁇ 2*S 20X ; take the image block under 5X magnification The coordinates of the upper left corner are The image size is 4*S 20X ⁇ 4*S 20X .
  • each sample pack contains N i sample image block groups, denoted as
  • the self-adaptive multi-magnification integrated ratio learning module combines various levels of magnification and the cross-entropy function of the integrated magnification to form a loss function to realize multi-magnification integrated learning; through self-adaptive ratio learning, adaptive dynamic adjustment of image block training weight, and iteratively train the neural network; the specific process is as follows:
  • f(x) such as ResNet, etc.
  • the multi-magnification integration probability of the kth category can be calculated from the magnification probabilities of all levels as
  • Loss op is the loss function under the magnification factor o p
  • ⁇ k is the training weight of the kth class training image block.
  • Integrated loss function The integrated loss function term is defined as:
  • Training image block weight adjustment give higher training weight to the difference training image blocks in B i that are lower than the proportional label threshold, and the weight is proportional to the difference value, and adaptive weight assignment is performed. That is, if Will
  • G. Scale fitting loss function through random upsampling (if The number of training image blocks in the middle is less than the number of training image blocks in a single training batch) or downsampling (if The number of training image blocks in a training batch is greater than the number of training image blocks in a single training batch), so that the training image blocks in a training batch are all from If the k-th tissue proportion is lower than the scale label, then the proportional fitting loss function term is defined as:
  • the final loss function Loss total is the sum of the magnification loss function items at all levels, the integrated loss function item and the proportional fitting loss function item, that is
  • Test/Application For test data Test example set after foreground segmentation Use the best model to perform probabilistic inference calculations on all image blocks in the sample package, Indicates the classification probability at each magnification. Indicates the multi-magnification integration probability of the image block on the WSI for the k-th class differentiation.
  • the classification threshold generally set as 0.5
  • the classification threshold generally set as 0.5
  • it will be classified into the kth type of differentiation. If at least one image block in the WSI is classified as this type of differentiation, then it is judged that the tissue has this type of differentiation.
  • the proportion of WSI containing the k-th differentiation is
  • the present invention also provides a tumor histopathological classification method based on adaptive proportional learning, characterized in that the method comprises the following steps:
  • Step 1 Obtain several pathological slices of tumor tissue and perform digital scanning, manually label the scanned pathological images according to the target category of the classification task, and construct a data set; for this step, refer to the implementation process of the proportional label labeling module;
  • Step 2 Under different magnifications, the thumbnail size of the pathological image is calculated according to the set target image block overlap rate, and the tumor tissue foreground is segmented using the difference distribution characteristics of the RGB channel and gray value of the thumbnail, Remove the tumor tissue background, and perform pixel-level segmentation on the thumbnail, each pixel corresponding to an image block; this step refers to the implementation process of the foreground segmentation and image block positioning module;
  • Step 3 Calculate the probability that the segmented image blocks in the data set belong to each classification task target category under different magnifications through the neural network model, and obtain the proportion of the pathological image containing each classification task target category as the calculation ratio;
  • Step 4 For each classification task target category, compare the calculated minimum value of probability under different magnifications with half of the maximum probability under different magnifications, and take the larger value as the image block in the classification task Integration probability under the target category; arrange the integration probability of all image blocks under the same category in descending order, select training image blocks according to the marked ratio threshold, and assign a category label to each training image block;
  • Step 5 According to the difference between the marked ratio threshold and the calculated ratio, a higher training weight is given to the training image block between the calculated ratio and the ratio threshold in the pathological image, and the weight is proportional to the difference, and the adaptive weight assignment is performed ;
  • Step 6 Adjust the proportion threshold according to the training result to iteratively train the model, and calculate the proportion of each classification task target category contained in the pathological image through the trained neural network model.
  • steps 3 to 6 please refer to the implementation process of the self-adaptive multi-magnification integration ratio learning module.
  • hepatocellular carcinoma esophageal adenocarcinoma, gastric adenocarcinoma, colon adenocarcinoma, rectal adenocarcinoma, pancreatic cancer, cervical adenocarcinoma, esophageal squamous cell carcinoma, head and neck squamous cell carcinoma, Cervical squamous cell carcinoma, sarcoma and other data sets.
  • the best model was applied to the gastrointestinal adenocarcinoma data collected by three tertiary hospitals (135, 95, and 351 cases respectively) to identify whether it contained hepatic differentiation, that is, hepatoid adenocarcinoma feature.
  • the present invention is aimed at the identification task of hepatoid adenocarcinoma of the gastrointestinal tract.
  • the detection rates of hepatic differentiation in the gastrointestinal tract were 0%, 8.5%, and 56%. The detection rates can reach 100%, 80%, and 93.8%, and the false positives are all around 10%. On the basis of reducing the additional workload as much as possible, the detection rate is maximized.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

一种基于自适应比例学习的肿瘤组织病理分类系统及方法，首先获取若干病理切片并进行数字化扫描，将扫描后的病理图像按照分类任务目标类别进行人工标注，构建数据集；然后利用RGB通道和灰度值的差值分布特征分割组织前景，构建包含多级放大倍数的图像块的训练数据组；最后进行多级放大倍数整合，组合了各级放大倍数和整合放大倍数的交叉熵函数形成损失函数，实现多放大倍数整合学习；通过自适应比例学习，对图像全局比例标签和未达到最低比例的图像块训练权重进行动态调整，扩大数据利用率，实现快速收敛。在日常肿瘤组织的病理检查中，在尽可能低地增加额外工作负担的基础上，最大限度地提升检出率。

Description

基于自适应比例学习的肿瘤组织病理分类系统及方法 技术领域
本发明涉及医疗图像处理及机器学习领域，尤其涉及一种基于自适应比例学习的肿瘤组织病理分类系统及方法。
背景技术
病理检查作为癌症诊疗的第一步，提供的组织病理信息对于癌症患者病情判断至关重要。但由于病理医生缺口大，较多的病理指标无法在日常病理检查中得到完整详细的调查，一些罕见的肿瘤分化如果未被病理医生识别则仅报告为中分化或低分化。胚胎发育来源接近的消化系统中，肝脏以外的器官肿瘤也有可能出现肝样分化，没有鳞状上皮分布但也可能出现鳞状细胞癌分化，以及较为常见的神经内分泌分化等等，这些类别分化的出现可能会影响患者预后。现有的病理报告中对相关类型分化的描述缺失，不利于精准的个性化医疗的开展，因此需要额外的识别方法或工具对其进行识别，有效补充病理报告。
现有方案包括两类，一类是人工阅片，一类是机器阅片。人工阅片，即由医生对病理样本制片染色后的切片图像进行检查，根据细胞分化形态将肿瘤细胞分为高、中、低等不同分化等级，以及根据细胞间的组织形态分为腺体状、巢状、实性片状等生长模式。但在病理检查采样多、病理图像尺寸大的客观条件下，病理医生无法完成不遗漏每一区域的详尽检查，容易遗漏一些罕见但倾略性较高的分化亚型。
机器阅片即利用机器学习算法(包括深度学习模型在内)根据病理图像特征与图像标签进行模型训练，当模型测试的准确性达到一定程度时可应用于病理特征识别。模型训练多采用监督学习的方式，即每一个训练最小单位都需要有类标签，为达到较高的识别准确率则需要大量的训练数据及标签，标注成本过高。目前针对是否为癌的分类模型较为常见，利用多示例学习可以有效降低标签成本，同时大规模的弱监督学习可以提升识别准确性。比例标签学习则利用数据包内类别所占比例进行机器学习，但在病理图像领域，精确的比例标签的获取成本约等同于监督学习，失去了弱监督学习低标签成本的优势。现有技术有如下缺点：
1.人工阅片耗时耗力，且存在个体间和个体内的差异性，难以覆盖整张病理切片，无法获取较细粒度的分化差异；
2.由于样本制备、染色标准、扫描设备的不同，数字病理切片图像的数据质量差异较大，且易存在包括笔迹、边缘伪影在内的非实质组织区域，在去除这些质量问题前不适合模型训练与直接应用，在进行自动化识别前应进行质量控制，保证数据质量；
3.现有机器学习方法标注成本过高：监督学习需要大量像素级别或图像块级别的标注标签；多示例学习难有绝对的完全标签，且训练约束太少，模型效果不佳；全尺寸病理图像的客观比例标签获取成本同监督学习。
4.现有模型通常仅用单一放大倍数或不规定放大倍数进行建模，而绝大多数肿瘤分化需要综合不同放大倍数的形态特征以得到准确判断。
发明内容
本发明目的在于针对现有技术的不足，提出一种基于自适应比例学习的肿瘤组织病理分类系统及方法。
本发明的目的是通过以下技术方案来实现的：一种基于自适应比例学习的肿瘤组织病理分类系统，该系统包括比例标签标注模块、前景分割与图像块定位模块和自适应多放大倍数整合比例学习模块；
所述比例标签标注模块获取若干肿瘤组织病理切片并进行数字化扫描，将扫描后的病理图像按照分类任务目标类别进行人工标注，构建数据集；
所述前景分割与图像块定位模块将病理图像在不同的放大倍数下，根据设定的目标图像块重叠率计算病理图像的缩略图尺寸并利用缩略图的RGB通道和灰度值的差值分布特征分割肿瘤组织前景，去除肿瘤组织背景，缩略图中每个像素点对应一个图像块；
所述自适应多放大倍数整合比例学习模块通过神经网络模型计算数据集中分割后的图像块在不同放大倍数下属于各个分类任务目标类别的概率，得到病理图像含有各个分类任务目标类别的比例，作为计算比例，对于每个分类任务目标类别，分别将计算的不同放大倍数下概率最小值与不同放大倍数下概率最大值的二分之一进行比较，取其中较大值作为图像块在该分类任务目标类别下的整合概率；将所有图像块同一类别下的整合概率进行降序排列，根据标注的比例阈值，选择训练图像块，并对每个训练图像块赋值类别标签；根据标注的比例阈值与计算比例的差值，对于病理图像中处于计算比例与比例阈值之间的训练图像块给予更高的训练权重，权重与差值呈正比，进行自适应权重赋值；使用权重调整后的训练图像块训练神经网络，损失函数为各级放大倍数损失函数项、整合损失函数项及比例拟合损失函数项之和，其中比例拟合损失函数项为若分类任务目标类别的计算比例低于比例阈值，则比例阈值与计算比例的差值作为比例拟合损失函数项，若计算比例大于等于比例阈值，则损失函数中比例拟合损失函数项为零，并根据训练结果调整比例阈值对模型迭代训练，通过训练好的神经网络模型计算病理图像含有各个分类任务目标类别的比例。
进一步地，所述比例标签标注模块共获取n张病理切片的全尺寸数字化扫描图像，构成数据集记为B＝{B 1，...，B i，...，B n}，其中B i为第i张病理图像；分类任务目标类别共计C类， 针对第i张病理图像B i，构建比例标签向量R i为：
Figure PCTCN2022124693-appb-000001
其中，
Figure PCTCN2022124693-appb-000002
为第i个病理图像中第k类的比例阈值，且满足
Figure PCTCN2022124693-appb-000003
T表示转置。
进一步地，所述前景分割与图像块定位模块计算病理图像的缩略图尺寸具体过程如下：记目标图像块尺寸为S tar(pixels)*S tar(pixels)，pixels为像素，目标图像块分辨率为R tar，目标图像块重叠率为O tar，对原始尺寸为W ori(pixels)×H ori(pixels)，原始分辨率为R ori的病理图像B i进行目标缩略图获取，W ori(pixels)和H ori(pixels)分别为原始病理图像尺寸的宽和高，则目标缩略图的宽和高尺寸分别为：
Figure PCTCN2022124693-appb-000004
目标缩略图对应病理图像B i的目标分割区域左上角点坐标为
Figure PCTCN2022124693-appb-000005
其中，mod表示取余数，将缩略图的目标分割区域，进行像素级分割，目标缩略图中每一个像素点对应一个的图像块，图像块的尺寸为
Figure PCTCN2022124693-appb-000006
进一步地，所述前景分割与图像块定位模块利用缩略图的RGB通道和灰度值的差值分布特征分割肿瘤组织前景具体过程如下：提取缩略图红V Red、绿V Green、蓝V Blue通道值以及灰度图值V Grey，计算V Red-V Grey，V Green-V Grey，V Blue-V Grey，V Red-V Green，V Blue-V Green这五种差值密度分布的分割阈值；若缩略图上的一像素点超过上述五个分割阈值的数量大于等于3，则该像素点对应的图像块被视为组织前景，否则，该像素点对应的图像块作为背景剔除。
进一步地，所述损失函数中各级放大倍数损失函数项如下：
Figure PCTCN2022124693-appb-000007
其中，op表示不同的放大倍数，ω k为第k类计算比例中的训练图像块的训练权重，
Figure PCTCN2022124693-appb-000008
为处于计算比例和比例阈值之间的训练图像块的训练权重；
Figure PCTCN2022124693-appb-000009
为第k类训练图像块的集合，
Figure PCTCN2022124693-appb-000010
为第k类中第j个图像块的整合概率，
Figure PCTCN2022124693-appb-000011
为第k类中第j个图像块的标签；
Figure PCTCN2022124693-appb-000012
为图像块在各放大倍数下归类为第k类的概率；N i为第i个病理图像中前景图像块的数量。
进一步地，所述损失函数中整合损失函数项Loss combine如下：
Figure PCTCN2022124693-appb-000013
进一步地，所述损失函数中比例拟合损失函数项为根据去除背景图像块后的病理图像中的训练图像块数量，通过随机升采样或者随机降采样，使得一个训练批次中的训练图像块均来自同一张病理图像：比例拟合损失函数项
Figure PCTCN2022124693-appb-000014
定义为：
Figure PCTCN2022124693-appb-000015
进一步地，所述自适应多放大倍数整合比例学习模块根据神经网络训练结果调整比例阈值具体公式如下：
Figure PCTCN2022124693-appb-000016
其中，
Figure PCTCN2022124693-appb-000017
表示调整后的比例阈值，
Figure PCTCN2022124693-appb-000018
表示调整前的比例阈值，α为调整率。
进一步地，训练图像块的具体选择过程如下：根据分类任务目标类别中的第k类对应多放大倍数整合概率进行降序排列，得
Figure PCTCN2022124693-appb-000019
此时
Figure PCTCN2022124693-appb-000020
对应的图像块在第k类中概率最高，
Figure PCTCN2022124693-appb-000021
对应的图像块在第k类中概率最低；对于病理图像B i中归类为第k类的图像块，根据标注的比例阈值，选择在比例阈值内的图像块作为训练图像块：
Figure PCTCN2022124693-appb-000022
其中，
Figure PCTCN2022124693-appb-000023
表示
Figure PCTCN2022124693-appb-000024
对应的训练图像块；赋值类别标签
Figure PCTCN2022124693-appb-000025
本发明还提供了一种基于自适应比例学习的肿瘤组织病理分类方法，该方法包括如下步骤：
步骤1：获取若干肿瘤组织病理切片并进行数字化扫描，将扫描后的病理图像按照分类任务目标类别进行人工标注，构建数据集；
步骤2：将病理图像在不同的放大倍数下，根据设定的目标图像块重叠率计算病理图像的缩略图尺寸并利用缩略图的RGB通道和灰度值的差值分布特征分割肿瘤组织前景，去除肿瘤组织背景，对缩略图进行像素级分割，每个像素点对应一个图像块；
步骤3：通过神经网络模型计算数据集中分割后的图像块在不同放大倍数下属于各个分类任务目标类别的概率，得到病理图像含有各个分类任务目标类别的比例，作为计算比例；
步骤4：对于每个分类任务目标类别，分别将计算的不同放大倍数下概率最小值与不同放大倍数下概率最大值的二分之一进行比较，取其中较大值作为图像块在该分类任务目标类别下的整合概率；将所有图像块同一类别下的整合概率进行降序排列，根据标注的比例阈值，选择训练图像块，并对每个训练图像块赋值类别标签；
步骤5：根据标注的比例阈值与计算比例的差值，对于病理图像中处于计算比例与比例阈值之间的训练图像块给予更高的训练权重，权重与差值呈正比，进行自适应权重赋值；使用权重调整后的训练图像块训练神经网络，损失函数为各级放大倍数损失函数项、整合损失函数项及比例拟合损失函数项之和，其中比例拟合损失函数项为若分类任务目标类别的计算比例低于比例阈值，则比例阈值与计算比例的差值作为比例拟合损失函数项，若计算比例大于等于比例阈值，则损失函数中比例拟合损失函数项为零；
步骤6：根据训练结果调整比例阈值对模型迭代训练，通过训练好的神经网络模型计算病理图像含有各个分类任务目标类别的比例。
本发明的有益效果：本发明通过比例标签的标定标准，以明确最低比例为标注体系进行图像比例标签的标注，降低标注成本和对标注精度的要求，扩大数据利用率；通过有效前景组织的快速提取，完成多放大倍数图像块预分割与坐标定位；最后通过自适应比例学习，提升数据利用率和训练效率；并且通过多放大倍数整合训练，提高识别准确率。本发明在日常肿瘤组织的病理检查中，在尽可能低地增加额外工作负担的基础上，最大限度地提升检出率。
附图说明
图1为本发明基于自适应比例学习的肿瘤组织病理分类的流程示意图；
图2为本发明比例标签标定示意图；
图3为本发明图像前景分割结果示例图。
具体实施方式
以下结合附图对本发明具体实施方式作进一步详细说明。
如图1所示，本发明提供的一种基于自适应比例学习的肿瘤组织病理分类系统，该系统 包括比例标签标注模块、前景分割与图像块定位模块和自适应多放大倍数整合比例学习模块；
所述比例标签标注模块获取若干肿瘤组织病理切片并进行数字化扫描，将扫描后的病理图像按照分类任务目标类别进行人工标注，构建数据集；具体过程如下
总数据集记为B，共包含n张经苏木精—伊红染色法(hematoxylin-eosin，HE)获得的肿瘤组织病理切片的全尺寸数字化扫描图像(Whole Slide Image,WSI)B i，即B＝{B 1，...，B i，...，B n}。分类任务目标类别共计C类，针对每张病理图像B i，构建比例标签向量为
Figure PCTCN2022124693-appb-000026
其中，
Figure PCTCN2022124693-appb-000027
为第i个样本图像中第k类的比例阈值，且满足
Figure PCTCN2022124693-appb-000028
比例阈值指人工粗略估计时的比例范围的最低值。假设B i中第k类面积占比主观估计约占50％，不经过详细勾画计算比例无法确定是48％或54％或是客观确定值，但大致在40-60％之间，则标定
Figure PCTCN2022124693-appb-000029
为(0，40％]之内的任意数值均可，设定比例阈值为40％。
以肝样分化和普通腺癌分化的分类目标为例，C＝3，分别为非癌变组织C 0、普通腺癌组织C 1、肝样腺癌组织C 2。图2病理诊断为肝细胞肝癌，人工评估时粗略判断虚线框内为C 2，框外为C 0，两者面积相近，则赋予
Figure PCTCN2022124693-appb-000030
所述前景分割与图像块定位模块将计算病理图像的缩略图尺寸并将缩略图分割出肿瘤组织前景，去除肿瘤组织背景，缩略图中每个像素点对应一个图像块；具体过程如下：
为得到不同放大倍数(op)下的图像块坐标，以20X放大倍数为基础，进行不同放大倍数的图像块分割。当放大倍数为20X时，目标图像块分辨率R tar＝0.5μm/pixel，记目标图像块尺寸为S tar(pixels)*S tar(pixels)，pixels为像素，目标图像块重叠率为O tar，对原始尺寸为W ori(pixels)×H ori(pixels)，原始分辨率为R ori的病理图像B i进行目标缩略图获取，W ori(pixels)和H ori(pixels)分别为原始病理图像尺寸的宽和高，则目标缩略图的宽W tar和高H tar尺寸分别为：
Figure PCTCN2022124693-appb-000031
该缩略图对应病理图像B i的目标分割区域左上角点坐标(W left，H up)为：
Figure PCTCN2022124693-appb-000032
其中，mod表示取余数。目标缩略图中每一个像素点对应一个20X放大倍数下的训练图像块，图像块的尺寸为：
Figure PCTCN2022124693-appb-000033
利用RGB通道和灰度值的差值分布特征分割组织前景，去除肿瘤组织背景，对缩略图进行像素级分割，每个像素点对应一个图像块；具体过程如下：提取该缩略图的红(V Red)、绿(V Green)、蓝(V Blue)通道值以及灰度图值(V Grey)。依据Otsu阈值分割算法，分别计算V Red-V Grey，V Green-V Grey，V Blue-V Grey，V Red-V Green，V Blue-V Green这五种差值密度分布的分割阈值，若该缩略图上的第m个像素点(a m，b m)通过上述5个分割阈值的数量大于等于3，则该像素点对应的图像块
Figure PCTCN2022124693-appb-000034
被视为组织前景，该图像块的坐标左上角为(W left+a m*S 20X，H up+b m*S 20X)；否则，该像素点对应的图像块作为背景剔除。如图3所示，经过前景分割后，图像上的各轮廓与无效背景得以识别，仅保留前景组织(图3中最右侧图的非白色区域)。
以20X图像块区域为中心，取10X放大倍数下图像块
Figure PCTCN2022124693-appb-000035
左上角坐标为
Figure PCTCN2022124693-appb-000036
图像大小为2*S 20X×2*S 20X；取5X放大倍数下图像块
Figure PCTCN2022124693-appb-000037
左上角坐标为
Figure PCTCN2022124693-appb-000038
图像大小为4*S 20X×4*S 20X。
最终得去除背景图像块后的总数据集
Figure PCTCN2022124693-appb-000039
包含n个样本包
Figure PCTCN2022124693-appb-000040
每个样本包内含N i个示例图像块组，记为
Figure PCTCN2022124693-appb-000041
所述自适应多放大倍数整合比例学习模块组合了各级放大倍数和整合放大倍数的交叉墒函数形成损失函数，实现多放大倍数整合学习；通过自适应比例学习，自适应的动态调整图像块训练权重，并对神经网络进行迭代训练；具体过程如下：
A.各级放大倍数概率推算：利用神经网络模型f(x)(如ResNet等)对样本包中所有图像块进行概率推理计算，得到各图像的概率矩阵组
Figure PCTCN2022124693-appb-000042
各放大倍数概率矩阵可表示为：
Figure PCTCN2022124693-appb-000043
图像块
Figure PCTCN2022124693-appb-000044
在各放大倍数下归类为第k类概率为
Figure PCTCN2022124693-appb-000045
B.整合概率计算：由各级放大倍数概率可计算其在第k类上的多放大倍数整合概率为
Figure PCTCN2022124693-appb-000046
C.各类概率排序与标签赋值：根据第k类对应多放大倍数整合概率进行降序排列，得
Figure PCTCN2022124693-appb-000047
此时
Figure PCTCN2022124693-appb-000048
对应的图像块在第k类中概率最高，
Figure PCTCN2022124693-appb-000049
对应的图像块在第k类中概率最低。对于B i中应当归类为第k类的训练图像块：
Figure PCTCN2022124693-appb-000050
其中，
Figure PCTCN2022124693-appb-000051
表示
Figure PCTCN2022124693-appb-000052
对应的图像块；赋值类别标签
Figure PCTCN2022124693-appb-000053
D.各级放大倍数损失函数：将所有得到赋值类别标签的训练图像块
Figure PCTCN2022124693-appb-000054
计算交叉墒，即
Figure PCTCN2022124693-appb-000055
Loss op为在放大倍数o p下的损失函数，ω k为第k类训练图像块的训练权重。
E.整合损失函数：整合损失函数项定义为：
Figure PCTCN2022124693-appb-000056
F.训练图像块权重调整：对B i中低于比例标签阈值的差值训练图像块给予更高的训练权重，权重与差值呈正比，进行自适应权重赋值。即，若
Figure PCTCN2022124693-appb-000057
将
Figure PCTCN2022124693-appb-000058
其中，
Figure PCTCN2022124693-appb-000059
为处于计算比例和比例阈值之间的训练图像块的训练权重，替换公式(4)、(5)中的ω k项。
G.比例拟合损失函数：通过随机升采样(若
Figure PCTCN2022124693-appb-000060
中训练图像块数量小于单训练批次训练图像块数量)或者降采样(若
Figure PCTCN2022124693-appb-000061
中训练图像块数量大于单训练批次训练图像块数量)，使得一个训练批次中的训练图像块均来自
Figure PCTCN2022124693-appb-000062
第k的组织比例若低于比例标签，则有比例拟合损失函数项定义为：
Figure PCTCN2022124693-appb-000063
H.比例标签动态更新：由于人工标记的比例标签纪录的明确最低比例，并不等同于客观比例。因此在训练迭代模型参数前的样本赋予标签步骤中，根据前一次模型计算的各类组织比例小幅度调整比例标签，调整率α根据实验选择，本发明中设置为α＝1％，进行自适应比例标签调整，以公式(8)中的
Figure PCTCN2022124693-appb-000064
更新公式(3)、(6)、(7)中的
Figure PCTCN2022124693-appb-000065
Figure PCTCN2022124693-appb-000066
其中，
Figure PCTCN2022124693-appb-000067
表示调整后的比例阈值，
Figure PCTCN2022124693-appb-000068
表示调整前的比例阈值。
I.最终损失函数Loss total为各级放大倍数损失函数项、整合损失函数项及比例拟合损失函数项之和，即
Figure PCTCN2022124693-appb-000069
J.最佳模型参数状态选择方式：通过N epoch次步骤A-I的循环训练迭代后，Loss total将趋于稳定。对于第epoch次模型，其计算概率P i与标定标签R i之间的匹配度为：
Figure PCTCN2022124693-appb-000070
匹配度越高，模型识别效果越好。计算N epoch次模型匹配度，选定具有最高匹配度的模型参数作为最佳模型参数。
K.测试/应用：对于测试数据
Figure PCTCN2022124693-appb-000071
经过前景分割后得测试示例集
Figure PCTCN2022124693-appb-000072
利用最佳模型对样本包中所有图像块进行概率推理计算，
Figure PCTCN2022124693-appb-000073
表示各放大倍数下的分类概率。
Figure PCTCN2022124693-appb-000074
表示WSI上图像块对于第k类分化的多放大倍数整合概率。
应用在肿瘤组织病理分类问题时，
Figure PCTCN2022124693-appb-000075
高于分类阈值(一般定为0.5)则分类至第k类分化， 若WSI中含有至少一个图像块分类为该类分化，则判断该组织具有该类分化。应用在肿瘤组织病理分类量化问题时，WSI含有第k类分化的比例为
Figure PCTCN2022124693-appb-000076
本发明还提供了一种基于自适应比例学习的肿瘤组织病理分类方法，其特征在于，该方法包括如下步骤：
步骤1：获取若干肿瘤组织病理切片并进行数字化扫描，将扫描后的病理图像按照分类任务目标类别进行人工标注，构建数据集；该步骤具体参考比例标签标注模块的实现过程；
步骤2：将病理图像在不同的放大倍数下，根据设定的目标图像块重叠率计算病理图像的缩略图尺寸并利用缩略图的RGB通道和灰度值的差值分布特征分割肿瘤组织前景，去除肿瘤组织背景，对缩略图进行像素级分割，每个像素点对应一个图像块；该步骤具体参考前景分割与图像块定位模块的实现过程；
步骤3：通过神经网络模型计算数据集中分割后的图像块在不同放大倍数下属于各个分类任务目标类别的概率，得到病理图像含有各个分类任务目标类别的比例，作为计算比例；
步骤4：对于每个分类任务目标类别，分别将计算的不同放大倍数下概率最小值与不同放大倍数下概率最大值的二分之一进行比较，取其中较大值作为图像块在该分类任务目标类别下的整合概率；将所有图像块同一类别下的整合概率进行降序排列，根据标注的比例阈值，选择训练图像块，并对每个训练图像块赋值类别标签；
步骤5：根据标注的比例阈值与计算比例的差值，对于病理图像中处于计算比例与比例阈值之间的训练图像块给予更高的训练权重，权重与差值呈正比，进行自适应权重赋值；使用权重调整后的训练图像块训练神经网络，损失函数为各级放大倍数损失函数项、整合损失函数项及比例拟合损失函数项之和，其中比例拟合损失函数项为若分类任务目标类别的计算比例低于比例阈值，则比例阈值与计算比例的差值作为比例拟合损失函数项，若计算比例大于等于比例阈值，则损失函数中比例拟合损失函数项为零；
步骤6：根据训练结果调整比例阈值对模型迭代训练，通过训练好的神经网络模型计算病理图像含有各个分类任务目标类别的比例。
步骤3～步骤6具体参考自适应多放大倍数整合比例学习模块的实现过程。
本发明实施例中收集TCGA(The Cancer Genome Atlas)数据库中的肝细胞肝癌、食管腺癌、胃腺癌、结肠腺癌、直肠腺癌、胰腺癌、宫颈腺癌、食管鳞癌、头颈鳞癌、宫颈鳞癌、肉瘤等数据集。对所有图像进行比例标签标定，以肝样分化和普通腺癌分化的分类目标为例，C＝3，分别为非癌变组织C 0、普通腺癌组织C 1、肝样腺癌组织C 2。其中仅有肝细胞肝癌中的图像含有C 2组织，其余数据均不含。
经过数据预处理和模型训练后，将最佳模型应用于三家三甲医院收集的胃肠道腺癌数据 (分别有135、95、351例)，识别其是否含有肝样分化，即肝样腺癌特征。本发明针对胃肠道肝样腺癌的识别任务，在三家三甲医院病理科的日常病理检查中，胃肠道肝样分化检出率为0％、8.5％、56％，本方法构建模型检出率可达100％、80％、93.8％，假阳性均在10％左右，在尽可能低地增加额外工作负担的基础上，最大限度地提升检出率。
上述实施例用来解释说明本发明，而不是对本发明进行限制，在本发明的精神和权利要求的保护范围内，对本发明作出的任何修改和改变，都落入本发明的保护范围。

Claims (10)

  1. 一种基于自适应比例学习的肿瘤组织病理分类系统，其特征在于，该系统包括比例标签标注模块、前景分割与图像块定位模块和自适应多放大倍数整合比例学习模块；
    所述比例标签标注模块获取若干肿瘤组织病理切片并进行数字化扫描，将扫描后的病理图像按照分类任务目标类别进行人工标注，构建数据集；
    所述前景分割与图像块定位模块将病理图像在不同的放大倍数下，根据设定的目标图像块重叠率计算病理图像的缩略图尺寸并利用缩略图的RGB通道和灰度值的差值分布特征分割肿瘤组织前景，去除肿瘤组织背景，缩略图中每个像素点对应一个图像块；
    所述自适应多放大倍数整合比例学习模块通过神经网络模型计算数据集中分割后的图像块在不同放大倍数下属于各个分类任务目标类别的概率，得到病理图像含有各个分类任务目标类别的比例，作为计算比例，对于每个分类任务目标类别，分别将计算的不同放大倍数下概率最小值与不同放大倍数下概率最大值的二分之一进行比较，取其中较大值作为图像块在该分类任务目标类别下的整合概率；将所有图像块同一类别下的整合概率进行降序排列，根据标注的比例阈值，选择训练图像块，并对每个训练图像块赋值类别标签；根据标注的比例阈值与计算比例的差值，对于病理图像中处于计算比例与比例阈值之间的训练图像块给予更高的训练权重，权重与差值呈正比，进行自适应权重赋值；使用权重调整后的训练图像块训练神经网络，损失函数为各级放大倍数损失函数项、整合损失函数项及比例拟合损失函数项之和，其中比例拟合损失函数项为若分类任务目标类别的计算比例低于比例阈值，则比例阈值与计算比例的差值作为比例拟合损失函数项，若计算比例大于等于比例阈值，则损失函数中比例拟合损失函数项为零，并根据训练结果调整比例阈值对模型迭代训练，通过训练好的神经网络模型计算病理图像含有各个分类任务目标类别的比例。
  2. 根据权利要求1所述的一种基于自适应比例学习的肿瘤组织病理分类系统，其特征在于，所述比例标签标注模块共获取n张病理切片的全尺寸数字化扫描图像，构成数据集记为B＝{B 1,...，B i，...，B n}，其中B i为第i张病理图像；分类任务目标类别共计C类，针对第i张病理图像B i，构建比例标签向量R i为：
    Figure PCTCN2022124693-appb-100001
    其中，
    Figure PCTCN2022124693-appb-100002
    为第i个病理图像中第k类的比例阈值，且满足
    Figure PCTCN2022124693-appb-100003
    T表示转置。
  3. 根据权利要求1所述的一种基于自适应比例学习的肿瘤组织病理分类系统，其特征在于，所述前景分割与图像块定位模块计算病理图像的缩略图尺寸具体过程如下：记目标图像块尺寸为S tar(pixels)*S tar(pixels)，pixels为像素，目标图像块分辨率为R tar，目标图像块 重叠率为O tar，对原始尺寸为W ori(pixels)×H ori(pixels)，原始分辨率为R ori的病理图像B i进行目标缩略图获取，W ori(pixels)和H ori(pixels)分别为原始病理图像尺寸的宽和高，则目标缩略图的宽和高尺寸分别为：
    Figure PCTCN2022124693-appb-100004
    目标缩略图对应病理图像B i的目标分割区域左上角点坐标为
    Figure PCTCN2022124693-appb-100005
    其中，mod表示取余数，将缩略图的目标分割区域，进行像素级分割，目标缩略图中每一个像素点对应一个的图像块，图像块的尺寸为
    Figure PCTCN2022124693-appb-100006
  4. 根据权利要求1所述的一种基于自适应比例学习的肿瘤组织病理分类系统，其特征在于，所述前景分割与图像块定位模块利用缩略图的RGB通道和灰度值的差值分布特征分割肿瘤组织前景具体过程如下：提取缩略图红V Red、绿V Green、蓝V Blue通道值以及灰度图值V Grey，计算V Red-V Grey，V Green-V Grey，V Blue-V Grey，V Red-V Green，V Blue-V Green这五种差值密度分布的分割阈值；若缩略图上的一像素点超过上述五个分割阈值的数量大于等于3，则该像素点对应的图像块被视为组织前景，否则，该像素点对应的图像块作为背景剔除。
  5. 根据权利要求2所述的一种基于自适应比例学习的肿瘤组织病理分类系统，其特征在于，所述损失函数中各级放大倍数损失函数项如下：
    Figure PCTCN2022124693-appb-100007
    其中，op表示不同的放大倍数，ω k为第k类计算比例中的训练图像块的训练权重，
    Figure PCTCN2022124693-appb-100008
    为处于计算比例和比例阈值之间的训练图像块的训练权重；
    Figure PCTCN2022124693-appb-100009
    为第k类训练图像块的集合，
    Figure PCTCN2022124693-appb-100010
    为第k类中第j个图像块的整合概率，
    Figure PCTCN2022124693-appb-100011
    为第k类中第j个图像块的标签；
    Figure PCTCN2022124693-appb-100012
    为图像块在各放大倍数下归类为第k类的概率；N i为第i个病理图像中前景图像块的数量。
  6. 根据权利要求5所述的一种基于自适应比例学习的肿瘤组织病理分类系统，其特征在于，所述损失函数中整合损失函数项Loss combine如下：
    Figure PCTCN2022124693-appb-100013
  7. 根据权利要求5所述的一种基于自适应比例学习的肿瘤组织病理分类系统，其特征在于，所述损失函数中比例拟合损失函数项为根据去除背景图像块后的病理图像中的训练图像块数量，通过随机升采样或者随机降采样，使得一个训练批次中的训练图像块均来自同一张病理图像：比例拟合损失函数项
    Figure PCTCN2022124693-appb-100014
    定义为：
    Figure PCTCN2022124693-appb-100015
  8. 根据权利要求5所述的一种基于自适应比例学习的肿瘤组织病理分类系统，其特征在于，所述自适应多放大倍数整合比例学习模块根据神经网络训练结果调整比例阈值具体公式如下：
    Figure PCTCN2022124693-appb-100016
    其中，
    Figure PCTCN2022124693-appb-100017
    表示调整后的比例阈值，
    Figure PCTCN2022124693-appb-100018
    表示调整前的比例阈值，α为调整率。
  9. 根据权利要求5所述的一种基于自适应比例学习的肿瘤组织病理分类系统，其特征在于，训练图像块的具体选择过程如下：根据分类任务目标类别中的第k类对应多放大倍数整合概率进行降序排列，得
    Figure PCTCN2022124693-appb-100019
    此时
    Figure PCTCN2022124693-appb-100020
    对应的图像块在第k类中概率最高，
    Figure PCTCN2022124693-appb-100021
    对应的图像块在第k类中概率最低；对于病理图像B i中归类为第k类的图像块，根据标注的比例阈值，选择在比例阈值内的图像块作为训练图像块：
    Figure PCTCN2022124693-appb-100022
    其中，
    Figure PCTCN2022124693-appb-100023
    表示
    Figure PCTCN2022124693-appb-100024
    对应的训练图像块；赋值类别标签
    Figure PCTCN2022124693-appb-100025
  10. 一种基于自适应比例学习的肿瘤组织病理分类方法，其特征在于，该方法包括如下步骤：
    步骤1：获取若干肿瘤组织病理切片并进行数字化扫描，将扫描后的病理图像按照分类任务目标类别进行人工标注，构建数据集；
    步骤2：将病理图像在不同的放大倍数下，根据设定的目标图像块重叠率计算病理图像的缩略图尺寸并利用缩略图的RGB通道和灰度值的差值分布特征分割肿瘤组织前景，去除肿瘤组织背景，对缩略图进行像素级分割，每个像素点对应一个图像块；
    步骤3：通过神经网络模型计算数据集中分割后的图像块在不同放大倍数下属于各个分类任务目标类别的概率，得到病理图像含有各个分类任务目标类别的比例，作为计算比例；
    步骤4：对于每个分类任务目标类别，分别将计算的不同放大倍数下概率最小值与不同放大倍数下概率最大值的二分之一进行比较，取其中较大值作为图像块在该分类任务目标类别下的整合概率；将所有图像块同一类别下的整合概率进行降序排列，根据标注的比例阈值，选择训练图像块，并对每个训练图像块赋值类别标签；
    步骤5：根据标注的比例阈值与计算比例的差值，对于病理图像中处于计算比例与比例阈值之间的训练图像块给予更高的训练权重，权重与差值呈正比，进行自适应权重赋值；使用权重调整后的训练图像块训练神经网络，损失函数为各级放大倍数损失函数项、整合损失函数项及比例拟合损失函数项之和，其中比例拟合损失函数项为若分类任务目标类别的计算比例低于比例阈值，则比例阈值与计算比例的差值作为比例拟合损失函数项，若计算比例大于等于比例阈值，则损失函数中比例拟合损失函数项为零；
    步骤6：根据训练结果调整比例阈值对模型迭代训练，通过训练好的神经网络模型计算病理图像含有各个分类任务目标类别的比例。
PCT/CN2022/124693 2021-11-03 2022-10-11 基于自适应比例学习的肿瘤组织病理分类系统及方法 Ceased WO2023078041A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111291695.X 2021-11-03
CN202111291695.XA CN113723573B (zh) 2021-11-03 2021-11-03 基于自适应比例学习的肿瘤组织病理分类系统及方法

Publications (1)

Publication Number Publication Date
WO2023078041A1 true WO2023078041A1 (zh) 2023-05-11

Family

ID=78686597

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/124693 Ceased WO2023078041A1 (zh) 2021-11-03 2022-10-11 基于自适应比例学习的肿瘤组织病理分类系统及方法

Country Status (2)

Country Link
CN (1) CN113723573B (zh)
WO (1) WO2023078041A1 (zh)

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116934635A (zh) * 2023-09-18 2023-10-24 苏州可帮基因科技有限公司 基于图像处理技术的病理图像质控方法及设备
CN116953006A (zh) * 2023-07-05 2023-10-27 长三角先进材料研究院 一种铸造材料扫描电镜图像缺陷识别与量化方法
CN116958509A (zh) * 2023-08-23 2023-10-27 福州大学 一种罕见目标在多场景中的数据集构建方法及系统
CN117058292A (zh) * 2023-07-28 2023-11-14 北京透彻未来科技有限公司 基于数字病理图像的色阶图渲染系统
CN117392693A (zh) * 2023-12-11 2024-01-12 苏州可帮基因科技有限公司 病理图像去笔迹的方法及设备
CN117392468A (zh) * 2023-12-11 2024-01-12 山东大学 基于多示例学习的癌症病理图像分类系统、介质及设备
CN117422912A (zh) * 2023-10-20 2024-01-19 哈尔滨工业大学 一种基于提示学习的组织病理图像多任务分类系统
CN117557558A (zh) * 2024-01-10 2024-02-13 汕头大学医学院 一种基于半监督学习的全切片病理图像分类方法
CN117649381A (zh) * 2023-11-27 2024-03-05 北京透彻未来科技有限公司 一种基于数字病理切片自动计算缩放倍率的装置
CN117670895A (zh) * 2024-02-01 2024-03-08 华南理工大学 基于切片重染色技术的免疫组化病理图像细胞分割方法
CN118262097A (zh) * 2024-04-10 2024-06-28 北京透彻未来科技有限公司 一种基于病理大模型的目标检测模型构建方法
CN118448056A (zh) * 2024-05-20 2024-08-06 暨南大学附属第一医院（广州华侨医院） 一种基于网络搜索机器学习的颅脑肿瘤可视化模型构建系统
CN118690289A (zh) * 2024-08-29 2024-09-24 苏州可帮基因科技有限公司 基于深度学习的肿瘤组织溯源方法及设备
CN118781423A (zh) * 2024-07-11 2024-10-15 大连医科大学附属第一医院 一种基于二阶池化混合监督学习的乳腺癌全切片病理图像分类方法
CN118799630A (zh) * 2024-06-20 2024-10-18 哈尔滨工业大学 一种基于自我激励多示例学习的病理图像分类方法
CN118941513A (zh) * 2024-07-16 2024-11-12 武汉兰丁云医学检验实验室有限公司 LD CytoMatrix 20玻片扫描图像质量自动化评估方法
CN119228789A (zh) * 2024-11-28 2024-12-31 神州医疗科技股份有限公司 面向病理图像的基于大模型的图像分析和诊断方法及系统
CN119625303A (zh) * 2024-11-21 2025-03-14 北京大学第一医院(北京大学第一临床医学院) 一种基于深度学习的肾脏组分分割方法和系统
CN120598873A (zh) * 2025-05-23 2025-09-05 郑州大学第一附属医院 基于深度学习的气管镜图像分析方法、系统及存储介质
CN120876925A (zh) * 2025-05-28 2025-10-31 南通大学附属医院 一种基于深度学习的肿瘤图像分类方法及系统
CN121504917A (zh) * 2026-01-12 2026-02-10 华中科技大学 一种脑卒中病灶检测方法及系统

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113723573B (zh) * 2021-11-03 2022-01-14 浙江大学 基于自适应比例学习的肿瘤组织病理分类系统及方法
CN115439843B (zh) * 2021-12-01 2025-11-14 万达信息股份有限公司 一种基于空间位置信息的免标注病理图像特征提取方法
CN114820502B (zh) * 2022-04-21 2023-10-24 济宁医学院附属医院 一种用于肠粘膜组织中蛋白激酶ck2的着色检测方法
CN115359031A (zh) * 2022-08-31 2022-11-18 浙江大学 一种数字病理图像制片质量评价方法
CN116228737B (zh) * 2023-03-29 2026-03-17 南京大学 一种病理切片肿瘤间质比量化方法
CN120279031B (zh) * 2025-06-12 2025-09-05 浙江大学 基于联邦持续学习的肝细胞癌病理微坏死识别系统及方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109528230A (zh) * 2018-11-21 2019-03-29 济南浪潮高新科技投资发展有限公司 一种基于多级变换网络的乳腺肿瘤分割方法及装置
US20190156159A1 (en) * 2017-11-20 2019-05-23 Kavya Venkata Kota Sai KOPPARAPU System and method for automatic assessment of cancer
CN113723573A (zh) * 2021-11-03 2021-11-30 浙江大学 基于自适应比例学习的肿瘤组织病理分类系统及方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108765408B (zh) * 2018-05-31 2021-09-10 杭州同绘科技有限公司 构建癌症病理图像虚拟病例库的方法以及基于卷积神经网络的多尺度癌症检测系统
CN111079862B (zh) * 2019-12-31 2023-05-16 西安电子科技大学 基于深度学习的甲状腺乳头状癌病理图像分类方法
CN113221978A (zh) * 2021-04-27 2021-08-06 浙江师范大学 基于弱监督学习的结直肠癌数字病理图像判别方法及系统

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190156159A1 (en) * 2017-11-20 2019-05-23 Kavya Venkata Kota Sai KOPPARAPU System and method for automatic assessment of cancer
CN109528230A (zh) * 2018-11-21 2019-03-29 济南浪潮高新科技投资发展有限公司 一种基于多级变换网络的乳腺肿瘤分割方法及装置
CN113723573A (zh) * 2021-11-03 2021-11-30 浙江大学 基于自适应比例学习的肿瘤组织病理分类系统及方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
YE QIANCHENG; ZHANG QI; TIAN YU; ZHOU TIANSHU; GE HONGBIN; WU JIAJUN; LU NA; BAI XUELI; LIANG TINGBO; LI JINGSONG: "Method of Tumor Pathological Micronecrosis Quantification Via Deep Learning From Label Fuzzy Proportions", IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, vol. 25, no. 9, 6 April 2021 (2021-04-06), Piscataway, NJ, USA , pages 3288 - 3299, XP011876398, ISSN: 2168-2194, DOI: 10.1109/JBHI.2021.3071276 *

Cited By (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116953006A (zh) * 2023-07-05 2023-10-27 长三角先进材料研究院 一种铸造材料扫描电镜图像缺陷识别与量化方法
CN117058292A (zh) * 2023-07-28 2023-11-14 北京透彻未来科技有限公司 基于数字病理图像的色阶图渲染系统
CN117058292B (zh) * 2023-07-28 2024-04-26 北京透彻未来科技有限公司 基于数字病理图像的色阶图渲染系统
CN116958509A (zh) * 2023-08-23 2023-10-27 福州大学 一种罕见目标在多场景中的数据集构建方法及系统
CN116934635B (zh) * 2023-09-18 2023-12-12 苏州可帮基因科技有限公司 基于图像处理技术的病理图像质控方法及设备
CN116934635A (zh) * 2023-09-18 2023-10-24 苏州可帮基因科技有限公司 基于图像处理技术的病理图像质控方法及设备
CN117422912A (zh) * 2023-10-20 2024-01-19 哈尔滨工业大学 一种基于提示学习的组织病理图像多任务分类系统
CN117649381A (zh) * 2023-11-27 2024-03-05 北京透彻未来科技有限公司 一种基于数字病理切片自动计算缩放倍率的装置
CN117392468A (zh) * 2023-12-11 2024-01-12 山东大学 基于多示例学习的癌症病理图像分类系统、介质及设备
CN117392468B (zh) * 2023-12-11 2024-02-13 山东大学 基于多示例学习的癌症病理图像分类系统、介质及设备
CN117392693B (zh) * 2023-12-11 2024-03-01 苏州可帮基因科技有限公司 病理图像去笔迹的方法及设备
CN117392693A (zh) * 2023-12-11 2024-01-12 苏州可帮基因科技有限公司 病理图像去笔迹的方法及设备
CN117557558A (zh) * 2024-01-10 2024-02-13 汕头大学医学院 一种基于半监督学习的全切片病理图像分类方法
CN117670895A (zh) * 2024-02-01 2024-03-08 华南理工大学 基于切片重染色技术的免疫组化病理图像细胞分割方法
CN117670895B (zh) * 2024-02-01 2024-04-19 华南理工大学 基于切片重染色技术的免疫组化病理图像细胞分割方法
CN118262097A (zh) * 2024-04-10 2024-06-28 北京透彻未来科技有限公司 一种基于病理大模型的目标检测模型构建方法
CN118448056A (zh) * 2024-05-20 2024-08-06 暨南大学附属第一医院（广州华侨医院） 一种基于网络搜索机器学习的颅脑肿瘤可视化模型构建系统
CN118799630A (zh) * 2024-06-20 2024-10-18 哈尔滨工业大学 一种基于自我激励多示例学习的病理图像分类方法
CN118781423A (zh) * 2024-07-11 2024-10-15 大连医科大学附属第一医院 一种基于二阶池化混合监督学习的乳腺癌全切片病理图像分类方法
CN118941513A (zh) * 2024-07-16 2024-11-12 武汉兰丁云医学检验实验室有限公司 LD CytoMatrix 20玻片扫描图像质量自动化评估方法
CN118690289A (zh) * 2024-08-29 2024-09-24 苏州可帮基因科技有限公司 基于深度学习的肿瘤组织溯源方法及设备
CN119625303A (zh) * 2024-11-21 2025-03-14 北京大学第一医院(北京大学第一临床医学院) 一种基于深度学习的肾脏组分分割方法和系统
CN119228789A (zh) * 2024-11-28 2024-12-31 神州医疗科技股份有限公司 面向病理图像的基于大模型的图像分析和诊断方法及系统
CN120598873A (zh) * 2025-05-23 2025-09-05 郑州大学第一附属医院 基于深度学习的气管镜图像分析方法、系统及存储介质
CN120876925A (zh) * 2025-05-28 2025-10-31 南通大学附属医院 一种基于深度学习的肿瘤图像分类方法及系统
CN121504917A (zh) * 2026-01-12 2026-02-10 华中科技大学 一种脑卒中病灶检测方法及系统

Also Published As

Publication number Publication date
CN113723573A (zh) 2021-11-30
CN113723573B (zh) 2022-01-14

Similar Documents

Publication Publication Date Title
CN113723573B (zh) 基于自适应比例学习的肿瘤组织病理分类系统及方法
CN111986150B (zh) 一种数字病理图像的交互式标注精细化方法
Tosta et al. Segmentation methods of H&E-stained histological images of lymphoma: a review
US8077958B2 (en) Computer-aided pathological diagnosis system
CN103020585B (zh) 一种免疫组织阳性细胞和阴性细胞识别方法
CN110648322B (zh) 一种子宫颈异常细胞检测方法及系统
Jadhav et al. Soybean leaf disease detection and severity measurement using multiclass SVM and KNN classifier
CN102096804A (zh) 骨扫描中肿瘤骨转移的图像识别方法
CN103020639A (zh) 一种白细胞自动识别计数方法
CN108734108B (zh) 一种基于ssd网络的裂纹舌识别方法
CN109147932B (zh) 癌细胞her2基因扩增分析方法及系统
Pang et al. Automatic segmentation of crop leaf spot disease images by integrating local threshold and seeded region growing
CN109033936A (zh) 一种宫颈脱落细胞核图像识别方法
Apou et al. Detection of lobular structures in normal breast tissue
CN113096184A (zh) 一种复杂背景下硅藻定位与识别方法
CN112990015A (zh) 一种病变细胞自动识别方法、装置和电子设备
CN115359031A (zh) 一种数字病理图像制片质量评价方法
CN111429461A (zh) 一种新型的重叠脱落上皮细胞分割方法
CN117496276B (zh) 肺癌细胞形态学分析、识别方法及计算机可读存储介质
CN110853030B (zh) 生物反应器病毒感染细胞质量评价方法
CN108664979A (zh) 基于图像识别的玉米叶部病虫害检测模型的构建方法及应用
US12394225B1 (en) Automatic recognition method and system of pathology image based on deep learning
CN110051384B (zh) 结合医学统计信息的胎儿颈背透明物位置检测方法及系统
Grandinetti Towards 3D reconstruction of neuronal cell distributions from histological human brain sections
CN115187852A (zh) 一种藏医尿诊悬浮物识别方法及装置

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22889067

Country of ref document: EP

Kind code of ref document: A1