CN113222903A - Full-section histopathology image analysis method and system - Google Patents

Full-section histopathology image analysis method and system Download PDF

Info

Publication number
CN113222903A
CN113222903A CN202110425450.5A CN202110425450A CN113222903A CN 113222903 A CN113222903 A CN 113222903A CN 202110425450 A CN202110425450 A CN 202110425450A CN 113222903 A CN113222903 A CN 113222903A
Authority
CN
China
Prior art keywords
image
data
full
model
slice
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110425450.5A
Other languages
Chinese (zh)
Inventor
郑元杰
姜岩芸
隋晓丹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Normal University
Original Assignee
Shandong Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Normal University filed Critical Shandong Normal University
Priority to CN202110425450.5A priority Critical patent/CN113222903A/en
Publication of CN113222903A publication Critical patent/CN113222903A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a method and a system for analyzing a full-slice histopathology image, which are used for acquiring a full-slice digital pathology image; amplifying the full-slice digital pathological image and cutting the full-slice digital pathological image into image blocks which can be used for analysis; removing the background image block without tissues and cells; and analyzing the residual image blocks by using a semi-supervised deep learning model, and displaying the information obtained by analysis in the original full-slice pathological image. The model established by the invention can realize the analysis and quantification of the full-section histopathology image, realize the model interaction through online learning, and improve the analysis performance.

Description

Full-section histopathology image analysis method and system
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a method and a system for analyzing a full-section histopathology image.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
Classification of cancer subtypes by pathological examination is a key process in diagnosing malignancies and selecting treatments. In recent years, a whole slide imaging technique (WSI) has been used, which scans a histopathology slide into whole-slice digital pathology images (WSIs) and analyzes them by computer-aided diagnosis (CAD). Despite the considerable research and development in microscopy and pathology analysis, quantification of information contained in a full-section pathology image is very difficult due to the large scale of the entire full-section pathology image, the fact that the analysis is very dependent on the clinical experience of an experienced observer, and the like. Furthermore, manually analyzing WSI is very time consuming.
Since the advent of WSI technology, automated tumor subtype classification has been an active research topic. CAD-based pathological image analysis is becoming more and more common in the clinical evaluation of WSI. Recently, deep learning approaches for clinical analysis and research of WSI have made significant progress, and large-scale data collection and analysis further reveal spatial information shared between different cancers. However, deep learning based approaches typically require large amounts of annotation data to train the model, which makes the model still require very expensive annotation images when training the model or migrating to a new medical task. Obviously, the acquisition of the annotation data corresponding to the histopathological image is very difficult. WSIs are large in size, require experienced pathologists to use special labeling tools, and take a lot of time and cost to annotate.
In summary, the existing deep learning technology is used for analyzing the histopathological image, and an effective solution is not yet available.
Disclosure of Invention
The invention aims to solve the problems and provides a method and a system for analyzing the full-slice histopathology image.
According to some embodiments, the invention adopts the following technical scheme:
a full-slice histopathological image analysis method comprises the following steps:
acquiring a full-slice digital pathological image;
amplifying the full-slice digital pathological image and cutting the full-slice digital pathological image into image blocks which can be used for analysis;
removing the background image block without tissues and cells;
and analyzing the residual image blocks by using a semi-supervised deep learning model, and displaying the information obtained by analysis in the original full-slice pathological image.
As an alternative embodiment, the specific process of analyzing the remaining image blocks by using the semi-supervised deep learning model includes:
dividing the full-slice pathological image block into labeled data and unlabeled data;
carrying out convolution operation on input image data by using an image classification model based on a convolution neural network to obtain class probability corresponding to an image;
calculating a loss function for different image data to optimally train the image classification model;
and carrying out image analysis by using the optimally trained model.
As a further limitation, for data with labels, after the data enters an image classification model, features are extracted by the image classification model to obtain predicted class distribution, and the consistency of the predicted class distribution and an original single label is constrained by using a cross entropy loss function.
As a further limitation, unlabeled data is transformed by data augmentation, and then the model is constrained using a consistency regularization method to keep model predictions unchanged before and after data transformation.
As a further limitation, for unlabeled data, a model under training is used to obtain a virtual label from unlabeled data, which serves as an artificial label for the enhanced unlabeled data.
As a further limitation, the image of the unmarked data after weak supervision is input into a classification model to predict class distribution, and when the predicted value of the maximum prediction type in class prediction is larger than a set threshold value, the prediction is retained, and a pseudo label in the form of a single label is generated.
By way of further limitation, the loss function includes an unlabeled data loss and a labeled data loss, wherein: for tagged data, a given set of data χ { (x)b，pb): b ∈ 1.. and B, each image xb corresponds to a single label pb, and the loss function is defined as:
Figure BDA0003029518890000041
CE (|) refers to the original label p of the databAnd class probability distribution p of model to input predictionsmodel(y | α (x)) which is a weakly enhanced image to the original tagged image x;
for unlabeled data, a set of data is given
Figure BDA0003029518890000045
Image ubWithout corresponding artificial labels, for data after data enhancement, the model predicts the class probability distribution q of the datab＝pmodel(y|α(ub) Using an average label or a pseudo label to obtain a virtual label corresponding to the image, wherein a loss function is defined as:
Figure BDA0003029518890000042
or
Figure BDA0003029518890000043
The overall loss function is defined as:
Figure BDA0003029518890000044
a full-slice histopathological image analysis system, comprising:
an image acquisition module configured to acquire a full-slice digital pathology image;
the image preprocessing module is configured to read the full-slice digital pathological image, export the image at a certain magnification, divide the image into image blocks which can be used for analysis and remove background image blocks without tissues and cells;
and the image analysis module is configured to analyze the residual image blocks by using a semi-supervised deep learning model and display the analyzed information in the original full-slice pathological image.
An electronic device comprising a memory and a processor, and computer instructions stored on the memory and executed on the processor, wherein the computer instructions, when executed by the processor, perform the steps of a method for full-slice histopathological image analysis as described above.
A computer readable storage medium for storing computer instructions which, when executed by a processor, perform the steps of a method for full-slice histopathological image analysis as described above.
Compared with the prior art, the invention has the beneficial effects that:
the invention uses the strategy of generating artificial labels through consistency regularization and virtual labels in a semi-supervision method for reference, and trains a deep learning classification model through a small amount of labeled images and a large amount of unlabeled images together. For unlabeled images, the estimation of weakly enhanced images can be used to generate artificial labels to be used as virtual labels for strongly enhanced images. By the method, the model can be trained only by part of the labeled images, the model performance of the model trained with a large number of labeled images is basically achieved, the requirements of the deep learning method on image labels are reduced, and the model can realize the analysis precision similar to that of a fully supervised model which is trained with a large number of labeled data relatively.
The invention utilizes the semi-supervision method to make the model easier to be transferred and applied to different types of histopathology image analysis, and the transfer learning cost is low and the implementation is easy. The method comprises offline learning and online learning, the model for completing the offline learning can realize interactive online learning on new data, and the method has better applicability and expansibility.
The method is based on a deep learning model and a simple and common deep learning classification framework. The deep learning model test process only passes through a convolution forward propagation process once, the analysis of image data can be realized, the calculation complexity is low, the analysis result of the full-section histopathology image can be obtained in a short time, and the operation speed is high.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
FIG. 1 is a flow of full-slice histopathology image analysis based on semi-supervised deep learning;
FIG. 2 is a semi-supervised deep learning image classification model based on consistency regularization and virtual labels according to the present embodiment;
FIG. 3 is an example diagram of a full-slice digital pathology;
fig. 4 is an example diagram of a pathological image block;
fig. 5 is a data enhancement result presentation diagram of a tissue pathology image block;
FIG. 6 is a diagram of a model of a semi-supervised method based on mean label;
FIG. 7 is a diagram of a pseudo label-based semi-supervised method model.
The specific implementation mode is as follows:
the invention is further described with reference to the following figures and examples.
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
A full-slice histopathological image analysis process based on semi-supervised deep learning, as shown in fig. 1, which is explained by taking the method applied to the histopathological image analysis of breast cancer as an example, the method includes the following steps:
step S1: a full-scale digital pathology image is acquired.
In particular, the data may be used with digital pathology images scanned after H & E staining, as well as with public data sets.
H & E staining pathological section preparation: paraffin fixation, slicing, sticking, H & E staining, sealing and the like to obtain pathological sections, and then scanning the pathological sections into full-size digital pathological images (WSI) by using a scanner. In the embodiment, paraffin fixation, slicing, pasting, dyeing, sealing and the like are carried out in a conventional manner without limitation; the specific manner in which the scanner obtains a full-size digital pathology image is not limited.
Disclosing a pathological section data set: the Cancer Genome Atlas (TCGA), a Cancer gene map dataset containing imaging data, pathology data, gene sequencing, survival information, and The like.
Step S2: the full-size digital pathology image is derived according to a certain magnification (usually 40 times magnification image or 20 times magnification image) and is cut into image blocks which can be used for analysis.
Specifically, an OpenSlide toolkit is introduced into the Python, and is read by using an OpenSlide to determine a down-sampling factor; setting the size of the cut block and whether the cut block is overlapped (the full-size pathological image is cut into 512 multiplied by 512 non-overlapped image blocks in the invention); and storing the cut image blocks in corresponding row and column names.
Step S3: background image patches without tissue and cells were removed.
Specifically, setting an image threshold, and distinguishing each pixel in the image into foreground and background pixels, wherein image blocks with more than or equal to 50% of foreground pixels are defined as foreground image blocks for subsequent analysis; image blocks with less than 50% of the foreground pixels are defined as background image blocks and are not used in subsequent computational analysis.
Step S4: image blocks from the full-slice pathology image are analyzed using a semi-supervised depth learning model.
Specifically, a specific model analysis task is formulated according to a specific task and a data label. The data is divided into labeled images and unlabeled images, and a consistency regularization strategy and a virtual label strategy are used.
Inputting a model, wherein input data come from a full-slice pathological image block and are divided into labeled data and unlabeled data;
a classification model, which is an image classification model based on a Convolutional Neural Network (CNN), and is input as an image, and the classification model is subjected to convolution operation to obtain a class probability corresponding to the image;
for the classification problem, given a set of data with a single label, a set of unlabeled data without a label. And (5) classifying the models. And for the data with the labels, after the data enters the model, extracting features from the model to obtain the predicted class distribution. The consistency of the predicted class distribution with the original single label is constrained using a cross entropy loss function. In addition, the data expansion of the tagged data is expected to obtain the expected class distribution. For unlabeled data, a model under training is used to somehow obtain virtual labels from unlabeled data, which can be used as artificial labels for the enhanced unlabeled data.
Note that data augmentation is a core idea of semi-supervised approaches. Unlabeled data is transformed by data augmentation, and then a model is constrained using a consistency regularization method to keep model prediction unchanged before and after data transformation.
Consistency regularization is a common method of training deep models. It relies on data enhancement techniques, which means that when a perturbed image of the same image is input, the model should correspond to a similar distribution of the prediction results. This consistency regularization method has been applied to SSL methods and has become an important component of the latest SSL technologies. The consistency regularization applied to unlabeled data is based on the following assumptions: when the input image is disturbed, the output of the model remains unchanged. This type of model uses unlabeled data to train the model through standard supervised classification loss L2-norm loss:
Figure BDA0003029518890000091
wherein, α (u)b) For the image after random enhancement, class distribution p is predicted through a modelmodel(y|α(ub)). Because alpha (#) is randomly enhanced, the two enhanced images in the formula are different, so that the predicted class distribution is different, and the similarity of effective features can be captured by the model from the images through consistency constraint.
The virtual tag borrows the predicted output of the model unpredicted data as an artificial tag, executing the constraint of consistency regularization. Virtual tagging provides a simpler and more effective strategy, which has proven in practice to significantly improve results.
In the method MixMatch, this problem is solved by increasing the average of the K model predictions. The method uses a classification model to respectively predict K random data expansions of unmarked data to obtain K class distribution predictions, and then averages the results of the K predictions
Figure BDA0003029518890000105
And generating a virtual label through sharpening.
Figure BDA0003029518890000101
Figure BDA0003029518890000102
Wherein the content of the first and second substances,
Figure BDA0003029518890000103
is the average label of K predictions. Sharpening function Sharpern (p, T)iThe entropy distribution of the prediction label is reduced, and the sharpening result is similar to a single label. The virtual label can be used as a manual label for model training.
In addition, a pseudo label strategy can be used for simplifying consistency regularization, images of unlabeled data after weak supervision are input into a classification model, class distribution is predicted, and when the predicted value of the maximum prediction type in class prediction is larger than a certain threshold value, the prediction is retained, and a pseudo label in a single label form is generated. Suppose qb＝pmodel(y|α(ub) Is given a random data extension alpha (u)b) And (4) obtaining class prediction through a classification model. Then, use
Figure BDA0003029518890000104
The maximum of the prediction distribution of the model is retained as a pseudo label, which is defined as:
Figure BDA0003029518890000111
wherein, 1(, is an index function, and means that when the maximum value of the prediction probability distribution is greater than the hyper-parameter β, the term constraint is established, and the prediction category distribution is generated by a pseudo label constraint model in the form of a single label. CE (|) means two probability distributions
Figure BDA0003029518890000112
And q isbCross entropy between. Using pseudo labels instead of class probability distributions, cross-entropy constrained model training can be used.
It should be noted that data augmentation is the basis for consistency regularization and virtual tagging. Various data expansion methods have been proposed in the existing literature, which can be simply divided into three categories: a simple enhancement method, a region level enhancement method and an automatic enhancement method. Most amplification methods are based on the first category: geometric transformations, such as flipping, clipping, affine transformations; pixel level content transitions such as inversion, noise, blur, sharpness, contrast interference. Several region-level enhancement methods, such as Cutout, randomly occlude or modify pixel values in an N-sized region of an image, thereby improving model performance using regularization. In addition, automatic augmentation, Fast automation, randautoment, etc. methods of automatic augmentation generate novel image data from the original dataset by training a subnet to search to select appropriate augmentation parameters or adjusting the augmentation parameters according to model training and the size of the dataset.
In the present invention, we use a method similar to RandAugment. An image processing conversion library is defined based on a Python Image Library (PIL), and contains K conversions, such as flipping, cropping, affine conversion, noise, blurring, and the like. Then, one operation in the library is randomly selected each time data expansion is performed, and the operation is performed N times.
A loss function, the total loss function comprising an unlabeled data loss and a labeled data loss.
For tagged data, a given set of data χ { (x)b，pb): b ∈ (1,..., B) }, each image xbCorresponding to a single tag pbThe loss function is defined as:
Figure BDA0003029518890000121
CE (|) refers to the original label p of the databAnd class probability distribution p of model to input predictionsmodel(y | α (x)) which is a weakly enhanced image to the original tagged image x.
For unlabeled data, a set of data is given
Figure BDA0003029518890000125
Image ubThere is no corresponding manual label. First, for data enhanced data, the model predicts the class probability distribution q of the datab＝pmodel(y|α(ub) ) then makeAnd obtaining a virtual label corresponding to the image in an average label or pseudo label mode. The loss function is defined as:
Figure BDA0003029518890000122
or
Figure BDA0003029518890000123
The overall loss function is defined as:
Figure BDA0003029518890000124
step S5: and displaying the analysis result in the original full-slice pathological image to obtain a quantitative result.
And predicting each image block category by using a depth learning model, wherein the row and column numbers of the image blocks can correspond to the original full-slice image, and the classification labels are displayed in the image.
A full-slice histopathological image analysis system, comprising:
an image acquisition module configured to acquire a full-slice digital pathology image;
the image preprocessing module is configured to amplify the full-slice digital pathological image, cut the full-slice digital pathological image into image blocks which can be used for analysis, and remove background image blocks without tissues and cells;
and the image analysis module is configured to analyze the residual image blocks by using a semi-supervised deep learning model and display the analyzed information in the original full-slice pathological image.
The system also provides the following functions:
off-line learning, wherein model training is completed on an original data set, and the original data set comprises tag data and non-tag data;
setting a data path, a pre-training model, a model storage path and the like, setting types of conversion in data expansion, virtual label strategies, hyper-parameters, and training parameters such as initialization, deviation, regularization, initial learning rate, learning rate reduction mode, optimization algorithm, iteration times and the like, and realizing the training of a semi-supervised deep learning model.
And in the stage of online learning and model test use, new data is used for fine tuning of the model.
The model testing process is similar to the training process, setting up input images, using models, etc. In one embodiment, the inputs include a test folder path, a test model, a test image, a test result output path.
In particular, model tuning may be implemented on test data. If the test data contains simple labels, part of the image is marked with label labels after the cutting block is led out. The model performs similar fine-tuning as the training process. And finally, displaying a test result, and displaying the analysis result of the full-section pathological image generated by the model.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive efforts by those skilled in the art based on the technical solution of the present invention.

Claims (10)

1. A full-section histopathology image analysis method is characterized by comprising the following steps: the method comprises the following steps:
acquiring a full-slice digital pathological image;
amplifying the full-slice digital pathological image and cutting the full-slice digital pathological image into image blocks which can be used for analysis;
removing the background image block without tissues and cells;
and analyzing the residual image blocks by using a semi-supervised deep learning model, and displaying the information obtained by analysis in the original full-slice pathological image.
2. The method of analyzing full-section histopathological image according to claim 1, wherein: the specific process for analyzing the residual image blocks by using the semi-supervised deep learning model comprises the following steps:
dividing the full-slice pathological image block into labeled data and unlabeled data;
carrying out convolution operation on input image data by using an image classification model based on a convolution neural network to obtain class probability corresponding to an image;
calculating a loss function for different image data to optimally train the image classification model;
and carrying out image analysis by using the optimally trained model.
3. The method of analyzing full-section histopathological image according to claim 2, wherein: for data with labels, after the data enters an image classification model, extracting features by the image classification model to obtain predicted class distribution, and constraining the consistency of the predicted class distribution and an original single label by using a cross entropy loss function.
4. The method of analyzing full-section histopathological image according to claim 2, wherein: unlabeled data is transformed by data augmentation, and then a model is constrained using a consistency regularization method to keep model prediction unchanged before and after data transformation.
5. The method of analyzing full-section histopathological image according to claim 2, wherein: for unlabeled data, a virtual label is obtained from unlabeled data by using a model in training, and the virtual label is used as an artificial label of the enhanced unlabeled data.
6. The method of analyzing full-section histopathological image according to claim 2, wherein: inputting the image of unmarked data after weak supervision into a classification model, predicting class distribution, and when the predicted value of the maximum prediction type in class prediction is larger than a set threshold value, reserving the prediction to generate a pseudo label in a single label form.
7. The method of analyzing full-section histopathological image according to claim 2, wherein: the loss function includes an unlabeled data loss and a labeled data loss, wherein: for tagged data, a given set of data χ { (x)b，pb): b ∈ (1,..., B) }, each image xbCorresponding to a single tag pbThe loss function is defined as:
Figure FDA0003029518880000021
CE (|) refers to the original label p of the databAnd class probability distribution p of model to input predictionsmodel(y | α (x)) which is a weakly enhanced image to the original tagged image x;
for unlabeled data, a given set of data u ═ ub: b ∈ (1,..., B) }, image ubWithout corresponding artificial labels, for data after data enhancement, the model predicts the class probability distribution q of the datab＝pmodel(y|α(ub) Using an average label or a pseudo label to obtain a virtual label corresponding to the image, wherein a loss function is defined as:
Figure FDA0003029518880000031
or
Figure FDA0003029518880000032
The overall loss function is defined as:
Figure FDA0003029518880000033
8. a full-section histopathology image analysis system is characterized in that: the method comprises the following steps:
an image acquisition module configured to acquire a full-slice digital pathology image;
the image preprocessing module is configured to amplify the full-slice digital pathological image, cut the full-slice digital pathological image into image blocks which can be used for analysis, and remove background image blocks without tissues and cells;
and the image analysis module is configured to analyze the residual image blocks by using a semi-supervised deep learning model and display the analyzed information in the original full-slice pathological image.
9. An electronic device, characterized by: comprising a memory and a processor and computer instructions stored on the memory and executed on the processor, which when executed by the processor, perform the steps of a method of full-slice histopathological image analysis according to any of claims 1-7.
10. A computer-readable storage medium characterized by: for storing computer instructions which, when executed by a processor, perform the steps of a method of full-slice histopathological image analysis according to any one of claims 1-7.
CN202110425450.5A 2021-04-20 2021-04-20 Full-section histopathology image analysis method and system Pending CN113222903A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110425450.5A CN113222903A (en) 2021-04-20 2021-04-20 Full-section histopathology image analysis method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110425450.5A CN113222903A (en) 2021-04-20 2021-04-20 Full-section histopathology image analysis method and system

Publications (1)

Publication Number Publication Date
CN113222903A true CN113222903A (en) 2021-08-06

Family

ID=77088023

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110425450.5A Pending CN113222903A (en) 2021-04-20 2021-04-20 Full-section histopathology image analysis method and system

Country Status (1)

Country Link
CN (1) CN113222903A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113764073A (en) * 2021-09-02 2021-12-07 宁波权智科技有限公司 Medical image analysis method and device
WO2023124605A1 (en) * 2021-12-28 2023-07-06 International Business Machines Corporation Three-dimensional delineation of tumor boundaries via supervised machine learning algorithms and augmented reality

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110222690A (en) * 2019-04-29 2019-09-10 浙江大学 A kind of unsupervised domain adaptation semantic segmentation method multiplying loss based on maximum two
CN111832627A (en) * 2020-06-19 2020-10-27 华中科技大学 Image classification model training method, classification method and system for suppressing label noise
CN112036335A (en) * 2020-09-03 2020-12-04 南京农业大学 Deconvolution-guided semi-supervised plant leaf disease identification and segmentation method
CN112435243A (en) * 2020-11-26 2021-03-02 山东第一医科大学附属省立医院(山东省立医院) Automatic analysis system and method for full-slice digital pathological image

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110222690A (en) * 2019-04-29 2019-09-10 浙江大学 A kind of unsupervised domain adaptation semantic segmentation method multiplying loss based on maximum two
CN111832627A (en) * 2020-06-19 2020-10-27 华中科技大学 Image classification model training method, classification method and system for suppressing label noise
CN112036335A (en) * 2020-09-03 2020-12-04 南京农业大学 Deconvolution-guided semi-supervised plant leaf disease identification and segmentation method
CN112435243A (en) * 2020-11-26 2021-03-02 山东第一医科大学附属省立医院(山东省立医院) Automatic analysis system and method for full-slice digital pathological image

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
DAVID BERTHELOT ET AL.: ""MixMatch: A Holistic Approach to"", 《33RD CONFERENCE ON NEURAL INFORMATION PROCESSING SYSTEMS (NEURIPS 2019)》 *
KIHYUK SOHN ET AL.: ""FixMatch: Simplifying Semi-Supervised Learning"", 《34TH CONFERENCE ON NEURAL INFORMATION PROCESSING SYSTEMS (NEURIPS 2020)》 *
林成创 等: ""机器视觉应用中的图像数据增广综述"", 《计算机科学与探索》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113764073A (en) * 2021-09-02 2021-12-07 宁波权智科技有限公司 Medical image analysis method and device
WO2023124605A1 (en) * 2021-12-28 2023-07-06 International Business Machines Corporation Three-dimensional delineation of tumor boundaries via supervised machine learning algorithms and augmented reality

Similar Documents

Publication Publication Date Title
Priego-Torres et al. Automatic segmentation of whole-slide H&E stained breast histopathology images using a deep convolutional neural network architecture
Wan et al. Robust nuclei segmentation in histopathology using ASPPU-Net and boundary refinement
CN112016682B (en) Video characterization learning and pre-training method and device, electronic equipment and storage medium
İnik et al. A new method for automatic counting of ovarian follicles on whole slide histological images based on convolutional neural network
US11935213B2 (en) Laparoscopic image smoke removal method based on generative adversarial network
CN112687327A (en) Cancer survival analysis system based on multitask and multi-mode
CN113222903A (en) Full-section histopathology image analysis method and system
Gadermayr et al. Multiple instance learning for digital pathology: A review of the state-of-the-art, limitations & future potential
CN113421240B (en) Mammary gland classification method and device based on ultrasonic automatic mammary gland full-volume imaging
Nguyen et al. Breast cancer histology image classification using deep learning
CN115601330A (en) Colonic polyp segmentation method based on multi-scale space reverse attention mechanism
Ding et al. Two-stage deep neural network via ensemble learning for melanoma classification
CN115409832A (en) Triple negative breast cancer classification method based on ultrasound image and omics big data
Kromp et al. Deep Learning architectures for generalized immunofluorescence based nuclear image segmentation
CN113724267B (en) Breast ultrasonic image tumor segmentation method and device
Xu et al. Histopathological tissue segmentation of lung cancer with bilinear cnn and soft attention
Bonechi Isic_Wsm: Generating weak segmentation maps for the ISIC archive
CN114037699A (en) Pathological image classification method, equipment, system and storage medium
CN117437423A (en) Weak supervision medical image segmentation method and device based on SAM collaborative learning and cross-layer feature aggregation enhancement
Gupta et al. Iterative learning to make the most of unlabeled and quickly obtained labeled data in histology
CN116469103A (en) Automatic labeling method for medical image segmentation data
CN113591791B (en) Lung cancer automatic identification system based on self-learning artificial intelligence
Dzyubachyk et al. Model-based approach for tracking embryogenesis in Caenorhabditis elegans fluorescence microscopy data
Vijayalakshmi et al. Liver tumor detection using CNN
CN114283406A (en) Cell image recognition method, device, equipment, medium and computer program product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination