CN115131565A - Histology image segmentation model based on semi-supervised learning - Google Patents
Histology image segmentation model based on semi-supervised learning Download PDFInfo
- Publication number
- CN115131565A CN115131565A CN202210858624.1A CN202210858624A CN115131565A CN 115131565 A CN115131565 A CN 115131565A CN 202210858624 A CN202210858624 A CN 202210858624A CN 115131565 A CN115131565 A CN 115131565A
- Authority
- CN
- China
- Prior art keywords
- model
- loss function
- level
- teacher
- segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V10/7753—Incorporation of unlabelled data, e.g. multiple instance learning [MIL]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/776—Validation; Performance evaluation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a histology image segmentation model based on semi-supervised learning, which comprises the following steps: the system comprises a teacher model, a student model, a multi-level mandatory consistency module and a total loss function for supervised training; the student model is trained through marked data and unmarked data, the teacher model is trained through unmarked data, and the multi-level forced consistency module adopts a multi-level consistency loss function to carry out consistency constraint on a segmentation prediction result of the teacher model and a segmentation prediction result of a variant of multi-level potential representation of an encoder in the student model when the student model and the teacher model are trained through the unmarked data; the histological image segmentation model is effective, a multi-level forced consistency module and a multi-level consistency loss function are provided, and prediction invariance of a model segmentation prediction result is enhanced by adding disturbance to multi-level potential representation of the model.
Description
Technical Field
The invention belongs to the technical field of image segmentation, and particularly relates to a histology image segmentation model based on semi-supervised learning.
Background
Accurate segmentation of cells and glands using histological images is an essential but challenging task in computer-aided diagnosis. The method of histological image segmentation by deep learning techniques with the help of a large amount of labeled data achieves advanced performance [1 ]. However, there is a challenging problem in the field of histological image analysis, namely that the performance improvement of the deep learning model requires a large amount of high-quality and well-labeled data support. However, unlike natural images, the annotation of medical images requires the involvement of experts with domain knowledge, and well-annotated data acquisition is a time-consuming and labor-intensive task.
In recent years, to solve the problem of labeling difficulty, more and more studies have been devoted to medical image segmentation by a limited amount of labeled data and a large amount of unlabeled data using semi-supervised learning techniques [2,4,5 ]. However, how to promote consistency between labeled data and unlabeled data poses a great challenge to the development of semi-supervised learning. Although current research focuses on formulating perturbations to consistently train labeled data with unlabeled data [2,4,5], existing consistency training methods focus primarily on formulating perturbations that apply to the input space and the advanced feature space, and neglect formulating perturbations in the hierarchical potential feature space of the deep network architecture. Also, in the Mean-Teacher architecture, which is commonly used for consistency training methods, Teacher models are often used to generate training targets for student models. However, it is difficult to determine whether a Teacher model performs better than a student model during the training process, and a low performance Teacher model presents a serious challenge for the training of the Mean-Teacher architecture.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide a histology image segmentation model based on semi-supervised learning, which solves the problems of difficult labeling in the field of computer-aided diagnosis and interference of a low-performance Teacher model in a Mean-Teacher architecture with model training.
The purpose of the invention is realized by the following technical scheme.
A semi-supervised learning based histological image segmentation model, comprising: the system comprises a teacher model, a student model, a multi-level mandatory consistency module and a total loss function for supervised training;
the teacher model and the student model are identical in structure, the teacher model and the student model both adopt DeepLabV3+ with hole convolution, and the DeepLabV3+ is composed of an encoder and a main decoder, wherein the encoder comprises a convolution block CB and four residual blocks, the four residual blocks are RB1, RB2, RB3 and RB4 respectively, each residual block is a pre-trained ResNet34, and RB3 and RB4 use hole convolution;
the student model is trained through marked data and unmarked data, the teacher model is trained through unmarked data, and the multi-level forced consistency module adopts a multi-level consistency loss function to carry out consistency constraint on a segmentation prediction result of the teacher model and a segmentation prediction result of a variant of multi-level potential representation of an encoder in the student model when the student model and the teacher model are trained through the unmarked data; the method for obtaining the segmentation prediction result of the multi-level potential representation variant of the encoder in the student model comprises the following steps: the output of each residual block of RB2, RB3, and RB4 in the student model potentially represents z h Obtaining variants by perturbation operations
Then generating a segmentation prediction result by an auxiliary decoder;
for annotated data, the student model passes through a supervised loss function L seg Training is carried out;
the multi-level consistency loss function comprises a learnable multi-level loss function L lh_c And self-directed multi-level consistency loss function L sgh_c ；
Total loss function L total The concrete expression is as follows:
L total ＝L seg +λ h_c (L lh_c +λ sgh_c L sgh_c )
wherein λ is sgh_c Is L sgh_c Coefficient of (a) sgh_c ＝0～1，λ h_c Expressed as:
wherein q is a scaling coefficient, v is equal to the current iteration number of training, and T is equal to the total iteration number of training.
In the above technical solution, the update strategy of the weights in the teacher model is as follows: in each training batch, the weight updating of the teacher model is based on the weight of the teacher model in the last training batch and the weight of the student model in the training batch, and the updating strategy is as follows: weight θ 'of teacher model in t-th training batch' t Comprises the following steps:
θ′ t ＝αθ′ t-1 +(1-α)θ t
wherein, theta' t-1 Represents the weight, θ, of the teacher model in the t-1 th training batch t Represents the weight of the student model in the t-th training batch, and alpha represents the updating of the student model theta by gradient descent in the total training process t Is measured by the exponential moving average of (d).
In the above technical scheme, there is a supervisory loss function L seg By a cross entropy loss function L ce Sum variance constrained cross-loss function L var Composition of a supervised loss function L seg Is represented as follows:
L seg ＝L ce +λ var L var
wherein L is ce Representing the cross entropy loss function, L var Represents a variance constrained cross-loss function, λ var Representing the weight of the variance constrained cross-loss function.
In the above technical solution, forLabeled data B for each training batch l Variance constrained cross-loss function L var As follows:
wherein D represents labeled data B of the training batch l Number of split instances, B d Represents all pixels, | B, contained in the d-th segmentation instance in the training batch d I represents B d Number of pixels in (1), p j Is represented by B d The prediction probability that the jth pixel in (j) is in the correct class, j 1 … … | B d |，u d Is shown in B d The average of the prediction probabilities for all pixels in the array.
In the above technical solution, the perturbation operation mode is random, specifically dropout or a noise layer using a feature level.
In the above technical solution, all the auxiliary decoders have the same structure, and include a hollow space convolution pooling pyramid layer and an upsampling layer.
In the above technical solution, the learnable multi-level loss function L lh_c Specifically, the following are shown:
wherein, B u Represents any one training batch with no labeled data, | B u I represents B u Number of pixels in, L mse Represents the mean square error function, H represents the number of levels in the multilevel mandatory consistency module,
representing the segmented prediction result of the k-th pixel by the student model main decoder,
represents the h auxiliary decoder pair of the student modelThe prediction result of the segmentation of k pixels, H1 … … H,
representing the learnable predicted probability of the teacher model for the kth pixel,
expressed as:
wherein u' k Can be expressed as:
In the above technical solution, a self-directed multi-level consistency loss function L sgh_c Specifically, the following are shown:
the invention has the following beneficial effects:
(1) the invention provides a histology image segmentation model based on semi-supervised learning, and results of two embodiments show the effectiveness of the histology image segmentation model;
(2) the invention provides a multi-level forced consistency module and a multi-level consistency loss function, and the prediction invariance of a model segmentation prediction result is enhanced by adding disturbance to multi-level potential representation of a model.
Drawings
FIG. 1 is a block diagram of a histological image segmentation model based on semi-supervised learning according to the present invention;
fig. 2 shows the experimental results after training.
Detailed Description
The technical scheme of the invention is further explained by combining specific examples.
Example 1
A semi-supervised learning based histological image segmentation model, comprising: the teacher model, the student model, the multi-level mandatory consistency (hierarchy mandatory consistency) module and the overall loss function used for supervising training, the teacher model and the student model have the same structure, the teacher model and the student model both use DeepLabV3+ [8] with cavity convolution, DeepLabV3+ is composed of an encoder and a main decoder (g), wherein, the encoder comprises a convolution block CB and four residual blocks, the four residual blocks are RB1, RB2, RB3 and RB4 respectively, each residual block is a pretrained ResNet34, RB3 and RB4 use convolution cavities, the expansion parameters of the cavity convolution are set to be 2 and 4 respectively, and the parameters between the teacher model and the student model are independent;
the update strategy of the weights in the teacher model is as follows: in each training batch, the weight updating of the teacher model is based on the weight of the teacher model in the last training batch and the weight of the student model in the training batch, and the updating strategy is as follows: weight θ 'of teacher model in t-th training batch' t Comprises the following steps:
θ′ t ＝αθ′ t-1 +(1-α)θ t
wherein, theta' t-1 Represents the weight, θ, of the teacher model in the t-1 th training batch t Represents the weight of the student model in the t-th training batch, and alpha represents the updating of the student model alpha by gradient descent in the total training process t The attenuation ratio of the Exponential Moving Average (α) of (1) is 0.99 in this example.
The student model is trained by the presence of marked data and the absence of marked data, the teacher model is trained by the absence of marked data, and a multilevel consistency enforcement module adopts a multilevel consistency loss function to carry out consistency constraint on a segmentation prediction result of the teacher model and a segmentation prediction result of a multi-level potential representation variant of an encoder in the student model when the student model and the teacher model are trained by the absence of marked data.
For annotated data, the student model passes through a supervised loss function L seg Training is carried out with a supervised loss function L seg From a Cross Entropy Loss function (Cross Engine Loss) L ce Sum Variance Constrained Cross Loss function (Variance Constrained Cross Loss) L var Composition, expressed as follows:
L seg ＝L ce +λ var L var
wherein L is ce Representing a Cross entropy loss function [9 ]]，L var Represents a variance constrained cross-loss function, λ var Represents the weight of the variance constrained cross-loss function, in this embodiment, λ var ＝0.1。
Variance constrained cross-loss function L var [9]The local constraint is performed on pixels belonging to the same segmentation instance, so as to solve the problem that when the segmentation instance in the image has uneven color or texture, the model cannot completely segment the whole segmentation instance. Labeled data B for each training batch l Variance constrained cross-loss function L var As follows:
wherein D represents labeled data B of the training batch l Number of split instances, B d Represents all pixels, | B, contained in the d-th segmentation instance in the training batch d I represents B d Number of pixels in, p j Is represented by B d The prediction probability that the jth pixel in (j) is in the correct class, j 1 … … | B d |，u d Is shown in B d The average of the prediction probabilities for all pixels in the array.
The method for obtaining the segmentation prediction result of the multi-level potential representation variant of the encoder in the student model comprises the following steps: as shown in figure 1 of the drawings, in which,the output potential representation z of each residual block of RB2, RB3, and RB4 in the student model h (z in FIG. 1) 1 、z 2 And z 3 ) Obtaining variants by perturbation operations
(in FIG. 1 are
And
) Then goes through the auxiliary decoder (in FIG. 1, it is
And
) And generating a segmentation prediction result.
The perturbation operation is random in manner, specifically dropout or using a noise floor at the feature level [11 ].
All the auxiliary decoders have the same structure and include an empty space convolutional Pooling Pyramid (Spatial Pyramid) layer and an upsampling layer, wherein four sampling rates of the empty space convolutional Pooling Pyramid layer are respectively set to be 6, 8, 18 and 24.
A multilevel mandatory consistency implementation module can provide stronger constraint for student model training, so that generalization of a student network is promoted. Furthermore, to ensure that the student model has a greater generalization capability, the present invention does not impose constraints on the original potential representation of each level of the student model encoder, but rather imposes constraints on variations of the original potential representation of each level of the encoder.
The multi-level consistency loss function comprises a learnable multi-level loss function L lh_c And self-directed multi-level consistency loss function L sgh_c 。
To prevent teacher models from obtaining high uncertainty estimates and enhance level consistencyA learnable multi-level consistency loss function L is provided lh_c Learnable multi-level loss function L lh_c Specifically, the following are shown:
wherein, B u Represents any one training batch with no labeled data, | B u I represents B u Number of pixels in, L mse Represents a Mean Squared Error function (Mean Squared Error) for calculating the difference between the segmentation prediction results of the teacher model and the student model, H represents the number of levels in a multilevel mandatory consistency implementation (H takes 3 in this embodiment),
representing the segmented prediction result of the k-th pixel by the student model master decoder (g in figure 1),
representing the result of the student model's H-th auxiliary decoder's segmentation prediction on the k-th pixel, H-1 … … H,
the learnable prediction probability of the k pixel of the teacher model is represented, more reliable prediction can be provided for the student model to guide,
expressed as:
wherein u' k Can be expressed as:
As can be seen from equation (4), when the teacher model produces unreliable results (high uncertainty), the learnable prediction probability of the teacher model for the k-th pixel
Approximate to the prediction result of the student model main decoder to the student model main decoder
Conversely, when the teacher model has confidence in the prediction (low uncertainty), the learnable prediction probability of the teacher model for the kth pixel
And the teacher model predicts the results
The same, and provide certain prediction as the object of student model learning.
Multi-level consistency loss function L through self-guidance sgh_c To ensure that the output of a plurality of auxiliary decoders and the output of a main decoder of the student model are consistent, the specific expression is as follows:
as can be seen from the formula (7), through the constraint of the self-guided multi-level consistency loss function, the student model takes the prediction of the main decoder as the guide, and the inconsistency among all decoders is minimized, so that the feature representation capability of the student model is enhanced.
Total loss function L total The concrete expression is as follows:
L total ＝L seg +λ h_c (L lh_c +λ sgh_c L sgh_c )
wherein λ is sgh_c Is L sgh_c Coefficient of (a) sgh_c 0 to 1, in this example 0.1, λ h_c Expressed as:
where q is a scaling factor, in this embodiment q is 0.1, v is equal to the current iteration number of training, and T is equal to the total iteration number of training.
The learnable multi-level loss function can ensure that the segmentation prediction result of the student model can be consistent with the segmentation prediction result of the teacher model, and can prevent the teacher model from obtaining high-uncertainty prediction. The self-guided multi-level consistency loss function can ensure that the partition prediction result of the auxiliary branch decoder of the student model can be consistent with the partition prediction result of the main branch decoder, thereby enhancing the feature representation capability of the student model.
Respective monsseg [15] (multi-organ nuclear segmentation data set) and CRAG [16] (colorectal adenocarcinoma data set) were prepared as data sets, and image blocks were extracted in a sliding window manner from the original images in the data sets, and the size of each image block was set to 128 × 128 for monseg and 480 × 480 for CRAG.
The data set is divided into labeled data and unlabeled data without labels, wherein the labeled data are 5%, 10% and 20% of the data set, and the rest of the data in the data set are the unlabeled data.
And substituting the labeled data and the unlabeled data in the data set into a student model in the histological image segmentation model for training, and substituting the unlabeled data in the data set into a teacher model in the histological image segmentation model for training.
For MoNuSeg, the number of images in a training batch is set to be 16, the total number of training iterations is set to be 500, and for a CRAG data set, the number of images in a training batch is set to be 8, and the total number of training iterations is set to be 300. In order to prevent overfitting, online data enhancement is performed in the training process, and the enhancement modes comprise random scaling, overturning, rotating and affine transformation. The experimental results (HCE) of the present invention are shown in table 1 and fig. 2, and table 1 shows the performance comparison results of the present invention with the most advanced semi-supervised learning method on mourseg and CRAG.
TABLE 1
For MoNuSeg, the invention achieves the highest results on two evaluation indexes, namely Dice and AJI. For CRAG, the present invention achieved the highest effect on all evaluation indexes (F1, Dice, Haus). The invention enables the segmentation performance and the robustness of the model to be continuously improved by encouraging multi-level consistency training.
Fig. 2 shows the segmented prediction results of mounseg and CRAG under the condition of labeled data as 5% and 10% in data set by the present invention and other semi-supervised learning methods. From the result of the segmentation prediction, the invention has better expandability for segmentation examples with different shapes, such as small cells or large glands.
Example 2
This example is substantially the same as example 1, with the only difference that: there are labels for 50% of the data set and the data set for MoNuSeg. The results of the experiment are shown in Table 2. Table 2 shows the performance of the present invention compared to the fully supervised method on the monuserg dataset. The invention (HCE) achieves the best effect on two evaluation indexes of F1 and Dice.
TABLE 2
The invention has been described in an illustrative manner, and it is to be understood that any simple variations, modifications or other equivalent changes which can be made by one skilled in the art without departing from the spirit of the invention fall within the scope of the invention.
[1]Sahasrabudhe,M.,Christodoulidis,S.,Salgado,R.,Michiels,S.,Loi,S.,André,F., Paragios,N.,Vakalopoulou,M.:Self-supervised Nuclei Segmentation in Histopathological Images Using Attention.In:International Conference on Medical Image Computing and ComputerAssisted Intervention.pp.393–402.Springer(2020).
[2]Yu,L.,Wang,S.,Li,X.,Fu,C.W.,Heng,P.A.:Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation.In:International Conference on Medical Image Computing and Computer-Assisted Intervention.pp.605–613.Springer(2019).
[3].Graham,S.,Chen,H.,Gamper,J.,Dou,Q.,Heng,P.A.,Snead,D.,Tsang,Y.W., Rajpoot,N.:MILD-Net:Minimal information loss dilated network for gland instance segmentation in colon histology images.Medical Image Analysis 52,199–211(2019)
[4]Xie,Y.,Zhang,J.,Liao,Z.,Verjans,J.,Shen,C.,Xia,Y.:Pairwise Relation Learning for Semi-supervised Gland Segmentation.In:International Conference on Medical Image Computing and Computer-Assisted Intervention.pp.417–427.Springer(2020).
[5]Li,X.,Yu,L.,Chen,H.,Fu,C.W.,Xing,L.,Heng,P.A.:Transformation-consistent self-ensembling model for semi-supervised medical image segmentation.IEEE Transactions on Neural Networks and Learning Systems pp.1–12(2020).
[6].Li,X.,Yu,L.,Chen,H.,Fu,C.W.,Xing,L.,Heng,P.A.:Transformation-consistent selfensembling model for semisupervised medical image segmentation.IEEE Transactions on Neural Networks and Learning Systems pp.1–12(2020). https://doi.org/10.1109/TNNLS.2020.2995319
[7].Li,Y.,Chen,J.,Xie,X.,Ma,K.,Zheng,Y.:Self-loop uncertainty:A novel pseudo- label for semi-supervised medical image segmentation.In:International Conference on Medical Image Computing and Computer-Assisted Intervention.pp.614–623.Springer(2020)
[8]Chen,L.C.,Papandreou,G.,Kokkinos,I.,Murphy,K.,Yuille,A.L.:DeepLab:Semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence 40(4),834–848(2017).
[9]Qu,H.,Yan,Z.,Riedlinger,G.M.,De,S.,Metaxas,D.N.:Improving nuclei/gland instance segmentation in histopathology images by full resolution neural network and spatial constrained loss.In:International Conference on Medical Image Computing and ComputerAssisted Intervention.pp.378–386.Springer(2019).
[10]Raza,S.E.A.,Cheung,L.,Shaban,M.,Graham,S.,Epstein,D.,Pelengaris,S.,Khan, M.,Rajpoot,N.M.:Micro-Net:A unified model for segmentation of various objects in microscopy images.Medical Image Analysis 52,160–173(2019)
[11]Ouali,Y.,Hudelot,C.,Tami,M.:Semi-supervised semantic segmentation with cross- consistency training.In:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.pp.12674–12684(2020)
[12]Tarvainen,A.,Valpola,H.:Mean teachers are better role models:Weight-averaged consistency targets improve semi-supervised deep learning results.In:Advances in Neural Information Processing Systems.pp.1195–1204(2017)
[13]Verma,V.,Lamb,A.,Kannala,J.,Bengio,Y.,Lopez-Paz,D.:Interpolation Consistency Training for Semi-supervised Learning.In:Proceedings of the 28th International Joint Conference on Artificial Intelligence.pp.3635–3641.IJCAI’19,AAAI Press(2019)
[14]Vu,T.H.,Jain,H.,Bucher,M.,Cord,M.,Pérez,P.:ADVENT:Adversarial entropy minimization for domain adaptation in semantic segmentation.In:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.pp.2517–2526(2019)
[15]Kumar,N.,Verma,R.,Anand,D.,Zhou,Y.,Onder,O.F.,Tsougenis,E.,Chen,H., Heng,P.A.,Li,J.,Hu,Z.,et al.:A multi-organ nucleus segmentation challenge.IEEE Transactions on Medical Imaging 39(5),1380–1391(2019)
[16]Awan,R.,Sirinukunwattana,K.,Epstein,D.,Jefferyes,S.,Qidwai,U.,Aftab,Z., Mujeeb,I.,Snead,D.,Rajpoot,N.:Glandular morphometrics for objective grading of colorectal adenocarcinoma histology images.Scientific Reports 7(1),1–12(2017)
[17]Xiang,T.,Zhang,C.,Liu,D.,Song,Y.,Huang,H.,Cai,W.:BiO-Net:Learning Recurrent Bidirectional Connections for Encoder-Decoder Architecture.In:International Conference on Medical Image Computing and Computer-Assisted Intervention.pp.74–84. Springer(2020)
[18]Xie,Y.,Lu,H.,Zhang,J.,Shen,C.,Xia,Y.:Deep segmentation-emendation model for gland instance segmentation.In:International Conference on Medical Image Computing and Computer-Assisted Intervention.pp.469–477.Springer(2019)
[19]Xie,Y.,Zhang,J.,Liao,Z.,Verjans,J.,Shen,C.,Xia,Y.:Pairwise Relation Learning for Semi-supervised Gland Segmentation.In:International Conference on Medical Image Computing and Computer-Assisted Intervention.pp.417–427.Springer(2020)
[20]Yu,L.,Wang,S.,Li,X.,Fu,C.W.,Heng,P.A.:Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation.In:International Conference on Medical Image Computing and Computer-Assisted Intervention.pp.605–613.Springer(2019).
Claims (8)
1. A histology image segmentation model based on semi-supervised learning, comprising: the system comprises a teacher model, a student model, a multi-level mandatory consistency module and a total loss function for supervised training;
the teacher model and the student model are identical in structure, the teacher model and the student model both adopt DeepLabV3+ with hole convolution, and the DeepLabV3+ is composed of an encoder and a main decoder, wherein the encoder comprises a convolution block and four residual blocks, the four residual blocks are RB1, RB2, RB3 and RB4 respectively, each residual block is a pre-trained ResNet34, and RB3 and RB4 use hole convolution;
the student model is trained through marked data and unmarked data, the teacher model is trained through unmarked data, and the multi-level forced consistency module adopts a multi-level consistency loss function to carry out consistency constraint on a segmentation prediction result of the teacher model and a segmentation prediction result of a variant of multi-level potential representation of an encoder in the student model when the student model and the teacher model are trained through the unmarked data; the method for obtaining the segmentation prediction result of the multi-level potential representation variant of the encoder in the student model comprises the following steps: the output potential representation z of each residual block of RB2, RB3, and RB4 in the student model h Obtaining variants by perturbation operations
Generating a segmentation prediction result by an auxiliary decoder;
for annotated data, the student model passes through a supervised loss function L seg Training is carried out;
the multi-level consistency loss function comprises a learnable multi-level loss function L lh_c And self-directed multi-level consistency loss function L sgh_c ；
Total loss function L total The concrete expression is as follows:
L total ＝L seg +λ h_c (L lh_c +λ sgh_c L sgh_c )
wherein λ is sgh_c Is L sgh_c Coefficient of (a) sgh_c ＝0～1，λ h_c Expressed as:
q is a scaling factor, v equals the current number of iterations of the training, and T equals the total number of iterations of the training.
2. The histological image segmentation model of claim 1, wherein the teacher model is a teacher modelThe update strategy of the medium weight is as follows: in each training batch, updating the weight of the teacher model according to the weight of the teacher model in the previous training batch and the weight of the student model in the training batch, wherein the updating strategy is as follows: weight θ 'of teacher model in t-th training batch' t Comprises the following steps:
θ′ t ＝αθ′ t-1 +(1-α)θ t
wherein, theta' t-1 Represents the weight, θ, of the teacher model in the t-1 th training batch t Represents the weight of the student model in the t-th training batch, and alpha represents the updating of the student model theta by gradient descent in the total training process t Is measured by the exponential moving average of (d).
3. The histological image segmentation model of claim 1, wherein there is a supervised loss function L seg By a cross entropy loss function L ce Sum variance constrained cross-loss function L var Composition of a supervised loss function L seg Is represented as follows:
L seg ＝L ce +λ var L var
wherein L is ce Representing the cross entropy loss function, L var Represents a variance constrained cross-loss function, λ var Representing the weight of the variance constrained cross-loss function.
4. The histological image segmentation model of claim 3, wherein labeled data B for each training batch l Variance constrained cross-loss function L var As follows:
wherein D represents labeled data B of the training batch l Number of split instances, B d Represents all pixels, | B, contained in the d-th segmentation instance in the training batch d I represents B d In (1)Number of pixels, p j Is represented by B d The prediction probability that the jth pixel in (j) is in the correct class, j 1 … … | B d |，u d Is shown in B d Average of all the pixel prediction probabilities.
5. The histological image segmentation model of claim 4, wherein the perturbation operation is performed in a random manner, in particular dropout or using a noise floor at a feature level.
6. The histological image segmentation model of claim 5, wherein all the auxiliary decoders have the same structure, including a hole space convolution pooling pyramid layer and an upsampling layer.
7. The histological image segmentation model of claim 6, wherein the learnable multi-level loss function L lh_c Specifically, the following are shown:
wherein, B u Represents any one training batch with no labeled data, | B u I represents B u Number of pixels in, L mse Represents the mean square error function, H represents the number of levels in the multi-level mandatory consistency module,
representing the segmented prediction result of the k-th pixel by the student model main decoder,
representing the result of the student model's H-th auxiliary decoder's segmentation prediction on the k-th pixel, H-1 … … H,
representing teachersThe learnable predicted probability of the model for the kth pixel,
expressed as:
wherein u' k Can be expressed as:
8. The histological image segmentation model of claim 7, wherein the self-guided multi-level consistency loss function L sgh_c Specifically, the following are shown:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210858624.1A CN115131565B (en) | 2022-07-20 | 2022-07-20 | Histological image segmentation model based on semi-supervised learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210858624.1A CN115131565B (en) | 2022-07-20 | 2022-07-20 | Histological image segmentation model based on semi-supervised learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115131565A true CN115131565A (en) | 2022-09-30 |
| CN115131565B CN115131565B (en) | 2023-05-02 |
Family
ID=83384021
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210858624.1A Active CN115131565B (en) | 2022-07-20 | 2022-07-20 | Histological image segmentation model based on semi-supervised learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115131565B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116188481A (en) * | 2023-04-27 | 2023-05-30 | 深圳市大数据研究院 | Image segmentation model training method, image processing method, system and electronic equipment |
| CN116205289A (en) * | 2023-05-05 | 2023-06-02 | 海杰亚（北京）医疗器械有限公司 | Animal organ segmentation model training method, segmentation method and related products |
| CN117058156A (en) * | 2023-06-20 | 2023-11-14 | 苏州大学 | Semi-supervised medical image segmentation method |
| CN117333874A (en) * | 2023-10-27 | 2024-01-02 | 江苏新希望科技有限公司 | Image segmentation method, system, storage medium and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070292005A1 (en) * | 2006-06-14 | 2007-12-20 | Motorola, Inc. | Method and apparatus for adaptive hierarchical processing of print images |
| CN109741332A (en) * | 2018-12-28 | 2019-05-10 | 天津大学 | A kind of image segmentation and mask method of man-machine coordination |
| WO2021157863A1 (en) * | 2020-02-05 | 2021-08-12 | 주식회사 스파이더코어 | Autoencoder-based graph construction for semi-supervised learning |
| CN113256639A (en) * | 2021-05-27 | 2021-08-13 | 燕山大学 | Coronary angiography blood vessel image segmentation method based on semi-supervised average teacher model |
| US20210279525A1 (en) * | 2020-03-05 | 2021-09-09 | Samsung Electronics Company, Ltd. | Hierarchy-preserving learning for multi-label classification |
| CN114283329A (en) * | 2021-11-16 | 2022-04-05 | 华能盐城大丰新能源发电有限责任公司 | Semi-supervised remote sensing image semantic segmentation method and equipment based on strong transformation |
-
2022
- 2022-07-20 CN CN202210858624.1A patent/CN115131565B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070292005A1 (en) * | 2006-06-14 | 2007-12-20 | Motorola, Inc. | Method and apparatus for adaptive hierarchical processing of print images |
| CN109741332A (en) * | 2018-12-28 | 2019-05-10 | 天津大学 | A kind of image segmentation and mask method of man-machine coordination |
| WO2021157863A1 (en) * | 2020-02-05 | 2021-08-12 | 주식회사 스파이더코어 | Autoencoder-based graph construction for semi-supervised learning |
| US20210279525A1 (en) * | 2020-03-05 | 2021-09-09 | Samsung Electronics Company, Ltd. | Hierarchy-preserving learning for multi-label classification |
| CN113256639A (en) * | 2021-05-27 | 2021-08-13 | 燕山大学 | Coronary angiography blood vessel image segmentation method based on semi-supervised average teacher model |
| CN114283329A (en) * | 2021-11-16 | 2022-04-05 | 华能盐城大丰新能源发电有限责任公司 | Semi-supervised remote sensing image semantic segmentation method and equipment based on strong transformation |
Non-Patent Citations (3)
| Title |
|---|
| QIANGGUO JIN 等: "Cascade knowledge diffusion network for skin lesion diagnosis and segmentation", 《ELSEVIER》 * |
| SHUMENG LI 等: "Histological consistency regularized mean teacher for Semi-supervised 3D left Atrium Segmentation", 《IEEE》 * |
| 江朝杰 等: "增量跨模态检索方法", 《增量跨模态检索方法》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116188481A (en) * | 2023-04-27 | 2023-05-30 | 深圳市大数据研究院 | Image segmentation model training method, image processing method, system and electronic equipment |
| CN116188481B (en) * | 2023-04-27 | 2023-10-20 | 深圳市大数据研究院 | Image segmentation model training method, image processing method, system and electronic equipment |
| CN116205289A (en) * | 2023-05-05 | 2023-06-02 | 海杰亚（北京）医疗器械有限公司 | Animal organ segmentation model training method, segmentation method and related products |
| CN117058156A (en) * | 2023-06-20 | 2023-11-14 | 苏州大学 | Semi-supervised medical image segmentation method |
| CN117333874A (en) * | 2023-10-27 | 2024-01-02 | 江苏新希望科技有限公司 | Image segmentation method, system, storage medium and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115131565B (en) | 2023-05-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Abdar et al. | A review of uncertainty quantification in deep learning: Techniques, applications and challenges | |
| CN115131565B (en) | Histological image segmentation model based on semi-supervised learning | |
| Guo et al. | Gan-based virtual-to-real image translation for urban scene semantic segmentation | |
| CN107886510A (en) | A kind of prostate MRI dividing methods based on three-dimensional full convolutional neural networks | |
| CN108399406A (en) | The method and system of Weakly supervised conspicuousness object detection based on deep learning | |
| CN112132149B (en) | Semantic segmentation method and device for remote sensing image | |
| Chen et al. | Learning linear regression via single-convolutional layer for visual object tracking | |
| CN113657387A (en) | Semi-supervised three-dimensional point cloud semantic segmentation method based on neural network | |
| CN114692732B (en) | Method, system, device and storage medium for updating online label | |
| Cai et al. | A robust interclass and intraclass loss function for deep learning based tongue segmentation | |
| Li et al. | Unsupervised domain adaptation for the histopathological cell segmentation through self-ensembling | |
| Cui et al. | Energy-based residual latent transport for unsupervised point cloud completion | |
| Huang et al. | ICMiF: Interactive cascade microformers for cross-domain person re-identification | |
| CN115761654B (en) | Vehicle re-identification method | |
| CN117058394A (en) | Zero sample semantic segmentation method | |
| CN116912620A (en) | Characterization reinforcement learning method with visual disturbance resistance | |
| Deng et al. | Power system stability assessment method based on GAN and GRU‐Attention using incomplete voltage data | |
| CN116148864A (en) | Radar echo extrapolation method based on DyConvGRU and Unet prediction refinement structure | |
| Chen et al. | Improving semantic segmentation with knowledge reasoning network | |
| Xu | Region selection for occluded person re-identification via policy gradient | |
| CN111797732B (en) | Video motion identification anti-attack method insensitive to sampling | |
| Sun et al. | Gppf: A general perception pre-training framework via sparsely activated multi-task learning | |
| Wang et al. | Cross-layer progressive attention bilinear fusion method for fine-grained visual classification | |
| Wang | Semi-supervised semantic segmentation network based on knowledge distillation | |
| Zhu et al. | Application research on improved CGAN in image raindrop removal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |