CN115131565B - Histological image segmentation model based on semi-supervised learning - Google Patents
Histological image segmentation model based on semi-supervised learning Download PDFInfo
- Publication number
- CN115131565B CN115131565B CN202210858624.1A CN202210858624A CN115131565B CN 115131565 B CN115131565 B CN 115131565B CN 202210858624 A CN202210858624 A CN 202210858624A CN 115131565 B CN115131565 B CN 115131565B
- Authority
- CN
- China
- Prior art keywords
- model
- loss function
- training
- level
- segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V10/7753—Incorporation of unlabelled data, e.g. multiple instance learning [MIL]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/776—Validation; Performance evaluation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a histological image segmentation model based on semi-supervised learning, which comprises the following steps: a teacher model, a student model, a multi-level forced consistency module and a total loss function for supervision training; training the student model by using marked data and unmarked data, training the teacher model by using unmarked data, and performing consistency constraint on a segmentation prediction result of the teacher model and a segmentation prediction result of a variant of a multi-level potential representation of an encoder in the student model by using a multi-level consistency loss function by using a multi-level forced consistency module when the student model and the teacher model are trained by using the unmarked data; the histological image segmentation model is effective, a multi-level forced consistency module and a multi-level consistency loss function are provided, and the prediction invariance of a model segmentation prediction result is enhanced by adding disturbance to multi-level potential representation of the model.
Description
Technical Field
The invention belongs to the technical field of image segmentation, and particularly relates to a histological image segmentation model based on semi-supervised learning.
Background
Accurate segmentation of cells and glands using histological images is an indispensable but challenging task in computer-aided diagnosis. Advanced performance is achieved by means of a large number of labeled data through a method of histological image segmentation by deep learning techniques [1]. However, there is a challenging problem in the field of histological image analysis, namely that the performance enhancement of deep learning models requires a large number of high quality and well-annotated data supports. However, unlike natural images, labeling of medical images requires expert participation with domain knowledge, and labeling good data acquisition is a time-consuming and labor-intensive task.
In recent years, in order to solve the problem of labeling difficulties, more and more research has been devoted to medical image segmentation with a limited amount of labeled data and a large amount of unlabeled data using semi-supervised learning techniques [2,4,5]. However, how to promote consistency between annotated data and unlabeled data presents a significant challenge to the development of semi-supervised learning. While research is currently focused on formulating perturbations to consistently train tagged and untagged data [2,4,5], existing consistency training methods focus primarily on formulating perturbations applied to input spaces and advanced feature spaces, and ignore formulating perturbations in hierarchical latent feature spaces of deep network architectures. Also, in the Mean-Teacher architecture commonly used in consistency training methods, a Teacher model is typically used to generate training targets for student models. However, it is difficult to determine whether the Teacher model performs better than the student model during the training process, and the low-performance Teacher model presents a serious challenge for training of the Mean-Teacher architecture.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide a histological image segmentation model based on semi-supervised learning, which solves the problems of labeling difficulty in the field of computer-aided diagnosis and low-performance Teacher model interference model training in a Mean-Teacher architecture.
The aim of the invention is achieved by the following technical scheme.
A semi-supervised learning based histological image segmentation model, comprising: a teacher model, a student model, a multi-level forced consistency module and a total loss function for supervision training;
the structure of the teacher model is the same as that of the student model, the teacher model and the student model are both composed of a coder and a main decoder by adopting deep LabV3+ with cavity convolution, wherein the coder comprises a convolution block CB and four residual blocks, the four residual blocks are RB1, RB2, RB3 and RB4 respectively, each residual block is a pre-trained ResNet34, and the RB3 and RB4 use cavity convolution;
training the student model by using marked data and unmarked data, training the teacher model by using unmarked data, and performing consistency constraint on a segmentation prediction result of the teacher model and a segmentation prediction result of a variant of a multi-level potential representation of an encoder in the student model by using a multi-level consistency loss function by using a multi-level forced consistency module when the student model and the teacher model are trained by using the unmarked data; the method for obtaining the segmentation prediction result of the variant of the multi-level potential representation of the encoder in the student model comprises the following steps: the output potential representation z of each residual block of RB2, RB3, and RB4 in the student model h Obtaining variants by perturbation operations
Generating a segmentation prediction result through an auxiliary decoder;
for marked data, the student model passes through a supervised loss function L seg Training is carried out;
the multi-level consistency loss function comprises a learnable multi-level loss function L lh_c And self-guided multi-level consistency loss function L sgh_c ；
Total loss function L total The concrete steps are as follows:
L total ＝L seg +λ h_c (L lh_c +λ sgh_c L sgh_c )
wherein lambda is sgh_c Is L sgh_c Coefficient lambda of (a) sgh_c ＝0～1，λ h_c Expressed as:
where q is a scaling factor, v is equal to the current number of iterations of the training, and T is equal to the total number of iterations of the training.
In the above technical solution, the update policy of the weights in the teacher model is: for each training batch, the weight update of the teacher model is based on the weight of the teacher model in the previous training batch and the weight of the student model in the present training batch, and the update strategy is as follows: weights θ 'for teacher model in the t training batch' t The method comprises the following steps:
θ′ t ＝αθ′ t-1 +(1-α)θ t
wherein, θ' t-1 Representing the weight, θ, of the teacher model in the t-1 th training lot t Representing the weight of the student model in the t-th training batch, α represents updating the student model θ using gradient descent during the overall training process t The decay rate of the exponential moving average of (a).
In the above technical solution, the supervised loss function L seg From cross entropy loss function L ce Sum-of-variance constraint cross-loss function L var Composition, supervised loss function L seg The expression is as follows:
L seg ＝L ce +λ var L var
wherein L is ce Represents a cross entropy loss function, L var Represents a variance constraint cross-loss function, lambda var Representing the weights of the variance constraint cross-loss function.
In the above technical solution, the labeled data B for each training lot l Variance constraint cross-loss function L var The following is indicated:
wherein D represents marked data B of the training batch l Number of middle split instancesOrder, B d Representing all pixels contained in the d-th segmentation instance in the training batch, |B d I represents B d The number of pixels in (p) j Representation B d Prediction probability of j-th pixel belonging to correct class, j= … … |b d |，u d Represented at B d An average of all pixel prediction probabilities in (a).
In the above technical solution, the manner of disturbance operation is random, specifically dropout or a noise layer using a feature level.
In the above technical solution, all the auxiliary decoders have the same structure, including a hole space convolution pooling pyramid layer and an up-sampling layer.
In the above technical solution, a learnable multi-level loss function L lh_c The concrete representation is as follows:
wherein B is u Represents any training batch without marked data, |B u I represents B u The number of pixels in L mse Represents the mean square error function, H represents the number of layers in the multi-layer forced consistency module,
representing the segmentation prediction result of the student model main decoder for the kth pixel,/for the student model main decoder>
Representing the prediction result of the division of the kth pixel by the H auxiliary decoder of the student model, h= … … H, < >>
Representing the learner-predicted probability of the teacher model for the kth pixel,>
expressed as:
wherein u' k Can be expressed as:
In the above technical solution, the self-guiding multi-level consistency loss function L sgh_c The concrete representation is as follows:
the beneficial effects of the invention are as follows:
(1) The invention provides a histological image segmentation model based on semi-supervised learning, and the results of two embodiments show the effectiveness of the histological image segmentation model;
(2) The invention provides a multi-level forced consistency module and a multi-level consistency loss function, and the prediction invariance of a model segmentation prediction result is enhanced by adding disturbance to multi-level potential representation of the model.
Drawings
FIG. 1 is a block diagram of a histological image segmentation model based on semi-supervised learning;
fig. 2 shows the experimental effect after training.
Detailed Description
The technical scheme of the invention is further described below with reference to specific embodiments.
Example 1
A semi-supervised learning based histological image segmentation model, comprising: the system comprises a teacher model, a student model, a multi-level forced consistency (Hierarchical consistencyenforcement) module and a total loss function for supervision training, wherein the teacher model and the student model have the same structure, the teacher model and the student model are both made of deep LabV3+ [8] with cavity convolution, the deep LabV3+ is composed of an encoder and a main decoder (g), the encoder comprises a convolution block CB and four residual blocks, the four residual blocks are RB1, RB2, RB3 and RB4 respectively, each residual block is a pre-trained ResNet34, RB3 and RB4 use cavity convolution, expansion parameters of the cavity convolution are respectively set to be 2 and 4 in sequence, and parameters between the teacher model and the student model are independent;
the update strategy of the weights in the teacher model is as follows: for each training batch, the weight update of the teacher model is based on the weight of the teacher model in the previous training batch and the weight of the student model in the present training batch, and the update strategy is as follows: weights θ 'for teacher model in the t training batch' t The method comprises the following steps:
θ′ t ＝αθ′ t-1 +(1-α)θ t
wherein, θ' t-1 Representing the weight, θ, of the teacher model in the t-1 th training lot t Representing the weight of the student model in the t training batch, alpha represents updating the student model alpha using gradient descent during the overall training process t An exponential moving average (Exponential Moving Average), in this example α=0.99.
The student model is trained by the marked data and the unmarked data, the teacher model is trained by the unmarked data, and the multi-level forced consistency (Hierarchical consistency enforcement) module adopts a multi-level consistency loss function to carry out consistency constraint on the segmentation prediction result of the teacher model and the segmentation prediction result of the variant of the multi-level potential representation of the encoder in the student model when the student model and the teacher model are trained by the unmarked data.
For marked data, the student model passes through a supervised loss function L seg Training, supervised loss function L seg By cross entropy loss function (Cross Entropy Loss) L ce And a variance constraint cross-loss function (Variance Constrained Cross Loss) L var The composition is as follows:
L seg ＝L ce +λ var L var
wherein L is ce Representing cross entropy loss function [9 ]]，L var Represents a variance constraint cross-loss function, lambda var Weights representing variance-constrained cross-loss functions, λ in this embodiment var ＝0.1。
Variance constraint cross-loss function L var [9]The local constraint is carried out on pixels belonging to the same segmentation example, so as to solve the problem that the model cannot completely segment the whole segmentation example when the segmentation example in the image has uneven color or texture. Noted data B for each training batch l Variance constraint cross-loss function L var The following is indicated:
wherein D represents marked data B of the training batch l Number of split instances, B d Representing all pixels contained in the d-th segmentation instance in the training batch, |B d I represents B d The number of pixels in (p) j Representation B d Prediction probability of j-th pixel belonging to correct class, j= … … |b d |，u d Represented at B d An average of all pixel prediction probabilities in (a).
The method for obtaining the segmentation prediction result of the variant of the multi-level potential representation of the encoder in the student model comprises the following steps: as shown in FIG. 1, the output potential of each residual block of RB2, RB3, and RB4 in the student model represents z h (z in FIG. 1) 1 、z 2 And z 3 ) Obtaining variants by perturbation operations
(is +.>
And->
) Then through an auxiliary decoder (FIG. 1)Middle->
And->
) And generating a segmentation prediction result.
The manner of the perturbation operation is random, in particular dropout or a noise floor using feature levels [11].
All auxiliary decoders are identical in structure, including a hole-space convolution pooling pyramid layer (Atrous Spatial Pyramid Pooling layer) and an upsampling layer, wherein the four sample rates of the hole-space convolution pooling pyramid layer are set to 6, 8, 18, and 24, respectively.
The multi-level forced consistency (Hierarchical consistency enforcement) module can provide stronger constraint for student model training, thereby promoting generalization of student networks. Furthermore, to ensure that the student model has greater generalization capability, the present invention does not impose constraints on the original potential representation of each level of the student model encoder, but rather imposes constraints on variants of the original potential representation of each level of the encoder.
The multi-level consistency loss function comprises a learnable multi-level loss function L lh_c And self-guided multi-level consistency loss function L sgh_c 。
In order to prevent the teacher model from obtaining high uncertainty estimation and strengthen the hierarchical consistency, a learnable multi-level consistency loss function L is provided lh_c Learnable multi-level loss function L lh_c The concrete representation is as follows:
wherein B is u Represents any training batch without marked data, |B u I represents B u The number of pixels in L mse Represents a mean square error function (Mean Squared Error) for calculating the difference between the segmentation prediction results of the teacher model and the student model, and H represents a multi-level strengthThe number of layers in the consistency (Hierarchical consistency enforcement) module, in this example H takes 3,
representing the segmentation prediction result of the student model main decoder (g in fig. 1) for the kth pixel,/for the k pixel>
Representing the prediction result of the division of the kth pixel by the H auxiliary decoder of the student model, h= … … H, < >>
Representing the learner-able prediction probability of the kth pixel by the teacher model, can provide more reliable predictions for the student model to guide,
expressed as:
wherein u' k Can be expressed as:
As can be seen from equation (4), when the teacher model produces unreliable results (high uncertainty), the teacher model's learned prediction probability for the kth pixel
Approximate prediction of student model main decoder to it +.>
Conversely, when the teacher model has confidence in the predictions (low uncertainty)Learner-predicted probability of teacher model for kth pixel>
Predicted outcome with teacher model->
The same, and provides a certain prediction as a goal of student model learning.
Multi-level consistency loss function L through self-guidance sgh_c To ensure that the outputs of the plurality of auxiliary decoders and the output of the main decoder of the student model are consistent, and the method is specifically expressed as follows:
as can be seen from the formula (7), the student model takes prediction of the main decoder as guidance through constraint of the self-guidance multi-level consistency loss function, and the inconsistency among all decoders is minimized, so that the characteristic representation capability of the student model is enhanced.
Total loss function L total The concrete steps are as follows:
L total ＝L seg +λ h_c (L lh_c +λ sgh_c L sgh_c )
wherein lambda is sgh_c Is L sgh_c Coefficient lambda of (a) sgh_c =0 to 1, in this example 0.1, λ h_c Expressed as:
where q is a scaling factor, in this embodiment q=0.1, v is equal to the current number of iterations of the training, and T is equal to the total number of iterations of the training.
The learnable multi-level loss function can ensure that the segmentation prediction result of the student model can be consistent with the segmentation prediction result of the teacher model, and can also prevent the teacher model from obtaining high-uncertainty prediction. The self-guiding multi-level consistency loss function can ensure that the segmentation prediction result of the auxiliary branch decoder of the student model can be consistent with the segmentation prediction result of the main branch decoder, thereby enhancing the characteristic representation capability of the student model.
MonUSeg [15] (multi-organ-core segmented dataset) and CRAG [16] (colorectal adenocarcinoma dataset) were prepared as datasets, respectively, and image blocks were truncated in the original image in the dataset in a sliding window manner, with MonUSeg, the image block size was set to 128×128, and with CRAG, the image block size was set to 480×480.
The data set is divided into marked data and unmarked data without marking, wherein the marked data is 5%, 10% and 20% of the data set, and the rest data in the data set is unmarked data.
Substituting the marked data and the unmarked data in the data set into the student model in the histological image segmentation model for training, and substituting the unmarked data in the data set into the teacher model in the histological image segmentation model for training.
For MonUSeg, the number of images in a training batch is set to be 16, the total number of training iterations is set to be 500, and for a CRAG data set, the number of images in a training batch is set to be 8, and the total number of training iterations is set to be 300. In order to prevent overfitting, online data enhancement is performed during training, including random scaling, flipping, rotation, and affine transformation. The experimental results (HCE) of the present invention are shown in table 1 and fig. 2, table 1 shows the results of the performance comparison of the present invention with the most advanced semi-supervised learning methods on MoNuSeg and CRAG.
TABLE 1
For MonUSeg, the present invention achieved the highest results on both the evaluation criteria of Dice and AJI. For CRAG, the present invention achieves the highest effect on all evaluation indexes (F1, dice, haus). The invention enables the model segmentation performance and robustness to be continuously improved by encouraging multi-level consistency training.
Fig. 2 shows the segmentation prediction results of the MoNuSeg and CRAG with labeled data of 5% and 10% in the dataset according to the invention and other semi-supervised learning methods. From the segmentation prediction results, the invention has better expandability for segmentation examples of different shapes, such as small cells or large glands.
Example 2
This embodiment is substantially the same as embodiment 1, except that: there are labeled data as 50% in the dataset and MoNuSeg in the dataset. The experimental results are shown in Table 2. Table 2 shows the performance of the present invention compared to the fully supervised method on the MoNuSeg dataset. The present invention (HCE) achieves the best results on both F1 and Dice evaluation criteria.
TABLE 2
The foregoing has described exemplary embodiments of the invention, it being understood that any simple variations, modifications, or other equivalent arrangements which would not unduly obscure the invention may be made by those skilled in the art without departing from the spirit of the invention.
[1]Sahasrabudhe,M.,Christodoulidis,S.,Salgado,R.,Michiels,S.,Loi,S.,André,F.,Paragios,N.,Vakalopoulou,M.:Self-supervised Nuclei Segmentation in Histopathological Images Using Attention.In:International Conference on Medical Image Computing and ComputerAssisted Intervention.pp.393–402.Springer(2020).
[2]Yu,L.,Wang,S.,Li,X.,Fu,C.W.,Heng,P.A.:Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation.In:International Conference on Medical Image Computing and Computer-Assisted Intervention.pp.605–613.Springer(2019).
[3].Graham,S.,Chen,H.,Gamper,J.,Dou,Q.,Heng,P.A.,Snead,D.,Tsang,Y.W.,Rajpoot,N.:MILD-Net:Minimal information loss dilated network for gland instance segmentation in colon histology images.Medical Image Analysis 52,199–211(2019)
[4]Xie,Y.,Zhang,J.,Liao,Z.,Verjans,J.,Shen,C.,Xia,Y.:Pairwise Relation Learning for Semi-supervised Gland Segmentation.In:International Conference on Medical Image Computing and Computer-Assisted Intervention.pp.417–427.Springer(2020).
[5]Li,X.,Yu,L.,Chen,H.,Fu,C.W.,Xing,L.,Heng,P.A.:Transformation-consistent self-ensembling model for semi-supervised medical image segmentation.IEEE Transactions on Neural Networks and Learning Systems pp.1–12(2020).
[6].Li,X.,Yu,L.,Chen,H.,Fu,C.W.,Xing,L.,Heng,P.A.:Transformation-consistent selfensembling model for semisupervised medical image segmentation.IEEE Transactions on Neural Networks and Learning Systems pp.1–12(2020).https://doi.org/10.1109/TNNLS.2020.2995319
[7].Li,Y.,Chen,J.,Xie,X.,Ma,K.,Zheng,Y.:Self-loop uncertainty:A novel pseudo-label for semi-supervised medical image segmentation.In:International Conference on Medical Image Computing and Computer-Assisted Intervention.pp.614–623.Springer(2020)
[8]Chen,L.C.,Papandreou,G.,Kokkinos,I.,Murphy,K.,Yuille,A.L.:DeepLab:Semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected CRFs.IEEE Transactions on Pattern Analysis and Machine Intelligence 40(4),834–848(2017).
[9]Qu,H.,Yan,Z.,Riedlinger,G.M.,De,S.,Metaxas,D.N.:Improving nuclei/gland instance segmentation in histopathology images by full resolution neural network and spatial constrained loss.In:International Conference on Medical Image Computing and ComputerAssisted Intervention.pp.378–386.Springer(2019).
[10]Raza,S.E.A.,Cheung,L.,Shaban,M.,Graham,S.,Epstein,D.,Pelengaris,S.,Khan,M.,Rajpoot,N.M.:Micro-Net:A unified model for segmentation of various objects in microscopy images.Medical Image Analysis 52,160–173(2019)
[11]Ouali,Y.,Hudelot,C.,Tami,M.:Semi-supervised semantic segmentation with cross-consistency training.In:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.pp.12674–12684(2020)
[12]Tarvainen,A.,Valpola,H.:Mean teachers are better role models:Weight-averaged consistency targets improve semi-supervised deep learning results.In:Advances in Neural Information Processing Systems.pp.1195–1204(2017)
[13]Verma,V.,Lamb,A.,Kannala,J.,Bengio,Y.,Lopez-Paz,D.:Interpolation Consistency Training for Semi-supervised Learning.In:Proceedings of the 28th International Joint Conference on Artificial Intelligence.pp.3635–3641.IJCAI’19,AAAI Press(2019)
[14]Vu,T.H.,Jain,H.,Bucher,M.,Cord,M.,Pérez,P.:ADVENT:Adversarial entropy minimization for domain adaptation in semantic segmentation.In:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.pp.2517–2526(2019)
[15]Kumar,N.,Verma,R.,Anand,D.,Zhou,Y.,Onder,O.F.,Tsougenis,E.,Chen,H.,Heng,P.A.,Li,J.,Hu,Z.,et al.:A multi-organ nucleus segmentation challenge.IEEE Transactions on Medical Imaging 39(5),1380–1391(2019)
[16]Awan,R.,Sirinukunwattana,K.,Epstein,D.,Jefferyes,S.,Qidwai,U.,Aftab,Z.,Mujeeb,I.,Snead,D.,Rajpoot,N.:Glandular morphometrics for objective grading of colorectal adenocarcinoma histology images.Scientific Reports 7(1),1–12(2017)
[17]Xiang,T.,Zhang,C.,Liu,D.,Song,Y.,Huang,H.,Cai,W.:BiO-Net:Learning Recurrent Bidirectional Connections for Encoder-Decoder Architecture.In:International Conference on Medical Image Computing and Computer-Assisted Intervention.pp.74–84.Springer(2020)
[18]Xie,Y.,Lu,H.,Zhang,J.,Shen,C.,Xia,Y.:Deep segmentation-emendation model for gland instance segmentation.In:International Conference on Medical Image Computing and Computer-Assisted Intervention.pp.469–477.Springer(2019)
[19]Xie,Y.,Zhang,J.,Liao,Z.,Verjans,J.,Shen,C.,Xia,Y.:Pairwise Relation Learning for Semi-supervised Gland Segmentation.In:International Conference on Medical Image Computing and Computer-Assisted Intervention.pp.417–427.Springer(2020)
[20]Yu,L.,Wang,S.,Li,X.,Fu,C.W.,Heng,P.A.:Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation.In:International Conference on Medical Image Computing and Computer-Assisted Intervention.pp.605–613.Springer(2019)。
Claims (6)
1. A semi-supervised learning based histological image segmentation model, comprising: a teacher model, a student model, a multi-level forced consistency module and a total loss function for supervision training;
the structure of the teacher model is the same as that of the student model, the teacher model and the student model are both composed of a coder and a main decoder by adopting deep LabV3+ with cavity convolution, wherein the coder comprises a convolution block and four residual blocks, the four residual blocks are RB1, RB2, RB3 and RB4 respectively, each residual block is a pre-trained ResNet34, and the RB3 and RB4 use cavity convolution;
training the student model by using marked data and unmarked data, training the teacher model by using unmarked data, and performing consistency constraint on a segmentation prediction result of the teacher model and a segmentation prediction result of a variant of a multi-level potential representation of an encoder in the student model by using a multi-level consistency loss function by using a multi-level forced consistency module when the student model and the teacher model are trained by using the unmarked data; the method for obtaining the segmentation prediction result of the variant of the multi-level potential representation of the encoder in the student model comprises the following steps: the output potential representation z of each residual block of RB2, RB3, and RB4 in the student model h Obtaining variants by perturbation operations
Generating a segmentation prediction result through an auxiliary decoder;
for marked data, the student model passes through a supervised loss function L seg Training is carried out;
the multi-level consistency loss function comprises a learnable multi-level loss function L lh_c And self-guided multi-level consistency loss function L sgh_c The method comprises the steps of carrying out a first treatment on the surface of the Learnable multi-level loss function L lh_c The concrete representation is as follows:
wherein B is u Represents any training batch without marked data, |B u I represents B u The number of pixels in L mse Represents the mean square error function, H represents the number of layers in the multi-layer forced consistency module,
representing the segmentation prediction result of the student model main decoder for the kth pixel,/for the student model main decoder>
Representing the prediction result of the division of the kth pixel by the H auxiliary decoder of the student model, h= … … H, < >>
Representing the learner-predicted probability of the teacher model for the kth pixel,>
expressed as:
wherein u' k Can be expressed as:
self-guiding multi-level consistency loss function L sgh_c The concrete representation is as follows:
total loss function L total The concrete steps are as follows:
L total ＝L seg +λ h_c (L lh_c +λ sgh_c L sgh_c )
wherein lambda is sgh_c Is L sgh_c Coefficient lambda of (a) sgh_c ＝0～1，λ h_c Expressed as:
q is a scaling factor, v is equal to the current iteration number of the training, and T is equal to the total iteration number of the training.
2. The histological image segmentation model according to claim 1, wherein the update strategy of weights in the teacher model is: for each training batch, the weight update of the teacher model is based on the weight of the teacher model in the previous training batch and the weight of the student model in the present training batch, and the update strategy is as follows: weights θ 'for teacher model in the t training batch' t The method comprises the following steps:
θ′ t ＝αθ′ t-1 +(1-α)θ t
wherein, θ' t-1 Representing the weight, θ, of the teacher model in the t-1 th training lot t Represents the t thWeights of student models in training batch, α represents updating student model θ using gradient descent during the overall training process t The decay rate of the exponential moving average of (a).
3. The histological image segmentation model according to claim 1, wherein the supervised loss function L seg From cross entropy loss function L ce Sum-of-variance constraint cross-loss function L var Composition, supervised loss function L seg The expression is as follows:
L seg ＝L ce +λ var L var
wherein L is ce Represents a cross entropy loss function, L var Represents a variance constraint cross-loss function, lambda var Representing the weights of the variance constraint cross-loss function.
4. A histological image segmentation model according to claim 3, wherein for each training batch there is labeling data B l Variance constraint cross-loss function L var The following is indicated:
wherein D represents marked data B of the training batch l Number of split instances, B d Representing all pixels contained in the d-th segmentation instance in the training batch, |B d I represents B d The number of pixels in (p) j Representation B d Prediction probability of j-th pixel belonging to correct class, j= … … |b d |，u d Represented at B d An average of all pixel prediction probabilities in (a).
5. The histological image segmentation model according to claim 4, wherein the manner of perturbation operation is random, in particular dropout or a noise layer using a feature level.
6. The histological image segmentation model of claim 5, wherein all ancillary decoders are structurally identical, including a hole-space convolution pooling pyramid layer and an upsampling layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210858624.1A CN115131565B (en) | 2022-07-20 | 2022-07-20 | Histological image segmentation model based on semi-supervised learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210858624.1A CN115131565B (en) | 2022-07-20 | 2022-07-20 | Histological image segmentation model based on semi-supervised learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115131565A CN115131565A (en) | 2022-09-30 |
| CN115131565B true CN115131565B (en) | 2023-05-02 |
Family
ID=83384021
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210858624.1A Active CN115131565B (en) | 2022-07-20 | 2022-07-20 | Histological image segmentation model based on semi-supervised learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115131565B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116188481B (en) * | 2023-04-27 | 2023-10-20 | 深圳市大数据研究院 | Image segmentation model training method, image processing method, system and electronic equipment |
| CN116205289B (en) * | 2023-05-05 | 2023-07-04 | 海杰亚（北京）医疗器械有限公司 | Animal organ segmentation model training method, segmentation method and related products |
| CN117333874A (en) * | 2023-10-27 | 2024-01-02 | 江苏新希望科技有限公司 | Image segmentation method, system, storage medium and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109741332A (en) * | 2018-12-28 | 2019-05-10 | 天津大学 | A kind of image segmentation and mask method of man-machine coordination |
| WO2021157863A1 (en) * | 2020-02-05 | 2021-08-12 | 주식회사 스파이더코어 | Autoencoder-based graph construction for semi-supervised learning |
| CN113256639A (en) * | 2021-05-27 | 2021-08-13 | 燕山大学 | Coronary angiography blood vessel image segmentation method based on semi-supervised average teacher model |
| CN114283329A (en) * | 2021-11-16 | 2022-04-05 | 华能盐城大丰新能源发电有限责任公司 | Semi-supervised remote sensing image semantic segmentation method and equipment based on strong transformation |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070292005A1 (en) * | 2006-06-14 | 2007-12-20 | Motorola, Inc. | Method and apparatus for adaptive hierarchical processing of print images |
| US20210279525A1 (en) * | 2020-03-05 | 2021-09-09 | Samsung Electronics Company, Ltd. | Hierarchy-preserving learning for multi-label classification |
-
2022
- 2022-07-20 CN CN202210858624.1A patent/CN115131565B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109741332A (en) * | 2018-12-28 | 2019-05-10 | 天津大学 | A kind of image segmentation and mask method of man-machine coordination |
| WO2021157863A1 (en) * | 2020-02-05 | 2021-08-12 | 주식회사 스파이더코어 | Autoencoder-based graph construction for semi-supervised learning |
| CN113256639A (en) * | 2021-05-27 | 2021-08-13 | 燕山大学 | Coronary angiography blood vessel image segmentation method based on semi-supervised average teacher model |
| CN114283329A (en) * | 2021-11-16 | 2022-04-05 | 华能盐城大丰新能源发电有限责任公司 | Semi-supervised remote sensing image semantic segmentation method and equipment based on strong transformation |
Non-Patent Citations (3)
| Title |
|---|
| Cascade knowledge diffusion network for skin lesion diagnosis and segmentation;Qiangguo Jin 等;《ElSEVIER》;1-13 * |
| Histological consistency regularized mean teacher for Semi-supervised 3D left Atrium Segmentation;Shumeng Li 等;《IEEE》;全文 * |
| 增量跨模态检索方法;江朝杰 等;《增量跨模态检索方法》;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115131565A (en) | 2022-09-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115131565B (en) | Histological image segmentation model based on semi-supervised learning | |
| Abdar et al. | A review of uncertainty quantification in deep learning: Techniques, applications and challenges | |
| CN111832501B (en) | Remote sensing image text intelligent description method for satellite on-orbit application | |
| CN112132149B (en) | Semantic segmentation method and device for remote sensing image | |
| Chen et al. | Weakly supervised histopathology image segmentation with sparse point annotations | |
| CN111340738A (en) | Image rain removing method based on multi-scale progressive fusion | |
| Cai et al. | A robust interclass and intraclass loss function for deep learning based tongue segmentation | |
| Yang et al. | Attention-based dynamic alignment and dynamic distribution adaptation for remote sensing cross-domain scene classification | |
| Li et al. | Two‐stage single image dehazing network using swin‐transformer | |
| Bi et al. | Entropy-weighted reconstruction adversary and curriculum pseudo labeling for domain adaptation in semantic segmentation | |
| Xu et al. | EI-HCR: An efficient end-to-end hybrid consistency regularization algorithm for semisupervised remote sensing image segmentation | |
| CN113850012A (en) | Data processing model generation method, device, medium and electronic equipment | |
| CN116580243A (en) | Cross-domain remote sensing scene classification method for mask image modeling guide domain adaptation | |
| CN116148864A (en) | Radar echo extrapolation method based on DyConvGRU and Unet prediction refinement structure | |
| Hudagi et al. | Bayes-probabilistic-based fusion method for image inpainting | |
| Wang et al. | Single image rain removal via cascading attention aggregation network on challenging weather conditions | |
| CN115797642A (en) | Self-adaptive image semantic segmentation algorithm based on consistency regularization and semi-supervision field | |
| Huang et al. | ICMiF: Interactive cascade microformers for cross-domain person re-identification | |
| Zhang et al. | Scale-progressive multi-patch network for image dehazing | |
| Chen et al. | Improving semantic segmentation with knowledge reasoning network | |
| Wang et al. | MFCANet: A road scene segmentation network based on Multi-Scale feature fusion and context information aggregation | |
| Wang | Semi-supervised semantic segmentation network based on knowledge distillation | |
| Zhengpeng et al. | A multimodal feature fusion image dehazing method with scene depth prior | |
| Chen et al. | Focal ViT: image transformer catches up with CNN on small datasets | |
| Li et al. | A multi-grained unsupervised domain adaptation approach for semantic segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |