CN114972311A - Semi-supervised white blood cell image classification method based on consistency regularization - Google Patents
Semi-supervised white blood cell image classification method based on consistency regularization Download PDFInfo
- Publication number
- CN114972311A CN114972311A CN202210710074.9A CN202210710074A CN114972311A CN 114972311 A CN114972311 A CN 114972311A CN 202210710074 A CN202210710074 A CN 202210710074A CN 114972311 A CN114972311 A CN 114972311A
- Authority
- CN
- China
- Prior art keywords
- consistency
- model
- sample
- supervised
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10056—Microscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30242—Counting objects in image
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Investigating Or Analysing Biological Materials (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a semi-supervised white blood cell image classification method based on consistency regularization. First, using a semi-supervised learning approach, unlabeled data is fully utilized in medical image analysis, reducing the need for labeled data. Secondly, an average teacher model is adopted, and exponential moving average is carried out on the weights of the student models to update the weights of the teacher models. In addition, consistency regularization is adopted, so that the consistency of the semantic relation of the sample is kept under different disturbances of the teacher model and the student model, and the network is encouraged to explore extra semantic information from input data to improve the network performance. In addition, intra-class differences and inter-class similarities between white blood cell images are mitigated with sample consistency. And finally, adopting a lightweight network ShuffleNet 2 for transfer learning for a shared network in the teacher model and the student model, reducing training parameters and improving the leukocyte classification precision.
Description
Technical Field
The invention belongs to the technical field of image recognition, and particularly relates to a semi-supervised white blood cell image classification method based on consistency regularization.
Background
White Blood Cells (also called leucocytes) are important components of immune Cells and play a vital role in protecting the human body from viruses, bacteria and pathogens. In the medical field, the differential counting and morphological analysis of leukocytes in blood smears is of great significance for diagnosing blood diseases such as leukemia. The classification of blood leukocytes is an important step in the automated analysis of leukocytes, and information such as the number of leukocytes, the proportion of each type of leukocyte, and morphology is an important index for the diagnosis of human blood diseases such as leukemia [1 ]. The leukocytes can be classified into five types, eosinophils, basophils, neutrophils, lymphocytes and monocytes according to the morphology and size of the leukocytes. When a human body suffers from certain blood diseases (such as dermatomyositis, lymphoma and leukemia), the total number of the white blood cells and the proportion of the different types of the white blood cells in the blood can be obviously changed. Therefore, it is of great importance to accurately classify leukocytes to aid in the diagnosis of some related diseases.
Machine learning-based leukocyte classification typically extracts blood cell features first, and then applies a classifier to achieve leukocyte classification. For example, Zhao et al [2] proposed an automatic leukocyte detection and classification method using threshold segmentation and morphological operations followed by classification using a Support Vector Machine (SVM) and random forests. In addition, the machine learning method for classifying the leucocytes also comprises a K-Nearest Neighbors (KNN) algorithm, a Bayesian algorithm and the like. However, it is difficult for the conventional machine learning method to capture image features whose classification performance is particularly stable.
In recent years, deep learning has enjoyed great success in the fields of computer vision and medical image analysis. For the leukocyte classification problem, et al [3] use a deep learning model to predict the classes of leukocytes and classify the cells into the corresponding classes by combining the results of the different methods. Kutlu et al [4] proposed a region-based convolutional neural network that uses a transfer learning method to classify leukocytes in peripheral blood smear images. For the data problem, since the deep neural network relies heavily on large sample data to avoid the overfitting phenomenon, researchers need to use data enhancement to obtain a large number of white blood cell images. Data enhancement includes conventional data enhancement and methods based on generating a countermeasure network (GAN) [5 ]. Horse et al [6] proposed a framework for generating a countermeasure network and a residual neural network (ResNet-34) based on deep convolution to improve classification performance of monochrome images.
In deep learning, medical image analysis uses a large amount of labeled data for full supervised learning, and excellent performance is achieved. However, in the medical field, collecting these tagged data requires experts to label the data and is time-consuming and labor-consuming, and the untagged data cannot be fully utilized.
Disclosure of Invention
The invention aims to provide a semi-supervised leukocyte image classification method based on consistency regularization, which adopts an average teacher model to perform mixed training on labeled data and unlabeled data, wherein a network model takes a lightweight ShuffleNet V2 as a backbone network. And finally, introducing sample consistency regularization to relieve the inter-class difference and intra-class correlation of the white blood cells so as to improve the classification performance and reduce the time and labor for marking data.
In order to achieve the purpose, the technical scheme of the invention is as follows: a semi-supervised leukocyte image classification method based on consistency regularization is characterized in that an average teacher model is adopted to carry out mixed training on labeled data and unlabelled data, and the average teacher model takes a lightweight ShuffleNet V2 as a backbone network; then, using consistency regularization to predict the consistency of the output of the teacher model and the output of the student model under different disturbances; and finally, introducing sample consistency regularization to relieve the inter-class difference and intra-class correlation of the white blood cells and realize white blood cell classification.
Compared with the prior art, the invention has the following beneficial effects: the invention provides a semi-supervised leukocyte classification method (CCS-Net) based on consistency regularization, which is used for training labeled data and unlabeled data in a mixed mode. Meanwhile, a teacher average model is adopted, a student model and a teacher model share a network, EMA is carried out on the student model weights to update the teacher model weights, a consistency regularization mechanism is adopted to ensure the consistency of output prediction between the student model and the teacher model under different disturbances, and the problems of inter-class difference and intra-class similarity of white blood cell images are relieved by citing sample consistency. In addition, the ShuffleNet V2 of transfer learning is used as a backbone network of the average teacher model, so that network training parameters are reduced, and the training speed is improved. In addition, an optimizer combining Adam and AMSgrad and a weighted cross entropy loss function are used, so that the convergence of model training is guaranteed, and the problem of unbalanced samples is solved. And finally, comparing the performance of the system with other semi-supervised classification methods, and setting an ablation experiment to compare different optimizers and loss weights beta. The result shows that the method provided by the invention achieves excellent results in the aspects of accuracy and training speed.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
FIG. 2 is a network model framework of the present invention.
FIG. 3 shows the structure of the ShuffLeNet V2 network.
FIG. 4 is an improved ShuffleNet V2 based on migration learning.
FIG. 5 is a cumulative distribution; (a) accuracy, (b) specificity, (c) sensitivity, (d) area under curve.
Detailed Description
The technical scheme of the invention is specifically explained below with reference to the accompanying drawings.
As shown in FIG. 1, the invention provides a Semi-Supervised leukocyte classification method based on consistency regularization, which is called CSS-Net (consistency Semi-Supervised Network), the method combines Semi-Supervised learning and consistency regularization, the overfitting problem is relieved, the classification performance is improved, and the CCS-Net Network structure is shown in FIG. 2. Firstly, a semi-supervised learning method is used, unlabelled data are fully utilized in medical image analysis, and the requirement on labeled data is reduced. Secondly, an average teacher model is adopted, and exponential moving average is carried out on the weights of the student models to update the weights of the teacher models. In addition, consistency regularization is adopted, so that the consistency of the semantic relation of the sample is kept under different disturbances of the teacher model and the student model, and the network is encouraged to explore extra semantic information from input data to improve the network performance. In addition, intra-class differences and inter-class similarities between white blood cell images are mitigated with sample consistency. And finally, adopting a lightweight network ShuffleNet V2 for transfer learning to the shared network in the teacher model and the student model, reducing training parameters and improving the leukocyte classification precision.
1. Average teacher model
Existing semi-supervised learning methods can be broadly divided into three types: (1) based on confrontation training, (2) based on graph, (3) based on consistency [7] . In semi-supervised learning, consistency regularization is a secondary objective function that encourages the prediction of the network to be similar across training samples. At this stage, the best known method in the consistency regularization model is Tarvainen et al [8] An average teacher model of average model weights is proposed to improve the quality of the consistency objective. The architecture of the network as a whole comprises two parts: teacher model and student model. The student model network parameters are obtained by inputting labeled data and then performing learning gradient descent, and the teacher model network parameters are obtained by Exponential Moving Average (EMA) of the student model network parameters.
Allowing for limited marking data and extraThe invention provides an effective loss function for making full use of the marked data to carry out semi-supervised learning. The number of the marked labels is N, the number of the unmarked labels is M, so that the marked data set is N
Set of unlabeled data
Wherein alpha is 20%, 50%, 80%, x i For the input blood leukocyte image, y i Is a genuine label that is one-hot encoded. The aim of the invention is to minimize the total loss:
wherein L is s To mark the input supervision loss, e.g. a weighted cross entropy loss function, is used to evaluate the output of the network on the tag data input. L is u The method is used for measuring the consistency of the same input under different disturbances for unsupervised consistency loss. F (-) is represented as a parameterized classification network model. μ is an ascending weighting factor empirically set to 1, controlling the balance between supervised and unsupervised losses. z and z' respectively denote the addition of different input perturbations, e.g. Gaussian noise, in the two models s And theta t Respectively representing the weights of the parameters of the student model and the teacher model,
the weight n representing the teacher's network in the training step is updated as follows:
where α is a smoothing coefficient that controls the weight update rate.
In order to ensure consistency of semantic output of the teacher model and the student model under different disturbances, a traditional consistency mechanism is reserved, and the mechanism is represented as follows:
wherein x is i Representing the unlabeled training set D U Per unlabeled training sample sampled.
The pathological information in the leukocyte images is different, so that the images of the same type are greatly different in visual effect, and the images of different types are also possibly extremely similar in visual effect, namely, the intra-class difference and the inter-class similarity. To alleviate this problem, Liu et al are cited [9] The proposed sample relation consistency mechanism regularizes the network and keeps the consistency of the relation between samples under different disturbances to improve the network performance. In each small batch input B, a case-level gram matrix is used [10] Constructing the relation between different samples, and representing the activation map of the l-th layer as A l ∈R B×HWC Wherein H and W are the space dimensions of the characteristic diagram, and C is the number of channels. Gram matrix G according to case level l ∈R B×B Expressed as:
G l ＝A l ×(A l ) T
i.e. the activation map of the ith sample and the activation map of the jth sample are inner-multiplied to show the similarity between the two. For G l Is normalized by L2 to obtain a final sample relation matrix R l The formula is expressed as:
thus according to the sample relation matrix R l The sample consistency loss function is found to be:
wherein x is trainingSet { D s ∪D U Small batches of samples, R l (x,θ s Z) and R l (x,θ t And z') are the sample relationship matrices for calculating x under different weights and perturbations, respectively. By minimizing L during training CR The method improves the robustness and the distinguishability of the network, and is helpful for obtaining more semantic information about the unmarked data under different disturbances.
The research object of the invention is to adopt semi-supervised learning aiming at labeled data and unlabeled data to improve the classification performance of white blood cells, and the overall loss function is expressed as:
Loss＝L S +μL U ＝L S +μ(L C +βL CR )
wherein L is C For conventional loss of consistency, L CR Is the sample consistency loss, and β is a hyperparameter that balances the conventional consistency loss and the sample consistency loss, and is typically set to 1.
2. Backbone network
ResNet and DenseNet in the deep convolutional neural network greatly improve the classification accuracy in image classification, but besides the accuracy, the calculation complexity is also an important index to be considered by the convolutional neural network, and an excessively complex network model increases the training cost and the training parameter quantity, so that ShuffleNet V2 is adopted in the invention [11] As a backbone network, shared by student and teacher models, ShuffleNetV2 is a lightweight network proposed by the open world in 2018, with a balance between speed and accuracy. As shown in fig. 3, which is a network structure of ShuffleNetV2, the input feature map is divided into two branches in the channel dimension by using channel mixing: the number of channels is C ' and C-C ', and C ' is C/2 in practical realization. Then the outputs of the two branches are merged to ensure the information confluence of the two branches. In the invention, ShuffleNet V2_1.0 is adopted for model training, and the network structure is shown in Table 1.
TABLE 1 ShuffleNet V2_1.0 [11] Structure of the device
3. Transfer learning
Transfer learning is a sub-domain of deep learning that focuses on transferring knowledge from a source domain to a target domain to enhance a target task. Two migration learning methods commonly used in medical image analysis include fine-tuning and Domain Adaptation (DA). Deep learning depends on a large amount of data, if the number of data sets is too small, the classification performance is possibly reduced due to the overfitting phenomenon, and aiming at the condition that the sample data size is insufficient, the model weight which is trained in advance is migrated to a new model by means of migration learning, so that the training time is shortened, and good classification accuracy can be obtained on a small sample.
To fully exploit the potential of ShuffLeNet V2 in a small amount of tagged data, ImageNet was used as the source domain, and migratory learning was applied to migrate the knowledge learned from ImageNet to the target domain. Instead of initializing all weights randomly, they are initialized with weights learned from the ImageNet dataset (except for the last connected layer). Since the class number of leukocytes is different from the 1000 classes of ImageNet, which corresponds to the output dimension of the last fully-connected layer, the weights of the last fully-connected layer in the pre-trained model are discarded. And (3) improving the ShuffleNet V2 network by combining with the transfer learning, freezing the characteristic layer and the weight of the model architecture, and reloading a new design module into the full connection layer of the network model, wherein the improved network is shown in FIG. 4.
In the new design module, four full connection layers are arranged, and BN is added behind the first connection layer [12] And the ReLU and the Dropout are added after the second full connection layer, the random loss probability is set to be 0.5, the stability of the network model is optimized, the convergence speed of the network model is accelerated, the generalization capability of the model is improved, and the overfitting problem is reduced.
4. Parameter setting (Parameter settings)
Adam optimizer for CCS-Net [13] And novel exponential moving average method AMSgrad [14] The network is optimized in combination, and the initial learning rate is set to be 1e-4, beta 1 Is 0.9，β 2 0.999, and the weight decay rate is set to 5 e-4. The data set is divided into a training set, a verification set and a test set according to the proportion of 7:1:2, and the training set is randomly divided into marked data and unmarked data according to the proportion of 20%, 50% and 80% respectively in the training process. The small batch of input images is set to 16 with the labeled input images set to 4. The number of iterations was set to 100, the EMA decay rate was 0.99, and the loss of consistency rose to 1 after the 30 th iteration.
5. Experimental data and evaluation
In order to test the classification accuracy and speed of CCS-Net provided by the invention, a data set enhanced by WGAN data is selected for experiment, and the total number of the experimental data sets is 1503. The comparison method used by the invention is also a medical image classification method based on semi-supervision, which is called Baseline [9] . In order to comprehensively evaluate the effectiveness of the consistency semi-supervised leukocyte classification method, a training set is subjected to network training according to labeled data and unlabeled data in a specified proportion.
The leukocyte classification experimental method was evaluated using five criteria of Accuracy (Accuracy, Acc), Precision (Precision, Pre), Sensitivity (sensivity, Sens), Specificity (Spec) and F1 metric (F1 score, F1). The calculation formula is as follows:
area Under the Curve (AUC) is defined as the Area Under the ROC Curve, and is used as an evaluation criterion of the algorithm, the value range is 0.5 to 1, and the larger the Area is, the better the classification performance is. Wherein TP + FP + TN + FN represents the total number of the dataset, TP + FP represents the number of samples in the dataset that are predicted to be positive, and TP + FN represents the number of samples in the dataset that are positive for the true label.
Table 2 lists labeled data in 20%, 50%, and 80% proportions, respectively, in the training samples, and the remaining data were the area under the curve, accuracy, sensitivity, specificity, precision, and F1 values obtained from consistent semi-supervised learning of unlabeled data. In a whole view, the CCS-Net method provided by the invention produces a classification result superior to Baseline. Baseline has low accuracy, sensitivity, specificity, precision and F1 value when only 20% of labeled data is used, and the performance is improved when the proportion of labeled data reaches 50%. However, when the ratio of the flag data is increased to 80%, the increase in the five evaluation indexes is reduced, and the performance tends to be stable. Conversely, when the marking data proportion is 20%, the other five indexes except the area index under the curve are all higher than the Baseline with the marking data proportion of 20%, which shows that the classification performance of the network provided by the invention is superior to that of the Baseline under the condition of little marking data. With the increase of the proportion of the marking data, the accuracy, sensitivity, specificity, precision and F1 of the two methods of Baseline and CCS-Net all steadily increase, when the proportion of the marking data is 50%, the sensitivity of CCS-Net is higher than 0.15% of the marking data of Baseline using 80%, but when the proportion of the marking data is 80%, the area under the curve of Baseline is higher than 0.03% of the marking data of CCS-Net using the same proportion. The CCS-Net method provided by the invention is superior to the Baseline comparison method in terms of overall results, and the classification performance of the white blood cells is improved.
Table 2 index values using 20%, 50%, and 80% of the marking data
Besides analyzing the overall classification effect of the white blood cell image, the invention also counts the accumulated distribution of the data set in terms of accuracy, sensitivity, specificity and area under curve. As shown in fig. 5, wherein the green, gray, and magenta bands x lines represent the curves of results for Baseline with 20%, 50%, and 80% of labeled data, respectively, and the blue-above triangle line, the orange-below triangle line, and the red line represent the results for CCS-Net training samples containing 20%, 50%, and 80% of labeled data, respectively. From the overall view, CCS-Net is marked as 20%, 50% and 80% higher than Baseline in training samples, respectively [9] The classification performance of the CCS-Net is improved to a certain extent along with the increase of the proportion of the labeled data, when the proportion of the labeled data is 80%, the cumulative distribution result of the four indexes is the best, but on the sensitivity curve, when the CCS-Net is trained on labeled data containing 20% of the training sample, the numerical value of the curve is gradually stabilized after being reduced and then increased, and the stability of the curve is lower than that of a Baseline method containing 20% of the labeled data. On the whole, CCS-Net and Baseline are sensitive to the quantity of marked data, the classification performance is better when the quantity of the marked data is larger, and the classification performance of CCS-Net is better than that of the Baseline method when the quantity of the marked data is small.
6. Parameter and time analysis (Parameter and time analysis)
In medical image classification, an excellent convolutional neural network can improve the performance of the network, but a computer-aided tool needs to consider not only the accuracy performance of classification, but also the operating efficiency and hardware conditions. The lightweight convolutional neural network can not only improve the classification performance, but also reduce the training parameter quantity and the training cost. Reducing the amount of parameters may ensure that less memory is used to implement the training of the network. Thus, the number of parameters and the training speed are taken into account when designing the network. Table 3 shows Baseline [9] And CCS-Net, wherein the training time is 80% of the labeled data. BaIn the seline method, a pretrained DenseNet121 is used as a backbone network, a cross entropy loss function and an Adam optimizer are used for training, and due to intensive connection between DenseBlock, the amount of training parameters is increased, the training speed is slowed down, and the training cost is further increased. Theoretical analysis and experimental verification prove that the accuracy and the training speed of the CCS-Net provided by the invention are improved compared with Baseline.
TABLE 3 comparison of Baseline, CCS-Net parameters and training time
Reference documents:
[1]Y Duan,J Wang,M Hu,et al.Leukocyte classification based on spatial and spectral features ofmicroscopic hyperspectral images[J].Optics&Laser Technology,112:530-538(2019).
[2]J.Zhao,M.Zhang,Z.Zhou,et al.Automatic detection and classification ofleukocytes using convolutional neural networks[J].Medical&biological engineering&computing,55(8):1287–1301(2017).
[3]W Yu,J Chang,C Yang,et al.Automatic classification of leukocytes using deep neural network.In Proceedings of 12th International Conference onASIC(ASICON).IEEE,Guiyang,China,pp.1041–1044(2017).
[4]H.Kutlu,E.Avci,F.
White blood cells detection and classification based on regional convolutional neural networks.Medical hypotheses,135:109472(2020).
[5]Goodfellow I,Pouget-Abadie J,Mirza M,et al.Generative adversarial nets[J].Advances in neural informationprocessing systems,27(2014).
[6]L Ma,R Shuai,X Ran,et al.Combining DC-GAN with ResNet for blood cell image classification[C].Medical&biological engineering&computing,58(6):1251-1264(2020).
[7]Srinidhi C L,Kim S W,Chen F D,et al.Self-supervised driven consistency training for annotation efficient histopathology image analysis[J].Medical ImageAnalysis,2022,75:102256.
[8]Tarvainen A,Valpola H.Mean teachers are better role models:Weight-averaged consistency targets improve semi-supervised deep learning results[J].Advances in neural information processing systems,2017,30.
[9]Liu Q,Yu L,Luo L,et al.Semi-supervised medical image classification with relation-driven self-ensembling model[J].IEEE transactions on medical imaging,2020,39(11):3429-3440.
[10]Gatys LA,Ecker A S,Bethge M.ANeural Algorithm of Artistic Style[J].Journal of Vision,2015.
[11]Ma N,Zhang X,Zheng H T,et al.ShuffleNet V2:Practical Guidelines for Efficient CNN Architecture Design[J].Springer,Cham,2018.
[12]Huang G,Liu Z,Van Der Maaten L,et al.Densely connected convolutional networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.2017:4700-4708.
[13]Kingma D P,Ba J.Adam:A method for stochastic optimization[J].arXiv preprint arXiv:1412.6980,2014.
[14]Reddi S J,Kale S,Kumar S.On the convergence of adam and beyond[J].arXiv preprint arXiv:1904.09237,2019.。
the above are preferred embodiments of the present invention, and all changes made according to the technical scheme of the present invention that produce functional effects do not exceed the scope of the technical scheme of the present invention belong to the protection scope of the present invention.
Claims (6)
1. A semi-supervised leukocyte image classification method based on consistency regularization is characterized in that an average teacher model is adopted to carry out mixed training on labeled data and unlabelled data, and the average teacher model takes a lightweight ShuffleNet V2 as a backbone network; then, using consistency regularization to predict the consistency of the output of the teacher model and the output of the student model under different disturbances; and finally, introducing sample consistency regularization to relieve the inter-class difference and intra-class correlation of the white blood cells and realize white blood cell classification.
2. The semi-supervised leukocyte image classification method based on consistency regularization as claimed in claim 1, wherein the overall architecture of the average teacher model comprises two parts: a teacher model and a student model; the student model network parameters are obtained by inputting labeled data to perform learning gradient descent, and the teacher model network parameters are obtained by performing exponential sliding average on the student model network parameters.
3. The semi-supervised leukocyte image classification method based on consistency regularization as claimed in claim 2, wherein an effective loss function is proposed to fully utilize labeled label data for semi-supervised learning in consideration of limited labeled label data and additional unlabeled label data, specifically as follows:
setting the number of marked label data as N, the number of unmarked label data as M, and the marked label data set as
Unlabeled labeled data set as
Wherein, delta belongs to 20 percent, 50 percent, 80 percent and x i For the input blood leukocyte image, y i A genuine label for one-hot encoding; the goal is to minimize the total loss:
wherein L is s For marking input supervision loss, evaluating the output of the network on the tag data input; l is u The method is unsupervised consistency loss and is used for measuring the consistency of the same input under different disturbances; f (-) is expressed as a parameterized classification network model; μ is an ascending weight factor, controlling the balance between supervised and unsupervised losses; z and z' respectively represent that different input disturbances are added into the teacher model and the student model; theta s And theta t Respectively representing the parameter weights of the student model and the teacher model; by using
The weight n representing the teacher model in the training step is updated as follows:
wherein, alpha is a smooth coefficient for controlling the update rate of the weight value;
in order to ensure the consistency of semantic output of the teacher model and the student model under different disturbances, a traditional consistency mechanism is reserved, and a traditional consistency loss function is expressed as follows:
wherein x is i Representing unlabeled tag data set D U Each of the unlabeled tag data of the middle sample,
representing an expected distance between the student model and the teacher model prediction;
in order to relieve the intra-class difference and inter-class similarity of the white blood cells, sample consistency regularization is introduced, an average teacher model is regularized, and the consistency of the relation between samples under different disturbances is maintained; in each small batch input B, use caseConstructing the relation between different samples by using a case-level gram matrix, and representing the activation map of the l-th layer as A l ∈R B×HWC H and W are space dimensions of the characteristic diagram, and C is the number of channels; according to case level gram matrix G l ∈R B×B Expressed as:
i.e. activation map of the ith sample
And activation map of jth sample
Inner products are made between the two to show the similarity between the two; for G l Is normalized by L2 to obtain a final sample relation matrix R l The formula is expressed as:
wherein the content of the first and second substances,
a case-level gram matrix representing the ith sample in the ith layer;
according to a sample relation matrix R l The sample consistency loss function is found to be:
wherein x is { D s ∪D U Small batches of samples, R l (x,θ s Z) and R l (x,θ t Z') are sample relation matrices for calculating x under different weights and disturbances, respectively; minimizing L during training CR ；
The overall loss function is expressed as:
Loss＝L S +μL U ＝L S +μ(L C +βL CR ) (7)
wherein L is C Is a conventional consistency loss function, L CR Is the sample consistency loss function, and β is a hyper-parameter that balances the traditional consistency loss function and the sample consistency loss function.
4. The semi-supervised leukocyte image classification method based on regularization of claim 2, wherein the teacher model and the student models share ShuffleNet V2.
5. The semi-supervised leukocyte image classification method based on consistency regularization as recited in claim 1 or 4, wherein the ShuffleNet V2 is a ShffleNet V2 improved based on migration learning, and has four fully connected layers, BN, ReLU and Dropout are added after the first fully connected layer, ReLU and Dropout are added after the second fully connected layer, and the random loss probability is set to be 0.5.
6. The semi-supervised leukocyte image classification method based on consistency regularization as recited in claim 1, wherein an Adam optimizer and a novel exponential moving average method AMSgrad are combined to optimize a network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210710074.9A CN114972311A (en) | 2022-06-22 | 2022-06-22 | Semi-supervised white blood cell image classification method based on consistency regularization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210710074.9A CN114972311A (en) | 2022-06-22 | 2022-06-22 | Semi-supervised white blood cell image classification method based on consistency regularization |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114972311A true CN114972311A (en) | 2022-08-30 |
Family
ID=82965126
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210710074.9A Pending CN114972311A (en) | 2022-06-22 | 2022-06-22 | Semi-supervised white blood cell image classification method based on consistency regularization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114972311A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116385809A (en) * | 2023-06-05 | 2023-07-04 | 山东第一医科大学附属省立医院(山东省立医院) | MRI brain tumor classification method and system based on semi-supervised learning |
| CN117173401A (en) * | 2022-12-06 | 2023-12-05 | 南华大学 | Semi-supervised medical image segmentation method and system based on cross guidance and feature level consistency dual regularization |
-
2022
- 2022-06-22 CN CN202210710074.9A patent/CN114972311A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117173401A (en) * | 2022-12-06 | 2023-12-05 | 南华大学 | Semi-supervised medical image segmentation method and system based on cross guidance and feature level consistency dual regularization |
| CN117173401B (en) * | 2022-12-06 | 2024-05-03 | 南华大学 | Semi-supervised medical image segmentation method and system based on cross guidance and feature level consistency dual regularization |
| CN116385809A (en) * | 2023-06-05 | 2023-07-04 | 山东第一医科大学附属省立医院(山东省立医院) | MRI brain tumor classification method and system based on semi-supervised learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112801212B (en) | White blood cell classification counting method based on small sample semi-supervised learning | |
| CN114972311A (en) | Semi-supervised white blood cell image classification method based on consistency regularization | |
| Dirvanauskas et al. | Embryo development stage prediction algorithm for automated time lapse incubators | |
| Elen et al. | Classifying white blood cells using machine learning algorithms | |
| Bani-Hani et al. | Classification of leucocytes using convolutional neural network optimized through genetic algorithm | |
| Sejuti et al. | An efficient method to classify brain tumor using CNN and SVM | |
| Liang et al. | Comparison detector for cervical cell/clumps detection in the limited data scenario | |
| Zhang et al. | Hybrid adversarial‐discriminative network for leukocyte classification in leukemia | |
| CN113159066B (en) | Fine-grained image recognition algorithm of distributed labels based on inter-class similarity | |
| Sanida et al. | Tomato leaf disease identification via two–stage transfer learning approach | |
| AlBdairi et al. | Identifying ethnics of people through face recognition: A deep CNN approach | |
| Sipes et al. | Using convolutional neural networks for automated fine grained image classification of acute lymphoblastic leukemia | |
| Dong et al. | A self-adaptive approach for white blood cell classification towards point-of-care testing | |
| CN112183237A (en) | Automatic white blood cell classification method based on color space adaptive threshold segmentation | |
| Dharani et al. | DEEPORCD: detection of oral cancer using deep learning | |
| Lerner et al. | A comparison of state-of-the-art classification techniques with application to cytogenetics | |
| Chen et al. | A skin detector based on neural network | |
| Chen et al. | Transmixnet: an attention based double-branch model for white blood cell classification and its training with the fuzzified training data | |
| Li et al. | Facial emotion recognition via stationary wavelet entropy and Biogeography-based optimization | |
| Wang et al. | A computer-assisted human peripheral blood leukocyte image classification method based on Siamese network | |
| CN114970862B (en) | PDL1 expression level prediction method based on multi-instance knowledge distillation model | |
| Zhang et al. | Automatic cotton spider mites' damage grading algorithm for depth residual network based on transfer learning | |
| CN112257787B (en) | Image semi-supervised classification method based on generation type dual-condition confrontation network structure | |
| Karthikeyan et al. | Machine Learning based Algorithmic approach for Detection and Classification of Leukemia | |
| Bhatia et al. | Integrating explainability into deep learning-based models for white blood cells classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |