CN110689089A - Active incremental training method for deep learning of multi-class medical image classification - Google Patents

Active incremental training method for deep learning of multi-class medical image classification Download PDF

Info

Publication number
CN110689089A
CN110689089A CN201910968185.8A CN201910968185A CN110689089A CN 110689089 A CN110689089 A CN 110689089A CN 201910968185 A CN201910968185 A CN 201910968185A CN 110689089 A CN110689089 A CN 110689089A
Authority
CN
China
Prior art keywords
samples
training
model
medical image
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910968185.8A
Other languages
Chinese (zh)
Inventor
段贵多
黄添喜
刘江明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Institute Of Electronic And Information Engineering University Of Electronic Science And Technology Of China
University of Electronic Science and Technology of China
Original Assignee
Guangdong Institute Of Electronic And Information Engineering University Of Electronic Science And Technology Of China
University of Electronic Science and Technology of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Institute Of Electronic And Information Engineering University Of Electronic Science And Technology Of China, University of Electronic Science and Technology of China filed Critical Guangdong Institute Of Electronic And Information Engineering University Of Electronic Science And Technology Of China
Priority to CN201910968185.8A priority Critical patent/CN110689089A/en
Publication of CN110689089A publication Critical patent/CN110689089A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Epidemiology (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Primary Health Care (AREA)
  • Radiology & Medical Imaging (AREA)
  • Pathology (AREA)
  • Databases & Information Systems (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an active increment training method for deep learning of multi-class medical image classification, which comprises the following steps: 1. performing primary data cleaning and preprocessing on the medical image data set; 2. randomly selecting initial data, and performing initial training on the network model; 3. testing the rest samples in the data set to obtain the correspondence between the predicted score and the lesion category; 4. performing cross expansion on the residual samples in the data set, and actively screening candidate samples; 5. performing further data set cleaning; 6. performing incremental training on the model; 7. and (5) testing the model after the incremental training, finishing the training if the accuracy rate is stable, and otherwise, repeating the steps from 4 to 7. The invention improves the AIFT (active enhancement learning method), and solves the problems of difficult medical image classification, low training efficiency and the like caused by data imbalance; the problem of poor application effect of deep learning in the field of pathological change classification is solved, and the auxiliary effect of diagnosing the state of an illness of a doctor is improved.

Description

Active incremental training method for deep learning of multi-class medical image classification
Technical Field
The invention relates to a medical image classification training method, in particular to an active increment training method for deep learning of multi-class medical image classification.
Background
With the emergence of new imaging technology and equipment in medicine and the development of computer technology, the medical image processing technology has increasingly influenced medical research and clinical practice, and has received high attention from scholars at home and abroad. In recent years, with the rise of Deep learning (also called Feature learning) methods, Feature learning has received a great deal of attention in the field of machine learning research. Deep learning utilizes a deep neural network to automatically learn effective feature representation in a data-driven mode, rapidly subverts a research framework based on artificial features in a plurality of machine learning related application fields, and becomes a new research paradigm. High-tech companies such as google, microsoft, IBM, hundredth and the like, which have big data, successively invest a large amount of resources to carry out the research and development of deep learning technology, and remarkable progress is made in the fields of voice, image, natural language, online advertisement and the like.
In recent years, an increasing patient population has led researchers to realize an efficient, high-quality and certain interpretable system for automatically classifying disease patterns, and rapid innovation and development of computer performance and deep learning techniques have provided the possibility for researchers to realize the system. However, in the medical field, the images themselves have strong specialization and specificity, and the task of classifying medical images through a computer also becomes a challenge.
Compared with other natural image data sets, the medical image data set has small labeled data amount and extremely unbalanced data among categories, the training of the model by using the data set is time-consuming and labor-consuming work, the result of the model has serious tendency, and serious misdiagnosis results can be caused in the classification task of the medical image. The active incremental learning method can effectively relieve the adverse effect of low data set quality on model performance, so that the active incremental learning method is more and more widely applied to the field of medical image classification.
Currently, active incremental learning and deep learning are combined as an emerging research direction to study a few. Wang and Shang et al, who are perhaps the earliest researchers combining active learning with deep learning, are based on a stack limit line Boltzmann machine and a stack auto-encoder. Stack et al apply active learning to improve the performance of convolutional neural networks for CAPTCHA recognition. Yang et al propose an active learning framework that reduces the annotation effort by judiciously suggesting the most efficient segmentation of the annotation region based on uncertainty and similarity information provided by a fully convolutional network. Their computational methods are very expensive because they require a set of models to be trained from scratch by the bootstrapping algorithm to compute their uncertainty measures from the divergence of these models, and the maximum set coverage problem must be solved to determine the most representative regions.
The difference between the AIFT algorithm (active-innovative fine-tuning) proposed by Zhou et al is that the above method is repeatedly retrained from scratch in each step, and the AIFT algorithm continuously fine-tunes (fine-tune) the convolutional neural network model in an incremental manner, thereby significantly improving the model training efficiency and reducing the labeling cost. And has high requirements on equipment, but some incremental training methods cannot achieve good effect in a glycogenopathy classification model due to the particularity of medical images such as fundus image data sets.
Disclosure of Invention
In order to solve the above problems, an object of the present invention is to provide an active incremental training method for deep learning multi-class medical image classification, which is easy to classify medical images and has high training efficiency.
The invention realizes the purpose through the following technical scheme:
an active incremental training method for deep learning multi-category medical image classification, comprising the following steps:
step 1, carrying out primary data cleaning and preprocessing on a medical image data set;
step 2, randomly selecting initial data, and performing initial training on the network model;
step 3, testing the rest samples in the data set on the model trained in the step 2 to obtain a prediction probability, and obtaining a prediction score of the sample by using a ranking decoding method on the output probability to realize the correspondence between the prediction score and the lesion category;
step 4, performing cross expansion on the residual samples in the data set, and actively screening candidate samples according to the test result in the step 3;
step 5, setting K values according to data proportions among different categories and carrying out further data set cleaning on the candidate samples obtained by screening in the step 4;
step 6, performing incremental training on the model, and adding a distillation item in a loss function of the incremental training;
and 7, testing the model after the incremental training, wherein the testing method is the same as that in the step 3, testing the accuracy of the model, finishing the training if the accuracy is stable, and otherwise, repeating the steps 4 to 7.
Preferably, the step 1 comprises the following steps:
step 1.1, data cleaning: in the medical image data set, low-quality samples which influence model training due to illumination, misoperation and the like in the data set are removed, and the number of samples in the fundus image data set before and after cleaning is as follows:
number of original samples Number of samples after washing Post-cleaning data ratio
Level 0 65741 63367 72.61％
Level 1 6618 6206 7.11％
Stage 2 14846 13155 15.07％
Grade 3 2301 2087 2.39％
4 stage 2177 1914 2.21％
Total of 91653 87269 100％
；
Step 1.2, image preprocessing: in medical image data set, in order to reduce the influence of different factors such as illumination, a bottom camera and the like on the image during data acquisition, green channel extraction and contrast enhancement operations are sequentially carried out on the image.
Preferably, the step 1.2 comprises the following steps:
step 1.2.1, green channel extraction: extracting a green channel from the color medical image, wherein the medical image is composed of red, green and blue channels, and the green channel contains more information;
step 1.2.2, contrast enhancement is carried out, average brightness is kept, and an image with better quality can be generated;
step 1.2.3, central region extraction: the black background of the image is removed, and the central part of the image is extracted.
Preferably, the step 2 comprises the following steps:
2.1, randomly selecting initial data, selecting a certain number of samples as an initial training set of the model according to the original proportion of samples among the categories in the data set, randomly selecting 5000 samples in the medical image data set, and respectively selecting 3600, 350, 750, 150 and 150 samples in 5 categories;
step 2.2, performing initial training on the model: and taking 5000 samples as a training set, and carrying out initial training on the network model.
Preferably, the step 3 comprises the following steps:
step 3.1, testing the model, testing samples which do not participate in training, wherein each sample can obtain a 5-dimensional probability vector P:
P＝[P0，P1，P2，P3，P4]；
and 3.2, decoding the probability vector to obtain a prediction score S, and setting a ranking decoding method in lesion classification as follows:
S＝0.1*P0+0.15*P1+0.2*P2+0.25*P3+0.3*P4；
step 3.3, obtaining a prediction score: based on the data ratio of the data set, S is converted into a continuous lesion level prediction result with a decimal between 0 and 4 in the lesion classification prediction, the smallest 72% of samples in S are predicted to be 0 to 0.4, the next 7% are 0.5 to 1.4, 15% are 1.5 to 2.4, 3% are 2.5 to 3.4, and the remaining 3% are 3.5 to 4.
Preferably, the step 4 comprises the following steps:
step 4.1, cross expansion of data: performing cross expansion on data in the data set, obtaining 5 amplification samples at five positions, namely the upper position, the lower position, the left position, the right position and the middle position of the medical image, turning the image, and performing the same operation to obtain 10 samples in total;
step 4.2, calculating the error between the true label of the candidate sample and the prediction result: obtaining an error between a prediction result and a real label according to the real label of the medical image and a sample prediction score S obtained by previous calculation:
wherein y is a real label of the sample, i is a sample serial number, and j is an amplified sample serial number of the same sample;
4.3, calculating the diversity of a group of expansion samples obtained after the cross expansion, and calculating the prediction probability P obtained through a classification network to obtain:
Figure BDA0002231199510000052
wherein k is the class in which the sample is predicted;
and 4.4, calculating the value of the sample, and calculating the value p of the sample through the error and diversity of a group of expansion samples obtained before:
Figure BDA0002231199510000053
and 4.5, screening a certain number of candidate samples as a training set of incremental training, and selecting n expansion sample groups with the highest value as the training set according to p obtained by calculating each group of expansion samples.
Preferably, the step 5 comprises the following steps:
step 5.1, further cleaning noise data of the selected training set, calculating the mean value and the variance of the prediction score S of each group of samples, and rejecting samples which are more than or less than 1 standard deviation of the mean value as noise samples;
step 5.2, selecting the number of the expansion samples according to the K value, setting the K value according to the number of each type of samples, wherein the K value is the maximum number selected in each group of expansion samples, if the number of the remaining samples after cleaning in the step 5.1 is less than the K value, all the samples are selected, otherwise, the K samples are randomly selected, and the K value in the medical image data set for lesion classification is set as shown in the following table:
categories Total number of data Data ratio Value of K
Grade 0 (No DR) 63367 72.61％ 3
Grade 1 (mild NPDR) 6206 7.11％ 7
Grade 2 (moderate NPDR) 13155 15.07％ 4
Grade 3 (Severe NPDR) 2087 2.39％ 10
Grade 4 (PDR) 1914 2.21％ 10
Total of 87269 100％ -
。
Preferably, the step 6 comprises the following steps:
6.1, during incremental training, adding distillation terms to the model loss function, wherein the distillation terms are the loss functions of the models trained for the last two times, so as to reduce possible catastrophic forgetting of the models after multiple incremental training, wherein lambda is1Is set to 0.4, lambda2Set to 0.1:
lossfine-tune＝lossnew+λ1loss1+λ2loss2
therein, lossfine-tuneIs a loss function of incremental training, lossnewIs the loss function of origin, loss1Is the loss function in the last training, loss2Is the loss function during the last training;
and 6.2, performing incremental training on the model by taking the data finally obtained in the step 5.2 as a training set of incremental training and taking the loss function obtained in the step 6.1 as a loss function of model training.
Preferably, the step 7 comprises the following steps:
7.1, testing the accuracy of the model result after the incremental training;
7.2, if the difference of the prediction accuracy of the model in the last 3 times is less than 0.5%, determining that the model is stable, and ending the model training;
and 7.3, if the accuracy of the model is not stable, repeating the steps 4 to 7 until the model is stable.
The invention has the beneficial effects that:
aiming at various characteristics of medical images, the invention improves the AIFT (active learning enhancement method), uses a multi-task model to replace the traditional classification network model, and aims to solve the problems of difficult classification of the medical images, low training efficiency and the like caused by unbalanced data; by extracting the global features and the local focus information of the medical image respectively, the information in the image with serious lesion degree can be fully utilized, and the problem of poor application effect of deep learning in the field of lesion classification caused by insufficient and unbalanced data volume is solved; in the process of extracting the local focus information, the invention outputs the labeling result of the local focus information in the medical image, displays 4 key focuses influencing the lesion classification result for a doctor, solves the problem of low interpretable degree of the deep learning model result, and improves the auxiliary effect on the doctor to diagnose the disease condition.
Drawings
FIG. 1 is a general flow diagram of the active incremental training method for deep learning multi-class medical image classification according to the present invention;
FIG. 2 is a flowchart of step 1 of the active incremental training method for deep learning multi-class medical image classification according to the present invention;
fig. 3 is an expanded fundus image set obtained by cross expansion in step 4.1 of the active incremental training method for deep learning multi-class medical image classification according to the embodiment of the present invention.
Detailed Description
The invention is further illustrated by the following examples and figures:
example (b):
the present invention will be described specifically below, taking an active increment training method for classification of a diabetic fundus image as an example; diabetes mellitus is short for diabetic retinopathy, is an eye complication caused by diabetes mellitus, and is also an eye disease with high blindness rate.
As shown in fig. 1, the active incremental training method for deep learning multi-class medical image classification according to the present invention includes the following seven steps:
step 1, carrying out primary data cleaning and pretreatment on an eye fundus image data set; as shown in fig. 2, this step includes the following steps:
step 1.1, data cleaning: in the fundus image dataset, removing low-quality samples which can influence model training in the dataset;
step 1.2, image preprocessing: in the fundus image data set, sequentially carrying out green channel extraction and contrast enhancement on the image; the method comprises the following steps:
step 1.2.1, green channel extraction: extracting a green channel from the color fundus image;
step 1.2.2, contrast enhancement is carried out, and average brightness is kept;
step 1.2.3, central region extraction: the black background of the image is removed, and the central part of the image is extracted.
Step 2, randomly selecting initial data, and performing initial training on the network model; the method comprises the following steps:
2.1, randomly selecting initial data, selecting a certain number of samples as an initial training set of the model according to the original proportion of the samples among the categories in the data set, randomly selecting 5000 samples in the fundus image data set, and respectively selecting 3600, 350, 750, 150 and 150 samples in 5 categories;
step 2.2, performing initial training on the model: and taking 5000 samples as a training set, and carrying out initial training on the network model.
Step 3, testing other samples in the data set on the model trained in the step 2 to obtain a prediction probability, and obtaining a prediction score of the sample by using a ranking decoding method on the output probability to realize the correspondence between the prediction score and the diabetes mellitus lesion category; the method comprises the following steps:
step 3.1, testing the model, testing samples which do not participate in training, wherein each sample can obtain a 5-dimensional probability vector P:
P＝[P0，P1，P2，P3，P4]；
and 3.2, decoding the probability vector to obtain a prediction score S, wherein in the classification of the diabetes mellitus lesion, the set ranking decoding method comprises the following steps:
S＝0.1*P0+0.15*P1+0.2*P2+0.25*P3+0.3*P4；
step 3.3, obtaining a prediction score: based on the data ratio of the data set, S is converted into a continuous lesion level prediction result, which is a decimal between 0 and 4 in the classification prediction of diabetes mellitus lesions, the smallest 72% of samples in S are predicted to be 0 to 0.4, the next 7% is 0.5 to 1.4, 15% is 1.5 to 2.4, 3% is 2.5 to 3.4, and the remaining 3% is 3.5 to 4.
Step 4, performing cross expansion on the residual samples in the data set, and actively screening candidate samples according to the test result in the step 3; the method comprises the following steps:
step 4.1, cross expansion of data: performing cross expansion on data in the data set, obtaining 5 amplified samples at five positions, namely the upper position, the lower position, the left position, the right position and the middle position of the fundus image, turning the image, and performing the same operation to obtain 10 samples in total;
step 4.2, calculating the error between the true label of the candidate sample and the prediction result: obtaining an error between a prediction result and a real label according to the real label of the fundus image and a sample prediction score S obtained by calculation:
Figure BDA0002231199510000091
wherein y is a real label of the sample, i is a sample serial number, and j is an amplified sample serial number of the same sample;
4.3, calculating the diversity of a group of expansion samples obtained after the cross expansion, and calculating the prediction probability P obtained through a classification network to obtain:
wherein k is the class in which the sample is predicted;
and 4.4, calculating the value of the sample, and calculating the value p of the sample through the error and diversity of a group of expansion samples obtained before:
Figure BDA0002231199510000102
and 4.5, screening a certain number of candidate samples as a training set of incremental training, and selecting n expansion sample groups with the highest value as the training set according to p obtained by calculating each group of expansion samples.
Step 5, setting K values according to data proportions among different categories and carrying out further data set cleaning on the candidate samples obtained by screening in the step 4; the method comprises the following steps:
step 5.1, further cleaning noise data of the selected training set, calculating the mean value and the variance of the prediction score S of each group of samples, and rejecting samples which are more than or less than 1 standard deviation of the mean value as noise samples;
step 5.2, selecting the number of the expansion samples according to the K value, setting the K value according to the number of each type of samples, wherein the K value is the maximum number selected in each group of expansion samples, if the number of the residual samples after cleaning in the step 5.1 is less than the K value, all the expansion samples are selected, otherwise, the K samples are randomly selected, and the K value in the fundus image data set classified by the glycogenopathy lesion is set as shown in the following table:
categories Total number of data Data ratio Value of K
Grade 0 (No DR) 63367 72.61％ 3
Grade 1 (mild NPDR) 6206 7.11％ 7
Grade 2 (moderate NPDR) 13155 15.07％ 4
Grade 3 (Severe NPDR) 2087 2.39％ 10
Grade 4 (PDR) 1914 2.21％ 10
Total of 87269 100％ -
。
Step 6, performing incremental training on the model, and adding a distillation item in a loss function of the incremental training; the method comprises the following steps:
6.1, during incremental training, adding distillation terms to the model loss function, wherein the distillation terms are the loss functions of the models trained for the last two times, so as to reduce possible catastrophic forgetting of the models after multiple incremental training, wherein lambda is1Is set to 0.4, lambda2Set to 0.1:
lossfine-tune＝lossnew+λ1loss1+λ2loss2
therein, lossfine-tuneIs a loss function of incremental training, lossnewIs the loss function of origin, loss1Is the loss function in the last training, loss2Is the loss function during the last training;
and 6.2, performing incremental training on the model by taking the data finally obtained in the step 5.2 as a training set of incremental training and taking the loss function obtained in the step 6.1 as a loss function of model training.
Step 7, testing the model after the incremental training, wherein the testing method is the same as that in the step 3, testing the accuracy of the model, ending the training if the accuracy is stable, and otherwise, repeating the steps 4 to 7; the method comprises the following steps:
7.1, testing the accuracy of the model result after the incremental training;
7.2, if the difference of the prediction accuracy of the model in the last 3 times is less than 0.5%, determining that the model is stable, and ending the model training;
and 7.3, if the accuracy of the model is not stable, repeating the steps 4 to 7 until the model is stable.
The embodiment provides an improved AIFT algorithm based on a multi-task network aiming at the characteristics of the classification task of the diabetes mellitus lesion and the data balance problem of a data set, and according to experimental detection, the improved AIFT method can effectively reduce the time required by training, improves the training efficiency, and is more suitable for the classification task of the diabetes mellitus lesion than the AIFT method; and the multi-task network slightly improves the performance of the diabetes mellitus lesion classification task due to the addition of the regression task in the classification network, so that the network model designed in this chapter can obtain a better effect in a shorter training time. However, for the classification task of diabetic retinopathy with extremely high requirements on accuracy and sensitivity, the effect of the multitask model trained and completed by using the improved AIFT algorithm still cannot completely meet the requirement.
The above embodiments are only preferred embodiments of the present invention, and are not intended to limit the technical solutions of the present invention, so long as the technical solutions can be realized on the basis of the above embodiments without creative efforts, which should be considered to fall within the protection scope of the patent of the present invention.

Claims (9)

1. An active incremental training method for deep learning multi-category medical image classification is characterized in that: the method comprises the following steps:
step 1, carrying out primary data cleaning and preprocessing on a medical image data set;
step 2, randomly selecting initial data, and performing initial training on the network model;
step 3, testing the rest samples in the data set on the model trained in the step 2 to obtain a prediction probability, and obtaining a prediction score of the sample by using a ranking decoding method on the output probability to realize the correspondence between the prediction score and the lesion category;
step 4, performing cross expansion on the residual samples in the data set, and actively screening candidate samples according to the test result in the step 3;
step 5, setting K values according to data proportions among different categories and carrying out further data set cleaning on the candidate samples obtained by screening in the step 4;
step 6, performing incremental training on the model, and adding a distillation item in a loss function of the incremental training;
and 7, testing the model after the incremental training, wherein the testing method is the same as that in the step 3, testing the accuracy of the model, finishing the training if the accuracy is stable, and otherwise, repeating the steps 4 to 7.
2. The active incremental training method for deep learning multi-class medical image classification as claimed in claim 1, wherein: the step 1 comprises the following steps:
step 1.1, data cleaning: in the medical image data set, low-quality samples which can influence model training in the data set are removed;
step 1.2, image preprocessing: in the medical image data set, the green channel extraction and contrast enhancement operations are sequentially performed on the image.
3. The active incremental training method for deep learning multi-class medical image classification as claimed in claim 2, wherein: the step 1.2 comprises the following steps:
step 1.2.1, green channel extraction: extracting a green channel from the color medical image;
step 1.2.2, contrast enhancement is carried out, and average brightness is kept;
step 1.2.3, central region extraction: the black background of the image is removed, and the central part of the image is extracted.
4. The active incremental training method for deep learning multi-class medical image classification as claimed in claim 1, wherein: the step 2 comprises the following steps:
2.1, randomly selecting initial data, selecting a certain number of samples as an initial training set of the model according to the original proportion of samples among the categories in the data set, randomly selecting 5000 samples in the medical image data set, and respectively selecting 3600, 350, 750, 150 and 150 samples in 5 categories;
step 2.2, performing initial training on the model: and taking 5000 samples as a training set, and carrying out initial training on the network model.
5. The active incremental training method for deep learning multi-class medical image classification as claimed in claim 1, wherein: the step 3 comprises the following steps:
step 3.1, testing the model, testing samples which do not participate in training, wherein each sample can obtain a 5-dimensional probability vector P:
P＝[P0，P1，P2，P3，P4]；
and 3.2, decoding the probability vector to obtain a prediction score S, and setting a ranking decoding method in lesion classification as follows:
S＝0.1*P0+0.15*P1+0.2*P2+0.25*P3+0.3*P4；
step 3.3, obtaining a prediction score: based on the data ratio of the data set, S is converted into a continuous lesion level prediction result with a decimal between 0 and 4 in the lesion classification prediction, the smallest 72% of samples in S are predicted to be 0 to 0.4, the next 7% are 0.5 to 1.4, 15% are 1.5 to 2.4, 3% are 2.5 to 3.4, and the remaining 3% are 3.5 to 4.
6. The active incremental training method for deep learning multi-class medical image classification as claimed in claim 1, wherein: the step 4 comprises the following steps:
step 4.1, cross expansion of data: performing cross expansion on data in the data set, obtaining 5 amplification samples at five positions, namely the upper position, the lower position, the left position, the right position and the middle position of the medical image, turning the image, and performing the same operation to obtain 10 samples in total;
step 4.2, calculating the error between the true label of the candidate sample and the prediction result: obtaining an error between a prediction result and a real label according to the real label of the medical image and a sample prediction score S obtained by previous calculation:
wherein y is a real label of the sample, i is a sample serial number, and j is an amplified sample serial number of the same sample;
4.3, calculating the diversity of a group of expansion samples obtained after the cross expansion, and calculating the prediction probability P obtained through a classification network to obtain:
Figure FDA0002231199500000032
wherein k is the class in which the sample is predicted;
and 4.4, calculating the value of the sample, and calculating the value p of the sample through the error and diversity of a group of expansion samples obtained before:
Figure FDA0002231199500000033
and 4.5, screening a certain number of candidate samples as a training set of incremental training, and selecting n expansion sample groups with the highest value as the training set according to p obtained by calculating each group of expansion samples.
7. The active incremental training method for deep learning multi-class medical image classification as claimed in claim 1, wherein: the step 5 comprises the following steps:
step 5.1, further cleaning noise data of the selected training set, calculating the mean value and the variance of the prediction score S of each group of samples, and rejecting samples which are more than or less than 1 standard deviation of the mean value as noise samples;
step 5.2, selecting the number of the expansion samples according to the K value, setting the K value according to the number of each type of samples, wherein the K value is the maximum number selected in each group of expansion samples, if the number of the remaining samples after cleaning in the step 5.1 is less than the K value, all the samples are selected, otherwise, the K samples are randomly selected, and the K value in the medical image data set for lesion classification is set as shown in the following table:
categories Total number of data Data ratio Value of K Grade 0 (No DR) 63367 72.61％ 3 Grade 1 (mild NPDR) 6206 7.11％ 7 Grade 2 (moderate NPDR) 13155 15.07％ 4 Grade 3 (Severe NPDR) 2087 2.39％ 10 Grade 4 (PDR) 1914 2.21％ 10 Total of 87269 100％ -
。
8. The active incremental training method for deep learning multi-class medical image classification as claimed in claim 7, wherein: the step 6 comprises the following steps:
6.1, during incremental training, adding distillation terms to the model loss function, wherein the distillation terms are the loss functions of the models trained for the last two times, so as to reduce possible catastrophic forgetting of the models after multiple incremental training, wherein lambda is1Is set to 0.4, lambda2Set to 0.1:
lossfine-tune＝lossnew+λ1loss1+λ2loss2
therein, lossfine-tuneIs a loss function of incremental training, lossnewIs the loss function of origin, loss1Is the loss function in the last training, loss2Is the loss function during the last training;
and 6.2, performing incremental training on the model by taking the data finally obtained in the step 5.2 as a training set of incremental training and taking the loss function obtained in the step 6.1 as a loss function of model training.
9. The active incremental training method for deep learning multi-class medical image classification as claimed in claim 1, wherein: the step 7 comprises the following steps:
7.1, testing the accuracy of the model result after the incremental training;
7.2, if the difference of the prediction accuracy of the model in the last 3 times is less than 0.5%, determining that the model is stable, and ending the model training;
and 7.3, if the accuracy of the model is not stable, repeating the steps 4 to 7 until the model is stable.
CN201910968185.8A 2019-10-12 2019-10-12 Active incremental training method for deep learning of multi-class medical image classification Pending CN110689089A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910968185.8A CN110689089A (en) 2019-10-12 2019-10-12 Active incremental training method for deep learning of multi-class medical image classification

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910968185.8A CN110689089A (en) 2019-10-12 2019-10-12 Active incremental training method for deep learning of multi-class medical image classification

Publications (1)

Publication Number Publication Date
CN110689089A true CN110689089A (en) 2020-01-14

Family

ID=69112546

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910968185.8A Pending CN110689089A (en) 2019-10-12 2019-10-12 Active incremental training method for deep learning of multi-class medical image classification

Country Status (1)

Country Link
CN (1) CN110689089A (en)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111430025A (en) * 2020-03-10 2020-07-17 清华大学 Disease diagnosis method based on medical image data amplification
CN111932534A (en) * 2020-09-22 2020-11-13 平安科技（深圳）有限公司 Medical image picture analysis method and device, electronic equipment and readable storage medium
CN112733963A (en) * 2021-02-01 2021-04-30 中国人民解放军海军航空大学航空作战勤务学院 General image target classification method and system
CN112926674A (en) * 2021-03-19 2021-06-08 广东好太太智能家居有限公司 Image classification prediction method and device based on support vector machine model
CN112991346A (en) * 2021-05-13 2021-06-18 深圳科亚医疗科技有限公司 Training method and training system for learning network for medical image analysis
CN113077001A (en) * 2021-04-07 2021-07-06 西南大学 Medical image classification system based on generative tensor network
CN113139568A (en) * 2021-02-22 2021-07-20 杭州深睿博联科技有限公司 Class prediction model modeling method and device based on active learning
CN113139612A (en) * 2021-05-07 2021-07-20 上海商汤临港智能科技有限公司 Image classification method, training method of classification network and related products
WO2021147217A1 (en) * 2020-01-20 2021-07-29 平安科技（深圳）有限公司 Medical image classification method and apparatus, and medium and electronic device
CN113555092A (en) * 2020-04-24 2021-10-26 辉达公司 Image annotation using one or more neural networks
CN113807528A (en) * 2020-06-16 2021-12-17 阿里巴巴集团控股有限公司 Model optimization method, device and storage medium
CN115908964A (en) * 2022-09-20 2023-04-04 国药(武汉)医学实验室有限公司 Medical image classification method, system, terminal and storage medium
CN116580268A (en) * 2023-07-11 2023-08-11 腾讯科技（深圳）有限公司 Training method of image target positioning model, image processing method and related products
CN117974672A (en) * 2024-04-02 2024-05-03 华侨大学 Method and device for detecting breast ultrasonic tumor lesion area based on global information

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108021916A (en) * 2017-12-31 2018-05-11 南京航空航天大学 Deep learning diabetic retinopathy sorting technique based on notice mechanism

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108021916A (en) * 2017-12-31 2018-05-11 南京航空航天大学 Deep learning diabetic retinopathy sorting technique based on notice mechanism

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
刘江明: ""眼底图像的糖网病变分类研究与实现"", 《万方学位论文》 *

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021147217A1 (en) * 2020-01-20 2021-07-29 平安科技（深圳）有限公司 Medical image classification method and apparatus, and medium and electronic device
CN111430025A (en) * 2020-03-10 2020-07-17 清华大学 Disease diagnosis method based on medical image data amplification
CN111430025B (en) * 2020-03-10 2023-07-25 清华大学 Disease diagnosis model training method based on medical image data augmentation
CN113555092A (en) * 2020-04-24 2021-10-26 辉达公司 Image annotation using one or more neural networks
CN113807528A (en) * 2020-06-16 2021-12-17 阿里巴巴集团控股有限公司 Model optimization method, device and storage medium
CN111932534B (en) * 2020-09-22 2021-07-06 平安科技（深圳）有限公司 Medical image picture analysis method and device, electronic equipment and readable storage medium
CN111932534A (en) * 2020-09-22 2020-11-13 平安科技（深圳）有限公司 Medical image picture analysis method and device, electronic equipment and readable storage medium
CN112733963B (en) * 2021-02-01 2023-02-21 中国人民解放军海军航空大学航空作战勤务学院 General image target classification method and system
CN112733963A (en) * 2021-02-01 2021-04-30 中国人民解放军海军航空大学航空作战勤务学院 General image target classification method and system
CN113139568A (en) * 2021-02-22 2021-07-20 杭州深睿博联科技有限公司 Class prediction model modeling method and device based on active learning
CN113139568B (en) * 2021-02-22 2022-05-10 杭州深睿博联科技有限公司 Class prediction model modeling method and device based on active learning
CN112926674A (en) * 2021-03-19 2021-06-08 广东好太太智能家居有限公司 Image classification prediction method and device based on support vector machine model
CN113077001A (en) * 2021-04-07 2021-07-06 西南大学 Medical image classification system based on generative tensor network
CN113139612A (en) * 2021-05-07 2021-07-20 上海商汤临港智能科技有限公司 Image classification method, training method of classification network and related products
CN112991346B (en) * 2021-05-13 2022-04-26 深圳科亚医疗科技有限公司 Training method and training system for learning network for medical image analysis
CN112991346A (en) * 2021-05-13 2021-06-18 深圳科亚医疗科技有限公司 Training method and training system for learning network for medical image analysis
CN115908964A (en) * 2022-09-20 2023-04-04 国药(武汉)医学实验室有限公司 Medical image classification method, system, terminal and storage medium
CN115908964B (en) * 2022-09-20 2023-12-12 国药(武汉)医学实验室有限公司 Medical image classification method, system, terminal and storage medium
CN116580268A (en) * 2023-07-11 2023-08-11 腾讯科技（深圳）有限公司 Training method of image target positioning model, image processing method and related products
CN116580268B (en) * 2023-07-11 2023-10-03 腾讯科技（深圳）有限公司 Training method of image target positioning model, image processing method and related products
CN117974672A (en) * 2024-04-02 2024-05-03 华侨大学 Method and device for detecting breast ultrasonic tumor lesion area based on global information

Similar Documents

Publication Publication Date Title
CN110689089A (en) Active incremental training method for deep learning of multi-class medical image classification
Li et al. Automatic detection of diabetic retinopathy in retinal fundus photographs based on deep learning algorithm
CN110992382B (en) Fundus image optic cup optic disc segmentation method and system for assisting glaucoma screening
CN108806792B (en) Deep learning face diagnosis system
CN112288706B (en) Automatic chromosome karyotype analysis and abnormality detection method
Harangi et al. Automatic screening of fundus images using a combination of convolutional neural network and hand-crafted features
CN110930418B (en) Retina blood vessel segmentation method fusing W-net and conditional generation confrontation network
CN112132817B (en) Retina blood vessel segmentation method for fundus image based on mixed attention mechanism
CN109086799A (en) A kind of crop leaf disease recognition method based on improvement convolutional neural networks model AlexNet
CN111680706A (en) Double-channel output contour detection method based on coding and decoding structure
US20180165413A1 (en) Gene expression data classification method and classification system
Luo et al. Retinal image classification by self-supervised fuzzy clustering network
CN114038037B (en) Expression label correction and identification method based on separable residual error attention network
CN113610118A (en) Fundus image classification method, device, equipment and medium based on multitask course learning
CN112435264A (en) 42CrMo single-phase metallographic structure segmentation method and system based on deep learning
CN115358337A (en) Small sample fault diagnosis method and device and storage medium
CN114882497A (en) Method for realizing fruit classification and identification based on deep learning algorithm
BAKIR et al. Detection of pneumonia from x-ray images using deep learning techniques
Zhang et al. Blood vessel segmentation in fundus images based on improved loss function
CN110458846B (en) Cell image segmentation method based on graph path search and deep learning
CN113011436A (en) Traditional Chinese medicine tongue color and fur color collaborative classification method based on convolutional neural network
CN112633301A (en) Traditional Chinese medicine tongue image greasy feature classification method based on depth metric learning
CN117036288A (en) Tumor subtype diagnosis method for full-slice pathological image
Chen et al. Chromosome segmentation via data simulation and shape learning
CN115661185A (en) Fundus image blood vessel segmentation method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20200114

WD01 Invention patent application deemed withdrawn after publication