CN113160119A - Diabetic retinopathy image classification method based on deep learning - Google Patents
Diabetic retinopathy image classification method based on deep learning Download PDFInfo
- Publication number
- CN113160119A CN113160119A CN202110159376.7A CN202110159376A CN113160119A CN 113160119 A CN113160119 A CN 113160119A CN 202110159376 A CN202110159376 A CN 202110159376A CN 113160119 A CN113160119 A CN 113160119A
- Authority
- CN
- China
- Prior art keywords
- model
- image
- diabetic retinopathy
- neural network
- network model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 206010012689 Diabetic retinopathy Diseases 0.000 title claims abstract description 63
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000013135 deep learning Methods 0.000 title claims abstract description 27
- 206010012601 diabetes mellitus Diseases 0.000 claims abstract description 39
- 238000003062 neural network model Methods 0.000 claims abstract description 36
- 238000007781 pre-processing Methods 0.000 claims abstract description 10
- 238000010606 normalization Methods 0.000 claims abstract description 5
- 238000012549 training Methods 0.000 claims description 30
- 238000011176 pooling Methods 0.000 claims description 15
- 238000010586 diagram Methods 0.000 claims description 10
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 9
- 230000000694 effects Effects 0.000 claims description 8
- 238000012360 testing method Methods 0.000 claims description 8
- 230000003321 amplification Effects 0.000 claims description 6
- 238000013528 artificial neural network Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 201000010099 disease Diseases 0.000 claims description 6
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 6
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 6
- 208000017442 Retinal disease Diseases 0.000 claims description 5
- 206010038923 Retinopathy Diseases 0.000 claims description 5
- 239000008280 blood Substances 0.000 claims description 4
- 210000004369 blood Anatomy 0.000 claims description 4
- 238000003708 edge detection Methods 0.000 claims description 4
- 230000006870 function Effects 0.000 claims description 3
- 230000003902 lesion Effects 0.000 claims description 3
- 210000002569 neuron Anatomy 0.000 claims description 3
- 201000007914 proliferative diabetic retinopathy Diseases 0.000 claims description 3
- 230000007704 transition Effects 0.000 claims description 3
- 238000003745 diagnosis Methods 0.000 abstract description 4
- 230000002265 prevention Effects 0.000 abstract description 2
- 238000001514 detection method Methods 0.000 description 5
- 201000004569 Blindness Diseases 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 208000009857 Microaneurysm Diseases 0.000 description 2
- 230000000740 bleeding effect Effects 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 231100000915 pathological change Toxicity 0.000 description 2
- 230000036285 pathological change Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000004393 visual impairment Effects 0.000 description 2
- 206010002329 Aneurysm Diseases 0.000 description 1
- 208000031226 Hyperlipidaemia Diseases 0.000 description 1
- 206010020772 Hypertension Diseases 0.000 description 1
- 208000006550 Mydriasis Diseases 0.000 description 1
- 206010038862 Retinal exudates Diseases 0.000 description 1
- 206010047571 Visual impairment Diseases 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 238000013399 early diagnosis Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 210000000416 exudates and transudate Anatomy 0.000 description 1
- 201000001421 hyperglycemia Diseases 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000002062 proliferating effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 208000029257 vision disease Diseases 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/60—Rotation of whole images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Mathematical Physics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Software Systems (AREA)
- Evolutionary Biology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Medical Informatics (AREA)
- Probability & Statistics with Applications (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a diabetic retinopathy image classification method based on deep learning, which improves the quality of an input network image after carrying out data set preprocessing operations such as normalization, image preprocessing, data set expansion and the like on a data set, and trains the processed data set by utilizing a deep learning neural network model to generate a diabetic retinopathy diagnosis model. Compared with other methods, the network model has higher recognition rate and proper algorithm complexity, can reduce the phenomenon of misdiagnosis caused by human factors, can greatly shorten the time for diagnosing the diabetic retinopathy, and has important significance for early prevention and treatment of patients with the diabetes mellitus.
Description
Technical Field
The invention relates to the technical field of deep learning, in particular to a diabetic retinopathy image classification method based on deep learning.
Background
Diabetic Retinopathy (DR), also known as diabetes mellitus, is an ocular manifestation of diabetes mellitus and is a leading cause of visual impairment and blindness in the world today. Researches prove that hyperglycemia, hypertension, hyperlipidemia and diabetes are important risk factors for diabetic retinopathy. According to WHO data statistics, the number of Chinese diabetics reaches 1.1 hundred million, and the Chinese diabetics are the countries with the largest number of diabetics. The prevalence of diabetic retinopathy in the diabetic population is 65%, which means that 6700 million people may suffer from diabetes mellitus in China. If the diabetes mellitus is not treated in time, the vision of the patient is sharply reduced after the diabetes mellitus develops to the later stage, and finally the patient is blinded, so that the normal life of the patient is seriously influenced. According to studies, the longer the period of diabetes, the higher the incidence of diabetic retinopathy, but the 90% lower the incidence of vision loss or blindness in patients with diabetes mellitus if effective early diagnosis and treatment is performed.
The traditional method for clinically diagnosing diabetic retinopathy is to perform comprehensive ophthalmologic detection including visual sensitivity, slit lamp, mydriasis and the like, and has a complex detection process and needs to consume certain manpower and material resources. Since the diabetic retinopathy has various focus types, if the fundus images can be automatically classified in the early screening process, a large amount of diagnosis time can be saved for clinicians, so that the research of realizing the diagnosis of the diabetes retinopathy by means of a machine learning and deep learning method is paid attention to by people.
At present, the precision of classifying the fundus images of patients by using a machine learning method is high, but most of machine learning algorithms at present need high-quality images as data sets, and classification and labeling of the fundus images by experienced ophthalmologists are needed when image data are acquired, so that a large amount of preparation work is still needed in the early stage of model building. Meanwhile, the method for diagnosing diabetic retinopathy by using deep learning is also rapidly developed, and most of the existing work is to indirectly detect the pathological changes by segmenting blood vessels and optic discs in fundus pictures. With the continuous increase of the scale and the depth of the deep learning network model, the accuracy of the deep learning network model in an image recognition task is rapidly improved, but the small medical image data set is the main reason for poor recognition effect of the deep learning method in the field.
At present, the detection of diabetic retinopathy mainly depends on the manual judgment of doctors, the work is time-consuming and labor-consuming, and the requirements on the related experience of the doctors are high. In the face of a large number of diabetics, the detection efficiency is urgently needed to be greatly improved, and the detection time is shortened.
Disclosure of Invention
The technical problem to be solved by the present invention is to provide a diabetic retinopathy image classification method based on deep learning, aiming at the defects of the prior art.
The technical scheme adopted by the invention for solving the technical problems is as follows: a diabetic retinopathy image classification method based on deep learning is constructed, and comprises the following steps:
acquiring diabetic retinopathy image data, preprocessing the image, and dividing a training set and a test set;
combining a ResNet neural network model and a DenseNet neural network model, constructing a deep neural network model, inputting diabetic retinopathy image data serving as a training set into the deep neural network model for training, adjusting network parameters and functions until an output result is an accurate classification result, inputting the diabetic retinopathy image data serving as a test set into the trained deep neural network model after training is finished, and verifying the accuracy of the deep neural network model;
and inputting the data of the diabetic retinopathy image shot in real time into the trained deep neural network model, and classifying the input diabetic retinopathy image by using the output result through calculation of the network model.
In the step of collecting the diabetic retinopathy image data, extracting high-resolution color images of two eyes of a plurality of diabetic patients, dividing the images into five grades of normal, mild, moderate, severe and proliferative diabetic retinopathy according to the degree of the retinopathy, and marking the grades by 0-4, wherein 0 corresponds to a normal health state, and 4 is the most serious state.
The image preprocessing mode at least comprises the operations of image denoising, image filtering, feature enhancement, normalization and data amplification on the marked high-resolution color image data of the two eyes of the diabetic patient.
In the step of carrying out image filtering on the high-resolution color image data of both eyes of the diabetic patient, the characteristics of the eyeground blood spots in the diabetic retinopathy image are enhanced in an edge detection algorithm mode, so that the accuracy of model identification is improved.
Wherein, in the step of performing feature enhancement on the high-resolution color image data of both eyes of the diabetic patient, the method comprises the following steps:
subtracting a Gaussian blur (GaussianBlur) image from an original image of a high-resolution color image of both eyes of a diabetic patient to obtain the difference between the two images;
and obtaining the final characteristic-enhanced diabetic patient binocular high-resolution color image by using GaussianBlur algorithm.
Wherein in the step of normalizing the diabetic patient binocular high resolution color image data, the data enhanced diabetic patient binocular high resolution color image is fixed to a resolution of 512x 512.
In the step of data amplification of the high-resolution color image data of both eyes of the diabetic patient, the images are zoomed, rotated, overturned and changed in brightness on the basis of not changing the disease grade of the original data set image, so that the quantity of the images in the diabetic retinopathy image data sets of all the disease grades is equal.
The deep neural network model for image classification is combined with the characteristics of a ResNet neural network model and a DenseNet neural network model, and comprises 5 blocks including a Dense Block representing a Dense Block and a Resnet Block representing a residual Block, and a transition layer is arranged between each Block for convolution nuclear pooling; the model input is pictures with the resolution of 512x512, and the pictures are classified through a Softmax classifier;
inputting a training set into a model, performing convolution operation on a picture, wherein the size of a convolution kernel is 7x7, the step length is 2, the length and the width of an output after the convolution operation are both 256, and the length and the width of an output feature map after a pooling layer are 128x128 after the maximum pooling operation; after convolutional encoding, the sample image is input into a dense block followed by a conversion layer for performing convolution kernel average pooling, and after passing through the dense block and the conversion layer, the length and width of the model output picture are both 16x 16; and further adopting a method of adding a residual block, wherein after the residual block, the output dimensionality of the model is 8x8x1024, performing average pooling on the output, then connecting the output with a layer of 1024 neurons, and finally classifying through a sofmax classifier to output a lesion classification result of the diabetic retinopathy image.
The classification effect of various identification networks is objectively evaluated by adopting two indexes of model accuracy and space complexity:
the indexes for evaluating the classification effect of the model have accuracy and average accuracy, and the calculation formulas are respectively as follows:
wherein m iscorrectRepresents the number of correctly classified samples, mtotalRepresenting the number of samples tested in total; n is the number of model classification categories, AccuracyiRepresenting the recognition accuracy of the ith category;
the quality of the model is evaluated by the spatial complexity of the model, the spatial complexity of the model determines the number of parameters of the model, the more the model parameters are, the larger the data size required by the training model is, and when the training data size is not enough to match the model parameters, the overfitting problem of the training model can be caused; the model space complexity is composed of two parts, namely the total parameter number of the model and the space occupation of each layer of output characteristic diagram, and the calculation formula is as follows:
d, K, M respectively represents the convolution layer number, the characteristic diagram size and the convolution kernel size of the convolution neural network, n represents the convolution layer label of the convolution neural network, C represents the output channel number of the convolution layer and is equal to the output channel number of the last convolution layer, the first term in the formula represents the total parameter number of the neural network model, and the second term represents the space occupation of the output characteristic diagram.
The invention provides a diabetic retinopathy image classification method based on deep learning, which improves the quality of an input network image after carrying out data set preprocessing operations such as normalization, image preprocessing, data set expansion and the like on a data set, and trains the processed data set by using a deep learning neural network model to generate a diabetic retinopathy diagnosis model. Compared with other methods, the network model has higher recognition rate and proper algorithm complexity, can reduce the phenomenon of misdiagnosis caused by human factors, can greatly shorten the time for diagnosing the diabetic retinopathy, and has important significance for early prevention and treatment of patients with the diabetes mellitus.
Drawings
The invention will be further described with reference to the accompanying drawings and examples, in which:
FIG. 1 is a schematic flow chart of a method for classifying images of diabetic retinopathy based on deep learning according to the present invention.
FIG. 2 is a schematic diagram of an architecture of a deep neural network model in the method for classifying diabetic retinopathy image based on deep learning according to the present invention.
Detailed Description
For a more clear understanding of the technical features, objects and effects of the present invention, embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
As shown in fig. 1, the present invention provides a method for classifying diabetic retinopathy image based on deep learning, comprising:
acquiring diabetic retinopathy image data, preprocessing the image, and dividing a training set and a test set;
combining a ResNet neural network model and a DenseNet neural network model, constructing a deep neural network model, inputting diabetic retinopathy image data serving as a training set into the deep neural network model for training, adjusting network parameters and functions until an output result is an accurate classification result, inputting the diabetic retinopathy image data serving as a test set into the trained deep neural network model after training is finished, and verifying the accuracy of the deep neural network model;
and inputting the data of the diabetic retinopathy image shot in real time into the trained deep neural network model, and classifying the input diabetic retinopathy image by using the output result through calculation of the network model.
In the step of collecting the diabetic retinopathy image data, the high-resolution color images of the two eyes of a plurality of diabetic patients are extracted, and the fundus image resolutions of the left eye and the fundus image resolution of the right eye are the same. The retinopathy is divided into five grades of normal, mild, moderate, severe and proliferative type diabetic retinopathy according to the degree of the retinopathy, and the pathological changes are mainly characterized by microaneurysms, hard exudation, cotton wool spots, bleeding spots and the like. And is labeled with a rating of 0-4, where 0 corresponds to a normal health state and 4 is the most severe state.
The image preprocessing mode at least comprises the operations of image denoising, image filtering, feature enhancement, normalization and data amplification on the marked high-resolution color image data of the two eyes of the diabetic patient.
In the step of carrying out image filtering on the high-resolution color image data of both eyes of the diabetic patient, the characteristics of the eyeground blood spots in the diabetic retinopathy image are enhanced in an edge detection algorithm mode, so that the accuracy of model identification is improved.
However, training tests show that the edge detection algorithm recognizes dust or other optical problems on the lens as blood spots, and the healthy original image is processed to recognize the proliferative diabetic retinopathy. Therefore, in order to improve the accuracy of model identification, it is necessary to avoid using the image subjected to the filter processing.
Wherein, in the step of performing feature enhancement on the high-resolution color image data of both eyes of the diabetic patient, the method comprises the following steps:
subtracting a Gaussian blur (GaussianBlur) image from an original image of a high-resolution color image of both eyes of a diabetic patient to obtain the difference between the two images;
and obtaining the final characteristic-enhanced diabetic patient binocular high-resolution color image by using GaussianBlur algorithm.
Bleeding points and hard exudate characteristics in the image become clear and visible, microaneurysm characteristics are amplified, and useful characteristics can be maximally learned by the classification model so as to help the model obtain a better training result.
Wherein in the step of normalizing the diabetic patient binocular high resolution color image data, the data enhanced diabetic patient binocular high resolution color image is fixed to a resolution of 512x 512.
The data set is analyzed, and images from the training and testing data sets are found to have different resolutions, aspect ratios, colors, and various cropping modes, and some data images have very poor quality and are out of focus. Training the data set image by using the deep convolutional neural network requires a fixed input dimension, so that the size of the fundus image needs to be adjusted to enable all data sets to have a fixed resolution. If the resolution of the image is too low, the detailed features of the fundus image are lost, and if the resolution is too high, the training cost of the model is increased, and the image is fixed to the resolution of 512x512 through experiments, so that the detailed features of the fundus image can be ensured on one hand, and the training speed of the model can be increased on the other hand.
In the step of data amplification of the high-resolution color image data of both eyes of the diabetic patient, the images are zoomed, rotated, overturned and changed in brightness on the basis of not changing the disease grade of the original data set image, so that the quantity of the images in the diabetic retinopathy image data sets of all the disease grades is equal. After data amplification, the influence on the categories with a large number of samples is small, and the categories with a small number are expanded and filled up to the categories with a large number.
As shown in fig. 2, the deep neural network model for image classification combines the characteristics of the ResNet neural network model and the DenseNet neural network model, and includes 5 blocks including a density Block representing a Dense Block and a ResNet Block representing a residual Block, and a transition layer is arranged between each Block for convolution kernel pooling; the model input is pictures with the resolution of 512x512, and the pictures are classified through a Softmax classifier;
inputting a training set into a model, performing convolution operation on a picture, wherein the size of a convolution kernel is 7x7, the step length is 2, the length and the width of an output after the convolution operation are both 256, and the length and the width of an output feature map after a pooling layer are 128x128 after the maximum pooling operation; after convolutional encoding, the sample image is input into a dense block followed by a conversion layer for performing convolution kernel average pooling, and after passing through the dense block and the conversion layer, the length and width of the model output picture are both 16x 16; and further adopting a method of adding a residual block, wherein after the residual block, the output dimensionality of the model is 8x8x1024, performing average pooling on the output, then connecting the output with a layer of 1024 neurons, and finally classifying through a sofmax classifier to output a lesion classification result of the diabetic retinopathy image.
The network model structure is shown in table 1:
TABLE 1 network model architecture
The classification effect of various identification networks is objectively evaluated by adopting two indexes of model accuracy and space complexity:
the indexes for evaluating the classification effect of the model have accuracy and average accuracy, and the calculation formulas are respectively as follows:
wherein m iscorrectRepresents the number of correctly classified samples, mtotalRepresenting the number of samples tested in total; n is the number of model classification categories, AccuracyiRepresenting the recognition accuracy of the ith category;
the quality of the model is evaluated by the spatial complexity of the model, the spatial complexity of the model determines the number of parameters of the model, the more the model parameters are, the larger the data size required by the training model is, and when the training data size is not enough to match the model parameters, the overfitting problem of the training model can be caused; the model space complexity is composed of two parts, namely the total parameter number of the model and the space occupation of each layer of output characteristic diagram, and the calculation formula is as follows:
d, K, M respectively represents the convolution layer number, the characteristic diagram size and the convolution kernel size of the convolution neural network, n represents the convolution layer label of the convolution neural network, C represents the output channel number of the convolution layer and is equal to the output channel number of the last convolution layer, the first term in the formula represents the total parameter number of the neural network model, and the second term represents the space occupation of the output characteristic diagram.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (9)
1. A diabetic retinopathy image classification method based on deep learning is characterized by comprising the following steps:
acquiring diabetic retinopathy image data, preprocessing the image, and dividing a training set and a test set;
combining a ResNet neural network model and a DenseNet neural network model, constructing a deep neural network model, inputting diabetic retinopathy image data serving as a training set into the deep neural network model for training, adjusting network parameters and functions until an output result is an accurate classification result, inputting the diabetic retinopathy image data serving as a test set into the trained deep neural network model after training is finished, and verifying the accuracy of the deep neural network model;
and inputting the data of the diabetic retinopathy image shot in real time into the trained deep neural network model, and classifying the input diabetic retinopathy image by using the output result through calculation of the network model.
2. The deep learning-based diabetic retinopathy image classification method according to claim 1, wherein in the step of collecting diabetic retinopathy image data, binocular high resolution color images of a plurality of diabetic patients are extracted, classified into five grades of normal, mild, moderate, severe and proliferative diabetic retinopathy according to the degree of retinopathy, and the grades are identified by 0 to 4, wherein 0 corresponds to a normal healthy state and 4 is the most severe state.
3. The deep learning-based diabetic retinopathy image classification method according to claim 2, wherein the image preprocessing at least comprises the operations of image denoising, image filtering, feature enhancement, normalization and data amplification of the labeled diabetic patient binocular high-resolution color image data.
4. The diabetic retinopathy image classification method based on deep learning of claim 3, wherein in the step of image filtering the high-resolution color image data of both eyes of the diabetic patient, the characteristics of the fundus blood spots in the diabetic retinopathy image are enhanced by adopting an edge detection algorithm so as to improve the accuracy of model identification.
5. The diabetic retinopathy image classification method based on deep learning of claim 3, wherein the step of performing feature enhancement on the high-resolution color image data of both eyes of the diabetic patient comprises the steps of:
subtracting a Gaussian blur (GaussianBlur) image from an original image of a high-resolution color image of both eyes of a diabetic patient to obtain the difference between the two images;
and obtaining the final characteristic-enhanced diabetic patient binocular high-resolution color image by using GaussianBlur algorithm.
6. The deep learning-based diabetic retinopathy image classification method according to claim 5, wherein in the step of normalizing the diabetic patient's binocular high-resolution color image data, the data-enhanced diabetic patient's binocular high-resolution color image is fixed to a resolution of 512x 512.
7. The method for classifying diabetic retinopathy based on deep learning of claim 3, wherein in the step of data expansion of the diabetic patient's binocular high resolution color image data, the operations of scaling, rotating, flipping and changing brightness are performed on the images without changing the disease level of the original data set image, so that the number of images in the diabetic retinopathy image data sets of each disease level is equal.
8. The deep learning-based diabetic retinopathy image classification method according to claim 1, characterized in that the deep neural network model for image classification combines the characteristics of the ResNet neural network model and the DenseNet neural network model, and comprises 5 blocks including density Block representing Dense blocks and ResNet Block representing residual blocks, and a transition layer is arranged between each Block for convolution nuclear pooling; the model input is pictures with the resolution of 512x512, and the pictures are classified through a Softmax classifier;
inputting a training set into a model, performing convolution operation on a picture, wherein the size of a convolution kernel is 7x7, the step length is 2, the length and the width of an output after the convolution operation are both 256, and the length and the width of an output feature map after a pooling layer are 128x128 after the maximum pooling operation; after convolutional encoding, the sample image is input into a dense block followed by a conversion layer for performing convolution kernel average pooling, and after passing through the dense block and the conversion layer, the length and width of the model output picture are both 16x 16; further adopting a method of adding a residual block, after the residual block, the output dimension of the model is 8x8x1024, performing average pooling on the output, then connecting the output with a layer of 1024 neurons, and finally passing through sofmax
And classifying by the classifier, and outputting a lesion classification result of the diabetic retinopathy image.
9. The diabetic retinopathy image classification method based on deep learning of claim 1 is characterized in that two indexes of model accuracy and spatial complexity are adopted to objectively evaluate the classification effect of various identification networks:
the indexes for evaluating the classification effect of the model have accuracy and average accuracy, and the calculation formulas are respectively as follows:
wherein m iscorrectRepresents the number of correctly classified samples, mtotalRepresenting the number of samples tested in total; n is the number of model classification categories, AccuracyiRepresenting the recognition accuracy of the ith category;
the quality of the model is evaluated by the spatial complexity of the model, the spatial complexity of the model determines the number of parameters of the model, the more the model parameters are, the larger the data size required by the training model is, and when the training data size is not enough to match the model parameters, the overfitting problem of the training model can be caused; the model space complexity is composed of two parts, namely the total parameter number of the model and the space occupation of each layer of output characteristic diagram, and the calculation formula is as follows:
d, K, M respectively represents the convolution layer number, the characteristic diagram size and the convolution kernel size of the convolution neural network, n represents the convolution layer label of the convolution neural network, C represents the output channel number of the convolution layer and is equal to the output channel number of the last convolution layer, the first term in the formula represents the total parameter number of the neural network model, and the second term represents the space occupation of the output characteristic diagram.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110159376.7A CN113160119A (en) | 2021-02-04 | 2021-02-04 | Diabetic retinopathy image classification method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110159376.7A CN113160119A (en) | 2021-02-04 | 2021-02-04 | Diabetic retinopathy image classification method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113160119A true CN113160119A (en) | 2021-07-23 |
Family
ID=76882765
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110159376.7A Pending CN113160119A (en) | 2021-02-04 | 2021-02-04 | Diabetic retinopathy image classification method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113160119A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114494196A (en) * | 2022-01-26 | 2022-05-13 | 南通大学 | Retina diabetic depth network detection method based on genetic fuzzy tree |
| CN115063383A (en) * | 2022-06-29 | 2022-09-16 | 北京理工大学 | Bright red mole segmentation method and device based on multi-color space adaptive fusion |
| CN116823760A (en) * | 2023-06-25 | 2023-09-29 | 深圳市眼科医院(深圳市眼病防治研究所) | Automatic identification method and system for retinopathy treatment mode of premature infant |
| CN117877692A (en) * | 2024-01-02 | 2024-04-12 | 珠海全一科技有限公司 | Personalized difference analysis method for retinopathy |
| CN117936079A (en) * | 2024-03-21 | 2024-04-26 | 中国人民解放军总医院第三医学中心 | Manifold learning-based diabetic retinopathy identification method, medium and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018201633A1 (en) * | 2017-05-04 | 2018-11-08 | 深圳硅基仿生科技有限公司 | Fundus image-based diabetic retinopathy identification system |
| CN109691979A (en) * | 2019-01-07 | 2019-04-30 | 哈尔滨理工大学 | A kind of diabetic retina image lesion classification method based on deep learning |
| CN109948719A (en) * | 2019-03-26 | 2019-06-28 | 天津工业大学 | A kind of eye fundus image quality automatic classification method based on the intensive module network structure of residual error |
| CN110210570A (en) * | 2019-06-10 | 2019-09-06 | 上海延华大数据科技有限公司 | The more classification methods of diabetic retinopathy image based on deep learning |
| CN111260551A (en) * | 2020-01-08 | 2020-06-09 | 华南理工大学 | Retina super-resolution reconstruction system and method based on deep learning |
-
2021
- 2021-02-04 CN CN202110159376.7A patent/CN113160119A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018201633A1 (en) * | 2017-05-04 | 2018-11-08 | 深圳硅基仿生科技有限公司 | Fundus image-based diabetic retinopathy identification system |
| CN109691979A (en) * | 2019-01-07 | 2019-04-30 | 哈尔滨理工大学 | A kind of diabetic retina image lesion classification method based on deep learning |
| CN109948719A (en) * | 2019-03-26 | 2019-06-28 | 天津工业大学 | A kind of eye fundus image quality automatic classification method based on the intensive module network structure of residual error |
| CN110210570A (en) * | 2019-06-10 | 2019-09-06 | 上海延华大数据科技有限公司 | The more classification methods of diabetic retinopathy image based on deep learning |
| CN111260551A (en) * | 2020-01-08 | 2020-06-09 | 华南理工大学 | Retina super-resolution reconstruction system and method based on deep learning |
Non-Patent Citations (2)
| Title |
|---|
| 李琼;柏正尧;刘莹芳;: "糖尿病性视网膜图像的深度学习分类方法", 中国图象图形学报, no. 10 * |
| 杜霞;: "基于深度卷积网络的糖尿病性视网膜病变分类", 现代计算机(专业版), no. 11 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114494196A (en) * | 2022-01-26 | 2022-05-13 | 南通大学 | Retina diabetic depth network detection method based on genetic fuzzy tree |
| CN114494196B (en) * | 2022-01-26 | 2023-11-17 | 南通大学 | Retinal diabetes mellitus depth network detection method based on genetic fuzzy tree |
| CN115063383A (en) * | 2022-06-29 | 2022-09-16 | 北京理工大学 | Bright red mole segmentation method and device based on multi-color space adaptive fusion |
| CN116823760A (en) * | 2023-06-25 | 2023-09-29 | 深圳市眼科医院(深圳市眼病防治研究所) | Automatic identification method and system for retinopathy treatment mode of premature infant |
| CN117877692A (en) * | 2024-01-02 | 2024-04-12 | 珠海全一科技有限公司 | Personalized difference analysis method for retinopathy |
| CN117936079A (en) * | 2024-03-21 | 2024-04-26 | 中国人民解放军总医院第三医学中心 | Manifold learning-based diabetic retinopathy identification method, medium and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113160119A (en) | Diabetic retinopathy image classification method based on deep learning | |
| CN108615051B (en) | Diabetic retina image classification method and system based on deep learning | |
| US10413180B1 (en) | System and methods for automatic processing of digital retinal images in conjunction with an imaging device | |
| SujithKumar et al. | Automatic detection of diabetic retinopathy in non-dilated RGB retinal fundus images | |
| CN110648344B (en) | Diabetes retinopathy classification device based on local focus characteristics | |
| CN112837805B (en) | Eyelid topological morphology feature extraction method based on deep learning | |
| CN111785363A (en) | AI-guidance-based chronic disease auxiliary diagnosis system | |
| Liu et al. | Segmenting nailfold capillaries using an improved U-net network | |
| Yao et al. | Generic features for fundus image quality evaluation | |
| CN114343563A (en) | Method, device and system for assisting dry eye diagnosis and typing through multi-modal fusion | |
| Su et al. | Superficial punctate keratitis grading for dry eye screening using deep convolutional neural networks | |
| Muddamsetty et al. | Multi-level quality assessment of retinal fundus images using deep convolution neural networks | |
| CN114549520A (en) | Retina pathological section analysis system based on full convolution attention enhancement network | |
| Consejo et al. | Detection of subclinical keratoconus with a validated alternative method to corneal densitometry | |
| Phridviraj et al. | A bi-directional Long Short-Term Memory-based Diabetic Retinopathy detection model using retinal fundus images | |
| Chakrabarty et al. | A novel approach to glaucoma screening using computer vision | |
| Triyadi et al. | Deep learning in image classification using vgg-19 and residual networks for cataract detection | |
| CN114334124A (en) | Pathological myopia detection system based on deep neural network | |
| Sampson et al. | Evaluation of focus and deep learning methods for automated image grading and factors influencing image quality in adaptive optics ophthalmoscopy | |
| CN111292285A (en) | Automatic screening method for diabetes mellitus based on naive Bayes and support vector machine | |
| Kanakatte et al. | A decision support system for retinal image defect detection | |
| CN111291706B (en) | Retina image optic disc positioning method | |
| Resita et al. | Color RGB and structure GLCM method to feature extraction system in endoscope image for the diagnosis support of otitis media disease | |
| Azeroual et al. | Convolutional Neural Network for Segmentation and Classification of Glaucoma. | |
| Gilchrist | Computer processing of ocular photographs—A review |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210723 |
|
| RJ01 | Rejection of invention patent application after publication |