CN116152229B - Method for constructing diabetic retinopathy diagnosis model and diagnosis model - Google Patents

Method for constructing diabetic retinopathy diagnosis model and diagnosis model Download PDF

Info

Publication number
CN116152229B
CN116152229B CN202310397924.9A CN202310397924A CN116152229B CN 116152229 B CN116152229 B CN 116152229B CN 202310397924 A CN202310397924 A CN 202310397924A CN 116152229 B CN116152229 B CN 116152229B
Authority
CN
China
Prior art keywords
loss function
relative position
model
image
prediction result
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310397924.9A
Other languages
Chinese (zh)
Other versions
CN116152229A (en
Inventor
欧阳继红
刘思光
李成溪
孟庆奕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jilin University
Original Assignee
Jilin University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jilin University filed Critical Jilin University
Priority to CN202310397924.9A priority Critical patent/CN116152229B/en
Publication of CN116152229A publication Critical patent/CN116152229A/en
Application granted granted Critical
Publication of CN116152229B publication Critical patent/CN116152229B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30041—Eye; Retina; Ophthalmic
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Quality & Reliability (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a method for constructing a diabetic retinopathy diagnosis model and the diagnosis model, and belongs to the technical field of deep learning. The construction method comprises the following steps: acquiring a fundus image and preprocessing to obtain an input image; inputting the input image into a backbone network for feature extraction to obtain a feature image containing advanced features; designing an improved classifier, inputting the feature map into the improved classifier, and obtaining a classification prediction result and a regression prediction result through calculation; designing a relative position loss function additional term, wherein the classification prediction result in the previous step is used for calculating a cross entropy loss function, and the regression prediction result is used for calculating the relative position loss function additional term; and combining the cross entropy loss function and the relative position loss function additional term to obtain a final joint loss function so as to construct a diagnosis model. The model constructed by the invention can learn the continuity information of the diabetic retinopathy process, and the model performance is obviously improved.

Description

Method for constructing diabetic retinopathy diagnosis model and diagnosis model
Technical Field
The invention relates to the technical field of deep learning, in particular to a method for constructing a diabetic retinopathy diagnosis model and the diagnosis model.
Background
In recent years, diabetic retinopathy (Diabetic Retinopathy, abbreviated as DR) is one of the most common diabetic ophthalmic disease complications, and is a preventable disease. In clinic, the eye specialist with high experience is mainly relied on to observe and evaluate the color fundus images to determine the disease stage grade, wherein the pathology stage grade adopts DR international five stage standard to make an optimized diagnosis and treatment scheme. The deep learning can effectively extract pathological features in the image according to the marked data and obtain a better diagnosis result, but the actual situation can face a plurality of challenges.
The problem of DR staging has a significant feature in that the disease progression is continuous. However, the existing DR stage method based on deep learning generally regards the problem as a classification task with discrete categories, and ignores the continuity attribute of the lesion process. This corresponds to discarding a significant portion of the prior knowledge during modeling, which undoubtedly negatively affects the performance of the model. The main reason for this is that the model end classifier cannot directly model and learn the continuity information of the labels in the training data due to the structural limitation of the model end classifier.
Disclosure of Invention
Aiming at the problems, the invention aims to provide a method for constructing a diabetic retinopathy diagnosis model and the diagnosis model, and aims to improve the performance of the model by designing a brand-new classifier structure and a loss function so that the model can learn the continuity information of a DR pathological change process.
In order to solve the technical problems, the invention provides the following scheme:
in one aspect, a method for constructing a diabetic retinopathy diagnostic model is provided, comprising the steps of:
acquiring a fundus image and preprocessing to obtain an input image;
inputting the input image into a backbone network for feature extraction to obtain a feature image containing advanced features;
designing an improved classifier, inputting the feature map into the improved classifier, and obtaining a classification prediction result and a regression prediction result through calculation;
designing a relative position loss function additional term, wherein the classification prediction result in the previous step is used for calculating a cross entropy loss function, and the regression prediction result in the previous step is used for calculating the relative position loss function additional term;
and combining the cross entropy loss function and the relative position loss function additional term to obtain a final joint loss function so as to construct a diagnosis model.
Preferably, the step of preprocessing the fundus image specifically includes:
firstly, removing black areas around fundus images;
secondly, the sizes of fundus images are regulated uniformly, and the resolution of the regulated images is 256 multiplied by 256;
finally, fundus image enhancement is performed using the following formula to improve the brightness and contrast differences of fundus images:
Figure SMS_1
（1）
Figure SMS_2
（2）
in the formula (1),
Figure SMS_4
for the preprocessed fundus image, +.>
Figure SMS_7
Representing standard deviation +.>
Figure SMS_9
Is a gaussian convolution of (c) and,
Figure SMS_5
representing pixel points in the fundus image; equation (2) is a weighted sum of pre-and post-enhancement fundus images, wherein the parameters used +.>
Figure SMS_6
, />
Figure SMS_8
, />
Figure SMS_10
,/>
Figure SMS_3
4, -4, 10 and 128, respectively.
Preferably, a ResNet50 network containing ImageNet pre-training parameters is selected as the backbone network for the encoder to construct the diagnostic model.
Preferably, the process of designing an improved classifier is as follows:
feature graphs obtained via backbone network are fed into full connection layer and vector representation with dimension of 6 is output
Figure SMS_11
The components are
Figure SMS_12
The method comprises the steps of carrying out a first treatment on the surface of the Wherein->
Figure SMS_13
After Softmax the probability of i for the input sample label is obtained +.>
Figure SMS_14
，/>
Figure SMS_15
The method comprises the steps of carrying out a first treatment on the surface of the The Softmax calculation procedure is as follows:
Figure SMS_16
(3)
classification results given to input samples
Figure SMS_17
Obtained by subscript of the largest element in P, the formula is:
Figure SMS_18
(4)
Figure SMS_19
obtaining +.>
Figure SMS_20
This process is shown in equation (5):
Figure SMS_21
(5)
here, the
Figure SMS_22
Consider as the regression prediction value of the input sample; />
Figure SMS_23
The greater the value representing the greater the extent of pathology of the sample; considering the value range of the input sample label of 0-4, for the convenience of calculation, the predicted value +.>
Figure SMS_24
Amplified to get->
Figure SMS_25
：
Figure SMS_26
(6)
Figure SMS_27
The regression prediction value is regarded as the regression prediction value of the lesion degree of the input sample.
Preferably, the process of designing the relative position loss function additional term is as follows:
assume any two input samples of a training set
Figure SMS_30
And->
Figure SMS_32
Their label is->
Figure SMS_34
And->
Figure SMS_29
Regression prediction of lesion extentThe value is +.>
Figure SMS_31
And->
Figure SMS_33
Their tag distance and predicted value distance are +.>
Figure SMS_35
And->
Figure SMS_28
The method comprises the steps of carrying out a first treatment on the surface of the To avoid interference of the symbol to the model training process, the absolute values of the two distances are taken when the distance difference is calculated:
Figure SMS_36
(7)
likewise, consider cancellation
Figure SMS_37
The influence of the sign of (2) on the model is verified by experiments, and the final selection pair is +>
Figure SMS_38
Taking the square as the relative position loss;
for the model training process, one batch contains
Figure SMS_39
In the case of several input samples, the distance difference between any two input samples is calculated, and +.>
Figure SMS_40
Secondary times; and then averaged as an additional term to the relative position loss function of the batch
Figure SMS_41
：
Figure SMS_42
(8)
The final joint loss function consists of the cross entropy loss function and the relative position loss function additional term together, and is specifically shown as follows:
Figure SMS_43
(9)
wherein the method comprises the steps of
Figure SMS_44
For a super parameter adjusting the ratio of the two loss functions, +.>
Figure SMS_45
Is a cross entropy loss function.
In another aspect, a diagnostic model of diabetic retinopathy is provided, comprising:
the preprocessing unit is used for preprocessing the acquired fundus image to obtain an input image;
the backbone network unit is used for extracting the characteristics of the input image to obtain a characteristic image containing advanced characteristics;
the improved classifier unit is used for calculating the feature map by utilizing the improved classifier to respectively obtain a classification prediction result and a regression prediction result;
and the loss function unit is used for calculating a cross entropy loss function by using the classification prediction result, calculating a relative position loss function additional term by using the regression prediction result, and combining the two to obtain a final joint loss function.
Preferably, the preprocessing unit is specifically configured to:
firstly, removing black areas around fundus images;
secondly, the sizes of fundus images are regulated uniformly, and the resolution of the regulated images is 256 multiplied by 256;
finally, fundus image enhancement is performed using the following formula to improve the brightness and contrast differences of fundus images:
Figure SMS_46
（1）
Figure SMS_47
（2）
in the formula (1),
Figure SMS_48
for the preprocessed fundus image, +.>
Figure SMS_52
Representing standard deviation +.>
Figure SMS_54
Is a gaussian convolution of (c) and,
Figure SMS_49
representing pixel points in the fundus image; equation (2) is a weighted sum of pre-and post-enhancement fundus images, wherein the parameters used +.>
Figure SMS_51
, />
Figure SMS_53
, />
Figure SMS_55
,/>
Figure SMS_50
4, -4, 10 and 128, respectively.
Preferably, in the backbone network unit, a ResNet50 network containing ImageNet pre-training parameters is selected as a backbone network of the encoder forming the diagnostic model.
Preferably, in the improved classifier unit, the feature map obtained via the backbone network is fed into the fully connected layer, and a vector representation with dimension of 6 is output
Figure SMS_56
The component is->
Figure SMS_57
The method comprises the steps of carrying out a first treatment on the surface of the Wherein->
Figure SMS_58
After Softmax the probability of i for the input sample label is obtained +.>
Figure SMS_59
，/>
Figure SMS_60
The method comprises the steps of carrying out a first treatment on the surface of the The Softmax calculation procedure is as follows:
Figure SMS_61
(3)
classification results given to input samples
Figure SMS_62
Obtained by subscript of the largest element in P, the formula is:
Figure SMS_63
(4)
Figure SMS_64
after Sigmoid function, get +.>
Figure SMS_65
This process is shown in equation (5):
Figure SMS_66
(5)
here, the
Figure SMS_67
Consider as the regression prediction value of the input sample; />
Figure SMS_68
The greater the value representing the greater the extent of pathology of the sample; considering the value range of the input sample label of 0-4, for the convenience of calculation, the predicted value +.>
Figure SMS_69
Amplified to get->
Figure SMS_70
：
Figure SMS_71
(6)
Figure SMS_72
The regression prediction value is regarded as the regression prediction value of the lesion degree of the input sample.
Preferably, in the loss function unit, the relative position loss function additional term is designed as follows:
assume any two input samples of a training set
Figure SMS_73
And->
Figure SMS_76
Their label is->
Figure SMS_78
And->
Figure SMS_74
Regression prediction value of lesion degree is +.>
Figure SMS_77
And->
Figure SMS_79
Their tag distance and predicted value distance are +.>
Figure SMS_80
And->
Figure SMS_75
The method comprises the steps of carrying out a first treatment on the surface of the To avoid interference of the symbol to the model training process, the absolute values of the two distances are taken when the distance difference is calculated:
Figure SMS_81
(7)
likewise, consider cancellation
Figure SMS_82
The influence of the sign of (2) on the model is verified by experiments, and the final selection pair is +>
Figure SMS_83
Taking the square as the relative position loss;
for the model training process, one batch contains
Figure SMS_84
In the case of several input samples, the distance difference between any two input samples is calculated, and +.>
Figure SMS_85
Secondary times; and then averaged as an additional term to the relative position loss function of the batch
Figure SMS_86
：
Figure SMS_87
(8)
The final joint loss function consists of the cross entropy loss function and the relative position loss function additional term together, and is specifically shown as follows:
Figure SMS_88
(9)
wherein the method comprises the steps of
Figure SMS_89
For a super parameter adjusting the ratio of the two loss functions, +.>
Figure SMS_90
Is a cross entropy loss function.
Compared with the prior art, the technical scheme provided by the invention has the following beneficial effects:
according to the method for constructing the diabetic retinopathy diagnosis model and the diagnosis model, provided by the invention, the model can learn the continuity information of the DR pathological change process by designing a brand-new classifier structure and a loss function, so that the performance of the model is obviously improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a general flow chart of a method of constructing a diagnostic model of diabetic retinopathy;
FIGS. 2 a-2 c are contrast plots showing fundus image preprocessing before and after preprocessing;
fig. 3 is a schematic diagram of the structure of a double-ended classifier.
While particular structures have been shown for purposes of clarity and understanding of the embodiments of the invention, it is not intended to limit the invention to the particular structures and environments disclosed, and modifications within the scope of the invention will occur to those skilled in the art.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. It will be apparent that the described embodiments are some, but not all, embodiments of the invention. All other embodiments, which can be made by a person skilled in the art without creative efforts, based on the described embodiments of the present invention fall within the protection scope of the present invention.
The embodiment of the invention firstly provides a method for constructing a diabetic retinopathy diagnosis model, and referring to fig. 1, the method comprises the following steps:
s1, acquiring a fundus image and preprocessing to obtain an input image;
s2, inputting the input image into a backbone network for feature extraction to obtain a feature image containing advanced features;
s3, designing an improved classifier, inputting the feature map into the improved classifier, and obtaining a classification prediction result and a regression prediction result through calculation;
s4, designing a relative position loss function additional item, wherein the classification prediction result in the previous step is used for calculating a cross entropy loss function, and the regression prediction result in the previous step is used for calculating the relative position loss function additional item;
s5, combining the cross entropy loss function and the relative position loss function additional term to obtain a final joint loss function so as to construct a diagnosis model.
In the present embodiment, this newly constructed diagnostic model is designated as RL-ResNet. The RL-res net is mainly composed of a backbone network and an improved double-end classifier, and after the preprocessed fundus image is input into the backbone network, features are extracted through the backbone network, and become a set of feature images containing advanced features, and the feature images are sent into the improved classifier. In the classifier, the feature map is calculated to obtain a classification prediction result and a regression prediction result, and the two results are used for calculating a cross entropy loss function and a relative position loss function addition term designed by the invention, wherein the cross entropy loss function and the relative position loss function addition term are used for modeling continuous information in the label. The two are combined to obtain a final joint loss function so as to realize the training of model parameters.
In particular, since fundus images have differences in illumination conditions and photographing apparatuses during sampling, there is a great difference in size and color between pictures. Meanwhile, some pictures have the problems of overexposure, more noise and the like. In order to train a network using samples of consistent color range, pre-processing and enhancement of the acquired fundus image is required.
In step S1, the step of preprocessing the fundus image specifically includes:
firstly, removing black areas around fundus images; black areas around the fundus image have a great negative influence on the classification result, and therefore need to be removed;
secondly, the sizes of fundus images are adjusted uniformly, and the resolution of the images after adjustment in the embodiment of the invention is 256×256;
finally, fundus image enhancement is performed using the following formula to improve the brightness and contrast differences of fundus images:
Figure SMS_91
（1）
Figure SMS_92
（2）
in the formula (1),
Figure SMS_93
for the preprocessed fundus image, +.>
Figure SMS_96
Representing standard deviation +.>
Figure SMS_98
Is a gaussian convolution of (c) and,
Figure SMS_95
representing pixel points in the fundus image; equation (2) is a weighted sum of pre-and post-enhancement fundus images, wherein the parameters used +.>
Figure SMS_97
, />
Figure SMS_99
, />
Figure SMS_100
,/>
Figure SMS_94
4, -4, 10 and 128, respectively.
Fig. 2 a-2 c show the preprocessing results of three different fundus images. As can be seen from the figure, by the above pretreatment, the black areas around the fundus image become gray, reducing the influence on the classification result. The sharpness of blood vessels and lesion areas and the contrast with surrounding areas are significantly improved.
Further, in the embodiment of the invention, a ResNet50 network containing ImageNet pre-training parameters is selected as a backbone network of the diagnostic model formed by the encoder.
The ResNet50 network has proven to be very efficient in extracting image high-level semantic information, which is critical to medical image tasks. Therefore, the invention selects the ResNet50 network containing the ImageNet pre-training parameters as an encoder to form the backbone network for providing the diagnosis model, so that the backbone network has better initialization parameters and the convergence speed of the backbone network in the training process is accelerated.
The improved classifier designed in the invention is a double-end classifier, and the structure of the classifier is shown in figure 3.
Feature graphs obtained via backbone network are fed into full connection layer and vector representation with dimension of 6 is output
Figure SMS_101
The components are
Figure SMS_102
The method comprises the steps of carrying out a first treatment on the surface of the Wherein->
Figure SMS_103
After Softmax the probability of i for the input sample label is obtained +.>
Figure SMS_104
，/>
Figure SMS_105
The method comprises the steps of carrying out a first treatment on the surface of the The Softmax calculation procedure is as follows:
Figure SMS_106
(3)
classification results given to input samples
Figure SMS_107
Obtained by subscript of the largest element in P, the formula is:
Figure SMS_108
(4)
existing deep learning-based DR staging methods typically use P-dependent cross entropy loss functions for network training, a process that does not require the use of
Figure SMS_109
。/>
Figure SMS_110
Calculation is only required when using a trained model. If one wants to model tag continuity information, one needs to use +.>
Figure SMS_111
Related information. But the gradient information of the corresponding term of the loss function cannot be transferred to the shallow layer of the model through the process shown in the formula (4). That is, if use +.>
Figure SMS_112
Modeling tag continuity information and constructing corresponding loss function additional items, wherein the part of information cannot influence the training process of the model, namely, the newly added additional items are invalid. The invention thus envisages +_ in figure 3>
Figure SMS_113
Corresponding structure.
In FIG. 3
Figure SMS_114
Obtaining +.>
Figure SMS_115
This process is shown in equation (5):
Figure SMS_116
(5)
here, the
Figure SMS_117
I.e. can be regarded as regression predictions of the input samples; />
Figure SMS_118
The greater the value representing the greater the extent of pathology of the sample; for example->
Figure SMS_119
Corresponding to DR0 stage, and->
Figure SMS_120
Then the corresponding DR4 stage.
Considering the value range of the input sample label of 0-4, for the convenience of calculation, the predicted value is calculated
Figure SMS_121
Amplified to get->
Figure SMS_122
：
Figure SMS_123
(6)
Figure SMS_124
The regression prediction value is regarded as the regression prediction value of the lesion degree of the input sample. For example->
Figure SMS_125
Indicating that the extent of lesions in the input samples is between DR2 and DR3, closer to DR2.
In the embodiment of the invention, the paired sample regression prediction value is usedThe relative distance between the two is used for modeling the lesion continuity and counting a loss function additional term, namely a relative position loss function additional term (Relative Position Loss, short for RePLoss) used for
Figure SMS_126
And (3) representing.
The process of designing the relative position loss function additional term is as follows:
assume any two input samples of a training set
Figure SMS_128
And->
Figure SMS_131
Their label is->
Figure SMS_133
And->
Figure SMS_129
Regression prediction value of lesion degree is +.>
Figure SMS_130
And->
Figure SMS_132
Their tag distance and predicted value distance are +.>
Figure SMS_134
And->
Figure SMS_127
The method comprises the steps of carrying out a first treatment on the surface of the To avoid interference of the symbol to the model training process, the absolute values of the two distances are taken when the distance difference is calculated:
Figure SMS_135
(7)
likewise, consider cancellation
Figure SMS_136
Influence of the sign of (2) on the modelExperiments prove that the final selection pair +.>
Figure SMS_137
The relative position loss is obtained as a square.
For the model training process, a Batch (Batch) contains
Figure SMS_138
In the case of several input samples, the distance difference between any two input samples is calculated, and +.>
Figure SMS_139
Secondary times; then taking the average as a relative position loss function additional term for this batch +.>
Figure SMS_140
：
Figure SMS_141
(8)
The final joint loss function consists of the cross entropy loss function and the relative position loss function additional term together, and is specifically shown as follows:
Figure SMS_142
(9)
wherein the method comprises the steps of
Figure SMS_143
For a super parameter adjusting the ratio of the two loss functions, +.>
Figure SMS_144
Is a cross entropy loss function.
The calculation procedure for a single sample is as follows:
Figure SMS_145
(10)
Figure SMS_146
onehot, which is a sample tag, indicates that when the sample belongs to category i, i.e. the sample tag is i,/>
Figure SMS_150
Otherwise->
Figure SMS_151
；/>
Figure SMS_147
Is classified as +.>
Figure SMS_148
Probability of individual categories. For a batch +.>
Figure SMS_152
Sample number->
Figure SMS_153
For each of them->
Figure SMS_149
Average value of (2):
Figure SMS_154
(11)
aiming at the defects of the prior art, the invention designs a brand-new classifier structure and a loss function, so that the model can learn the DR pathological change process continuity information, thereby obviously improving the model performance.
Accordingly, embodiments of the present invention also provide a diagnostic model of diabetic retinopathy, the diagnostic model comprising:
the preprocessing unit is used for preprocessing the acquired fundus image to obtain an input image;
the backbone network unit is used for extracting the characteristics of the input image to obtain a characteristic image containing advanced characteristics;
the improved classifier unit is used for calculating the feature map by utilizing the improved classifier to respectively obtain a classification prediction result and a regression prediction result;
and the loss function unit is used for calculating a cross entropy loss function by using the classification prediction result, calculating a relative position loss function additional term by using the regression prediction result, and combining the two to obtain a final joint loss function.
Further, the preprocessing unit is specifically configured to:
firstly, removing black areas around fundus images;
secondly, the sizes of fundus images are regulated uniformly, and the resolution of the regulated images is 256 multiplied by 256;
finally, fundus image enhancement is performed using the following formula to improve the brightness and contrast differences of fundus images:
Figure SMS_155
（1）
Figure SMS_156
（2）
in the formula (1),
Figure SMS_157
for the preprocessed fundus image, +.>
Figure SMS_160
Representing standard deviation +.>
Figure SMS_162
Is a gaussian convolution of (c) and,
Figure SMS_159
representing pixel points in the fundus image; equation (2) is a weighted sum of pre-and post-enhancement fundus images, wherein the parameters used +.>
Figure SMS_161
, />
Figure SMS_163
, />
Figure SMS_164
,/>
Figure SMS_158
4, -4, 10 and 128, respectively.
Further, in the backbone network unit, a ResNet50 network containing ImageNet pre-training parameters is selected as a backbone network of the diagnostic model formed by the encoder.
Further, in the improved classifier unit, the feature map obtained through the backbone network is sent to the full connection layer, and a vector representation with 6 dimensions is output
Figure SMS_165
The component is->
Figure SMS_166
The method comprises the steps of carrying out a first treatment on the surface of the Wherein->
Figure SMS_167
After Softmax the probability of i for the input sample label is obtained +.>
Figure SMS_168
，/>
Figure SMS_169
The method comprises the steps of carrying out a first treatment on the surface of the The Softmax calculation procedure is as follows:
Figure SMS_170
(3)
classification results given to input samples
Figure SMS_171
Obtained by subscript of the largest element in P, the formula is:
Figure SMS_172
(4)
Figure SMS_173
obtaining +.>
Figure SMS_174
This process is shown in equation (5):
Figure SMS_175
(5)
here, the
Figure SMS_176
Consider as the regression prediction value of the input sample; />
Figure SMS_177
The greater the value representing the greater the extent of pathology of the sample; considering the value range of the input sample label of 0-4, for the convenience of calculation, the predicted value +.>
Figure SMS_178
Amplified to get->
Figure SMS_179
：
Figure SMS_180
(6)
Figure SMS_181
The regression prediction value is regarded as the regression prediction value of the lesion degree of the input sample.
Further, in the loss function unit, the relative position loss function additional term is designed as follows:
assume any two input samples of a training set
Figure SMS_182
And->
Figure SMS_186
Their labels are/>
Figure SMS_188
And->
Figure SMS_184
Regression prediction value of lesion degree is +.>
Figure SMS_185
And->
Figure SMS_187
Their tag distance and predicted value distance are +.>
Figure SMS_189
And->
Figure SMS_183
The method comprises the steps of carrying out a first treatment on the surface of the To avoid interference of the symbol to the model training process, the absolute values of the two distances are taken when the distance difference is calculated:
Figure SMS_190
(7)
likewise, consider cancellation
Figure SMS_191
The influence of the sign of (2) on the model is verified by experiments, and the final selection pair is +>
Figure SMS_192
The relative position loss is obtained as a square.
For the model training process, one batch contains
Figure SMS_193
In the case of several input samples, the distance difference between any two input samples is calculated, and +.>
Figure SMS_194
Secondary times; and then averaged as an additional term to the relative position loss function of the batch
Figure SMS_195
：
Figure SMS_196
(8)
The final joint loss function consists of the cross entropy loss function and the relative position loss function additional term together, and is specifically shown as follows:
Figure SMS_197
(9)
wherein the method comprises the steps of
Figure SMS_198
For a super parameter adjusting the ratio of the two loss functions, +.>
Figure SMS_199
Is a cross entropy loss function.
The calculation procedure for a single sample is as follows:
Figure SMS_200
(10)
Figure SMS_202
onehot, which is a sample tag, indicates that when the sample belongs to category i, i.e. the sample tag is i,/>
Figure SMS_205
Otherwise->
Figure SMS_207
；/>
Figure SMS_203
Is classified as +.>
Figure SMS_204
Probability of individual categories. For a batch +.>
Figure SMS_206
Sample number->
Figure SMS_208
For each of them
Figure SMS_201
Average value of (2):
Figure SMS_209
(11)
according to the diagnosis model provided by the invention, through designing a brand-new classifier structure and a loss function, the model can learn the DR pathological change process continuity information, so that the performance of the model is obviously improved.
Embodiments of the present invention also provide an electronic device that may vary greatly in configuration or performance, and may include one or more processors (central processing units, CPU) and one or more memories, where the memories store at least one instruction that is loaded and executed by the processors to implement the steps of the method for constructing a diabetic retinopathy diagnostic model described above.
In an exemplary embodiment, a computer readable storage medium, such as a memory comprising instructions executable by a processor in a terminal to perform the method of constructing a diabetic retinopathy diagnostic model described above, is also provided. For example, the computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal device comprising the element.
References in the specification to "one embodiment," "an example embodiment," "some embodiments," etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the relevant art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
The invention is intended to cover any alternatives, modifications, equivalents, and variations that fall within the spirit and scope of the invention. In the following description of preferred embodiments of the invention, specific details are set forth in order to provide a thorough understanding of the invention, and the invention will be fully understood to those skilled in the art without such details. In other instances, well-known methods, procedures, flows, components, circuits, and the like have not been described in detail so as not to unnecessarily obscure aspects of the present invention.
Those of ordinary skill in the art will appreciate that all or a portion of the steps in implementing the methods of the embodiments described above may be implemented by a program that instructs associated hardware, and the program may be stored on a computer readable storage medium, such as: ROM/RAM, magnetic disks, optical disks, etc.
The foregoing description of the preferred embodiments of the invention is not intended to limit the invention to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the invention are intended to be included within the scope of the invention.

Claims (6)

1. The method for constructing the diabetic retinopathy diagnosis model is characterized by comprising the following steps of:
acquiring a fundus image and preprocessing to obtain an input image;
inputting the input image into a backbone network for feature extraction to obtain a feature image containing advanced features;
designing an improved classifier, inputting the feature map into the improved classifier, and obtaining a classification prediction result and a regression prediction result through calculation;
designing a relative position loss function additional term, wherein the classification prediction result in the previous step is used for calculating a cross entropy loss function, and the regression prediction result in the previous step is used for calculating the relative position loss function additional term;
the cross entropy loss function and the relative position loss function additional term are combined to obtain a final joint loss function so as to realize the construction of a diagnosis model;
the process of designing the improved classifier is as follows:
feature graphs obtained via backbone network are fed into full connection layer and vector representation with dimension of 6 is output
Figure QLYQS_1
The component is->
Figure QLYQS_2
The method comprises the steps of carrying out a first treatment on the surface of the Wherein->
Figure QLYQS_3
Through the process ofSoftmaxObtaining the input sample label asiProbability of->
Figure QLYQS_4
，/>
Figure QLYQS_5
；SoftmaxThe calculation process is as follows:
Figure QLYQS_6
（3）
classification results given to input samples
Figure QLYQS_7
From the following componentsPThe subscript of the largest element in (2) is obtained, and the formula is as follows:
Figure QLYQS_8
（4）
Figure QLYQS_9
through the process ofSigmoidAfter the function get +.>
Figure QLYQS_10
This process is shown in equation (5):
Figure QLYQS_11
（5）
here, the
Figure QLYQS_12
Consider as the regression prediction value of the input sample; />
Figure QLYQS_13
The greater the value representing the greater the extent of pathology of the sample; considering the value range of the input sample label of 0-4, for the convenience of calculation, the predicted value +.>
Figure QLYQS_14
Amplified to get->
Figure QLYQS_15
：
Figure QLYQS_16
（6）
Figure QLYQS_17
The regression prediction value is regarded as the regression prediction value of the lesion degree of the input sample;
the process of designing the relative position loss function additional term is as follows:
assume any two input samples of a training set
Figure QLYQS_19
And->
Figure QLYQS_22
Their label is->
Figure QLYQS_24
And->
Figure QLYQS_20
Regression prediction value of lesion degree is +.>
Figure QLYQS_21
And->
Figure QLYQS_23
Their tag distance and predicted value distance are +.>
Figure QLYQS_25
And->
Figure QLYQS_18
The method comprises the steps of carrying out a first treatment on the surface of the To avoid interference of the symbol to the model training process, the absolute values of the two distances are taken when the distance difference is calculated:
Figure QLYQS_26
(7)
likewise, consider cancellation
Figure QLYQS_27
The influence of the sign of (2) on the model is verified by experiments, and the final selection pair is +>
Figure QLYQS_28
Taking the square as the relative position loss;
for the model training process, one batch contains
Figure QLYQS_29
In the case of several input samples, the distance difference between any two input samples is calculated, and +.>
Figure QLYQS_30
Secondary times; and then averaged as an additional term to the relative position loss function of the batch
Figure QLYQS_31
：
Figure QLYQS_32
(8)
The final joint loss function consists of the cross entropy loss function and the relative position loss function additional term together, and is specifically shown as follows:
Figure QLYQS_33
(9)
wherein the method comprises the steps of
Figure QLYQS_34
For a super parameter adjusting the ratio of the two loss functions, +.>
Figure QLYQS_35
Is a cross entropy loss function.
2. The method for constructing a diabetic retinopathy diagnostic model according to claim 1, wherein the step of preprocessing the fundus image specifically includes:
firstly, removing black areas around fundus images;
secondly, the sizes of fundus images are regulated uniformly, and the resolution of the regulated images is 256 multiplied by 256;
finally, fundus image enhancement is performed using the following formula to improve the brightness and contrast differences of fundus images:
Figure QLYQS_36
（1）
Figure QLYQS_37
（2）
in the formula (1),
Figure QLYQS_39
for the preprocessed fundus image, +.>
Figure QLYQS_41
Representing standard deviation +.>
Figure QLYQS_43
Gaussian convolution of>
Figure QLYQS_40
Representing pixel points in the fundus image; equation (2) is a weighted sum of pre-and post-enhancement fundus images, wherein the parameters used +.>
Figure QLYQS_42
, />
Figure QLYQS_44
, />
Figure QLYQS_45
, />
Figure QLYQS_38
4, -4, 10 and 128, respectively.
3. The method for constructing a diagnostic model of diabetic retinopathy according to claim 1, wherein a network of ResNet50 containing image Net pre-training parameters is selected as a backbone network of the diagnostic model.
4. A diagnostic model of diabetic retinopathy, comprising:
the preprocessing unit is used for preprocessing the acquired fundus image to obtain an input image;
the backbone network unit is used for extracting the characteristics of the input image to obtain a characteristic image containing advanced characteristics;
the improved classifier unit is used for calculating the feature map by utilizing the improved classifier to respectively obtain a classification prediction result and a regression prediction result;
the loss function unit is used for calculating a cross entropy loss function by using the classification prediction result, calculating a relative position loss function additional term by using the regression prediction result, and combining the two to obtain a final joint loss function;
in the improved classifier unit, a feature map obtained through a backbone network is sent to a full-connection layer, and a vector representation with the dimension of 6 is output
Figure QLYQS_46
The component is->
Figure QLYQS_47
The method comprises the steps of carrying out a first treatment on the surface of the Wherein->
Figure QLYQS_48
After Softmax the probability of i for the input sample label is obtained +.>
Figure QLYQS_49
， />
Figure QLYQS_50
The method comprises the steps of carrying out a first treatment on the surface of the The Softmax calculation procedure is as follows:
Figure QLYQS_51
(3)
classification results given to input samples
Figure QLYQS_52
Obtained by subscript of the largest element in P, the formula is:
Figure QLYQS_53
(4)
Figure QLYQS_54
obtaining +.>
Figure QLYQS_55
This process is shown in equation (5):
Figure QLYQS_56
(5)
here, the
Figure QLYQS_57
Consider as the regression prediction value of the input sample; />
Figure QLYQS_58
The greater the value representing the greater the extent of pathology of the sample; considering the value range of the input sample label of 0-4, for the convenience of calculation, the predicted value +.>
Figure QLYQS_59
Amplified to get->
Figure QLYQS_60
：
Figure QLYQS_61
(6)
Figure QLYQS_62
The regression prediction value is regarded as the regression prediction value of the lesion degree of the input sample;
in the loss function unit, the relative position loss function additional terms are designed as follows:
assume any two input samples of a training set
Figure QLYQS_64
And->
Figure QLYQS_66
Their label is->
Figure QLYQS_68
And->
Figure QLYQS_65
Regression prediction value of lesion degree is +.>
Figure QLYQS_67
And->
Figure QLYQS_69
Their tag distance and predicted value distance are +.>
Figure QLYQS_70
And->
Figure QLYQS_63
The method comprises the steps of carrying out a first treatment on the surface of the To avoid interference of the symbol to the model training process, the absolute values of the two distances are taken when the distance difference is calculated:
Figure QLYQS_71
(7)
likewise, consider cancellation
Figure QLYQS_72
The influence of the sign of (2) on the model is determined byExperiments prove that the final selection pair +.>
Figure QLYQS_73
Taking the square as the relative position loss;
for the model training process, one batch contains
Figure QLYQS_74
In the case of several input samples, the distance difference between any two input samples is calculated, and +.>
Figure QLYQS_75
Secondary times; and then averaged as an additional term to the relative position loss function of the batch
Figure QLYQS_76
：
Figure QLYQS_77
(8)
The final joint loss function consists of the cross entropy loss function and the relative position loss function additional term together, and is specifically shown as follows:
Figure QLYQS_78
(9)
wherein the method comprises the steps of
Figure QLYQS_79
For a super parameter adjusting the ratio of the two loss functions, +.>
Figure QLYQS_80
Is a cross entropy loss function.
5. The diabetic retinopathy diagnostic model according to claim 4, wherein the preprocessing unit is specifically configured to:
firstly, removing black areas around fundus images;
secondly, the sizes of fundus images are regulated uniformly, and the resolution of the regulated images is 256 multiplied by 256;
finally, fundus image enhancement is performed using the following formula to improve the brightness and contrast differences of fundus images:
Figure QLYQS_81
（1）
Figure QLYQS_82
（2）
in the formula (1),
Figure QLYQS_83
for the preprocessed fundus image, +.>
Figure QLYQS_84
A gaussian convolution with standard deviation sigma is represented,
Figure QLYQS_85
representing pixel points in the fundus image; equation (2) is a weighted sum of pre-and post-enhancement fundus images, wherein the parameters used +.>
Figure QLYQS_86
,/>
Figure QLYQS_87
, />
Figure QLYQS_88
, />
Figure QLYQS_89
4, -4, 10 and 128, respectively.
6. The diabetic retinopathy diagnostic model according to claim 4, wherein the backbone network unit selects a ResNet50 network containing ImageNet pre-training parameters as a backbone network of the diagnostic model.
CN202310397924.9A 2023-04-14 2023-04-14 Method for constructing diabetic retinopathy diagnosis model and diagnosis model Active CN116152229B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310397924.9A CN116152229B (en) 2023-04-14 2023-04-14 Method for constructing diabetic retinopathy diagnosis model and diagnosis model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310397924.9A CN116152229B (en) 2023-04-14 2023-04-14 Method for constructing diabetic retinopathy diagnosis model and diagnosis model

Publications (2)

Publication Number Publication Date
CN116152229A CN116152229A (en) 2023-05-23
CN116152229B true CN116152229B (en) 2023-07-11

Family

ID=86350944

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310397924.9A Active CN116152229B (en) 2023-04-14 2023-04-14 Method for constructing diabetic retinopathy diagnosis model and diagnosis model

Country Status (1)

Country Link
CN (1) CN116152229B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114271763A (en) * 2021-12-20 2022-04-05 合肥中纳医学仪器有限公司 Mask RCNN-based gastric cancer early identification method, system and device
CN114445620A (en) * 2022-01-13 2022-05-06 国网江苏省电力有限公司苏州供电分公司 Target segmentation method for improving Mask R-CNN
CN114492574A (en) * 2021-12-22 2022-05-13 中国矿业大学 Pseudo label loss unsupervised countermeasure domain adaptive picture classification method based on Gaussian uniform mixing model
CN114529819A (en) * 2022-02-23 2022-05-24 合肥学院 Household garbage image recognition method based on knowledge distillation learning

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019075410A1 (en) * 2017-10-13 2019-04-18 Ai Technologies Inc. Deep learning-based diagnosis and referral of ophthalmic diseases and disorders
CN108960257A (en) * 2018-07-06 2018-12-07 东北大学 A kind of diabetic retinopathy grade stage division based on deep learning
CN112651938B (en) * 2020-12-24 2023-12-19 平安科技（深圳）有限公司 Training method, device, equipment and storage medium for video disc image classification model
CN114387201B (en) * 2021-04-08 2023-01-17 透彻影像科技(南京)有限公司 Cytopathic image auxiliary diagnosis system based on deep learning and reinforcement learning
CN113902743A (en) * 2021-12-08 2022-01-07 武汉爱眼帮科技有限公司 Method and device for identifying diabetic retinopathy based on cloud computing

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114271763A (en) * 2021-12-20 2022-04-05 合肥中纳医学仪器有限公司 Mask RCNN-based gastric cancer early identification method, system and device
CN114492574A (en) * 2021-12-22 2022-05-13 中国矿业大学 Pseudo label loss unsupervised countermeasure domain adaptive picture classification method based on Gaussian uniform mixing model
CN114445620A (en) * 2022-01-13 2022-05-06 国网江苏省电力有限公司苏州供电分公司 Target segmentation method for improving Mask R-CNN
CN114529819A (en) * 2022-02-23 2022-05-24 合肥学院 Household garbage image recognition method based on knowledge distillation learning

Also Published As

Publication number Publication date
CN116152229A (en) 2023-05-23

Similar Documents

Publication Publication Date Title
Elangovan et al. Glaucoma assessment from color fundus images using convolutional neural network
CN108021916B (en) Deep learning diabetic retinopathy sorting technique based on attention mechanism
Kou et al. Microaneurysms segmentation with a U-Net based on recurrent residual convolutional neural network
Oyelade et al. CovFrameNet: An enhanced deep learning framework for COVID-19 detection
CN111938569A (en) Eye ground multi-disease classification detection method based on deep learning
CN110689025A (en) Image recognition method, device and system, and endoscope image recognition method and device
Panda et al. Deep convolutional neural network-based patch classification for retinal nerve fiber layer defect detection in early glaucoma
Lian et al. Deblurring retinal optical coherence tomography via a convolutional neural network with anisotropic and double convolution layer
Hussein et al. Retinex theory for color image enhancement: A systematic review
CN115601299A (en) Intelligent liver cirrhosis state evaluation system and method based on images
CN110660048B (en) Leather surface defect detection method based on shape characteristics
CN113240655A (en) Method, storage medium and device for automatically detecting type of fundus image
Wan et al. Optimized-Unet: novel algorithm for parapapillary atrophy segmentation
CN110827963A (en) Semantic segmentation method for pathological image and electronic equipment
Singh et al. Optimized convolutional neural network for glaucoma detection with improved optic-cup segmentation
CN116152229B (en) Method for constructing diabetic retinopathy diagnosis model and diagnosis model
CN113643261A (en) Lung disease diagnosis method based on frequency attention network
CN110503636B (en) Parameter adjustment method, focus prediction method, parameter adjustment device and electronic equipment
CN111931544B (en) Living body detection method, living body detection device, computing equipment and computer storage medium
CN115578783B (en) Device and method for identifying eye diseases based on eye images and related products
Wang et al. A r-cnn based approach for microaneurysm detection in retinal fundus images
Valério et al. Lesions multiclass classification in endoscopic capsule frames
Anjugam et al. Study of Deep Learning Approaches for Diagnosing Covid-19 Disease using Chest CT Images
Puthren et al. Automated glaucoma detection using global statistical parameters of retina fundus images
Sharma et al. Advancement in Diabetic Retinopathy Diagnosis Techniques: Automation and Assistive Tools

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant