CN114170089B - Method for classifying diabetic retinopathy and electronic equipment - Google Patents
Method for classifying diabetic retinopathy and electronic equipment Download PDFInfo
- Publication number
- CN114170089B CN114170089B CN202111540274.6A CN202111540274A CN114170089B CN 114170089 B CN114170089 B CN 114170089B CN 202111540274 A CN202111540274 A CN 202111540274A CN 114170089 B CN114170089 B CN 114170089B
- Authority
- CN
- China
- Prior art keywords
- super
- feature map
- resolution
- image
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 22
- 206010012689 Diabetic retinopathy Diseases 0.000 title claims abstract description 21
- 230000004927 fusion Effects 0.000 claims abstract description 24
- 238000000605 extraction Methods 0.000 claims abstract description 17
- 230000003321 amplification Effects 0.000 claims abstract description 5
- 238000003199 nucleic acid amplification method Methods 0.000 claims abstract description 5
- 238000010586 diagram Methods 0.000 claims description 18
- 230000004913 activation Effects 0.000 claims description 14
- 230000006870 function Effects 0.000 claims description 14
- 238000011176 pooling Methods 0.000 claims description 7
- 238000011144 upstream manufacturing Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 238000013178 mathematical model Methods 0.000 claims description 4
- 238000007499 fusion processing Methods 0.000 claims description 2
- 238000003745 diagnosis Methods 0.000 abstract description 6
- 238000013135 deep learning Methods 0.000 abstract description 4
- 230000000694 effects Effects 0.000 abstract description 3
- 238000012549 training Methods 0.000 description 14
- 230000003902 lesion Effects 0.000 description 7
- 238000012360 testing method Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 2
- 206010012601 diabetes mellitus Diseases 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 210000001525 retina Anatomy 0.000 description 2
- 208000017442 Retinal disease Diseases 0.000 description 1
- 206010038923 Retinopathy Diseases 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 235000003642 hunger Nutrition 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000035755 proliferation Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000004256 retinal image Effects 0.000 description 1
- 230000037351 starvation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Medical Informatics (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and electronic equipment for classifying diabetic retinopathy, wherein the method comprises the steps of super-resolution reconstruction, classifying super-resolution images by utilizing an image classification network, and the like, the super-resolution reconstruction network comprises a preliminary convolution layer, a deep feature extraction unit, a feature fusion unit and an image amplification module, the preliminary convolution layer is used for carrying out shallow feature extraction on fundus images, the deep feature extraction unit comprises jump connection and a plurality of residual error attention modules, and the image amplification module is used for carrying out super-resolution reconstruction on second feature images. The invention increases the resolution of fundus images by utilizing the super-resolution technology, improves the accuracy of classifying diabetic retinopathy based on a deep learning method under the condition of limited hardware, meets the requirement of actual diagnosis, and relieves the situation of medical resource shortage; the super-resolution reconstruction network features have high extraction efficiency and good reconstruction effect.
Description
Technical Field
The invention belongs to the technical field of medical treatment and image processing, and particularly relates to a method and electronic equipment for classifying diabetic retinopathy.
Background
Diabetes causes a number of complications, of which Diabetic Retinopathy (DR) is one of the more severe, which can lead to impaired and even blinding vision in patients. With the improvement of the physical living standard of people, the average age of people is gradually increased, the number of people suffering from diabetes in China is rapidly increased, and the number of patients needing to carry out retinopathy diagnosis is rapidly increased. On the other hand, conventional manual diagnosis depends on clinical experience of doctors, and qualified medical staff has a long culture period, and medical resources are increasingly deficient compared with the number of patients which are rapidly increased, so that a plurality of patients cannot be diagnosed and treated in time, and the disease is worsened.
Automatic classification of fundus retinal images using artificial intelligence techniques, particularly deep learning, to aid in diagnosis of diabetic retinopathy is an effective approach to address medical resource starvation. However, accurate diagnosis depends on high-quality image input, but economic level in many areas of China is not high, medical equipment conditions are behind, resolution ratio of retina images obtained through shooting is generally low, so that misdiagnosis rate of an image classification method based on deep learning is high, and actual requirements are difficult to meet.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a method for classifying diabetic retinopathy and electronic equipment so as to improve the accuracy of classifying fundus retina images.
In order to achieve the above object, the present invention adopts the following solutions: a method for diabetic retinopathy classification, comprising the steps of:
s1, acquiring a fundus image, acquiring a trained super-resolution reconstruction network, and performing super-resolution reconstruction on the fundus image by using the super-resolution reconstruction network to obtain a super-resolution image with resolution greater than that of the fundus image;
s2, acquiring a trained image classification network, inputting the super-resolution image into the image classification network, and classifying the super-resolution image by using the image classification network;
the super-resolution reconstruction network includes:
the preliminary convolution layer is used for carrying out shallow layer feature extraction on the fundus image to obtain a first feature map;
the deep feature extraction unit is connected with the output end of the preliminary convolution layer, the deep feature extraction unit comprises jump connection and a plurality of residual attention modules, the residual attention modules are sequentially connected end to end, and the first feature map is input into each residual attention module through the jump connection;
the feature fusion unit is used for fusing the feature graphs output by the residual attention modules to obtain a second feature graph;
the image amplification module is used for carrying out super-resolution reconstruction on the second feature map to obtain the super-resolution image;
the mathematical model of the residual attention module is as follows:
F1＝σ 1 (f 31 (K n ))
F2＝σ 2 (f 32 (K n ))
F3＝σ 3 (f 33 (F2))
F4＝f ca (f T (F1,F2,K 0 ))
F5＝f m (F4,[F1,F3])
K n+1 ＝σ 4 (f 1 (F5))+K n
wherein K is 0 Representing the first feature map, K n K is a feature map of upstream output 0 And K n At the same time as input to the residual attention module, f 31 、f 32 And f 33 All represent convolution operations with convolution kernel size 3*3, f 1 Representing a convolution operation with a convolution kernel size 1*1, σ 1 、σ 2 、σ 3 Sum sigma 4 All represent a nonlinear activation function ReLU, f T Represents a ternary fusion module for fusing F1, F2 and K 0 Fusion, f ca Representing the channel attention module, [ ·]Representing the splicing operation of the characteristic graphs m (F4，[F1,F3]) Represents K after F1 and F3 are spliced and then are fused with F4 n+1 And the characteristic diagram represents the output of the residual attention module.
Further, the ternary fusion module may be expressed as the following formula:
F T1 ＝σ T1 (f T1 ([F1,F2,K 0 ]))
F T2 ＝σ T2 (F1+F2)
F T ＝σ T3 (f T3 ([F T1 ,F T2 ]))
wherein F1, F2 and K 0 At the same time, f is used as the input of the ternary fusion module T1 Representing a convolution operation with a convolution kernel size of 1*1, f T3 Representing a convolution operation with a convolution kernel size of 3*3, σT 1 、σT 2 And sigma T 3 All represent nonlinear activation functions ReLU, [. Cndot.]Representing the splicing operation of the characteristic graphs in the characteristic graphs, F T And outputting the data to the ternary fusion module.
Further, the mathematical model of the channel attention module is:
Fa1＝δ a1 (f a12 (σ a1 (f a11 (p v (F T )))))
Fa2＝δ a2 (f a22 (σ a2 (f a21 (p M (F T )))))
Fa3＝δ a3 (f a32 (σ a3 (f a31 (p Max (F T )))))
wherein F is T Representing the output of the ternary fusion module, F T As input to the channel attention module, p v Representing global differential pooling operations, p M Representing global average pooling operations, p Max Representing global max pooling operations, f a11 、f a12 、f a21 、f a22 、f a31 And f a32 All represent full join operation, sigma a1 、σ a2 Sum sigma a3 All represent a nonlinear activation function ReLU, delta a1 、δ a2 And delta a3 Are all sigmoid activation functions, fa1, fa2, and Fa3 are channel attention diagrams output by the channel attention module.
Further, the fusion process of the feature map obtained after the feature map F1 and the feature map F3 are spliced with the channel attention map comprises the following steps:
a1, the feature map obtained after the feature map F1 and the feature map F3 are spliced is multiplied by three channel attention force maps output by the channel attention module respectively to obtain a first enhancement feature map, a second enhancement feature map and a third enhancement feature map,
a2, splicing the first enhancement feature map, the second enhancement feature map and the third enhancement feature map in the channel direction.
Further, the image classification network is MobileNet V3.
The invention also provides an electronic device for diabetic retinopathy classification, comprising a processor and a memory, the memory storing a computer program, the processor being adapted to execute the method for diabetic retinopathy classification as described above by loading the computer program.
The beneficial effects of the invention are as follows:
(1) The resolution of the fundus image is increased by utilizing the super-resolution technology, the accuracy of classifying the diabetic retinopathy based on the deep learning method is improved under the limited hardware condition, so that the actual diagnosis needs are met, the burden of medical staff is reduced, the situation of medical resource shortage is relieved, and the condition of a patient is increased due to the fact that the patient does not visit in time is avoided;
(2) In the residual attention module, the input feature map is subjected to feature extraction through two branches, one branch is subjected to convolution operation of 3*3, the other branch is subjected to convolution operation of two 3*3 in series, which is equivalent to the convolution operation of 5*5, and because super-resolution reconstruction is a more apparent task than classification, in the invention, the feature map output by the front end convolution layer of the residual attention module is input into a ternary fusion module to generate a channel attention map, and because the feature map contains relatively more apparent feature information, the generated channel attention map is better in modulation effect, and is favorable for reconstructing a super-division image with better effect;
(3) In order to make full use of information in the low-resolution image, the first feature map is input into each residual attention module through jump connection, and after the first feature map is fused with the feature map in the residual attention module, the channel attention map generated by the channel attention module is more beneficial to improving the extraction ratio of high-frequency information by modulation and reducing the redundancy of useless information;
(4) For different residual attention modules, as depth positions of the residual attention modules in a network are different, scales of information in F1 and F2 are also different, in order to enable a first feature map to be well fused with F1 feature maps and F2 feature maps in different residual attention modules, the inventor creatively designs a ternary fusion module, the F1, F2 and the first feature map are fused by splicing and dimension reduction, the obtained feature map is spliced with the feature map after summation and activation of the F1 and the F2, and finally, feature extraction is carried out through a 3*3 convolution layer, so that specific feature information in the first feature map and abstract feature information in the F1 and F2 are well balanced, and the first feature map input into the different residual attention modules can be adaptively fused with information of different scales, so that useful feature information extraction efficiency is improved;
(5) For an attention mechanism (such as CBAM) comprising a plurality of modules, attention force generated by different attention modules is multiplied with a feature diagram in sequence in a serial mode, after the upstream attention force is used for modulating the feature diagram, the importance and the duty ratio of different information are changed, and the attention module positioned at the back is interfered.
Drawings
FIG. 1 is a schematic diagram of a super-resolution reconstruction network structure according to the present invention;
FIG. 2 is a schematic diagram of a residual attention module structure according to the present invention;
FIG. 3 is a schematic diagram of a ternary fusion module according to the present invention;
FIG. 4 is a schematic diagram of a first branch structure of the channel attention module according to the present invention;
FIG. 5 is a schematic diagram of a second branch structure of the channel attention module according to the present invention;
FIG. 6 is a schematic diagram of a third branch structure of the channel attention module according to the present invention;
FIG. 7 is a schematic diagram of a feature fusion unit according to the present invention;
FIG. 8 is a schematic diagram of an image magnification module according to the present invention;
in the accompanying drawings:
the system comprises a 1-fundus image, a 2-preliminary convolution layer, a 3-deep feature extraction unit, a 4-jump connection, a 5-residual attention module, a 51-ternary fusion module, a 52-channel attention module, a 53-first branch, a 54-second branch, a 55-third branch, a 6-feature fusion unit, a 7-image amplification module and an 8-super-resolution image.
Detailed Description
The invention is further described below with reference to the accompanying drawings:
examples:
the fundus image 1 is collected, a fundus image dataset is made, and the dataset is divided into three parts, a first training set, a second training set, and a test set. The first training set, the second training set and the test set all contain lesion information marked by human experts, and the lesion information comprises five categories of no lesions, slight lesions, medium lesions, serious lesions and proliferation lesions.
Super-resolution reconstruction network training:
the super-resolution reconstruction network structure is shown in fig. 1, and in this embodiment, the super-resolution reconstruction network includes sixteen residual attention modules 5. The first training set image is downsampled to obtain a low-resolution fundus image 1, and then the super-resolution reconstruction network is trained by using the low-resolution fundus image 1 and the first training set image. An L2 loss function optimization model is adopted in the training process.
The primary convolution layer 2 is realized by adopting a convolution layer with the convolution kernel size of 3*3, the number of channels of the first feature map output by the primary convolution layer 2 is 48, and the output end of the primary convolution layer 2 is connected with the input end of the deep feature extraction unit 3. For the first residual attention module 5, the first feature map is input to the first residual attention module 5 from both the upstream end and the jump connection 4, and the input end of the subsequent residual attention module 5 receives as input the feature map output by the previous residual attention module 5.
As shown in fig. 2, the residual attention module 5 is configured such that feature extraction is performed through two branches from the feature map input upstream to the residual attention module 5, the number of channels of the feature maps (F1, F3) output from the two branches is 48, and after the channels are spliced in the channel direction, the number of channels becomes 96.
The structure of the ternary fusion module 51 is shown in fig. 3, the number of channels of three feature maps (first feature map, F1, F2) input into the ternary fusion module 51 is 48, and after the three feature maps are subjected to splicing, 1*1 convolution and activation operations, a first intermediate feature map with the number of channels being 48 is output. On the other hand, after F1 and F2 are summed and activated, a second intermediate feature map with 48 channels is output. The first intermediate profile and the second intermediate profile are then subjected to a stitching, 3*3 convolution and activation function to output 96 profiles as inputs to the channel attention module 52.
In this embodiment, the channel attention module 52 includes a first branch 53 (shown in FIG. 4), a second branch 54 (shown in FIG. 5), and a third branch 55 (shown in FIG. 6), each of which includes a global pooling operation and two fully connected operations. In each branch, the input node number of the first full connection layer is 96, the output node number is 24, the input node number of the second full connection layer is 24, and the output node number is 96. After modulating the characteristic graphs after the splicing of the F1 and the F3 respectively, the three channels are spliced in the channel direction, then a 1*1 convolution layer and a ReLU activation function are input, and the characteristic graph with 48 channels is output.
The structure of the feature fusion unit 6 is as shown in fig. 7, all feature graphs output by the residual attention module 5 are spliced in the channel direction, then the channel number is changed to 48 by convolution with a 1*1 convolution layer, and finally a second feature graph is output through a ReLU activation function.
The image amplifying module 7 may be implemented by adopting the existing deconvolution and sub-pixel convolution operations, in this embodiment, the structure of the image amplifying module 7 is shown in fig. 8, the image amplifying module 7 includes two amplifying components and a tail end convolution layer, the tail end convolution layer is disposed at the tail end of the image amplifying component, the two amplifying components are sequentially connected end to end, and the amplifying component includes a deconvolution layer and a ReLU activation function which are sequentially connected. The convolution kernel sizes of the tail end convolution layer and the deconvolution layer are 3*3, the length and width sizes of the feature images are doubled when the feature images pass through one amplifying assembly, and the number of channels is unchanged. For the end convolution layer, the number of input channels is 48 and the number of output channels is 3 for the super-resolution image 8.
Image classification network training:
and performing super-resolution reconstruction on the images in the second training set by using the super-resolution reconstruction network after training, and then training an image classification network by using the obtained super-resolution images 8. The image classification network adopts MobileNet V3, and cross entropy is adopted as a loss function in the training process.
Classification of diabetic retinopathy:
inputting the images in the test set into a super-resolution reconstruction network trained before, inputting the reconstructed and output images into an image classification network trained before, and calculating the accuracy of model classification by combining the labeling information.
As a comparison experiment, the super-resolution reconstruction network provided by the invention is replaced by a SAN model, the SAN model and the image classification network are retrained under the identical conditions (comprising a training set, a framework, epoch, batchsize, magnification and the like) according to the method, and the test is carried out on the same test set, so that the accuracy of the classification result finally obtained by adopting the super-resolution reconstruction network provided by the invention is 7.4% higher than that obtained by adopting the SAN network.
The foregoing examples merely illustrate specific embodiments of the invention, which are described in greater detail and are not to be construed as limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention.
Claims (6)
1. A method for classifying diabetic retinopathy, characterized by: the method comprises the following steps:
s1, acquiring a fundus image, acquiring a trained super-resolution reconstruction network, and performing super-resolution reconstruction on the fundus image by using the super-resolution reconstruction network to obtain a super-resolution image with resolution greater than that of the fundus image;
s2, acquiring a trained image classification network, inputting the super-resolution image into the image classification network, and classifying the super-resolution image by using the image classification network;
the super-resolution reconstruction network includes:
the preliminary convolution layer is used for carrying out shallow layer feature extraction on the fundus image to obtain a first feature map;
the deep feature extraction unit is connected with the output end of the preliminary convolution layer, the deep feature extraction unit comprises jump connection and a plurality of residual attention modules, the residual attention modules are sequentially connected end to end, and the first feature map is input into each residual attention module through the jump connection;
the feature fusion unit is used for fusing the feature graphs output by the residual attention modules to obtain a second feature graph;
the image amplification module is used for carrying out super-resolution reconstruction on the second feature map to obtain the super-resolution image;
the mathematical model of the residual attention module is as follows:
F1＝σ 1 (f 31 (K n ))
F2＝σ 2 (f 32 (K n ))
F3＝σ 3 (f 33 (F2))
F4＝f ca (f T (F1,F2,K 0 ))
F5＝f m (F4,[F1,F3])
K n+1 ＝σ 4 (f 1 (F5))+K n
wherein K is 0 Representing the first feature map, K n K is a feature map of upstream output 0 And K n At the same time as input to the residual attention module, f 31 、f 32 And f 33 All represent convolution operations with convolution kernel size 3*3, f 1 Representing a convolution operation with a convolution kernel size 1*1, σ 1 、σ 2 、σ 3 Sum sigma 4 All represent a nonlinear activation function ReLU, f T Represents a ternary fusion module for fusing F1, F2 and K 0 Fusion, f ca Representing the channel attention module, [ ·]Representing the splicing operation of the characteristic graphs m (F4，[F1,F3]) Represents K after F1 and F3 are spliced and then are fused with F4 n+1 And the characteristic diagram represents the output of the residual attention module.
2. The method for diabetic retinopathy classification as claimed in claim 1, wherein: the ternary fusion module may be expressed as the following formula:
F T1 ＝σ T1 (f T1 ([F1,F2,K 0 ]))
F T2 ＝σ T2 (F1+F2)
F T ＝σ T3 (f T3 ([F T1 ,F T2 ]))
wherein F1, F2 and K 0 At the same time, f is used as the input of the ternary fusion module T1 Representing a convolution operation with a convolution kernel size of 1*1, f T3 Representing a convolution operation with a convolution kernel size 3*3, σ T1 、σ T2 Sum sigma T3 All represent nonlinear activation functions ReLU, [. Cndot.]Representing the splicing operation of the characteristic graphs in the characteristic graphs, F T And outputting the data to the ternary fusion module.
3. The method for diabetic retinopathy classification as claimed in claim 1, wherein: the mathematical model of the channel attention module is as follows:
Fa1＝δ a1 (f a12 (σ a1 (f a11 (p v (F T )))))
Fa2＝δ a2 (f a22 (σ a2 (f a21 (p M (F T )))))
Fa3＝δ a3 (f a32 (σ a3 (f a31 (p Max (F T )))))
wherein F is T Representing the output of the ternary fusion module, F T As input to the channel attention module, p v Representing global differential pooling operations, p M Representing global average pooling operations, p Max Representing global max pooling operations, f a11 、f a12 、f a21 、f a22 、f a31 And f a32 All represent full join operation, sigma a1 、σ a2 Sum sigma a3 All represent a nonlinear activation function ReLU, delta a1 、δ a2 And delta a3 Are all sigmoid activation functions, fa1, fa2, and Fa3 are channel attention diagrams output by the channel attention module.
4. A method for diabetic retinopathy classification as claimed in claim 3, wherein: the fusion process of the characteristic diagram obtained after the characteristic diagram F1 and the characteristic diagram F3 are spliced and the channel attention diagram comprises the following steps:
a1, the feature map obtained after the feature map F1 and the feature map F3 are spliced is multiplied by three channel attention force maps output by the channel attention module respectively to obtain a first enhancement feature map, a second enhancement feature map and a third enhancement feature map,
a2, splicing the first enhancement feature map, the second enhancement feature map and the third enhancement feature map in the channel direction.
5. The method for diabetic retinopathy classification as claimed in claim 1, wherein: the image classification network is MobileNet V3.
6. An electronic device for diabetic retinopathy classification, characterized by: comprising a processor and a memory, said memory storing a computer program for executing the method for diabetic retinopathy classification according to any of claims 1-5 by loading said computer program.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111160828X | 2021-09-30 | ||
| CN202111160828 | 2021-09-30 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114170089A CN114170089A (en) | 2022-03-11 |
| CN114170089B true CN114170089B (en) | 2023-07-07 |
Family
ID=80486882
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111540274.6A Active CN114170089B (en) | 2021-09-30 | 2021-12-16 | Method for classifying diabetic retinopathy and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114170089B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114882203B (en) * | 2022-05-20 | 2024-05-28 | 江阴萃合智能装备有限公司 | Image super-resolution reconstruction method for power grid inspection robot |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200034948A1 (en) * | 2018-07-27 | 2020-01-30 | Washington University | Ml-based methods for pseudo-ct and hr mr image estimation |
| CN111192200A (en) * | 2020-01-02 | 2020-05-22 | 南京邮电大学 | Image super-resolution reconstruction method based on fusion attention mechanism residual error network |
| CN111461983A (en) * | 2020-03-31 | 2020-07-28 | 华中科技大学鄂州工业技术研究院 | Image super-resolution reconstruction model and method based on different frequency information |
| CN111626300A (en) * | 2020-05-07 | 2020-09-04 | 南京邮电大学 | Image semantic segmentation model and modeling method based on context perception |
| US20210133925A1 (en) * | 2019-11-05 | 2021-05-06 | Moxa Inc. | Device and Method of Handling Image Super-Resolution |
| CN113128583A (en) * | 2021-04-15 | 2021-07-16 | 重庆邮电大学 | Medical image fusion method and medium based on multi-scale mechanism and residual attention |
| CN113139907A (en) * | 2021-05-18 | 2021-07-20 | 广东奥普特科技股份有限公司 | Generation method, system, device and storage medium for visual resolution enhancement |
| CN113298717A (en) * | 2021-06-08 | 2021-08-24 | 浙江工业大学 | Medical image super-resolution reconstruction method based on multi-attention residual error feature fusion |
-
2021
- 2021-12-16 CN CN202111540274.6A patent/CN114170089B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200034948A1 (en) * | 2018-07-27 | 2020-01-30 | Washington University | Ml-based methods for pseudo-ct and hr mr image estimation |
| US20210133925A1 (en) * | 2019-11-05 | 2021-05-06 | Moxa Inc. | Device and Method of Handling Image Super-Resolution |
| CN111192200A (en) * | 2020-01-02 | 2020-05-22 | 南京邮电大学 | Image super-resolution reconstruction method based on fusion attention mechanism residual error network |
| CN111461983A (en) * | 2020-03-31 | 2020-07-28 | 华中科技大学鄂州工业技术研究院 | Image super-resolution reconstruction model and method based on different frequency information |
| CN111626300A (en) * | 2020-05-07 | 2020-09-04 | 南京邮电大学 | Image semantic segmentation model and modeling method based on context perception |
| CN113128583A (en) * | 2021-04-15 | 2021-07-16 | 重庆邮电大学 | Medical image fusion method and medium based on multi-scale mechanism and residual attention |
| CN113139907A (en) * | 2021-05-18 | 2021-07-20 | 广东奥普特科技股份有限公司 | Generation method, system, device and storage medium for visual resolution enhancement |
| CN113298717A (en) * | 2021-06-08 | 2021-08-24 | 浙江工业大学 | Medical image super-resolution reconstruction method based on multi-attention residual error feature fusion |
Non-Patent Citations (3)
| Title |
|---|
| "Image Super-Resolution Using Very Deep Residual Channel Attention Networks";Yulun Zhang et ai.;arXiv;第1-16页 * |
| "胰岛素抵抗在2 型糖尿病时血清维生素D3 和骨密度 变化中的意义";吕燕等;《中国骨质疏松杂志》;第20卷(第4期);第372-374页 * |
| "通道注意力与残差级联的图像超分辨率重建";蔡体健等;《光学精密工程》;第29卷(第1期);第142-149页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114170089A (en) | 2022-03-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110232383B (en) | Focus image recognition method and focus image recognition system based on deep learning model | |
| CN112132817B (en) | Retina blood vessel segmentation method for fundus image based on mixed attention mechanism | |
| CN110399929B (en) | Fundus image classification method, fundus image classification apparatus, and computer-readable storage medium | |
| CN111815574B (en) | Fundus retina blood vessel image segmentation method based on rough set neural network | |
| CN113888412B (en) | Image super-resolution reconstruction method for diabetic retinopathy classification | |
| CN112215755B (en) | Image super-resolution reconstruction method based on back projection attention network | |
| WO2020168820A1 (en) | Yolo convolutional neural network-based cholelithiasis ct medical image data enhancement method | |
| CN114037624B (en) | Image enhancement method and device for classifying diabetic nephropathy | |
| CN111091575B (en) | Medical image segmentation method based on reinforcement learning method | |
| CN114170089B (en) | Method for classifying diabetic retinopathy and electronic equipment | |
| CN116563533A (en) | Medical image segmentation method and system based on target position priori information | |
| Sahu et al. | An application of deep dual convolutional neural network for enhanced medical image denoising | |
| CN111462004B (en) | Image enhancement method and device, computer equipment and storage medium | |
| CN116958535B (en) | Polyp segmentation system and method based on multi-scale residual error reasoning | |
| Yue et al. | Deep pyramid network for low-light endoscopic image enhancement | |
| Iqbal et al. | LDMRes-Net: Enabling real-time disease monitoring through efficient image segmentation | |
| CN116309507A (en) | AIS focus prediction method for performing feature fusion on CTP under attention mechanism | |
| CN114066873B (en) | Method and device for detecting osteoporosis by utilizing CT (computed tomography) image | |
| CN115330600A (en) | Lung CT image super-resolution method based on improved SRGAN | |
| CN115115736A (en) | Image artifact removing method, device and equipment and storage medium | |
| CN114649092A (en) | Auxiliary diagnosis method and device based on semi-supervised learning and multi-scale feature fusion | |
| CN113796850A (en) | Parathyroid MIBI image analysis system, computer device, and storage medium | |
| CN110969117A (en) | Fundus image segmentation method based on Attention mechanism and full convolution neural network | |
| CN116246067B (en) | CoA Unet-based medical image segmentation method | |
| CN114004784B (en) | Method for detecting bone condition based on CT image and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220622 Address after: 610021 No. 10, Qingyun South Street, Jinjiang District, Chengdu, Sichuan Applicant after: CHENGDU SECOND PEOPLE'S Hospital Address before: No.82, 2nd section, 2nd Ring Road, Chengdu, Sichuan 610000 Applicant before: AFFILIATED HOSPITAL OF CHENGDU University |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |