CN112669260B - Fundus image optic disc macula lutea detection method and device based on deep neural network - Google Patents
Fundus image optic disc macula lutea detection method and device based on deep neural network Download PDFInfo
- Publication number
- CN112669260B CN112669260B CN202011427915.2A CN202011427915A CN112669260B CN 112669260 B CN112669260 B CN 112669260B CN 202011427915 A CN202011427915 A CN 202011427915A CN 112669260 B CN112669260 B CN 112669260B
- Authority
- CN
- China
- Prior art keywords
- optic disc
- model
- segmentation
- module
- pseudo label
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention discloses a fundus image optic disc macula lutea detection method and a device based on a deep neural network, which comprises the following steps: giving a data set containing a plurality of fundus color photographs, giving corresponding optic disc center position labels or whole optic disc region labels and macular center position labels, and dividing the data set into a training set and a verification set; preprocessing an original fundus color photograph to obtain an enhanced domain image, labeling a macular position of a video disc, and establishing a pseudo label image; establishing a model for positioning the macula lutea of the optic disc and segmenting the optic disc, and performing model training and verification on a training set and a verification set by utilizing an enhanced domain diagram and a pseudo label diagram; and positioning the optic disc and the macula lutea on the verification set by adopting the model constructed in the S13, extracting the region of interest, and segmenting by using the model to obtain the final macula lutea, optic disc positioning and optic disc segmentation results. The invention simultaneously covers two tasks of optic disc and yellow spot detection, is reliable and is easy to realize.
Description
Technical Field
The invention relates to the technical field of computer vision and image processing, in particular to a method and a device for detecting eyeground image optic disc macula lutea based on a deep neural network.
Background
The retinal fundus image analysis and processing is a hot point problem in the current computer-aided diagnosis field, namely, a fundus color photograph is input, and information such as organ structures and focus of the fundus is output by a computer, so that a doctor is assisted in diagnosis and treatment, meanwhile, the labor cost of the doctor can be saved, and large-scale screening and the like are realized. The structures of the optic discs and the macula lutea in the fundus images can provide rich medical information, so that the algorithm for automatically detecting the optic discs and the macula lutea by the computer has very important significance. Currently, for the detection work of the optic disc and the macula lutea, the image processing method according to the algorithm can be divided into a method based on deep learning and a method based on traditional image processing.
The conventional image processing method is generally based on the color information of the optic disc and the macula lutea, has complex procedures and poor generalization, and is gradually replaced by the recent emerging deep learning-based method. The method based on deep learning can be roughly divided into two ideas, namely detection according to the color characteristics of the optic disc or the macular region and detection according to the fundus structure information. The detection based on the color features has the characteristics of simple thought, accurate result under normal imaging conditions and slight pathological changes, but failure possibly caused by poor imaging conditions or serious pathological changes. Detection based on structural information is characterized by being more robust, but generally results worse than the former in non-extreme cases.
After retrieval, the following are available in the prior art:
[1]B.Al-Bander,W.Al-Nuaimy,B.M.Williams,and Y.Zheng,“Multiscale sequential convolutional neural networks for simultaneous detection of fovea and optic disc,”Biomedical Signal Processing and Control,vol.40,pp.91–101,2018。
[2]X.Meng,X.Xi,L.Yang,G.Zhang,Y.Yin,and X.Chen,“Fast and effective optic disk localization based on convolutional neural network,”Neurocomputing,vol.312,pp.285–295,2018.
[3]K.-K.Maninis,J.Pont-Tuset,P.Arbelaez,and L.Van Gool,“Deep retinal image understanding,”in International conference on medical image computing and computer-assisted intervention.Springer,2016,pp.140–148
[4]J.Son,S.J.Park,and K.-H.Jung,“Towards accurate segmentation of retinal vessels and the optic disc in fundoscopic images with generative adversarial networks,”Journal of digital imaging,vol.32,no.3,pp.499–512,2019.
[5]Z.Gu,J.Cheng,H.Fu,K.Zhou,H.Hao,Y.Zhao,T.Zhang,S.Gao,and J.Liu,“Ce-net:Context encoder network for 2d medical image segmentation,”IEEE Transactions on Medical Imaging,pp.1–1,2019.
in [1], the authors treat disc and macula localization as regression problems, designing a Convolutional Neural Network (CNN), while predicting the positions of both. However, this type of method cannot determine the reliability of the result from the result output by the model. In [2], an author introduces an anomaly detection idea, but a large amount of data set support is needed, and the method is not suitable for popularization. In [3] [4] [5], various authors designed it. Different networks and learning algorithms are used for dividing the optic disc area, but the experiment only aims at the image block of the optic disc area, and the positioning process of the image block of the optic disc area is not available.
The above methods all have their own limitations. Therefore, it is urgently needed to provide fundus image optic disc macula lutea detection which covers two tasks of optic disc and macula lutea detection and is reliable and easy to realize.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a method and a device for detecting the optic disc macula lutea of the fundus image based on the deep neural network, which cover two tasks of optic disc and macula lutea detection, designs a new algorithm and considers color information and structure information, so that each index of optic disc and macula lutea detection is improved, and meanwhile, experiments of the invention are verified on a public data set and are easy to reproduce.
In order to solve the technical problems, the invention is realized by the following technical scheme:
the invention provides a fundus image optic disc macula lutea detection method based on a deep neural network, which comprises the following steps:
s11: establishing a data set;
giving a data set containing a plurality of fundus color photographs, giving corresponding optic disc center position labels or whole optic disc region labels and macular center position labels, and dividing the data set into a training set and a verification set;
s12: pre-treating;
preprocessing an original fundus color photograph to obtain an enhanced domain image, labeling a macular position of a video disc, and establishing a pseudo label image;
s13: establishing a model;
establishing a model for positioning macula lutea of the optic disc and segmenting the optic disc, and performing model training and verification on the training set and the verification set which are divided in the S11 by using the enhanced domain map and the pseudo label map obtained in the S12;
s14: testing the model;
and positioning the optic disc and the macula lutea on the verification set divided in the step S11 by adopting the model constructed in the step S13, extracting the region of interest, and segmenting by using the model constructed in the step S13 to obtain the final macula lutea, optic disc positioning and optic disc segmentation results.
Preferably, the preprocessing in S12 includes: gaussian filtering and background subtraction.
Preferably, the original fundus oculi is color-photographed I in S12 ori Preprocessing the image to obtain an enhanced domain image I eh Further comprises the following steps:
I eh ＝4(I ori -G(σ)*I ori )+0.5，
where G (σ) is a gaussian filter, σ is its variance, and σ represents the image convolution operation, and σ is set to 1/30 of the image field radius.
Preferably, the creating the pseudo tag map in S12 further includes: and generating a circular area with the radius of a preset radius and the center of the circle positioned in the center of the optic disc and/or the yellow spots.
Preferably, the model in S13 is trained to minimize a loss function, wherein the loss function used is a regression loss function constructed by using a difference between a real coordinate label and a regression network regression prediction result; a segmentation loss function is constructed by utilizing the gap between the pseudo label graph and the pseudo label segmentation result; and constructing a segmentation loss function by using the difference between the optic disc region label and the optic disc region segmentation structure.
Preferably, in S13, the real coordinate labels are used for model training to obtain a regression network, and the pseudo label graph is used for model training to obtain a pseudo label segmentation network; carrying out model training by using real optic disc region labels to obtain an optic disc segmentation network; further, the air conditioner is provided with a fan,
in the step S14, the model constructed in the step S13 is used for testing, including a pseudo tag segmentation result output by a pseudo tag segmentation network, a macula lutea output by a regression network, a optic disc positioning result, and a optic disc segmentation result;
when the region shape of the pseudo label segmentation result is a regular quasi-circular shape, the pseudo label segmentation result output by the pseudo label segmentation network is used as a final yellow spot and optic disc positioning result; and when the region of the pseudo label segmentation structure is in an irregular shape, positioning the yellow spots and the optic discs output by the regression network to obtain a final yellow spot and optic disc positioning result.
Preferably, the shape of the region of the pseudo tag segmentation result is determined according to a shape factor SI, where the shape factor SI is:
wherein C is the perimeter of the pseudo label segmentation result region, and S is the area of the pseudo label segmentation result region;
and when the SI is within the preset range, judging that the SI is in a round-like shape, and when the SI exceeds the preset range, judging that the SI is in an irregular shape.
Preferably, the regression loss function minimization problem is defined as:
wherein, θ represents the model parameter, and n is the number of images trained each time, i.e. the number of batchs. In the following, it is assumed that P denotes a coordinate vector, I denotes an output graph, pre in the subscript denotes a model prediction value, and gt denotes a true annotation value.
Preferably, the segmentation loss function minimization problem is defined as:
wherein, θ represents the model parameter, and n is the number of images trained each time, i.e. the number of batchs. The other binomial loss functions are respectively cross entropy loss L CE And Dice coefficient loss L Dice ；
The cross entropy loss is:
the Dice coefficient loss is:
wherein H and W respectively represent the height and width of the image, the unit is pixel, K is the number of categories and is the number of target categories plus the number of backgrounds, and the coordinates (i, j, K) represent the value of the ith row and jth column of the image.
The invention also provides a fundus image optic disc macula lutea detection device based on the deep neural network, which is used for realizing the fundus image optic disc macula lutea detection method based on the deep neural network, and comprises the following steps: the system comprises a data set establishing module, a preprocessing module, a model establishing module and a model testing module; wherein the content of the first and second substances,
the data set establishing module is used for giving a data set containing a plurality of fundus color photographs, giving corresponding optic disc center position labels or whole optic disc region labels and macular center position labels, and dividing the data set into a training set and a verification set;
the preprocessing module is used for preprocessing an original fundus color photograph to obtain an enhanced domain image, labeling the position of the macula lutea of a video disc and establishing a pseudo label image;
the model establishing module is used for establishing a model for the macular location of the optic disc and the segmentation of the optic disc, and performing model training and verification on a training set and a verification set which are divided in the data set establishing module by utilizing the enhanced domain map and the pseudo label map which are obtained by the preprocessing module;
the model testing module is used for positioning the optic disc and the yellow spot on the verification set divided in the data set establishing module by adopting the model established by the model establishing module, extracting the region of interest, and segmenting by using the model established by the model establishing module to obtain the final yellow spot, optic disc positioning and optic disc segmenting results.
Compared with the prior art, the invention has the following advantages:
(1) According to the fundus image optic disc macula lutea detection method and device based on the deep neural network, the fundus color photos are labeled and preprocessed, so that color information and structural information are considered, and a positioning result is more accurate and robust;
(2) According to the fundus image optic disc macula lutea detection method and device based on the deep neural network, model training is carried out through the enhanced domain graph and the pseudo label graph to obtain two networks, and segmentation precision is improved;
(3) The fundus image optic disc macula lutea detection method and device based on the deep neural network provided by the invention simultaneously cover two detection tasks of optic disc and macula lutea, and simultaneously realize macula lutea positioning and optic disc segmentation.
Of course, it is not necessary for any product in which the invention is practiced to achieve all of the above-described advantages at the same time.
Drawings
Embodiments of the invention are further described below with reference to the accompanying drawings:
FIG. 1 is a flowchart of a method for detecting macula of an eye fundus image optic disc based on a deep neural network according to an embodiment of the present invention;
FIG. 2 is a diagram of a pseudo tag segmentation network (NetF) and a video disc segmentation network (NetS) according to a preferred embodiment of the present invention;
FIG. 3 is a schematic diagram of an input module according to a preferred embodiment of the present invention;
FIG. 4 is a diagram of a ResNet basic module A according to a preferred embodiment of the present invention;
FIG. 5 is a diagram of a ResNet basic module B according to a preferred embodiment of the present invention;
FIG. 6 is a block diagram of a DAC module according to a preferred embodiment of the present invention;
FIG. 7 is a diagram of an RMP module according to a preferred embodiment of the present invention;
FIG. 8 is a schematic diagram of a Dec module according to a preferred embodiment of the present invention;
FIG. 9 is a schematic diagram of an output module according to a preferred embodiment of the present invention;
fig. 10 is a schematic diagram of a fully connected module according to a preferred embodiment of the invention.
Detailed Description
The following examples are given for the detailed implementation and the specific operation procedures, but the scope of the present invention is not limited to the following examples.
Fig. 1 is a flowchart of a fundus image optic disc macula detecting method based on a deep neural network according to an embodiment of the present invention.
Referring to fig. 1, the method for detecting the macula of a fundus image optic disc based on a deep neural network of the present embodiment includes:
s11: establishing a data set;
giving a data set containing a plurality of fundus color photographs, giving corresponding optic disc center position labels or whole optic disc region labels and macular center position labels, and dividing the data set into a training set and a verification set;
s12: pre-treating;
preprocessing an original fundus color photograph to obtain an enhanced domain image, labeling a macular position of a video disc, and establishing a pseudo label image;
s13: establishing a model;
establishing a model for positioning the macula lutea of the optic disc and segmenting the optic disc, and performing model training and verification on the training set and the verification set which are divided in the S11 by using the enhanced domain map and the pseudo label map obtained in the S12;
s14: testing the model;
and positioning the optic disc and the macula lutea on the verification set divided in the step S11 by adopting the model constructed in the step S13, extracting the region of interest, and segmenting by using the model constructed in the step S13 to obtain the final macula lutea, optic disc positioning and optic disc segmentation results.
In a preferred embodiment, the position of the macula of the data set is calibrated, i.e. coordinated
Incorporating disc position calibration
For the data to be divided into disks, a complete disk region map I is included gt . In one embodiment, model training uses 4-fold cross-validation, i.e., the data set is divided into four parts, three of which are used as the training set and the other is used as the validation set, and the process is repeated until all four parts of data are used as the validation set.
In a preferred embodiment, the preprocessing in S12 includes: gaussian filtering, background subtraction. S12 specifically includes: for original RGB eyeground color chart I ori Preprocessing the image to obtain an enhanced image I eh The process can be represented as:
I eh ＝4(I ori -G(σ)*I ori )+0.5
where G (σ) is a gaussian filter, σ is its variance, σ represents the image convolution operation, and σ is set to 1/30 of the image field radius.
In a preferred embodiment, the step of creating the pseudo tag map in S12 specifically includes: position calibration of optic disc and macula
And
generating a pseudo label map of a video disc
To include a center of a circle
Radius of
To
A binary plot of a circle of one fifth distance. Similarly, a false label map of the macula lutea can be generated
In a preferred embodiment, the model training in S13 includes: and performing model training by using the real coordinate labels to obtain a regression network (NetP), performing model training by using the pseudo label graph to obtain a pseudo label segmentation network (NetF), and performing model training by using the real optic disc region labels to obtain an optic disc segmentation network (NetS). The model training comprises the following steps: the device comprises an input module for feature extraction, a Res module, a DAC module for encoding, an RMP module, a Dec module for decoding, an output module and a full-connection module. The input module, the Res module, the DAC module, the RMP module, the Dec module and the output module are connected to form a pseudo tag segmentation network (NetF) and a video disc segmentation network (NetS), a loss function is constructed by utilizing the difference between a pseudo tag or video disc area and a pseudo tag or video disc segmentation result, a training set training model is input, and network parameters are updated in an iterative mode. The input module, the Res module, the DAC module, the RMP module and the full-connection module are connected to form a regression network (NetP), a loss function is constructed by using the difference between the real position calibration and the regression result, a training set and a verification set training model are input, and network parameters are updated in an iterative mode. A schematic diagram of a network model of a pseudo tag segmentation network (NetF) and a optic disc segmentation network (NetS) is shown in fig. 2, wherein Res denotes a Res module, DAC denotes a Dense aperture Convolution module (DAC), RMP denotes a Residual Multi-kernel pooling module (RMP), dec denotes a decoder module,
representing a tensor addition.
In the preferred embodiment, the input module in S13 is used to perform convolution, batch normalization, nonlinear excitation, maximum pooling and summation on the two images. Specifically, the method comprises the following steps: the RGB domain fundus image and the enhanced domain image after the second step of preprocessing are respectively input into an input module through an upper path and a lower path, the processing modes of the two paths in the input module are the same, and the RGB domain fundus image and the enhanced domain image sequentially pass through convolution (Conv), batch standardization (BN), linear rectification function excitation (Relu) and maximum pooling (Maxpool). The convolution kernel size of the convolution layer is 7 × 7, the number of input channels is 3, the number of output channels is 64, the step size is 2, and the number of filled pixels (pad) is 3. Through the processing, the two paths of characteristic images of the RGB domain eyeground image and the enhanced domain image are respectively obtained, and the two characteristic images are added and then output to the next module. The process is shown in figure 3.
In the preferred embodiment, the feature map output by the input module in S13 is passed through 4 Res modules to extract deeper features. The Res module is a module proposed by ResNet and is formed by connecting a series of ResNet basic modules in series. The ResNet basic module can be divided into A, B, as shown in FIGS. 4 and 5. The Res1 module is formed by connecting 3 ResNet basic modules B in series; the Res2 module is formed by connecting 1 ResNet basic module A and 3 ResNet basic modules B in series; the Res3 module is formed by connecting 1 ResNet basic module A and 5 ResNet basic modules B in series; the Res4 module is formed by connecting 1 ResNet basic module A and 2 ResNet basic modules B in series. In fig. 4 and 5, conv3x3 and Conv1x1 denote convolutions with convolution kernel sizes of 3 × 3 and 1 × 1, respectively, and s =2 denotes a step size of 2, that is, a downsampling operation is included.
The number of all convolution input channels in the Res1 module is the number of the characteristic diagram channels of the previous stage, namely 64, and the number of output channels is also 64; the number of input channels of two convolutional layers with the step length of 2 in the Res2 module is 64, the number of output channels is 128, and the number of input channels and the number of output channels of the rest convolutional layers are 128; the number of input channels of two convolutional layers with the step length of 2 in the Res3 module is 128, the number of output channels is 256, and the number of input channels and the number of output channels of the rest convolutional layers are 256; the input channel number of two convolutional layers with the step length of 2 in the Res4 module is 256, the output channel number is 512, and the input channel number and the output channel number of the rest convolutional layers are 512.
In the preferred embodiment, the DAC module in S13 comprises a series of hole convolution layers of different convolution kernel sizes and hole sizes. The RMP block contains pooling at scales of 2, 3, 5 and 6, upsampling at scales of 2, 3, 5 and 6, respectively, and a tensor stitching process. Specifically, the method comprises the following steps: extracting the information of different scales from the 14 × 14 and 512 channel feature map extracted by the Res4 module through the DAC module to obtain a 14 × 14 and 512 channel feature map containing information of different scales; and performing multi-scale pooling and upsampling by an RMP module, and performing tensor splicing on the channel dimension to obtain a feature map of a 14 multiplied by 14, 516 channel. The DAC module is shown in fig. 6, where the numbers before Conv indicate the convolution kernel size, rate indicates the hole size, and channel indicates the number of input and output channels. The RMP block is shown in fig. 7, where posing represents the maximum pooling operation, the previous number represents the pooling metric, the number before Conv represents the convolution kernel size, the convolution input channel is 512, the output channel is 1, upsample represents upsampling, and Concatenate represents tensor stitching.
In the preferred embodiment, the signature output by the RMP block in S13 is decoded by 4 Dec blocks. The schematic diagram of the Dec block is shown in fig. 8. In the figure, convT3x3 represents a deconvolution operation with a convolution kernel size of 3x3, with a step size of 2. Specifically, the method comprises the following steps: for a plurality of Dec modules, if the number of characteristic diagram channels input by each Dec module is Ch, the number of input channels of the first Conv1x1 convolution layer of the Dec module is Ch, and the number of output channels is Ch
[·]Represents rounding down; the number of input and output channels of the first deconvolution layer of the Dec module is equal to
The number of input channels of the second Conv1x1 convolution layer of the Dec module is
Number of channels outputWhich is a fixed value, in this embodiment the 4 Dec modules are 256, 128, 64 in that order. And (3) adding the feature maps output by the first 3 Dec modules and the feature maps output by the Res3, res2 and Res1 modules respectively to tensors, and inputting the tensor sum to the next Dec module. As shown in fig. 2.
In a preferred embodiment, the feature map output by the last Dec module is analyzed by an output module to obtain a final output result, and a schematic diagram of the output module is shown in fig. 9. Specifically, the method comprises the following steps: in the figure, convT4x4 represents a deconvolution layer with convolution kernel size of 4x4, the step size is 2, the input channel of the layer is 64, and the output channel is 32 in the invention; conv3x3 represents a convolution layer with convolution kernel size of 3x3, wherein the input channel of the convolution layer is 32, and the output channel of the convolution layer is 32; conv1x1 represents a convolutional layer with a convolutional kernel size of 1x1, which in the present invention has an input channel of 32, the number of output channels equal to the number of categories of the problem plus one, for a pseudo label split network (NetF), the number of output channels is 3, and for a video disc split network (NetS), the number of output channels is 2.
In the preferred embodiment, the regression loss function minimization problem in S13 is defined as:
wherein, θ represents the model parameter, and n is the number of images trained each time, i.e. the number of batchs. In the following, it is assumed that P denotes a coordinate vector, I denotes an output graph, pre in the subscript denotes a model prediction value, and gt denotes a true annotation value.
In the preferred embodiment, the segmentation loss function minimization problem in S13 is defined as:
wherein, θ represents the model parameter, and n is the number of images trained each time, i.e. the number of batchs. The other binomial loss functions are respectively cross entropy loss L CE And Dice coefficient loss L Dice ；
The cross entropy loss is:
the Dice coefficient loss is:
wherein H and W respectively represent the height and width of the image, the unit is pixel, K is the number of categories and is the number of target categories plus the number of backgrounds, and the coordinates (i, j, K) represent the value of the ith row and jth column of the image. In one embodiment, H and W are both set to 448, K is 3 for the pseudo label split network (NetF) and K is 2 for the video disc split network (NetS).
In a preferred embodiment, the regression network (NetP) input module, res module, DAC module, RMP module and (NetF), (NetS) have the same structure and connection. The RMP module is connected to a full-connection module, and a schematic structural diagram of the full-connection module is shown in fig. 10.
In a preferred embodiment, the segmentation performed by using the model constructed in S13 in S14 includes the pseudo label segmentation result output by the pseudo label segmentation network and the macula lutea, optic disc positioning, and optic disc segmentation result output by the regression network. When the region shape of the pseudo label segmentation result is a regular quasi-circle shape, the pseudo label segmentation result output by a pseudo label segmentation network (NetF) is used as a final macular and optic disc positioning result; and when the region shape of the pseudo label segmentation structure is an irregular shape, positioning the macula lutea and the optic disc output by a regression network (NetP) as a final macula lutea and optic disc positioning result.
Further, the region shape of the pseudo tag segmentation result is determined according to a shape factor SI, which is:
wherein C is the perimeter of the pseudo label segmentation result region, and S is the area of the pseudo label segmentation result region;
and when the SI is within the preset range, judging that the SI is in a round-like shape, and when the SI exceeds the preset range, judging that the SI is in an irregular shape. Theoretically, if the segmentation result area of the pseudo label is circular, the SI is 4 pi, and the SI being too high or too low can indicate that the segmentation result area is irregular in shape. In one embodiment, two thresholds T are set min =11 and T max =12.2, when T min ≤SI≤T max And taking the center of the pseudo label segmentation region output by the NetF as the positioning result of the optic disk and the yellow spot. When SI is<T min Or SI > T max Or when the division result output by the NetF includes a plurality of regions, the NetP output is taken as the positioning result of the optic disk and the macula lutea.
In a preferred embodiment, after the positioning of the optic disc and the macula lutea is completed in S14, an image block with the center located at the center of the optic disc and the side length 1.6 times the distance from the optic disc to the macula lutea is cut out, and the image block is input into the NetS to obtain a segmentation result of the optic disc region.
The effect of the fundus image optic disc macula detecting method based on the deep neural network of the above embodiment is verified below with reference to a specific example. After four-fold cross validation of different methods, experimental results are obtained as shown in tables 1 and 2:
TABLE 1 comparison of the optic disc macula orientation ED on different data validation sets by different methods (best results are shown bold)
TABLE 2 optic disc segmentation Dice comparison of different methods on different data validation sets (best results are shown bold)
As can be seen from the results in tables 1 and 2, compared with other methods based on a deep neural network, the method for detecting macula lutea of an optic disc provided by the present invention has better results on a verification set, is closer to a true calibration, and has more comprehensive functions than other methods.
In one embodiment, there is further provided a depth neural network-based fundus image optic disc macula detecting apparatus for implementing the depth neural network-based fundus image optic disc macula detecting method of the above embodiment, which includes: the system comprises a data set establishing module, a preprocessing module, a model establishing module and a model testing module; wherein the content of the first and second substances,
the data set establishing module is used for giving a data set containing a plurality of fundus color photographs, giving corresponding optic disc center position labels or whole optic disc region labels and macula lutea center position labels, and dividing the data set into a training set and a verification set;
the preprocessing module is used for preprocessing an original fundus color photograph to obtain an enhanced domain map, labeling the position of the macula lutea of a video disc and establishing a pseudo label map;
the model establishing module is used for establishing a model for the macular location and optic disc segmentation of the optic disc, and performing model training and verification on a training set and a verification set which are divided in the data set establishing module by utilizing an enhanced domain map and a pseudo label map which are obtained by the preprocessing module;
the model testing module is used for positioning the optic disc and the yellow spot on the verification set divided in the data set establishing module by adopting the model established by the model establishing module, then extracting the region of interest, and segmenting by using the model established by the model establishing module to obtain the final yellow spot, optic disc positioning and optic disc segmenting results.
The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, and not to limit the invention. Any modifications and variations within the scope of the description, which may occur to those skilled in the art, are intended to be within the scope of the invention.
Claims (9)
1. A fundus image optic disc macula lutea detection method based on a deep neural network is characterized by comprising the following steps:
s11: establishing a data set;
giving a data set containing a plurality of fundus color photographs, giving corresponding optic disc center position labels or whole optic disc region labels and macular center position labels, and dividing the data set into a training set and a verification set;
s12: pre-treating;
preprocessing an original fundus color photograph to obtain an enhanced domain image, labeling a macular position of a video disc, and establishing a pseudo label image;
s13: establishing a model;
establishing a model for positioning macula lutea of the optic disc and segmenting the optic disc, and performing model training and verification on the training set and the verification set which are divided in the S11 by using the enhanced domain map and the pseudo label map obtained in the S12;
the model in the S13 is trained to minimize a loss function, wherein the loss function is a regression loss function constructed by using the difference between the real coordinate label and the regression network regression prediction result; a segmentation loss function is constructed by utilizing the gap between the pseudo label graph and the pseudo label segmentation result; constructing a segmentation loss function by using the difference between the real optic disc region label and the optic disc region segmentation structure;
in the step S13, model training is carried out by using the real coordinate labels to obtain a regression network, model training is carried out by using the pseudo label graph to obtain a pseudo label segmentation network, and model training is carried out by using the real optic disc region labels to obtain an optic disc segmentation network; respectively inputting a training set and a verification set training model based on the obtained loss function, and iteratively updating each network parameter;
s14: testing the model;
and positioning the optic disc and the macula lutea on the verification set divided in the step S11 by adopting the model constructed in the step S13, extracting the region of interest, and segmenting by using the model constructed in the step S13 to obtain the final macula lutea, optic disc positioning and optic disc segmentation results.
2. The deep neural network-based fundus image optic disc macula detecting method according to claim 1, characterized in that the preprocessing in S12 includes: gaussian filtering and background subtraction.
3. The deep neural network-based fundus image optic disc macula detecting method of claim 2, wherein in the S12 original fundus color photograph I ori Preprocessing to obtain an enhanced domain map I eh Further comprises the following steps:
I eh ＝4(I ori -G(σ)*I ori )+0.5，
where G (σ) is a gaussian filter, σ is its variance, and σ represents the image convolution operation, and σ is set to 1/30 of the image field radius.
4. The method for detecting the macula of an eye fundus image optic disc based on a deep neural network of claim 1, wherein the establishing of the pseudo tag map in S12 further comprises: and generating a circular area with the radius of a preset radius and the center of the circle positioned in the center of the optic disc and/or the yellow spots.
5. The deep neural network-based fundus image optic disc macula detecting method according to claim 1, characterized in that the model constructed in S13 in S14 is subjected to a test including a pseudo label segmentation result output by a pseudo label segmentation network, macula lutea output by a regression network, a optic disc positioning result, and a optic disc segmentation result;
when the region shape of the pseudo label segmentation result is a regular quasi-circular shape, the pseudo label segmentation result output by the pseudo label segmentation network is used as a final yellow spot and optic disc positioning result; and when the region of the pseudo label segmentation structure is in an irregular shape, positioning the yellow spots and the optic discs output by the regression network to obtain a final yellow spot and optic disc positioning result.
6. The method of detecting a fundus image based on a deep neural network for the macula lutea of an optic disc according to claim 5, wherein the region shape of the pseudo tag segmentation result is determined in accordance with a shape factor SI that is:
wherein C is the perimeter of the pseudo label segmentation result region, and S is the area of the pseudo label segmentation result region;
and when the SI is within the preset range, judging that the SI is in a round-like shape, and when the SI exceeds the preset range, judging that the SI is in an irregular shape.
7. The deep neural network-based fundus image optic disc macula detecting method of claim 1, wherein the regression loss function minimization problem is defined as:
wherein, theta represents a model parameter, and n is the number of images trained each time, namely the number of batchs; let P denote the coordinate vector, I denote the output graph, pre in the subscript denotes the model prediction value, gt denotes the true annotation value.
8. The method for fundus image-based macular disc detection by a deep neural network according to claim 1, wherein the segmentation loss function minimization problem is defined as:
wherein, theta represents a model parameter, and n is the number of images trained each time, namely the number of batchs; the other binomial loss functions are respectively cross entropy loss L CE And Dice coefficient loss L Dice ；
The cross entropy loss is:
the Dice coefficient loss is:
wherein H and W respectively represent the height and width of the image, the unit is pixel, K is the number of categories and is the number of target categories plus the number of backgrounds, and the coordinates (i, j, K) represent the value of the ith row and jth column of the image.
9. A fundus image optic disc macula lutea detecting apparatus based on a deep neural network, for realizing the fundus image optic disc macula lutea detecting method based on a deep neural network according to any one of claims 1 to 8, comprising: the system comprises a data set establishing module, a preprocessing module, a model establishing module and a model testing module; wherein the content of the first and second substances,
the data set establishing module is used for giving a data set containing a plurality of fundus color photographs, giving corresponding optic disc center position labels or whole optic disc region labels and macular center position labels, and dividing the data set into a training set and a verification set;
the preprocessing module is used for preprocessing an original fundus color photograph to obtain an enhanced domain image, labeling the position of the macula lutea of a video disc and establishing a pseudo label image;
the model establishing module is used for establishing a model for the macular location of the optic disc and the segmentation of the optic disc, and performing model training and verification on a training set and a verification set which are divided in the data set establishing module by utilizing the enhanced domain map and the pseudo label map which are obtained by the preprocessing module; in the model building module, the model is trained to minimize a loss function, wherein the loss function is a regression loss function built by using the difference between a real coordinate label and a regression network regression prediction result; a segmentation loss function is constructed by utilizing the gap between the pseudo label graph and the pseudo label segmentation result; constructing a segmentation loss function by using the difference between the real optic disc region label and the optic disc region segmentation structure; carrying out model training by using a real coordinate label to obtain a regression network, carrying out model training by using a pseudo label graph to obtain a pseudo label segmentation network, and carrying out model training by using a real optic disc region label to obtain an optic disc segmentation network; respectively inputting a training set and a verification set training model based on the obtained loss function, and iteratively updating each network parameter;
the model testing module is used for positioning the optic disc and the yellow spot on the verification set divided in the data set establishing module by adopting the model established by the model establishing module, extracting the region of interest, and segmenting by using the model established by the model establishing module to obtain the final yellow spot, optic disc positioning and optic disc segmenting results.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011427915.2A CN112669260B (en) | 2020-12-07 | 2020-12-07 | Fundus image optic disc macula lutea detection method and device based on deep neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011427915.2A CN112669260B (en) | 2020-12-07 | 2020-12-07 | Fundus image optic disc macula lutea detection method and device based on deep neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112669260A CN112669260A (en) | 2021-04-16 |
| CN112669260B true CN112669260B (en) | 2023-02-28 |
Family
ID=75401580
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011427915.2A Active CN112669260B (en) | 2020-12-07 | 2020-12-07 | Fundus image optic disc macula lutea detection method and device based on deep neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112669260B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116777794B (en) * | 2023-08-17 | 2023-11-03 | 简阳市人民医院 | Cornea foreign body image processing method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109598733A (en) * | 2017-12-31 | 2019-04-09 | 南京航空航天大学 | Retinal fundus images dividing method based on the full convolutional neural networks of depth |
| CN110084824A (en) * | 2019-04-26 | 2019-08-02 | 山东财经大学 | Tongue body image partition method, system, equipment and medium based on symmetrical level set |
| CN110598652A (en) * | 2019-09-18 | 2019-12-20 | 上海鹰瞳医疗科技有限公司 | Fundus data prediction method and device |
| CN111028230A (en) * | 2019-12-24 | 2020-04-17 | 贵州大学 | Fundus image optic disc and macula lutea positioning detection algorithm based on YOLO-V3 |
-
2020
- 2020-12-07 CN CN202011427915.2A patent/CN112669260B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109598733A (en) * | 2017-12-31 | 2019-04-09 | 南京航空航天大学 | Retinal fundus images dividing method based on the full convolutional neural networks of depth |
| CN110084824A (en) * | 2019-04-26 | 2019-08-02 | 山东财经大学 | Tongue body image partition method, system, equipment and medium based on symmetrical level set |
| CN110598652A (en) * | 2019-09-18 | 2019-12-20 | 上海鹰瞳医疗科技有限公司 | Fundus data prediction method and device |
| CN111028230A (en) * | 2019-12-24 | 2020-04-17 | 贵州大学 | Fundus image optic disc and macula lutea positioning detection algorithm based on YOLO-V3 |
Non-Patent Citations (4)
| Title |
|---|
| Automatic detection and segmentation of optic disc and fovea in retinal images;Renoh Johnson Chalakkal 等;《IET Image Processing》;20181130;第2100-2110页 * |
| 基于深度学习的眼底图像分割与辅助诊断研究;高宏杰;《中国优秀硕士学位论文全文数据库 信息科技辑》;20200215;正文第3-5章 * |
| 眼底图像中自动定位黄斑中心凹的快速定位方法研究;杨波 等;《中国医学装备》;20190731;第16卷(第7期);第33-36页 * |
| 融合残差注意力机制的UNet视盘分割;侯向丹等;《中国图象图形学报》;20200916(第09期);第1915-1929页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112669260A (en) | 2021-04-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110889853B (en) | Tumor segmentation method based on residual error-attention deep neural network | |
| CN110889852B (en) | Liver segmentation method based on residual error-attention deep neural network | |
| CN109118495B (en) | Retinal vessel segmentation method and device | |
| Zhang et al. | Automated semantic segmentation of red blood cells for sickle cell disease | |
| CN108764342B (en) | Semantic segmentation method for optic discs and optic cups in fundus image | |
| CN112581458B (en) | Image processing method and device | |
| Wu et al. | Ultrasound image segmentation method for thyroid nodules using ASPP fusion features | |
| CN112669260B (en) | Fundus image optic disc macula lutea detection method and device based on deep neural network | |
| Zhao et al. | D2a u-net: Automatic segmentation of covid-19 lesions from ct slices with dilated convolution and dual attention mechanism | |
| Zhang et al. | An o-shape neural network with attention modules to detect junctions in biomedical images without segmentation | |
| CN115272647A (en) | Lung image recognition processing method and system | |
| Jian et al. | Dual-Branch-UNet: A Dual-Branch Convolutional Neural Network for Medical Image Segmentation. | |
| CN112529915B (en) | Brain tumor image segmentation method and system | |
| CN113781468A (en) | Tongue image segmentation method based on lightweight convolutional neural network | |
| CN116563189B (en) | Medical image cross-contrast synthesis method and system based on deep learning | |
| CN111754530A (en) | Prostate ultrasonic image segmentation and classification method | |
| Iqbal et al. | LDMRes-Net: A Lightweight Neural Network for Efficient Medical Image Segmentation on IoT and Edge Devices | |
| Tai et al. | Using deep learning to evaluate the segmentation of liver cell from biopsy image | |
| CN114882282A (en) | Neural network prediction method for colorectal cancer treatment effect based on MRI and CT images | |
| WO2022227193A1 (en) | Liver region segmentation method and apparatus, and electronic device and storage medium | |
| Dickson et al. | A Dual Channel Multiscale Convolution U-Net Methodfor Liver Tumor Segmentation from Abdomen CT Images | |
| Alazawee et al. | Analyzing and detecting hemorrhagic and ischemic strokebased on bit plane slicing and edge detection algorithms | |
| CN114445419A (en) | Lung segment segmentation method, device and system based on bronchial topological structure | |
| CN114267408A (en) | SNP data and MRI data based classification method and terminal equipment | |
| Solak et al. | Adrenal Tumor Segmentation on U-Net: A Study About Effect of Different Parameters in Deep Learning. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |