CN115035127A - Retinal vessel segmentation method based on generative confrontation network - Google Patents
Retinal vessel segmentation method based on generative confrontation network Download PDFInfo
- Publication number
- CN115035127A CN115035127A CN202210805978.XA CN202210805978A CN115035127A CN 115035127 A CN115035127 A CN 115035127A CN 202210805978 A CN202210805978 A CN 202210805978A CN 115035127 A CN115035127 A CN 115035127A
- Authority
- CN
- China
- Prior art keywords
- vessel segmentation
- network
- retinal
- image
- retinal vessel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 78
- 210000001210 retinal vessel Anatomy 0.000 title claims abstract description 45
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000012549 training Methods 0.000 claims abstract description 36
- 238000007781 pre-processing Methods 0.000 claims abstract description 13
- 210000004204 blood vessel Anatomy 0.000 claims description 28
- 238000011176 pooling Methods 0.000 claims description 27
- 210000001525 retina Anatomy 0.000 claims description 14
- 238000012937 correction Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 230000004256 retinal image Effects 0.000 abstract description 5
- 238000013527 convolutional neural network Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 11
- 238000012545 processing Methods 0.000 description 11
- 230000007246 mechanism Effects 0.000 description 10
- 230000006870 function Effects 0.000 description 8
- 201000010099 disease Diseases 0.000 description 7
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 7
- 230000035945 sensitivity Effects 0.000 description 6
- 230000004913 activation Effects 0.000 description 5
- 238000005457 optimization Methods 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 238000005070 sampling Methods 0.000 description 4
- 230000009466 transformation Effects 0.000 description 4
- 230000015556 catabolic process Effects 0.000 description 3
- 238000006731 degradation reaction Methods 0.000 description 3
- 238000003745 diagnosis Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 239000011800 void material Substances 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 239000012141 concentrate Substances 0.000 description 2
- 230000008602 contraction Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 239000003814 drug Substances 0.000 description 2
- 208000030533 eye disease Diseases 0.000 description 2
- 230000005484 gravity Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000009191 jumping Effects 0.000 description 2
- 230000003902 lesion Effects 0.000 description 2
- 210000000056 organ Anatomy 0.000 description 2
- 230000001575 pathological effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000740 bleeding effect Effects 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 210000000416 exudates and transudate Anatomy 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000002911 mydriatic effect Effects 0.000 description 1
- 238000002203 pretreatment Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a retinal vessel segmentation method based on a generative confrontation network, which comprises the following steps: preprocessing a fundus retinal image set to obtain a training image set, constructing a retinal blood vessel segmentation model based on GAN, training the retinal blood vessel segmentation model by using the training image set, segmenting a fundus retinal image to be segmented by using the trained retinal blood vessel segmentation model, outputting segmented sub-images, and splicing the segmented sub-images to obtain a complete retinal blood vessel segmentation image; the invention combines with the Ladder Net, introduces the concept of a plurality of pairs of coders and decoders of the Ladder Net into the original improved U-shaped network structure, connects two improved U-shaped networks together in parallel, improves the traditional residual error module into a parameter sharing residual error module, and trains through a circulation layer formed by convolutional layers with two sharing parameters, thereby capturing complex characteristic points in images and improving the segmentation precision.
Description
Technical Field
The invention belongs to the field of fundus retina image processing, and particularly relates to a retinal blood vessel segmentation method based on a generative confrontation network.
Background
The eye, which occupies more than 80 percent of the total information of the body, is therefore an indispensable organ of the human body, whose health is inseparable from human life. At present, doctors mainly rely on fundus images when diagnosing eye diseases, and the states of some organs are displayed through transformation of the fundus images. Retinal blood vessels can be clearly observed in fundus images, and are the most basic basis for doctors to diagnose and prevent certain diseases. The division of the ocular fundus blood vessels is a decisive condition for doctors to diagnose the conditions of patients according to ocular fundus images, and has a main significance for clinical medicine, the key judgment index of whether the human body is healthy is the morphological structure (such as diameter, length and the like) of the retinal blood vessels, and the transformation of the retinal blood vessels can reflect certain diseases of the human body to a large extent, so that the analysis of the retinal blood vessels is necessary for the doctors to effectively diagnose the diseases.
When diagnosing the diseases of the fundus retina, manual segmentation of retinal blood vessels according to experience is a common method for doctors at present. However, this method has the problems of dense distribution of ocular fundus blood vessels and low contrast, and may have lesions such as bleeding spots and exudates, and many tiny blood vessels, which are combined with the influence of lesion noise, so that the traditional manual segmentation becomes huge and cumbersome, and the blood vessel segmentation depending on manual segmentation is inefficient, susceptible to subjectivity and high in error rate.
In order to prevent potential fundus diseases and improve diagnosis efficiency of the fundus diseases, related medical images need to be processed and analyzed by means of technologies such as image processing, computer vision, deep learning and the like, an advanced and accurate retinal vessel segmentation algorithm is designed, and related pathological structures can be effectively quantified and visualized, so that accurate diagnosis and accurate treatment of disease conditions can be realized by computer assistance and even a doctor is replaced.
In recent years, with the rapid development of image processing and analysis techniques, medical image processing using a computer has been widely applied to various subjects and fields of medicine. Traditional machine learning algorithms sometimes require manual selection of features based on experimentation, which is not conducive to automated implementation. Various algorithm network structures in deep learning are used for processing retinal vessel segmentation and obtain great achievements, a single network training mode of a traditional convolutional neural network is widely applied, but the problems of low sensitivity and low precision of fine vessel segmentation still exist, compared with the convolutional neural network, a generative confrontation network comprises a generator part and a discriminator part which are formed by two arbitrary sub-networks, and the two sub-networks are used for confrontation training and are mutually cooperated and optimized, so that a vessel characteristic optimization segmentation model can be better learned.
Disclosure of Invention
The invention aims to provide a retinal blood vessel segmentation method based on a generative countermeasure network, which aims to solve the problems of low sensitivity and small blood vessel segmentation in the prior art and realize automatic retinal blood vessel segmentation and high-precision good segmentation effect of small blood vessels.
The invention adopts the following technical scheme: a retinal vessel segmentation method based on a generative confrontation network is characterized by comprising the following steps:
step S1: preprocessing a fundus retina image set to obtain a training image set,
step S2: constructing a retinal vessel segmentation model based on the GAN,
step S3: training a retinal vessel segmentation model by using a training image set,
step S4: segmenting the fundus retina image to be segmented by utilizing a trained retina blood vessel segmentation model, outputting a segmented subimage,
step S5: splicing the sub-images after segmentation to obtain a complete retina blood vessel segmentation image,
wherein, the network for generating the retinal vessel segmentation model is formed by connecting two coding and decoding U-shaped networks in parallel by using Ladder Net,
each of the codec U-type networks includes:
four first structure blocks, a hollow convolution module and four second structure blocks which are connected in sequence,
each first structure block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence,
each second structure block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence.
Furthermore, the discrimination network of the retinal vessel segmentation model comprises four third structural blocks, an average pooling layer and a full-link layer which are sequentially connected, and each third structural block comprises a convolution layer, a residual module and a pooling layer which are sequentially connected.
Further, the preprocessing method is to sequentially perform RGB green channel conversion, contrast-limited histogram equalization, and gamma correction.
Further, the pretreatment method further comprises the following steps: expansion and rotation.
The invention has the beneficial effects that: the generator model is a coding and decoding structure of a convolutional neural network, an improved U-shaped network model with a residual error module, a cavity convolution and an attention mechanism is further improved, a plurality of pairs of coder-decoder concepts of a Ladder Net are introduced into the original improved U-shaped network structure by combining the Ladder Net, the two improved U-shaped networks are connected together in parallel, the traditional residual error module is improved into a parameter sharing residual error module, a circulating layer formed by convolutional layers with two sharing parameters is set for training, and therefore complex characteristic points in an image are captured, and the segmentation precision is improved; the discriminator model of the invention introduces Res Block to prevent the network degradation problem, and generates a network model with good segmentation performance through the mutual alternate iterative training of the two modules.
Drawings
FIG. 1 is a diagram showing the effect of the fundus image preprocessing according to the present invention;
FIG. 2 is a schematic view of the fundus image expansion process of the present invention;
FIG. 3 is a diagram of the overall architecture of the generative countermeasure network of the present invention;
FIG. 4 is a diagram of a generator network model architecture of the present invention;
FIG. 5 is a diagram of a discriminator network model structure according to the present invention;
FIG. 6 is a graph of the results of the segmentation of the DRIVE data set according to example 1 of the present invention;
FIG. 7 is a STARE data set segmentation result chart according to embodiment 1 of the present invention.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
The invention discloses a retinal vessel segmentation method based on a generative confrontation network, which comprises the following steps as shown in figure 3:
step S1: preprocessing a fundus retina image set to obtain a training image set,
step S2: constructing a retinal vessel segmentation model based on the GAN,
step S3: training a retinal vessel segmentation model by using a training image set,
step S4: segmenting the fundus retinal image to be segmented by using the trained retinal vessel segmentation model, outputting a segmented sub-image,
step S5: splicing the sub-images after segmentation to obtain a complete retina blood vessel segmentation image,
wherein, as shown in fig. 4, the network for generating the retinal vessel segmentation model is formed by connecting two codec U-type networks in parallel by using Ladder Net,
each of the codec U-type networks includes:
four first structure blocks, a hollow convolution module and four second structure blocks which are connected in sequence,
each first structure block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence,
each second structure block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence.
As shown in fig. 5, the discrimination network of the retinal vessel segmentation model includes four third structure blocks, an average pooling layer, and a full-link layer, which are connected in sequence, and each of the third structure blocks includes a convolution layer, a residual error module, and a pooling layer, which are connected in sequence.
The preprocessing method comprises the steps of sequentially carrying out RGB green channel conversion, histogram equalization with limited contrast and gamma correction, and further comprises the following steps: expansion and rotation.
Before step S1, in order to obtain the fundus retinal image in the public DRIVE and STARE data set as a data set, the preprocessing includes performing RGB green channel conversion, contrast-limited histogram equalization and gamma correction on the data set to perform image enhancement preprocessing, as shown in fig. 1, to enhance the contrast between the background and the blood vessel and reduce noise interference, thereby facilitating the segmentation of the subsequent blood vessel; the preprocessing also comprises the steps of expanding the data set and expanding the number of the data set by rotating, and as shown in FIG. 2, the overfitting problem of the training model caused by the excessively small data amount is avoided.
Step S1, extracting a green channel with high contrast between blood vessels and the background from the fundus image of the training set, and observing fine blood vessel textures well; then CLAHE is equalized by using a contrast-limited self-adaptive histogram to enhance data so as to increase the contrast of a local area of the image and reduce the interference of background noise to a certain extent; finally, gamma correction is used, nonlinear operation is carried out on the image, pixels of the corrected image and pixels of the corrected image are in an exponential relation before correction, too white or too dark of the fundus image caused by uneven light is corrected, and the contrast ratio of the image background and blood vessels is increased by detecting a dark color area and a light color area in the fundus image and increasing the specific gravity difference of the dark color area and the light color area; and expanding the preprocessed data set, and performing rotation operation to expand the number of the data set, so that the over-fitting problem of the undersize data in the training model is avoided.
The coding and decoding U-shaped network is an improved U-shaped structure in the convolutional neural network, and the two improved coding and decoding U-shaped networks are connected in parallel to form a W-shaped network model. Because the U-shaped convolutional neural network only has one pair of coders and decoders, and the information flow path is limited, the concept of a plurality of pairs of coders and decoders of the Ladder Net is combined, two improved coding and decoding U-shaped network structures are connected together in parallel, and compared with the original network structure, the U-shaped convolutional neural network has a plurality of information flow paths.
The generation network is responsible for extracting the characteristic information of an input image, and two coding and decoding U-shaped networks are connected in parallel by utilizing a Ladder Net, wherein each coding and decoding U-shaped network comprises four first structure blocks, a hollow convolution module and four second structure blocks which are connected in sequence, each first structure block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence, and each second structure block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence.
The sizes of the convolution layers of the generated network are all 3 multiplied by 3, the number of the characteristic channels is 64, a Shared-parameter residual module (Shared-weight Resblock) is adopted to replace the original convolution module, the Relu activation function and the BN layer are used for solving the problem of poor network generalization capability caused by excessive network layer number, the characteristic utilization rate is improved, and the step length of the pooling layer is 2.
The original convolutional layer module at the bottom of the coding and decoding U-shaped network is replaced by a hole convolutional module, the characteristic diagram is reduced to 1/2, a hole rate parameter with the size of 3 is increased by the hole convolutional module compared with the original convolutional block, the receptive field is amplified, more image details are reserved, and the problem that the blood vessel segmentation details are broken incompletely is solved.
The judgment network decoding part of the retinal vessel segmentation model is responsible for recovering image detail information and is used for carrying out up-sampling on the extracted features, the judgment network comprises four third structural blocks, an average pooling layer and a full-connection layer which are connected in sequence, and each third structural block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence.
The size of the convolution layer of the network is judged to be 3 x 3, the structure is the same as that of the generated network, and the highest decoding layer is the convolution layer of 1x 1 for outputting the segmentation result. A residual error module is introduced into the discrimination network, the used convolutions are convolution operations with the size of 3 multiplied by 3, the input data firstly extract features through convolution with the size of 3 multiplied by 3 and the step length of 2, the training speed is accelerated through BN, the nonlinear change of the network architecture is increased through Relu activation function, and then the convolution with the size of 3 multiplied by 3 and the step length of 1 is used for extracting the features and simultaneously ensuring the resolution of the feature map to be unchanged; then, after passing through a Res Block structure, reducing the dimension by using a Max pooling of a maximum pooling layer, and totally carrying out the same down-sampling operation for four times; and then carrying out global average pooling Avg pooling operation by adopting an average pooling check feature map with the same size as the feature map being 20, and finally obtaining a final output score by adopting a layer of full-connection layer FC operation, wherein the final output score is used for judging the truth of the input image.
An Attention mechanism is introduced in a decoding stage and an encoding stage, and an Attention mechanism is utilized to carry out weighting operation similar to a mask mechanism on features connected from a contraction path through jumping, so that important feature weights are larger, secondary feature weights are small, and even the important feature weights are directly discarded, and a network is made to concentrate on processing the important features;
because only one pair of coder-decoders exist and the information flow path is limited, the improved U-shaped network structure is combined with a plurality of pairs of coder-decoder concepts of the Ladder Net, two improved U-shaped network structures are connected together in parallel, the parameter sharing residual concept is utilized to reduce the parameter quantity and improve the ownership of the information flow, the two U-shaped network structures are connected through hop level, namely the channel numbers are added and connected in parallel, the improved W-shaped network structure can be seen from the model, compared with the original network structure, the improved W-shaped network structure has a plurality of information flow paths, and therefore the retinal vessel segmentation network model has the capability of capturing complex features and obtains higher accuracy.
The loss function of the generative countermeasure network is:
wherein, x represents real image data, Z represents random noise, G (Z) represents the output result of the generated model, D (x) represents the output of the discrimination model to the real training sample, D (G (Z)) represents the output of the generated model generated image, P (Z)) represents the output of the generated model generated image, and date (x) Representing the distribution of real training samples, P Z (Z) represents the input noise profile and E represents the mathematical expectation. For any input image, the discrimination model D outputs a [0,1 ]]The value of the interval. The optimization process can be considered as optimization of the generative model and the discriminant model.
The method aims at the problems of low sensitivity and fine vessel under-segmentation of the traditional convolutional neural network single network training, and carries out optimization and improvement. For a generator model, for a coding and decoding structure of a convolutional neural network, an improved U-shaped network model with a residual error module, a cavity convolution and an attention mechanism is proposed for further transformation, a plurality of pairs of coder-decoder concepts of a Ladder Net are introduced into the original improved U-shaped network structure by combining the Ladder Net, the two improved U-shaped networks are connected together in parallel, the traditional residual error module is improved into a parameter sharing residual error module, a circulating layer formed by convolutional layers with two sharing parameters is set for training, and therefore complex feature points in an image are captured, and therefore segmentation accuracy is improved; for the discriminator model, Res Block is introduced to prevent the network degradation problem.
And through the mutual alternate iterative training of the two models, a network model with good segmentation performance is generated. Experimental research proves that the algorithm model can be used for driving and STARE fundus data sets, the accuracy, specificity and sensitivity are respectively 96.84% and 96.87%, 98.46% and 98.61%, 83.26% and 83.94%, and blood vessel characteristics can be better learned, so that the algorithm model can be trained to realize segmentation of retinal blood vessels more accurately and sensitively.
Example 1
1. Obtaining data
The fundus image is enhanced, and the contrast of the target blood vessel of the fundus image and the background is improved so as to highlight the details of the blood vessel. The experimental subject is a DRIVE data set, which is one of the most widely applied standard libraries in the field of retinal image processing, the storage format of the image of the DRIVE data base is JPEG, the image is captured by a Canon CR53CCD non-mydriatic camera, the image has a 45-degree view field angle, the image with the size of 768 x 584 pixels is obtained, the view field is circular, and the DRIVE directly divides 40 colored fundus images into two groups, namely a training set and a testing set, wherein the two groups are respectively 20, and the driving method is more convenient to use without the need of additional grouping by users. The STATE data set is a database established by medical institutions in the United STATEs for the research of full-automatic diagnosis of human eye diseases, pictures in the data set are shot by a TopCon-50 camera at an angle of 35 degrees, the size of the pictures is 700 x 605 pixels, and the total number of the pictures is 20, wherein 20 pictures which can be used for comparing results with expert manual annotation pictures comprise 10 fundus pictures of normal and pathological eyeground pictures respectively and are provided with expert manual annotation segmentation pictures.
The data set is used for preprocessing the part, specifically, a green channel with high contrast between blood vessels and the background is extracted from the fundus image of the training set, so that fine blood vessel textures can be observed well; then, CLAHE is equalized by using a self-adaptive histogram with limited contrast to enhance data so as to increase the contrast of a local area of the image and reduce the interference of background noise to a certain extent; and finally, gamma correction is used, the image is subjected to nonlinear operation, the corrected image pixels and the corrected image pixels are in an exponential relationship, the over-white or over-dark of the fundus image caused by uneven light is corrected, and the contrast between the image background and the blood vessel is increased by detecting a dark color area and a light color area in the fundus image and increasing the specific gravity difference of the dark color area and the light color area. The effect schematic diagram is shown in fig. 1, and the fundus image, the green channel image, the CLAHE processing and the gamma conversion processing are sequentially performed.
Since the development of deep learning is not independent of the support of large-scale data, small batches of data not only do not achieve good results but also may cause overfitting problems. However, since the data acquisition of medical images is difficult and the data volume is often small, the data must be augmented when deep learning network training is used. And (3) expanding a training set data set, carrying out patch processing on image enhanced random cutting, and taking small blocks of 48 multiplied by 48 as the data set. The experiment needs to select a center in a random mode from a complete fundus retina image to cut out sub-images with uniform size, and the selection range of the sub-images comprises plaques out of the visual field, so that the neural network can learn how to distinguish the visual field boundary from the blood vessel.
2. Constructing retinal vessel segmentation model
A generator model part:
the structure is that after a U-shaped structure in a convolutional neural network is improved, two improved U-shaped networks are connected in parallel to form a W-shaped network model. Firstly, a coding part of a U-shaped network structure is responsible for extracting characteristic information of an input image, a coding stage is composed of four structural blocks and used for extracting characteristics of the image, each structural block is composed of a down-sampling layer and a convolution layer, the size of each convolution layer is 3 multiplied by 3, the number of characteristic channels is 64, a Shared-parameter residual error module (Shared-weights reblock) is adopted to replace an original convolution module, a Relu activation function and a BN layer are used for solving the problem that the network generalization capability is poor due to the fact that the number of network layers is too large, the problem that the network generalization capability is poor is used for improving the characteristic utilization rate, and the structural blocks are connected through a pooling layer with the step length of 2.
And secondly, replacing the original convolutional layer Module at the bottom of the U-shaped network model with a void Convolution Module (scaled Convolution Module), reducing the characteristic diagram to 1/2, increasing a void rate parameter with the size of 3 compared with the original convolutional block by the void Convolution Module added in the text, amplifying a receptive field, reserving more image details, and preventing the problem of incomplete fracture of the blood vessel segmentation details.
And thirdly, the decoding part is responsible for recovering image detail information, and is composed of four structural blocks in the decoding stage and used for performing upsampling on the extracted features, each structural block is composed of an upsampling layer and a convolutional layer, the size of the convolutional layer is 3 x 3, the structure of the convolutional layer is the same as that of the coding module, and the highest decoding layer is a 1x 1 convolutional layer and used for outputting a segmentation result. An Attention mechanism is introduced in a decoding stage and an encoding stage, and an Attention mechanism is utilized to carry out a weighting operation similar to a mask mechanism on the features connected from a contraction path through jumping, so that the important feature weight is larger, the secondary feature weight is small and even directly discarded, and the network is led to concentrate on processing the important features. The specific operation is that the part receives an eigen map from jump connection and an eigen map from up sampling, the two eigen maps are respectively adjusted to half of the number of channels through convolution of 1x 1, the sizes of the two tensors are ensured to be identical, then corresponding elements of the two tensors are added to obtain a result, the result is changed into a tensor which has the same size as the input eigen map but has the number of channels of 1 through convolution of 1x 1, the tensor is changed into a value range of 0-1 through a Sigmoid function, the result is used as a weight matrix and is multiplied by corresponding elements of the tensor obtained by the jump connection through a broadcasting mechanism, so that the numerical value in the jump connection obtains different importance degrees through weighting of the weight, and the probability of retinal vessel information loss is effectively reduced through multi-feature fusion in combination with a jump connection structure of an original U-Net network, and the segmentation precision is improved, and the image information is restored.
Finally, since only one pair of coders and decoders has limited information flow path, an improved U-shaped network structure is combined with a plurality of pairs of coders and decoders of Ladder Net, the next path is connected through skip level, namely the number of channels is added and connected in parallel, the structure is the same as the encoding stage, namely an encoding path and a decoding path, and a parameter sharing residual concept is utilized, the module consists of two 3 x 3 convolutions, and the two convolutions are different from a traditional residual module in that the two convolutions share parameters and are added with Relu activation functions, BN, Dropout and other optimization functions, so that the situations of overfitting, gradient explosion and the like are prevented, the number of parameters is reduced, and the ownership of information flow is improved. Two U-shaped channels are connected through skip level, namely, the channel numbers are added and connected in parallel, the improved W-shaped network structure can be seen from the model, and compared with the original network structure, the improved W-shaped network structure has a plurality of information flow paths, so that the retinal blood vessel segmentation network model has the capability of capturing complex features and obtains higher accuracy, and the model is a partial complete structure schematic diagram of a generator model as shown in fig. 4.
The discriminator model part:
a residual error module is introduced into the discriminator, the used convolutions are convolution operations of 3 multiplied by 3, the input data are subjected to convolution with the size of 3 multiplied by 3 and the step length of 2 to extract features, the training speed is accelerated through BN, the nonlinear change of a network architecture is increased through a Relu activation function, and the convolution with the size of 3 multiplied by 3 and the step length of 1 is used for extracting the features and ensuring the resolution of a feature map to be unchanged; then, after passing through a Res Block structure, performing dimensionality reduction by using a Max pooling layer, and performing the same downsampling operation for four times in total; and then carrying out global average pooling Avg pooling operation by adopting an average pooling check characteristic diagram which is 20 and the size of the characteristic diagram, and finally obtaining a final output score by adopting a layer of full-connection layer FC operation. The final output score is used to determine whether the input image is "true" or false, as shown in fig. 5, which is a partial complete structural diagram of the discriminator model.
GAN retinal vessel segmentation section:
the design of the retinal vessel segmentation network model is divided into a generation model G and a discrimination model D, wherein G and D are both convolutional neural network structures, the training target of G is to generate a false-spurious segmentation result, so that D cannot correctly distinguish positive and negative samples, the training target of D is to maximize the classification accuracy, the output of positive samples approaches to 1, and the output of negative samples approaches to 0. The training method of G and D is independent alternate iterative training, and mutual confrontation is carried out until Nash balance is achieved, and GAN can train a network model which is better to fit real data.
The method of the invention is used for carrying out experiments on a DRIVE and STARE fundus image database, the retina segmentation results are shown in fig. 6 and 7, the left column in fig. 6 and 7 is an original fundus image, the middle column is a standard fundus blood vessel image which is manually segmented, and the right column is the fundus blood vessel segmentation result obtained in the embodiment.
Through observation, the model algorithm is relatively good in retinal vessel segmentation result, basically consistent with a standard image result manually segmented by an expert, and more obvious and clear in characteristic segmentation of main vessels.
Aiming at the defects of the blood vessel segmentation, the method constructs a new improved algorithm model based on the GAN. For a generator model, for a coding and decoding structure of a convolutional neural network, an improved U-shaped network model with a residual error module, a cavity convolution and an attention mechanism is proposed for further transformation, a plurality of pairs of coder-decoder concepts of a Ladder Net are introduced into the original improved U-shaped network structure by combining the Ladder Net, the two improved U-shaped networks are connected together in parallel, the traditional residual error module is improved into a parameter sharing residual error module, a circulating layer formed by convolutional layers with two sharing parameters is set for training, and therefore complex feature points in an image are captured, and therefore segmentation accuracy is improved; for the discriminator model, Res Block is introduced to prevent the network degradation problem. Through alternate iterative training of the generated model and the discriminant model, the generated model is continuously optimized, a network model with good segmentation capability is finally obtained, and test experiments are performed on two public fundus data sets of DRIVE and STARE. The accuracy of vessel segmentation on the database of DRIVE and STARE reaches 0.9684 and 0.9687, the specificity reaches 0.9846 and 0.9861, the sensitivity reaches 0.8326 and 0.8394, compared with other convolutional networks, the sensitivity is greatly improved, the problems of fine vessel under-segmentation and over-segmentation are greatly improved, and the overall performance of the network is superior to that of the existing segmentation algorithm.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (4)
1. A retinal vessel segmentation method based on a generative confrontation network is characterized by comprising the following steps:
step S1: preprocessing a fundus retina image set to obtain a training image set,
step S2: constructing a retinal vessel segmentation model based on the GAN,
step S3: training a retinal vessel segmentation model by using a training image set,
step S4: segmenting the fundus retina image to be segmented by utilizing a trained retina blood vessel segmentation model, outputting a segmented subimage,
step S5: splicing the sub-images after segmentation to obtain a complete retina blood vessel segmentation image,
wherein, the network for generating the retinal vessel segmentation model is formed by connecting two coding and decoding U-shaped networks in parallel by using Ladder Net,
each of the codec U-type networks includes:
four first structure blocks, a hollow convolution module and four second structure blocks which are connected in sequence,
each first structure block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence,
each second structure block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence.
2. The retinal vessel segmentation method based on the generative countermeasure network as claimed in claim 1, wherein the discriminative network of the retinal vessel segmentation model comprises four third structural blocks, an average pooling layer, and a full-link layer, which are connected in sequence, and each of the third structural blocks comprises a convolution layer, a residual module, and a pooling layer, which are connected in sequence.
3. The retinal vessel segmentation method based on the generative countermeasure network as claimed in claim 1 or 2, wherein the preprocessing method comprises sequential RGB green channel conversion, contrast-limited histogram equalization and gamma correction.
4. The retinal vessel segmentation method based on the generative countermeasure network as claimed in claim 1 or 2, wherein the preprocessing method further comprises: expansion and rotation.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210805978.XA CN115035127A (en) | 2022-07-08 | 2022-07-08 | Retinal vessel segmentation method based on generative confrontation network |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210805978.XA CN115035127A (en) | 2022-07-08 | 2022-07-08 | Retinal vessel segmentation method based on generative confrontation network |
Publications (1)
Publication Number | Publication Date |
---|---|
CN115035127A true CN115035127A (en) | 2022-09-09 |
Family
ID=83129016
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210805978.XA Pending CN115035127A (en) | 2022-07-08 | 2022-07-08 | Retinal vessel segmentation method based on generative confrontation network |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115035127A (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116152197A (en) * | 2023-02-21 | 2023-05-23 | 北京长木谷医疗科技有限公司 | Knee joint segmentation method, knee joint segmentation device, electronic equipment and computer readable storage medium |
CN116188492A (en) * | 2023-02-21 | 2023-05-30 | 北京长木谷医疗科技有限公司 | Hip joint segmentation method, device, electronic equipment and computer readable storage medium |
-
2022
- 2022-07-08 CN CN202210805978.XA patent/CN115035127A/en active Pending
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116152197A (en) * | 2023-02-21 | 2023-05-23 | 北京长木谷医疗科技有限公司 | Knee joint segmentation method, knee joint segmentation device, electronic equipment and computer readable storage medium |
CN116188492A (en) * | 2023-02-21 | 2023-05-30 | 北京长木谷医疗科技有限公司 | Hip joint segmentation method, device, electronic equipment and computer readable storage medium |
CN116188492B (en) * | 2023-02-21 | 2024-04-26 | 北京长木谷医疗科技股份有限公司 | Hip joint segmentation method, device, electronic equipment and computer readable storage medium |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109635862B (en) | Sorting method for retinopathy of prematurity plus lesion | |
CN110930418B (en) | Retina blood vessel segmentation method fusing W-net and conditional generation confrontation network | |
CN112132817A (en) | Retina blood vessel segmentation method for fundus image based on mixed attention mechanism | |
CN115035127A (en) | Retinal vessel segmentation method based on generative confrontation network | |
CN113256641B (en) | Skin lesion image segmentation method based on deep learning | |
CN108764342B (en) | Semantic segmentation method for optic discs and optic cups in fundus image | |
CN112446891A (en) | Medical image segmentation method based on U-Net network brain glioma | |
CN113012163A (en) | Retina blood vessel segmentation method, equipment and storage medium based on multi-scale attention network | |
CN113724206B (en) | Fundus image blood vessel segmentation method and system based on self-supervision learning | |
Li et al. | Multi-stage attention-unet for wireless capsule endoscopy image bleeding area segmentation | |
CN115375711A (en) | Image segmentation method of global context attention network based on multi-scale fusion | |
CN114038564A (en) | Noninvasive risk prediction method for diabetes | |
CN111242949B (en) | Fundus image blood vessel segmentation method based on full convolution neural network multi-scale features | |
CN112869697A (en) | Judgment method for simultaneously identifying stage and pathological change characteristics of diabetic retinopathy | |
CN111462082A (en) | Focus picture recognition device, method and equipment and readable storage medium | |
Miao et al. | Classification of diabetic retinopathy based on multiscale hybrid attention mechanism and residual algorithm | |
CN113421250A (en) | Intelligent fundus disease diagnosis method based on lesion-free image training | |
CN110992309B (en) | Fundus image segmentation method based on deep information transfer network | |
CN117036288A (en) | Tumor subtype diagnosis method for full-slice pathological image | |
Xiao et al. | SE-MIDNet Based on Deep Learning for Diabetic Retinopathy Classification | |
CN116092667A (en) | Disease detection method, system, device and storage medium based on multi-mode images | |
CN115049682A (en) | Retina blood vessel segmentation method based on multi-scale dense network | |
CN114519722A (en) | Carotid artery extraction method based on convolutional neural network | |
CN117078697B (en) | Fundus disease seed detection method based on cascade model fusion | |
Vinta et al. | Segmentation and Classification of Interstitial Lung Diseases Based on Hybrid Deep Learning Network Model |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |