CN115035127A - Retinal vessel segmentation method based on generative confrontation network - Google Patents

Retinal vessel segmentation method based on generative confrontation network Download PDF

Info

Publication number
CN115035127A
CN115035127A CN202210805978.XA CN202210805978A CN115035127A CN 115035127 A CN115035127 A CN 115035127A CN 202210805978 A CN202210805978 A CN 202210805978A CN 115035127 A CN115035127 A CN 115035127A
Authority
CN
China
Prior art keywords
vessel segmentation
network
retinal
image
retinal vessel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210805978.XA
Other languages
Chinese (zh)
Inventor
陈伟
朱振宁
冯蕊
穆倩
丁婉莹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian University of Science and Technology
Original Assignee
Xian University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian University of Science and Technology filed Critical Xian University of Science and Technology
Priority to CN202210805978.XA priority Critical patent/CN115035127A/en
Publication of CN115035127A publication Critical patent/CN115035127A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/40Image enhancement or restoration using histogram techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/90Dynamic range modification of images or parts thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/26Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/44Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10024Color image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30041Eye; Retina; Ophthalmic

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a retinal vessel segmentation method based on a generative confrontation network, which comprises the following steps: preprocessing a fundus retinal image set to obtain a training image set, constructing a retinal blood vessel segmentation model based on GAN, training the retinal blood vessel segmentation model by using the training image set, segmenting a fundus retinal image to be segmented by using the trained retinal blood vessel segmentation model, outputting segmented sub-images, and splicing the segmented sub-images to obtain a complete retinal blood vessel segmentation image; the invention combines with the Ladder Net, introduces the concept of a plurality of pairs of coders and decoders of the Ladder Net into the original improved U-shaped network structure, connects two improved U-shaped networks together in parallel, improves the traditional residual error module into a parameter sharing residual error module, and trains through a circulation layer formed by convolutional layers with two sharing parameters, thereby capturing complex characteristic points in images and improving the segmentation precision.

Description

Retinal vessel segmentation method based on generative confrontation network
Technical Field
The invention belongs to the field of fundus retina image processing, and particularly relates to a retinal blood vessel segmentation method based on a generative confrontation network.
Background
The eye, which occupies more than 80 percent of the total information of the body, is therefore an indispensable organ of the human body, whose health is inseparable from human life. At present, doctors mainly rely on fundus images when diagnosing eye diseases, and the states of some organs are displayed through transformation of the fundus images. Retinal blood vessels can be clearly observed in fundus images, and are the most basic basis for doctors to diagnose and prevent certain diseases. The division of the ocular fundus blood vessels is a decisive condition for doctors to diagnose the conditions of patients according to ocular fundus images, and has a main significance for clinical medicine, the key judgment index of whether the human body is healthy is the morphological structure (such as diameter, length and the like) of the retinal blood vessels, and the transformation of the retinal blood vessels can reflect certain diseases of the human body to a large extent, so that the analysis of the retinal blood vessels is necessary for the doctors to effectively diagnose the diseases.
When diagnosing the diseases of the fundus retina, manual segmentation of retinal blood vessels according to experience is a common method for doctors at present. However, this method has the problems of dense distribution of ocular fundus blood vessels and low contrast, and may have lesions such as bleeding spots and exudates, and many tiny blood vessels, which are combined with the influence of lesion noise, so that the traditional manual segmentation becomes huge and cumbersome, and the blood vessel segmentation depending on manual segmentation is inefficient, susceptible to subjectivity and high in error rate.
In order to prevent potential fundus diseases and improve diagnosis efficiency of the fundus diseases, related medical images need to be processed and analyzed by means of technologies such as image processing, computer vision, deep learning and the like, an advanced and accurate retinal vessel segmentation algorithm is designed, and related pathological structures can be effectively quantified and visualized, so that accurate diagnosis and accurate treatment of disease conditions can be realized by computer assistance and even a doctor is replaced.
In recent years, with the rapid development of image processing and analysis techniques, medical image processing using a computer has been widely applied to various subjects and fields of medicine. Traditional machine learning algorithms sometimes require manual selection of features based on experimentation, which is not conducive to automated implementation. Various algorithm network structures in deep learning are used for processing retinal vessel segmentation and obtain great achievements, a single network training mode of a traditional convolutional neural network is widely applied, but the problems of low sensitivity and low precision of fine vessel segmentation still exist, compared with the convolutional neural network, a generative confrontation network comprises a generator part and a discriminator part which are formed by two arbitrary sub-networks, and the two sub-networks are used for confrontation training and are mutually cooperated and optimized, so that a vessel characteristic optimization segmentation model can be better learned.
Disclosure of Invention
The invention aims to provide a retinal blood vessel segmentation method based on a generative countermeasure network, which aims to solve the problems of low sensitivity and small blood vessel segmentation in the prior art and realize automatic retinal blood vessel segmentation and high-precision good segmentation effect of small blood vessels.
The invention adopts the following technical scheme: a retinal vessel segmentation method based on a generative confrontation network is characterized by comprising the following steps:
step S1: preprocessing a fundus retina image set to obtain a training image set,
step S2: constructing a retinal vessel segmentation model based on the GAN,
step S3: training a retinal vessel segmentation model by using a training image set,
step S4: segmenting the fundus retina image to be segmented by utilizing a trained retina blood vessel segmentation model, outputting a segmented subimage,
step S5: splicing the sub-images after segmentation to obtain a complete retina blood vessel segmentation image,
wherein, the network for generating the retinal vessel segmentation model is formed by connecting two coding and decoding U-shaped networks in parallel by using Ladder Net,
each of the codec U-type networks includes:
four first structure blocks, a hollow convolution module and four second structure blocks which are connected in sequence,
each first structure block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence,
each second structure block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence.
Furthermore, the discrimination network of the retinal vessel segmentation model comprises four third structural blocks, an average pooling layer and a full-link layer which are sequentially connected, and each third structural block comprises a convolution layer, a residual module and a pooling layer which are sequentially connected.
Further, the preprocessing method is to sequentially perform RGB green channel conversion, contrast-limited histogram equalization, and gamma correction.
Further, the pretreatment method further comprises the following steps: expansion and rotation.
The invention has the beneficial effects that: the generator model is a coding and decoding structure of a convolutional neural network, an improved U-shaped network model with a residual error module, a cavity convolution and an attention mechanism is further improved, a plurality of pairs of coder-decoder concepts of a Ladder Net are introduced into the original improved U-shaped network structure by combining the Ladder Net, the two improved U-shaped networks are connected together in parallel, the traditional residual error module is improved into a parameter sharing residual error module, a circulating layer formed by convolutional layers with two sharing parameters is set for training, and therefore complex characteristic points in an image are captured, and the segmentation precision is improved; the discriminator model of the invention introduces Res Block to prevent the network degradation problem, and generates a network model with good segmentation performance through the mutual alternate iterative training of the two modules.
Drawings
FIG. 1 is a diagram showing the effect of the fundus image preprocessing according to the present invention;
FIG. 2 is a schematic view of the fundus image expansion process of the present invention;
FIG. 3 is a diagram of the overall architecture of the generative countermeasure network of the present invention;
FIG. 4 is a diagram of a generator network model architecture of the present invention;
FIG. 5 is a diagram of a discriminator network model structure according to the present invention;
FIG. 6 is a graph of the results of the segmentation of the DRIVE data set according to example 1 of the present invention;
FIG. 7 is a STARE data set segmentation result chart according to embodiment 1 of the present invention.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
The invention discloses a retinal vessel segmentation method based on a generative confrontation network, which comprises the following steps as shown in figure 3:
step S1: preprocessing a fundus retina image set to obtain a training image set,
step S2: constructing a retinal vessel segmentation model based on the GAN,
step S3: training a retinal vessel segmentation model by using a training image set,
step S4: segmenting the fundus retinal image to be segmented by using the trained retinal vessel segmentation model, outputting a segmented sub-image,
step S5: splicing the sub-images after segmentation to obtain a complete retina blood vessel segmentation image,
wherein, as shown in fig. 4, the network for generating the retinal vessel segmentation model is formed by connecting two codec U-type networks in parallel by using Ladder Net,
each of the codec U-type networks includes:
four first structure blocks, a hollow convolution module and four second structure blocks which are connected in sequence,
each first structure block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence,
each second structure block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence.
As shown in fig. 5, the discrimination network of the retinal vessel segmentation model includes four third structure blocks, an average pooling layer, and a full-link layer, which are connected in sequence, and each of the third structure blocks includes a convolution layer, a residual error module, and a pooling layer, which are connected in sequence.
The preprocessing method comprises the steps of sequentially carrying out RGB green channel conversion, histogram equalization with limited contrast and gamma correction, and further comprises the following steps: expansion and rotation.
Before step S1, in order to obtain the fundus retinal image in the public DRIVE and STARE data set as a data set, the preprocessing includes performing RGB green channel conversion, contrast-limited histogram equalization and gamma correction on the data set to perform image enhancement preprocessing, as shown in fig. 1, to enhance the contrast between the background and the blood vessel and reduce noise interference, thereby facilitating the segmentation of the subsequent blood vessel; the preprocessing also comprises the steps of expanding the data set and expanding the number of the data set by rotating, and as shown in FIG. 2, the overfitting problem of the training model caused by the excessively small data amount is avoided.
Step S1, extracting a green channel with high contrast between blood vessels and the background from the fundus image of the training set, and observing fine blood vessel textures well; then CLAHE is equalized by using a contrast-limited self-adaptive histogram to enhance data so as to increase the contrast of a local area of the image and reduce the interference of background noise to a certain extent; finally, gamma correction is used, nonlinear operation is carried out on the image, pixels of the corrected image and pixels of the corrected image are in an exponential relation before correction, too white or too dark of the fundus image caused by uneven light is corrected, and the contrast ratio of the image background and blood vessels is increased by detecting a dark color area and a light color area in the fundus image and increasing the specific gravity difference of the dark color area and the light color area; and expanding the preprocessed data set, and performing rotation operation to expand the number of the data set, so that the over-fitting problem of the undersize data in the training model is avoided.
The coding and decoding U-shaped network is an improved U-shaped structure in the convolutional neural network, and the two improved coding and decoding U-shaped networks are connected in parallel to form a W-shaped network model. Because the U-shaped convolutional neural network only has one pair of coders and decoders, and the information flow path is limited, the concept of a plurality of pairs of coders and decoders of the Ladder Net is combined, two improved coding and decoding U-shaped network structures are connected together in parallel, and compared with the original network structure, the U-shaped convolutional neural network has a plurality of information flow paths.
The generation network is responsible for extracting the characteristic information of an input image, and two coding and decoding U-shaped networks are connected in parallel by utilizing a Ladder Net, wherein each coding and decoding U-shaped network comprises four first structure blocks, a hollow convolution module and four second structure blocks which are connected in sequence, each first structure block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence, and each second structure block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence.
The sizes of the convolution layers of the generated network are all 3 multiplied by 3, the number of the characteristic channels is 64, a Shared-parameter residual module (Shared-weight Resblock) is adopted to replace the original convolution module, the Relu activation function and the BN layer are used for solving the problem of poor network generalization capability caused by excessive network layer number, the characteristic utilization rate is improved, and the step length of the pooling layer is 2.
The original convolutional layer module at the bottom of the coding and decoding U-shaped network is replaced by a hole convolutional module, the characteristic diagram is reduced to 1/2, a hole rate parameter with the size of 3 is increased by the hole convolutional module compared with the original convolutional block, the receptive field is amplified, more image details are reserved, and the problem that the blood vessel segmentation details are broken incompletely is solved.
The judgment network decoding part of the retinal vessel segmentation model is responsible for recovering image detail information and is used for carrying out up-sampling on the extracted features, the judgment network comprises four third structural blocks, an average pooling layer and a full-connection layer which are connected in sequence, and each third structural block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence.
The size of the convolution layer of the network is judged to be 3 x 3, the structure is the same as that of the generated network, and the highest decoding layer is the convolution layer of 1x 1 for outputting the segmentation result. A residual error module is introduced into the discrimination network, the used convolutions are convolution operations with the size of 3 multiplied by 3, the input data firstly extract features through convolution with the size of 3 multiplied by 3 and the step length of 2, the training speed is accelerated through BN, the nonlinear change of the network architecture is increased through Relu activation function, and then the convolution with the size of 3 multiplied by 3 and the step length of 1 is used for extracting the features and simultaneously ensuring the resolution of the feature map to be unchanged; then, after passing through a Res Block structure, reducing the dimension by using a Max pooling of a maximum pooling layer, and totally carrying out the same down-sampling operation for four times; and then carrying out global average pooling Avg pooling operation by adopting an average pooling check feature map with the same size as the feature map being 20, and finally obtaining a final output score by adopting a layer of full-connection layer FC operation, wherein the final output score is used for judging the truth of the input image.
An Attention mechanism is introduced in a decoding stage and an encoding stage, and an Attention mechanism is utilized to carry out weighting operation similar to a mask mechanism on features connected from a contraction path through jumping, so that important feature weights are larger, secondary feature weights are small, and even the important feature weights are directly discarded, and a network is made to concentrate on processing the important features;
because only one pair of coder-decoders exist and the information flow path is limited, the improved U-shaped network structure is combined with a plurality of pairs of coder-decoder concepts of the Ladder Net, two improved U-shaped network structures are connected together in parallel, the parameter sharing residual concept is utilized to reduce the parameter quantity and improve the ownership of the information flow, the two U-shaped network structures are connected through hop level, namely the channel numbers are added and connected in parallel, the improved W-shaped network structure can be seen from the model, compared with the original network structure, the improved W-shaped network structure has a plurality of information flow paths, and therefore the retinal vessel segmentation network model has the capability of capturing complex features and obtains higher accuracy.
The loss function of the generative countermeasure network is:
Figure BDA0003737658670000071
wherein, x represents real image data, Z represents random noise, G (Z) represents the output result of the generated model, D (x) represents the output of the discrimination model to the real training sample, D (G (Z)) represents the output of the generated model generated image, P (Z)) represents the output of the generated model generated image, and date (x) Representing the distribution of real training samples, P Z (Z) represents the input noise profile and E represents the mathematical expectation. For any input image, the discrimination model D outputs a [0,1 ]]The value of the interval. The optimization process can be considered as optimization of the generative model and the discriminant model.
The method aims at the problems of low sensitivity and fine vessel under-segmentation of the traditional convolutional neural network single network training, and carries out optimization and improvement. For a generator model, for a coding and decoding structure of a convolutional neural network, an improved U-shaped network model with a residual error module, a cavity convolution and an attention mechanism is proposed for further transformation, a plurality of pairs of coder-decoder concepts of a Ladder Net are introduced into the original improved U-shaped network structure by combining the Ladder Net, the two improved U-shaped networks are connected together in parallel, the traditional residual error module is improved into a parameter sharing residual error module, a circulating layer formed by convolutional layers with two sharing parameters is set for training, and therefore complex feature points in an image are captured, and therefore segmentation accuracy is improved; for the discriminator model, Res Block is introduced to prevent the network degradation problem.
And through the mutual alternate iterative training of the two models, a network model with good segmentation performance is generated. Experimental research proves that the algorithm model can be used for driving and STARE fundus data sets, the accuracy, specificity and sensitivity are respectively 96.84% and 96.87%, 98.46% and 98.61%, 83.26% and 83.94%, and blood vessel characteristics can be better learned, so that the algorithm model can be trained to realize segmentation of retinal blood vessels more accurately and sensitively.
Example 1
1. Obtaining data
The fundus image is enhanced, and the contrast of the target blood vessel of the fundus image and the background is improved so as to highlight the details of the blood vessel. The experimental subject is a DRIVE data set, which is one of the most widely applied standard libraries in the field of retinal image processing, the storage format of the image of the DRIVE data base is JPEG, the image is captured by a Canon CR53CCD non-mydriatic camera, the image has a 45-degree view field angle, the image with the size of 768 x 584 pixels is obtained, the view field is circular, and the DRIVE directly divides 40 colored fundus images into two groups, namely a training set and a testing set, wherein the two groups are respectively 20, and the driving method is more convenient to use without the need of additional grouping by users. The STATE data set is a database established by medical institutions in the United STATEs for the research of full-automatic diagnosis of human eye diseases, pictures in the data set are shot by a TopCon-50 camera at an angle of 35 degrees, the size of the pictures is 700 x 605 pixels, and the total number of the pictures is 20, wherein 20 pictures which can be used for comparing results with expert manual annotation pictures comprise 10 fundus pictures of normal and pathological eyeground pictures respectively and are provided with expert manual annotation segmentation pictures.
The data set is used for preprocessing the part, specifically, a green channel with high contrast between blood vessels and the background is extracted from the fundus image of the training set, so that fine blood vessel textures can be observed well; then, CLAHE is equalized by using a self-adaptive histogram with limited contrast to enhance data so as to increase the contrast of a local area of the image and reduce the interference of background noise to a certain extent; and finally, gamma correction is used, the image is subjected to nonlinear operation, the corrected image pixels and the corrected image pixels are in an exponential relationship, the over-white or over-dark of the fundus image caused by uneven light is corrected, and the contrast between the image background and the blood vessel is increased by detecting a dark color area and a light color area in the fundus image and increasing the specific gravity difference of the dark color area and the light color area. The effect schematic diagram is shown in fig. 1, and the fundus image, the green channel image, the CLAHE processing and the gamma conversion processing are sequentially performed.
Since the development of deep learning is not independent of the support of large-scale data, small batches of data not only do not achieve good results but also may cause overfitting problems. However, since the data acquisition of medical images is difficult and the data volume is often small, the data must be augmented when deep learning network training is used. And (3) expanding a training set data set, carrying out patch processing on image enhanced random cutting, and taking small blocks of 48 multiplied by 48 as the data set. The experiment needs to select a center in a random mode from a complete fundus retina image to cut out sub-images with uniform size, and the selection range of the sub-images comprises plaques out of the visual field, so that the neural network can learn how to distinguish the visual field boundary from the blood vessel.
2. Constructing retinal vessel segmentation model
A generator model part:
the structure is that after a U-shaped structure in a convolutional neural network is improved, two improved U-shaped networks are connected in parallel to form a W-shaped network model. Firstly, a coding part of a U-shaped network structure is responsible for extracting characteristic information of an input image, a coding stage is composed of four structural blocks and used for extracting characteristics of the image, each structural block is composed of a down-sampling layer and a convolution layer, the size of each convolution layer is 3 multiplied by 3, the number of characteristic channels is 64, a Shared-parameter residual error module (Shared-weights reblock) is adopted to replace an original convolution module, a Relu activation function and a BN layer are used for solving the problem that the network generalization capability is poor due to the fact that the number of network layers is too large, the problem that the network generalization capability is poor is used for improving the characteristic utilization rate, and the structural blocks are connected through a pooling layer with the step length of 2.
And secondly, replacing the original convolutional layer Module at the bottom of the U-shaped network model with a void Convolution Module (scaled Convolution Module), reducing the characteristic diagram to 1/2, increasing a void rate parameter with the size of 3 compared with the original convolutional block by the void Convolution Module added in the text, amplifying a receptive field, reserving more image details, and preventing the problem of incomplete fracture of the blood vessel segmentation details.
And thirdly, the decoding part is responsible for recovering image detail information, and is composed of four structural blocks in the decoding stage and used for performing upsampling on the extracted features, each structural block is composed of an upsampling layer and a convolutional layer, the size of the convolutional layer is 3 x 3, the structure of the convolutional layer is the same as that of the coding module, and the highest decoding layer is a 1x 1 convolutional layer and used for outputting a segmentation result. An Attention mechanism is introduced in a decoding stage and an encoding stage, and an Attention mechanism is utilized to carry out a weighting operation similar to a mask mechanism on the features connected from a contraction path through jumping, so that the important feature weight is larger, the secondary feature weight is small and even directly discarded, and the network is led to concentrate on processing the important features. The specific operation is that the part receives an eigen map from jump connection and an eigen map from up sampling, the two eigen maps are respectively adjusted to half of the number of channels through convolution of 1x 1, the sizes of the two tensors are ensured to be identical, then corresponding elements of the two tensors are added to obtain a result, the result is changed into a tensor which has the same size as the input eigen map but has the number of channels of 1 through convolution of 1x 1, the tensor is changed into a value range of 0-1 through a Sigmoid function, the result is used as a weight matrix and is multiplied by corresponding elements of the tensor obtained by the jump connection through a broadcasting mechanism, so that the numerical value in the jump connection obtains different importance degrees through weighting of the weight, and the probability of retinal vessel information loss is effectively reduced through multi-feature fusion in combination with a jump connection structure of an original U-Net network, and the segmentation precision is improved, and the image information is restored.
Finally, since only one pair of coders and decoders has limited information flow path, an improved U-shaped network structure is combined with a plurality of pairs of coders and decoders of Ladder Net, the next path is connected through skip level, namely the number of channels is added and connected in parallel, the structure is the same as the encoding stage, namely an encoding path and a decoding path, and a parameter sharing residual concept is utilized, the module consists of two 3 x 3 convolutions, and the two convolutions are different from a traditional residual module in that the two convolutions share parameters and are added with Relu activation functions, BN, Dropout and other optimization functions, so that the situations of overfitting, gradient explosion and the like are prevented, the number of parameters is reduced, and the ownership of information flow is improved. Two U-shaped channels are connected through skip level, namely, the channel numbers are added and connected in parallel, the improved W-shaped network structure can be seen from the model, and compared with the original network structure, the improved W-shaped network structure has a plurality of information flow paths, so that the retinal blood vessel segmentation network model has the capability of capturing complex features and obtains higher accuracy, and the model is a partial complete structure schematic diagram of a generator model as shown in fig. 4.
The discriminator model part:
a residual error module is introduced into the discriminator, the used convolutions are convolution operations of 3 multiplied by 3, the input data are subjected to convolution with the size of 3 multiplied by 3 and the step length of 2 to extract features, the training speed is accelerated through BN, the nonlinear change of a network architecture is increased through a Relu activation function, and the convolution with the size of 3 multiplied by 3 and the step length of 1 is used for extracting the features and ensuring the resolution of a feature map to be unchanged; then, after passing through a Res Block structure, performing dimensionality reduction by using a Max pooling layer, and performing the same downsampling operation for four times in total; and then carrying out global average pooling Avg pooling operation by adopting an average pooling check characteristic diagram which is 20 and the size of the characteristic diagram, and finally obtaining a final output score by adopting a layer of full-connection layer FC operation. The final output score is used to determine whether the input image is "true" or false, as shown in fig. 5, which is a partial complete structural diagram of the discriminator model.
GAN retinal vessel segmentation section:
the design of the retinal vessel segmentation network model is divided into a generation model G and a discrimination model D, wherein G and D are both convolutional neural network structures, the training target of G is to generate a false-spurious segmentation result, so that D cannot correctly distinguish positive and negative samples, the training target of D is to maximize the classification accuracy, the output of positive samples approaches to 1, and the output of negative samples approaches to 0. The training method of G and D is independent alternate iterative training, and mutual confrontation is carried out until Nash balance is achieved, and GAN can train a network model which is better to fit real data.
The method of the invention is used for carrying out experiments on a DRIVE and STARE fundus image database, the retina segmentation results are shown in fig. 6 and 7, the left column in fig. 6 and 7 is an original fundus image, the middle column is a standard fundus blood vessel image which is manually segmented, and the right column is the fundus blood vessel segmentation result obtained in the embodiment.
Through observation, the model algorithm is relatively good in retinal vessel segmentation result, basically consistent with a standard image result manually segmented by an expert, and more obvious and clear in characteristic segmentation of main vessels.
Aiming at the defects of the blood vessel segmentation, the method constructs a new improved algorithm model based on the GAN. For a generator model, for a coding and decoding structure of a convolutional neural network, an improved U-shaped network model with a residual error module, a cavity convolution and an attention mechanism is proposed for further transformation, a plurality of pairs of coder-decoder concepts of a Ladder Net are introduced into the original improved U-shaped network structure by combining the Ladder Net, the two improved U-shaped networks are connected together in parallel, the traditional residual error module is improved into a parameter sharing residual error module, a circulating layer formed by convolutional layers with two sharing parameters is set for training, and therefore complex feature points in an image are captured, and therefore segmentation accuracy is improved; for the discriminator model, Res Block is introduced to prevent the network degradation problem. Through alternate iterative training of the generated model and the discriminant model, the generated model is continuously optimized, a network model with good segmentation capability is finally obtained, and test experiments are performed on two public fundus data sets of DRIVE and STARE. The accuracy of vessel segmentation on the database of DRIVE and STARE reaches 0.9684 and 0.9687, the specificity reaches 0.9846 and 0.9861, the sensitivity reaches 0.8326 and 0.8394, compared with other convolutional networks, the sensitivity is greatly improved, the problems of fine vessel under-segmentation and over-segmentation are greatly improved, and the overall performance of the network is superior to that of the existing segmentation algorithm.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims (4)

1. A retinal vessel segmentation method based on a generative confrontation network is characterized by comprising the following steps:
step S1: preprocessing a fundus retina image set to obtain a training image set,
step S2: constructing a retinal vessel segmentation model based on the GAN,
step S3: training a retinal vessel segmentation model by using a training image set,
step S4: segmenting the fundus retina image to be segmented by utilizing a trained retina blood vessel segmentation model, outputting a segmented subimage,
step S5: splicing the sub-images after segmentation to obtain a complete retina blood vessel segmentation image,
wherein, the network for generating the retinal vessel segmentation model is formed by connecting two coding and decoding U-shaped networks in parallel by using Ladder Net,
each of the codec U-type networks includes:
four first structure blocks, a hollow convolution module and four second structure blocks which are connected in sequence,
each first structure block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence,
each second structure block comprises a convolution layer, a residual error module and a pooling layer which are connected in sequence.
2. The retinal vessel segmentation method based on the generative countermeasure network as claimed in claim 1, wherein the discriminative network of the retinal vessel segmentation model comprises four third structural blocks, an average pooling layer, and a full-link layer, which are connected in sequence, and each of the third structural blocks comprises a convolution layer, a residual module, and a pooling layer, which are connected in sequence.
3. The retinal vessel segmentation method based on the generative countermeasure network as claimed in claim 1 or 2, wherein the preprocessing method comprises sequential RGB green channel conversion, contrast-limited histogram equalization and gamma correction.
4. The retinal vessel segmentation method based on the generative countermeasure network as claimed in claim 1 or 2, wherein the preprocessing method further comprises: expansion and rotation.
CN202210805978.XA 2022-07-08 2022-07-08 Retinal vessel segmentation method based on generative confrontation network Pending CN115035127A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210805978.XA CN115035127A (en) 2022-07-08 2022-07-08 Retinal vessel segmentation method based on generative confrontation network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210805978.XA CN115035127A (en) 2022-07-08 2022-07-08 Retinal vessel segmentation method based on generative confrontation network

Publications (1)

Publication Number Publication Date
CN115035127A true CN115035127A (en) 2022-09-09

Family

ID=83129016

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210805978.XA Pending CN115035127A (en) 2022-07-08 2022-07-08 Retinal vessel segmentation method based on generative confrontation network

Country Status (1)

Country Link
CN (1) CN115035127A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116152197A (en) * 2023-02-21 2023-05-23 北京长木谷医疗科技有限公司 Knee joint segmentation method, knee joint segmentation device, electronic equipment and computer readable storage medium
CN116188492A (en) * 2023-02-21 2023-05-30 北京长木谷医疗科技有限公司 Hip joint segmentation method, device, electronic equipment and computer readable storage medium

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116152197A (en) * 2023-02-21 2023-05-23 北京长木谷医疗科技有限公司 Knee joint segmentation method, knee joint segmentation device, electronic equipment and computer readable storage medium
CN116188492A (en) * 2023-02-21 2023-05-30 北京长木谷医疗科技有限公司 Hip joint segmentation method, device, electronic equipment and computer readable storage medium
CN116188492B (en) * 2023-02-21 2024-04-26 北京长木谷医疗科技股份有限公司 Hip joint segmentation method, device, electronic equipment and computer readable storage medium

Similar Documents

Publication Publication Date Title
CN109635862B (en) Sorting method for retinopathy of prematurity plus lesion
CN110930418B (en) Retina blood vessel segmentation method fusing W-net and conditional generation confrontation network
CN112132817A (en) Retina blood vessel segmentation method for fundus image based on mixed attention mechanism
CN115035127A (en) Retinal vessel segmentation method based on generative confrontation network
CN113256641B (en) Skin lesion image segmentation method based on deep learning
CN108764342B (en) Semantic segmentation method for optic discs and optic cups in fundus image
CN112446891A (en) Medical image segmentation method based on U-Net network brain glioma
CN113012163A (en) Retina blood vessel segmentation method, equipment and storage medium based on multi-scale attention network
CN113724206B (en) Fundus image blood vessel segmentation method and system based on self-supervision learning
Li et al. Multi-stage attention-unet for wireless capsule endoscopy image bleeding area segmentation
CN115375711A (en) Image segmentation method of global context attention network based on multi-scale fusion
CN114038564A (en) Noninvasive risk prediction method for diabetes
CN111242949B (en) Fundus image blood vessel segmentation method based on full convolution neural network multi-scale features
CN112869697A (en) Judgment method for simultaneously identifying stage and pathological change characteristics of diabetic retinopathy
CN111462082A (en) Focus picture recognition device, method and equipment and readable storage medium
Miao et al. Classification of diabetic retinopathy based on multiscale hybrid attention mechanism and residual algorithm
CN113421250A (en) Intelligent fundus disease diagnosis method based on lesion-free image training
CN110992309B (en) Fundus image segmentation method based on deep information transfer network
CN117036288A (en) Tumor subtype diagnosis method for full-slice pathological image
Xiao et al. SE-MIDNet Based on Deep Learning for Diabetic Retinopathy Classification
CN116092667A (en) Disease detection method, system, device and storage medium based on multi-mode images
CN115049682A (en) Retina blood vessel segmentation method based on multi-scale dense network
CN114519722A (en) Carotid artery extraction method based on convolutional neural network
CN117078697B (en) Fundus disease seed detection method based on cascade model fusion
Vinta et al. Segmentation and Classification of Interstitial Lung Diseases Based on Hybrid Deep Learning Network Model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination