CN106920227A - Based on the Segmentation Method of Retinal Blood Vessels that deep learning is combined with conventional method - Google Patents
Based on the Segmentation Method of Retinal Blood Vessels that deep learning is combined with conventional method Download PDFInfo
- Publication number
- CN106920227A CN106920227A CN201611228597.0A CN201611228597A CN106920227A CN 106920227 A CN106920227 A CN 106920227A CN 201611228597 A CN201611228597 A CN 201611228597A CN 106920227 A CN106920227 A CN 106920227A
- Authority
- CN
- China
- Prior art keywords
- retinal
- layer
- network
- segmentation
- convolutional layers
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
Landscapes
- Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Eye Examination Apparatus (AREA)
- Image Analysis (AREA)
Abstract
Based on the Segmentation Method of Retinal Blood Vessels that deep learning is combined with conventional method, it is related to computer vision and area of pattern recognition.Using two kinds of gray level images all as the training sample of network, done corresponding data amplification for the few problem of retinal image data includes elastic deformation, smothing filtering etc. to the present invention, expands the broad applicability of the invention.The present invention splits depth network by building the retinal vessel of FCN HNED, the process for realizing autonomous learning of the network high degree, the convolution feature of whole image can not only be shared, reduce feature redundancy, the generic of multiple pixels can be recovered from abstract feature again, the CLAHE figures and Gauss matched filtering figure of retinal vascular images are input into network respectively respectively makes its blood vessel segmentation figure for obtaining be weighted averagely so as to obtain more preferably more complete retinal vessel segmentation probability graph, this kind of robustness and accuracy that improve blood vessel segmentation of processing mode high degree.
Description
Technical field
It is that one kind is mutually tied based on deep learning with conventional method the present invention relates to computer vision and area of pattern recognition
The Segmentation Method of Retinal Blood Vessels of conjunction.
Background technology
Fundus imaging can judge whether exception by retina image-forming, wherein for the observation of retinal vessel
It is quite important.The diseases such as glaucoma, cataract and diabetes can all cause the lesion of retina optical fundus blood vessel.PVR
Patient increases year by year, if can not treat in time, it will usually cause the patient with these diseases for a long time to bear great pain very
To blindness.However, retinopathy is to carry out Artificial Diagnosis by specialist at present, specialist is first to the eye fundus image of patient
The mark of manual blood vessel is carried out, then, then the relevant parameters such as required external caliber, bifurcation angle is measured.Wherein, manual markings
The process of blood vessel probably needs or so two hours, and diagnosis process is taken a significant amount of time, and in order to use manpower and material resources sparingly, automation is carried
The method for taking blood vessel is particularly important.The burden of specialist can not only be mitigated, it is also possible to effectively solve remote districts and lack
The problem of weary specialist.In view of the importance of retinal vessel segmentation, domestic and foreign scholars have done many researchs, have substantially divided non-prison
Superintend and direct and measure of supervision.
Non-supervisory method is that blood vessel target is extracted by certain rule, including matched filtering, and Morphological scale-space, blood vessel is chased after
Track, multiscale analysis scheduling algorithm.Supervised learning is also called pixel characteristic sorting technique or machine learning techniques.Will by training
Each pixel classifications is judged as blood vessel or non-vascular.It is broadly divided into two processes：Feature extraction and classification.Feature extraction phases
Generally include the methods such as Gabor filtering, Gauss matched filtering, morphology enhancing.The grader that sorting phase is generally included has
The graders such as Bayesian (naive Bayesian), SVM.But, this kind can not well consider each picture for the judgement of pixel
Element and contacting between the pixel of field around it.Thus occur in that CNN, it can be judged according to the feature of image block in imago
Element is blood vessel or non-vascular, and by carrying out the automatic learning characteristic of sandwich construction, the feature for making these abstract is conducive to center
The classification of pixel judges.But, each pixel classify seldom is related to global information so that in the feelings for locally having lesion
Under condition, classification failure；Secondly, each image at least hundreds of thousands pixel, if judged one by one so that storage overhead is big, meter
Calculate efficiency very low.
The content of the invention
For the deficiency of existing algorithm, the present invention proposes a kind of view being combined with conventional method based on deep learning
Film blood vessel segmentation method.First, do it according to the characteristics of retinal vessel targetedly to pre-process, including carry out CLAHE (limitations
Property contrast self-adapting histogram equilibrium) treatment enables that retinal vessel and background, with contrast higher, carry out height
The conventional method of this matched filtering causes that the minute blood vessel of retina is strengthened well, and the present invention is proposed two kinds of gray-scale maps
As all as the training sample of network.On this basis, we have done corresponding number for the few problem of retinal image data
Include elastic deformation, smothing filtering etc. according to amplification, not only cause data volume increase be conducive to the study of deep learning network with
Training, it is often more important that simulate with the retinal images in the case of various, can be processed by of the invention
To good retinal vessel segmentation figure, the broad applicability of the invention is expanded.
Secondly, the present invention splits depth network by building the retinal vessel of FCN-HNED, by FCN (Fully
Convolutional Network) network end-point obtains blood vessel probability graph and shallow-layer information HNED (HolisticallyNested
Edge Detection) blood vessel probability graph carried out good fusion, the retinal vessel segmentation figure needed for obtaining us, should
The process for realizing autonomous learning of network high degree, can not only share the convolution feature of whole image, reduce feature superfluous
It is remaining, can recover the generic of multiple pixels from abstract feature again, to realize a kind of end-to-end, pixel to pixel is regarded
The method of retinal vasculature dividing method, this global input and global output is both simple and effective.Work as in retinal vessel detection
The CLAHE figures and Gauss matched filtering figure of retinal vascular images are input into the blood that network obtains it by the middle present invention respectively respectively
Pipe segmentation figure is weighted averagely splits probability graph so as to obtain more preferably more complete retinal vessel, and this kind of processing mode is very big
The robustness and accuracy that improve blood vessel segmentation of degree.
Adopt the following technical scheme that herein：
1st, pre-process
1) green channel higher to contrast in tri- passages of RGB of colored retinal images is extracted.Its
Secondary, due to the problem of shooting angle etc., the brightness of the retinal fundus images for collecting is often uneven, or diseased region
Domain due to it is excessively bright or excessively it is dark show in the picture contrast it is not high the problems such as be difficult to be distinguished with background, so, we are carried out
Normalized.Then, the retinal images after normalization are carried out with CLAHE treatment and improves retinal fundus images quality,
The brightness of weighing apparatus eye fundus image, makes it be more suitable for Subsequent vessel and extracts.
Retinal vessel after CLAHE treatment is capable of high degree while blood vessel is strengthened with background contrasts
The self character of retinal vessel is kept, however, because wherein minute blood vessel is much like with background, working as in follow-up deep learning
In can not split well, be directed to this, the characteristics of the present invention is moved towards using the cross section gray-scale map of blood vessel in Gauss,
Retinal vessel after CLAHE is processed carries out Gauss matched filtering treatment so that minute blood vessel is capable of the table of high degree
Reveal and.Because the direction of blood vessel is arbitrary, therefore, herein using 12 Gaussian kernel templates of different directions come to retina
Image carries out matched filtering, using its peak response as the pixel response.Dimensional Gaussian matched filtering kernel function k (x, y)
It is represented by：
Wherein σ represents the variance of Gaussian curve, and L represents the retinal blood length of tube that y-axis is truncated, the width of filter window
Selection [- 3 σ, 3 σ] is the span of kernel function x, selects less σ numerical value to be set to 0.5 so that minute blood vessel can be very big
Degree is strengthened.
In order to fully take into account the overall permanence and the wherein characteristic of minute blood vessel of retinal images, we are by CLAHE
Retinal vessel figure and Gauss matched filtering figure after treatment can greatly lift network segmentation all as the sample of training
Performance.
2nd, data amplification and structure training sample
Because training depth network needs substantial amounts of data, only existing retinal images are used to train far from enough.In
It is the expansion for needing to carry out training data different modes, increases data volume, improves training and Detection results.Data amplification side
Formula：
1) pretreated image is carried out into inferior translation on left and right and is respectively 20 pixels, realize the translation of e-learning
Consistency.
2) image after 1) processing carries out 45 ° respectively, and 90 °, 125 °, 180 ° of rotation intercepts maximum rectangle therein,
It is this to convert the rotation robustness for not only increasing training data, and be original 5 times by data augmentation.
3) in the middle of general data amplification, the blooming that retinal images are likely to occur never is considered, however, this hair
It is bright in view of in all cases, for example the imprudence movement of the shake of camera or patient, can all make retinal images one
Determine the obscure portions in degree, so, the present invention will 2) process after image set choose wherein 25% carry out 3 × 3 and 5 respectively ×
5 medium filtering fuzzy operation so that network can have broad applicability for the retinal images of various fog-levels.
4) it is conventional in the middle of conventional retinal image data amplification simply to translate, scaling, rotation etc., much up to not
To the consideration of the various situations to retinal images, in consideration of it, the present invention considers the different of the vessel directions shape of retina etc.
Property, we take 25% and enter row stochastic elastic deformation to the image set after 3) treatment, and the item data expands mode for retina
The segmentation of blood vessel has very important significance, and it can help e-learning to the complicated retinal vessel in various directions, have
Split the lifting of accuracy rate beneficial to retinal vessel in practical application.
5) it is applied to the image of any size due to FCN, we carry out 50% and 75% contracting to the image after 4) treatment
Treatment is put, so that amplification data.
Certainly, we carry out same treatment for expert's standard drawing (ground truth) that retinal vessel is split,
So as to be corresponded with sample.Collect as checking using the 3/4 of the good training sample data of component as training set, 1/4.
3rd, FCN-HNED network structions
FCN networks：General FCN Internets are mainly made up of 5 parts, input layer, convolutional layer, down-sampled layer, up-sampling
Layer (warp lamination) and output layer.The network of structure is in the present invention：
Input layer, two convolutional layers (C1, C2), the first down-sampled layer (pool1), two convolutional layers (C3, C4), the second drop
Sample level (pool2), two convolutional layers (C5, C6), the 3rd down-sampled layer (pool3), two convolutional layers (C7, C8), the 4th drop
Sample level (pool4), two convolutional layers (C9, C10), the first up-sampling layer (U1), two convolutional layers (C11, C12), on second
Sample level (U2), two convolutional layers (C13, C14), the 3rd up-sampling layer (U3), two convolutional layers (C15, C16) are adopted on the 4th
Sample layer (U4), two convolutional layers (C17, C18), destination layer (output layer).Form U-shaped depth network structure symmetrical before and after
Frame.
Because the feature resolution of the low layer of FCN networks is higher, and high layer information embodies stronger semantic information, for
The regions such as the parts of lesions of retinal images blood vessel classification have good robustness, but simultaneously FCN networks finally obtain with
The output of input sample formed objects can but lose the detailed information of many less targets and part, thus, the present invention will be shallow
The retinal vessel information of layer is in the method for rim detection HNED (Holisticallynested edge detection) in depth
Learn abundant multi-layer information expression in the case of degree supervision, largely solve object edge fuzzy problem.I.e. we
By C2, a softmax grader is added after C4, C6, C8 layer respectively, so as to by the information of hidden layer by ground
Truth be label in the case of study obtain retinal vessel probability graph, be referred to as side output 1, side output 2, side output 3,
Side output 4.On this basis, we are merged four side outputs with last output layer, so as to form FCN-HNED's
Network structure, complementation is carried out by shallow-layer information and output layer information, obtains multiple dimensioned, at many levels, more close with target sample
Fusion feature figure, for becoming more meticulous for blood vessel of segmentation plays very big effect, so that step of refining that need not be subsequently special is come
Carry out becoming more meticulous for retinal vessel.
Convolutional layer of the invention all obtains an equal amount of characteristic pattern by way of zero padding, and pooling layers of result is
So that feature is reduced, parameter is reduced, but pooling layers of purpose and is not only in that this.The present invention can be subtracted using max-pooling
The skew of the estimation average that small convolutional layer parameter error is caused, more retains texture information.Of the invention maximum pond layer is adopted
Sample rate is 2.Up-sampling is the process of bilinear interpolation.
All with ReLU except Softmax classification layers, loss function is intersection to activation primitive in the building process of whole model
Entropy.
Training：The training of network can be carried out after FCN-HNED network structions well carry the automated characterization of image
Take and learning process, 128 images of per generation input stop after network convergence.
Test：The CLAHE figures and Gauss matched filtering figure of every retinal images green channel figure are separately input to
The network for training is tested, and the retinal vessel segmentation figure for respectively obtaining fusion is referred to asWithIt is rightWith
It is weighted and averagely obtains last retinal vessel segmentation probability graph.4th, post-process
Binaryzation is carried out to obtaining retinal vessel probability graph in test and obtains segmentation figure.
Beneficial effect
1st, the different qualities according to retinal vessel of the invention, using targetedly data processing method, training data
Quality directly determines whether the model that training is obtained is reliable, and whether accuracy rate reaches required level, and the present invention is using fuzzy
Operation, elastic deformation etc., simulate various retina data being likely to occur well, while expand data reaching enough
Many quantity is avoiding training over-fitting, it is also possible to for follow-up detection provides help, and then improve retinal vessel segmentation standard
True rate.
2nd, the image after the treatment of retinal images and Gauss matched filtering after the present invention processes CLAHE is input into respectively
Network is trained study, the abundant study for the property of retinal vessel is obtained under each performance level, Er Qiegao
This matched filtering figure fully compensate for CLAHE treatment figure for the unsharp deficiency of minute blood vessel, greatly improve retina
The performance of blood vessel segmentation.
3rd, method of the present invention by building deep learning network FCN-HNED, can be rapidly performed by retinal images
Automatic Feature Extraction, it can carry out feature extraction, study to retinal images to retinal fundus images from different levels
In each pixel and the relation around it between multiple neighborhoods, by its retinal vessel figure, the good table of advanced features
Reveal and, so that it has distinguished the internal feature of blood vessel and non-vascular well, realize end-to-end, the blood of pixel to pixel
Pipe is split, than many times of the classification judging efficiency lifting of traditional single pixel.
4th, the present invention is exported with the end of FCN networks using four sides output of shallow-layer feature and carries out height and merge, so that
Realize becoming more meticulous and robustness for blood vessel segmentation.So that blood vessel segmentation figure and the manual segmentation figure of expert reach it is consistent well
Property.Meanwhile, the automation for realizing retinal vessel segmentation of high degree greatly reduces drain on manpower and material resources.
Brief description of the drawings
Fig. 1 is overall flow figure of the invention；
Fig. 2 is vessel cross-sections intensity profile figure；(a) one section of vessel graph (b) gray level
Fig. 3 is pretreating effect figure；After image (c) Gauss matched filtering after (a) original image (b) CLAHE treatment
Image
Fig. 4 is FCN-HNED network structures；
Fig. 5 is retinal vessel segmentation result.(a) original image (b) retinal vessel segmentation figure (c) first expert's hand
Dynamic segmentation figure
Specific embodiment
It is specifically described below in conjunction with the accompanying drawings：
Technology frame chart of the invention is as shown in Figure 1.Specific implementation step is as follows respectively：
1st, pre-process
Same pretreatment is all carried out to each width retinal fundus images either training set or test set.
1) green channel higher to contrast in tri- passages of RGB of colored retinal images is extracted.Its
Secondary, due to the problem of shooting angle etc., the brightness of the retinal fundus images for collecting is often uneven, or diseased region
Domain due to it is excessively bright or excessively it is dark show in the picture contrast it is not high the problems such as be difficult to be distinguished with background, so, we are carried out
Then retinal images after normalization, are carried out CLAHE treatment and improve retinal fundus images quality, by normalized
The brightness of weighing apparatus eye fundus image, makes it be more suitable for Subsequent vessel and extracts.
Retinal vessel after CLAHE treatment is capable of high degree while blood vessel is strengthened with background contrasts
The self character of retinal vessel is kept, however, because wherein minute blood vessel is much like with background, working as in follow-up deep learning
In can not split well, be directed to this, the characteristics of the present invention is moved towards using the cross section gray-scale map of blood vessel in Gauss,
Retinal images are carried out to do dead matched filtering treatment.As shown in Fig. 2 (a) is blood vessel gray-scale map, (b) is the cross section of blood vessel
Gray value, the cross section of tiny blood vessel is also presented Gauss trend, so, the present invention CLAHE is processed after retina
Blood vessel carries out Gauss matched filtering treatment.Because the direction of blood vessel is arbitrary, therefore, herein using 12 height of different directions
This core template carries out matched filtering to retinal images, finds response of the corresponding peak response as the pixel.
In order to fully take into account the overall permanence and the wherein characteristic of minute blood vessel of retinal images, we are by CLAHE
Retinal vessel figure and Gauss matched filtering figure after treatment can greatly lift network segmentation all as the sample of training
Performance.
2nd, data amplification and structure training sample
Because training depth network needs substantial amounts of data, only existing retinal images are used to train far from enough.In
It is the data augmentation for needing to carry out training data different modes, increases data volume, improves training and Detection results.Data are expanded
Mode：
1) pretreated image is carried out into inferior translation on left and right and is respectively 20 pixels, realize the translation of e-learning
Consistency.
2) image after 1) processing carries out 45 ° respectively, and 90 °, 125 °, 180 ° of rotation intercepts maximum rectangle therein,
It is this to convert the rotation robustness for not only increasing training data, and be original 5 times by data augmentation.
3) in the middle of general data amplification, medium filtering is not always used, however, the present invention is considered in various situations
Under, for example the imprudence movement of the shake of camera or patient, can all make retinal images obscure portions to a certain extent
Situation, so, the image after 2) present invention will be processed takes wherein 25% image and carries out 3 × 3 and 5 × 5 medium filtering mould respectively
Paste operation so that network can have broad applicability for the retinal images of various fog-levels.
4) it is conventional in the middle of conventional retinal image data amplification simply to translate, scaling, rotation etc., much up to not
To the consideration of the various situations to retinal images, in consideration of it, the present invention considers the different of the vessel directions shape of retina etc.
Property, we take 25% and enter row stochastic elastic deformation to the image after 3) treatment, and the item data expands mode for retinal blood
The segmentation of pipe has very important significance, and it can help e-learning to the complicated retinal vessel in various directions, favorably
Retinal vessel splits the lifting of accuracy rate in practical application.
5) it is applied to the image of any size due to FCN, we carry out 50% and 75% contracting to the image after 4) treatment
Treatment is put, so that amplification data.
Certainly, we carry out same treatment for expert's standard drawing (ground truth) that retinal vessel is split,
So as to be corresponded with sample.Collect as checking using the 3/4 of the good training sample data of component as training set, 1/4.
3rd, FCN-HNED network structions and training and test process
FCN networks：General FCN Internets are mainly made up of 5 parts, input layer, convolutional layer, down-sampled layer, up-sampling
Layer (warp lamination) and output layer.The network of structure is in the present invention：Input layer, two convolutional layers (C1, C2), first is down-sampled
Layer (pool1), two convolutional layers (C3, C4), the second down-sampled layer (pool2), two convolutional layers (C5, C6), the 3rd is down-sampled
Layer (pool3), two convolutional layers (C7, C8), the 4th down-sampled layer (pool4), two convolutional layers (C9, C10), the first up-sampling
Layer (U1), two convolutional layers (C11, C12), the second up-sampling layer (U2), two convolutional layers (C13, C14), the 3rd up-sampling layer
(U3), two convolutional layers (C15, C16), the 4th up-sampling layer (U4), two convolutional layers (C17, C18), destination layer (output layer).
Form U-shaped depth network architecture symmetrical before and after.
Wherein convolution process is realized as follows：
f(X；W, b)=W*sX+b (2)
Wherein, f (X；W, b) to be output as characteristic pattern, X is the input feature vector figure of preceding layer, and W and b is convolution kernel and skew
Value, *sConvolution operation is represented, unlike traditional CNN networks, last full articulamentum is all changed and does convolutional layer by FCN networks, but
It is to cause that characteristic pattern is less and less by sequence of operations such as convolution and down-samplings, returns to image same with input picture
Sample size, FCN is using up-sampling operation deconvolution in other words conj.or perhaps.
Middle convolutional layer of the invention all obtains an equal amount of characteristic pattern by way of zero padding, symmetrical U-shaped
3 × 3 filtering convolution kernels that all repeated application two is tightly connected in network carry out convolution operation, and step-length is 1, each convolutional layer back
There is a ReLU activation primitive, pooling layers of result is so that feature is reduced, and parameter is reduced, but pooling layers of purpose
And this is not only in that, it can keep certain consistency to rotate, translate etc., this structure is 2 × 2 with core, and step-length is 2 max-
Pooling layers, the skew of the estimation average that convolutional layer parameter error is caused can be reduced, more retain texture information.At each
During down-sampling, the number of characteristic pattern all can be double, up-samples then opposite.In addition, in last layer with 1 × 1
By target mapping of standard output be trained 64 characteristic patterns by convolution kernel.
All with ReLU except Softmax classification layers, loss function is intersection to activation primitive in the building process of whole model
Entropy.
HNED structures：Blood vessel segmentation is regarded as rim detection problem by we, and we use the network supervised based on depth
Obtain four blood vessel probability graphs of shallow-layer FCN networks.I.e. we add a softmax points by C2, C4, C6, after C8 respectively
Class device, supervises network so that by the information of hidden layer with retinal vessel probability by the depth with Standard Segmentation result as target
The form of figure shows, and is referred to as side output 1, side output 2, side output 3, side and exports 4, realizes multiple dimensioned Feature Mapping
The study of figure.
Because the low-level feature resolution ratio of FCN networks is higher, and high layer information embodies stronger semantic information, for regarding
The blood vessel classification in the regions such as the parts of lesions of nethike embrane image has good robustness, but finally obtains identical with input sample big
Small output can but lose the detailed information of many less targets and part, thus, the retinal vessel by shallow-layer of the invention
Information learns abundant multi-layer information expression in the method for rim detection HNED in the case of depth supervision, largely solves
Object edge of having determined fuzzy problem.On this basis, we are merged four side outputs with last output layer, so that shape
Into the network structure of FCN-HNED, if as shown in figure 4, input picture size be 512 × 512, by C1, C2 all be 64 3
× 3 wave filter obtains 64 characteristic patterns, and C1 is caused by way of to original image zero padding, and it is 512 that C2 characteristic patterns keep size
× 512, by down-sampled so that characteristic pattern is double, when reaching lowermost end C9 and C10,1024 characteristic pattern sizes are 32 × 32,
Convolution realization afterwards is similar with front, and the implementation of up-sampling is bilinear interpolation.The network structure is by shallow-layer information
Four sides probability graph that runs off vascular carries out Mutually fusion with the output layer blood vessel probability graph of FCN networks, is obtained more preferably by training
The characteristic pattern more close with target sample, for becoming more meticulous for blood vessel of segmentation plays very big effect, so that need not be follow-up
Special refines step to carry out becoming more meticulous for retinal vessel.
Fusion process：In order to directly using the output probability figure after side output probability figure and FCN up-samplings, we are to it
Merged：Wherein, σ () represents sigmoid functions,Represent that m-th side is defeated
Go out, hmIt is respectively the blending weight of four side outputs and FCN finally outputs with h, original fusion weights are all set to 1/5.Added
Weighing the loss function for merging is：
Wherein, Y represents that standard blood vessel segmentation figure i.e. ground truth, Dist () represent the probability after fusion
Figure the distance between with standard blood vessel segmentation figure, i.e. difference degree, by way of study adjusting weights moves closer to convergence, I
Minimize its loss function by SDG (gradient descent method).
Training：The training of network can be carried out after FCN-HNED network structions well carry the automated characterization of image
Take and learning process, be carried out in two steps：The first step, manually chooses some and compares intuitively 1280, picture, first to building herein
Model be trained, per generation input 128 images, when model convergence after, model parameter is preserved because this
1280 pictures contents are relatively directly perceived simple, and than more visible, the convergence rate of model is than very fast for the semantic information of blood vessel non-vascular；
Second step, is trained again on complete or collected works' training set to model, but the initial value of model parameter in mono- Walk using obtaining
Parameter, so greatly reduce the training time of model so that the convergence rate of block mold is accelerated.
Training：After each image training data is successively calculated by convolutional neural networks algorithm, output one is obtained
Blood vessel probability graph after individual fusion, calculates the error of the probability graph and each pixel generic in corresponding standard drawing.According to
Minimum error principle, by each layer parameter in the depth convolutional neural networks that error calculation carries out constructed by successively feedback modifiers.
When error is gradually reduced to tend towards stability, it is believed that network has been restrained, training terminates, detection model needed for generation.
Test：The CLAHE figures and Gauss matched filtering figure of every retinal fundus images green channel figure are input into respectively
Tested to the network for having trained, the retinal vessel segmentation figure for respectively obtaining fusion is referred to asWithIt is right
WithIt is weighted averagely so as to obtain more vessel informations, has also obtained last retinal vessel segmentation probability graph.
4 post processings
Binaryzation is carried out to the comprehensive retinal vessel probability graph for obtaining and obtains segmentation figure, showed consistent with expert's segmentation
Binary map.Parameter evaluation is carried out by segmentation result, more than 96% accuracy rate is obtained, as shown in Figure 5.
Claims (1)
1. the Segmentation Method of Retinal Blood Vessels being combined with conventional method based on deep learning, it is characterised in that including following step
Suddenly：
(1), pre-process
Tri- passage Green passages of RGB to colored retinal images are extracted, and are normalized, to normalization
Retinal images afterwards carry out CLAHE treatment, and the retinal vessel after CLAHE is processed carries out Gauss matched filtering treatment,
Retinal vessel figure and Gauss matched filtering figure after CLAHE is processed are all as the sample of training；
(2), data amplification and structure training sample
Data expand mode：
1) pretreated image is carried out into inferior translation on left and right and is respectively 20 pixels, realize the translation invariant of e-learning
Property；
2) image after 1) processing carries out 45 ° respectively, and 90 °, 125 °, 180 ° of rotation intercepts maximum rectangle therein；
3) image set after 2) processing chooses wherein 25% and carries out 3 × 3 and 5 × 5 medium filtering fuzzy operation respectively；
4) image set after 3) processing takes 25% and enters row stochastic elastic deformation；
5) image after 4) processing carries out 50% and 75% scaling treatment, so that amplification data；
Carry out same treatment for the expert standard drawing ground truth that retinal vessel is split, so as to a pair of sample 1
Should；
(3), FCN-HNED network structions
The network of structure is：
Input layer, two convolutional layers (C1, C2), the first down-sampled layer (pool1), two convolutional layers (C3, C4), second is down-sampled
Layer (pool2), two convolutional layers (C5, C6), the 3rd down-sampled layer (pool3), two convolutional layers (C7, C8), the 4th is down-sampled
Layer (pool4), two convolutional layers (C9, C10), the first up-sampling layer (U1), two convolutional layers (C11, C12), the second up-sampling
Layer (U2), two convolutional layers (C13, C14), the 3rd up-sampling layer (U3), two convolutional layers (C15, C16), the 4th up-sampling layer
(U4), two convolutional layers (C17, C18), destination layer (output layer)；Form U-shaped depth network architecture symmetrical before and after；
By C2, a softmax grader is added after C4, C6, C8 layer respectively, so as to by the information of hidden layer by ground
Truth be label in the case of study obtain retinal vessel probability graph, be referred to as side output 1, side output 2, side output 3,
Side output 4；Four side outputs are merged with last output layer, so as to form the network structure of FCN-HNED；
Training：The training of network can be carried out after FCN-HNED network structions well to carry out to the Automatic Feature Extraction of image and
Learning process, 128 images of per generation input, stops after network convergence；
Test：The CLAHE figures and Gauss matched filtering figure of every retinal images green channel figure are separately input to train
Good network is tested, and the retinal vessel segmentation figure for respectively obtaining fusion is referred to asWithIt is rightWithCarry out
Weighted average obtains last retinal vessel segmentation probability graph；
(4) post-process
Binaryzation is carried out to obtaining retinal vessel probability graph in test and obtains segmentation figure.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611228597.0A CN106920227B (en) | 2016-12-27 | 2016-12-27 | The Segmentation Method of Retinal Blood Vessels combined based on deep learning with conventional method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611228597.0A CN106920227B (en) | 2016-12-27 | 2016-12-27 | The Segmentation Method of Retinal Blood Vessels combined based on deep learning with conventional method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106920227A true CN106920227A (en) | 2017-07-04 |
| CN106920227B CN106920227B (en) | 2019-06-07 |
Family
ID=59453388
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611228597.0A Active CN106920227B (en) | 2016-12-27 | 2016-12-27 | The Segmentation Method of Retinal Blood Vessels combined based on deep learning with conventional method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106920227B (en) |
Cited By (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108122236A (en) * | 2017-12-18 | 2018-06-05 | 上海交通大学 | Iterative eye fundus image blood vessel segmentation method based on distance modulated loss |
| CN108230322A (en) * | 2018-01-28 | 2018-06-29 | 浙江大学 | A kind of eyeground feature detection device based on weak sample labeling |
| CN108492302A (en) * | 2018-03-26 | 2018-09-04 | 北京市商汤科技开发有限公司 | Nervous layer dividing method and device, electronic equipment, storage medium, program |
| CN108510473A (en) * | 2018-03-09 | 2018-09-07 | 天津工业大学 | The FCN retinal images blood vessel segmentations of convolution and channel weighting are separated in conjunction with depth |
| CN108665461A (en) * | 2018-05-09 | 2018-10-16 | 电子科技大学 | A kind of breast ultrasound image partition method corrected based on FCN and iteration sound shadow |
| CN108765422A (en) * | 2018-06-13 | 2018-11-06 | 云南大学 | A kind of retinal images blood vessel automatic division method |
| CN108830155A (en) * | 2018-05-10 | 2018-11-16 | 北京红云智胜科技有限公司 | A kind of heart coronary artery segmentation and knowledge method for distinguishing based on deep learning |
| CN108986124A (en) * | 2018-06-20 | 2018-12-11 | 天津大学 | In conjunction with Analysis On Multi-scale Features convolutional neural networks retinal vascular images dividing method |
| CN109087302A (en) * | 2018-08-06 | 2018-12-25 | 北京大恒普信医疗技术有限公司 | A kind of eye fundus image blood vessel segmentation method and apparatus |
| CN109118495A (en) * | 2018-08-01 | 2019-01-01 | 沈阳东软医疗系统有限公司 | A kind of Segmentation Method of Retinal Blood Vessels and device |
| CN109191476A (en) * | 2018-09-10 | 2019-01-11 | 重庆邮电大学 | The automatic segmentation of Biomedical Image based on U-net network structure |
| CN109285157A (en) * | 2018-07-24 | 2019-01-29 | 深圳先进技术研究院 | Myocardium of left ventricle dividing method, device and computer readable storage medium |
| CN109426773A (en) * | 2017-08-24 | 2019-03-05 | 浙江宇视科技有限公司 | A kind of roads recognition method and device |
| CN109448006A (en) * | 2018-11-01 | 2019-03-08 | 江西理工大学 | A kind of U-shaped intensive connection Segmentation Method of Retinal Blood Vessels of attention mechanism |
| CN109523522A (en) * | 2018-10-30 | 2019-03-26 | 腾讯科技（深圳）有限公司 | Processing method, device, system and the storage medium of endoscopic images |
| CN109523569A (en) * | 2018-10-18 | 2019-03-26 | 中国科学院空间应用工程与技术中心 | A kind of remote sensing image dividing method and device based on more granularity network integrations |
| CN109528155A (en) * | 2018-11-19 | 2019-03-29 | 复旦大学附属眼耳鼻喉科医院 | A kind of intelligent screening system and its method for building up suitable for the concurrent open-angle glaucoma of high myopia |
| CN109886982A (en) * | 2019-04-24 | 2019-06-14 | 数坤（北京）网络科技有限公司 | A kind of blood-vessel image dividing method, device and computer memory device |
| CN110120047A (en) * | 2019-04-04 | 2019-08-13 | 平安科技（深圳）有限公司 | Image Segmentation Model training method, image partition method, device, equipment and medium |
| CN110222726A (en) * | 2019-05-15 | 2019-09-10 | 北京字节跳动网络技术有限公司 | Image processing method, device and electronic equipment |
| CN110276763A (en) * | 2018-03-15 | 2019-09-24 | 中南大学 | It is a kind of that drawing generating method is divided based on the retinal vessel of confidence level and deep learning |
| CN110309849A (en) * | 2019-05-10 | 2019-10-08 | 腾讯医疗健康(深圳)有限公司 | Blood-vessel image processing method, device, equipment and storage medium |
| CN110415231A (en) * | 2019-07-25 | 2019-11-05 | 山东浪潮人工智能研究院有限公司 | A kind of CNV dividing method based on attention pro-active network |
| WO2019228195A1 (en) * | 2018-05-28 | 2019-12-05 | 中兴通讯股份有限公司 | Method and apparatus for perceiving spatial environment |
| CN110796643A (en) * | 2019-10-18 | 2020-02-14 | 四川大学 | Rail fastener defect detection method and system |
| CN111091132A (en) * | 2020-03-19 | 2020-05-01 | 腾讯科技（深圳）有限公司 | Image recognition method and device based on artificial intelligence, computer equipment and medium |
| CN111541911A (en) * | 2020-04-21 | 2020-08-14 | 腾讯科技（深圳）有限公司 | Video detection method and device, storage medium and electronic device |
| CN112132817A (en) * | 2020-09-29 | 2020-12-25 | 汕头大学 | Retina blood vessel segmentation method for fundus image based on mixed attention mechanism |
| CN112465842A (en) * | 2020-12-22 | 2021-03-09 | 杭州电子科技大学 | Multi-channel retinal vessel image segmentation method based on U-net network |
| CN112950638A (en) * | 2019-12-10 | 2021-06-11 | 深圳华大生命科学研究院 | Image segmentation method and device, electronic equipment and computer readable storage medium |
| US11080850B2 (en) * | 2018-01-16 | 2021-08-03 | Electronics And Telecommunications Research Institute | Glaucoma diagnosis method using fundus image and apparatus for the same |
| EP3745347A4 (en) * | 2018-01-26 | 2021-12-15 | BOE Technology Group Co., Ltd. | Image processing method, processing apparatus and processing device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120178099A1 (en) * | 2011-01-10 | 2012-07-12 | Indian Association For The Cultivation Of Science | Highly fluorescent carbon nanoparticles and methods of preparing the same |
| CN105825509A (en) * | 2016-03-17 | 2016-08-03 | 电子科技大学 | Cerebral vessel segmentation method based on 3D convolutional neural network |
| CN106096654A (en) * | 2016-06-13 | 2016-11-09 | 南京信息工程大学 | A kind of cell atypia automatic grading method tactful based on degree of depth study and combination |
| CN106203327A (en) * | 2016-07-08 | 2016-12-07 | 清华大学 | Lung tumor identification system and method based on convolutional neural networks |
-
2016
- 2016-12-27 CN CN201611228597.0A patent/CN106920227B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120178099A1 (en) * | 2011-01-10 | 2012-07-12 | Indian Association For The Cultivation Of Science | Highly fluorescent carbon nanoparticles and methods of preparing the same |
| CN105825509A (en) * | 2016-03-17 | 2016-08-03 | 电子科技大学 | Cerebral vessel segmentation method based on 3D convolutional neural network |
| CN106096654A (en) * | 2016-06-13 | 2016-11-09 | 南京信息工程大学 | A kind of cell atypia automatic grading method tactful based on degree of depth study and combination |
| CN106203327A (en) * | 2016-07-08 | 2016-12-07 | 清华大学 | Lung tumor identification system and method based on convolutional neural networks |
Non-Patent Citations (1)
| Title |
|---|
| SAINING XIE ET AL.: "Holistically-Nested Edge Detection", 《2015 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION》 * |
Cited By (47)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109426773A (en) * | 2017-08-24 | 2019-03-05 | 浙江宇视科技有限公司 | A kind of roads recognition method and device |
| CN108122236B (en) * | 2017-12-18 | 2020-07-31 | 上海交通大学 | Iterative fundus image blood vessel segmentation method based on distance modulation loss |
| CN108122236A (en) * | 2017-12-18 | 2018-06-05 | 上海交通大学 | Iterative eye fundus image blood vessel segmentation method based on distance modulated loss |
| US11080850B2 (en) * | 2018-01-16 | 2021-08-03 | Electronics And Telecommunications Research Institute | Glaucoma diagnosis method using fundus image and apparatus for the same |
| EP3745347A4 (en) * | 2018-01-26 | 2021-12-15 | BOE Technology Group Co., Ltd. | Image processing method, processing apparatus and processing device |
| CN108230322B (en) * | 2018-01-28 | 2021-11-09 | 浙江大学 | Eye ground characteristic detection device based on weak sample mark |
| CN108230322A (en) * | 2018-01-28 | 2018-06-29 | 浙江大学 | A kind of eyeground feature detection device based on weak sample labeling |
| CN108510473A (en) * | 2018-03-09 | 2018-09-07 | 天津工业大学 | The FCN retinal images blood vessel segmentations of convolution and channel weighting are separated in conjunction with depth |
| CN110276763A (en) * | 2018-03-15 | 2019-09-24 | 中南大学 | It is a kind of that drawing generating method is divided based on the retinal vessel of confidence level and deep learning |
| CN110276763B (en) * | 2018-03-15 | 2021-05-11 | 中南大学 | Retina blood vessel segmentation map generation method based on credibility and deep learning |
| CN108492302A (en) * | 2018-03-26 | 2018-09-04 | 北京市商汤科技开发有限公司 | Nervous layer dividing method and device, electronic equipment, storage medium, program |
| CN108492302B (en) * | 2018-03-26 | 2021-04-02 | 北京市商汤科技开发有限公司 | Neural layer segmentation method and device, electronic device and storage medium |
| CN108665461A (en) * | 2018-05-09 | 2018-10-16 | 电子科技大学 | A kind of breast ultrasound image partition method corrected based on FCN and iteration sound shadow |
| CN108830155A (en) * | 2018-05-10 | 2018-11-16 | 北京红云智胜科技有限公司 | A kind of heart coronary artery segmentation and knowledge method for distinguishing based on deep learning |
| WO2019228195A1 (en) * | 2018-05-28 | 2019-12-05 | 中兴通讯股份有限公司 | Method and apparatus for perceiving spatial environment |
| CN108765422A (en) * | 2018-06-13 | 2018-11-06 | 云南大学 | A kind of retinal images blood vessel automatic division method |
| CN108986124A (en) * | 2018-06-20 | 2018-12-11 | 天津大学 | In conjunction with Analysis On Multi-scale Features convolutional neural networks retinal vascular images dividing method |
| CN109285157A (en) * | 2018-07-24 | 2019-01-29 | 深圳先进技术研究院 | Myocardium of left ventricle dividing method, device and computer readable storage medium |
| WO2020019740A1 (en) * | 2018-07-24 | 2020-01-30 | 深圳先进技术研究院 | Left ventricle myocardium segmentation method and device, and computer readable storage medium |
| CN109118495A (en) * | 2018-08-01 | 2019-01-01 | 沈阳东软医疗系统有限公司 | A kind of Segmentation Method of Retinal Blood Vessels and device |
| CN109118495B (en) * | 2018-08-01 | 2020-06-23 | 东软医疗系统股份有限公司 | Retinal vessel segmentation method and device |
| CN109087302A (en) * | 2018-08-06 | 2018-12-25 | 北京大恒普信医疗技术有限公司 | A kind of eye fundus image blood vessel segmentation method and apparatus |
| CN109191476A (en) * | 2018-09-10 | 2019-01-11 | 重庆邮电大学 | The automatic segmentation of Biomedical Image based on U-net network structure |
| CN109191476B (en) * | 2018-09-10 | 2022-03-11 | 重庆邮电大学 | Novel biomedical image automatic segmentation method based on U-net network structure |
| CN109523569A (en) * | 2018-10-18 | 2019-03-26 | 中国科学院空间应用工程与技术中心 | A kind of remote sensing image dividing method and device based on more granularity network integrations |
| CN109523522A (en) * | 2018-10-30 | 2019-03-26 | 腾讯科技（深圳）有限公司 | Processing method, device, system and the storage medium of endoscopic images |
| CN109523522B (en) * | 2018-10-30 | 2023-05-09 | 腾讯医疗健康(深圳)有限公司 | Endoscopic image processing method, device, system and storage medium |
| CN109448006A (en) * | 2018-11-01 | 2019-03-08 | 江西理工大学 | A kind of U-shaped intensive connection Segmentation Method of Retinal Blood Vessels of attention mechanism |
| CN109448006B (en) * | 2018-11-01 | 2022-01-28 | 江西理工大学 | Attention-based U-shaped dense connection retinal vessel segmentation method |
| CN109528155A (en) * | 2018-11-19 | 2019-03-29 | 复旦大学附属眼耳鼻喉科医院 | A kind of intelligent screening system and its method for building up suitable for the concurrent open-angle glaucoma of high myopia |
| WO2020199593A1 (en) * | 2019-04-04 | 2020-10-08 | 平安科技（深圳）有限公司 | Image segmentation model training method and apparatus, image segmentation method and apparatus, and device and medium |
| CN110120047B (en) * | 2019-04-04 | 2023-08-08 | 平安科技（深圳）有限公司 | Image segmentation model training method, image segmentation method, device, equipment and medium |
| CN110120047A (en) * | 2019-04-04 | 2019-08-13 | 平安科技（深圳）有限公司 | Image Segmentation Model training method, image partition method, device, equipment and medium |
| CN109886982A (en) * | 2019-04-24 | 2019-06-14 | 数坤（北京）网络科技有限公司 | A kind of blood-vessel image dividing method, device and computer memory device |
| CN110309849A (en) * | 2019-05-10 | 2019-10-08 | 腾讯医疗健康(深圳)有限公司 | Blood-vessel image processing method, device, equipment and storage medium |
| CN110222726A (en) * | 2019-05-15 | 2019-09-10 | 北京字节跳动网络技术有限公司 | Image processing method, device and electronic equipment |
| CN110415231A (en) * | 2019-07-25 | 2019-11-05 | 山东浪潮人工智能研究院有限公司 | A kind of CNV dividing method based on attention pro-active network |
| CN110796643A (en) * | 2019-10-18 | 2020-02-14 | 四川大学 | Rail fastener defect detection method and system |
| CN112950638A (en) * | 2019-12-10 | 2021-06-11 | 深圳华大生命科学研究院 | Image segmentation method and device, electronic equipment and computer readable storage medium |
| CN112950638B (en) * | 2019-12-10 | 2023-12-29 | 深圳华大生命科学研究院 | Image segmentation method, device, electronic equipment and computer readable storage medium |
| CN111091132A (en) * | 2020-03-19 | 2020-05-01 | 腾讯科技（深圳）有限公司 | Image recognition method and device based on artificial intelligence, computer equipment and medium |
| CN111541911A (en) * | 2020-04-21 | 2020-08-14 | 腾讯科技（深圳）有限公司 | Video detection method and device, storage medium and electronic device |
| CN111541911B (en) * | 2020-04-21 | 2024-05-14 | 深圳市雅阅科技有限公司 | Video detection method and device, storage medium and electronic device |
| CN112132817A (en) * | 2020-09-29 | 2020-12-25 | 汕头大学 | Retina blood vessel segmentation method for fundus image based on mixed attention mechanism |
| CN112132817B (en) * | 2020-09-29 | 2022-12-06 | 汕头大学 | Retina blood vessel segmentation method for fundus image based on mixed attention mechanism |
| CN112465842A (en) * | 2020-12-22 | 2021-03-09 | 杭州电子科技大学 | Multi-channel retinal vessel image segmentation method based on U-net network |
| CN112465842B (en) * | 2020-12-22 | 2024-02-06 | 杭州电子科技大学 | Multichannel retinal blood vessel image segmentation method based on U-net network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106920227B (en) | 2019-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106920227B (en) | The Segmentation Method of Retinal Blood Vessels combined based on deep learning with conventional method | |
| CN111815574B (en) | Fundus retina blood vessel image segmentation method based on rough set neural network | |
| CN107437092B (en) | The classification method of retina OCT image based on Three dimensional convolution neural network | |
| Kumar et al. | An automated early diabetic retinopathy detection through improved blood vessel and optic disc segmentation | |
| CN106408564B (en) | A kind of method for processing fundus images based on deep learning, apparatus and system | |
| CN110930416B (en) | MRI image prostate segmentation method based on U-shaped network | |
| CN109615632B (en) | Fundus image optic disc and optic cup segmentation method based on semi-supervision condition generation type countermeasure network | |
| CN109191476A (en) | The automatic segmentation of Biomedical Image based on U-net network structure | |
| CN108095683A (en) | The method and apparatus of processing eye fundus image based on deep learning | |
| CN110276356A (en) | Eye fundus image aneurysms recognition methods based on R-CNN | |
| CN112508864B (en) | Retinal vessel image segmentation method based on improved UNet + | |
| CN108764342B (en) | Semantic segmentation method for optic discs and optic cups in fundus image | |
| JP2019192215A (en) | 3d quantitative analysis of retinal layers with deep learning | |
| CN111612856A (en) | Retina neovascularization detection method and imaging method for color fundus image | |
| Manning et al. | Image analysis and machine learning-based malaria assessment system | |
| CN114581434A (en) | Pathological image processing method based on deep learning segmentation model and electronic equipment | |
| CN109087310A (en) | Dividing method, system, storage medium and the intelligent terminal of Meibomian gland texture region | |
| Kumar et al. | Image processing in diabetic related causes | |
| CN114648806A (en) | Multi-mechanism self-adaptive fundus image segmentation method | |
| CN113012093B (en) | Training method and training system for glaucoma image feature extraction | |
| Argade et al. | Automatic detection of diabetic retinopathy using image processing and data mining techniques | |
| CN117611824A (en) | Digital retina image segmentation method based on improved UNET | |
| Maya et al. | Detection of retinal lesions based on deep learning for diabetic retinopathy | |
| Pradhan et al. | Diabetic retinopathy detection on retinal fundus images using convolutional neural network | |
| Sri et al. | Diabetic retinopathy classification using deep learning technique |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |