CN109064453B - Neural network model for fundus image blood vessel segmentation - Google Patents
Neural network model for fundus image blood vessel segmentation Download PDFInfo
- Publication number
- CN109064453B CN109064453B CN201810764191.7A CN201810764191A CN109064453B CN 109064453 B CN109064453 B CN 109064453B CN 201810764191 A CN201810764191 A CN 201810764191A CN 109064453 B CN109064453 B CN 109064453B
- Authority
- CN
- China
- Prior art keywords
- layer
- blood vessel
- image
- convolutional
- characteristic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 210000004204 blood vessel Anatomy 0.000 title claims abstract description 233
- 238000003062 neural network model Methods 0.000 title claims abstract description 33
- 230000011218 segmentation Effects 0.000 title abstract description 27
- 238000005457 optimization Methods 0.000 claims abstract description 65
- 238000000605 extraction Methods 0.000 claims abstract description 56
- 238000005070 sampling Methods 0.000 claims abstract description 49
- 238000012545 processing Methods 0.000 claims abstract description 30
- 238000003475 lamination Methods 0.000 claims description 18
- 239000000284 extract Substances 0.000 claims description 10
- 230000004927 fusion Effects 0.000 claims description 9
- 239000008280 blood Substances 0.000 claims description 6
- 210000004369 blood Anatomy 0.000 claims description 6
- 238000013527 convolutional neural network Methods 0.000 claims description 6
- 238000000034 method Methods 0.000 description 28
- 230000008569 process Effects 0.000 description 9
- 230000004256 retinal image Effects 0.000 description 9
- 238000012549 training Methods 0.000 description 8
- 230000002792 vascular Effects 0.000 description 7
- 206010012689 Diabetic retinopathy Diseases 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 210000005036 nerve Anatomy 0.000 description 2
- 210000001210 retinal vessel Anatomy 0.000 description 2
- 201000004569 Blindness Diseases 0.000 description 1
- 208000010412 Glaucoma Diseases 0.000 description 1
- 206010020772 Hypertension Diseases 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000002526 effect on cardiovascular system Effects 0.000 description 1
- 208000002780 macular degeneration Diseases 0.000 description 1
- 210000004218 nerve net Anatomy 0.000 description 1
- 230000002207 retinal effect Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30101—Blood vessel; Artery; Vein; Vascular
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Eye Examination Apparatus (AREA)
Abstract
The invention relates to a neural network model for fundus image blood vessel segmentation.A blood vessel characteristic processing layer at the highest layer is connected with a blood vessel characteristic optimizing layer at the lowest layer through backward short connection; each blood vessel characteristic optimization layer is connected with a blood vessel characteristic optimization layer higher by one layer through a forward short connection; each blood vessel characteristic optimization layer is used for acquiring an up-sampling image of the connected blood vessel characteristic processing layer; the lowest blood vessel feature optimization layer is also used for acquiring a feature image of the lowest blood vessel feature extraction layer; and each blood vessel characteristic optimization layer is also used for sequentially carrying out blood vessel characteristic extraction and nonlinear processing on each acquired image to obtain a nonlinear image corresponding to each image, and sending each acquired image to a higher blood vessel characteristic optimization layer through forward short connection. According to the invention, high-level information is transmitted to a low level through a backward short connection, and low-level information is transmitted to a high level through a forward short connection, so that characteristics of all levels are fully fused, and the blood vessel segmentation is more accurate.
Description
Technical field
The present embodiments relate to field of computer technology, and in particular to a kind of nerve for eye fundus image blood vessel segmentation
Network model.
Background technique
Retinal fundus images analysis facilitates oculist and handles cardiovascular and ophthalmology disease diagnosis, screening and control
It treats, such as macular degeneration, diabetic retinopathy, glaucoma, hypertension etc..If be not treated in time, these diseases may be led
Cause blindness.Blood vessel segmentation is the basic step of retinal images analysis, facilitates diabetic retinopathy and central foveal area
Positioning.But in clinical practice, the blood vessel manually in mark retinal images is time-consuming and needs a large amount of experiences.Therefore,
Retinal vessel is divided automatically to be necessary for reducing label time.
The automatic splitting scheme of retinal images blood vessel can be divided into two classes in recent decades: unsupervised scheme and have supervision
Scheme.
1) unsupervised scheme: unsupervised scheme can be divided into three subclasses: matched filtering, blood vessel tracing and based on model
Algorithm.Matched filtering is designed using piecewise linear approximation and the class Gaussian Profile of retinal vessel in matched filter
Core.Blood vessel tracing divides the blood vessel between two o'clock using local message, and the center of blood vessel longitudinal cross-section is by sum of the grayscale values blood
Pipe curvature determines.The model that algorithm based on model uses includes vessel sections model, movable contour model and is based on horizontal
The geometrical model of collection.
2) there is supervision scheme: having supervision scheme that can be counted as two classification problems of pixel scale.Each pixel belongs to
Blood vessel or non-vascular.There is supervision scheme that can be divided into three subclasses: Pixel-level classification method, patch (image block) grade segmentation side
Method and image level dividing method.Pixel-level method determines its attribute using classifier pixel-by-pixel, by providing a square window
Mouthful predict that the class label of each pixel, class label are used to indicate pixel and belong to blood vessel or non-vascular.For example, using deep
Degree convolutional neural networks classify to blood vessel pixel and non-vascular pixel.When handling extensive retinal images, Pixel-level
Method is time-consuming and is difficult to meet clinical requirement.In a sense, blood vessel identification can be counted as a semantic segmentation and ask
Topic.Common dividing method is made of patch grades of dividing methods and image level dividing method.The input of patch grades of dividing methods
It is a patch (image block), output is the corresponding vessel graph of patch.By piecing together the corresponding vessel graph of each patch one
Get up to obtain the blood vessel segmentation figure of retinal images.In addition, introducing hyper parameter patch size compared with image level dividing method
And original retinal images are divided into multiple patch, so that patch grades of dividing methods are more complicated and time-consuming.Image fraction
Segmentation method exports the blood vessel segmentation figure of original retinal images using original retinal images as input.
As it can be seen that neural network model is usually directed to the more additional conditions for needing to meet, to image in unsupervised scheme
Quality requirement is higher, and the blood vessel precision being partitioned into is lower.And have in supervision scheme, neural network model needs successively to extract figure
As feature, many useful information are lost, the parameter for causing neural network model to learn cannot portray blood vessel feature completely.
Summary of the invention
Of the existing technology in order to solve the problems, such as, at least one embodiment of the present invention provides a kind of for eyeground figure
As the neural network model of blood vessel segmentation.
For this purpose, the embodiment of the present invention proposes a kind of neural network model for eye fundus image blood vessel segmentation, comprising:
Multiple blood vessel feature extraction layers；
The multiple blood vessel feature extraction layer extracts the blood vessel feature in eye fundus image based on default hierarchic sequence；Lowermost layer
Blood vessel feature extraction layer extract eye fundus image in blood vessel feature, obtain characteristic image；The blood vessel feature extraction of not least layer
Layer carries out blood vessel feature extraction based on the characteristic image that low one layer of blood vessel feature extraction layer obtains, and obtains characteristic image；
Multiple blood vessel characteristic processing layers；
Multiple blood vessel feature extraction layers of the multiple blood vessel characteristic processing layer and not least layer connect one to one；Each
The blood vessel characteristic processing layer carries out up-sampling treatment to the characteristic image that the blood vessel feature extraction layer of connection obtains, and obtains adopting
Sampled images；
Multiple blood vessel characteristic optimization layers；
The blood vessel feature extraction layer of the blood vessel characteristic optimization layer connection lowermost layer of lowermost layer, multiple blood vessels of not least layer are special
Multiple blood vessel characteristic processing layers of sign optimization layer and not least layer connect one to one；Top blood vessel characteristic processing layer passes through
Backward short connection is connect with the blood vessel characteristic optimization layer of lowermost layer；Each blood vessel characteristic optimization layer by it is preceding to short connection with
High one layer of blood vessel characteristic optimization layer connection；Each blood vessel characteristic optimization layer, for obtaining the blood vessel characteristic processing of connection
The up-sampling image that layer obtains；The blood vessel characteristic optimization layer of lowermost layer, is also used to obtain the blood vessel feature extraction layer of lowermost layer
Characteristic image；Each blood vessel characteristic optimization layer is also used to successively carry out blood vessel feature extraction to each image of acquisition
Each of and nonlinear processing, obtain the corresponding non-linearization image of each image, and will acquire before passing through to short connection
Image is sent to high one layer of blood vessel characteristic optimization layer；
Blood-vessel image fused layer；
The blood-vessel image fused layer is separately connected each blood vessel characteristic optimization layer；The blood-vessel image fused layer base
In each preset fusion weight of blood vessel characteristic optimization layer, each blood vessel characteristic optimization layer of fusion obtains non-linear
Change image, obtains blood-vessel image.
In some embodiments, the multiple blood vessel feature extraction layer, comprising:
Convolutional layer conv1_2, convolutional layer conv2_2, convolutional layer conv3_3 and the volume of convolutional neural networks model VGG16
Lamination conv4_3.
In some embodiments, the multiple blood vessel characteristic processing layer, comprising:
Second up-sampling layer, third up-sampling layer and the 4th up-sampling layer；
Convolutional layer conv2_2 connection the second up-sampling layer；
The convolutional layer conv3_3 connection third up-samples layer；
Convolutional layer conv4_3 connection the 4th up-sampling layer.
In some embodiments, the multiple blood vessel characteristic optimization layer, comprising: the first convolutional layer, the second convolutional layer, third
Convolutional layer and Volume Four lamination；
Convolutional layer conv1_2 connection first convolutional layer；
The second up-sampling layer connects second convolutional layer；
The third up-sampling layer connects the third convolutional layer；
The 4th up-sampling layer connects the Volume Four lamination；
The 4th up-sampling layer is connect to short connection with first convolutional layer after passing through；
First convolutional layer is connect to short connection with second convolutional layer by preceding；
Second convolutional layer is connect to short connection with the third convolutional layer by preceding；
The third convolutional layer is connect to short connection with the Volume Four lamination by preceding.
In some embodiments, multiple blood vessel characteristic optimization layers, further includes: the first non-linear layer, second non-linear
Layer, third non-linear layer and the 4th non-linear layer；
First convolutional layer connects first non-linear layer；
Second convolutional layer connects second non-linear layer；
The third convolutional layer connects the third non-linear layer；
The Volume Four lamination connects the 4th non-linear layer.
In some embodiments, first non-linear layer, second non-linear layer, the third non-linear layer and institute
It states the 4th non-linear layer and is separately connected the blood-vessel image fused layer.
As it can be seen that at least one embodiment of the embodiment of the present invention, by it is rear to short connection high layer information is passed to it is low
Low level information is passed to high level to short connection by preceding, features at different levels is sufficiently merged, so that blood vessel segmentation is more accurate by layer.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below will be in embodiment or description of the prior art
Required attached drawing is briefly described, it should be apparent that, the accompanying drawings in the following description is only some realities of the invention
Example is applied, it for those of ordinary skill in the art, without any creative labor, can also be attached according to these
Figure obtains other attached drawings.
Fig. 1 is that a kind of structure of the neural network model for eye fundus image blood vessel segmentation provided in an embodiment of the present invention is shown
It is intended to；
Fig. 2 is the structure of another neural network model for eye fundus image blood vessel segmentation provided in an embodiment of the present invention
Schematic diagram.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiments of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill people
Member's every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
It should be noted that, in this document, the relational terms of such as " first " and " second " or the like are used merely to one
A entity or operation with another entity or operate distinguish, without necessarily requiring or implying these entities or operation it
Between there are any actual relationship or orders.
As shown in Figure 1, the present embodiment discloses a kind of neural network model for the identification of eye fundus image blood vessel, it may include:
Eye fundus image input layer, multiple blood vessel feature extraction layers, multiple blood vessel characteristic processing layers, multiple blood vessel characteristic optimization layers, blood vessel
Image co-registration layer and blood-vessel image output layer.
In the present embodiment, eye fundus image input layer, eye fundus image for receiving input.Although being appreciated that in Fig. 1 only
One eye fundus image input layer is shown, but those skilled in the art can setting be more in neural network model according to actual needs
A eye fundus image input layer, to receive multiple eye fundus images.
In some embodiments, eye fundus image input layer can be omitted.
In the present embodiment, multiple blood vessel feature extraction layers extract the blood vessel spy in eye fundus image based on default hierarchic sequence
Sign.In Fig. 1, the level of blood vessel feature extraction layer successively increases from left to right namely the blood vessel feature extraction layer of the leftmost side is most
The blood vessel feature extraction layer of low layer, the blood vessel feature extraction layer of the rightmost side are top blood vessel feature extraction layers.
In the present embodiment, the blood vessel feature extraction layer of lowermost layer extracts the blood vessel feature in eye fundus image, obtains characteristic pattern
Picture.It is special that the blood vessel feature extraction layer of not least layer based on the characteristic image that low one layer of blood vessel feature extraction layer obtains carries out blood vessel
Sign is extracted, and characteristic image is obtained.
In the present embodiment, blood vessel feature extraction layer realizes blood vessel feature extraction using convolution operation.Convolution is image procossing
Common method, give input picture, to obtained after input picture convolution output image.It is defeated for exporting each pixel in image
Enter the weighted average of pixel in a zonule in image.Average weighted weight is defined by a function, which is known as rolling up
Product core.Convolution kernel used in convolution operation can be by training neural network model to obtain in advance, and training method can be continued to use existing
Technology, this embodiment is not repeated.
In the present embodiment, multiple blood vessel feature extraction layers of multiple blood vessel characteristic processing layers and not least layer, which correspond, to be connected
It connects.Each blood vessel characteristic processing layer carries out up-sampling treatment to the characteristic image that the blood vessel feature extraction layer of connection obtains, and obtains
Up-sample image.
In the present embodiment, the up-sampling treatment of blood vessel characteristic processing layer can continue to use the prior art, and this embodiment is not repeated.
In the present embodiment, the blood vessel feature extraction layer of the blood vessel characteristic optimization layer connection lowermost layer of lowermost layer.Not least layer
Multiple blood vessel characteristic processing layers of multiple blood vessel characteristic optimization layers and not least layer connect one to one.Top blood vessel is special
Sign process layer is connect to short connection with the blood vessel characteristic optimization layer of lowermost layer after passing through.Each blood vessel characteristic optimization layer by it is preceding to
Short connection is connect with high one layer of blood vessel characteristic optimization layer.Each blood vessel characteristic optimization layer, for obtaining the blood vessel feature of connection
The up-sampling image that process layer obtains；The blood vessel characteristic optimization layer of lowermost layer, is also used to obtain the blood vessel feature extraction of lowermost layer
The characteristic image of layer；Each blood vessel characteristic optimization layer is also used to successively carry out blood vessel feature extraction to each image of acquisition
Each of and nonlinear processing, obtain the corresponding non-linearization image of each image, and will acquire before passing through to short connection
Image is sent to high one layer of blood vessel characteristic optimization layer.
In the present embodiment, by preceding to short connection, the semantic information of low layer is passed into high level, it is excellent to reduce blood vessel feature
The semantic gap changed between layer solves lowermost layer so that the non-linearization image that blood vessel characteristic optimization layer obtains is more accurate
The big problem of the obtained characteristic image noise of blood vessel feature extraction layer so that the vessel graph that blood-vessel image fused layer obtains
As more accurate.
In the present embodiment, by rear to short connection, high-rise structural information is passed into low layer, solves top blood
The fuzzy problem of the characteristic image that pipe feature extraction layer obtains, so that the blood-vessel image that blood-vessel image fused layer obtains is more
Accurately.
In the present embodiment, 1 × 1 convolution kernel and convolutional neural networks frame (Convolutional can be based on
Architecture for Fast Feature Embedding, CAFFE) provide attended operation realize before to it is short connection and
Backward short connection, details are not described herein for specific implementation code.The disclosed nerve for being used for eye fundus image blood vessel segmentation of the present embodiment
Network model can be described as supervising full convolutional neural networks model (S-DSN) for short connection depth.
In the present embodiment, blood-vessel image fused layer is separately connected each blood vessel characteristic optimization layer.Blood-vessel image fused layer base
In the preset fusion weight of each blood vessel characteristic optimization layer, the non-linearization image that each blood vessel characteristic optimization layer obtains is merged,
Obtain blood-vessel image.
In the present embodiment, by merging weight to each blood vessel characteristic optimization layer is default, so that blood-vessel image fused layer will
The non-linearization image that each blood vessel characteristic optimization layer obtains is merged according to corresponding fusion weight, is obtained more accurate
Blood-vessel image.Fusion weight can be by training neural network model to obtain in advance, and training method can continue to use the prior art, this implementation
Example repeats no more.
In the present embodiment, blood-vessel image output layer, the blood-vessel image obtained for the image co-registration layer that runs off vascular.
In some embodiments, blood-vessel image output layer can be omitted, and blood-vessel image fused layer can after obtaining blood-vessel image
Directly run off vascular image.
As it can be seen that the disclosed neural network model for eye fundus image blood vessel segmentation of the present embodiment will to short connection after passing through
High layer information passes to low layer, and low level information is passed to high level to short connection by preceding, features at different levels are sufficiently merged, so that blood
Pipe segmentation is more accurate.
Eye fundus image blood vessel is carried out based on the neural network model for being used for eye fundus image blood vessel segmentation disclosed in the present embodiment
The process of segmentation is described as follows step 1 to step 6:
Step 1: eye fundus image input layer receives the eye fundus image of input.
Step 2: multiple blood vessel feature extraction layers extract the blood vessel feature in eye fundus image based on default hierarchic sequence, obtain
Multiple characteristic images.Specifically, the blood vessel feature extraction layer of lowermost layer extracts the blood vessel feature in eye fundus image, obtains characteristic pattern
Picture.It is special that the blood vessel feature extraction layer of not least layer based on the characteristic image that low one layer of blood vessel feature extraction layer obtains carries out blood vessel
Sign is extracted, and characteristic image is obtained.
Step 3: the feature that multiple blood vessel characteristic processing layers respectively obtain multiple blood vessel feature extraction layers of not least layer
Image carries out up-sampling treatment, obtains up-sampling image.
Step 4: the blood vessel feature that the up-sampling image that this layer obtains is passed to connection by each blood vessel characteristic processing layer is excellent
Change layer.The blood vessel characteristic optimization layer of lowermost layer also obtains the characteristic image of the blood vessel feature extraction layer of lowermost layer.Top blood
The up-sampling image that this layer obtains is passed to the blood vessel characteristic optimization layer of lowermost layer by pipe characteristic processing layer to short connection after passing through.
Each blood vessel characteristic optimization layer successively carries out blood vessel feature extraction and nonlinear processing to each image of acquisition, obtains
The corresponding non-linearization image of each image.Each blood vessel characteristic optimization layer connects each image that will acquire hair to short by preceding
Give high one layer of blood vessel characteristic optimization layer.
Step 5: blood-vessel image fused layer is based on the preset fusion weight of each blood vessel characteristic optimization layer, merges each blood vessel
The non-linearization image that characteristic optimization layer obtains, obtains blood-vessel image.
Step 6: blood-vessel image output layer runs off vascular the blood-vessel image that image co-registration layer obtains.
In conclusion the disclosed neural network model for eye fundus image blood vessel segmentation of the present embodiment pass through after to short company
It connects and high layer information is passed into low layer, low level information is passed into high level to short connection by preceding, features at different levels is sufficiently merged, makes
It is more accurate to obtain blood vessel segmentation.The low disadvantage of traditional unsupervised scheme low efficiency, precision is overcome, while having overcome supervision side
In case, neural network model needs successively to extract characteristics of image, loses many useful information, neural network model is caused to learn
Parameter the shortcomings that cannot portraying blood vessel feature completely.
In some embodiments, as shown in Fig. 2, multiple blood vessel feature extraction layers, comprising: convolutional neural networks model
Convolutional layer conv1_2, convolutional layer conv2_2, convolutional layer conv3_3 and the convolutional layer conv4_3 of VGG16.
It should be noted that convolutional layer conv1_2, convolutional layer conv2_2, convolutional layer conv3_3 and convolutional layer in Fig. 2
Between conv4_3 and indirect connection, it is omitted and convolutional layer conv1_2, convolutional layer in neural network model shown in Fig. 2
Conv2_2, convolutional layer conv3_3 and the associated layer of convolutional layer conv4_3.Associated layer is layer used in VGG16, this
The neural network model of embodiment can be continued to use directly, and for the sake of clarity, these associated layers are omitted in Fig. 2.Specifically,
Associated layer specifically: convolutional layer conv1_1, non-linear layer relu1_1, non-linear layer relu1_2, pond layer pool1, volume
Lamination conv2_1, non-linear layer relu2_2, pond layer pool2, convolutional layer conv3_1, non-linear layer relu3_1, convolutional layer
Conv3_2, non-linear layer relu3_2, non-linear layer relu3_3, pond layer pool3, convolutional layer conv4_1, non-linear layer
Relu4_1, convolutional layer conv4_2 and non-linear layer relu4_2.The VGG16 of function and company in to(for) the above associated layer
The relationship of connecing explicitly defines, and the present embodiment is directly continued to use, therefore repeats no more.
In some embodiments, eye fundus image input layer can be omitted, and eye fundus image, which directly inputs, gives convolutional layer conv1_
1。
In the present embodiment, convolutional layer conv1_1, convolutional layer conv2_2, convolutional layer conv3_3 and convolutional layer conv4_3 are again
It can be described as side output (side-output) layer.
Based on upper one embodiment, as shown in Fig. 2, multiple blood vessel characteristic processing layers, comprising: the second up-sampling layer, third
Up-sample layer and the 4th up-sampling layer.
As shown in Fig. 2, convolutional layer conv2_2 connection second up-samples layer；Convolutional layer conv3_3 connection third up-sampling
Layer；Convolutional layer conv4_3 connection the 4th up-samples layer.
Based on upper one embodiment, as shown in Fig. 2, multiple blood vessel characteristic optimization layers, comprising: the first convolutional layer, volume Two
Lamination, third convolutional layer and Volume Four lamination.
Convolutional layer conv1_2 the first convolutional layer of connection.
Second up-sampling layer connects the second convolutional layer.
Third up-samples layer and connects third convolutional layer.
4th up-sampling layer connects Volume Four lamination.
4th up-sampling layer is connect to short connection with the first convolutional layer after passing through.
First convolutional layer is connect to short connection with the second convolutional layer by preceding.
Second convolutional layer is connect to short connection with third convolutional layer by preceding.
Third convolutional layer is connect to short connection with Volume Four lamination by preceding.
Based on upper one embodiment, as shown in Fig. 2, multiple blood vessel characteristic optimization layers, further includes: the first non-linear layer,
Two non-linear layers, third non-linear layer and the 4th non-linear layer；
First convolutional layer connects the first non-linear layer.
Second convolutional layer connects the second non-linear layer.
Third convolutional layer connects third non-linear layer.
Volume Four lamination connects the 4th non-linear layer.
In the present embodiment, the neural network model for eye fundus image blood vessel segmentation is total to there are four blood vessel characteristic optimization layer,
Each blood vessel characteristic optimization layer includes convolutional layer and non-linear layer.For example, the first convolutional layer and the first non-linear layer constitute one
A blood vessel characteristic optimization layer.
Based on upper one embodiment, as shown in Fig. 2, the first non-linear layer, the second non-linear layer, third non-linear layer and
Four non-linear layers are separately connected blood-vessel image fused layer.
Eye fundus image blood vessel segmentation is carried out based on the neural network model for being used for eye fundus image blood vessel segmentation disclosed in Fig. 2
Process is described as follows step 1 to step 6:
Step 1: eye fundus image input layer receives the eye fundus image of input.
Step 2: convolutional layer conv1_2 extracts the blood vessel feature in eye fundus image, obtains characteristic image.Convolutional layer conv2_
2 carry out blood vessel feature extraction based on the obtained characteristic image of convolutional layer conv1_2 again, obtain new characteristic image.Convolutional layer
Conv3_3 carries out blood vessel feature extraction based on the obtained characteristic image of convolutional layer conv2_2 again, obtains new characteristic image.
Convolutional layer conv4_3 carries out blood vessel feature extraction based on the obtained characteristic image of convolutional layer conv3_3 again, obtains new feature
Image.
Step 3: the second up-sampling layer, third up-sampling layer and the 4th up-sampling layer are respectively to convolutional layer conv2_2, convolution
The characteristic image that layer conv3_3 and convolutional layer conv4_3 is obtained carries out up-sampling treatment, obtains up-sampling image.
Step 4: the up-sampling figure that the second up-sampling layer, third up-sampling layer and the 4th up-sampling layer respectively obtain this layer
Image relaying gives the second convolutional layer, third convolutional layer and Volume Four lamination.Convolutional layer conv2_2 passes to obtained characteristic image
First convolutional layer.The up-sampling image that this layer obtains to short connection is passed to the first convolutional layer after passing through by the 4th up-sampling layer.
First convolutional layer, the second convolutional layer, third convolutional layer and Volume Four lamination carry out blood vessel feature to each image of acquisition respectively
It extracts.First convolutional layer, the second convolutional layer and third convolutional layer transmit before passing through respectively to the short each image that will acquire that connects
To high one layer of blood vessel characteristic optimization layer.First non-linear layer, the second non-linear layer, third non-linear layer and the 4th non-linear layer
Each image after the first convolutional layer, the second convolutional layer, third convolutional layer and Volume Four lamination process of convolution is carried out respectively non-
Linearization process obtains non-linearization image.
Step 5: blood-vessel image fused layer is based on the preset fusion weight of each blood vessel characteristic optimization layer, and fusion first is non-thread
The non-linearization image that property layer, the second non-linear layer, third non-linear layer and the 4th non-linear layer obtain, obtains blood-vessel image.
Step 6: blood-vessel image output layer runs off vascular the blood-vessel image that image co-registration layer obtains.
In the present embodiment, using publicly available retinal images blood vessel segmentation data set DRIVE to neural network model
It is trained.DRIVE is made of 40 colored fundus photographs.40 colored fundus photographs are divided into a training set and a test
Collection respectively includes 20 images.
In the present embodiment, the image in training set is pre-processed and obtains that example is trained to shine, for inputting nerve net
Network model.Pretreatment includes: 90 ° of rotations, 180 ° of rotations and 270 ° of rotations, flip horizontal and flip vertical, and to image ruler
One of very little 0.5 times and 0.8 times of scaling is a variety of.
In the present embodiment, the parameter setting during training neural network model is as follows:
For the S-DSN of image level, input data is eye fundus image, and output data is the corresponding blood of eye fundus image
Pipe image.For the S-DSN of patch rank, eye fundus image is divided into 9 patch.9 patch are made after 2 times of up-samplings
The S-DSN of patch rank is inputted for input data.
In the present embodiment, learning rate is set as 10-8, weight attenuation coefficient is 0.0005, momentum 0.9.Side output
(side-output) layer is four layers.Neural network model training learning rate after 25000 iteration reduces by 10 times, 50000 times
Preset parameter is as neural network model final argument after iteration.
It should be noted that, in this document, the terms "include", "comprise" or its any other variant are intended to non-row
His property includes, so that the process, method, article or the device that include a series of elements not only include those elements, and
And further include other elements that are not explicitly listed, or further include for this process, method, article or device institute it is intrinsic
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including being somebody's turn to do
There is also other identical elements in the process, method of element, article or device.
It will be appreciated by those of skill in the art that although some embodiments described herein include other embodiments is wrapped
Certain features for including rather than other feature, but the combination of the feature of different embodiments mean in the scope of the present invention it
It is interior and form different embodiments.
Although the embodiments of the invention are described in conjunction with the attached drawings, but those skilled in the art can not depart from this hair
Various modifications and variations are made in the case where bright spirit and scope, such modifications and variations are each fallen within by appended claims
Within limited range.
Claims (6)
1. a kind of neural network model for the identification of eye fundus image blood vessel characterized by comprising
Multiple blood vessel feature extraction layers；
The multiple blood vessel feature extraction layer extracts the blood vessel feature in eye fundus image based on default hierarchic sequence；The blood of lowermost layer
Pipe feature extraction layer extracts the blood vessel feature in eye fundus image, obtains characteristic image；The blood vessel feature extraction layer base of not least layer
Blood vessel feature extraction is carried out in the characteristic image that low one layer of blood vessel feature extraction layer obtains, obtains characteristic image；
Multiple blood vessel characteristic processing layers；
Multiple blood vessel feature extraction layers of the multiple blood vessel characteristic processing layer and not least layer connect one to one；It is each described
Blood vessel characteristic processing layer carries out up-sampling treatment to the characteristic image that the blood vessel feature extraction layer of connection obtains, and obtains up-sampling figure
Picture；
Multiple blood vessel characteristic optimization layers；
The blood vessel feature extraction layer of the blood vessel characteristic optimization layer connection lowermost layer of lowermost layer, multiple blood vessel features of not least layer are excellent
The multiple blood vessel characteristic processing layers for changing layer and not least layer connect one to one；Top blood vessel characteristic processing layer passes through backward
Short connection is connect with the blood vessel characteristic optimization layer of lowermost layer；Each blood vessel characteristic optimization layer passes through preceding to short connection and high by one
The blood vessel characteristic optimization layer connection of layer；Each blood vessel characteristic optimization layer, the blood vessel characteristic processing layer for obtaining connection obtain
The up-sampling image arrived；The blood vessel characteristic optimization layer of lowermost layer, is also used to obtain the feature of the blood vessel feature extraction layer of lowermost layer
Image；Each blood vessel characteristic optimization layer, be also used to successively carry out each image of acquisition blood vessel feature extraction and
Nonlinear processing obtains the corresponding non-linearization image of each image, and connects each image that will acquire to short by preceding
It is sent to high one layer of blood vessel characteristic optimization layer；
Blood-vessel image fused layer；
The blood-vessel image fused layer is separately connected each blood vessel characteristic optimization layer；The blood-vessel image fused layer is based on every
A preset fusion weight of the blood vessel characteristic optimization layer merges the non-linearization figure that each blood vessel characteristic optimization layer obtains
Picture obtains blood-vessel image.
2. neural network model according to claim 1, which is characterized in that the multiple blood vessel feature extraction layer, comprising:
Convolutional layer conv1_2, convolutional layer conv2_2, convolutional layer conv3_3 and the convolutional layer of convolutional neural networks model VGG16
conv4_3。
3. neural network model according to claim 2, which is characterized in that the multiple blood vessel characteristic processing layer, comprising:
Second up-sampling layer, third up-sampling layer and the 4th up-sampling layer；
Convolutional layer conv2_2 connection the second up-sampling layer；
The convolutional layer conv3_3 connection third up-samples layer；
Convolutional layer conv4_3 connection the 4th up-sampling layer.
4. neural network model according to claim 3, which is characterized in that the multiple blood vessel characteristic optimization layer, comprising:
First convolutional layer, the second convolutional layer, third convolutional layer and Volume Four lamination；
Convolutional layer conv1_2 connection first convolutional layer；
The second up-sampling layer connects second convolutional layer；
The third up-sampling layer connects the third convolutional layer；
The 4th up-sampling layer connects the Volume Four lamination；
The 4th up-sampling layer is connect to short connection with first convolutional layer after passing through；
First convolutional layer is connect to short connection with second convolutional layer by preceding；
Second convolutional layer is connect to short connection with the third convolutional layer by preceding；
The third convolutional layer is connect to short connection with the Volume Four lamination by preceding.
5. neural network model according to claim 4, which is characterized in that multiple blood vessel characteristic optimization layers also wrap
It includes: the first non-linear layer, the second non-linear layer, third non-linear layer and the 4th non-linear layer；
First convolutional layer connects first non-linear layer；
Second convolutional layer connects second non-linear layer；
The third convolutional layer connects the third non-linear layer；
The Volume Four lamination connects the 4th non-linear layer.
6. neural network model according to claim 5, which is characterized in that first non-linear layer, described second non-
Linear layer, the third non-linear layer and the 4th non-linear layer are separately connected the blood-vessel image fused layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810764191.7A CN109064453B (en) | 2018-07-12 | 2018-07-12 | Neural network model for fundus image blood vessel segmentation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810764191.7A CN109064453B (en) | 2018-07-12 | 2018-07-12 | Neural network model for fundus image blood vessel segmentation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109064453A CN109064453A (en) | 2018-12-21 |
| CN109064453B true CN109064453B (en) | 2019-07-23 |
Family
ID=64816301
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810764191.7A Active CN109064453B (en) | 2018-07-12 | 2018-07-12 | Neural network model for fundus image blood vessel segmentation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109064453B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110399833B (en) * | 2019-07-25 | 2023-03-24 | 上海鹰瞳医疗科技有限公司 | Identity recognition method, modeling method and equipment |
| CN110610480B (en) * | 2019-08-02 | 2020-07-24 | 成都上工医信科技有限公司 | MCASPP neural network eyeground image optic cup optic disc segmentation model based on Attention mechanism |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106355575A (en) * | 2016-08-31 | 2017-01-25 | 上海交通大学 | Supervised learning type ocular fundus vessel extraction method based on neural network |
| CN108198211A (en) * | 2017-11-20 | 2018-06-22 | 海纳医信（北京）软件科技有限责任公司 | The processing method and processing device of eye fundus image, storage medium, processor |
-
2018
- 2018-07-12 CN CN201810764191.7A patent/CN109064453B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106355575A (en) * | 2016-08-31 | 2017-01-25 | 上海交通大学 | Supervised learning type ocular fundus vessel extraction method based on neural network |
| CN108198211A (en) * | 2017-11-20 | 2018-06-22 | 海纳医信（北京）软件科技有限责任公司 | The processing method and processing device of eye fundus image, storage medium, processor |
Non-Patent Citations (1)
| Title |
|---|
| Deeply supervised neural network with short connections for retinal vessel segmentation;Song Guo等;《arXiv:1803.03963v1》;20180311;第1-14页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109064453A (en) | 2018-12-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108921169B (en) | A kind of eye fundus image blood vessel segmentation method | |
| Yu et al. | Robust optic disc and cup segmentation with deep learning for glaucoma detection | |
| CN108876775B (en) | Method for rapidly detecting diabetic retinopathy | |
| CN106408562B (en) | Eye fundus image Segmentation Method of Retinal Blood Vessels and system based on deep learning | |
| WO2021253939A1 (en) | Rough set-based neural network method for segmenting fundus retinal vascular image | |
| EP3659067B1 (en) | Method of modifying a retina fundus image for a deep learning model | |
| CN110211087B (en) | Sharable semiautomatic marking method for diabetic fundus lesions | |
| CN112883962B (en) | Fundus image recognition method, fundus image recognition apparatus, fundus image recognition device, fundus image recognition program, and fundus image recognition program | |
| CN107679525A (en) | Image classification method, device and computer-readable recording medium | |
| CN112132817A (en) | Retina blood vessel segmentation method for fundus image based on mixed attention mechanism | |
| CN109886273A (en) | A kind of CMR classification of image segmentation system | |
| CN110097554A (en) | The Segmentation Method of Retinal Blood Vessels of convolution is separated based on intensive convolution sum depth | |
| CN108388890A (en) | A kind of neonatal pain degree assessment method and system based on human facial expression recognition | |
| CN106682616A (en) | Newborn-painful-expression recognition method based on dual-channel-characteristic deep learning | |
| CN109410204A (en) | A kind of processing of cortex cataract image and Enhancement Method based on CAM | |
| CN110837803A (en) | Diabetic retinopathy grading method based on depth map network | |
| CN109635812B (en) | The example dividing method and device of image | |
| CN115205300A (en) | Fundus blood vessel image segmentation method and system based on cavity convolution and semantic fusion | |
| CN109671094A (en) | A kind of eye fundus image blood vessel segmentation method based on frequency domain classification | |
| CN109064453B (en) | Neural network model for fundus image blood vessel segmentation | |
| CN109087296A (en) | A method of extracting human region in CT image | |
| CN107145885A (en) | A kind of individual character figure character recognition method and device based on convolutional neural networks | |
| CN106886991A (en) | A kind of fuzziness automatic grading method based on colored eyeground figure | |
| CN109919915A (en) | Retina fundus image abnormal region detection method and device based on deep learning | |
| CN104463216A (en) | Eye movement pattern data automatic acquisition method based on computer vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220511 Address after: 519000 unit Z, room 615, 6th floor, main building, No. 10, Keji 1st Road, Gangwan Avenue, Tangjiawan Town, Xiangzhou District, Zhuhai City, Guangdong Province (centralized office area) Patentee after: Zhuhai Quanyi Technology Co.,Ltd. Address before: Room 701-3, No. 2, yard 9, Ronghua South Road, Beijing 100176 Patentee before: BEIJING SHANGGONG YIXIN TECHNOLOGY CO.,LTD. |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240131 Address after: 519000 6th floor, building 1, No. 88, Keji 6th Road, high tech Zone, Zhuhai, Guangdong Province Patentee after: Quanyi medical (Zhuhai) Co.,Ltd. Country or region after: China Address before: 519000 unit Z, room 615, 6th floor, main building, No. 10, Keji 1st Road, Gangwan Avenue, Tangjiawan Town, Xiangzhou District, Zhuhai City, Guangdong Province (centralized office area) Patentee before: Zhuhai Quanyi Technology Co.,Ltd. Country or region before: China |