CN110335254A - Eye fundus image compartmentalization deep learning method, apparatus and equipment and storage medium - Google Patents
Eye fundus image compartmentalization deep learning method, apparatus and equipment and storage medium Download PDFInfo
- Publication number
- CN110335254A CN110335254A CN201910498239.9A CN201910498239A CN110335254A CN 110335254 A CN110335254 A CN 110335254A CN 201910498239 A CN201910498239 A CN 201910498239A CN 110335254 A CN110335254 A CN 110335254A
- Authority
- CN
- China
- Prior art keywords
- eye fundus
- fundus image
- image
- template
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Eye Examination Apparatus (AREA)
- Image Analysis (AREA)
Abstract
This application involves a kind of eye fundus image compartmentalization deep learning method, apparatus and equipment and storage mediums, and wherein method includes: acquisition eye fundus image, and construct corresponding image template based on the eye fundus image got；Wherein, the different zones in different image moulds characterization eye fundus image；By choosing corresponding current desired template in image template, and based on selected current desired template, eye fundus image is handled, the eye fundus image that obtains that treated；By treated, eye fundus image is input to the neural network model constructed in advance, is trained to neural network model.By constructing image template, so that used eye fundus image is no longer whole picture eye fundus image when to neural network model training, but based on selected current desired template to the eye fundus image image that carries out that treated.To which this also just effectively eliminates unnecessary disturbing factor, the robustness and generalization of the neural network model after finally improving training.
Description
Technical field
This disclosure relates to technical field of medical image processing more particularly to a kind of eye fundus image compartmentalization deep learning side
Method, device and equipment and storage medium.
Background technique
It is the technology driven that deep learning, which is with data, has become the preferred solution in image domains at present.
In numerous medical images, the characteristics of colored ophthalmoscopic image, is that complexity is low, obtains convenient and be easy diagnosis, these features make
Obtaining ophthalmoscopic image can be used the preferable classification of depth learning technology acquirement and detection effect.It is existing to be based on colored ophthalmoscopic image
Classification of diseases and detection technique be essentially all that the image scale of construction, picture quality and are strongly dependent upon using depth learning technology
And mark accuracy.In the related art, using depth learning technology carry out ophthalmoscopic image classification and detection when, be mostly with
Entire image carries out model learning as learning materials, and it is more that this allows for the interference during model learning, to influence
The robustness and generalization of model.
Summary of the invention
In view of this, the present disclosure proposes a kind of eye fundus image compartmentalization deep learning method, apparatus and equipment and storages
Medium, it is possible to prevente effectively from the unnecessary interference during model learning, improves the robustness and generalization of model.
According to the one side of the disclosure, a kind of eye fundus image compartmentalization deep learning method is provided, comprising:
Eye fundus image is obtained, and corresponding image template is constructed based on the eye fundus image got；
Wherein, different described image moulds characterizes the different zones in the eye fundus image；
By choosing corresponding current desired template in described image template, and based on the selected current desired mould
Plate handles the eye fundus image, the eye fundus image that obtains that treated；
By treated, eye fundus image is input to the neural network model constructed in advance, carries out to the neural network model
Training.
In one possible implementation, corresponding image template, packet are constructed based on the eye fundus image got
It includes:
Using algorithm of target detection in the eye fundus image optic disk region and macular region carry out detection and localization, obtain
Corresponding object detection results；
According to the object detection results, structured analysis is carried out to the eye fundus image, obtains corresponding described image
Template.
In one possible implementation, according to the object detection results, structuring is carried out to the eye fundus image
Analysis, comprising:
When the object detection results are the optic disk region and the visible macular region, by the eye fundus image
Contour line moved respectively along the center in the optic disk region and the direction of both ends of the central junction line of the macular region, obtain
Corresponding first profile and the second profile；
Wherein, the first profile and second profile are tangent with the edge in the optic disk region；
According to the first profile and second profile respectively with the intersection situation of the eye fundus image, generate corresponding
Described image template.
In one possible implementation, according to the object detection results, structuring is carried out to the eye fundus image
Analysis, comprising:
It in the object detection results is the optic disk region as it can be seen that when the invisible macular region, by the eyeground
Direction of both ends of the contour line of image respectively along the connecting line at the center at the center and eye fundus image in the optic disk region is moved
It is dynamic, obtain corresponding third profile and fourth contoured；
Wherein, the third profile and the fourth contoured are tangent with the edge in the optic disk region；
According to the third profile and the fourth contoured respectively with the intersection situation of the eye fundus image, generate corresponding
Described image template.
In one possible implementation, the object detection results are based on, structuring is carried out to the eye fundus image
Analysis, comprising:
When the object detection results are that the macular region is visible, the optic disk region is invisible, according to being positioned
The eye fundus image is carried out region division, generates corresponding described image template by the macular region out.
In one possible implementation, based on the selected current desired template, to the eye fundus image into
Row processing, comprising:
The pixel value in the region of eye fundus image corresponding to the current desired template by selection remains original image area
The pixel value in domain, and zero is set by the pixel value of eye fundus image corresponding to the image template that do not choose.
In one possible implementation, the neural network model constructed in advance includes basic network and multiple sons
Network；
Wherein, the output end of the basic network is the input terminal of each sub-network；
The basic network includes that successively cascade multiple convolution combination layers, each sub-network include cascade convolution
Layer and full articulamentum.
According to another aspect of the present disclosure, a kind of eye fundus image compartmentalization deep learning device is additionally provided, comprising:
Image template constructs module, is configured as obtaining eye fundus image, and based on the eye fundus image building got
Corresponding image template；Wherein, different described image templates characterizes the different zones in the eye fundus image；
Eye fundus image processing module is configured as by choosing corresponding current desired template, and base in described image template
In the selected current desired template, the eye fundus image is handled, the eye fundus image that obtains that treated；
Network model training module, is configured as that eye fundus image is input to the neural network mould constructed in advance by treated
Type is trained the neural network model.
According to the one side of the disclosure, a kind of eye fundus image compartmentalization deep learning equipment is additionally provided, comprising:
Processor；
Memory for storage processor executable instruction；
Wherein, the processor is configured to realizing any eye fundus image in front when executing the executable instruction
Compartmentalization deep learning method.
According to another aspect of the present disclosure, a kind of non-volatile computer readable storage medium storing program for executing is additionally provided, is stored thereon
There are computer program instructions, any eye fundus image area in front is realized when the computer program instructions are executed by processor
Domain deep learning method.
By based on get eye fundus image building characterization eye fundus image in different zones image template, and then again by
Choose corresponding current desired template in image template, and based on selected current desired template to eye fundus image at
Reason, the eye fundus image that obtains that treated.Finally, by treated, eye fundus image is input to the neural network mould constructed in advance again
Type is trained neural network model, and used eye fundus image is no longer when this is allowed for neural network model training
Whole picture eye fundus image, but treated image is carried out to eye fundus image based on selected current desired template.To this
Unnecessary disturbing factor is just effectively eliminated, the robustness and generalization of the neural network model after finally improving training.
According to below with reference to the accompanying drawings to detailed description of illustrative embodiments, the other feature and aspect of the disclosure will become
It is clear.
Detailed description of the invention
Comprising in the description and constituting the attached drawing of part of specification and specification together illustrates the disclosure
Exemplary embodiment, feature and aspect, and for explaining the principles of this disclosure.
Fig. 1 shows the flow chart of the eye fundus image compartmentalization deep learning method of the embodiment of the present disclosure；
Fig. 2 shows the process design schematic diagrames of the eye fundus image compartmentalization deep learning method of the embodiment of the present disclosure；
Fig. 3 is shown in the eye fundus image compartmentalization deep learning method of the embodiment of the present disclosure, is view in object detection results
The regular schematic diagram of image template is constructed when disk area and visible macular region；
Fig. 4 is shown in the eye fundus image compartmentalization deep learning method of the embodiment of the present disclosure, is view in object detection results
The regular schematic diagram of image template is constructed when disk area is visible but macular region is invisible；
Fig. 5 is shown in the eye fundus image compartmentalization deep learning method of the embodiment of the present disclosure, is Huang in object detection results
The regular schematic diagram of image template is constructed when spot region is visible, optic disk region is invisible；
Fig. 6 shows neural network model constructed in the eye fundus image compartmentalization deep learning method of the embodiment of the present disclosure
In basic network schematic network structure；
Fig. 7 shows neural network model constructed in the eye fundus image compartmentalization deep learning method of the embodiment of the present disclosure
In sub-network schematic network structure；
Fig. 8 shows the block diagram of the eye fundus image compartmentalization deep learning device of the embodiment of the present disclosure；
Fig. 9 shows the block diagram of the eye fundus image compartmentalization deep learning equipment of the embodiment of the present disclosure.
Specific embodiment
Various exemplary embodiments, feature and the aspect of the disclosure are described in detail below with reference to attached drawing.It is identical in attached drawing
Appended drawing reference indicate element functionally identical or similar.Although the various aspects of embodiment are shown in the attached drawings, remove
It non-specifically points out, it is not necessary to attached drawing drawn to scale.
Dedicated word " exemplary " means " being used as example, embodiment or illustrative " herein.Here as " exemplary "
Illustrated any embodiment should not necessarily be construed as preferred or advantageous over other embodiments.
In addition, giving numerous details in specific embodiment below to better illustrate the disclosure.
It will be appreciated by those skilled in the art that without certain details, the disclosure equally be can be implemented.In some instances, for
Method, means, element and circuit well known to those skilled in the art are not described in detail, in order to highlight the purport of the disclosure.
Fig. 1 shows the flow chart of the eye fundus image compartmentalization deep learning method of the embodiment of the present disclosure.Fig. 2 shows the disclosure
The process design schematic diagram of the eye fundus image compartmentalization deep learning method of embodiment.Refering to fig. 1 and Fig. 2, the embodiment of the present disclosure
Eye fundus image compartmentalization deep learning method include: step S100, eye fundus image is obtained, and based on the eye fundus image got
Construct corresponding image template.Herein, it should be noted that the number of image template can be multiple, different image template
Characterize the different zones in eye fundus image.
Such as: image template may include optic disk region template, macular region template, upper hemal arch region template, under
At least one of nasal side region template by hemal arch region template, macula lutea peripheral vessels region template and optic disk.Wherein,
Optic disk region template corresponds to the optic disk region in eye fundus image.Macular region template corresponds to the macular region in eye fundus image.On
Hemal arch region template corresponds to the partial region being located at optic disk overlying regions position in eye fundus image.Lower hemal arch region
Template then corresponds to the partial region for being located at optic disk region lower position in eye fundus image.Macula lutea peripheral vessels region template pair
Answer the partial region of macular region ambient side in eye fundus image.Nasal side region template, which then corresponds to, by optic disk is located at view in eye fundus image
Partial region of the disk area far from macular region side.
Simultaneously, it should be further noted that the number and type of constructed image template can be according to different eyeground figures
As different, herein without specifically limiting.
S100 is being executed the step, after constructing corresponding image template, i.e., executable step S200, by being selected in image template
Corresponding current desired template is taken, and based on selected current desired template, eye fundus image is handled, after obtaining processing
Eye fundus image.And then step S300 is executed again, by treated, eye fundus image is input to the neural network model constructed in advance,
Neural network model is trained.
The eye fundus image compartmentalization deep learning method of the embodiment of the present disclosure as a result, by based on the eyeground figure got
As the image template of different zones in building characterization eye fundus image, and then again by choosing corresponding current desired mould in image template
Plate, and eye fundus image being handled based on selected current desired template, the eye fundus image that obtains that treated.Finally, again
By treated, eye fundus image is input to the neural network model constructed in advance, is trained to neural network model, this just makes
To neural network model training when used eye fundus image be no longer whole picture eye fundus image, but based on selected current
Required template carries out treated image to eye fundus image.To which this also just effectively eliminates unnecessary disturbing factor, finally
The robustness and generalization of neural network model after improving training.
In one possible implementation, in step S100, corresponding image is constructed based on the eye fundus image got
Template can be accomplished by the following way.
That is, first using algorithm of target detection in eye fundus image optic disk region and macular region carry out detection and localization,
Obtain corresponding object detection results.Here, it should be pointed out that under normal circumstances, optic disk region and Huang in eye fundus image
Spot region is obvious, in order to guarantee the accuracy of target detection, in use algorithm of target detection to the optic disk in eye fundus image
It, can be using the Faster Rcnn model based on deep learning to optic disk region when region and macular region carry out detection and localization
It is detected with macular region.Wherein, the Faster Rcnn model based on deep learning is image procossing net commonly used in the art
Network model, herein no longer repeats it.
After obtaining corresponding object detection results, and then further according to object detection results, structure is carried out to eye fundus image
Change analysis, obtains corresponding image template.
Herein, it is noted that when carrying out detection and localization to different eye fundus images using algorithm of target detection, institute
Obtained object detection results may be different.Therefore, structuring is being carried out to eye fundus image according to different object detection results
Analysis, obtained image template would also vary from.
For example, object detection results may include that optic disk region and macular region are visible (that is, optic disk region and Huang
The equal detection and localization of spot region arrives), optic disk region is visible but macular region is invisible (that is, view of the detection and localization into eye fundus image
Disk area, macular region are not arrived by detection and localization), optic disk region is invisible but macular region is visible (that is, detection and localization is to eyeground
Macular region in image, optic disk region are not arrived by detection and localization) and optic disk region and macular region it is invisible (that is, eye
Optic disk region and macular region in base map picture are not arrived by detection and localization) four kinds.
Wherein, when object detection results are optic disk region and invisible macular region, at this point, showing current acquired
The eye fundus image arrived is invalid.That is, current accessed eye fundus image not can be used as number of training according to for neural network mould
The training study of type.Therefore, it is sightless as a result, and by the base map of opening one's eyes that optic disk region and macular region can directly be exported
As discarding, directly acquires next eye fundus image and carry out detection and localization.
When object detection results are optic disk region and visible macular region, then show current accessed eyeground figure
As effectively, can be used as the training that sample data carries out neural network model.It as a result, can be according to object detection results to eyeground
Image carries out structured analysis, to obtain corresponding image template.
Wherein, when object detection results are optic disk region and visible macular region, structuring is carried out to eye fundus image
Analysis, can be accomplished by the following way.
When object detection results are optic disk region and visible macular region, by the contour line of eye fundus image respectively along view
The direction of both ends of the central junction line at the center and macular region of disk area is mobile, to obtain first profile and the second profile.
Wherein, first profile and the second profile are tangent with the edge in optic disk region.
In turn, corresponding image is generated respectively with the intersection situation of eye fundus image further according to first profile and the second profile
Template.
Herein, it should be noted that the contour line of eye fundus image can be circle, or rectangle.Herein without
It limits.Correspondingly, the shape at optic disk region and the edge of macular region both can be circle, or rectangle.Before use
Algorithm of target detection described in face (such as: Faster Rcnn model) carries out detection and localization to eye fundus image, orients optic disk region
When with macular region, the edge of usual optic disk region and macular region is defined using rectangle frame.In the embodiment of the present disclosure
In eye fundus image compartmentalization deep learning method, for the ease of operation, rectangle frame can be fitted to by circle by fit approach,
So that the edge (that is, profile) in optic disk region and macular region that detection and localization goes out is rounded.
In order to illustrate more clearly of object detection results be optic disk region and visible macular region when, image template
Building rule, is below circle with the contour line of eye fundus image, and the edge in optic disk region and macular region that detection and localization goes out is in
It is illustrated for rectangle frame.
Refering to Fig. 3, when object detection results are optic disk region and visible macular region, with the center in optic disk region with
The both ends of the connecting line at the center of macular region be moving direction, by the contour line of eye fundus image respectively to the both ends of connecting line into
Row movement, to move out two dotted outlines (that is, first profile and second profile).Wherein, the distance moved is to move respectively
It is dynamic as the both sides of edges of optic disk region (rectangle or circle) or so it is tangent until.That is, the edge of first profile and optic disk region
Side is tangent, and the other side of the second profile and the edge in optic disk region is tangent.
In turn, corresponding image is generated respectively with the intersection situation of eye fundus image further according to first profile and the second profile
Template.That is, the corresponding optic disk region template (that is, region 1) of optic disk Area generation refering to Fig. 3, in corresponding eye fundus image；It is corresponding
Macular region in eye fundus image generates corresponding macular region template (that is, region 2)；Corresponding first profile, the second profile and
In the profile area defined of eye fundus image, the Area generation positioned at optic disk overlying regions goes up hemal arch region mould accordingly
Plate (that is, region 3)；In the profile area defined of corresponding first profile, the second profile and eye fundus image, it is located at optic disk region
The Area generation of lower section descends hemal arch region template (that is, region 4) accordingly；Corresponding first profile and eye fundus image cross-shaped portion
Point region in, remove macular region except the corresponding macula lutea peripheral vessels region template of other parts Area generation (that is,
Region 5)；The second profile of correspondence optic disk side nasal side region template corresponding with the Area generation of the intersection of eye fundus image (that is,
Region 6).
Further, when object detection results are that optic disk region is visible, macular region is invisible, at this time to eye fundus image
Carry out structured analysis, can by by the contour line of eye fundus image respectively along the center in optic disk region and the center of eye fundus image
Connecting line direction of both ends it is mobile, obtain the mode of corresponding third profile and fourth contoured to realize.Wherein, third profile
It is tangent with the edge in optic disk region with fourth contoured.Third profile and fourth round are obtained by the profile of mobile eye fundus image again
After exterior feature, and then corresponding image template is generated respectively with the intersection situation of eye fundus image further according to third profile and fourth contoured.
Herein, it is noted that the shape of the contour line of eye fundus image and the edge shape in optic disk region are also can be with
It is configured according to the actual situation.Such as: the shape of the contour line of eye fundus image can be circle, or rectangle.Optic disk
The edge shape in region can be circle, or rectangle frame.Herein without specifically limiting.
For example, the contour line of eye fundus image is circle, and the edge shape in optic disk region is rectangle frame.In target detection
When being as a result that optic disk region is visible, macular region is invisible, refering to Fig. 4, by the contour line of eye fundus image respectively along optic disk region
Center and eye fundus image center connecting line both ends it is mobile, until mobile obtained third profile and fourth contoured difference
With the edge in optic disk region it is tangent until.
And then corresponding image mould is generated respectively with the intersection situation of eye fundus image further according to third profile and fourth contoured
Plate.That is, the corresponding optic disk region template (that is, region 1 ') of optic disk Area generation refering to Fig. 4, in corresponding eye fundus image；It is corresponding
In the profile area defined of first profile, the second profile and eye fundus image, positioned at the Area generation phase of optic disk overlying regions
The upper hemal arch region template (that is, region 3 ') answered；The profile of corresponding first profile, the second profile and eye fundus image is surrounded
Region in, the Area generation below optic disk region descends hemal arch region template (that is, region 4 ') accordingly；Corresponding the
One profile macular region corresponding with the Area generation of eye fundus image intersection and macula lutea peripheral vessels region template (that is,
Region 5 ')；Nasal side region template by corresponding second profile optic disk corresponding with the Area generation of the intersection of eye fundus image
(that is, region 6 ').
It further, at this time can basis when object detection results are that macular region is visible, optic disk region is invisible
The volume macular region oriented carries out region division to eye fundus image, generates corresponding image template.
That is, when object detection results are that macular region is visible, optic disk region is invisible, can be generated at this time refering to Fig. 5
Two kinds of templates.That is, directly generate corresponding macular region template (that is, region 2 ') for macular region is corresponding, remaining region (that is,
Other regions in eye fundus image in addition to macular region) correspond to remaining region template (that is, region 7) of generation.
Herein, it is noted that after generating corresponding image template based on different object detection results buildings,
When the number of image template is multiple, multiple images template can carry out record storage in the form of form assembly list, so as to
Current desired template can be faster searched out when the subsequent selection for carrying out image template, so that the choosing of current desired template
It takes more efficiently.
It, can be by step S200, by the figure constructed after constructing corresponding image template by any of the above-described kind of mode
As selecting current desired template in template, and based on the current desired template selected, eye fundus image is handled.
Herein, it is noted that, can be according to current when choosing current desired template in the image template by constructing
Detection demand is chosen.I.e., it is generally the case that different eyeground diseases needs to pay close attention to the different zones in eye fundus image.
Such as: glaucoma disease needs to combine optic disk region, positioned at the upper hemal arch region of optic disk overlying regions and positioned at optic disk region
The lower hemal arch region of lower section is judged.Age related macular degeneration then needs to be judged according to macular region.Diabetes view
Nethike embrane disease then needs that other regions other than the optic disk region in eye fundus image is combined to be judged.
As a result, according to current detection demand, by selecting current desired template in the image template that constructs, and based on choosing
The current desired template taken out handles eye fundus image, so that finally carrying out used when the training of neural network model
Eye fundus image more has specific aim.
It is also desirable to explanation, current detection demand can be according to the function of currently trained neural network model
To determine.Such as: currently the function of trained neural network model can be then determined current when carrying out the detection of glaucoma disease
Detection demand is that glaucoma disease detects demand, therefore detects demand according to determining glaucoma disease, the figure generated by building
As choosing corresponding optic disk region template, upper hemal arch region template and lower hemal arch region template these three moulds in template
Board group is closed.Then, then based on selected form assembly eye fundus image is handled, so that eye fundus image is made by treated
For training sample, study is trained to neural network model.
In one possible implementation, when being handled based on selected current desired template eye fundus image,
It can be accomplished by the following way.
That is, the pixel value in the region of eye fundus image corresponding to current desired template by selection remains original image area
The pixel value in domain, and zero is set by the pixel value of eye fundus image corresponding to the image template that do not choose.
For example, example is still detected as with mentioned-above glaucoma disease, selected form assembly at this time includes view
Disk area template, upper hemal arch region template and lower hemal arch region template.As a result, based on these three selected figures
It, can be by optic disk region, upper hemal arch region and the lower blood vessel in eye fundus image when handling as template eye fundus image
The pixel value of bow region remains unchanged, by its in addition to optic disk region, upper hemal arch region and lower hemal arch region
He is all set to zero by the pixel value in region, the eye fundus image that obtains that treated.It then, then will treated eye fundus image conduct
Training sample data, which are input in the neural network model built in advance, to be trained.
It should also be noted that being constructed in advance in the eye fundus image compartmentalization deep learning method of the embodiment of the present disclosure
Good neural network model may include basic network and multiple sub-networks.Wherein, referring to Fig.2, basic network is as multiple sons
The shared network of network, output end are the input terminal of each sub-network.That is, the output of basic network is the defeated of each sub-network
Enter.It should also be noted that, in one possible implementation, the number of basic network is one, and the number of sub-network can
(such as: the disease quantity for needing to detect) to carry out flexible setting according to actual needs.Each sub-network is for detecting a seed type
Disease.
Refering to Fig. 6, in one possible implementation, basic network includes successively cascade multiple convolution combination layers,
It is mainly responsible for the feature extraction of image.Wherein, in the basic network of the embodiment of the present disclosure, convolution combination layer may include convolution
The mature internetwork connection modes such as layer, pond layer, batch normalization layer, residual error connection.Details are not described herein again.
Refering to Fig. 7, in one possible implementation, the depth of each sub-network is shallower, and each sub-network may include
Cascade convolutional layer and full articulamentum, the final differentiation after being mainly used for basic network feature extraction.Wherein, the classification of diagnostic horizon
It is different that number can detect (differentiation) based on different diseases.
It should be pointed out that in the eye fundus image compartmentalization deep learning method of the embodiment of the present disclosure, to entire nerve
The training of network model is broadly divided into two stages:
The training stage of basic network:
The initial parameter of basic network is obtained by the training of common data sets ImageNet essence, and micro- using eye fundus image progress
It adjusts.Such as: the more classification and Detections of eye fundus image that diabetic retinopathy can be used are finely adjusted.It is theoretical based on transfer learning,
The network parameter of basic network after fine tuning is used for the classification task of different eye fundus images, and the network parameter of basic network exists
After determination, it will not change again with the training of next sub-network.
The training of each sub-network:
Basic network and each sub-network are connected in series, and all-ones subnet network shares a basic network, and each subnet
Network (carries out treated eyeground figure to eye fundus image namely based on selected current desired template using corresponding eye fundus image
Picture) it is trained.Only the network parameter of sub-network is updated when training, finally determines the network parameter of each sub-network.This
Outside, when being trained to sub-network, each sub-network corresponds to respective image template combination, can be arranged by form assembly
It is obtained in table.
The eye fundus image compartmentalization deep learning method of the embodiment of the present disclosure as a result, by constructing image template, by model
Study is combined together with current detection demand, is avoided the interference of regions of non-interest during model learning, is enhanced mould
The robustness and generalization of type.Also, in the eye fundus image compartmentalization deep learning method of the embodiment of the present disclosure, pass through building
The neural network model that basic network and multiple sub-networks are combined into realizes and differentiates for the detection of each disease and customized pair
The network model answered can not only effectively improve the efficiency of model learning in this way, while can also realize eye fundus image regionality
The effect of explanation.Further, also a variety of to reach eye fundus image in such a way that multiple sub-networks share a basic network
The detection of disease differentiates and classification, realizes the function of multitask output, significantly reduces the complexity of network model and superfluous
Remaining avoids the excessively huge situation of network structure.
Correspondingly, based on any eye fundus image compartmentalization deep learning method in front, the disclosure additionally provides one
Kind eye fundus image compartmentalization deep learning device.Due to the work of the eye fundus image compartmentalization deep learning device of the embodiment of the present disclosure
The principle for making the eye fundus image compartmentalization deep learning method of principle and the embodiment of the present disclosure is same or similar, therefore repeats place
It repeats no more.
Refering to Fig. 8, the eye fundus image compartmentalization deep learning device 100 of the embodiment of the present disclosure may include image template structure
Model block 110, eye fundus image processing module 120 and network model training module 130.
Wherein, image template constructs module 110, is configured as obtaining eye fundus image, and based on the eye fundus image got
Construct corresponding image template；Wherein, the different zones in different image templates characterization eye fundus image.
Eye fundus image processing module 120 is configured as by choosing corresponding current desired template in image template, and is based on
Selected current desired template, handles eye fundus image, the eye fundus image that obtains that treated.
Network model training module 130, is configured as that eye fundus image is input to the nerve net constructed in advance by treated
Network model, is trained neural network model.
In one possible implementation, image template building module 110 may include detection and localization submodule and knot
Structure analyzes submodule (not shown).Wherein, detection and localization submodule is configured as using algorithm of target detection to eyeground figure
Optic disk region and macular region as in carry out detection and localization, obtain corresponding object detection results；Structural analysis submodule, quilt
It is configured to according to object detection results, structured analysis is carried out to eye fundus image, obtains corresponding image template.
In one possible implementation, structural analysis submodule may include that first movement unit and first generate list
First (not shown).Wherein, first movement unit is configured as in object detection results being that optic disk region and macular region are equal
Thus it is clear that when, by the contour line of eye fundus image respectively along optic disk region center and macular region central junction line direction of both ends
It is mobile, obtain corresponding first profile and the second profile.Wherein, the edge phase of first profile and the second profile with optic disk region
It cuts.First generation unit is configured as generating phase respectively with the intersection situation of eye fundus image according to first profile and the second profile
The image template answered.
In one possible implementation, structural analysis submodule can also include that the second mobile unit and second generate
Unit (not shown).Wherein, the second mobile unit is configured as in object detection results being optic disk region as it can be seen that macula lutea
When region is invisible, by the contour line of eye fundus image respectively along optic disk region center and eye fundus image center connecting line
Direction of both ends is mobile, obtains corresponding third profile and fourth contoured.Wherein, third profile and fourth contoured with optic disk region
Edge it is tangent.Second generation unit, be configured as according to third profile and fourth contoured respectively with the phase friendship of eye fundus image
Condition generates corresponding image template.
In one possible implementation, structural analysis submodule can also include that area division unit (does not show in figure
Out).Wherein, area division unit is configured as when object detection results are that macular region is visible, optic disk region is invisible,
According to the macular region oriented, eye fundus image is subjected to region division, generates corresponding image template.
In one possible implementation, eye fundus image processing module 120 includes that submodule is arranged (in figure not in pixel value
It shows).Wherein, submodule is arranged in pixel value, is configured as the region of eye fundus image corresponding to the current desired template that will be chosen
Pixel value remain the pixel value in original image region, and by the pixel value of eye fundus image corresponding to the image template that do not choose
It is set as zero.
It in one possible implementation, can also include that network model constructs module (not shown).Wherein, net
Network model construction module is configured as building neural network model.Wherein, constructed neural network model includes basic network
With multiple sub-networks；The output end of basic network is the input terminal of each sub-network；.Wherein, basic network includes successively cascade
Multiple convolution combination layers, each sub-network include cascade convolutional layer and full articulamentum.
Further, according to another aspect of the present disclosure, a kind of eye fundus image compartmentalization deep learning equipment is additionally provided.
Refering to Fig. 9, embodiment of the present disclosure eye fundus image compartmentalization deep learning equipment 200 includes processor 210 and is used at storage
Manage the memory 220 of 210 executable instruction of device.Wherein, realize that front is appointed when processor 210 is configured as executing executable instruction
Eye fundus image compartmentalization deep learning method described in one.
Herein, it should be noted that processor 210 can be general processor, such as: CPU (Central Processing
Unit/Processor, central processing unit), it can also be artificial intelligent processor.Wherein, artificial intelligence process device refers to using
In the processor (IPU) for executing artificial intelligence operation, such as: including GPU (Graphics Processing Unit, graphics process
Unit), NPU (Neural-Network Processing Unit, neural-network processing unit), DSP (Digital Signal
Process, digital signal processing unit), field programmable gate array (Field-Programmable Gate Array,
FPGA) one of chip or combination.The disclosure to the concrete type of artificial intelligence process device with no restriction.
It should also be noted that the number of processor 210 can be one or more.Meanwhile in the embodiment of the present disclosure
It can also include input unit 230 and output device 240 in eye fundus image processing equipment 200.Wherein, processor 210, storage
It can be connected, can also be connected by other means by bus between device 220, input unit 230 and output device 240, this
Place is without specifically limiting.
Memory 220 is used as a kind of computer readable storage medium, can be used for storing software program, journey can be performed in computer
Sequence and various modules, as: program or module corresponding to the eye fundus image compartmentalization deep learning method of the embodiment of the present disclosure.Place
The software program or module that reason device 210 is stored in memory 220 by operation, thereby executing eye fundus image processing equipment 200
Various function application and data processing.
Input unit 230 can be used for receiving the number or signal of input.Wherein, signal can for generate with equipment/terminal/
The related key signals of user setting and function control of server.Output device 240 may include that display screen etc. shows equipment.
According to another aspect of the present disclosure, a kind of non-volatile computer readable storage medium storing program for executing is additionally provided, is stored thereon
There are computer program instructions, any eye fundus image area in front is realized when computer program instructions are executed by processor 210
Domain deep learning method.
The presently disclosed embodiments is described above, above description is exemplary, and non-exclusive, and
It is not limited to disclosed each embodiment.Without departing from the scope and spirit of illustrated each embodiment, for this skill
Many modifications and changes are obvious for the those of ordinary skill in art field.The selection of term used herein, purport
In the principle, practical application or technological improvement to the technology in market for best explaining each embodiment, or lead this technology
Other those of ordinary skill in domain can understand each embodiment disclosed herein.
Claims (10)
1. a kind of eye fundus image compartmentalization deep learning method characterized by comprising
Eye fundus image is obtained, and corresponding image template is constructed based on the eye fundus image got；
Wherein, different described image moulds characterizes the different zones in the eye fundus image；
It is right by choosing corresponding current desired template in described image template, and based on the selected current desired template
The eye fundus image is handled, the eye fundus image that obtains that treated；
By treated, eye fundus image is input to the neural network model constructed in advance, instructs to the neural network model
Practice.
2. the method according to claim 1, wherein based on the corresponding figure of eye fundus image building got
As template, comprising:
Using algorithm of target detection in the eye fundus image optic disk region and macular region carry out detection and localization, obtain corresponding
Object detection results；
According to the object detection results, structured analysis is carried out to the eye fundus image, obtains corresponding described image template.
3. according to the method described in claim 2, it is characterized in that, according to the object detection results, to the eye fundus image
Carry out structured analysis, comprising:
When the object detection results are the optic disk region and the visible macular region, by the wheel of the eye fundus image
Profile is moved along the center in the optic disk region and the direction of both ends of the central junction line of the macular region respectively, is obtained corresponding
First profile and the second profile；
Wherein, the first profile and second profile are tangent with the edge in the optic disk region；
According to the first profile and second profile respectively with the intersection situation of the eye fundus image, generate corresponding described
Image template.
4. according to the method described in claim 2, it is characterized in that, according to the object detection results, to the eye fundus image
Carry out structured analysis, comprising:
It in the object detection results is the optic disk region as it can be seen that when the invisible macular region, by the eye fundus image
Contour line moved respectively along the center in the optic disk region and the direction of both ends of the connecting line at the center of the eye fundus image, obtain
To corresponding third profile and fourth contoured；
Wherein, the third profile and the fourth contoured are tangent with the edge in the optic disk region；
According to the third profile and the fourth contoured respectively with the intersection situation of the eye fundus image, generate corresponding described
Image template.
5. according to the method described in claim 2, it is characterized in that, the object detection results are based on, to the eye fundus image
Carry out structured analysis, comprising:
When the object detection results are that the macular region is visible, the optic disk region is invisible, according to what is oriented
The eye fundus image is carried out region division, generates corresponding described image template by the macular region.
6. method according to any one of claims 1 to 5, which is characterized in that based on the selected current desired mould
Plate handles the eye fundus image, comprising:
The pixel value in the region of eye fundus image corresponding to the current desired template by selection remains original image region
Pixel value, and zero is set by the pixel value of eye fundus image corresponding to the image template that do not choose.
7. method according to any one of claims 1 to 5, which is characterized in that the neural network model constructed in advance
Including basic network and multiple sub-networks；
Wherein, the output end of the basic network is the input terminal of each sub-network；
The basic network includes successively cascade multiple convolution combination layers, each sub-network include cascade convolutional layer and
Full articulamentum.
8. a kind of eye fundus image compartmentalization deep learning device characterized by comprising
Image template constructs module, is configured as obtaining eye fundus image, and corresponding based on the eye fundus image building got
Image template；Wherein, different described image templates characterizes the different zones in the eye fundus image；
Eye fundus image processing module is configured as by choosing corresponding current desired template in described image template, and is based on institute
The current desired template chosen, handles the eye fundus image, the eye fundus image that obtains that treated；
Network model training module, is configured as that eye fundus image is input to the neural network model constructed in advance by treated,
The neural network model is trained.
9. a kind of eye fundus image compartmentalization deep learning equipment characterized by comprising
Processor；
Memory for storage processor executable instruction；
Wherein, the processor is configured to being realized described in any one of claim 1 to 7 when executing the executable instruction
Method.
10. a kind of non-volatile computer readable storage medium storing program for executing, is stored thereon with computer program instructions, which is characterized in that institute
It states and realizes method described in any one of claim 1 to 7 when computer program instructions are executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910498239.9A CN110335254B (en) | 2019-06-10 | 2019-06-10 | Fundus image regionalization deep learning method, device and equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910498239.9A CN110335254B (en) | 2019-06-10 | 2019-06-10 | Fundus image regionalization deep learning method, device and equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110335254A true CN110335254A (en) | 2019-10-15 |
| CN110335254B CN110335254B (en) | 2021-07-27 |
Family
ID=68140886
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910498239.9A Active CN110335254B (en) | 2019-06-10 | 2019-06-10 | Fundus image regionalization deep learning method, device and equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110335254B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111292296A (en) * | 2020-01-20 | 2020-06-16 | 京东方科技集团股份有限公司 | Training set acquisition method and device based on eye recognition model |
| CN114219761A (en) * | 2021-11-15 | 2022-03-22 | 中山大学中山眼科中心 | Image processing method based on fundus images |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110137157A1 (en) * | 2009-12-08 | 2011-06-09 | Canon Kabushiki Kaisha | Image processing apparatus and image processing method |
| CN105243669A (en) * | 2015-10-15 | 2016-01-13 | 四川和生视界医药技术开发有限公司 | Method for automatically identifying and distinguishing eye fundus images |
| CN107209933A (en) * | 2014-08-25 | 2017-09-26 | 新加坡科技研究局 | For assessing retinal images and the method and system of information being obtained from retinal images |
| CN107256410A (en) * | 2017-05-26 | 2017-10-17 | 北京郁金香伙伴科技有限公司 | To the method and device of class mirror image image classification |
| CN108520522A (en) * | 2017-12-31 | 2018-09-11 | 南京航空航天大学 | Retinal fundus images dividing method based on the full convolutional neural networks of depth |
| CN108596895A (en) * | 2018-04-26 | 2018-09-28 | 上海鹰瞳医疗科技有限公司 | Eye fundus image detection method based on machine learning, apparatus and system |
| CN108717696A (en) * | 2018-05-16 | 2018-10-30 | 上海鹰瞳医疗科技有限公司 | Macula lutea image detection method and equipment |
| CN109199322A (en) * | 2018-08-31 | 2019-01-15 | 福州依影健康科技有限公司 | A kind of macula lutea detection method and a kind of storage equipment |
| CN109325942A (en) * | 2018-09-07 | 2019-02-12 | 电子科技大学 | Eye fundus image Structural Techniques based on full convolutional neural networks |
| CN109662686A (en) * | 2019-02-01 | 2019-04-23 | 北京致远慧图科技有限公司 | A kind of fundus flavimaculatus localization method, device, system and storage medium |
-
2019
- 2019-06-10 CN CN201910498239.9A patent/CN110335254B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110137157A1 (en) * | 2009-12-08 | 2011-06-09 | Canon Kabushiki Kaisha | Image processing apparatus and image processing method |
| CN107209933A (en) * | 2014-08-25 | 2017-09-26 | 新加坡科技研究局 | For assessing retinal images and the method and system of information being obtained from retinal images |
| CN105243669A (en) * | 2015-10-15 | 2016-01-13 | 四川和生视界医药技术开发有限公司 | Method for automatically identifying and distinguishing eye fundus images |
| CN107256410A (en) * | 2017-05-26 | 2017-10-17 | 北京郁金香伙伴科技有限公司 | To the method and device of class mirror image image classification |
| CN108520522A (en) * | 2017-12-31 | 2018-09-11 | 南京航空航天大学 | Retinal fundus images dividing method based on the full convolutional neural networks of depth |
| CN109598733A (en) * | 2017-12-31 | 2019-04-09 | 南京航空航天大学 | Retinal fundus images dividing method based on the full convolutional neural networks of depth |
| CN108596895A (en) * | 2018-04-26 | 2018-09-28 | 上海鹰瞳医疗科技有限公司 | Eye fundus image detection method based on machine learning, apparatus and system |
| CN108717696A (en) * | 2018-05-16 | 2018-10-30 | 上海鹰瞳医疗科技有限公司 | Macula lutea image detection method and equipment |
| CN109199322A (en) * | 2018-08-31 | 2019-01-15 | 福州依影健康科技有限公司 | A kind of macula lutea detection method and a kind of storage equipment |
| CN109325942A (en) * | 2018-09-07 | 2019-02-12 | 电子科技大学 | Eye fundus image Structural Techniques based on full convolutional neural networks |
| CN109662686A (en) * | 2019-02-01 | 2019-04-23 | 北京致远慧图科技有限公司 | A kind of fundus flavimaculatus localization method, device, system and storage medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111292296A (en) * | 2020-01-20 | 2020-06-16 | 京东方科技集团股份有限公司 | Training set acquisition method and device based on eye recognition model |
| CN111292296B (en) * | 2020-01-20 | 2024-06-18 | 京东方科技集团股份有限公司 | Training set acquisition method and device based on eye recognition model |
| CN114219761A (en) * | 2021-11-15 | 2022-03-22 | 中山大学中山眼科中心 | Image processing method based on fundus images |
| CN114219761B (en) * | 2021-11-15 | 2024-05-28 | 中山大学中山眼科中心 | Image processing method based on fundus image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110335254B (en) | 2021-07-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220415087A1 (en) | Method, Device, Electronic Equipment and Storage Medium for Positioning Macular Center in Fundus Images | |
| CN108717869B (en) | Auxiliary system for diagnosing diabetic retinal complications based on convolutional neural network | |
| CN110197493A (en) | Eye fundus image blood vessel segmentation method | |
| CN111656357A (en) | Artificial intelligence-based ophthalmic disease diagnosis modeling method, device and system | |
| CN109671076A (en) | Blood vessel segmentation method, apparatus, electronic equipment and storage medium | |
| CN108510473A (en) | The FCN retinal images blood vessel segmentations of convolution and channel weighting are separated in conjunction with depth | |
| CN108764286A (en) | The classifying identification method of characteristic point in a kind of blood-vessel image based on transfer learning | |
| CN110335254A (en) | Eye fundus image compartmentalization deep learning method, apparatus and equipment and storage medium | |
| CN109637660B (en) | Tongue diagnosis analysis method and system based on deep convolutional neural network | |
| WO2017028011A1 (en) | Method and device for processing blood pressure measurement data | |
| CN111242933B (en) | Retinal image artery and vein classification device, apparatus, and storage medium | |
| CN109190682A (en) | A kind of classification method and equipment of the brain exception based on 3D nuclear magnetic resonance image | |
| CN109816666A (en) | Symmetrical full convolutional neural networks model building method, eye fundus image blood vessel segmentation method, apparatus, computer equipment and storage medium | |
| CN110070531A (en) | For detecting the model training method of eyeground picture, the detection method and device of eyeground picture | |
| CN108694994A (en) | Noninvasive cardiac infarction disaggregated model construction method based on stack self-encoding encoder and support vector machines | |
| US11037296B1 (en) | Deriving target data from selected brain data | |
| CN108416371A (en) | A kind of diabetic retinopathy automatic testing method | |
| CN108720868A (en) | Blood flow imaging method, apparatus and computer readable storage medium | |
| CN110335276A (en) | Medical image segmentation model, method, storage medium and electronic equipment | |
| CN109583331A (en) | Human wrist arteries and veins mouth position precise positioning method based on deep learning | |
| CN106355574A (en) | Intra-abdominal adipose tissue segmentation method based on deep learning | |
| KR20220097859A (en) | Method and apparatus for distinguishing lesion | |
| CN109978796A (en) | Optical fundus blood vessel Picture Generation Method, device and storage medium | |
| CN110503636A (en) | Parameter regulation means, lesion prediction technique, parameter adjustment controls and electronic equipment | |
| CN113850804A (en) | Retina image generation system and method based on generation countermeasure network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |