CN108095683A - The method and apparatus of processing eye fundus image based on deep learning - Google Patents

The method and apparatus of processing eye fundus image based on deep learning Download PDF

Info

Publication number
CN108095683A
CN108095683A CN201610992731.8A CN201610992731A CN108095683A CN 108095683 A CN108095683 A CN 108095683A CN 201610992731 A CN201610992731 A CN 201610992731A CN 108095683 A CN108095683 A CN 108095683A
Authority
CN
China
Prior art keywords
eye fundus
image
fundus image
deep learning
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201610992731.8A
Other languages
Chinese (zh)
Inventor
白海龙
丁鹏
王璋
徐通
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Feather Care Cabbage Information Technology Co Ltd
Original Assignee
Beijing Feather Care Cabbage Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Feather Care Cabbage Information Technology Co Ltd filed Critical Beijing Feather Care Cabbage Information Technology Co Ltd
Priority to CN201610992731.8A priority Critical patent/CN108095683A/en
Publication of CN108095683A publication Critical patent/CN108095683A/en
Pending legal-status Critical Current

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/12Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for looking at the eye fundus, e.g. ophthalmoscopes
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/0016Operational features thereof
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/14Arrangements specially adapted for eye photography
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30041Eye; Retina; Ophthalmic

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Surgery (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Ophthalmology & Optometry (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biophysics (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The present invention provides a kind of method and apparatus of the accurately and reliably processing eye fundus image based on deep learning, to solve the problems of the prior art.The method of the processing eye fundus image based on deep learning of the present invention, including:Obtain multiple original sample eye fundus images and corresponding lesion class parameter;Multiple original sample eye fundus images are carried out with image pretreatment operation and image data enhancing operation, obtains multiple training sample eye fundus images;Depth network training is carried out to multiple training sample eye fundus images and corresponding lesion class parameter, obtains eyeground pathological changes grade identification model;Test eye fundus image is inputted into eyeground pathological changes grade identification model, obtains lesion grade recognition result.

Description

The method and apparatus of processing eye fundus image based on deep learning
Technical field
The present invention relates to technical field of computer vision, a kind of particularly processing eye fundus image based on deep learning Method and apparatus.
Background technology
Diabetic retinopathy is one of microvascular complication of diabetes.Chronic hyperglycemia damages retina capillary Blood vessel causes capillary vessel leak and blocking, in turn results in hypopsia and even blinds.In 2015, global diabetes were diseased Number is up to 4.15 hundred million, wherein, 93,000,000 patient's visions are influenced be subject to different degrees of diabetic retinopathy.And if When receive diagnose and treat, the vision impairment of Patients With Diabetic Retinopathy, which can usually slow down, even to reverse.However, by In the disease early symptom unobvious, Most patients could not actively go to a doctor and miss the best opportunity for the treatment of in time.
At present, clinically the detection of diabetic retinopathy there is still a need for by the doctor Jing Guo professional training passing through people Work checks the fundus photograph of patient to complete.The process more takes, and patient often arrives two days later the one of shooting fundus photograph Diagnostic result can be just obtained, easily causes treatment delay.In addition, the medical resource available for diabetic retinopathy detection More rare and distribution is uneven, and the whole world only has about 200,000 oculists, and remote deficiency carries out regular eye to all diabetics Section checks, and 3/4 diabetic lives in the serious deficient low middle income country of medical resource.With patient of diabetes The growth year by year of person's quantity also will increasingly increase the demand of medical resource.
Under these circumstances, research and development with Automatic analysis fundus photograph and can sieve diabetic retinopathy The demand for the computerized algorithm looked into becomes urgent.The existing automatic examination technical solution of diabetic retinopathy is mainly from shape Characteristics of lesion is detected in state, is broadly divided into three steps:
First, image preprocessing is carried out.Color is carried out to retinal images using histogram equalization and brightness is returned One change is handled, and makes exposure and contrast uniform, ensures the stability of processed image.
Secondly, eyeground anatomical structure segmentation is carried out.Split the major anatomical structure on retinal images, such as optic disk and view Film blood vessel excludes the interference of lesion detection during these structures handle next step.Optic disk segmentation generally uses Threshold segmentation or area Domain grows partitioning algorithm, the luminance difference of each pixel and neighborhood pixels in movement images, if difference value is less than some threshold value, These pixels are divided into the same area.The region of brightness maximum is identified as optic disk in image after segmentation.Blood vessel segmentation one As use edge detection or simple neural network algorithm.
Then, retinopathy detection is carried out.For retinopathy detection mainly include microaneurysm, blutpunkte and The detection of exudation.On the image, microaneurysm is the circular red point of small size, and blutpunkte is larger-size dotted, sheet or fire Flame shape red area is oozed out for the obvious yellow in the larger edge of brightness or white area, in different shape, size.Capilary Knurl and blutpunkte are close with retinal vessel in color, ooze out it is close with optic disk in color and brightness, therefore detect this The influence of rejecting retina anatomical structure is needed before a little lesions.After rejecting retina anatomical structure and influencing, microaneurysm and go out These red features of blood point mostly detect under green channel, and ooze out and then mostly detected under luminance channel.The lesion detected is special Sign can be used for carrying out diabetic retinopathy degree rough classification.
Prior art requires retinal structure in image based on the image gathered by specific retinal camera It is clearly apparent with characteristics of lesion, therefore it is only applicable to the eye fundus image after the mydriasis by the shooting of specific retina camera or into image quality Amount is higher to exempt from mydriasis image.The retina camera brand and model clinically actually used is different, and the image of acquisition is in color, exposure It is totally different in light, contrast and clarity, and most of is to exempt from mydriasis image.These conditions prevent prior art from Clinically there is good performance.
Existing scheme technically existing defects.First, existing scheme can not accurately identify it is neighbouring or coincide with dissection knot The characteristics of lesion of structure, such as exudation near optic disk will be identified that optic disk, and the blutpunkte of near vessels will be identified that Blood vessel and meet with reject, missing inspection.Second, existing scheme is only capable of identifying several specific lesions, but for international diabetic keratopathy view The non-Accretive Type characteristics of lesion of the severe such as vein beading sample of film lesion grade scale division changes, intraretinal microvascular abnormality, And the formation of the characteristics of lesion of Accretive Type such as new vessels, using existing scheme None- identified.And these lesions exactly refer to Show the main feature of high lesion degree and high blindness risk.Whether existing scheme is for diabetic retinopathy judgement It accurately, but then can not accurate evaluation for actual lesion degree.Third, prior art is computationally intensive, to each image It all needs to carry out complicated cumbersome processing and computing, the computer performance for using terminal is more demanding.
The content of the invention
In view of this, the present invention provides a kind of method and dress of the accurately and reliably processing eye fundus image based on deep learning It puts, to solve the problems of the prior art.
First aspect present invention proposes a kind of method of the processing eye fundus image based on deep learning, including:Step A:It obtains Take multiple original sample eye fundus images and corresponding lesion class parameter;Step B:To the multiple original sample eye fundus image Image pretreatment operation and image data enhancing operation are carried out, obtains multiple training sample eye fundus images;Step C:To multiple institutes It states training sample eye fundus image and corresponding lesion class parameter carries out depth network training, obtain the identification of eyeground pathological changes grade Model;Step D:Test eye fundus image is inputted into the eyeground pathological changes grade identification model, obtains lesion grade recognition result.
Optionally, described image pretreatment operation includes:Color averaging operation goes black surround operation, picture size unified Change at least one of operation.
Optionally, the step of carrying out image pretreatment operation to the multiple original sample eye fundus image includes:Use Gauss The original sample eye fundus image is subtracted the office by the local color average value of original sample eye fundus image described in Fuzzy Calculation Portion's color average value and then the neutral ash of compensation, obtain color handling averagely result images;Cut the color handling averagely Surrounding's black surround of result images obtains cutting handling result image;The cutting handling result image is subjected to equal proportion scaling To pre-set dimension, to obtain scaling handling result image.
Optionally, the data enhancement operations include:Data based on Arbitrary Rotation enhance, based on mirror image operation Data enhancing, based on scaled operation data enhancing at least one of.
Optionally, the step C includes:Using including X convolutional layer, Y full articulamentum, X+Y active coatings, Z pond The network structure for changing layer and a loss layer carries out depth network training to multiple training sample eye fundus images, obtains eye Bottom lesion grade identification model, wherein X >=10,1≤Y≤5,3≤Z≤5.
Second aspect of the present invention proposes a kind of device of the processing eye fundus image based on deep learning, including:Acquisition module, For obtaining multiple original sample eye fundus images and corresponding lesion class parameter;Operation module, for the multiple original Beginning sample eye fundus image carries out image pretreatment operation and image data enhancing operation, obtains multiple training sample eye fundus images; Training module, for carrying out depth network instruction to multiple training sample eye fundus images and corresponding lesion class parameter Practice, obtain eyeground pathological changes grade identification model;Test module inputs the eyeground pathological changes grade knowledge for that will test eye fundus image Other model obtains lesion grade recognition result.
Optionally, described image pretreatment operation includes:Color averaging operation goes black surround operation, picture size unified Change at least one of operation..
Optionally, the operation module is additionally operable to:The local color of the original sample eye fundus image is calculated with Gaussian Blur The original sample eye fundus image is subtracted the local color average value and then compensates neutral ash, obtains color by color average value Handling averagely result images;Surrounding's black surround of the color handling averagely result images is cut, obtains cutting handling result Image;The cutting handling result image is subjected to equal proportion and zooms to pre-set dimension, to obtain scaling handling result image.
Optionally, the data enhancement operations include:Data based on Arbitrary Rotation enhance, based on mirror image operation Data enhancing, based on scaled operation data enhancing at least one of.
Optionally, the training module is additionally operable to:Using including X convolutional layer, Y full articulamentum, X+Y active coatings, Z The network structure of a pond layer and a loss layer carries out depth network training to multiple training sample eye fundus images, obtains To eyeground pathological changes grade identification model, wherein X >=10,1≤Y≤5,3≤Z≤5.
Technical scheme handles eye fundus image to identify lesion based on neutral net and depth learning technology Grade at least has the advantages that:(1) accuracy of algorithm is greatly improved, reduces and fails to report.(2) disease can be identified Disease whether there is, and can further confirm that lesion degree.(3) algorithm simplicity is apparent, in test application after the completion of model training Stage is low to hardware requirement.
Description of the drawings
Attached drawing does not form inappropriate limitation of the present invention for more fully understanding the present invention.Wherein:
Fig. 1 is the key step of the method for the processing eye fundus image based on deep learning of embodiment according to the present invention Schematic diagram;
Fig. 2 is the main modular of the device of the processing eye fundus image based on deep learning of embodiment according to the present invention Schematic diagram;
Fig. 3 be according to the present invention specific embodiment based on deep learning processing eye fundus image method in depth Practise the schematic diagram of network.
Specific embodiment
It explains below in conjunction with attached drawing to the exemplary embodiment of the present invention, including embodiment of the present invention Various details should think them only exemplary to help understanding.Therefore, those of ordinary skill in the art should recognize Know, various changes and modifications can be made to embodiment described herein, without departing from scope and spirit of the present invention. Equally, for clarity and conciseness, the description to known function and structure is omitted in following description.
The shortcomings that having been elaborated in the prior art in background technology.The deep learning used in technical scheme The neutral net used in scheme has the characteristics of extraction object high-level characteristic.Since high-level characteristic information is low-level image feature information Linear processes conversion, therefore deep-neural-network can more be extracted compared to shallow-layer network can portray object to be described Substantive characteristics, so as to lift scheme effect.
Fig. 1 is the key step of the method for the processing eye fundus image based on deep learning of embodiment according to the present invention Schematic diagram.As shown in Figure 1, the method for the processing eye fundus image based on deep learning of the embodiment mainly includes following step Rapid A to step D.
Step A:Obtain multiple original sample eye fundus images and corresponding lesion class parameter.
In one particular embodiment of the present invention, we gather a large amount of eye fundus images especially diabetic retinopathy Become the eye fundus image of patient as original sample eye fundus image, and lesion class parameter is demarcated to them.
Step B:Multiple original sample eye fundus images are carried out with image pretreatment operation and image data enhancing operation, is obtained Multiple training sample eye fundus images.The purpose of image pretreatment operation is carried out primarily to improving sample quality, carries out image The purpose of data enhancement operations is primarily to increase sample size, the training pattern step be conducive to below of so working along both lines In obtain more preferably model.It should be noted that image data enhancing operation is performed after image pretreatment operation can be first carried out, Image pretreatment operation is performed after image data enhancing operation can also be first carried out.
Image pretreatment operation can include:Color averaging operation goes black surround operation, picture size to unitize in operation At least one.
In one particular embodiment of the present invention, image pretreatment operation is carried out to multiple original sample eye fundus images Step includes:(1) the local color average value of original sample eye fundus image is calculated with Gaussian Blur, by original sample eye fundus image It subtracts local color average value and then compensates neutral ash, obtain color handling averagely result images;(2) color equalization is cut Surrounding's black surround of handling result image obtains cutting handling result image;(3) handling result image will be cut and carries out equal proportion contracting It puts to pre-set dimension, to obtain scaling handling result image.
Data enhancement operations can include:Data enhancing based on Arbitrary Rotation, the data based on mirror image operation increase By force, at least one of data enhancing based on scaled operation.
Step C:Depth network training is carried out to multiple training sample eye fundus images and corresponding lesion class parameter, is obtained To eyeground pathological changes grade identification model.Eyeground pathological changes grade identification model is based on deep learning convolutional neural networks.Model is layer Shape structure, mainly including convolutional layer (convolutional layer), active coating (activation layer), pond layer (pooling layer), full articulamentum (fully connected layer) and loss layer (loss layer).
In one particular embodiment of the present invention, swash using including more than X convolutional layer, Y full articulamentums, X+Y The network structure of layer living, Z pond layer and a loss layer carries out depth network training to multiple training sample eye fundus images, Obtain eyeground pathological changes grade identification model.Detailed process is:Each convolutional layer and an active coating pairing are formed into a volume Each full articulamentum and an active coating pairing are formed a full articulamentum-active coating pair, by institute by lamination-active coating pair Some convolutional layer-active coatings are to being divided into several groups connected in sequence, and every group of convolutional layer-active coating to being inserted into a pond below Layer.All full articulamentum-active coatings pair are sequentially connected with behind the last one pond layer.Full articulamentum-active coating pair of most end For exporting the class parameter for judging lesion degree.Full articulamentum-active coating of most end to connecting a loss layer, the damage below Layer is lost for the deviation between the lesion grade of computation model prediction and true lesion grade.The deviation will successively pass back to net In network model, every layer of model parameter is adjusted according to the size of the deviation.Network model after adjustment handles instruction again Practice sample eye fundus image, the deviation being calculated is re-used for correction model parameter, so cycles, until deviation be down to it is acceptable Scope.So far, the training completion of deep learning network.
Convolution layer number determines the degree to be processed image higher-dimensionization and abstract, and convolution layer number is more, to figure The degree of image height dimensionization and abstract is deeper.Full connection layer number determines the complexity of the function representated by network model, Full connection layer number is more, and the function that model represents is more complicated.By multiple pilot scale study, preferably X >=10,1≤Y≤5 and 3 ≤Z≤5.In the embodiment for meeting the value condition, appropriate higher-dimension and abstract have been carried out to training sample eye fundus image, Meet the complexity that the problem of diabetic keratopathy lesion degree judges is carried out to the feature extracted by eye fundus image simultaneously.
It should be noted that depth network training instrument Caffe (http can be used:// Caffe.berkeleyvision.org/ model training) is carried out.Using when this instrument except use network structure file it Outside, also need to define solver files.Solver files give the method i.e. backpropagation of parameter of optimal model (training) Algorithm.Key parameter includes but not limited to:Basic learning rate (base learning rate) scope 0.0001 to 0.01;Study Momentum (momentum) scope 0.9 to 0.99;Weight penalty coefficient (weight_decay) scope 0.0001 to 0.001;Epoch Scope determined by the quantity of training set image, can neatly be determined according to actual conditions.
Step D:Test eye fundus image is inputted into eyeground pathological changes grade identification model, obtains lesion grade recognition result.
Fig. 2 is the main modular of the device of the processing eye fundus image based on deep learning of embodiment according to the present invention Schematic diagram.As shown in Fig. 2, the device of the processing eye fundus image based on deep learning of the embodiment can mainly include:It obtains Modulus block 100, operation module 200, training module 300 and test module 400.Acquisition module 100 is used to obtain multiple original samples This eye fundus image and corresponding lesion class parameter;Operation module 200 is used to carry out figure to multiple original sample eye fundus images As pretreatment operation and image data enhancing operation, multiple training sample eye fundus images are obtained;Training module 300 is used for multiple Training sample eye fundus image and corresponding lesion class parameter carry out depth network training, obtain eyeground pathological changes grade identification mould Type;Test module 400 obtains lesion grade identification knot for that will test eye fundus image input eyeground pathological changes grade identification model Fruit.
In the operation module 200 of the embodiment of the present invention, image pretreatment operation can include:Color averaging operation, Go black surround operation, picture size unitize operation at least one of.
In one particular embodiment of the present invention, operation module 200 is additionally operable to:Original sample eye is calculated with Gaussian Blur Original sample eye fundus image is subtracted local color average value and then compensates neutral ash, obtained by the local color average value of base map picture To color handling averagely result images;Surrounding's black surround of color handling averagely result images is cut, obtains cutting processing knot Fruit image;Handling result image progress equal proportion will be cut and zoom to pre-set dimension, to obtain scaling handling result image.
In the operation module 200 of the embodiment of the present invention, data enhancement operations can include:Based on Arbitrary Rotation At least one of data enhancing, the data enhancing based on mirror image operation, the data enhancing based on scaled operation.
In one particular embodiment of the present invention, training module 300 can be also used for:Using including X convolutional layer, Y The network structure of a full articulamentum, X+Y active coating, Z pond layer and a loss layer is to multiple training sample eyes Base map picture carries out depth network training, obtains eyeground pathological changes grade identification model, wherein X >=10,1≤Y≤5,3≤Z≤5.
Technical scheme handles eye fundus image to identify lesion based on neutral net and depth learning technology Grade at least has the advantages that:(1) accuracy of algorithm is greatly improved, reduces and fails to report.(2) disease can be identified Disease whether there is, and can further confirm that lesion degree.(3) algorithm simplicity is apparent, in test application after the completion of model training Stage is low to hardware requirement.In practical applications, day common computer is just sufficient for service requirement.It is the equal of the training stage It is once hard, later test phase time time easily, with conventional method it is time time hard compared with it is advantageous.
For those skilled in the art is made to more fully understand technical scheme, a specific embodiment is described below It further illustrates.
First, data acquisition and calibration
The eye fundus image of a large amount of acquisition eye fundus images especially Patients With Diabetic Retinopathy.Calibration process can join According to newest international diabetic retinopathy clinical rating scales, according to lesion severity and the new green blood of retina is whether there is Pipe, is divided into Pyatyi:Without lesion, slightly without Accretive Type, moderate is without Accretive Type, and severe is without Accretive Type and Accretive Type.This Pyatyi The corresponding lesion class parameter of lesion degree is 0,1,2,3 and 4.Lesion class parameter is bigger, and the lesion degree of instruction is got over Deep, blindness risk is higher.
2nd, data prediction and data enhancing
A. the local color average value of image is calculated with Gaussian Blur.With the local color average value correction map picture (original image subtracts average value) and local color average value is compensated as neutral ash, to eliminate the figure come by shooting condition different band The difference of exposure and color as between.This step pretreatment will impact the picture quality of eyeground vision periphery, therefore must go Image information in addition to 0.9 times of eyeground radius of view.
B. the black surround for removing image peripheral is cut out, rejects invalid data.
C. downscaled images length and width dimensions in proportion make long edge lengths reduce to 500 ± 100 pixels, and it is short with neutral ash to fill out lining While make it isometric with long side.This step processing retains the original scale of feature in image, makes the eyeground visual field after diminution in image still For circle, and on the basis of enough image details are retained, model calculation speed is improved.Pretreated image data with Its corresponding lesion grade mark information (the lesion degree label represented using discrete digital) is converted to database through being packaged Form, the input as deep learning network model.
D. data enhancing is implemented.The distribution of data of all categories is possible and uneven in data set.Random from database When extracting data formation batch, probability and data bulk of all categories that data of all categories are selected are inversely proportional, i.e., data volume is smaller The probability that is selected of classification it is higher, the probability that the bigger classification of data volume is selected is lower.After this otherness sampling operation, Data category distribution into model should be generally uniform, makes a small number of classifications that can also fully be learnt by model.Optionally, it is of all categories Sampling weight otherness can gradually decay with the propulsion of model training, make to finally enter the data category distribution of model with it is original It is distributed identical.For reduce over-fitting risk, increase data volume, after each batch is formed carry out data enhancing processing, i.e., according to Certain probability overturns every image (flip horizontal or flip vertical), rotate (0 ° to 360 °) and amplification (0% to 15%) operate, on the basis of key feature is retained, artificial expanding data amount.
3rd, training pattern
Using neutral net shown in Fig. 3 come training pattern.Picture by pretreatment is subjected to first convolutional layer (volume Product computing):Using the convolution kernel (length and width all be 7 pixels) of a fixed size, with certain step-length, (convolution kernel is every time to the right Mobile 2 pixels) scanning picture in its entirety.The position that a convolution kernel in office is swept to, by the parameter value inside convolution kernel and corresponding position The pixel value put does dot-product operation, obtains an output valve, this output valve finally is done nonlinear change via active coating again The final output valve of correspondence position is obtained afterwards.Such one secondary picture will become another " figure after a convolutional layer Piece ", height, width and the number of active lanes of this secondary new " picture " can be all previously designated.Convolution operation passes through volume The pixel value of original image is done linear, additive and then does nonlinear transformation by product core, and final purpose is to extract the height of a secondary picture Dimensional information.Such as when this task of object classification is done, several convolutional layers extraction low dimensional feature of beginning, bag Include edge, inflection point of object etc.;Several convolutional layers extraction below is exactly the high-dimensional feature for having more semantic feature, such as Global shape of object etc..
Then picture convolutional layer one obtained passes through next convolutional layer two, it is therefore an objective to extraction first convolution operation of ratio More high-dimensional information, and so on.
By the feature image obtained after several convolution operations (it can be appreciated that a pictures, but this when The number of active lanes of this characteristic pattern is not necessarily the only tri- passages of GRB, but can with pre-set multichannel, such as 128 logical Road, 256 passages etc.), by a pond layer.Pond layer does not include any parameter, and the purpose of this layer is by feature image Then dimensionality reduction finds local maximum response.For example one 256 can be multiplied by the pictures of 256 pixels and become 128 by pondization operation It is multiplied by 128 pixels.Such as four pixel values inside the grid of a 2x2 pixel are 1-10-3-4 respectively, then pondization operates By 10, this pixel value takes out, as output.The translation invariance of object in the picture that pondization operation utilizes, that is if Have specific objective in picture, and though then specific objective appear in picture where, the purpose of pondization operation is specific It finds response position of the target in feature image.
The characteristic pattern that pond layer is obtained sequentially is put into new convolutional layer again, and output result is to be grasped by new convolution The characteristic pattern of work.
Then, the characteristic pattern that previous step exports is put into a pond layer, gives characteristic pattern dimensionality reduction.
After dimensionality reduction several times, the feature image of output is put into a full articulamentum.Full attended operation is grasped with convolution Make similar, be all to be weighted after summation operation to do nonlinear transformation.The difference is that convolution operation is the office of network neural member Portion connects, and attended operation is that each neuron between two layers has connection relation entirely.The purpose of full attended operation be High-dimensional information is further extracted in characteristic pattern entire scope.The output of full articulamentum is not a feature image, but one Feature vector.
Similarly, the upper output connected entirely is passed sequentially through into addition several full connections again.
Finally, the numerical value of the representative lesion degree the last one full articulamentum neuron exported is calculated through loss layer, is asked It must predict the deviation of lesion degree and actual lesion degree.The deviation declines dust by gradient and successively passes back to network model In, every layer of model parameter is adjusted according to the size of the deviation.Network model after adjustment handles training sample again Eye fundus image, the deviation being calculated are re-used for correction model parameter, so cycle, until deviation is down to tolerance interval. So far, the training completion of deep learning network.
4th, tested using model
The model that present invention training is completed can be loaded in the computing terminal with general computing capability and run, and hardware is matched somebody with somebody Put require it is relatively low.
The experimental results showed that the model that the training of the embodiment of the present invention is completed is used to judge the new samples eyeground of random sampling During the lesion degree grade of image, accuracy of judgement degree can be more than 90%, reach and manually demarcate consistent level.
Above-mentioned specific embodiment, does not form limiting the scope of the invention.Those skilled in the art should be bright It is white, depending on design requirement and other factors, various modifications, combination, sub-portfolio and replacement can occur.It is any Modifications, equivalent substitutions and improvements made within the spirit and principles in the present invention etc., should be included in the scope of the present invention Within.

Claims (10)

  1. A kind of 1. method of the processing eye fundus image based on deep learning, which is characterized in that including:
    Step A:Obtain multiple original sample eye fundus images and corresponding lesion class parameter;
    Step B:Image pretreatment operation and image data enhancing operation are carried out to the multiple original sample eye fundus image, is obtained Multiple training sample eye fundus images;
    Step C:Depth network training is carried out to multiple training sample eye fundus images and corresponding lesion class parameter, is obtained To eyeground pathological changes grade identification model;
    Step D:Test eye fundus image is inputted into the eyeground pathological changes grade identification model, obtains lesion grade recognition result.
  2. 2. the method for the processing eye fundus image according to claim 1 based on deep learning, which is characterized in that described image Pretreatment operation includes:Color averaging operation, go black surround operation, picture size unitize operation at least one of.
  3. 3. the method for the processing eye fundus image according to claim 1 based on deep learning, which is characterized in that described more The step of a original sample eye fundus image progress image pretreatment operation, includes:
    The local color average value of the original sample eye fundus image is calculated with Gaussian Blur, by the original sample eye fundus image It subtracts the local color average value and then compensates neutral ash, obtain color handling averagely result images;
    Surrounding's black surround of the color handling averagely result images is cut, obtains cutting handling result image;
    The cutting handling result image is subjected to equal proportion and zooms to pre-set dimension, to obtain scaling handling result image.
  4. 4. the method for the processing eye fundus image according to claim 1 based on deep learning, which is characterized in that the data Enhancing operation includes:Data enhancing based on Arbitrary Rotation, the data based on mirror image operation enhance, based on scaled At least one of data enhancing of operation.
  5. 5. the method for the processing eye fundus image according to claim 1 based on deep learning, which is characterized in that the step C includes:Using including X convolutional layer, Y full articulamentum, X+Y active coatings, the network of Z pond layer and a loss layer Structure carries out depth network training to multiple training sample eye fundus images, obtains eyeground pathological changes grade identification model, wherein X >=10,1≤Y≤5,3≤Z≤5.
  6. 6. a kind of device of the processing eye fundus image based on deep learning, which is characterized in that including:
    Acquisition module, for obtaining multiple original sample eye fundus images and corresponding lesion class parameter;
    Operation module, for carrying out image pretreatment operation and image data enhancing behaviour to the multiple original sample eye fundus image Make, obtain multiple training sample eye fundus images;
    Training module, for carrying out depth network to multiple training sample eye fundus images and corresponding lesion class parameter Training, obtains eyeground pathological changes grade identification model;
    Test module inputs the eyeground pathological changes grade identification model for that will test eye fundus image, obtains the identification of lesion grade As a result.
  7. 7. the device of the processing eye fundus image according to claim 6 based on deep learning, which is characterized in that described image Pretreatment operation includes:Color averaging operation, go black surround operation, picture size unitize operation at least one of.
  8. 8. the device of the processing eye fundus image according to claim 6 based on deep learning, which is characterized in that the operation Module is additionally operable to:
    The local color average value of the original sample eye fundus image is calculated with Gaussian Blur, by the original sample eye fundus image It subtracts the local color average value and then compensates neutral ash, obtain color handling averagely result images;
    Surrounding's black surround of the color handling averagely result images is cut, obtains cutting handling result image;
    The cutting handling result image is subjected to equal proportion and zooms to pre-set dimension, to obtain scaling handling result image.
  9. 9. the device of the processing eye fundus image according to claim 6 based on deep learning, which is characterized in that the data Enhancing operation includes:Data enhancing based on Arbitrary Rotation, the data based on mirror image operation enhance, based on scaled At least one of data enhancing of operation.
  10. 10. the device of the processing eye fundus image according to claim 6 based on deep learning, which is characterized in that the instruction Practice module to be additionally operable to:Using including X convolutional layer, Y full articulamentum, X+Y active coatings, Z pond layer and one lose The network structure of layer carries out depth network training to multiple training sample eye fundus images, obtains eyeground pathological changes grade identification mould Type, wherein X >=10,1≤Y≤5,3≤Z≤5.
CN201610992731.8A 2016-11-11 2016-11-11 The method and apparatus of processing eye fundus image based on deep learning Pending CN108095683A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610992731.8A CN108095683A (en) 2016-11-11 2016-11-11 The method and apparatus of processing eye fundus image based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610992731.8A CN108095683A (en) 2016-11-11 2016-11-11 The method and apparatus of processing eye fundus image based on deep learning

Publications (1)

Publication Number Publication Date
CN108095683A true CN108095683A (en) 2018-06-01

Family

ID=62203608

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610992731.8A Pending CN108095683A (en) 2016-11-11 2016-11-11 The method and apparatus of processing eye fundus image based on deep learning

Country Status (1)

Country Link
CN (1) CN108095683A (en)

Cited By (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108876775A (en) * 2018-06-12 2018-11-23 广州图灵人工智能技术有限公司 The rapid detection method of diabetic retinopathy
CN108876776A (en) * 2018-06-13 2018-11-23 东软集团股份有限公司 A kind of method of generating classification model, eye fundus image classification method and device
CN108921179A (en) * 2018-06-22 2018-11-30 电子科技大学 A kind of infant hemangioma diseased region color automatically extract and quantization method
CN109087306A (en) * 2018-06-28 2018-12-25 众安信息技术服务有限公司 Arteries iconic model training method, dividing method, device and electronic equipment
CN109271990A (en) * 2018-09-03 2019-01-25 北京邮电大学 A kind of semantic segmentation method and device for RGB-D image
CN109427060A (en) * 2018-10-30 2019-03-05 腾讯科技(深圳)有限公司 A kind of method, apparatus, terminal device and the medical system of image identification
CN109464120A (en) * 2018-10-31 2019-03-15 深圳市第二人民医院 A kind of screening for diabetic retinopathy method, apparatus and storage medium
CN109523535A (en) * 2018-11-15 2019-03-26 首都医科大学附属北京友谊医院 A kind of preprocess method of lesion image
CN109602391A (en) * 2019-01-04 2019-04-12 平安科技(深圳)有限公司 Automatic testing method, device and the computer readable storage medium of fundus hemorrhage point
CN109858429A (en) * 2019-01-28 2019-06-07 北京航空航天大学 A kind of identification of eye fundus image lesion degree and visualization system based on convolutional neural networks
CN109857663A (en) * 2019-01-26 2019-06-07 北京工业大学 Automatic test platform of the crucial word drive in conjunction with image similarity
CN109903297A (en) * 2019-03-08 2019-06-18 数坤(北京)网络科技有限公司 Coronary artery dividing method and system based on disaggregated model
CN109993757A (en) * 2019-04-17 2019-07-09 山东师范大学 A kind of retinal images lesion region automatic division method and system
CN110021052A (en) * 2019-04-11 2019-07-16 北京百度网讯科技有限公司 The method and apparatus for generating model for generating eye fundus image
CN110101361A (en) * 2019-04-23 2019-08-09 深圳市新产业眼科新技术有限公司 Based on big data on-line intelligence diagnostic platform and its operation method and storage medium
CN110211087A (en) * 2019-01-28 2019-09-06 南通大学 The semi-automatic diabetic eyeground pathological changes mask method that can share
CN110236483A (en) * 2019-06-17 2019-09-17 杭州电子科技大学 A method of the diabetic retinopathy detection based on depth residual error network
CN110327013A (en) * 2019-05-21 2019-10-15 北京至真互联网技术有限公司 Eye fundus image detection method, device and equipment and storage medium
CN110348477A (en) * 2019-06-04 2019-10-18 上海联影智能医疗科技有限公司 Medical image processing method, storage medium and computer equipment
CN110473198A (en) * 2019-08-21 2019-11-19 北京大恒普信医疗技术有限公司 Deep learning system and method
CN110490138A (en) * 2019-08-20 2019-11-22 北京大恒普信医疗技术有限公司 A kind of data processing method and device, storage medium, electronic equipment
CN110555845A (en) * 2019-09-27 2019-12-10 上海鹰瞳医疗科技有限公司 Fundus OCT image identification method and equipment
CN111080643A (en) * 2019-12-31 2020-04-28 上海鹰瞳医疗科技有限公司 Method and device for classifying diabetes and related diseases based on fundus images
CN111374672A (en) * 2018-12-29 2020-07-07 西安思博探声生物科技有限公司 Intelligent knee pad and knee joint injury early warning method
CN112184647A (en) * 2020-09-22 2021-01-05 清华大学深圳国际研究生院 Vascular lesion grading identification method for fundus image based on migration convolution network
CN112789686A (en) * 2018-10-02 2021-05-11 翰林大学产学合作团 Device and method for diagnosing stomach pathological changes by deep learning of stomach endoscope images
CN112927228A (en) * 2021-04-14 2021-06-08 依未科技(北京)有限公司 Image evaluation method and device, and model training method and device
CN112950737A (en) * 2021-03-17 2021-06-11 中国科学院苏州生物医学工程技术研究所 Fundus fluorescence radiography image generation method based on deep learning
WO2021174821A1 (en) * 2020-08-07 2021-09-10 平安科技(深圳)有限公司 Fundus color photo image grading method and apparatus, computer device, and storage medium
CN114612389A (en) * 2022-02-21 2022-06-10 浙江大学 Fundus image quality evaluation method and device based on multi-source multi-scale feature fusion
CN117197145A (en) * 2023-11-08 2023-12-08 天津医科大学眼科医院 Cataract lesion image recognition method and system based on deep learning

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120257164A1 (en) * 2011-04-07 2012-10-11 The Chinese University Of Hong Kong Method and device for retinal image analysis
CN104835150A (en) * 2015-04-23 2015-08-12 深圳大学 Learning-based eyeground blood vessel geometric key point image processing method and apparatus
CN104881683A (en) * 2015-05-26 2015-09-02 清华大学 Cataract eye fundus image classification method based on combined classifier and classification apparatus

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120257164A1 (en) * 2011-04-07 2012-10-11 The Chinese University Of Hong Kong Method and device for retinal image analysis
CN104835150A (en) * 2015-04-23 2015-08-12 深圳大学 Learning-based eyeground blood vessel geometric key point image processing method and apparatus
CN104881683A (en) * 2015-05-26 2015-09-02 清华大学 Cataract eye fundus image classification method based on combined classifier and classification apparatus

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
HARRY PRATT等: "Convolutional Neural Networks for Diabetic Retinopathy", 《PROCEDIA COMPUTER SCIENCE》 *

Cited By (46)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108876775A (en) * 2018-06-12 2018-11-23 广州图灵人工智能技术有限公司 The rapid detection method of diabetic retinopathy
CN108876775B (en) * 2018-06-12 2022-10-18 湖南大学 Method for rapidly detecting diabetic retinopathy
CN108876776A (en) * 2018-06-13 2018-11-23 东软集团股份有限公司 A kind of method of generating classification model, eye fundus image classification method and device
CN108921179A (en) * 2018-06-22 2018-11-30 电子科技大学 A kind of infant hemangioma diseased region color automatically extract and quantization method
CN108921179B (en) * 2018-06-22 2022-03-18 电子科技大学 Automatic extraction and quantification method for colors of lesion areas of infantile hemangioma
CN109087306A (en) * 2018-06-28 2018-12-25 众安信息技术服务有限公司 Arteries iconic model training method, dividing method, device and electronic equipment
CN109271990A (en) * 2018-09-03 2019-01-25 北京邮电大学 A kind of semantic segmentation method and device for RGB-D image
CN112789686A (en) * 2018-10-02 2021-05-11 翰林大学产学合作团 Device and method for diagnosing stomach pathological changes by deep learning of stomach endoscope images
US11410306B2 (en) 2018-10-30 2022-08-09 Tencent Technology (Shenzhen) Company Limited Method, apparatus, system, and storage medium for recognizing medical image
US11610310B2 (en) 2018-10-30 2023-03-21 Tencent Technology (Shenzhen) Company Limited Method, apparatus, system, and storage medium for recognizing medical image
CN109427060A (en) * 2018-10-30 2019-03-05 腾讯科技(深圳)有限公司 A kind of method, apparatus, terminal device and the medical system of image identification
CN109464120A (en) * 2018-10-31 2019-03-15 深圳市第二人民医院 A kind of screening for diabetic retinopathy method, apparatus and storage medium
CN109523535A (en) * 2018-11-15 2019-03-26 首都医科大学附属北京友谊医院 A kind of preprocess method of lesion image
CN109523535B (en) * 2018-11-15 2023-11-17 首都医科大学附属北京友谊医院 Pretreatment method of lesion image
CN111374672B (en) * 2018-12-29 2024-01-19 西安思博探声生物科技有限公司 Intelligent knee pad and knee joint injury early warning method
CN111374672A (en) * 2018-12-29 2020-07-07 西安思博探声生物科技有限公司 Intelligent knee pad and knee joint injury early warning method
CN109602391A (en) * 2019-01-04 2019-04-12 平安科技(深圳)有限公司 Automatic testing method, device and the computer readable storage medium of fundus hemorrhage point
CN109857663A (en) * 2019-01-26 2019-06-07 北京工业大学 Automatic test platform of the crucial word drive in conjunction with image similarity
CN110211087A (en) * 2019-01-28 2019-09-06 南通大学 The semi-automatic diabetic eyeground pathological changes mask method that can share
CN109858429A (en) * 2019-01-28 2019-06-07 北京航空航天大学 A kind of identification of eye fundus image lesion degree and visualization system based on convolutional neural networks
CN110211087B (en) * 2019-01-28 2023-06-30 南通大学 Sharable semiautomatic marking method for diabetic fundus lesions
CN109858429B (en) * 2019-01-28 2021-01-19 北京航空航天大学 Eye ground image lesion degree identification and visualization system based on convolutional neural network
CN109903297A (en) * 2019-03-08 2019-06-18 数坤(北京)网络科技有限公司 Coronary artery dividing method and system based on disaggregated model
CN110021052A (en) * 2019-04-11 2019-07-16 北京百度网讯科技有限公司 The method and apparatus for generating model for generating eye fundus image
CN109993757A (en) * 2019-04-17 2019-07-09 山东师范大学 A kind of retinal images lesion region automatic division method and system
CN110101361A (en) * 2019-04-23 2019-08-09 深圳市新产业眼科新技术有限公司 Based on big data on-line intelligence diagnostic platform and its operation method and storage medium
CN110327013A (en) * 2019-05-21 2019-10-15 北京至真互联网技术有限公司 Eye fundus image detection method, device and equipment and storage medium
CN110327013B (en) * 2019-05-21 2022-02-15 北京至真互联网技术有限公司 Fundus image detection method, device and equipment and storage medium
CN110348477A (en) * 2019-06-04 2019-10-18 上海联影智能医疗科技有限公司 Medical image processing method, storage medium and computer equipment
CN110236483A (en) * 2019-06-17 2019-09-17 杭州电子科技大学 A method of the diabetic retinopathy detection based on depth residual error network
CN110236483B (en) * 2019-06-17 2021-09-28 杭州电子科技大学 Method for detecting diabetic retinopathy based on depth residual error network
CN110490138A (en) * 2019-08-20 2019-11-22 北京大恒普信医疗技术有限公司 A kind of data processing method and device, storage medium, electronic equipment
CN110473198A (en) * 2019-08-21 2019-11-19 北京大恒普信医疗技术有限公司 Deep learning system and method
CN110555845A (en) * 2019-09-27 2019-12-10 上海鹰瞳医疗科技有限公司 Fundus OCT image identification method and equipment
CN111080643A (en) * 2019-12-31 2020-04-28 上海鹰瞳医疗科技有限公司 Method and device for classifying diabetes and related diseases based on fundus images
WO2021174821A1 (en) * 2020-08-07 2021-09-10 平安科技(深圳)有限公司 Fundus color photo image grading method and apparatus, computer device, and storage medium
CN112184647B (en) * 2020-09-22 2023-07-04 清华大学深圳国际研究生院 Method for classifying and identifying vascular lesions on fundus images based on migration convolution network
CN112184647A (en) * 2020-09-22 2021-01-05 清华大学深圳国际研究生院 Vascular lesion grading identification method for fundus image based on migration convolution network
CN112950737A (en) * 2021-03-17 2021-06-11 中国科学院苏州生物医学工程技术研究所 Fundus fluorescence radiography image generation method based on deep learning
CN112950737B (en) * 2021-03-17 2024-02-02 中国科学院苏州生物医学工程技术研究所 Fundus fluorescence contrast image generation method based on deep learning
CN112927228A (en) * 2021-04-14 2021-06-08 依未科技(北京)有限公司 Image evaluation method and device, and model training method and device
CN114612389B (en) * 2022-02-21 2022-09-06 浙江大学 Fundus image quality evaluation method and device based on multi-source multi-scale feature fusion
CN114612389A (en) * 2022-02-21 2022-06-10 浙江大学 Fundus image quality evaluation method and device based on multi-source multi-scale feature fusion
US11842490B2 (en) 2022-02-21 2023-12-12 Zhejiang University Fundus image quality evaluation method and device based on multi-source and multi-scale feature fusion
CN117197145A (en) * 2023-11-08 2023-12-08 天津医科大学眼科医院 Cataract lesion image recognition method and system based on deep learning
CN117197145B (en) * 2023-11-08 2024-02-13 天津医科大学眼科医院 Cataract lesion image recognition system based on deep learning

Similar Documents

Publication Publication Date Title
CN108095683A (en) The method and apparatus of processing eye fundus image based on deep learning
Sinthanayothin Image analysis for automatic diagnosis of diabetic retinopathy
CN108021916A (en) Deep learning diabetic retinopathy sorting technique based on notice mechanism
CN106920227A (en) Based on the Segmentation Method of Retinal Blood Vessels that deep learning is combined with conventional method
CN108764342B (en) Semantic segmentation method for optic discs and optic cups in fundus image
CN107180421A (en) A kind of eye fundus image lesion detection method and device
CN109544540A (en) A kind of diabetic retina picture quality detection method based on image analysis technology
CN112017185B (en) Focus segmentation method, device and storage medium
CN110516685A (en) Lenticular opacities degree detecting method based on convolutional neural networks
CN110335241A (en) The method to be scored automatically after enteroscopy INTESTINAL CLEANSING
US20230022921A1 (en) System and method for analyzing corneal lesion using anterior ocular segment image, and computer-readable recording medium
Lestari et al. Retinal blood vessel segmentation using Gaussian filter
CN106846293A (en) Image processing method and device
Adalarasan et al. Automatic detection of blood vessels in digital retinal images using soft computing technique
CN112957005A (en) Automatic identification and laser photocoagulation region recommendation algorithm for fundus contrast image non-perfusion region
Zhang et al. Hierarchical detection of red lesions in retinal images by multiscale correlation filtering
CN117372284B (en) Fundus image processing method and fundus image processing system
Choudhary et al. ANN glaucoma detection using cup-to-disk ratio and neuroretinal rim
CN106846301A (en) Retinal images sorting technique and device
AU2020100868A4 (en) Automatic diabetic retinopathy detection and estimation from multispectral images using machine learning algorithms
Chowdhury et al. Transfer learning approach for diabetic retinopathy detection using efficient network with 2 phase training
Mathias et al. Categorization of Diabetic Retinopathy and Identification of Characteristics to Assist Effective Diagnosis
Dhivya et al. Automated Identification of Diabetic Retinopathy Using Artificial Neutral Network
Patil et al. Screening and detection of diabetic retinopathy by using engineering concepts
Kumari et al. Automated process for retinal image segmentation and classification via deep learning based cnn model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20180601

WD01 Invention patent application deemed withdrawn after publication