CN116824203A - Glaucoma recognition device and recognition method based on neural network - Google Patents
Glaucoma recognition device and recognition method based on neural network Download PDFInfo
- Publication number
- CN116824203A CN116824203A CN202310297714.2A CN202310297714A CN116824203A CN 116824203 A CN116824203 A CN 116824203A CN 202310297714 A CN202310297714 A CN 202310297714A CN 116824203 A CN116824203 A CN 116824203A
- Authority
- CN
- China
- Prior art keywords
- image
- glaucoma
- disc
- optic
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 208000010412 Glaucoma Diseases 0.000 title claims abstract description 210
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 92
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000007781 pre-processing Methods 0.000 claims abstract description 60
- 230000011218 segmentation Effects 0.000 claims abstract description 26
- 238000000605 extraction Methods 0.000 claims abstract description 18
- 238000013135 deep learning Methods 0.000 claims abstract description 13
- 238000010801 machine learning Methods 0.000 claims abstract description 11
- 230000006870 function Effects 0.000 claims description 96
- 238000012549 training Methods 0.000 claims description 85
- 238000002372 labelling Methods 0.000 claims description 45
- 239000011159 matrix material Substances 0.000 claims description 5
- 238000003709 image segmentation Methods 0.000 description 21
- 238000010586 diagram Methods 0.000 description 10
- 238000012545 processing Methods 0.000 description 10
- 238000004422 calculation algorithm Methods 0.000 description 7
- 238000004590 computer program Methods 0.000 description 6
- 230000015654 memory Effects 0.000 description 6
- 238000010606 normalization Methods 0.000 description 5
- 230000003902 lesion Effects 0.000 description 4
- 238000005457 optimization Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000012216 screening Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 201000004569 Blindness Diseases 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 206010025421 Macule Diseases 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 210000003484 anatomy Anatomy 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 208000030533 eye disease Diseases 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000002427 irreversible effect Effects 0.000 description 1
- 238000007477 logistic regression Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000002207 retinal effect Effects 0.000 description 1
- 210000003786 sclera Anatomy 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000002759 z-score normalization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Medical Informatics (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Databases & Information Systems (AREA)
- Geometry (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Quality & Reliability (AREA)
- Radiology & Medical Imaging (AREA)
- Eye Examination Apparatus (AREA)
- Image Analysis (AREA)
Abstract
The disclosure describes a recognition device and a recognition method for glaucoma based on a neural network, wherein the recognition device comprises an input unit, a preprocessing unit, a segmentation unit, a feature extraction unit and a classification unit; the input unit is used for receiving a first image; the preprocessing unit is used for preprocessing the first image to obtain a first preprocessed image; the segmentation unit is used for inputting the first preprocessed image into the artificial neural network based on deep learning to generate a video disc area image and a video cup area image; the feature extraction unit acquires a plurality of glaucoma features based on the optic disc area image and the optic cup area image corresponding to the first preprocessing image; the classification unit is used for inputting the characteristic information comprising glaucoma characteristics corresponding to the first preprocessing image into a classifier based on machine learning for classification so as to obtain glaucoma classification results. According to the scheme, the accuracy of glaucoma identification can be improved.
Description
The application relates to a training method, a training device, an identification method and a divisional application of a patent application of an identification system, wherein the application date is 7/18/2020, the application number is 2020107013737, and the application name is glaucoma.
Technical Field
The disclosure relates specifically to a device and method for identifying glaucoma based on a neural network.
Background
Glaucoma has now become the global second blinding eye disease. Global primary glaucoma patients have exceeded millions of people, with more than one patient likely developing blindness to both eyes. Glaucoma may develop irreversible eye blindness if not diagnosed early, so early glaucoma screening is of great importance.
Among the techniques for glaucoma screening, fundus camera technology provides an economical and accurate way for early glaucoma screening. Medical studies have demonstrated that glaucoma can be found early by measuring the disk ratio of the disk (the ratio of disk radius to disk radius, simply referred to as cup-disk ratio) by imaging the fundus. With the development of artificial intelligence technology in recent years, cup-to-disc ratio can be calculated using artificial intelligence technology to achieve automatic glaucoma identification, such as the method of automatic glaucoma identification described in patent document (CN 109829877 a). In the above patent document, first, an image processing algorithm is used to perform preliminary positioning of the optic disc in the fundus image, and a depth convolution neural network is used to divide the optic disc region and the optic cup region from the preliminary positioning region, and then the cup-disc ratio is calculated and whether glaucoma exists in the fundus image is judged.
However, in the glaucoma recognition method, a complex image processing algorithm is required to perform preliminary positioning on the optic disc, and accuracy of optic disc positioning affects accuracy of subsequent optic cup or optic disc segmentation, thereby affecting accuracy of cup-to-disc ratio calculation. In addition, in other existing glaucoma identification methods, glaucoma is identified by using a cup-to-disc ratio, and other features of a optic cup or a optic disc extracted by using a convolutional neural network are not used for the identification of glaucoma, so that the accuracy of glaucoma identification is still to be improved.
Disclosure of Invention
In view of the above-described conventional circumstances, an object of the present disclosure is to provide a training method, training device, identification method, and identification system that can accurately identify glaucoma.
To this end, a first aspect of the present disclosure provides a training device based on glaucoma identification, comprising: an acquisition module that acquires a fundus image and a glaucoma classification tag thereof, a preprocessed fundus image obtained by preprocessing the fundus image, and a labeling image obtained by labeling the fundus image, the labeling image including a disc labeling image labeling a disc region and a cup labeling image labeling a cup region; an image segmentation network which is an artificial neural network based on deep learning and is trained by the pre-processing fundus image, the labeling image and a spatial weighting map to output a probability that each pixel point in the pre-processing fundus image belongs to a optic disc and a probability that each pixel point in the pre-processing fundus image belongs to a optic cup, and generates a optic disc region image and a optic cup region image based on the probability that each pixel point in the pre-processing fundus image belongs to the optic disc and the probability that each pixel point in the pre-processing fundus image belongs to the optic cup, wherein the spatial weighting map is generated by weighting each pixel point in the pre-processing fundus image based on a preset distance threshold and a optic disc distance, the optic disc distance is the shortest distance from each pixel point in the pre-processing fundus image to the optic disc region in the optic disc labeling image, and in the training of the artificial neural network, a first loss function is obtained by weighting a loss function of each pixel point in the pre-processing fundus image based on the spatial weighting map, and the artificial neural network is optimized based on the first loss function; a feature extraction module that obtains glaucoma features based on the optic disc region image and the optic cup region image; and a classifier trained based on machine learning by feature information including the glaucoma feature and the glaucoma classification tag to output probabilities belonging to glaucoma, in the training of the classifier, a second loss function is obtained, and the classifier is optimized based on the second loss function.
In the present disclosure, an image segmentation network is trained based on a pre-processed fundus image, a labeling image, and a spatial weighting map, a loss function of each pixel point in the pre-processed fundus image is weighted based on the spatial weighting map to obtain a first loss function, the image segmentation network is trained and optimized with the first loss function, a optic disc region image and a optic cup region image obtained with the image segmentation network are used to obtain glaucoma features, a classifier is trained based on feature information including the glaucoma features and a glaucoma classification tag, and the classifier is optimized based on a second loss function to obtain a classifier capable of recognizing glaucoma. In this case, the image segmentation network can be trained without preliminary positioning of the video disc, the problem of inaccurate segmentation of the video cup and the video disc due to inaccurate positioning of the video disc is improved, and the features extracted based on the image segmentation network can be utilized in combination with the image segmentation network and the classifier to identify glaucoma. Thereby, accuracy of glaucoma recognition can be improved.
Further, in the training device for glaucoma recognition according to the first aspect of the present disclosure, optionally, the glaucoma feature includes at least one of a vertical diameter of a optic disc in the optic disc region image and a vertical diameter of a optic cup in the optic cup region image, a horizontal diameter of a optic disc in the optic disc region image and a horizontal diameter of a optic cup in the optic cup region image, an area of a optic disc in the optic disc region image, and an area of a optic cup in the optic cup region image. In this case, a plurality of features can be extracted for identification of glaucoma based on the optic disc area image and the optic cup area image. Thereby, accuracy of glaucoma recognition can be improved.
In addition, in the training device for glaucoma recognition according to the first aspect of the present disclosure, optionally, by comparing the optic disc distance of each pixel point in the preprocessed fundus image with the preset distance threshold, the weight of the pixel point whose optic disc distance is smaller than the preset distance threshold is made to be a first preset value, and the weight of the pixel point whose optic disc distance is greater than or equal to the preset distance threshold is made to be a second preset value, where the first preset value is greater than the second preset value. In this case, the influence of the optic disc area can be increased and the image segmentation network can be trained without preliminary positioning of the optic disc. Therefore, the problem of inaccurate segmentation of the optic cup and the optic disc caused by inaccurate positioning of the optic disc can be solved, and the accuracy of glaucoma identification is further improved.
Further, in the training device for glaucoma identification according to the first aspect of the present disclosure, optionally, the training device is optimized based on a total loss function, which is determined according to the first loss function and the second loss function. In this case, the training device can be optimized with the total loss function. Thereby, accuracy of glaucoma recognition can be improved.
In addition, in the training device for glaucoma recognition according to the first aspect of the present disclosure, optionally, the feature information further includes at least one of age, sex, and medical history. In this case, the classifier may be trained based on different combinations of feature information. Thus, a classifier with better performance can be obtained.
A second aspect of the present disclosure provides a training method for glaucoma identification, comprising: acquiring a fundus image and a glaucoma classification label thereof, a preprocessed fundus image obtained by preprocessing the fundus image, and a labeling image obtained by labeling the fundus image, wherein the labeling image comprises a video disc labeling image labeled with a video disc area and a video cup labeling image labeled with a video cup area; training an artificial neural network based on deep learning based on the pre-processing fundus image, the labeling image and a spatial weighting map to output the probability that each pixel point in the pre-processing fundus image belongs to a video disc and the probability that each pixel point in the pre-processing fundus image belongs to a video cup, and generating a video disc area image and a video cup area image based on the probability that each pixel point in the pre-processing fundus image belongs to the video disc and the probability that each pixel point in the pre-processing fundus image belongs to the video cup, wherein the spatial weighting map is generated by weighting each pixel point in the pre-processing fundus image based on a preset distance threshold and a video disc distance, the video disc distance is the shortest distance from each pixel point in the pre-processing fundus image to the video disc area in the video disc labeling image, weighting a loss function of each pixel point in the pre-processing fundus image based on the spatial weighting map to obtain a first loss function in training of the artificial neural network, and optimizing the artificial neural network based on the first loss function; acquiring glaucoma features based on the optic disc area image and the optic cup area image; and training a machine learning based classifier based on the feature information including the glaucoma feature and the glaucoma classification tag to output a probability of belonging to glaucoma, in the training of the classifier, obtaining a second loss function, and optimizing the classifier based on the second loss function.
In the present disclosure, an artificial neural network is trained based on a pre-processed fundus image, a labeling image, and a spatial weighting map, a loss function of each pixel point in the pre-processed fundus image is weighted based on the spatial weighting map to obtain a first loss function, the artificial neural network is trained and optimized using the first loss function, a disc region image and a cup region image obtained by the artificial neural network are used to obtain glaucoma features, a classifier is trained based on feature information including the glaucoma features and a glaucoma classification tag, and the classifier is optimized based on a second loss function to obtain a classifier capable of recognizing glaucoma. Under the condition, the artificial neural network can be trained without preliminary positioning of the video disc, the problem of inaccurate segmentation of the video cup and the video disc caused by inaccurate positioning of the video disc is solved, and the characteristics extracted based on the artificial neural network can be utilized by combining the artificial neural network and the classifier to identify glaucoma. Thereby, accuracy of glaucoma recognition can be improved.
In addition, in the training method for glaucoma recognition according to the second aspect of the present disclosure, optionally, the glaucoma feature includes at least one of a vertical diameter of a optic disc in the optic disc region image and a vertical diameter of a optic cup in the optic cup region image, a horizontal diameter of a optic disc in the optic disc region image and a horizontal diameter of a optic cup in the optic cup region image, a surface area of a optic disc in the optic disc region image, and a surface area of a optic cup in the optic cup region image. In this case, a plurality of features can be extracted for identification of glaucoma based on the optic disc area image and the optic cup area image. Thereby, accuracy of glaucoma recognition can be improved.
In addition, in the training method for glaucoma recognition according to the second aspect of the present disclosure, optionally, by comparing the optic disc distance of each pixel point in the preprocessed fundus image with the preset distance threshold, the weight of the pixel point whose optic disc distance is smaller than the preset distance threshold is made to be a first preset value, and the weight of the pixel point whose optic disc distance is greater than or equal to the preset distance threshold is made to be a second preset value, where the first preset value is greater than the second preset value. In this case, the influence of the optic disc area can be increased and the artificial neural network can be trained without preliminary positioning of the optic disc. Therefore, the problem of inaccurate segmentation of the optic cup and the optic disc caused by inaccurate positioning of the optic disc can be solved, and the accuracy of glaucoma identification is further improved.
In addition, in the training method for glaucoma identification according to the second aspect of the present disclosure, optionally, the training method is optimized based on a total loss function, which is determined according to the first loss function and the second loss function. In this case, the total loss function can be used to optimize the artificial neural network and classifier training process. Thereby, accuracy of glaucoma recognition can be improved.
In addition, in the training method for glaucoma recognition according to the second aspect of the present disclosure, optionally, the feature information further includes at least one of age, sex, and medical history. In this case, the classifier may be trained based on different combinations of feature information. Thus, a classifier with better performance can be obtained.
A third aspect of the present disclosure provides an identification method of glaucoma identification, comprising: receiving a fundus image; preprocessing the fundus image to obtain a preprocessed fundus image; inputting the preprocessed fundus image into an artificial neural network obtained by the training method to obtain the probability that each pixel point in the preprocessed fundus image belongs to a video disc and the probability that each pixel point in the preprocessed fundus image belongs to a video cup, and generating a video disc area image and a video cup area image based on the probability that each pixel point in the preprocessed fundus image belongs to the video disc and the probability that each pixel point in the preprocessed fundus image belongs to the video cup; acquiring glaucoma features based on the optic disc area image and the optic cup area image; and inputting the characteristic information comprising the glaucoma characteristics into a classifier obtained by the training method to classify so as to obtain glaucoma classification results. In the present disclosure, a received fundus image is preprocessed to obtain a preprocessed fundus image, the preprocessed fundus image is segmented by using an artificial neural network obtained by the above-described training method to generate a optic disc region image and a cup region image, glaucoma features are acquired based on the optic disc region image and the cup region image, and glaucoma classification results are obtained by using a classifier obtained by the above-described training method and based on feature information including the glaucoma features. In this case, the preprocessed fundus image can be segmented without preliminary positioning of the optic disc, the problem of inaccurate segmentation of the optic cup and the optic disc due to inaccurate positioning of the optic disc can be solved, and the characteristics extracted based on the artificial neural network can be utilized in combination with the artificial neural network and the classifier to identify glaucoma. Thereby, accuracy of glaucoma recognition can be improved.
A fourth aspect of the present disclosure provides an identification system for glaucoma identification, comprising: an input unit for receiving a fundus image; a preprocessing unit for preprocessing the fundus image to obtain a preprocessed fundus image; a segmentation unit for inputting the preprocessed fundus image into an artificial neural network obtained by the training method to obtain probabilities that each pixel point in the preprocessed fundus image belongs to a video disc and a video cup, and generating a video disc area image and a video cup area image based on the probabilities that each pixel point in the preprocessed fundus image belongs to the video disc and the video cup; a feature extraction unit that acquires glaucoma features based on the optic disc region image and the optic cup region image; and a classification unit for inputting the characteristic information including the glaucoma characteristic into the classifier obtained by the training method to classify so as to obtain a glaucoma classification result. In the present disclosure, the preprocessing unit preprocesses the fundus image received by the input unit to obtain a preprocessed fundus image, the segmentation unit segments the preprocessed fundus image by using the artificial neural network obtained by the training method to generate a optic disc region image and a cup region image, the feature extraction unit obtains glaucoma features based on the optic disc region image and the cup region image, and the classification unit obtains glaucoma classification results by using the classifier obtained by the training method and based on feature information including the glaucoma features. In this case, the preprocessed fundus image can be segmented without preliminary positioning of the optic disc, the problem of inaccurate segmentation of the optic cup and the optic disc due to inaccurate positioning of the optic disc can be solved, and the characteristics extracted based on the artificial neural network can be utilized in combination with the artificial neural network and the classifier to identify glaucoma. Thereby, accuracy of glaucoma recognition can be improved.
According to the present disclosure, a training method, a training device, an identification method, and an identification system for glaucoma identification are provided that can accurately identify glaucoma.
Drawings
Embodiments of the present disclosure will now be explained in further detail by way of example only with reference to the accompanying drawings, in which:
fig. 1 is a schematic diagram of an electronic device of an identification system showing glaucoma identification in accordance with examples of the present disclosure.
Fig. 2 is a flow chart illustrating a training method of glaucoma identification related to examples of the present disclosure.
Fig. 3 is a schematic diagram showing labeling of fundus images to form a labeling image according to an example of the present disclosure.
Fig. 4 is a flowchart illustrating a training method of glaucoma identification according to an example of the present disclosure.
Fig. 5 is a block diagram illustrating a training apparatus for glaucoma identification in accordance with examples of the present disclosure.
Fig. 6 is a block diagram illustrating a training apparatus for glaucoma identification in accordance with examples of the present disclosure.
Fig. 7 is a flowchart illustrating an identification method of glaucoma identification according to an example of the present disclosure.
Fig. 8 is a block diagram illustrating an identification system for glaucoma identification in accordance with examples of the present disclosure.
Description of the reference numerals:
1 … electronic equipment, 110 … input equipment, 120 … server, 121 … processor, 122 … memory, 130 … output equipment, P210 … fundus image, P220 … optic disc labeling image, P230 … optic cup labeling image, 2 … training device, 210 … acquisition module, 220 … image segmentation network, 230 … feature extraction module, 240 … classifier, 250 … optimization module, 3 … identification system, 310 … input unit, 320 … preprocessing unit, 330 … segmentation unit, 340 … feature extraction unit, and 350 … classification unit.
Detailed Description
Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the following description, the same members are denoted by the same reference numerals, and overlapping description thereof is omitted. In addition, the drawings are schematic, and the ratio of the sizes of the components to each other, the shapes of the components, and the like may be different from actual ones.
Fig. 1 is a schematic diagram of an electronic device of an identification system showing glaucoma identification in accordance with examples of the present disclosure.
In some examples, referring to fig. 1, an identification system (which may also be simply referred to as an "identification system") of glaucoma identification, to which the present disclosure relates, may be implemented by means of an electronic device 1. As shown in fig. 1, the electronic device 1 may include an input device 110, a server 120, and an output device 130. The input device 110 is for receiving data. The server 120 may be configured to process data received by the input device 110 to obtain processing results. The output device 130 is used to display the processing result obtained by the server 120.
Input device 110 may include, but is not limited to, a keyboard, mouse, touch screen, scanner, camera, etc. The server 120 may include one or more processors 121 and one or more memories 122. The processor 121 may comprise a central processing unit, a graphics processing unit, and any other electronic components capable of processing data, capable of executing computer program instructions. Memory 122 may be used to store computer program instructions. Output devices 130 may include, but are not limited to, displays, printers, projectors, plotters, and the like.
In some examples, the identification system may be stored in the memory 122 in the form of computer program instructions. The processor 121 classifies fundus images received by the input device 110 to obtain glaucoma classification results by executing computer program instructions stored in the memory 122, and displays the obtained glaucoma classification results through the output device 130.
Hereinafter, a training method (may also be simply referred to as "training method") of glaucoma recognition of the present disclosure is described in detail with reference to fig. 2. Fig. 2 is a flow chart illustrating a training method of glaucoma identification related to examples of the present disclosure.
In the present embodiment, the training method for glaucoma recognition may include acquiring fundus images and glaucoma classification tags thereof, preprocessing and labeling the fundus images (step S210); training an artificial neural network based on deep learning (step S220); glaucoma features are acquired based on the optic disc region image and the optic cup region image (step S230) and a machine learning based classifier is trained (step S240). Under the condition, the artificial neural network can be trained without preliminary positioning of the video disc, the problem of inaccurate segmentation of the video cup and the video disc caused by inaccurate positioning of the video disc is solved, and the characteristics extracted based on the artificial neural network can be utilized by combining the artificial neural network and the classifier to identify glaucoma. Thereby, accuracy of glaucoma recognition can be improved.
Fig. 3 is a schematic diagram showing labeling of fundus images to form a labeling image according to an example of the present disclosure. Fig. 3 (a) shows a fundus image P210, fig. 3 (b) shows a disc labeling image P220, and fig. 3 (c) shows a cup labeling image P230.
In some examples, as described above, in step S210, a fundus image and its glaucoma classification tag may be acquired, and the fundus image may be preprocessed and labeled.
In some examples, in step S210, a fundus image may be acquired. The fundus image may be an image about the fundus taken by a fundus camera or other fundus camera apparatus. As an example of the fundus image, for example, fig. 3 (a) shows a fundus image P210 photographed by a fundus camera.
In some examples, the fundus image may be a color fundus image. The colorful fundus image can clearly present abundant fundus information such as optic disc, optic cup, macula, blood vessel and the like. The fundus image may be one of RGB mode, CMYK mode, lab mode, and gradation mode.
In some examples, the fundus image may include an area of the optic disc and the optic cup. In medicine, the optic disc and the optic cup have well-defined anatomies. That is, the optic disc is defined as the edge of the posterior aperture of the sclera, bounded by the inner rim of the scleral ring. The optic cup is defined as the range from the scleral plate to the retinal plane.
In some examples, the plurality of fundus images may constitute a training dataset. The training data set may include a training set and a test set. For example, 5 to 20 ten thousand fundus images, for example, from a partner hospital and from which patient information is removed, can be selected as a training set (training set), for example, 5000 to 20000 fundus images as a test set (testing set).
In addition, in some examples, in step S210, a glaucoma class label of the fundus image may be acquired. In some examples, the glaucoma classification tag may be classified by a plurality of clinical professionals on fundus images of the patient to form the glaucoma classification tag. Specifically, after fundus images from a partner hospital are collected and patient information is removed, fundus images may be classified according to consultation results of 3 or more clinical specialists to form glaucoma classification tags with respect to fundus images.
In some examples, the glaucoma classification tag may include both glaucoma and non-glaucoma class tags. By letting the training method according to the present disclosure learn both glaucoma and non-glaucoma classification, the training method is enabled with the ability to determine whether glaucoma lesions are present in the fundus image of the patient. In the present embodiment, the training method may be further learned to determine which glaucoma lesion the fundus image of the patient is and to classify the glaucoma lesion.
In addition, in some examples, in step S210, a pre-processed fundus image may be acquired. The pre-processing fundus image may be obtained by pre-processing the fundus image. In some examples, preprocessing may include cropping and normalizing the fundus image.
In some examples, the bottom-of-eye image may be cropped. In general, since the fundus image acquired in step S210 may have problems of different image formats, different sizes, and the like, the fundus image may be cut using a manual processing method or an image processing algorithm to be converted into an image of a fixed standard form. The fixed standard form means that the image includes a fundus region of interest and the proportion of the optic disc region in the fundus region is uniform, and the format of the image is uniform and the size is the same.
In some examples, the size of the fundus image after preprocessing may be unified as a fundus image of 512×512 or 1024×1024 pixels.
In some examples, the bottom-of-eye image may be normalized. In general, since the fundus image acquired in step S210 may have problems of uneven brightness and large contrast difference, normalization processing may be performed on the fundus image to overcome the variability of different fundus images. In some examples, the normalization may be a z-score normalization such that the processed fundus image has a mean value of 0 and a standard deviation of 1. In some examples, the normalization may be a maximum-minimum normalization, where the linear transformation of the original pixel values of the bottom-of-eye image maps the pixel values between 0 and 1.
In addition, in some examples, in step S210, noise reduction, graying processing, and the like may be performed on the bottom-eye image. In some examples, the bottom-of-eye image may be scaled, flipped, translated, etc. In this case, the amount of data for training the artificial neural network can be increased. Thus, the generalization ability of the artificial neural network can be improved. In addition, in some examples, in step S210, a labeling image may be acquired. The annotation image may be obtained by annotating the fundus image. In some examples, the annotation image may include a disc annotation image that annotates the disc region and a cup annotation image that annotates the cup region (see fig. 3).
In some examples, the optic disc annotation image and the optic cup annotation image in the annotation image may be combined into one annotation image as the actual value of the artificial neural network, or may be separated into two annotation images as the actual value of the artificial neural network.
In some examples, as described above, the annotation image may include a disc annotation image and a cup annotation image. For example, as shown in FIG. 3, in some examples, a disc region in the bottom-of-eye image P210 may be annotated to obtain a disc annotation image P220. The disc labeling image P220 may include a disc area A1 (see fig. 3 (b)). In some examples, the optic cup region in the bottom-eye image P210 may be annotated to obtain a cup annotated image P230. The visual cup label image P230 may contain a visual cup area A2 (see fig. 3 (c)).
In some examples, manual labeling of optic disc regions and optic cup regions in fundus images may be performed by an experienced physician. Thus, the accuracy of labeling the optic disc area and the cup area can be improved. In some examples, the bottom-of-eye image may be annotated using a dataset annotation tool, which may be, for example, a LabelImg tool (image annotation tool).
In some examples, preprocessing of the annotation image may also be included along with preprocessing of the fundus image. Therefore, the size of the marked image and the size of the preprocessed fundus image can be kept consistent all the time, and further the artificial neural network training is facilitated.
In step S220, the artificial neural network based on deep learning may be trained. In some examples, the preprocessed fundus image and the annotation image obtained in step S210 may be acquired and the artificial neural network based on deep learning trained in conjunction with a spatial weighting map. In other examples, the artificial neural network may also be trained directly with fundus images.
In some examples, the deep learning based artificial neural network may be trained based on the pre-processed fundus image, the annotation image, and the spatial weighting map. In some examples, the deep learning-based artificial neural network may be an artificial neural network for image semantic segmentation. For example, it may be an artificial neural network based on the Unet network or its modified type. The Unet network is an artificial neural network for image semantic segmentation and can comprise a feature extraction and upsampling part. Wherein the feature extraction section may include a plurality of encoding layers, for example, may include five encoding layers (a first encoding layer, a second encoding layer, a third encoding layer, a fourth encoding layer, and a fifth encoding layer, respectively), the first encoding layer having the preprocessed fundus image as an input. In some examples, the coding layer may comprise a series of convolutional layers, a batch normalization layer, an activation layer, and a max-pooling layer. The upsampling section may include a plurality of decoding layers, for example, may include four decoding layers (a first last decoding layer, a second last decoding layer, a third last decoding layer, a fourth last decoding layer, respectively) for outputting a probability that each pixel point in each of the pre-processed fundus images belongs to the optic disc and a probability that each pixel point belongs to the optic cup. In this case, the probability that each pixel point in the pre-processed fundus image belongs to the optic disc and the probability that each pixel point belongs to the optic cup can be obtained based on the artificial neural network.
In some examples, the spatial weighting map may be generated by weighting individual pixel points in the pre-processed fundus image based on a preset distance threshold and disc distance. In some examples, the spatial weighting map may be consistent with the size of the pre-processed fundus image, where the spatial weighting map in the present disclosure may be an image or matrix. If the spatial weighting map is an image, the spatial weighting map may be identical to the size of the pre-processed fundus image, and the pixel value in the spatial weighting map at the position corresponding to each pixel in the pre-processed fundus image is the weight of each pixel in the pre-processed fundus image. If the spatial weighting map is a matrix, the rows and columns of the matrix may be identical to the size of the pre-processed fundus image, for example if the size of the pre-processed fundus image is 512 x 512, the spatial weighting map is a 512 x 512 matrix. The element values in the spatial weighting map corresponding to the respective pixel points in the pre-processing fundus image are weights of the respective pixel points in the pre-processing fundus image.
In some examples, the preset distance threshold may be set according to the size of the optic disc region in the optic disc labeling image. In some examples, the preset distance threshold may be 0.1 times the diameter (vertical or horizontal) of the optic disc region in the optic disc labeling image. In some examples, the optic disc distance may be the shortest distance from each pixel point in the pre-processed fundus image to the optic disc region in the optic disc labeling image.
As described above, the spatial weighting map may be generated by weighting each pixel point in the pre-processed fundus image based on a preset distance threshold and disc distance. Specifically, in some examples, the optic disc distance of each pixel point in the pre-processed fundus image may be compared to a preset distance threshold. And enabling the weight of the pixel point with the video disc distance smaller than the preset distance threshold value to be a first preset value. And enabling the weight of the pixel points with the video disc distance being greater than or equal to the preset distance threshold value to be a second preset value. Wherein the first preset value is greater than the second preset value. In this case, the influence of the optic disc area can be increased and the artificial neural network can be trained without preliminary positioning of the optic disc. Therefore, the problem of inaccurate segmentation of the optic cup and the optic disc caused by inaccurate positioning of the optic disc can be solved, and the accuracy of glaucoma identification is further improved.
In some examples, the first preset value may be 0.8 to 1. For example, the first preset value may be 0.82, 0.85, 0.9, or 0.95, etc. The second preset value may be 0 to 0.2. For example, the second preset value may be 0.02, 0.05, 0.1, 0.15, or the like.
In some examples, in training of the artificial neural network, the loss functions of the individual pixels in the pre-processed fundus image may be weighted based on the spatial weighting map to obtain a first loss function, and the artificial neural network may be optimized based on the first loss function.
In general, the loss function can be used to calculate the loss, the merits of metric model predictions. Wherein the difference between the predicted value and the true value of the model based on the artificial neural network with respect to a single sample may be referred to as a loss. The smaller the loss, the better the model. A single sample in the present disclosure may refer to each pixel point in the pre-processed fundus image.
In some examples, the loss function may be optimized using Adam (adaptive moment estimation ) optimization algorithm. For example, the initial learning rate may be set to 0.001, and the learning rate may be reduced when the artificial neural network is continuously trained for a plurality of rounds to stop the decrease of the loss function. Therefore, the training efficiency of the artificial neural network can be improved, and the memory can be saved.
In some examples, the loss function may be a predefined loss function. In some examples, the loss function may be a cross entropy loss function, a Dice loss function, or the like. Wherein the cross entropy loss function is a function that measures the difference between the true distribution and the predicted distribution, and the Dice loss function is a set similarity measure function. Therefore, a proper loss function can be selected according to the requirements, and the training efficiency of the artificial neural network can be improved.
Specifically, taking the cross entropy loss function as an example, the loss function loss of each pixel point i,j The method comprises the following steps:
wherein c represents the category of predicting each pixel point in the preprocessed fundus image, and the predicted category comprises two categories of a visual cup and a visual disk. (i, j) represents coordinates of pixel points in the pre-processed fundus image.The value representing the pixel point with the coordinates (i, j) in the cup-marked image or the disc-marked image is used as the true value of the pixel point with the coordinates (i, j) in the preprocessing fundus image. />The predicted value of the pixel point with coordinates (i, j) in the pre-processing fundus image is represented. Alpha c Weights for each category.
In some examples, the optic disc distance of each pixel point in the pre-processed fundus image may be compared to a preset distance threshold. And enabling the weight of the pixel point with the video disc distance smaller than the preset distance threshold value to be a first preset value. The first preset value may be, for example, 1. And enabling the weight of the pixel points with the video disc distance being greater than or equal to the preset distance threshold value to be a second preset value. The second preset value may be, for example, 0. Then the individual pixel values or element values w in the spatial weighting map i,j (i.e., the weights of the loss functions of the individual pixel points in the pre-processed fundus image) are:
Wherein d i,j Is the shortest distance from pixel (i, j) to the disc area in the disc area image. D is a preset distance threshold. In some examples, the preset distance threshold may be set to 0.1 times the diameter (vertical or horizontal) of the optic disc region in the optic disc labeling image.
In some examples, the loss function of each pixel point may be spatially weighted using a spatial weighting map, so as to obtain a first loss function L1 of the artificial neural network:
L1=∑ i,j (w i,j *loss i,j ) … … (3)
Wherein w is i,j For preprocessing weights of pixel points with coordinates (i, j) in fundus images i,j Is a loss function for preprocessing a pixel point with coordinates (i, j) in the fundus image. Thus, the artificial neural network can be trained based on the first loss function to optimize the output of the artificial neural network.
In addition, in some examples, in step S220, a optic disc region image and a cup region image may be generated based on the probability that each pixel point in the pre-processed fundus image belongs to the optic disc and the probability that each pixel point belongs to the cup. Specifically, in some examples, the disc region image and the cup region image may be generated based on the probability that each pixel point in the pre-processed fundus image belongs to the optic disc and the probability that each pixel point belongs to the cup in the fourth last decoding layer of the aforementioned Unet network. For example, the probability of a video disc may be set to be white when 100%, the probability of a video disc may be set to be black when 0%, and the probability of a video disc may be set to be gray when 50%. In some examples, the optic disc region image and the optic cup region image may also be grayscale images.
In step S230, the optic disc region image and the optic cup region image obtained in step S220 may be acquired, and glaucoma features may be acquired based on the optic disc region image and the optic cup region image.
In some examples, glaucoma features may be obtained from a disc area image and a cup area image. In some examples, the glaucoma feature may include at least one of a vertical diameter of a optic disc in the optic disc region image and a vertical diameter of a optic cup in the optic cup region, a horizontal diameter of a optic disc in the optic disc region image and a horizontal diameter of a optic cup in the optic cup region, an area of a optic disc in the optic disc region image, and an area of a optic cup in the optic cup region. In this case, a plurality of features can be extracted for identification of glaucoma based on the optic disc area image and the optic cup area image. Thereby, accuracy of glaucoma recognition can be improved.
In step S240, a machine learning based classifier may be trained. In some examples, a machine learning based classifier may be trained based on the feature information and glaucoma classification tags to output probabilities of belonging to glaucoma. The characteristic information may include glaucoma characteristics. Glaucoma features may be obtained from step S230. The glaucoma classification tag may be obtained by step S210. Glaucoma class labels may include both glaucoma and non-glaucoma class labels.
In some examples, the characteristic information may also include at least one of age, gender, medical history. In some examples, the feature information may be combined to form a plurality of feature combinations, and the classifier trained based on the plurality of feature combinations to obtain classifier performance corresponding to each feature combination. For example, the feature combination may include the area of the optic disc in the optic disc region image, the area of the optic cup in the optic cup region, age, and the like. In this case, the classifier may be trained based on different combinations of feature information. Thus, a classifier with better performance can be obtained.
Additionally, in some examples, the machine learning based classifier may include, but is not limited to, a random forest algorithm, a support vector machine algorithm, and a logistic regression algorithm based classifier, or a deep learning based artificial neural network.
In some examples, in step S240, a probability of belonging to glaucoma may be output. In some examples, the probability of glaucoma may be a probability of whether a glaucoma lesion is present in the fundus image. In some examples, a probability threshold may be set. For example, glaucoma may be identified as glaucoma when the probability of glaucoma is greater than a probability threshold. Glaucoma is identified as non-glaucoma when the probability of glaucoma is less than or equal to the probability threshold.
Additionally, in some examples, in step S240, a second loss function may be obtained and the classifier optimized based on the second loss function. In some examples, the second loss function may be a cross entropy loss function. Thereby, the classifier can be optimized with the second loss function.
The second loss function L2 may be as shown in equation (4):
L2=-Y*log(Y pred )–(1-Y)*log(1-Y pred ) … … (4)
Where Y represents the true value of the fundus image (i.e., glaucoma class label). Ypred represents a predicted value of glaucoma of the fundus image output from the classifier.
Fig. 4 is a flowchart illustrating a training method of glaucoma identification according to an example of the present disclosure. In some examples, as shown in fig. 4, the training method may further include optimizing the trained artificial neural network and classifier based on the total loss function (step S250). In this case, the total loss function can be used to optimize the artificial neural network and classifier training process. Thereby, accuracy of glaucoma recognition can be improved.
In some examples, the total loss function in step S250 may be determined from the first loss function and the second loss function. In some examples, weights may be assigned to the first and second loss functions, respectively, and a sum of the weighted first and second loss functions may be taken as a total loss function.
In some examples, by performing steps S210 through S240, a trained artificial neural network and classifier may be obtained. In this case, the trained artificial neural network and classifier may be further optimized based on the total loss function.
For example, taking the first loss function L1 and the second loss function L2 as examples, the total loss function L may be obtained based on the first loss function L1 and the second loss function L2, as shown in the following formula (5):
l=βl1+γl2 … … (5)
Where β is the weight of the first loss function and γ is the weight of the second loss function. In this case, the trained artificial neural network and classifier can be optimized with the total loss function. Thereby, accuracy of glaucoma recognition can be improved. In some examples, the artificial neural network and classifier may also be trained directly based on the total loss function.
The training device (which may also be referred to simply as a "training device") for glaucoma identification of the present disclosure is described in detail below in conjunction with fig. 5. The training device disclosed by the disclosure is used for realizing the training method. Fig. 5 is a block diagram illustrating a training apparatus for glaucoma identification in accordance with examples of the present disclosure.
In some examples, the respective components of the training device 2 correspond to the respective steps of the training method, and may be functional modules that are required to be established for implementing the respective steps of the training method. As shown in fig. 5, the training apparatus 2 may include an acquisition module 210, an image segmentation network 220, a feature extraction module 230, and a classifier 240.
In some examples, the acquisition module 210 may be used to acquire fundus images and their glaucoma class labels, and may be used to pre-process and annotate fundus images. The image segmentation network 220 may be an artificial neural network based on deep learning. The image segmentation network 220 may obtain a disc region image and a cup region image. The feature extraction module 230 may be used to obtain glaucoma features based on the optic disc area image and the optic cup area image. Classifier 240 may be a machine learning based classifier. Classifier 240 may obtain probabilities of belonging to glaucoma. In this case, the image segmentation network can be trained without preliminary positioning of the video disc, the problem of inaccurate segmentation of the video cup and the video disc due to inaccurate positioning of the video disc is improved, and the features extracted based on the image segmentation network can be utilized in combination with the image segmentation network and the classifier to identify glaucoma. Thereby, accuracy of glaucoma recognition can be improved.
In some examples, the acquisition module 210 may acquire fundus images. The fundus image may be an image about the fundus taken by a fundus camera or other fundus camera apparatus. The fundus image may be one of RGB mode, CMYK mode, lab mode, gray mode, or the like. In some examples, the acquisition module 210 may acquire a glaucoma class label of the fundus image. Glaucoma class labels may be two classes of labels, glaucoma and non-glaucoma. In some examples, the acquisition module 210 may obtain a preprocessed fundus image by preprocessing the fundus image. In some examples, the acquisition module 210 may obtain a labeling image by labeling the fundus image. The annotation image may include a disc annotation image that annotates the disc region and a cup annotation image that annotates the cup region. The specific description may refer to step S210, and will not be repeated here.
In some examples, the image segmentation network 220 may be a deep learning based artificial neural network. In some examples, the image segmentation network 220 may be trained by preprocessing fundus images, annotation images, and spatial weighting maps. In some examples, the image segmentation network 220 may output a probability that each pixel point in the pre-processed fundus image belongs to the optic disc and a probability that each pixel point belongs to the optic cup. In some examples, image segmentation network 220 may generate a optic disc region image and a optic cup region image based on the probabilities that individual pixels in the pre-processed fundus image belong to the optic disc and the probabilities that individual pixels belong to the optic cup. The specific description may refer to step S220, and will not be repeated here.
In some examples, in training of the image segmentation network 220, the loss functions of individual pixels in the pre-processed fundus image may be weighted based on a spatial weighting map to obtain a first loss function. In some examples, the artificial neural network may be optimized based on the first loss function. In some examples, the spatial weighting map may be generated by weighting individual pixel points in the pre-processed fundus image based on a preset distance threshold and disc distance. Specifically, in some examples, the optic disc distance of each pixel point in the pre-processed fundus image may be compared to a preset distance threshold. And enabling the weight of the pixel point with the video disc distance smaller than the preset distance threshold value to be a first preset value. And enabling the weight of the pixel points with the video disc distance being greater than or equal to the preset distance threshold value to be a second preset value. Wherein the first preset value is greater than the second preset value. In this case, the influence of the optic disc area can be increased and the image segmentation network can be trained without preliminary positioning of the optic disc. Therefore, the problem of inaccurate segmentation of the optic cup and the optic disc caused by inaccurate positioning of the optic disc can be solved, and the accuracy of glaucoma identification is further improved. In some examples, the optic disc distance may be the shortest distance from each pixel point in the pre-processed fundus image to the optic disc region in the optic disc labeling image. The specific description may refer to step S220, and will not be repeated here.
In some examples, the feature extraction module 230 may obtain glaucoma features based on the optic disc region image and the optic cup region image. In some examples, the glaucoma feature may include at least one of a vertical diameter of a optic disc in the optic disc region image and a vertical diameter of a optic cup in the optic cup region, a horizontal diameter of a optic disc in the optic disc region image and a horizontal diameter of a optic cup in the optic cup region, an area of a optic disc in the optic disc region image, and an area of a optic cup in the optic cup region. In this case, a plurality of features can be extracted for identification of glaucoma based on the optic disc area image and the optic cup area image. Thereby, accuracy of glaucoma recognition can be improved. The specific description may refer to step S230, and will not be repeated here.
In some examples, classifier 240 may be a machine learning based classifier. In some examples, classifier 240 may be trained by the feature information and glaucoma classification tags to output probabilities of belonging to glaucoma. The characteristic information may include glaucoma characteristics. In some examples, in training of the classifier, a second loss function may be obtained and the classifier may be optimized based on the second loss function. In some examples, the characteristic information may also include at least one of age, gender, medical history. In this case, the classifier may be trained based on different combinations of feature information. Thus, a classifier with better performance can be obtained. The specific description may refer to step S240, and will not be repeated here.
Fig. 6 is a block diagram illustrating a training apparatus for glaucoma identification in accordance with examples of the present disclosure. As shown in fig. 6, in some examples, the training apparatus 2 further comprises an optimization module 250.
In some examples, the optimization module 250 may optimize the training device 2 based on the total loss function. In some examples, the total loss function may be determined from the first loss function and the second loss function. In some examples, weights may be assigned to the first and second loss functions, respectively, and a sum of the weighted first and second loss functions may be taken as a total loss function. In this case, the training device can be optimized with the total loss function. Thereby, accuracy of glaucoma recognition can be improved. The specific description may refer to step S250, and will not be repeated here.
The identification method (may also be simply referred to as "identification method") of glaucoma identification of the present disclosure is described in detail below in connection with fig. 7. Fig. 7 is a flowchart illustrating an identification method of glaucoma identification according to an example of the present disclosure.
In this embodiment, as shown in fig. 7, the recognition method may include receiving a fundus image (step S310), preprocessing the fundus image (step S320), dividing the preprocessed fundus image into a disc region image and a cup region image using an artificial neural network (step S330), acquiring glaucoma characteristics (step S340), and obtaining glaucoma classification results using a classifier (step S350). In this case, the preprocessed fundus image can be segmented without preliminary positioning of the optic disc, the problem of inaccurate segmentation of the optic cup and the optic disc due to inaccurate positioning of the optic disc can be solved, and the characteristics extracted based on the artificial neural network can be utilized in combination with the artificial neural network and the classifier to identify glaucoma. Thereby, accuracy of glaucoma recognition can be improved.
In step S310, a fundus image may be received. The fundus image may be an image about the fundus taken by a fundus camera or other fundus camera apparatus. In other examples, the fundus image may be a picture pre-stored in the user terminal. The user terminal may include, but is not limited to, a notebook computer, tablet computer, cell phone, desktop computer, or the like. In some examples, the fundus image may be a color fundus image. The fundus image may be one of RGB mode, CMYK mode, lab mode, gray mode, or the like.
In some examples, the identification method may be stored in a server in the form of a computer program, and the server may receive the fundus image by executing the computer program stored in the server.
In step S320, the fundus image received in step S310 may be acquired and preprocessed to obtain a preprocessed fundus image. In some examples, the preprocessing may include cropping, normalizing, etc., the bottom-of-eye image. Thereby, the fundus image can be converted into an image of a fixed standard form and the variability of different fundus images can be overcome. The fixed standard form means that the image includes a fundus region of interest and the proportion of the optic disc region in the fundus region is uniform, and the format of the image is uniform and the size is the same. In some examples, the size of the fundus image after preprocessing may be unified as a fundus image of 512×512 or 1024×1024 pixels. In some examples, the bottom-of-eye image may be noise reduced, grayed out, etc. The operations of cutting, normalizing, etc. in the identification method can be analogous to the relevant descriptions of the operations of cutting, normalizing, etc. in step S210 in the training method.
In step S330, the preprocessed fundus image generated in step S320 may be input into an artificial neural network obtained using the above-described training method, to obtain the probability that each pixel point in the preprocessed fundus image belongs to the optic disc and the probability that each pixel point belongs to the optic cup. In some examples, the optic disc region image and the optic cup region image may be generated based on a probability that each pixel point in the pre-processed fundus image belongs to the optic disc and a probability that each pixel point belongs to the optic cup. Thus, the preprocessed fundus image can be divided into a disc region image and a cup region image based on the artificial neural network. The disc area image and the cup area image in the identification method can be analogized with the related description of the disc area image and the cup area image in step S220 in the training method.
In step S340, glaucoma features may be acquired based on the optic disc area image and the optic cup area image obtained in step S330. In some examples, the glaucoma feature may include at least one of a vertical diameter of a optic disc in the optic disc region image and a vertical diameter of a optic cup in the optic cup region, a horizontal diameter of a optic disc in the optic disc region image and a horizontal diameter of a optic cup in the optic cup region, an area of a optic disc in the optic disc region image, and an area of a optic cup in the optic cup region. In this case, a plurality of features can be extracted for identification of glaucoma based on the optic disc area image and the optic cup area image. Thereby, accuracy of glaucoma recognition can be improved.
In step S350, the feature information may be input to the classifier obtained by the above-described training method to classify, so as to obtain a glaucoma classification result. In some examples, the characteristic information may include glaucoma characteristics obtained in step S340. In some examples, the characteristic information may also include at least one of age, gender, medical history. In some examples, the glaucoma classification result may be both glaucoma and non-glaucoma classification.
The identification system (which may also be referred to simply as "identification system") for glaucoma identification of the present disclosure is described in detail below in connection with fig. 8. The identification system disclosed by the disclosure is used for realizing the identification method. Fig. 8 is a block diagram illustrating an identification system for glaucoma identification in accordance with examples of the present disclosure.
In the present embodiment, as shown in fig. 8, the recognition system 3 for glaucoma recognition may include an input unit 310, a preprocessing unit 320, a segmentation unit 330, a feature extraction unit 340, and a classification unit 350.
In some examples, the input unit 310 may be used to receive fundus images. The preprocessing unit 320 may be used to preprocess the fundus image. The segmentation unit 330 may be used to segment the pre-processed fundus image into a optic disc region image and a cup region image using an artificial neural network. The feature extraction unit 340 may acquire glaucoma features. The classification unit 350 may be configured to obtain glaucoma classification results using a classifier. In this case, the preprocessed fundus image can be segmented without preliminary positioning of the optic disc, the problem of inaccurate segmentation of the optic cup and the optic disc due to inaccurate positioning of the optic disc can be solved, and the characteristics extracted based on the artificial neural network can be utilized in combination with the artificial neural network and the classifier to identify glaucoma. Thereby, accuracy of glaucoma recognition can be improved.
In some examples, the input unit 310 may receive fundus images. The fundus image may be an image about the fundus taken by a fundus camera or other fundus camera apparatus. In other examples, the fundus image may be a picture pre-stored in the user terminal. The user terminal may include, but is not limited to, a notebook computer, tablet computer, cell phone, desktop computer, or the like. In some examples, the fundus image may be a color fundus image. The fundus image may be one of RGB mode, CMYK mode, lab mode, gray mode, or the like. The specific description may refer to step S310, and will not be repeated here.
In some examples, the preprocessing unit 320 may acquire the fundus image received by the input unit 310 and preprocess the fundus image to obtain a preprocessed fundus image. In some examples, the preprocessing may include cropping, normalizing, etc., the bottom-of-eye image. Thereby, the fundus image can be converted into an image of a fixed standard form and the variability of different fundus images can be overcome. In some examples, the bottom-of-eye image may be noise reduced, grayed out, etc. The specific description may refer to step S320, and will not be repeated here.
In some examples, the segmentation unit 330 may input the preprocessed fundus image generated in the preprocessing unit 320 into the artificial neural network obtained using the training method described above, and may obtain the probability that each pixel point in the preprocessed fundus image belongs to the optic disc and the probability that each pixel point belongs to the optic cup. In some examples, the optic disc region image and the optic cup region image may be generated based on a probability that each pixel point in the pre-processed fundus image belongs to the optic disc and a probability that each pixel point belongs to the optic cup. Thus, the preprocessed fundus image can be divided into a disc region image and a cup region image based on the artificial neural network. The specific description may refer to step S330, and will not be repeated here.
In some examples, the feature extraction unit 340 may acquire glaucoma features based on the optic disc region image and the optic cup region image obtained by the segmentation unit 330. The specific description may refer to step S340, and will not be repeated here.
In some examples, the classification unit 350 may input the feature information into the classifier obtained using the above-described training method to classify to obtain glaucoma classification results. In some examples, the feature information may include glaucoma features obtained by the feature extraction unit 340. In some examples, the characteristic information further includes at least one of age, gender, medical history. In some examples, the glaucoma classification result may be both glaucoma and non-glaucoma classification.
While the disclosure has been described in detail in connection with the drawings and embodiments, it should be understood that the foregoing description is not intended to limit the disclosure in any way. Modifications and variations of the present disclosure may be made as desired by those skilled in the art without departing from the true spirit and scope of the disclosure, and such modifications and variations fall within the scope of the disclosure.
Claims (10)
1. The glaucoma recognition device based on the neural network is characterized by comprising an input unit, a preprocessing unit, a segmentation unit, a feature extraction unit and a classification unit; the input unit is used for receiving a first image, and the first image is a fundus image to be identified; the preprocessing unit is used for preprocessing the first image to obtain a first preprocessed image, wherein the first preprocessed image is a preprocessed fundus image corresponding to the first image; the segmentation unit is used for inputting the first preprocessing image into an artificial neural network based on deep learning to generate a video disc area image and a video cup area image, wherein the artificial neural network is obtained based on a second image, a labeling image corresponding to the second image and a spatial weighting graph generated based on a preset distance threshold and video disc distances of all pixel points in the second preprocessing image corresponding to the second image, the spatial weighting graph is used for weighting loss functions of all pixel points in the second preprocessing image to obtain a first loss function for optimizing the artificial neural network so as to train the artificial neural network without carrying out preliminary positioning on a video disc, and the second image is a fundus image for training the artificial neural network; the feature extraction unit obtains a plurality of glaucoma features based on a disc region image and a cup region image corresponding to the first preprocessed image; the classification unit is used for inputting the characteristic information comprising glaucoma characteristics corresponding to the first preprocessing image into a classifier based on machine learning to classify so as to obtain glaucoma classification results.
2. The identification device of claim 1, wherein the artificial neural network is based on a Unet network or the artificial neural network is based on a modified type Unet network.
3. The recognition device according to claim 1, wherein the labeling image corresponding to the second image includes a disc labeling image labeled with a disc region and a cup labeling image labeled with a cup region, the preset distance threshold is set according to the size of the disc region in the disc labeling image, and the disc distance is the shortest distance from each pixel point in the pre-processed fundus image to the disc region in the disc labeling image.
4. The identification device of claim 1, wherein the spatial weighting map is an image or a matrix and the spatial weighting map is consistent with the size of the second pre-processed image.
5. The identification device of claim 1, wherein the glaucoma feature comprises at least one of a vertical diameter of a optic disc in the optic disc region image and a vertical diameter of a optic cup in the optic cup region image, a horizontal diameter of a optic disc in the optic disc region image and a horizontal diameter of a optic cup in the optic cup region image, a surface area of a optic disc in the optic disc region image, and a surface area of a optic cup in the optic cup region image.
6. The apparatus according to claim 3, wherein in the generated spatial weighting map, a weight of a pixel whose optic disc distance is smaller than the preset distance threshold is set to a first preset value, and a weight of a pixel whose optic disc distance is greater than or equal to the preset distance threshold is set to a second preset value, and the first preset value is greater than the second preset value.
7. The apparatus according to claim 1, wherein the classifier is configured to output a probability of belonging to glaucoma and is obtained by training based on feature information including glaucoma features corresponding to the second preprocessed image obtained by extracting a disc region image and a cup region image generated by the artificial neural network for the second preprocessed image and a glaucoma classification tag.
8. The identification device of claim 7, wherein the glaucoma classification tag is a glaucoma and non-glaucoma class tag.
9. The identification device of claim 1, wherein in training of the classifier, a second loss function is obtained and the classifier is optimized based on the second loss function, the classifier and the artificial neural network being optimized based on a total loss function, the total loss function being determined from the first loss function and the second loss function.
10. A method for identifying glaucoma based on a neural network, comprising: receiving a first image, wherein the first image is a fundus image to be identified; preprocessing the first image to obtain a first preprocessed image, wherein the first preprocessed image is a preprocessed fundus image corresponding to the first image; inputting the first preprocessed image into a deep learning-based artificial neural network to generate a disc region image and a cup region image, wherein the artificial neural network is obtained based on a second image, a labeling image corresponding to the second image, and a spatial weighting map generated based on a preset distance threshold and a disc distance of each pixel point in the second preprocessed image corresponding to the second image, the spatial weighting map is used for weighting a loss function of each pixel point in the second preprocessed image to obtain a first loss function for optimizing the artificial neural network so as to train the artificial neural network without preliminary positioning of a disc, and the second image is a fundus image used for training the artificial neural network; acquiring a plurality of glaucoma features based on a disc area image and a cup area image corresponding to the first pre-processed image; and inputting the characteristic information comprising glaucoma characteristics corresponding to the first preprocessing image into a classifier based on machine learning for classification so as to obtain glaucoma classification results.
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911228725 | 2019-12-04 | ||
CN2019112287255 | 2019-12-04 | ||
CN202010701373.7A CN113011450B (en) | 2019-12-04 | 2020-07-18 | Training method, training device, recognition method and recognition system for glaucoma recognition |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010701373.7A Division CN113011450B (en) | 2019-12-04 | 2020-07-18 | Training method, training device, recognition method and recognition system for glaucoma recognition |
Publications (1)
Publication Number | Publication Date |
---|---|
CN116824203A true CN116824203A (en) | 2023-09-29 |
Family
ID=76383107
Family Applications (6)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202311798837.0A Pending CN117764957A (en) | 2019-12-04 | 2020-07-18 | Glaucoma image feature extraction training system based on artificial neural network |
CN202310321384.6A Pending CN116343008A (en) | 2019-12-04 | 2020-07-18 | Glaucoma recognition training method and training device based on multiple features |
CN202010702643.6A Active CN113012093B (en) | 2019-12-04 | 2020-07-18 | Training method and training system for glaucoma image feature extraction |
CN202310297714.2A Pending CN116824203A (en) | 2019-12-04 | 2020-07-18 | Glaucoma recognition device and recognition method based on neural network |
CN202010701373.7A Active CN113011450B (en) | 2019-12-04 | 2020-07-18 | Training method, training device, recognition method and recognition system for glaucoma recognition |
CN202311806582.8A Pending CN117788407A (en) | 2019-12-04 | 2020-07-18 | Training method for glaucoma image feature extraction based on artificial neural network |
Family Applications Before (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202311798837.0A Pending CN117764957A (en) | 2019-12-04 | 2020-07-18 | Glaucoma image feature extraction training system based on artificial neural network |
CN202310321384.6A Pending CN116343008A (en) | 2019-12-04 | 2020-07-18 | Glaucoma recognition training method and training device based on multiple features |
CN202010702643.6A Active CN113012093B (en) | 2019-12-04 | 2020-07-18 | Training method and training system for glaucoma image feature extraction |
Family Applications After (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010701373.7A Active CN113011450B (en) | 2019-12-04 | 2020-07-18 | Training method, training device, recognition method and recognition system for glaucoma recognition |
CN202311806582.8A Pending CN117788407A (en) | 2019-12-04 | 2020-07-18 | Training method for glaucoma image feature extraction based on artificial neural network |
Country Status (1)
Country | Link |
---|---|
CN (6) | CN117764957A (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113768460B (en) * | 2021-09-10 | 2023-11-14 | 北京鹰瞳科技发展股份有限公司 | Fundus image analysis system, fundus image analysis method and electronic equipment |
CN115578577A (en) * | 2021-10-11 | 2023-01-06 | 深圳硅基智能科技有限公司 | Eye ground image recognition device and method based on tight frame marks |
US11941809B1 (en) * | 2023-07-07 | 2024-03-26 | Healthscreen Inc. | Glaucoma detection and early diagnosis by combined machine learning based risk score generation and feature optimization |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101909141B (en) * | 2009-06-03 | 2012-06-27 | 晨星软件研发(深圳)有限公司 | Method and device for adjusting television image |
WO2011059409A1 (en) * | 2009-11-16 | 2011-05-19 | Jiang Liu | Obtaining data for automatic glaucoma screening, and screening and diagnostic techniques and systems using the data |
SG11201501348RA (en) * | 2012-08-24 | 2015-04-29 | Agency Science Tech & Res | Methods and systems for automatic location of optic structures in an image of an eye, and for automatic retina cup-to-disc ratio computation |
CN106651888B (en) * | 2016-09-28 | 2019-06-07 | 天津工业大学 | Colour eye fundus image optic cup dividing method based on multi-feature fusion |
CN106408564B (en) * | 2016-10-10 | 2019-04-02 | 北京新皓然软件技术有限责任公司 | A kind of method for processing fundus images based on deep learning, apparatus and system |
CN106725295A (en) * | 2016-11-29 | 2017-05-31 | 瑞达昇科技(大连)有限公司 | A kind of miniature check-up equipment, device and its application method |
WO2018215855A1 (en) * | 2017-05-23 | 2018-11-29 | Indian Institute Of Science | Automated fundus image processing techniques for glaucoma prescreening |
CN108122236B (en) * | 2017-12-18 | 2020-07-31 | 上海交通大学 | Iterative fundus image blood vessel segmentation method based on distance modulation loss |
CN108520522A (en) * | 2017-12-31 | 2018-09-11 | 南京航空航天大学 | Retinal fundus images dividing method based on the full convolutional neural networks of depth |
CN108921227B (en) * | 2018-07-11 | 2022-04-08 | 广东技术师范学院 | Glaucoma medical image classification method based on capsule theory |
CN109829877A (en) * | 2018-09-20 | 2019-05-31 | 中南大学 | A kind of retinal fundus images cup disc ratio automatic evaluation method |
CN109215039B (en) * | 2018-11-09 | 2022-02-01 | 浙江大学常州工业技术研究院 | Method for processing fundus picture based on neural network |
CN109658395B (en) * | 2018-12-06 | 2022-09-09 | 代黎明 | Optic disc tracking method and system and eye fundus collection device |
CN109658423B (en) * | 2018-12-07 | 2022-12-09 | 中南大学 | Automatic optic disk cup segmentation method for color fundus picture |
CN109919938B (en) * | 2019-03-25 | 2022-12-09 | 中南大学 | Method for obtaining optic disc segmentation atlas of glaucoma |
CN110110782A (en) * | 2019-04-30 | 2019-08-09 | 南京星程智能科技有限公司 | Retinal fundus images optic disk localization method based on deep learning |
CN110473188B (en) * | 2019-08-08 | 2022-03-11 | 福州大学 | Fundus image blood vessel segmentation method based on Frangi enhancement and attention mechanism UNet |
-
2020
- 2020-07-18 CN CN202311798837.0A patent/CN117764957A/en active Pending
- 2020-07-18 CN CN202310321384.6A patent/CN116343008A/en active Pending
- 2020-07-18 CN CN202010702643.6A patent/CN113012093B/en active Active
- 2020-07-18 CN CN202310297714.2A patent/CN116824203A/en active Pending
- 2020-07-18 CN CN202010701373.7A patent/CN113011450B/en active Active
- 2020-07-18 CN CN202311806582.8A patent/CN117788407A/en active Pending
Also Published As
Publication number | Publication date |
---|---|
CN113012093A (en) | 2021-06-22 |
CN113011450B (en) | 2023-04-07 |
CN116343008A (en) | 2023-06-27 |
CN117788407A (en) | 2024-03-29 |
CN117764957A (en) | 2024-03-26 |
CN113012093B (en) | 2023-12-12 |
CN113011450A (en) | 2021-06-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20220198657A1 (en) | Method and system for selecting embryos | |
US20210228071A1 (en) | System and method of otoscopy image analysis to diagnose ear pathology | |
KR102339915B1 (en) | Systems and methods for guiding a user to take a selfie | |
US10430946B1 (en) | Medical image segmentation and severity grading using neural network architectures with semi-supervised learning techniques | |
US11657602B2 (en) | Font identification from imagery | |
US20190191988A1 (en) | Screening method for automated detection of vision-degenerative diseases from color fundus images | |
US20180336683A1 (en) | Multi-Label Semantic Boundary Detection System | |
CN110211087B (en) | Sharable semiautomatic marking method for diabetic fundus lesions | |
CN113011450B (en) | Training method, training device, recognition method and recognition system for glaucoma recognition | |
CN114846507A (en) | Method and system for non-invasive gene detection using Artificial Intelligence (AI) models | |
JP2011013732A (en) | Information processing apparatus, information processing method, and program | |
JP7294695B2 (en) | Program, Information Recording Medium, Classification Apparatus, and Classification Method Based on Trained Model | |
CN113240655B (en) | Method, storage medium and device for automatically detecting type of fundus image | |
US11532036B2 (en) | Digital image ordering using object position and aesthetics | |
US20240249515A1 (en) | Method and apparatus for training image recognition model, device, and medium | |
CN114037868B (en) | Image recognition model generation method and device | |
RU2768797C1 (en) | Method and system for determining synthetically modified face images on video | |
US12079985B2 (en) | Diabetic retinopathy detection using machine learning | |
Parameshachari et al. | U-Net based Segmentation and Transfer Learning Based-Classification for Diabetic-Retinopathy Diagnosis | |
JP2012234497A (en) | Object identification device, object identification method, and program | |
CN114066850A (en) | Image binarization method based on classification framework | |
CN111242047A (en) | Image processing method and apparatus, electronic device, and computer-readable storage medium | |
Taha et al. | Deep Learning for Malaria Diagnosis: Leveraging Convolutional Neural Networks for Accurate Parasite Detection | |
Chauhan et al. | Deep Learning Solution for Diabetic Retinopathy | |
Basnet | Automated Quality Assessment of Printed Objects Using Subjective and Objective Methods Based on Imaging and Machine Learning Techniques |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |