KR20230029559A - A method and apparatus for Deep Learning-Based Determination of Retinal Nerve Fiber Layer Thickness in Color Fundus Photograph for Glaucoma Screening - Google Patents

Abstract

The present invention relates to a method and an apparatus for determining a retinal nerve fiber layer thickness from a color fundus image for a glaucoma test on the basis of deep learning. The method for determining a retinal nerve fiber layer thickness from a color fundus image for a glaucoma test on the basis of deep learning, according to one embodiment of the present invention, may comprise the steps of: (a) acquiring a color fundus image for a user; (b) segmenting the color fundus image into a plurality of sub-regions; and (c) calculating a retinal nerve fiber layer thickness corresponding to each of the plurality of sub-regions by applying each of the segmented plurality of sub-regions to a thickness estimation deep learning model.

Description

A method and apparatus for Deep Learning-Based Determination of Retinal Nerve Fiber Layer Thickness in Color Fundus Photograph for Glaucoma Screening}

The present invention relates to a deep learning-based retinal nerve fiber layer thickness determination method and apparatus in a color fundus image for glaucoma examination, and more particularly, to a deep learning-based retinal nerve fiber layer thickness determination method and apparatus for each sub-region of a color fundus image. It is about.

Glaucoma is a chronic and irreversible disease caused by high intraocular pressure. The increased pressure can damage the optic nerve and lead to blindness.

Optimal treatment for glaucoma currently does not exist, and because conventional treatment can impede the progression of functional impairment and visual impairment, glaucoma must be detected early to prevent major vision loss.

Because glaucoma is known to be age-related, its prevalence is expected to increase worldwide due to the aging population. The prevalence of glaucoma in particular is very serious in many developing countries, possibly because most cases go undiagnosed or suboptimally managed due to a lack of diagnostic and screening tools.

Therefore, the availability of low-cost imaging devices for quick and easy screening would be very useful for glaucoma treatment in a resource-poor environment, but research on this is insufficient.

[Patent Document 1] Korean Patent Publication No. 10-2020-0049182

The present invention has been created to solve the above problems, and an object thereof is to provide a deep learning-based retinal nerve fiber layer thickness determination method and apparatus in a color fundus image for glaucoma examination.

In addition, the present invention is a method for calculating the retinal nerve fiber layer thickness corresponding to each of the plurality of sub-regions by applying each of a plurality of sub-regions segmented from a color fundus image to a thickness estimation deep learning model. and to provide an apparatus for that purpose.

The objects of the present invention are not limited to the objects mentioned above, and other objects not mentioned will be clearly understood from the description below.

In order to achieve the above objects, a deep learning-based retinal nerve fiber layer thickness determination method in a color fundus image for glaucoma examination according to an embodiment of the present invention includes (a) a color fundus image for a user obtaining; (b) segmenting the color fundus image into a plurality of subregions; and (c) calculating a retinal nerve fiber layer thickness corresponding to each of the plurality of sub-regions by applying a thickness estimation deep learning model to each of the plurality of sub-regions. there is.

In an embodiment, the step (b) may include extracting an optic disc region from the color fundus image by performing preprocessing on the color fundus image; and dividing the optic nerve head region into the plurality of sub-regions.

In an embodiment, the deep learning-based retinal nerve fiber layer thickness determination method in the color fundus image for glaucoma examination includes, after the step (c), classifying whether or not the user has glaucoma according to the retinal nerve fiber layer thickness; can include more.

In embodiments, the plurality of subregions may include superior-temporal (ST), inferior (II) and inferior-temporal (IT) regions.

In an embodiment, an apparatus for determining a retinal nerve fiber layer thickness based on deep learning in a color fundus image for glaucoma examination includes an acquisition unit acquiring a color fundus image for a user; and dividing the color fundus image into a plurality of sub-regions, and applying each of the divided sub-regions to a deep learning model for estimating a thickness to obtain a retinal nerve fiber layer thickness corresponding to each of the plurality of sub-regions. ); may include a control unit that calculates.

In an embodiment, the controller may perform preprocessing on the color fundus image to extract an optic disc region from the color fundus image, and divide the optic disc region into the plurality of sub-regions.

In an embodiment, the control unit may classify whether or not the user has glaucoma according to the thickness of the retinal nerve fiber layer.

In embodiments, the plurality of subregions may include superior-temporal (ST), inferior (II) and inferior-temporal (IT) regions.

Specific details for achieving the above objects will become clear with reference to embodiments to be described later in detail in conjunction with the accompanying drawings.

However, the present invention is not limited to the embodiments disclosed below, and may be configured in a variety of different forms, so that the disclosure of the present invention is complete and those of ordinary skill in the art to which the present invention belongs ( It is provided hereafter to fully inform the "ordinary skilled person") of the scope of the invention.

According to an embodiment of the present invention, glaucoma examination can be performed more accurately by considering low cost and evaluation quality provided by OCT and fundus images.

The effects of the present invention are not limited to the above-mentioned effects, and the potential effects expected by the technical features of the present invention will be clearly understood from the description below.

1A is a diagram illustrating a deep learning-based retinal nerve fiber layer thickness determination process in a color fundus image for glaucoma examination according to an embodiment of the present invention.
1B is a diagram illustrating an example of the architecture of a thickness estimation deep learning model according to an embodiment of the present invention.
2 is a diagram showing a graph of actual and predicted RNFL thickness according to an embodiment of the present invention.
3A is a diagram illustrating labeling classification of RNFL thinning levels according to an embodiment of the present invention.
3B is a diagram illustrating labeling classification of RNFL thinning levels for a steady state according to an embodiment of the present invention.
3C is a diagram illustrating labeling classification of RNFL thinning levels for some damage conditions according to an embodiment of the present invention.
3D is a diagram illustrating labeling classification of RNFL thinning levels for a complete damage condition according to an embodiment of the present invention.
4A is a diagram illustrating RNFL classification for an NTG group according to an embodiment of the present invention.
4B is a diagram illustrating RNFL classification for a suspicious group according to an embodiment of the present invention.
4C is a diagram illustrating RNFL classification for a normal group according to an embodiment of the present invention.
5A is a diagram illustrating a global RNFL-based confusion matrix according to an embodiment of the present invention.
5B is a diagram illustrating upper and lower RNFL-based confusion matrices according to an embodiment of the present invention.
5C is a diagram illustrating sensitivity and specificity screening results based on a combination of two subregions according to an embodiment of the present invention.
6 is a diagram illustrating a deep learning-based retinal nerve fiber layer thickness determination method in a color fundus image for glaucoma examination according to an embodiment of the present invention.
7 is a diagram showing a functional configuration of an apparatus for determining the thickness of a retinal nerve fiber layer based on deep learning in a color fundus image for glaucoma examination according to an embodiment of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail.

Various features of the invention disclosed in the claims may be better understood in consideration of the drawings and detailed description. Devices, methods, manufacturing methods, and various embodiments disclosed in the specification are provided for illustrative purposes. The disclosed structural and functional features are intended to enable a person skilled in the art to specifically implement various embodiments, and are not intended to limit the scope of the invention. The disclosed terms and phrases are intended to provide an easy-to-understand description of the various features of the disclosed invention, and are not intended to limit the scope of the invention.

In describing the present invention, if it is determined that a detailed description of related known technologies may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted.

Hereinafter, a deep learning-based retinal nerve fiber layer thickness determination method and apparatus in a color fundus image for glaucoma examination according to an embodiment of the present invention will be described.

1A is a diagram illustrating a deep learning-based retinal nerve fiber layer thickness determination process in a color fundus image for glaucoma examination according to an embodiment of the present invention. 1B is a diagram illustrating an example of the architecture of a thickness estimation deep learning model according to an embodiment of the present invention.

Referring to FIGS. 1A and 1B , according to the present invention, a thickness estimation deep learning model for predicting RNFL thickness around optic disc regions in fundus photographs for glaucoma examination may be used.

For example, the thickness estimation deep learning model is based on CNN (Convolutional Neural Network), and retinal nerve fiber layer (RNFL) thickness measured by fundus images and OCT can be used for model learning and validation.

A model trained using data sets obtained from patients with normal tension glaucoma (NTG) was able to estimate RNFL thickness along 12 optic disc regions from fundus photographs alone.

Screening based on regional RNFL estimation results in higher sensitivity compared to using global average RNFL thickness, using intuitive thickness labels to identify localized damage to the optic nerve head compared to standard databases can provide.

This may be because the local defects of the RNFL are not well reflected in the global average thickness.

According to the present invention, the initial stage of glaucoma can be diagnosed using local RNFL defect estimation, and it can also be applied to tracking the progression of glaucoma.

In addition, according to the present invention, the point-of-care tool can be used for glaucoma examination in a low-resource environment.

Screening tests based on the global mean of RNFL are poor at detecting early-stage glaucoma, whereas, as in the present invention, when using local RNFL thinning, screening sensitivity can be high.

Therefore, quantitative regional information can be used as a reference evaluation to develop highly accurate and sensitive screening protocols for early detection of glaucoma.

In one embodiment, data may be obtained from patients with normal tension glaucoma (NTG) for a screening protocol for early-stage glaucoma.

For example, NTG is a subcategory of primary open angled glaucoma (POAG) with an intraocular pressure (IOP) within the normal range.

NTG can show symptoms similar to other types of glaucoma, such as glaucomatous optic neuropathy and corresponding visual field (VF) defects.

In one embodiment, we can focus on the localized RNFL defect commonly observed in both high IOP POAG and NTG.

Therefore, a comprehensive glaucoma screening protocol using RNFL thickness can be tested using the data set of NTG patients.

According to the present invention, a thickness estimation deep learning model that infers local RNFL thickness values from color fundus images may be used.

The model according to the present invention can infer regional RNFL thickness from each optic disc image segmented along 12 sub-regions.

The predicted RNFL thickness is classified into three levels of thinning compared to the standard reference and can be used for glaucoma detection.

In one embodiment, data may be collected for DL model construction and validation from 303 patients diagnosed with normal tension glaucoma (NTG).

For example, an enrolled NTG patient may have a persistent peak IOP of 21 mmHg or less without treatment for glaucoma, a normal opening angle, typical glaucomatous optic nerve and visual field changes, and no ocular or systemic disorders causing optic nerve damage.

A total of 453 eye examinations consisting of color fundus images and spectral domain OCT scans can be acquired.

The collected color fundus images may be pre-processed.

For example, a 3216x2136 sized color fundus image can be processed in 4 steps to distinguish the optic disc region.

First, a color fundus image can be converted to a gray scale image.

Second, Gaussian filtering can be performed on the converted grayscale image with a kernel size (eg, 65x65).

Thirdly, since the center of the optic disc is the brightest in the fundus picture, the brightest point can be identified by scanning the maximum intensity pixels in the filtered image.

Fourth, an optic nerve head region of a certain size (eg, 320x320) pixels with a specific shape (eg, circular shape) centered on the center of the optic nerve may be extracted.

In one embodiment, the center of the optic nerve disc can be identified in the extracted color fundus image and classified into an image data set free of artifacts such as whitening, blur, twilight, camera artifact, dust, and darkness.

OCT measurements can be obtained from reports generated by analysis software.

A total of 940 pairs of color fundus images and OCT scans could be collected within 180 days (mean 31.4 days, standard deviation 62.9 days).

The fundus image was sectioned along 30° to create a scalloped optic disc image in the direction of each subregion and matched the corresponding RNFL thickness, which could occupy 11,280 data pairs.

For example, a summary of the collected raw data can be shown as <Table 1>.

Collected data for training Number of patients 303 Number of eye examinations 453 Number of images 940 Number of parted images 11,280 Patients’ age (years)

13.954.7

13.9 Gender (% of female) 48.3 Intraocular pressure (mmHg) 13.87

3.01 Global RNFL (

) 84.31

16.21

In addition, 583 fundus photographs can be obtained from normal patients.

Although the glaucoma specialist selected 103 suspected patients from fundus images of normal patients, OCT measurement may not be performed in normal patients.

Additional fundus images of normal patients can be used to confirm the applicability of the approach according to the present invention.

In one embodiment, a Convolutional Neural Network (CNN) may be used to predict RNFL thickness in color fundus images.

Referring to FIG. 1B , the architecture of the thickness estimation deep learning model according to the present invention may be composed of four convolution blocks and fully connected layers.

For example, each convolution block may consist of a double convolution layer with a kernel size of 3x3 and a stride of 2.

After the convolutional layer, batch normalization and max-pooling layers can be used for stable and fast learning.

The depth of the convolution layer can be increased from 16 to 128 to extract enough image features for accurate estimation of the RNFL thickness.

A fully connected layer with 128 nodes is placed after the final convolution block, and high-dimensional features extracted from the convolution block can be obtained.

The final node may output the estimated RNFL thickness from the merged and weighted features that are converted to numeric values of the input color fundus image.

Non-linearity can be implemented by Rectified Linear Unit (ReLU) activation in each convolutional and fully connected layer.

Data pairs collected for training can be randomly divided into 80% (9,024 pairs) and 20% (2,256 pairs) for training and validation, respectively.

32 randomly sampled pairs at each stage of training can form a mini-batch.

To increase the diversity and heterogeneity of the training set, the color fundus image data can be augmented by applying random contrast, brightness, hue, and saturation.

The mean squared error (MSE) is used for the loss function and the model parameters can be updated by the optimizer.

According to the present invention, for comparison with the prior art, the model can be further trained in the same way to predict the global mean RNFL thickness in whole optic disc images.

In one embodiment, for RNFL thickness estimation and classification for glaucoma screening, two trained CNN models can predict RNFL thickness values in whole and cross-sectional optic disc images.

For a given fundus image input, the CNN can output the RNFL thickness in um scale.

For each eye, a total of 13 RNFL values including a global average RNFL corresponding to 360 degrees and 12 regional RNFLs corresponding to each direction (30 degrees) can be predicted.

In one embodiment, the global area of the fundus image may refer to a partial area (eg, a circular area) corresponding to a preprocessed 360 degrees of the fundus image. In one embodiment, the local region of the fundus image may refer to a sub-region corresponding to a specific angle (eg, 30 degrees) obtained by dividing the global region into a plurality of regions.

The thinning level of RNFL is classified into three levels: green, yellow and red by comparison between the measured OCT and the normative database. In one embodiment, the thinning level may be referred to as a thickness level or a term having equivalent technical meaning.

Green can represent within the normal range between 95% and 5% of the baseline data, yellow represent a borderline range between 5% and 1% of the baseline data, and red represent an out-of-normal range outside 1% of the standard baseline data.

For screening, we can use the predicted thinning level of the RNFL determined by comparison between the CNN predicted RNFL thickness and a normative database.

Referring to the prior art investigating the screening ability of RNFL measurement using OCT, according to the present invention, glaucoma can be determined when an abnormal or borderline level is included in a region of interest for a specific fundus image.

Many sub-region combinations can be tested to ascertain local dependencies of screening performance.

2 is a diagram showing a graph of actual and predicted RNFL thickness according to an embodiment of the present invention.

Referring to FIG. 2, for model evaluation and local RNFL thinning levels, the relationship between the RNFL thickness estimated from the optic nerve disc image by the trained CNN and the OCT measurements of the validation set for regional and global average RNFL can be confirmed. .

The mean absolute error (MAE) between prediction and OCT measured for the local RNFL prediction of the trained model is 14.93 μm, and the R-squared value and Pearson's correlation coefficient can be 0.594 and 0.771, respectively.

In the case of global RNFL prediction, mean absolute error (MAE) was 8.37 μm, R-squared was 0.547, and Pearson's correlation was 0.739.

The predicted values can confirm a strong correlation with the ground truth as shown in Fig. 2 by the linear trend across a wide range of RNFL thicknesses in both local and global averages.

MAE for local RNFL thickness prediction may be more important than global average RNFL prediction. In one embodiment, local RNFL thickness may be referred to as regional RNFL thickness or a term having equivalent technical meaning.

The higher MAE in local RNFL predictions can be attributed to reduced information due to the optic disc images being sectioned for input to the DL model.

Compared with the prior art, it can be seen that the MAE and correlation for the global average RNFL prediction of the architecture of the model according to the present invention achieve a similar error range of 7.39 μm.

The correlation between predicted RNFL and OCT-measured RNFL confirms that the CNN architecture according to the present invention was trained as desired and works properly for RNFL thickness estimation from optic disc images.

A summary comparison between average OCT measurements and trained CNN predictions for RNFL in the test data set along the region can be shown in Table 2.

Region Acronym OCT measurement CNN Prediction Global average G.A.

16.483.5

16.4 83.6

12.2 Superior temporal ST 106.5

35.6 99.2

23.6 Superior SS 104.2

29.7 101.9

21.0 Superior nasal SN 98.6

25.4 90.4

19.8 Nasal Superior NS 76.2

23.1 74.6

16.5 Nasal NN 56.4

14.7 60.8

11.4 Nasal inferior NI 64.4

15.5 70.3

14.3 Inferior nasal IN 89.4

24.5 89.2

19.6 Inferior II 103.3

37.5 100.7

25.8 Inferior temporal IT 92.3

42.2 90.9

27.5 Temporal inferior TI 67.1

18.4 70.5

12.9 Temporal TT 62.5

11.5 67.4

12.7 Temporal superior TS 81.0

19.7 77.4

15.7

3A is a diagram illustrating labeling classification of RNFL thinning levels according to an embodiment of the present invention. 3B is a diagram illustrating labeling classification of RNFL thinning levels for a steady state according to an embodiment of the present invention. 3C is a diagram illustrating labeling classification of RNFL thinning levels for some damage conditions according to an embodiment of the present invention. 3D is a diagram illustrating labeling classification of RNFL thinning levels for a complete damage condition according to an embodiment of the present invention.

Referring to Figures 3a to 3d, predictions can be confirmed in the verification set for various RNFL thickness processes.

The inner color label may indicate an actual thinning level in OCT measurement, and the outer color label may indicate a predicted level in CNN estimation according to the present invention.

Referring to FIG. 3A , examples of global averages and actual and predicted RNFL thinning levels for each subregion can be seen.

Fig. 3b shows that all areas are normal, Fig. 3c shows normal but partially damaged RNFL, and Fig. 3d shows a case where RNFL thinning progresses acutely in severe cases.

The prediction of the RNFL thinning level may not always be accurate because there is an error in the RNFL thickness prediction in this example.

Because of the margin of error in local RNFL prediction over the range of 12-way borderlines, most prediction failures at thinning levels can occur between normal and borderline or between abnormal and borderline levels.

However, the inferential function of focal RNFL thinning levels in color fundus images may be useful for screening for glaucoma when OCT measurements are not available.

The condition can be briefly distinguished between the various levels of progression of RNFL deficits.

4A is a diagram illustrating RNFL classification for an NTG group according to an embodiment of the present invention. 4B is a diagram illustrating RNFL classification for a suspicious group according to an embodiment of the present invention. 4C is a diagram illustrating RNFL classification for a normal group according to an embodiment of the present invention.

Referring to FIGS. 4A to 4C , for local RNFL thinning and glaucoma tests, the prediction protocol according to the present invention can be applied to three groups to confirm different RNFL thinning levels among separate patient groups.

In this case, FIG. 4a may show prediction results of the eye in a glaucoma (NTG) patient group, FIG. 4b a suspected patient group, and FIG. 4c a normal patient group.

The color grid can represent predictions of RNFL thinning levels in color fundus images using a CNN trained on global averages and using 12 local parts for 30 randomly selected cases in each group.

A scatter point may indicate a predicted RNFL value for a corresponding color at a pale color level. As a result, it can be confirmed that the classification of the global mean RNFL thinning level may not be appropriate for differentiating individual glaucomatous eyes.

As shown in FIGS. 4A and 4B , it may be a result of classifying the average RNFL of the most suspicious eye with glaucoma in the normal range.

However, clear distinctions can be made in the level of RNFL thinning by classified regions.

In particular, directional thinning of RNFL can be clearly seen in the superior and inferior regions in glaucomatous eyes.

More specifically, directional thinning can be predicted in suspected patients in the superior-temporal (ST), inferior (II) and inferior-temporal (IT) regions.

Significant thinning of the superior and inferior regions correlates strongly with the early stages of the glaucomatous progression and may thin much faster than other regions.

The results of RNFL thickness estimated from color fundus images using trained CNNs can show similar trends to previous results.

Therefore, predicting RNFL thinning levels along 12 regions in optic disc images using a trained CNN may be useful for screening glaucoma patients with OCT-based RNFL measurements.

A summary of the estimated RNFLs in the three groups, with global averages and means and standard deviations for the 12 regions, is shown in Table 3.

Glaucoma Suspicious Normal G.A.

9.577.2

9.5 89.4

9.9 95.6

8.3 ST 78.1

17.9 106.9

21.7 131.0

15.5 SS 80.1

17.7 114.0

18.6 126.3

14.5 SN 73.2

16.0 101.3

18.3 113.1

14.3 NS 64.1

14.3 86.0

15.5 99.0

15.7 NN 58.5

13.4 69.1

11.6 74.7

10.8 NI 65.0

12.2 78.2

13.0 87.4

14.5 IN 75.7

15.4 101.3

16.2 113.9

15.4 II 81.1

18.2 115.1

21.9 134.5

16.1 IT 74.2

15.5 100.7

21.0 126.1

15.7 TI 64.4

12.3 73.3

12.4 82.6

12.1 TT 64.3

12.9 66.4

9.5 76.1

10.8 TS 66.0

13.6 79.0

12.6 89.9

12.0

5A is a diagram illustrating a global RNFL-based confusion matrix according to an embodiment of the present invention. 5B is a diagram illustrating upper and lower RNFL-based confusion matrices according to an embodiment of the present invention. 5C is a diagram illustrating sensitivity and specificity screening results based on a combination of two subregions according to an embodiment of the present invention.

Referring to Figures 5a to 5c, the results of the screening test can be confirmed.

For testing, 208 color fundus images can be separately collected from glaucoma patient data that were not used for model training.

In the normal case, 313 color fundus images can be collected from the normal eye in the center of the screen data.

The screening rules for distinguishing glaucomatous eyes can simply identify the predicted categories of average and regional RNFL thickness.

1) Glaucoma when the level of thinning is abnormal or borderline in screening by average RNFL

2) In the case of screening using local RNFL, one of the thinning levels of the upper or lower regions (i.e., ST, SS, SN, IN, II, and IT) judged as glaucoma is abnormal or borderline.

The results can be confirmed through the confusion matrices of FIGS. 5A and 5B.

A screening result based on the global mean RNFL thickness may have a sensitivity of 0.308 and a specificity of 0.99.

As a result of screening using regional information, sensitivity may be 0.942 and specificity may be 0.821.

Comparing the two confusion matrices, it is clear that screening based on regional information performs much better to discriminate between glaucomatous eyes.

In addition, subregions that are important for screening performance in terms of sensitivity and specificity can be further investigated.

A screening test can be performed by combining two regional subsections out of 12 regions. The criterion for determining glaucoma may be the same as before.

An eye in which one of the degree of thinning is abnormal or borderline can be judged as glaucoma.

Referring to FIG. 5C , the sensitivity and specificity of all possible combinations of screening results can be confirmed.

The results may indicate that the screening sensitivity is relatively high when the upper or lower regions are included.

Therefore, for the screening of glaucoma patients, upper and lower RNFL defects may be important, at least in the case of normal tension glaucoma.

Referring to FIG. 5C , the sensitivity and specificity of the screening results using the two subregions for all possible combinations can be displayed as a color map. In this case, when the ST, II, and IT regions are included, it can be confirmed that the screening sensitivity is relatively higher than other combinations and the specificity is lower than other combinations.

In one embodiment, a correlation value (eg, sensitivity, specificity) between two sub-regions among a plurality of sub-regions is calculated, and two sub-regions corresponding to a case where the calculated correlation value is greater than a threshold value are determined; , it is possible to calculate the retinal nerve fiber layer thickness corresponding to each of the plurality of sub-regions by applying the determined two sub-regions to a thickness estimation deep learning model.

That is, the thickness of the retinal nerve fiber layer can be calculated with higher accuracy by inputting only the sub-region determined in consideration of the correlation value into the model instead of inputting all the sub-regions into the model.

For example, if one superior-temporal (ST), inferior (II), and inferior-temporal (IT) subregion is included, the sensitivity of screening may be higher than 0.92.

Interestingly, when these regions are included, the specificity is lower than 0.85 and worse than the other combinations, probably due to the wide spectrum of normal ranges.

For example, in FIG. 5C , S denotes superior, N denotes nasal, I denotes inferior, and T denotes temporal.

In an embodiment, a color fundus image may be acquired, and only when a resolution value of the obtained color fundus image is greater than or equal to a threshold value, the color fundus image may be divided into a plurality of sub-regions.

In one embodiment, the color fundus image is divided into a plurality of sub-regions, a resolution value for each of the plurality of sub-regions is calculated, and a thickness is estimated for at least one sub-region when each resolution value is greater than a threshold value. The thickness of the retinal nerve fiber layer corresponding to each of the at least one subregion may be calculated by applying the deep learning model.

In addition, when other risk factors such as intraocular pressure, history of diabetes, high blood pressure, and high myopia are included in the thickness estimation deep learning model, the screening ability according to the present invention can be further improved.

In one embodiment, additional quantitative values including edge thinning, notching, and cup-to-disc ratio in the color fundus image may be considered for better reliability of the glaucoma examination.

In one embodiment, each of the plurality of sub-regions is applied to a thickness estimation deep learning model to calculate a feature corresponding to each of the plurality of sub-regions, and weights determined based on edge thinning, notching, and cup-to-disk ratio (weight) can be applied to the feature to more accurately calculate the retinal nerve fiber layer thickness.

Moreover, since glaucoma is known to be age-related, its prevalence can be expected to increase worldwide due to the aging of the population.

In particular, the prevalence of glaucoma in many developing countries is very high, possibly because most cases go undiagnosed or are managed suboptimally due to a lack of diagnostic and screening tools.

Therefore, the availability of low-cost imaging devices for quick and easy screening could be of great help in the treatment of glaucoma in resource-poor settings.

For future work, fully developed CNN models can be integrated with low-cost, portable fundus devices that facilitate diagnosis even with an ophthalmoscope.

Given the low cost of this approach and the evaluation quality provided by OCT and color fundus images, our DL model could support novel measurement and screening of glaucoma and other ocular conditions in resource-poor settings.

6 is a diagram illustrating a deep learning-based retinal nerve fiber layer thickness determination method in a color fundus image for glaucoma examination according to an embodiment of the present invention. In one embodiment, each step of FIG. 6 may be performed by the thickness determining device 700 of FIG. 7 .

Referring to FIG. 6 , step S601 is a step of obtaining a color fundus image for a user.

Step S603 is a step of dividing the color fundus image into a plurality of sub-regions.

In one embodiment, an optic disc region may be extracted from the color fundus image by performing pre-processing on the color fundus image, and the optic disc region may be segmented into a plurality of sub-regions.

In one embodiment, the size of each sub-region may be adjusted in response to the resolution value of each of the plurality of sub-regions. For example, the larger the resolution value is, the larger the size of the corresponding sub-region is, and the smaller the resolution value is, the smaller the size of the corresponding sub-region can be adjusted.

In one embodiment, the multiple subregions may include superior-temporal (ST), inferior (II) and inferior-temporal (IT) regions.

Step S605 is a step of calculating a retinal nerve fiber layer thickness corresponding to each of the plurality of sub-regions by applying the thickness estimation deep learning model to each of the plurality of divided sub-regions.

In one embodiment, after step S605, whether or not the user has glaucoma may be classified according to the thickness of the retinal nerve fiber layer.

7 is a diagram showing a functional configuration of an apparatus 700 for determining the thickness of the retinal nerve fiber layer based on deep learning in a color fundus image for glaucoma examination according to an embodiment of the present invention.

Referring to FIG. 7 , the thickness determination device 700 may include an acquisition unit 710, a control unit 720, a storage unit 730, and a display unit 740.

The acquisition unit 710 may obtain a color fundus image of the user. In one embodiment, the acquisition unit 710 may include an fundus imaging device.

In one embodiment, the acquisition unit 710 may include a communication unit. In this case, the communication unit may include at least one of a wired communication module and a wireless communication module. All or part of the communication unit may be referred to as a 'transmitter', a 'receiver', or a 'transceiver'.

The control unit 720 divides the color fundus image into a plurality of sub-regions, and applies each of the divided sub-regions to a deep learning model for estimating the thickness to determine the retinal nerve fiber layer thickness corresponding to each of the plurality of regions. ) can be calculated.

In one embodiment, the controller 720 may include at least one processor or microprocessor, or may be part of a processor. Also, the controller 720 may be referred to as a communication processor (CP). The controller 720 may control the operation of the thickness determining device 700 according to various embodiments of the present disclosure.

The storage unit 730 may store at least one of a color fundus image, a retinal nerve fiber layer thickness, and a classification result of whether or not the user has glaucoma.

In one embodiment, the storage unit 730 may include volatile memory, non-volatile memory, or a combination of volatile and non-volatile memory. Also, the storage unit 730 may provide stored data according to the request of the control unit 720 .

The display unit 740 may indicate information processed by the thickness determining device 700 . For example, the display unit may include a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, an Organic LED (OLED) display, and a Micro Electro Mechanical Systems (MEMS) display. It may include at least one of a display and an electronic paper display.

Referring to FIG. 7 , the thickness determination device 700 may include an acquisition unit 710, a control unit 720, a storage unit 730, and a display unit 740. In various embodiments of the present invention, the thickness determination device 700 has more components than the components described in FIG. 7 or less components than those described in FIG. 7 because the configurations described in FIG. 7 are not essential. there is.

The above description is only illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention.

The various embodiments disclosed herein may be performed out of order, concurrently or separately.

In one embodiment, at least one step may be omitted or added in each figure described herein, may be performed in reverse order, or may be performed concurrently.

The embodiments disclosed herein are not intended to limit the technical spirit of the present invention, but are intended to explain, and the scope of the present invention is not limited by these embodiments.

The protection scope of the present invention should be interpreted according to the claims, and all technical ideas within the equivalent range should be understood to be included in the scope of the present invention.

700: thickness determination device
710: acquisition unit
720: control unit
730: storage unit
740: display unit

Claims (8)

(a) obtaining a color fundus image for the user;
(b) segmenting the color fundus image into a plurality of subregions; and
(c) calculating a retinal nerve fiber layer thickness corresponding to each of the plurality of sub-regions by applying a thickness estimation deep learning model to each of the plurality of sub-regions;
including,
Deep learning-based retinal nerve fiber layer thickness determination method in color fundus images for glaucoma examination.
According to claim 1,
In step (b),
extracting an optic disc region from the color fundus image by pre-processing the color fundus image; and
dividing the optic nerve head region into the plurality of sub-regions;
including,
Deep learning-based retinal nerve fiber layer thickness determination method in color fundus images for glaucoma examination.
According to claim 1,
After step (c),
classifying whether or not the user has glaucoma according to the thickness of the retinal nerve fiber layer;
Including more,
Deep learning-based retinal nerve fiber layer thickness determination method in color fundus images for glaucoma examination.
According to claim 1,
The plurality of subregions include superior-temporal (ST), inferior (II) and inferior-temporal (IT) regions,
Deep learning-based retinal nerve fiber layer thickness determination method in color fundus images for glaucoma examination.
an acquisition unit that acquires a color fundus image of the user; and
segmenting the color fundus image into multiple subregions;
a control unit for calculating a retinal nerve fiber layer thickness corresponding to each of the plurality of sub-regions by applying a thickness estimation deep learning model to each of the plurality of sub-regions;
including,
Deep learning-based retinal nerve fiber layer thickness determination device in color fundus images for glaucoma examination.
According to claim 5,
The control unit,
Preprocessing is performed on the color fundus image to extract an optic disc region from the color fundus image;
Dividing the optic nerve head region into the plurality of sub-regions,
Deep learning-based retinal nerve fiber layer thickness determination device in color fundus images for glaucoma examination.
According to claim 5,
The control unit,
Classifying whether or not the user has glaucoma according to the thickness of the retinal nerve fiber layer,
Deep learning-based retinal nerve fiber layer thickness determination device in color fundus images for glaucoma examination.
According to claim 5,
The plurality of subregions include superior-temporal (ST), inferior (II) and inferior-temporal (IT) regions,
Deep learning-based retinal nerve fiber layer thickness determination device in color fundus images for glaucoma examination.

Images