WO2013041977A1

WO2013041977A1 - Method of retinal image enhancement and tool therefor

Info

Publication number: WO2013041977A1
Application number: PCT/IB2012/050647
Authority: WO
Inventors: Samit Dilipkumar DESAI; Poornima MOHANACHANDRAN
Original assignee: BANERJI, Shyamol
Priority date: 2011-09-20
Filing date: 2012-02-13
Publication date: 2013-03-28

Abstract

In one aspect, the invention provides a method for enhancing a retinal image. The method includes steps for extracting at least one of a R, G, or B planes from the retinal image as an extracted plane; applying a filtering technique for noise suppression on the extracted plane to obtain a filtered plane; performing contrast enhancement on the filtered plane to obtain an enhanced image plane; and reconstructing a gray image using the enhanced image plane. The method then involves reconstructing the retinal image using the gray image to obtain an enhanced color image. In another aspect, the invention provides a tool for retinal image visualization based on the method of the invention. The method and tool of the invention allows for improved viewing of retinal images and facilitates diagnosis and treatment of retinal conditions of patients.

Description

METHOD OF RETINAL IMAGE ENHANCEMENT AND TOOL THEREFOR

TECHNICAL FIELD

[0001] The invention relates generally to retinal image enhancement and more specifically to a method and a tool for retinal image enhancement to facilitate viewing and analysis of retinal images.

BACKGROUND

[0002] Analyzing retinal fundus image is important for early detection of multiple diseases related to the retina, such as but not limited to Retinopathy of Prematurity (ROP), Diabetic Retinopathy(DR) or Aggressive Posterior Retinopathy of Prematurity (APROP). These diseases are related to vasculature abnormalities of retina. ROP and APROP which affects neonates, is incomplete vasculaturization of retina and leads to retinal detachment and/or loss of vision. Diabetic Retinopathy refers to damage of retina due to diabetes. In these conditions, information of retinal vasculature is important to accurately determine the nature and severity of disease. Digital fundus photography is widely used for diagnosis and follow up treatments. However, digital color fundus images obtained from fundus cameras have limitations of low contrast (such as contrast between retinal vasculature and background), image noise and non-uniform illumination, especially towards periphery of the retina. This adversely affects utility of images for diagnosis and treatment planning.

[0003] To circumvent the limitations presented by noise and non-uniform illumination, direct examination of retina by ophthalmoscope is done. Alternatively to obtain high contrast images that allow for facile analysis, invasive techniques involving the use of contrast agents are required. Such contrast agents pose problems related to allergic reactions or side effects for the patient and demands presence of an expert physician.

[0004] In this regard, there is a need to address the noise and low contrast condition in retinal fundus images without the use of a contrast agent that allows for the detection of features that are clinically subtle or undetectable, significantly helping diagnosis and treatment planning.

BRIEF DESCRIPTION [0005] In one aspect, the invention provides a method for enhancing a retinal fundus image. The method comprises extracting at least one of the R, G, or B image planes, or their combination as an extracted plane. The method then comprises applying a filtering technique for noise suppression on the extracted plane to obtain a filtered plane. The method then includes the steps for performing contrast enhancement on the filtered plane to obtain an enhanced image plane; and reconstructing a gray image using the enhanced image plane. This enhanced gray image itself is used for visualization and disease diagnosis. The method further comprises reconstructing the retinal colour image using the gray image to obtain an enhanced color image.

[0006] In another aspect, the invention provides a tool for retinal image visualization.

The tool comprises an image enhancement module for extracting at least one of R, G, B plane from the retinal image as an extracted plane, applying a filtering technique for noise suppression on the extracted plane to obtain a filtered plane, performing contrast enhancement of the filtered plane to obtain an enhanced image plane, and reconstructing a gray image using the enhanced image plane. The tool further comprises a color enhancement module for reconstructing the retinal image using the gray image to obtain an enhanced color image. The tool also includes modules like vesselness processing module, display module and navigation module.

[0007] The tool provides measurement capabilities to quantify properties of vessels such as but not limited to vessel density, length and tortuosity. As would be appreciated by those skilled in the art, the measurement capabilities aid objective evaluation of the disease.

DRAWINGS

[0008] These and other features, aspects, and advantages of the present invention will become better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:

[0009] FIG. 1 is a flowchart representation of exemplary steps involved in the method of the invention;

[0010] FIG. 2 is a retinal image as obtained from an ophthalmic fundus camera; [0011] FIG. 3 is a gray image obtained using the method of the invention,

[0012] FIG. 4 is a color enhanced retinal image using the steps of the method of the invention;

[0013] FIG. 5 is a color vessel image using the steps of the method of the invention;

[0014] FIG. 6 is a block diagrammatic representation of the software tool of the invention; and

[0015] FIG. 7-10 illustrate screen shots from the tool showing the different images for quick reference of the physician.

DETAILED DESCRIPTION

[0016] As used herein and in the claims, the singular forms "a," "an," and "the" include the plural reference unless the context clearly indicates otherwise.

[0017] As used herein, the phrase "retinal image" includes any image of a primate's eye using known techniques. Such known techniques for obtaining an image of the eye include, for example, but not limited to, indirect ophthalmoscopy, fundus imaging and the like. Using the techniques known in the art, several aspects of the eye may be imaged and recorded, thus allowing for analysis of the eye. The images may also be referred to as ophthalmic images, or fundus image, and the like in the art, the exact nomenclature used depending on some factors, such as the type of equipment used, or part of the eye being imaged, and so on, or combinations thereof. All such images are contemplated to be included within the scope of the invention. The retinal image useful in the invention is a digital image, wherein the image is obtained directly from an imaging instrument as a digital image, or an analog image that has been digitized using techniques known in the art. Retinal image most useful in the invention include those that are made available in an electronic file format. Useful file formats include, for example, bmp, jpg, png, tiff, gif, and the like. Just like any electronic image, a retinal image comprises a plurality of pixels.

[0018] As noted herein, in one aspect the invention provides a method for enhancing a retinal image. The method is depicted in a flowchart representation in FIG. 1 and is assigned numeral 10. The method of the invention 10 comprises providing the retinal image such as a retinal fundus image comprising an original R, G, B planes, shown in FIG. 1 by numeral 12. As stated herein, the retinal image comprises a plurality of pixels. Each pixel carries red, green and blue color information or values, represented in an image as R, G, and B planes. The identity of the pixel and its corresponding R, G and B planes may be represented in a tabular form, text form, or in any other convenient form known to those of ordinary skill in the art. Each of this information is stored in a suitable form, such as an 8-bit string or a 16- bit string, or the like.

[0019] The method then involves extracting at least one of the R, G, or B planes from the retinal image as an extracted plane, as represented by numeral 14 in FIG. 1. The choice of R, G or B plane to be extracted depends on the image being used, and the analysis to be conducted. . In one instance the combination of the R G and B planes can be their weighted average.

[0020] Without being bound to any theory, it is known that different components of the eye provide a differential absorption at different colors (Hani et al. Retinal vasculature enhancement using independent component analysis, J. Biomed. Sci Engg., (2009) Vol. 2, pp 543-549). For example, hemoglobin absorbs relatively higher level of green spectra compared to its background resulting in best contrast in G plane. Thus, extracting individual color responses from the color planes provides the capability of enhancing the response from a particular component of the eye. The method then includes applying a filtering technique for noise suppression on the extracted plane to obtain a filtered plane, as shown in FIG. 1 by numeral 16. Noise suppression filtering techniques are known in the art, and may include, for example, but not limited to anisotropic diffusion filtering technique. Other techniques are known to those skilled in the art, and are contemplated to be within the scope of the invention.

[0021] The method also comprises performing contrast enhancement of the filtered plane to obtain enhanced plane, represented by numeral 18 in FIG. 1. Then at step 19, the method includes reconstructing a gray image using the enhanced image plane. This provides for enhanced images, which then allows for better analysis of the retinal image for more accurate and faster diagnosis of the eye. Further a combination of the responses from different components of the eye can be enhanced to obtain gray images with different image information for better diagnosis. [0022] Contrast enhancement methods are known to those skilled in the art, and include techniques such as, but not limited to, Histogram Equalization, Adaptive Histogram Equalization, Contrast Limited Adaptive Histogram Equalization (CLAHE), independent component analysis (ICA) [as described in for example Hani et al. Retinal vasculature enhancement using independent component analysis, J. Biomed. Sci Engg., (2009) Vol. 2, pp 543-549]. In one particular embodiment, the enhancement technique used is the Contrast Limited Adaptive Histogram Equalization method. The chosen method is applied to the extracted color plane. In one exemplary embodiment, the extracted plane is the G plane and G values for every pixel of an image are extracted. After applying a suitable filtering technique, contrast enhancement is done using CLAHE. After contrast enhancement, the enhanced G plane, i.e G' plane is obtained. This is then represented in a suitable manner along with the pixel identity, such as tabular form, textual form, and the like. G' is a gray image that is used for visualization and disease diagnosis .

[0023] Histogram equalization with its modification is commonly used to enhance the image contrast. However, histogram equalization tends to over-enhance the image and results in noisy appearance of the output image. Another technique for contrast improvement CLAHE improves the image, however, it creates artefacts in the enhanced image and the selection of contrast gain limit is image-dependent. Thus, applying contrast enhancement in itself does not provide an improved image that allows for facile analysis. When noise suppression steps are applied and subsequently contrast enhancement steps are applied, the image obtained are superior in the contrast and gain, and allows for easy analysis.

[0024] The method further comprises reconstructing the retinal image using the gray image to obtain an enhanced color image, as depicted by numeral 20 in FIG. 1. As an illustration using the aforementioned exemplary embodiment, each pixel is now represented by the enhanced color planes such as R, G' and B, or R',G,B or R,G,B'. Subsequently, all the pixels are now combined to provide an enhanced color image. Techniques to regenerate an image based on pixel information are known to those skilled in the art. The enhanced color image provides images of better clarity. Such improved retinal images enable better and more accurate disease diagnosis and treatment planning.

[0025] Reverting back to FIG. 1, in a further embodiment, the method of the invention includes a step 22 applying a vesselness measure on the enhanced color image. Vesselness measure allows for visualizing blood vessels within the retinal image as contrasted with non-vasculature of the eye. An exemplary vesselness measure is obtained using Franji vesselness measure method, known to those skilled in the art. In a Franji vesselness measure technique, the probability of whether a pixel belongs to a vessel region or not is provided as a number ranging between 0 and 1. A probability greater than a certain value is considered to be a vessel, while anything less is considered to be a non-vessel feature of the eye. The exact value to be used depends on various factors, which factors are known to one skilled in the art, and the value to be used may be for example, 0.99 in one embodiment, 0.95 in another embodiment, and 0.90 in yet another embodiment, and 0.80 in a further embodiment.

[0026] The probability values from the vesselness measure may also be used to create a pseudo-color image. For example, if the cut-off probability value is considered to be 0.95, all probability values greater than 0.99 may be colored red, probability values lying between 0.98 and 0.99 may be colored blue, while probability values between 0.95 and 0.98 may be colored pink, and other probability values are colored white. Thus, a pseudo-color image is obtained that provides for improved visualization of the original retinal image. This enables better visualization and disease diagnosis.

[0027] FIG. 2, 3, 4 and 5 illustrate different images as described herein above. FIG. 2 is a retinal image (original image) as obtained from a fundus camera using TOPCON ® TRC- NW7SF), while FIG. 3 is a gray image obtained using the techniques described above and FIG. 4 is a color enhanced retinal image using the steps of the method of the invention as described above and FIG. 5 is a color vessel image using the steps of the invention. As can be seen, several features of the retina such as vasculature are readily evident in the enhanced color image as compared to the original retinal image.

[0028] Thus, the method of the invention comprises three distinct components: "Gray

Enhancement (GE)", "Color Enhancement (CE)" and "Vesselness Measure (VNM)" to provide improved images and data. The combination of these individual components in the method of the invention is useful in a variety of different applications. For instance, color enhancement allows for better visualization of image attributes such as capillary non- perfusion zones and edge of vascular loops in APROP. The vesselness measure is useful in determining vessel attributes such as determining if the vessels have reached the ora serrata ('ora' is edge of the retina), analyzing occluded vessels, delineating vascular and avascular area in case of APROP, determining vessel edges in a selected area of the image, determining vessel growth in and around an area of surgery, determining leaky vessels, and the like, and combinations thereof. The method is also useful for generating a layered image with prominent and choroidal vessels.. As used herein ROP stands for Retinopathy of Prematurity. APROP refers to Aggressive Posterior ROP. Other aspects enabled by the method of the invention include, for example, vasculature density estimation, a vessel count in region of interest, vessel tracing, measuring and displaying length, tortuosity, diameter, rate of change of diameter and so on, and combinations thereof.

[0029] The exemplary steps of the method of the invention may be advantageously implemented in the form of an algorithm that is provided in a suitable programming language executed on a computing device to execute the instructions. The software program product may be made available as an executable file through any hardware such as Compact Disc (CD), Digital Versatile/Video Disc (DVD), flash drive, cartridges, EEPROM, and combinations thereof, or a downloadable and executable file from a suitable location such as an internet site. The software program may also include necessary authorizations and security requirements built into it. The software may further include several use levels such as manager, administrator, user and the like.

[0030] Thus, in another aspect of the invention provides a software program tool based on the method of the invention. The software tool is shown in a block diagrammatic representation in FIG. 6 and represented by numeral 24. The tool comprises a retinal image receiving module 26 for receiving a retinal image, an image processing module 28 for extracting at least one of R, G, B plane from the retinal image as an extracted plane, applying a filtering technique for noise suppression on the extracted plane to obtain a filtered plane, performing contrast enhancement of the filtered plane to obtain an enhanced image plane, and reconstructing a gray image 30 using the enhanced image plane. The tool further comprises a color enhancement module 32 for reconstructing the retinal image using the gray image to obtain an enhanced color image 34. Further, the image enhancement processing module 28 and/or color enhancement module 32 may be configured to enable detecting image attributes 36 such as at least one of capillary non perfusion areas, determining an edge of pathology, delineating vascular and avascular area in case of APROP, determining vessel edges in a selected area of the image, and the like, and combinations thereof.

[0031] The tool 24 also comprises a vesselness processing module 38 for detecting one or more vessels from the enhanced color image using a vesselness measure as described herein. The vesselness processing module 38 is also given the capability of applying a pseudo color on the detected one or more vessels to obtain a color vessel image 40. The vesselness processing module 38 is also configured for providing enhanced images for improved analysis for vessel attributes 42, such as, for example, at least one of generating a layered image with prominent and choroidal vessels, analyzing occluded vessels, determining APROP area, determining if vessels have reached ORA, determining vessel edges in a selected zone, determining vessel growth in and around an area of surgery, determining leaky vessels, and the like, and combinations thereof.

[0032] The tool 24 further comprises a display module 42 for displaying the gray image, the enhanced color image and the color vessel image as well as for displaying the image attributes and vessel attributes. The display module 42 may be a monitor of sufficient resolution that allows for facile viewing and analysis. Suitable resolution useful in the invention may be, for example 1024x768. The display module 42 is configured to display enhanced images for improved analysis, such as for example, at least one of a color map of retina for vessel density, a vessel count in region of interest, tracing a vessel , measuring and displaying length, tortuosity, diameter, rate of change of diameter, and the like. The display module 44 may also be configured for registration and stitching together images and data from other modalities with the retinal image to obtain further enhanced images for improved analysis using multiple modalities, thus enabling more accurate diagnosis of conditions of the subject.

[0033] The tool also comprises a navigation module (not shown) for identifying different parts of retina on the enhanced color image and the color vessel image, and for navigating along a vessel on the color vessel image.

[0034] FIG. 7 illustrates a screen shot from the tool showing the original retinal image

27, gray image 30, color enhanced image 34 and color vessel image 40 as described herein. FIG. 8 shows a screenshot for am image for tortuosity measurement. FIG. 9 shows a screenshot for vessel mapping and vessel measurement; and FIG. 10 shows a comparative view of images for APROP visualizations and diagnosis.

[0035] It would be appreciated by those skilled in the art that the method and tool described herein allow for significantly improving the retinal images to render clearer visualization of different image components that in turn is extremely useful in more accurate diagnosis and treatment planning or surgery planning for of the different diseases related to retina or eye in general.

[0036] While only certain features of the invention have been illustrated and described herein, many modifications and changes will occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the invention.

Claims

We Claim:

1. A method for enhancing a retinal image, the method comprising: extracting at least one of a R, G, or B planes from the retinal image as an extracted plane; applying a filtering technique for noise suppression on the extracted plane to obtain a filtered plane; performing contrast enhancement on the filtered plane to obtain an enhanced image plane; and reconstructing a gray image using the enhanced image plane.

2. The method of claim 1 further comprising reconstructing the retinal image using the gray image to obtain an enhanced color image.

3. The method of claim 1 further comprising displaying at least one of the gray image, the enhanced color image, or both on an output device.

4. The method of claim 1 further comprising applying a vesselness measure for detecting one or more vessels from the enhanced image plane.

5. The method of claim 4 further comprising applying a pseudo color on the detected one or more vessels to obtain a color vessel image.

6. The method of claim 4 further comprising obtaining quantitative information on vesselness measure.

7. The method of claim 1 and 5 further comprising storing the gray image, enhanced color image and the color vessel image.

8. A tool for retinal image visualization, the tool comprising: an image processing module for extracting at least one of R, G, B plane from the retinal image as an extracted plane, applying a filtering technique for noise suppression on the extracted plane to obtain a filtered plane, performing contrast enhancement of the filtered plane to obtain an enhanced image plane, and reconstructing a gray image using the enhanced image plane.

9. The tool of claim 8 further comprising a color enhancement module for reconstructing the retinal image using the gray image to obtain an enhanced color image.

10. The tool of claim 8 further comprising a vesselness processing module for detecting one or more vessels from the enhanced image plane using a vesselness measure, and applying a pseudo color on the detected one or more vessels to obtain a color vessel image.

11. The tool of claim 9 or 10 further comprising a display module configured to display at least one of the gray image, the enhanced color image, the color vessel image or combinations thereof.

12. The tool of claim 10 wherein the image processing module is configured for detecting one or more selected image attributes.

13. The tool of claim 10 wherein the vesselness processing module is configured for at least one of generating a layered image with prominent and choroidal vessels, or for analyzing the one or more vessels.

14. The tool of claim 9 or 10 further comprising a navigation module for identifying different regions of retina on the enhanced color image and the color vessel image, and for navigating along a vessel on the color vessel image.

15. A system comprising a tool of claim 8.