WO2011007126A2 - Disease determination - Google Patents
Disease determination Download PDFInfo
- Publication number
- WO2011007126A2 WO2011007126A2 PCT/GB2010/001330 GB2010001330W WO2011007126A2 WO 2011007126 A2 WO2011007126 A2 WO 2011007126A2 GB 2010001330 W GB2010001330 W GB 2010001330W WO 2011007126 A2 WO2011007126 A2 WO 2011007126A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- value
- processing
- generating
- linear
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10056—Microscopic image
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
Definitions
- the present invention relates to methods and apparatus suitable for use in the determination of the presence or absence of disease. More particularly, but not exclusively, the invention relates to methods for analysing corneal images to determine an indication of likelihood of disease.
- Automated image analysis may be used to reduce manual workloads in determining properties of images.
- Image analysis is now used in a variety of different fields.
- a variety of image analysis techniques are used to process medical images so as to provide data indicating whether an image includes features indicative of disease.
- Image analysis techniques for the processing of medical images in this way should be reliable both from the point of view of reliably detecting all features which are indicative of disease and from the point of view of not incorrectly detecting features which are not indicative of disease.
- CCM Corneal Confocal Microscopy
- corneal images indicating properties of the cornea can be difficult to capture effectively. Patients' eyes tend to move during image acquisition and the clarity of an image may therefore be poor.
- a method of generating data providing a quantitative indication of the presence or absence of disease from an image of a patient comprises processing the image to identify at least one linear structure, processing the at least one identified linear structure to generate data indicating properties of the at least one identified linear structure and processing the data indicating properties of the at least one identified linear structure to generate the data providing a quantitative indication of the presence or absence of disease.
- a quantitative indication may be provided.
- Providing a quantitative indication in this way allows a severity or confidence of the determination of the presence or absence of disease to be indicated.
- the quantitative indication may be used to classify the patient into categories such as no disease, mild disease, moderate disease and severe disease. The quantitative indication therefore provides additional information over prior art methods which may be useful for making clinical decisions relating to the patient.
- the generated data indicating properties of the at least one identified linear structure may comprise a plurality of data values and processing the data indicating properties of the at least one identified linear structure may comprise combining the data values.
- Each of the data values may have an associated weight, and the data values may be combined in accordance with the respective associated weight.
- the generated data indicating properties of the at least one identified linear structure may comprise a plurality of data values and processing the data indicating properties of the at least one identified linear structure may comprise processing the data values with a trained classifier to generate the quantitative indication of the presence or absence of disease.
- the quantitative indication may be a confidence of the presence or absence of disease.
- the image may be a corneal confocal microscopy (CCM) image.
- CCM corneal confocal microscopy
- the disease may be a neuropathy.
- the neuropathy may be a somatic or autonomic peripheral neuropathy.
- Processing the image to identify at least one linear structure may comprise processing the image to generate an enhanced image and processing the enhanced image to identify the at least one linear structure.
- Processing the image to generate an enhanced image may comprise generating a respective first value for each of a plurality of points in the image; generating a respective second value for each of a plurality of points in the image; and combining the first values and the second values to generate the enhanced image.
- Generating a respective first value for each of a plurality of points in the image may comprise applying a first model to the image.
- Generating a respective second value for each of a plurality of points in the image may comprise applying a second model to the image.
- Processing the image to identify at least one linear structure may comprise: processing the image at a plurality of scales to generate a plurality of values for each of a plurality of image locations; for each of the plurality of image locations, processing the plurality of values to generate an indication of whether the image location represents a linear structure in the image; and processing the generated indications of whether the image location represents a linear structure in the image to identify at least one linear structure.
- the identified linear structure may represent a nerve fiber.
- the properties of the linear structure may be indicative of properties of the nerve fiber selected from the group consisting of: fiber length, fiber density, fiber branch density, fiber width and fiber tortuosity.
- a computer implemented method for screening a population for disease comprising, at the computer: receiving a plurality of images, each image being taken from a member of said population; and processing each of the plurality of images according to the first aspect of the invention to determine a quantitative indication of the presence or absence of disease from each member of the population.
- a population may be screened and the respective quantitative indications may be used to compare members of the population.
- the comparison may be used to determine, for example, an order in which patients should be seen.
- a method of generating output data providing an indication of the presence or absence of disease from an image of a patient comprises generating a respective first value for each of a plurality of points in the image; generating a respective second value for each of a plurality of points in the image; combining the first values and second values to generate an enhanced image; processing the enhanced image to identify at least one linear structure in the enhanced image; processing the at least one identified linear structure to generate data indicating properties of the at least one identified linear structure; and generating the output data providing an indication of the presence or absence of disease based upon the data indicating properties of the at least one identified linear structure.
- the image may be a corneal image although processing other types of medical image in this way may provide a useful indication of the presence of absence of disease.
- Generating a respective first value for each of a plurality of points may comprise applying a first model to the image and generating a respective second value for each of a plurality of points may comprise applying a second model to the image.
- the point in the enhanced image may take a greater (or lesser) value than if the predetermined criterion is not satisfied. That is, the value for a point in the enhanced image may be generated using a 'step' function (or a function similar to a step function) such that if the criterion is satisfied the point in the enhanced image may have a value on a first side of the 'step' while if the criterion is not satisfied, the point in the enhanced image may have a value on the other side of the 'step'.
- Combining the first values and the second values to generate the enhanced image may comprise, for each of a plurality of points in the image: determining whether a first value for a respective point and a second value for a respective point satisfy a predetermined criterion; if the first value and the second value satisfy the predetermined criterion, setting the point to a first output value; and if the first value and the second value do not satisfy the predetermined criterion, setting the point to a second output value.
- the enhanced image can provide a relatively large value for a point if a predetermined criterion involving the first and second values is satisfied, and a relatively small value for a point if a predetermined criterion involving the first and second values is not satisfied. Contrast between structures representing nerve fibers in an image and other areas can thereby be increased based upon the first and second values and structures representing nerve fibers can thus be more easily identified and properties of the structures can be more readily determined.
- the first output value may be based upon the first value and the second value.
- the first output value may be the result of an arithmetic operation, such as addition, applied to the first and second values.
- the second output value may be a constant value, for example zero.
- the first value may be greater than the second value.
- the first value may be less than the second value.
- the predetermined criterion may therefore be based upon the relative values of the first and second values.
- the first model may be based upon an estimate of orientation of structures in the image.
- the first model may provide a respective value for each of a plurality of points in the image, and each value may be based upon an estimate of orientation at a particular point in the image.
- the method may further comprise determining an estimate of orientation of structures of the image by applying a least-squares estimation operation to the image to generate a local orientation estimate for each of a plurality of points in the image.
- Determining the estimate of orientation of structures of the image may further comprise applying a smoothing operation to the local orientation estimate, for example applying a Gaussian smoothing operation.
- the first model may be a Gabor model, although it will be appreciated that any model arranged to enhance the visibility of linear structures in an image can effectively be used.
- Processing the enhanced image to identify at least one linear structure in the enhanced image may comprise applying a threshold to the enhanced image to generate a thresholded image.
- the enhanced image may be generated in such a way that pixels representing linear structures have relatively high intensity, and a comparison of pixel intensity with a threshold can therefore be used to differentiate pixels representing linear structures from pixels not representing linear structures.
- the at least one linear structure may be a connected area in the thresholded image.
- Processing the enhanced image to identify at least one linear structure in the enhanced image may comprise identifying a plurality of linear structures in the enhanced image and processing the identified plurality of linear structures to form a single linear structure from the identified plurality of linear structures.
- Processing the identified plurality of linear structures to form a single linear structure may comprise selecting a pair of linear structures; for the selected pair of linear structures, determining a cost associated with a connection between the pair of linear structures; processing the cost associated with the selected pair of linear structures with respect to a predetermined threshold; and determining whether to form a single linear structure from the pair of linear structures based upon the processing.
- a determination can be made, for a particular pair of linear structures as to whether a connection between the pair of linear structures can be made sufficiently 'easily'.
- the term 'cost' is used herein to refer to any parameter (e.g. an energy) which can be used as a basis for determination of whether a pair of linear structures should be connected.
- Selecting a pair of linear structures may comprise selecting a plurality of pairs of linear structures from the identified plurality of linear structures; for each of the pairs of linear structures, determining a cost associated with a connection between the pair of linear structures; and selecting a pair associated with an optimal cost.
- Selecting the plurality of pairs may comprise determining a probability associated with each of a plurality of pairs of linear structures, the probability indicating a probability that the pair represent a single linear structure, and selecting the plurality of pairs based upon the determined probabilities.
- the probability may be determined based upon at least one property of the linear structures, the at least one property being selected from the group consisting of: intensity of the linear structures, size of the linear structures, distance between the linear structures, orientation of the linear structures, and orientation of a connection between the linear structures.
- a method for processing images to assess the condition of a patient comprising: processing a first image obtained from the patient at a first time using a method according to the first aspect of the invention; processing a second image obtained from the patient at a second time using a method according to the first aspect of the invention; and comparing results of processing the first image and processing the second image to assess a change in the condition of the patient between the first time and the second time.
- the fourth aspect of the invention therefore provides a method for monitoring a change in the condition of a patient over time.
- the method may be useful in assessing the natural history of a disease in longitudinal studies and in assessing the efficacy of a medicament, for example in clinical trials.
- the second image may be obtained after the patient has been subjected to therapy.
- the therapy may comprise a medicament.
- the second aspect of the invention can be used to monitor the efficacy of a treatment applied to the patient.
- a method of enhancing a corneal image comprising: generating an estimate of orientation of structures of the corneal image; and enhancing linear structures in the corneal image based upon the estimate of orientation of structures.
- the inventors have found that enhancing a corneal image based upon estimates of orientation of structures in the corneal image allows a corneal image to be enhanced in such a way that linear structures can be clearly identified.
- Enhancing linear structures in the corneal image may comprise applying a first model to the corneal image to generate a respective first value for each of a plurality of points in the corneal image; applying a second model to the corneal image to generate a respective second value for each of a plurality of points in the corneal image; and combining the first values and the second values to generate the enhanced corneal image.
- the estimate of orientation of structures of the image may be generated by applying a least-squares estimation operation to the corneal image to generate a local orientation estimate for each of a plurality of points in the corneal image.
- Determining the estimate of orientation of structures of the image may further comprise applying a smoothing operation to the local orientation estimate.
- the first model may be applied to the corneal image based upon the estimate of orientation of structures.
- a method of generating output data providing an indication of the presence or absence of disease from a corneal image comprises identifying an elongate structure representing a nerve fibre in the corneal image, the elongate structure defining a path; generating data based upon a part of the elongate structure, the data indicating a property of the elongate structure in a direction transverse to the path of the part of the elongate structure; and generating the output data providing an indication of the presence or absence of disease based upon the generated data.
- a measure transverse to the path of a nerve fiber can be used to determine an indication of the presence or absence of disease from a corneal image.
- Generating data based upon the part of the elongate structure may comprise determining a width of the part of the elongate structure.
- Generating data based upon the part of the elongate structure may comprise determining an intensity at a plurality of points of the elongate structure along the direction transverse to the part of the elongate structure. In this way an intensity profile, or variance in intensity across a nerve fiber may be used in the determination of the presence or absence of disease.
- the direction transverse to the part of the elongate structure may be based upon an estimate of orientation of the part of the elongate structure.
- the direction transverse to the part of the path may be substantially perpendicular to a direction indicated by the estimate of orientation of the part of the elongate structure.
- the data indicating a property of the elongate structure may therefore be a cross section of the elongate structure.
- the method may further comprise: generating respective data based upon each of a plurality of parts of the elongate structure, each of the respective data indicating the property of the elongate structure in a direction transverse to the path of a respective one of the plurality of parts of the elongate structure; and the output data may be based upon an average of the respective data. Determining a property at a plurality of locations along an elongate structure in this way avoids anomalous results that may be provided by a single measurement.
- Generating data based upon a part of the elongate structure may comprise: determining a length of the elongate structure; determining an area of the elongate structure; and the data may be based upon the determined length and the determined area. In this way an indication of an average width of the elongate structure may be determined based upon an average number of pixels per pixel length.
- the method may further comprise: identifying a plurality of elongate structures, each elongate structure representing a nerve fibre in the corneal image; generating respective data based upon a part of each of the elongate structures, each respective data indicating a property of a respective elongate structure in a direction transverse to the path of the part of the respective elongate structure; and generating the output data based upon the generated respective data.
- identifying a plurality of elongate structures in an image allows the determination of the presence or absence of disease to be based upon an overall property of elongate structures across the image.
- Generating the output data may comprise generating data indicative of a distribution of the generated data. For example the distribution of generated data from each elongate structure in the image may be considered and the distribution may provide an indication of disease. The distribution may be processed using a trained classifier.
- Generating the output data providing an indication of the presence or absence of disease may comprise: generating further data based upon the part of the elongate structure, the further data indicating a property of the elongate structure in a direction along the path of the part of the elongate structure; and comparing the data and the further data. For example the variance in intensity across an elongate structure compared to intensity along an elongate structure may provide an indication of the presence or absence of disease.
- Identifying the linear structures may comprise enhancing the corneal image using a method according to the second aspect of the invention.
- the corneal images may be corneal confocal microscopy images.
- the disease may be a nerve disorder.
- nerve disorders are neuropathies, for example peripheral neuropathies such as diabetic neuropathy.
- aspects of the invention can be implemented in any convenient form. For example computer programs may be provided to carry out the methods described herein. Such computer programs may be carried on appropriate computer readable media which term includes appropriate tangible storage devices (e.g. discs). Aspects of the invention can also be implemented by way of appropriately programmed computers.
- Figure 1 is a schematic illustration of a system for analysis of images according to an embodiment of the present invention
- Figure 1A is a schematic illustration showing a computer of the system of Figure 1 in further detail
- Figure 2 is an image suitable for processing using the system of Figure 1 ;
- Figure 3 is a flowchart showing processing carried out to analyse an image in the system of Figure 1 ;
- Figure 4 is a flowchart showing processing to enhance an image
- Figure 5 is a flowchart showing a part of the processing of Figure 4 in further detail
- Figure 6 is a flowchart showing processing to extract parts of an image showing nerve fibers
- Figure 7 is a flowchart showing processing to connect linear structures in an image.
- Figure 8 is a schematic illustration of part of an image in which it is desirable to determine a width of a nerve fiber at a particular point on the nerve fiber.
- an image capture device 1 is arranged to capture a digital image 2 of an eye 3.
- the digital image 2 is a Corneal Confocal Microscopy (CCM) image showing features of the cornea of the eye 3.
- the image 2 is stored in a database 4 for processing by a computer 5.
- Images such as the image 2 of Figure 1 may be collected from a population for screening for a disease such as, for example, a peripheral neuropathy such as somatic neuropathy or autonomic neuropathy. Additionally, images such as image 2 of Figure 1 may be collected from a patient at intervals to assess deterioration of a condition of the patient or to determine the efficacy of any therapy given to the patient to improve the condition.
- the image capture device 1 may be a CCM camera such as a Heidelberg Retina Tomograph (HRT3) manufactured by Heidelberg Engineering, Heidelberg, Germany, or any image acquisition device suitable for capturing a CCM image of an eye.
- HRT3 Heidelberg Retina Tomograph
- Figure 1A shows the computer 5 in further detail.
- the computer comprises a CPU 5a which is configured to read and execute instructions stored in a volatile memory 5b which takes the form of a random access memory.
- the volatile memory 5b stores instructions for execution by the CPU 5a and data used by those instructions.
- the digital image 2 may be stored in the volatile memory 5b.
- the Computer 5 further comprises non-volatile storage in the form of a hard disc drive 5c.
- the digital image 2 may be stored on the hard disc drive 5c.
- the computer 5 further comprises an I/O interface 5d to which are connected peripheral devices used in connection with the computer 5. More particularly, a display 5e is configured so as to display output from the computer 5. The display 5e may, for example, display a representation of the digital image 2. Additionally, the display 5e may display images generated by processing of the digital image 2. Input devices are also connected to the I/O interface 5d. Such input devices include a keyboard 5f and a mouse 5g which allow user interaction with the computer 5.
- a network interface 5h allows the computer 5 to be connected to an appropriate computer network so as to receive and transmit data from and to other computing devices.
- the CPU 5a, volatile memory 5b, hard disc drive 5c, I/O interface 5d, and network interface 5h are connected together by a bus 5i.
- FIG. 2 a CCM image of the cornea of a patient suitable for processing by the computer 5 of Figure 1 is shown.
- the CCM image shows clearly defined nerve fibers 7 as well as other linear structures that are less clear such as linear structure 8 which may indicate a nerve fiber at a deeper level in the cornea than nerve fiber 7, or may be noise in the image.
- Linear structure 9 appears in an area of noise in the image and is difficult to identify.
- Circled areas 10 indicate parts of relatively clear linear structures where a clear linear structure becomes hard to identify or disappears. For example, sections 11a and 11 b appear to be two parts of the same nerve fiber, however in the area 10 the exact path followed by the linear structure is not clear and is difficult to determine.
- step S1 a CCM image of dimensions M x N pixels is received and at step S2 the image is processed to enhance the quality of the image. Processing to enhance the quality of an input image is described in further detail below with reference to Figures 4 and 5.
- step S3 parts of the enhanced image representing nerve fibers are identified. Identification of parts of an image representing nerve fibers is described in further detail below with reference to Figure 6.
- step S4 the identified parts of the image representing nerve fibers are quantified according to measures of peripheral neuropathy as will be described in further detail below with reference to Figure 7.
- step S5 it is determined whether the identified parts of the image representing nerve fibers are indicative of peripheral neuropathy based upon the quantification of step S4.
- step S6 an image P is generated by processing the input image to remove high frequency noise components in the image and to normalise the image intensity such that all pixel values are in the range 0 to 1.
- the pre-processing of step S6 is not intended to affect linear structures in the image but instead to only remove background noise.
- an estimate of the orientation of each of a plurality of blocks of the input image is determined.
- two orthogonal gradient components are determined, and these components are combined to provide an estimate of orientation.
- These estimates can be represented in the form of an image ⁇ , where each pixel in the image has an orientation value determined by a gradient estimate generated from a block centred at that pixel.
- the estimates of orientation are then smoothed to generate a further image ⁇ .
- the pre-processed image P output from step S6 is processed by taking, for each pixel (i,j) a block of pixels centred on the pixel (i,j) and determining an orientation estimate for that pixel (i,j) from pixel values within the block of pixels
- the estimates of orientation are determined as set out in equations (1) to (3) below. More particularly an x-gradient component is determined using equation (1), a y- gradient component is determined using equation (2) and the x-gradient and y- gradient components are combined according to equation (3) to provide an estimate of orientation for the particular pixel.
- ⁇ is the width of the block centred at pixel (i,j).
- ⁇ x (u,v) and 9 y (u,v) are the gradients computed at each pixel (u,v) for u in the range ⁇ - j to , + ⁇ and v in the range j - ⁇ to j + ⁇ .
- the gradients ⁇ x (u,v) and ⁇ y (u,v) may be computed in any convenient way, for example using a Sobel operator or Canny operator.
- Values of the image ⁇ are processed to extract x and y vector field components, which are then filtered using a low-pass Gaussian filter G.
- ⁇ y (U) £ lG(u,v) ⁇ ,(i-U,i-v) (7) ⁇ ⁇ ⁇ ⁇
- ⁇ ⁇ x co ⁇ is the size of the kernel of G.
- the estimated orientation ⁇ may not be correct in all cases due to the local nature of the estimate. Applying the low-pass filter G therefore corrects some errors given that the orientation varies slowly.
- the low-pass Gaussian filter G is applied to each pixel in the image ⁇ in turn to reduce errors at near-nerve fiber and non-nerve fiber regions.
- An estimate of local orientation of structures ⁇ is determined from the smoothed vector field components according to equation (8).
- the image ⁇ provides an acceptably accurate estimate of gradient direction for each pixel of the original image.
- step S8 linear structures in the pre-processed image output from step S6 are enhanced using the estimates of local orientation of structures ⁇ output from step S7.
- Linear structure enhancement is described in further detail below with reference to Figure 5.
- An image may not be suitable for further processing if, for example, the size of linear structures in the image is below a particular size (i.e. if linear structures comprise less than a particular number of pixels), which is taken as an indication that the image is of a layer of the cornea which does not contain nerve fibers but is instead a layer of the cornea containing cells.
- Quantitative features of an image may be used to determine a quantitative indicator of validity, used to determine if an image is valid.
- the quantitative indicator may be determined using a trained classifier such as a support vector machine to classify images as suitable or not suitable for further processing based upon a distribution of sizes of linear structures in the image.
- step S9 determines that the image output from step S8 is suitable for processing to determine the presence or absence of disease
- the image is processed to extract nerve fibers as is described below with reference to Figure 6.
- the processing to enhance linear structures of step S8 of Figure 4 is shown in further detail in Figure 5.
- the enhancement of linear structures uses the estimate of local orientation of structures - determined according to equation (8). It will be recalled that the estimate of local orientation of structures ⁇ is an image in which each pixel represents an orientation associated with a corresponding pixel of the image P. In areas of the image P representing nerve fibers the image ⁇ has lines of substantially constant values. In background areas of the image P (i.e. areas that are not representative of nerve fibers) pixel values of the image ⁇ provide a noisy image due to the lack of structure and uniform direction in background areas.
- the image ⁇ is used as an input to a foreground model h, with which the image P is combined to enhance linear structures.
- the image P is additionally combined with a background model of a type described below.
- the pre-processed image P is received and at step S16 the local orientation estimate ⁇ generated at step S7 of Figure 4 as described above is received.
- the foreground model h is generated based upon the local orientation estimate ⁇ .
- the foreground model h is a 2D Gabor filter defined by equation (9): where:
- x, y are respectively horizontal and vertical indices of a pixel of the input image
- ⁇ is the orientation of the Gabor filter h, as determined by the local orientation estimation ⁇ at the pixel x,y received at step S16;
- f is the frequency of the cosine wave
- ⁇ x and ⁇ y are the standard deviations of the Gaussian envelope along the x and y axis respectively;
- ⁇ is an offset of the cosine function which generally takes a value 0 (i.e. no offset is applied), although other values may be used where it is desirable to apply an offset to the model h;
- ⁇ is a parameter which allows the aspect ratio of the foreground model h to be adjusted.
- the Gabor filter is a cosine function modulated by a Gaussian function.
- the value f defines the number of cycles of the cosine function within a predetermined distance.
- the value f is selected such that the spatial extent of a single cycle of the cosine function is approximately the same width as a linear structure that it is desirable to enhance (i.e. a nerve fiber).
- the Gaussian function determines the contribution of pixels at varying distances from a pixel which is currently being processed.
- the values ⁇ x and ⁇ y define the widths of the Gaussian function in the x and y directions respectively The larger the value ⁇ x , the greater the contribution of pixels at some distance from a particular pixel which is currently the subject of processing.
- the value ⁇ defines the ratio between the widths ⁇ x and ⁇ y .
- ⁇ has a value 1
- the Gaussian function is symmetric, i.e. has equal width in both the x and y dimensions, and where ⁇ has any other value, the Gaussian function is asymmetric, i.e. has different widths in the x and y dimensions.
- An asymmetric Gaussian function may be advantageous in some embodiments as this may magnify the response of the function to nerve fibers, given the linear nature of the nerve fibers.
- the values X 9 and y ⁇ define respectively the x and y-axes of the filter coordinate frame after a clockwise rotation of the Cartesian axis by an angle of ( ⁇ /2 - ⁇ ).
- the Gabor filter is rotated so that its local x-axis is perpendicular to the fiber direction and is applied across the direction of the fibers as determined by the local orientation estimation ⁇ .
- f 1/9
- an image P h is generated by pixel-wise convolving the foreground model h with the image P.
- the Gabor filter h enhances parts of the image P representing nerve fibers that are oriented in the direction of the filter (as defined by ⁇ ), and reduces visibility of parts of the image P having an orientation different to ⁇ .
- Applying the filter h to the image P increases the contrast between the foreground and the "noisy" background, as well as reducing noise around the fiber structure. Additionally the filter h increases the likelihood of detecting nerve fibers that are not represented clearly in the image P due to the prior knowledge of the general structure of the fiber and the assumption of continuous flow of the fibers along a particular known direction as determined from the local orientation estimation ⁇ .
- model h is a Gabor model
- any model arranged to enhance the visibility of linear structures in an image can be used, for example a model such as the model described in R.N. Dixon and CJ. Taylor “Automated Asbestos Fibre Counting", Machine Aided Image Analysis, Institute of Physics, London, 1979, pp.178-185 may be used.
- the background model g is a Gaussian kernel generated according to equation (10): where:
- x, y are respectively horizontal and vertical indices of a pixel of the input image
- ⁇ x1 , ⁇ y i are the spread in, respectively, the x and y direction of the kernel; and ⁇ is a weight that may be applied to g, although ⁇ generally takes a value 1
- an image P 9 is generated by pixel-wise convolving the image P with the background model g.
- the image P 9 is a smoothed version of the image P. It will be appreciated that background models other than the Gaussian kernel of equation (10) that smooth an image can be used.
- a check is performed for each pixel (x,y) to determine whether the value of the image P h (x,y) is greater than the value of the image P g (x,y). For each pixel for which the value P h (x,y) is greater than the value P g (x,y) a corresponding pixel in an image S is generated according to equation (11):
- k is a parameter that controls the sharpness of the change between a pixel being highlighted (i.e. treated as foreground) and being reduced to a constant value of zero (i.e. treated as background), which occurs when F g (x,y) is equal to zero, i.e. when the foreground response, h(x,y) and the background response, g(x,y) are equal.
- step function of equation (11a) provides an image S that is an approximation to the image S generated at step S21 above using logical statements and equation (11) given that R n takes a positive value where h is greater than g and a negative value or zero otherwise.
- the denominator of the right hand side of equation (11a) tends to 1 where h is greater than g, and S(x,y) therefore also tends to 1 , and the denominator tends to infinity otherwise causing S(x,y) to tend to 0.
- generation of the image S of step S21 may be more computationally efficient to implement than implementation of equation (11a).
- a Gabor filter is used as a foreground model. It will however be appreciated that wavelets other than the Gabor filter, for example the Daubechies wavelet, can be used. Additionally wavelets having different frequencies could be applied to the image and averaged or the values of f, ⁇ x and ⁇ y could be adaptively set according to appropriate local values determined by pre-processing the image. It will be appreciated that some of the processing shown in Figure 5 can be carried out in parallel. In particular, S17 and S18 can be carried out in parallel with steps S19 and S20.
- the identification of parts of the image representing nerve fibers at step S3 of Figure 3 is shown in further detail in Figure 6.
- the identification of parts of the image representing nerve fibers generally comprises a pre-processing part A in which a skeletonised binary image ⁇ is generated, followed by iterative processing B using the skeletonised binary image ⁇ .
- the iterative processing B is described in further detail below with reference to Figure 7.
- a binary image T is generated by thresholding the image S. Areas of the image S that are greater than the threshold are set to a value '1 ' and areas that are not greater than the threshold are set to a value 1 O'.
- pixels in the image T having a value of '1' which do not, together with other pixels having a value of '1' form an area exceeding a predetermined size are set to a value 1 O'.
- the size of an area is determined by the number of connected pixels in the area and the number of pixels is compared to a threshold to determine if the area is smaller than the predetermined size.
- step S27 the image output from step S26 is dilated and at step S28 the image output from step S27 is eroded.
- the dilation and erosion operators of steps S27 and S28 open up the binary image and fill small gaps between and within areas representing nerve fibers. Dilation and erosion can be carried out using any suitable structuring element, for example a square 3x3 structuring element.
- step S29 the image output from step S28 is skeletonised to generate the image ⁇ .
- the iterative processing B of Figure 6 is intended to identify discrete linear structures in the processed image which represent a single linear structure in the cornea. Such identified discrete linear structures are connected areas.
- the processing described with reference to Figure 7 is concerned with identifying which structures to combine.
- a model ⁇ is initialised based upon the enhanced image S and the local orientation estimation ⁇ .
- the model ⁇ is defined as follows:
- ⁇ i ⁇ e m
- ⁇ is the magnitude of S at pixel i
- pi is the value of local orientation estimation ⁇ at pixel i
- j is the complex number V-1.
- each ⁇ s is a connected area of pixels having value 1 in the image ⁇ . Given the processing described above, it is assumed that each connected area represent a linear structure in the cornea.
- Each ⁇ s is defined as follows:
- p r A s is the relative size of the area ⁇ s determined by dividing the number of pixels in ⁇ Sl by the number of pixels in the image, M x N.
- a pair of linear structures ⁇ s, ⁇ are selected such that the pair ⁇ s , ⁇ , has not previously been selected.
- a probability ⁇ s t of a connection between the selected structures is determined. The probability of a connection ⁇ s t in ⁇ is defined as follows:
- ⁇ s t ⁇ ' g p ⁇ s p ⁇ s t ⁇ 'l. p ⁇ 5 ,. t , ⁇ 6 s t j where each of the ⁇ ' s t , 1 ⁇ i ⁇ 6, are in the range 0 to 1 and where:
- ⁇ ' 8i , p ⁇ where p£ ; and p£ ; are the values of the pixels represented by the pixel descriptors q s and q t respectively in the image S, where q s represents a pixel in ⁇ s and q t represents a pixel in ⁇ , and q s and q, are selected such that the Euclidean distance between the pixels represented by the pixel descriptors is minimal across all possible pixels in ⁇ s , ⁇ t ;
- ⁇ 2 s t p ⁇ s; p2 ; where p ⁇ and p ⁇ ; are the relative sizes of the areas ⁇ s and ⁇ t respectively; ⁇ 3 s , where
- q s -q t defines the distance between the pixels represented by the pixel descriptors q s and q t (used in the determination of ⁇ ' s t ) and D is the maximum possible distance between points in the image;
- ⁇ wm[( ⁇ u - ⁇ s )+ ⁇ -( ⁇ u - ⁇ s )+ ⁇ ];
- the value ⁇ ' s t is such that linear structures with high intensity are more likely to be connected to each other than linear structures with low intensity.
- the value ⁇ 2 s t is such that larger linear structures are more likely to be connected than smaller linear structures.
- the value ⁇ 3 s t is such that linear structures that are located close together are more likely to be connected than linear structures that are spaced far apart.
- the value ⁇ 4 s is such that linear structures with ends with similar orientations are more likely to be connected than linear structures with ends with different orientations.
- the value ⁇ 5 s t is such that if the linear structures s, t are connected in the processed image then no further connection will be made.
- the value ⁇ 6 s t is such that if a connecting line between the ends of two linear structures has a similar orientation to the orientation of the two ends of the linear structures a connection is more likely to be made than if the connecting line has a different orientation to the orientation of the ends of the two linear structures.
- step S34 it is determined if there are more pairs of linear structures for which a probability of a connection has not been determined. If it is determined that there are more pairs of linear structures then processing returns to step S32 where a further pair is selected for processing. If it is determined that all possible pairs of linear structures s, t have been processed to determine a probability ⁇ s t then at step S35 a set ⁇ of the most likely connection for each linear object ⁇ s is determined by determining a linear structure ⁇ t for which the value ⁇ s t is a maximum for the linear object ⁇ s and adding the connection (s,t) to the set ⁇ .
- step S36 the minimum energy e s t associated with a connection between each of the pairs of linear structures (s,t) in ⁇ is determined according to equation (12) below: where:
- the value e s , t is the energy required to make a direct connection between pixels q s and q,.
- step S39 the image ⁇ is output. If it is determined at step S38 that e s t is smaller than e thre s h then at step S40 the image ⁇ is updated to include the connection u s , and the two linear structures s, t become a single linear object.
- step S41 the set of linear structures I " is updated based upon the updated image ⁇ and joined linear structures s, t. Processing passes from step S41 to step S31 and the processing to determine connections in the image is repeated based upon the updated model.
- the image ⁇ output from step S39 (updated with any connections whose energy does not exceed the threshold e thr es h ) is used to determine nerve fibers in the area of the eye represented by the CCM image received at step S1. Areas with a value '1 ' are determined to be nerve fibre and areas with a value O' are determined to be background.
- an input image 2 may be processed to generate a plurality of scaled images 12.
- the input image 2 and each of the scaled images 12 may each be processed using the two models of equations (9) and (10).
- the model of equation (9) is applied at a plurality of different orientations to each of the input image 2 and each of the scaled images 12 to generate a respective plurality of images indicated by one of S ⁇ x.y) to S n (X, y).
- a vector of pixel values 13 may be determined for each pixel 14 (corresponding to one of the pixels of the input image 2).
- the vector of pixel values 13 comprises values taken from each of the images in each of the plurality of images Si(x,y) to S n (X, y).
- the vector of pixel values 13 for each pixel may be processed using a trained classifier 15 to provide an output 16 indicating whether the pixel is part of a linear structure and indicating the orientation of the linear structure of which the pixel is a part.
- an input image is received.
- the input image is smoothed, for example using a Gaussian smoothing operation and at step S53 the image is sub-sampled such that the image comprises fewer pixels than the input image.
- Sub-sampling may be carried out in any convenient way, for example by taking an average of pixel values which are used to form the basis of a pixel in the sub-sampled image.
- the processing of steps S52 and S53 is repeated a plurality of times to generate a plurality of images, each generated image being based upon a previous smoothed and sub-sampled image.
- step S54 it is determined whether a predetermined number of iterations of steps S52 to S54 have been carried out to generate respective smoothed and sub-sampled images. If the predetermined number of iterations has not been carried out then processing continues at steps S52 and S53 where a further smoothed and sub-sampled image is generated based upon the previous smoothed and sub-sampled image. Otherwise at step S55 the input image and each smoothed and sub-sampled image output from step S53 is processed to generate a plurality of images S(x,y) as described above. Each of the plurality of images S(x,y) for each image is generated using a respective one of a predetermined number of orientations ⁇ for the foreground model h and background model g. For example, eight orientations may be used to generate eight images.
- step S56 pixel values are determined for each pixel in the original image based upon each of the images S(x,y) generated at step S55 (where each image S(x,y) has a respective scale, and orientation associated with the model h).
- the images output from step S53 are sub-sampled and contain fewer pixels than the input image, and correspondingly the images S(x,y) contain fewer pixels than the input image
- values for pixel locations in the original image are determined based upon values for pixels in the image S(x,y) that represent the pixel location of the pixel in the original image. Pixel locations that are represented by a pixel in the image S(x,y) may be determined based upon the sub-sampling of step S53.
- the image value at a particular pixel in the output image S n (x,y), that has been generated by smoothing and sub-sampling at steps S52 and S53, will consist of contributions from a plurality of pixels in the input image.
- the number of pixels in the input image that contribute to a particular pixel value in the generation of S n (x,y) at step S55 will increase as n increases, i.e. as more smoothing is applied to the input image.
- a plurality of pixel locations in the input image take a value based upon a particular pixel in the image output from step S53, with a pixel in an image S n (X, y) that is generated based upon an image that has been generated by smoothing and sub-sampling the input image a large number of times providing a value to a relatively large number of pixel locations in the input image relative to an image Si(x,y) that is generated based upon the input image (or an image that has been generated by smoothing and sub-sampling the input image a relatively small number of times).
- a feature vector is generated for each pixel in the input image, where values of the feature vector are the values determined at step S56 from each of the images S(x,y).
- each feature vector comprising values for a pixel location in the original image is processed using a trained classifier which provides an indication of whether a pixel represents a linear structure in the image, together with an indication of an orientation of any linear structure represented by the pixel.
- a set of linear structures F can be determined from the output of the classifier, based upon classifications of pixels in the input image determined by the classifier.
- properties of nerve fibers in the input image can be determined based upon the set of identified linear structures F.
- the following criteria are determined from the image ⁇ in order to determine an indication of the presence or absence of disease in the patient from whom the corneal image was captured:
- Each of the parameters can be determined in any convenient way from the nerve fibers identified by the processing of Figure 3.
- fiber length may be determined by calculating the total number of pixels in image ⁇ .
- Fiber density may be determined by calculating a count of the number of fibers in the image or a maximum fiber density in an area of a predetermined size.
- Fiber branches may be identified based upon locations at which fibers coincide. The number of such locations may be summed to give a total number of branches per image.
- Fiber tortuosity may be determined according to the method described in Corneal Nerve Tortuosity in Diabetic Patients with Neuropathy, Kallinikos et. al., Investigative Ophthalmology & Visual Science, February 2004, Vol. 45, No. 2, or in any other suitable way.
- the measure of fiber width may be generated directly from the image 2 of Figure 1 or from any other suitable image generated from the image 2, for example from the pre- processed image P or the binary image T. Additionally, nerve fibers may be identified in one image, such as the skeletonised image ⁇ , and the width of identified nerve fibers may be measured in another image, such as the image 2 of Figure 1 or an enhanced image based upon image 2. Measuring nerve fibers in image 2, or an enhanced image based upon image 2, may result in a more accurate indication of nerve fiber width.
- a measure of a width of a nerve fiber at a particular point on the nerve fiber may be determined in any convenient way.
- width of a nerve fiber may be determined using the estimate of local orientation of structures ⁇ .
- Figure 9 shows part of an image 2 in which a width of a nerve fiber 17 at a particular point q is indicated by cross section 18. The direction in which to measure the width, indicated by cross section 17, is selected so as to be perpendicular to the direction indicated by the local orientation of structures 19 of the point q.
- One suitable measure of fiber width at a particular point on a nerve fiber is a number of sequential pixels along the cross section 18 whose pixel value each exceed a predetermined threshold.
- the number of sequential pixels may be determined by thresholding the image S in which linear structures have been enhanced such that all values in the image S having a value less than the threshold are set to a constant value (for example 0).
- the number of pixels along the cross section 18 which have a value different to the constant value may be counted so as to provide a measure of fiber width.
- an indication of width of a nerve fiber at a particular point on a nerve fiber may be determined by processing values along cross section 18 in the input image 2 or an image generated from the input image 2 (for example, a Gaussian smoothed image generated from the input image 2) to generate second derivative values of values along the cross section 18.
- Edges of the nerve fiber along the cross section 18 are located at points having maximal gradient (that is points at which first derivative values are maximal), and as such are indicated as zero values in the second derivative values.
- a value indicating a distance between points having second derivative values of zero is therefore also indicative of width of the nerve fiber along the cross section 18 and can be used as an alternative to the number of pixels along the cross section 18 described above.
- the measures of width of nerve fiber described above can be used to determine an average width of a nerve fiber by determining the width of the nerve fiber at each possible point along the nerve fiber according to one of the methods (each possible point being determined, for example, by a pixel in the skeletonised image ⁇ ), or by determining the width of the nerve fiber at a plurality of intervals along the nerve fiber, and averaging the determined widths.
- An indication of an average fiber width in the image may be generated, determined by processing a plurality of widths, each width being determined from a respective nerve fiber identified in the image.
- an average width of a nerve fiber may be determined by determining the length of the nerve fiber I n , for example by determining the number of pixels in the skeletonisation of the nerve fiber in the image ⁇ , determining the total number of pixels which exceed a predetermined threshold in the nerve fiber t n in a non-skeletonised image (for example the image P or the binary image T) and dividing t n by I n .
- the value Vlkir provides an indication of the average width, in pixels, across the length of the nerve fiber. Determining a total length of all fibers in the image and a total number of pixels of all fibers in the image allows an average fiber width across an image to be determined in a similar way.
- the measure of fiber width may be a measure indicative of fiber width distribution such as, for example, a histogram indicating the distribution of measured widths in the image.
- the histogram may indicate average widths determined for each nerve fiber or may indicate a plurality of measured widths determined from each of a plurality of nerve fibers.
- the distribution of the measured widths of nerve fibers in the histogram may be used to determine an indication of the presence or absence of disease in the patient from whom the corneal image was captured, for example by training a classifier on a training set of images to classify fiber width distributions as indicative of the presence or absence of disease.
- low fiber width values are indicative of nerve damage or repair and therefore may be associated with disease or recovery from disease, respectively.
- nerve fiber width distribution indicates that a disproportionately large number of nerve fiber widths are relatively low (as indicated by a distribution which is asymmetric, with a greater number of fiber widths having a width lower than an average fiber width than fiber widths having a width higher than an average fiber width), this may be indicative of nerve fiber beading and may be indicative of neuropathy.
- the measure of fiber width may be based upon pixel intensities determined in a direction transverse the nerve fiber.
- the direction transverse the nerve fiber may be determined based upon the estimate of local orientation of structures ⁇ at the point at which the measure of fiber width is determined.
- a sum of pixel values along cross section 18 of Figure 9 may be generated and used as an indication of fiber width at a particular point on a nerve fiber.
- the distribution of sums of pixel intensities along a plurality of cross sections 18 at a plurality of points in the image may be classified using a classifier trained on a training set of images to classify pixel intensity distributions as indicative of the presence or absence of disease.
- Distribution of fiber width based upon pixel intensities may again be used to indicate, for example, nerve fiber beading.
- the classifier may further use pixel intensity distribution in a direction along the path of the nerve fiber, as indicated by the estimate of local orientation of structures ⁇ , and may classify images based upon a comparison of variation in pixel intensity across a nerve fiber and along a nerve fiber.
- pixel intensities in a direction transverse the nerve fiber may be processed to generate a plot of pixel values along the cross section 18.
- the plot of pixel values may be integrated to determine a value indicative of an area under the plot and the value indicative of an area may be used as a value indicative of width.
- any suitable image can be used, including, without limitation, the input image, and images generated from the input image by or during the processing set out above.
- each of short fiber length, low fiber density, low fiber branch density and high fiber tortuosity are indicators of neuropathy and these indicators are combined to generate the indication of the presence or absence of disease.
- the parameters may be combined in any convenient way. For example a manually classified training set of images may be processed to generate a set of weights for the criteria and the generated set of weights applied to values generated from an image captured from a patient to output a value indicating a confidence that the patient has neuropathy.
- the confidence may be used as a measure of the severity of neuropathy of the patient and used to assess regeneration of the nerve and improvement of neuropathy in the patient, for example in response to a treatment.
- a classifier may be trained on a training set of images to classify images as indicative of the presence or absence of disease based upon some or all of the measures indicative of fiber length, fiber density, fiber branch density, fiber width and fiber tortuosity, determined from a previously unseen image.
- FIG. 3 The processing of Figure 3 has been described with reference to a single image. It will be appreciated however that the presence or absence of disease may be determined with reference to a plurality of images captured from a patient. For example an average of each of the measures described above may be calculated across all valid images captured from a patient. Additionally each of the images may have an associated weight applied to the values determined from the image. Weights for images may be determined based upon a quantitative indicator of image validity.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Eye Examination Apparatus (AREA)
Abstract
A method of generating data providing a quantitative indication of the presence or absence of disease from a corneal image of a patient. The method comprises processing the image to identify at least one linear structure, processing the at least one identified linear structure to generate data indicating properties of the at least one identified linear structure and processing the data indicating properties of the at least one identified linear structure to generate the data providing a quantitative indication of the presence or absence of disease.
Description
DISEASE DETERMINATION
The present invention relates to methods and apparatus suitable for use in the determination of the presence or absence of disease. More particularly, but not exclusively, the invention relates to methods for analysing corneal images to determine an indication of likelihood of disease.
Automated image analysis may be used to reduce manual workloads in determining properties of images. Image analysis is now used in a variety of different fields. In particular, a variety of image analysis techniques are used to process medical images so as to provide data indicating whether an image includes features indicative of disease. Image analysis techniques for the processing of medical images in this way should be reliable both from the point of view of reliably detecting all features which are indicative of disease and from the point of view of not incorrectly detecting features which are not indicative of disease.
Features of the cornea of the eye can be used to determine indications of disease, in particular peripheral neuropathy. One type of corneal image that may be used to determine indications of disease is a Corneal Confocal Microscopy (CCM) image. CCM is highly sensitive and images are prone to noise. Additionally, CCM images capture only a single two-dimensional layer of the cornea and features that are indicative of disease may move between two-dimensional layers. Breaks in a feature shown in a CCM image may occur when the feature moves between a captured layer and other layers of the cornea. An automated system that is able to properly identify features of a CCM image that are useful in determining indications of disease is therefore desirable but has not, heretofore, been available.
Furthermore, corneal images indicating properties of the cornea can be difficult to capture effectively. Patients' eyes tend to move during image acquisition and the clarity of an image may therefore be poor.
It is an object of some embodiments of the present invention to obviate or mitigate at least some of the problems set out above.
According to a first aspect of the invention there is provided a method of generating data providing a quantitative indication of the presence or absence of disease from an image of a patient. The method comprises processing the image to identify at least one linear structure, processing the at least one identified linear structure to generate data indicating properties of the at least one identified linear structure and processing the data indicating properties of the at least one identified linear structure to generate the data providing a quantitative indication of the presence or absence of disease.
That is, whereas prior art methods simply provide an indication of presence or absence of disease, the inventors have realised that by combining data indicating properties of at least one linear structure in an image, a quantitative indication may be provided. Providing a quantitative indication in this way allows a severity or confidence of the determination of the presence or absence of disease to be indicated. For example, the quantitative indication may be used to classify the patient into categories such as no disease, mild disease, moderate disease and severe disease. The quantitative indication therefore provides additional information over prior art methods which may be useful for making clinical decisions relating to the patient.
The generated data indicating properties of the at least one identified linear structure may comprise a plurality of data values and processing the data indicating properties of the at least one identified linear structure may comprise combining the data values.
Each of the data values may have an associated weight, and the data values may be combined in accordance with the respective associated weight.
The generated data indicating properties of the at least one identified linear structure may comprise a plurality of data values and processing the data indicating properties of the at least one identified linear structure may comprise processing the data
values with a trained classifier to generate the quantitative indication of the presence or absence of disease.
The quantitative indication may be a confidence of the presence or absence of disease.
The image may be a corneal confocal microscopy (CCM) image. Use of CCM images provides a non-invasive way of detecting nerve damage. Additionally, CCM enables nerve damage to be detected earlier than prior art methods and allows neuropathy to be detected in patients earlier than other methods.
The disease may be a neuropathy. The neuropathy may be a somatic or autonomic peripheral neuropathy.
Processing the image to identify at least one linear structure may comprise processing the image to generate an enhanced image and processing the enhanced image to identify the at least one linear structure.
Processing the image to generate an enhanced image may comprise generating a respective first value for each of a plurality of points in the image; generating a respective second value for each of a plurality of points in the image; and combining the first values and the second values to generate the enhanced image.
Generating a respective first value for each of a plurality of points in the image may comprise applying a first model to the image. Generating a respective second value for each of a plurality of points in the image may comprise applying a second model to the image.
For points of the image representing linear structures, the first value may be greater than the second value. For points of the image not representing linear structures, the first value may be less than the second value.
Processing the image to identify at least one linear structure may comprise: processing the image at a plurality of scales to generate a plurality of values for each of a plurality of image locations; for each of the plurality of image locations, processing the plurality of values to generate an indication of whether the image location represents a linear structure in the image; and processing the generated indications of whether the image location represents a linear structure in the image to identify at least one linear structure.
The identified linear structure may represent a nerve fiber.
The properties of the linear structure may be indicative of properties of the nerve fiber selected from the group consisting of: fiber length, fiber density, fiber branch density, fiber width and fiber tortuosity.
According to a second aspect of the invention there is provided a computer implemented method for screening a population for disease, the method comprising, at the computer: receiving a plurality of images, each image being taken from a member of said population; and processing each of the plurality of images according to the first aspect of the invention to determine a quantitative indication of the presence or absence of disease from each member of the population.
In this way, a population may be screened and the respective quantitative indications may be used to compare members of the population. The comparison may be used to determine, for example, an order in which patients should be seen.
According to a third aspect of the invention there is provided a method of generating output data providing an indication of the presence or absence of disease from an image of a patient. The method comprises generating a respective first value for each of a plurality of points in the image; generating a respective second value for each of a plurality of points in the image; combining the first values and second values to generate an enhanced image; processing the enhanced image to identify at least one linear structure in the enhanced image; processing the at least one identified linear structure to generate data indicating properties of the at least one
identified linear structure; and generating the output data providing an indication of the presence or absence of disease based upon the data indicating properties of the at least one identified linear structure. The image may be a corneal image although processing other types of medical image in this way may provide a useful indication of the presence of absence of disease.
In this way, structures relevant to determining the presence or absence of disease can effectively be identified. More particularly, by enhancing an image before identifying linear structures by generating a first value and a second value for each point in the image and combining the first and second values, the inventors have found that linear structures useful in the determination of the presence or absence of disease can be more effectively identified.
Generating a respective first value for each of a plurality of points may comprise applying a first model to the image and generating a respective second value for each of a plurality of points may comprise applying a second model to the image.
For each point in the enhanced image, if the first value and the second value for the point satisfy a predetermined condition, the point in the enhanced image may take a greater (or lesser) value than if the predetermined criterion is not satisfied. That is, the value for a point in the enhanced image may be generated using a 'step' function (or a function similar to a step function) such that if the criterion is satisfied the point in the enhanced image may have a value on a first side of the 'step' while if the criterion is not satisfied, the point in the enhanced image may have a value on the other side of the 'step'. Combining the first values and the second values to generate the enhanced image may comprise, for each of a plurality of points in the image: determining whether a first value for a respective point and a second value for a respective point satisfy a predetermined criterion; if the first value and the second value satisfy the predetermined criterion, setting the point to a first output value; and if the first value and the second value do not satisfy the predetermined criterion, setting the point to a second output value.
In this way, the enhanced image can provide a relatively large value for a point if a predetermined criterion involving the first and second values is satisfied, and a relatively small value for a point if a predetermined criterion involving the first and second values is not satisfied. Contrast between structures representing nerve fibers in an image and other areas can thereby be increased based upon the first and second values and structures representing nerve fibers can thus be more easily identified and properties of the structures can be more readily determined.
The first output value may be based upon the first value and the second value. For example, the first output value may be the result of an arithmetic operation, such as addition, applied to the first and second values. The second output value may be a constant value, for example zero. For points of the image representing linear structures, the first value may be greater than the second value. For points of the image not representing linear structures, the first value may be less than the second value. The predetermined criterion may therefore be based upon the relative values of the first and second values.
The first model may be based upon an estimate of orientation of structures in the image. The first model may provide a respective value for each of a plurality of points in the image, and each value may be based upon an estimate of orientation at a particular point in the image. The inventors have found that processing an image with reference to an estimate of orientation of linear structures in the image allows an image to be enhanced in such a way that linear structures are more clearly discernable in the enhanced image.
The method may further comprise determining an estimate of orientation of structures of the image by applying a least-squares estimation operation to the image to generate a local orientation estimate for each of a plurality of points in the image.
Determining the estimate of orientation of structures of the image may further comprise applying a smoothing operation to the local orientation estimate, for example applying a Gaussian smoothing operation.
The first model may be a Gabor model, although it will be appreciated that any model arranged to enhance the visibility of linear structures in an image can effectively be used.
Processing the enhanced image to identify at least one linear structure in the enhanced image may comprise applying a threshold to the enhanced image to generate a thresholded image. For example, the enhanced image may be generated in such a way that pixels representing linear structures have relatively high intensity, and a comparison of pixel intensity with a threshold can therefore be used to differentiate pixels representing linear structures from pixels not representing linear structures.
The at least one linear structure may be a connected area in the thresholded image.
Processing the enhanced image to identify at least one linear structure in the enhanced image may comprise identifying a plurality of linear structures in the enhanced image and processing the identified plurality of linear structures to form a single linear structure from the identified plurality of linear structures.
Processing the identified plurality of linear structures to form a single linear structure may comprise selecting a pair of linear structures; for the selected pair of linear structures, determining a cost associated with a connection between the pair of linear structures; processing the cost associated with the selected pair of linear structures with respect to a predetermined threshold; and determining whether to form a single linear structure from the pair of linear structures based upon the processing. In this way, a determination can be made, for a particular pair of linear structures as to whether a connection between the pair of linear structures can be made sufficiently 'easily'. The term 'cost' is used herein to refer to any parameter (e.g. an energy) which can be used as a basis for determination of whether a pair of linear structures should be connected.
Selecting a pair of linear structures may comprise selecting a plurality of pairs of linear structures from the identified plurality of linear structures; for each of the pairs
of linear structures, determining a cost associated with a connection between the pair of linear structures; and selecting a pair associated with an optimal cost.
Selecting the plurality of pairs may comprise determining a probability associated with each of a plurality of pairs of linear structures, the probability indicating a probability that the pair represent a single linear structure, and selecting the plurality of pairs based upon the determined probabilities.
The probability may be determined based upon at least one property of the linear structures, the at least one property being selected from the group consisting of: intensity of the linear structures, size of the linear structures, distance between the linear structures, orientation of the linear structures, and orientation of a connection between the linear structures.
According to a fourth aspect of the invention there is provided a method for processing images to assess the condition of a patient, the method comprising: processing a first image obtained from the patient at a first time using a method according to the first aspect of the invention; processing a second image obtained from the patient at a second time using a method according to the first aspect of the invention; and comparing results of processing the first image and processing the second image to assess a change in the condition of the patient between the first time and the second time.
The fourth aspect of the invention therefore provides a method for monitoring a change in the condition of a patient over time. In this way, the method may be useful in assessing the natural history of a disease in longitudinal studies and in assessing the efficacy of a medicament, for example in clinical trials.
The second image may be obtained after the patient has been subjected to therapy. The therapy may comprise a medicament. In this way, the second aspect of the invention can be used to monitor the efficacy of a treatment applied to the patient.
According to a fifth aspect of the invention there is provided a method of enhancing a corneal image, the method comprising: generating an estimate of orientation of structures of the corneal image; and enhancing linear structures in the corneal image based upon the estimate of orientation of structures.
The inventors have found that enhancing a corneal image based upon estimates of orientation of structures in the corneal image allows a corneal image to be enhanced in such a way that linear structures can be clearly identified.
Enhancing linear structures in the corneal image may comprise applying a first model to the corneal image to generate a respective first value for each of a plurality of points in the corneal image; applying a second model to the corneal image to generate a respective second value for each of a plurality of points in the corneal image; and combining the first values and the second values to generate the enhanced corneal image.
The estimate of orientation of structures of the image may be generated by applying a least-squares estimation operation to the corneal image to generate a local orientation estimate for each of a plurality of points in the corneal image.
Determining the estimate of orientation of structures of the image may further comprise applying a smoothing operation to the local orientation estimate.
The first model may be applied to the corneal image based upon the estimate of orientation of structures.
According to a sixth aspect of the invention there is provided a method of generating output data providing an indication of the presence or absence of disease from a corneal image. The method comprises identifying an elongate structure representing a nerve fibre in the corneal image, the elongate structure defining a path; generating data based upon a part of the elongate structure, the data indicating a property of the elongate structure in a direction transverse to the path of the part of the elongate
structure; and generating the output data providing an indication of the presence or absence of disease based upon the generated data.
The inventors have realised that a measure transverse to the path of a nerve fiber can be used to determine an indication of the presence or absence of disease from a corneal image.
Generating data based upon the part of the elongate structure may comprise determining a width of the part of the elongate structure.
Generating data based upon the part of the elongate structure may comprise determining an intensity at a plurality of points of the elongate structure along the direction transverse to the part of the elongate structure. In this way an intensity profile, or variance in intensity across a nerve fiber may be used in the determination of the presence or absence of disease.
The direction transverse to the part of the elongate structure may be based upon an estimate of orientation of the part of the elongate structure.
The direction transverse to the part of the path may be substantially perpendicular to a direction indicated by the estimate of orientation of the part of the elongate structure. The data indicating a property of the elongate structure may therefore be a cross section of the elongate structure.
The method may further comprise: generating respective data based upon each of a plurality of parts of the elongate structure, each of the respective data indicating the property of the elongate structure in a direction transverse to the path of a respective one of the plurality of parts of the elongate structure; and the output data may be based upon an average of the respective data. Determining a property at a plurality of locations along an elongate structure in this way avoids anomalous results that may be provided by a single measurement.
Generating data based upon a part of the elongate structure may comprise: determining a length of the elongate structure; determining an area of the elongate
structure; and the data may be based upon the determined length and the determined area. In this way an indication of an average width of the elongate structure may be determined based upon an average number of pixels per pixel length.
The method may further comprise: identifying a plurality of elongate structures, each elongate structure representing a nerve fibre in the corneal image; generating respective data based upon a part of each of the elongate structures, each respective data indicating a property of a respective elongate structure in a direction transverse to the path of the part of the respective elongate structure; and generating the output data based upon the generated respective data. Considering a plurality of elongate structures in an image allows the determination of the presence or absence of disease to be based upon an overall property of elongate structures across the image.
Generating the output data may comprise generating data indicative of a distribution of the generated data. For example the distribution of generated data from each elongate structure in the image may be considered and the distribution may provide an indication of disease. The distribution may be processed using a trained classifier.
Generating the output data providing an indication of the presence or absence of disease may comprise: generating further data based upon the part of the elongate structure, the further data indicating a property of the elongate structure in a direction along the path of the part of the elongate structure; and comparing the data and the further data. For example the variance in intensity across an elongate structure compared to intensity along an elongate structure may provide an indication of the presence or absence of disease.
Identifying the linear structures may comprise enhancing the corneal image using a method according to the second aspect of the invention.
In each of the aspects of the invention, the corneal images may be corneal confocal microscopy images.
In each of the aspects of the invention the disease may be a nerve disorder. Particularly suitable nerve disorders are neuropathies, for example peripheral neuropathies such as diabetic neuropathy.
It should be appreciated that features described in the context of one aspect of the invention can be applied to other aspects of the invention.
Aspects of the invention can be implemented in any convenient form. For example computer programs may be provided to carry out the methods described herein. Such computer programs may be carried on appropriate computer readable media which term includes appropriate tangible storage devices (e.g. discs). Aspects of the invention can also be implemented by way of appropriately programmed computers.
Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:
Figure 1 is a schematic illustration of a system for analysis of images according to an embodiment of the present invention;
Figure 1A is a schematic illustration showing a computer of the system of Figure 1 in further detail;
Figure 2 is an image suitable for processing using the system of Figure 1 ;
Figure 3 is a flowchart showing processing carried out to analyse an image in the system of Figure 1 ;
Figure 4 is a flowchart showing processing to enhance an image;
Figure 5 is a flowchart showing a part of the processing of Figure 4 in further detail;
Figure 6 is a flowchart showing processing to extract parts of an image showing nerve fibers;
Figure 7 is a flowchart showing processing to connect linear structures in an image; and
Figure 8 is a schematic illustration of part of an image in which it is desirable to determine a width of a nerve fiber at a particular point on the nerve fiber.
Referring now to Figure 1 , an image capture device 1 is arranged to capture a digital image 2 of an eye 3. The digital image 2 is a Corneal Confocal Microscopy (CCM) image showing features of the cornea of the eye 3. The image 2 is stored in a database 4 for processing by a computer 5. Images such as the image 2 of Figure 1 may be collected from a population for screening for a disease such as, for example, a peripheral neuropathy such as somatic neuropathy or autonomic neuropathy. Additionally, images such as image 2 of Figure 1 may be collected from a patient at intervals to assess deterioration of a condition of the patient or to determine the efficacy of any therapy given to the patient to improve the condition. The image capture device 1 may be a CCM camera such as a Heidelberg Retina Tomograph (HRT3) manufactured by Heidelberg Engineering, Heidelberg, Germany, or any image acquisition device suitable for capturing a CCM image of an eye.
Figure 1A shows the computer 5 in further detail. It can be seen that the computer comprises a CPU 5a which is configured to read and execute instructions stored in a volatile memory 5b which takes the form of a random access memory. The volatile memory 5b stores instructions for execution by the CPU 5a and data used by those instructions. For example, in use, the digital image 2 may be stored in the volatile memory 5b.
The Computer 5 further comprises non-volatile storage in the form of a hard disc drive 5c. The digital image 2 may be stored on the hard disc drive 5c. The computer 5 further comprises an I/O interface 5d to which are connected peripheral devices used in connection with the computer 5. More particularly, a display 5e is configured
so as to display output from the computer 5. The display 5e may, for example, display a representation of the digital image 2. Additionally, the display 5e may display images generated by processing of the digital image 2. Input devices are also connected to the I/O interface 5d. Such input devices include a keyboard 5f and a mouse 5g which allow user interaction with the computer 5. A network interface 5h allows the computer 5 to be connected to an appropriate computer network so as to receive and transmit data from and to other computing devices. The CPU 5a, volatile memory 5b, hard disc drive 5c, I/O interface 5d, and network interface 5h, are connected together by a bus 5i.
Referring now to Figure 2, a CCM image of the cornea of a patient suitable for processing by the computer 5 of Figure 1 is shown. The CCM image shows clearly defined nerve fibers 7 as well as other linear structures that are less clear such as linear structure 8 which may indicate a nerve fiber at a deeper level in the cornea than nerve fiber 7, or may be noise in the image. Linear structure 9 appears in an area of noise in the image and is difficult to identify. Circled areas 10 indicate parts of relatively clear linear structures where a clear linear structure becomes hard to identify or disappears. For example, sections 11a and 11 b appear to be two parts of the same nerve fiber, however in the area 10 the exact path followed by the linear structure is not clear and is difficult to determine.
Referring now to Figure 3, processing carried out to analyse a CCM image is shown at a high level. At step S1 a CCM image of dimensions M x N pixels is received and at step S2 the image is processed to enhance the quality of the image. Processing to enhance the quality of an input image is described in further detail below with reference to Figures 4 and 5. At step S3 parts of the enhanced image representing nerve fibers are identified. Identification of parts of an image representing nerve fibers is described in further detail below with reference to Figure 6. At step S4 the identified parts of the image representing nerve fibers are quantified according to measures of peripheral neuropathy as will be described in further detail below with reference to Figure 7. At step S5 it is determined whether the identified parts of the image representing nerve fibers are indicative of peripheral neuropathy based upon the quantification of step S4.
Referring now to Figure 4, processing to enhance the quality of an input image is shown. At step S6 an image P is generated by processing the input image to remove high frequency noise components in the image and to normalise the image intensity such that all pixel values are in the range 0 to 1. The pre-processing of step S6 is not intended to affect linear structures in the image but instead to only remove background noise.
At step S7 an estimate of the orientation of each of a plurality of blocks of the input image is determined. For each of the blocks, two orthogonal gradient components are determined, and these components are combined to provide an estimate of orientation. These estimates can be represented in the form of an image θ, where each pixel in the image has an orientation value determined by a gradient estimate generated from a block centred at that pixel. The estimates of orientation are then smoothed to generate a further image Θ.
In more detail, the pre-processed image P output from step S6 is processed by taking, for each pixel (i,j) a block of pixels centred on the pixel (i,j) and determining an orientation estimate for that pixel (i,j) from pixel values within the block of pixels
The estimates of orientation are determined as set out in equations (1) to (3) below. More particularly an x-gradient component is determined using equation (1), a y- gradient component is determined using equation (2) and the x-gradient and y- gradient components are combined according to equation (3) to provide an estimate of orientation for the particular pixel.
u=ι— V=J—
where:
ω is the width of the block centred at pixel (i,j); and
δx(u,v) and 9y(u,v) are the gradients computed at each pixel (u,v) for u in the range■ - j to , + ~ and v in the range j - τ to j + τ .
The gradients δx(u,v) and δy(u,v) may be computed in any convenient way, for example using a Sobel operator or Canny operator.
Values of the image θ are processed to extract x and y vector field components, which are then filtered using a low-pass Gaussian filter G.
Vector field components Φx and Φy of θ in the x and y-axes for a pixel (i,j) are determined according to equations (4) and (5) respectively:
Smoothed vector field components Φx , Φy are determined from vector field components Φx and Φy according to equations (6) and (7) respectively.
Φ y(U)= £ lG(u,v)φ,(i-U,i-v) (7) ωφ ωφ
where ωφ x coφ is the size of the kernel of G.
The estimated orientation θ may not be correct in all cases due to the local nature of the estimate. Applying the low-pass filter G therefore corrects some errors given that the orientation varies slowly. The low-pass Gaussian filter G is applied to each pixel in the image θ in turn to reduce errors at near-nerve fiber and non-nerve fiber regions. An estimate of local orientation of structures Θ is determined from the smoothed vector field components according to equation (8).
In this way the image Θ provides an acceptably accurate estimate of gradient direction for each pixel of the original image.
Referring back to Figure 4, at step S8 linear structures in the pre-processed image output from step S6 are enhanced using the estimates of local orientation of structures Θ output from step S7. Linear structure enhancement is described in further detail below with reference to Figure 5.
At step S9 a determination is made as to whether the processed image output from step S8 is suitable for processing to determine the presence or absence of disease. If it is determined at step S9 that the image is not suitable for further processing then processing terminates. An image may not be suitable for further processing if, for example, the size of linear structures in the image is below a particular size (i.e. if linear structures comprise less than a particular number of pixels), which is taken as an indication that the image is of a layer of the cornea which does not contain nerve
fibers but is instead a layer of the cornea containing cells. Quantitative features of an image may be used to determine a quantitative indicator of validity, used to determine if an image is valid. The quantitative indicator may be determined using a trained classifier such as a support vector machine to classify images as suitable or not suitable for further processing based upon a distribution of sizes of linear structures in the image.
If the processing of step S9 determines that the image output from step S8 is suitable for processing to determine the presence or absence of disease, the image is processed to extract nerve fibers as is described below with reference to Figure 6.
The processing to enhance linear structures of step S8 of Figure 4 is shown in further detail in Figure 5. The enhancement of linear structures uses the estimate of local orientation of structures - determined according to equation (8). It will be recalled that the estimate of local orientation of structures Θ is an image in which each pixel represents an orientation associated with a corresponding pixel of the image P. In areas of the image P representing nerve fibers the image Θ has lines of substantially constant values. In background areas of the image P (i.e. areas that are not representative of nerve fibers) pixel values of the image Θ provide a noisy image due to the lack of structure and uniform direction in background areas.
With this in mind, the image Θ is used as an input to a foreground model h, with which the image P is combined to enhance linear structures. The image P is additionally combined with a background model of a type described below.
Referring to Figure 5, at step S15 the pre-processed image P is received and at step S16 the local orientation estimate Θ generated at step S7 of Figure 4 as described above is received. At step S17 the foreground model h is generated based upon the local orientation estimate Θ. The foreground model h is a 2D Gabor filter defined by equation (9):
where:
x, y are respectively horizontal and vertical indices of a pixel of the input image;
φ is the orientation of the Gabor filter h, as determined by the local orientation estimation Θ at the pixel x,y received at step S16;
f is the frequency of the cosine wave;
σx and σy are the standard deviations of the Gaussian envelope along the x and y axis respectively;
xφ = x cos(φ) + y sin(φ);
yφ = -x sin(φ) + y cos(φ);
θ is an offset of the cosine function which generally takes a value 0 (i.e. no offset is applied), although other values may be used where it is desirable to apply an offset to the model h; and
γ is a parameter which allows the aspect ratio of the foreground model h to be adjusted.
The Gabor filter is a cosine function modulated by a Gaussian function. The value f defines the number of cycles of the cosine function within a predetermined distance. The value f is selected such that the spatial extent of a single cycle of the cosine function is approximately the same width as a linear structure that it is desirable to enhance (i.e. a nerve fiber). The Gaussian function determines the contribution of pixels at varying distances from a pixel which is currently being processed. The values σx and σy define the widths of the Gaussian function in the x and y directions respectively The larger the value σx, the greater the contribution of pixels at some distance from a particular pixel which is currently the subject of processing. The value γ defines the ratio between the widths σx and σy. Where γ has a value 1 , the Gaussian function is symmetric, i.e. has equal width in both the x and y dimensions, and where γ has any other value, the Gaussian function is asymmetric, i.e. has different widths in the x and y dimensions. An asymmetric Gaussian function may be
advantageous in some embodiments as this may magnify the response of the function to nerve fibers, given the linear nature of the nerve fibers. The values X9 and yφ define respectively the x and y-axes of the filter coordinate frame after a clockwise rotation of the Cartesian axis by an angle of (π/2 - φ). The Gabor filter is rotated so that its local x-axis is perpendicular to the fiber direction and is applied across the direction of the fibers as determined by the local orientation estimation Θ.
Selection of f, σx and σy allows the Gabor filter to be tuned to the particular images and features. Suitable values for f, σx and σy have been determined as follows, although other values may be suitable according to the images to be processed: f = 1/9;
σx = 4; and
σy = 3.
At step S18 an image Ph is generated by pixel-wise convolving the foreground model h with the image P. The Gabor filter h enhances parts of the image P representing nerve fibers that are oriented in the direction of the filter (as defined by φ), and reduces visibility of parts of the image P having an orientation different to φ. Applying the filter h to the image P increases the contrast between the foreground and the "noisy" background, as well as reducing noise around the fiber structure. Additionally the filter h increases the likelihood of detecting nerve fibers that are not represented clearly in the image P due to the prior knowledge of the general structure of the fiber and the assumption of continuous flow of the fibers along a particular known direction as determined from the local orientation estimation Θ.
Whilst it has been described that the model h is a Gabor model, it will be appreciated that any model arranged to enhance the visibility of linear structures in an image can be used, for example a model such as the model described in R.N. Dixon and CJ. Taylor "Automated Asbestos Fibre Counting", Machine Aided Image Analysis, Institute of Physics, London, 1979, pp.178-185 may be used.
At step S19 a background model g is received. The background model g is a Gaussian kernel generated according to equation (10):
where:
x, y are respectively horizontal and vertical indices of a pixel of the input image;
σx1, σyi are the spread in, respectively, the x and y direction of the kernel; and α is a weight that may be applied to g, although α generally takes a value 1
(i.e. no weight is generally applied to g). Applying a weight other than 1 to g allows the effect of the model g to be varied relative to the model h.
At step S20 an image P9 is generated by pixel-wise convolving the image P with the background model g. The image P9 is a smoothed version of the image P. It will be appreciated that background models other than the Gaussian kernel of equation (10) that smooth an image can be used.
At step S21 a check is performed for each pixel (x,y) to determine whether the value of the image Ph(x,y) is greater than the value of the image Pg(x,y). For each pixel for which the value Ph(x,y) is greater than the value Pg(x,y) a corresponding pixel in an image S is generated according to equation (11):
S(x,y) = Ph(x,y) + Pg(x,y) (11)
In the case where Ph(x,y) is not greater than the value Pg(x,y) then the value S(x,y) is set to a value zero. Combining the values in this way means that areas likely to be linear structures are highlighted and areas that are likely to be background are reduced to a constant value of zero. The contrast between background and foreground areas is therefore increased.
Alternatively image S can be generated using a sigmoid function such as equation (11a):
where Fh(x,y) is the result of pixel-wise convolving the image P with Rp = h(x,y) + g(χ,y);
Fg (x, y) is the result of pixel-wise convolving the image P with Rn = h(x,y) - g(x,y); and
k is a parameter that controls the sharpness of the change between a pixel being highlighted (i.e. treated as foreground) and being reduced to a constant value of zero (i.e. treated as background), which occurs when Fg(x,y) is equal to zero, i.e. when the foreground response, h(x,y) and the background response, g(x,y) are equal.
It will be appreciated that the step function of equation (11a) provides an image S that is an approximation to the image S generated at step S21 above using logical statements and equation (11) given that Rn takes a positive value where h is greater than g and a negative value or zero otherwise. As such the denominator of the right hand side of equation (11a) tends to 1 where h is greater than g, and S(x,y) therefore also tends to 1 , and the denominator tends to infinity otherwise causing S(x,y) to tend to 0. It will be appreciated that generation of the image S of step S21 may be more computationally efficient to implement than implementation of equation (11a).
It has been described above that a Gabor filter is used as a foreground model. It will however be appreciated that wavelets other than the Gabor filter, for example the Daubechies wavelet, can be used. Additionally wavelets having different frequencies could be applied to the image and averaged or the values of f, σx and σy could be adaptively set according to appropriate local values determined by pre-processing the image.
It will be appreciated that some of the processing shown in Figure 5 can be carried out in parallel. In particular, S17 and S18 can be carried out in parallel with steps S19 and S20.
Referring back to Figure 3, the processing described above with reference to Figures 4 and 5 is carried out at step S2 of Figure 3. The identification of parts of the image representing nerve fibers at step S3 of Figure 3 is shown in further detail in Figure 6. The identification of parts of the image representing nerve fibers generally comprises a pre-processing part A in which a skeletonised binary image β is generated, followed by iterative processing B using the skeletonised binary image β. The iterative processing B is described in further detail below with reference to Figure 7.
Referring now to Figure 6, at step S25 a binary image T is generated by thresholding the image S. Areas of the image S that are greater than the threshold are set to a value '1 ' and areas that are not greater than the threshold are set to a value 1O'. At step S26 pixels in the image T having a value of '1' which do not, together with other pixels having a value of '1' form an area exceeding a predetermined size are set to a value 1O'. The size of an area is determined by the number of connected pixels in the area and the number of pixels is compared to a threshold to determine if the area is smaller than the predetermined size. At step S27 the image output from step S26 is dilated and at step S28 the image output from step S27 is eroded. The dilation and erosion operators of steps S27 and S28 open up the binary image and fill small gaps between and within areas representing nerve fibers. Dilation and erosion can be carried out using any suitable structuring element, for example a square 3x3 structuring element. At step S29 the image output from step S28 is skeletonised to generate the image β.
Referring now to Figure 7, the iterative processing B of Figure 6 is shown. The iterative processing B is intended to identify discrete linear structures in the processed image which represent a single linear structure in the cornea. Such identified discrete linear structures are connected areas. In general terms, the processing described with reference to Figure 7 is concerned with identifying which
structures to combine. At step S30 a model Ψ is initialised based upon the enhanced image S and the local orientation estimation Θ. The model Ψ is defined as follows:
Ψ = {ψu ψ2,••■, ΨL} where L = M x N is the number of pixels in the image;
Ψi = ^em where η is the magnitude of S at pixel i, (pi is the value of local orientation estimation Θ at pixel i and j is the complex number V-1.
At step S31 the set of linear structures r = {71,72, ..., γc} in the skeletonised image β is determined, where each γs is a connected area of pixels having value 1 in the image β. Given the processing described above, it is assumed that each connected area represent a linear structure in the cornea. Each γs is defined as follows:
γs = (p^,Qrs; ) where:
Qrs; = {Qi, α.2,■•- , Qh) is the set of pixel descriptors qk = (xk, yk, φk) in the connected area γs, where xk and yk are the coordinates of the pixel represented by the pixel descriptor qk in the image β and φk is the orientation of the pixel represented by the pixel descriptor qk determined from the image Θ; and
pr A s; is the relative size of the area γs determined by dividing the number of pixels in γSl by the number of pixels in the image, M x N.
At step S32 a pair of linear structures γs,γ, are selected such that the pair γs,γ, has not previously been selected. At step S33 a probability Λs t of a connection between the selected structures is determined. The probability of a connection Λs t in Λ is defined as follows:
Λs t = {λ'g pλspλstΛ'l. pλ5,. t,λ6 s t j where each of the λ's t , 1≤ i≤ 6, are in the range 0 to 1 and where:
λ'8i, = p^ where p£; and p£; are the values of the pixels represented by the pixel descriptors qs and qt respectively in the image S, where qs represents a pixel in γs and qt represents a pixel in γ, and qs and q, are selected such that the Euclidean distance between the pixels represented by the pixel descriptors is minimal across all possible pixels in γs, γt;
λ2 s t = p^s;p2; where p^ and p^; are the relative sizes of the areas γs and γt respectively;
λ3 s, where
qs -qt defines the distance between the pixels represented by the pixel descriptors qs and qt (used in the determination of λ's t) and D is the maximum possible distance between points in the image; λ4 s t = l— - where ξβt = wm[(φs - φ,)+ π,-(φβ - φ,)+ π] and φs, φ, π
are the orientations of the pixels represented by the pixel descriptors qs and qt respectively; λ5 s , = 1 - δst where δ« = {- 1I^- ; and
st [θ, otherwise λ6 s t
[l -fξua J where ust is a connecting line between linear structures s and t and:
ξ^ = wm[(φu - φs)+ π -(φu -φs)+ π];
ξ(£ = min[(φv -φt)+ π,-(φu -φt)+ π] where: φ - iarfx —— where -^- is the gradient of the
IΔXSJ Δxst
connecting line between pixels represented by the pixel descriptors qs and q,.
The probability Λs t for a connection between linear structures (γS; γt) is determined by multiplying the values
The value λ's t is such that linear structures with high intensity are more likely to be connected to each other than linear structures with low intensity. The value λ2 s t is such that larger linear structures are more likely to be connected than smaller linear structures. The value λ3 s t is such that linear structures that are located close together are more likely to be connected than linear structures that are spaced far apart. The value λ4 s, is such that linear structures with ends with similar orientations are more likely to be connected than linear structures with ends with different orientations. The value λ5 s t is such that if the linear structures s, t are connected in
the processed image then no further connection will be made. The value λ6 s t is such that if a connecting line between the ends of two linear structures has a similar orientation to the orientation of the two ends of the linear structures a connection is more likely to be made than if the connecting line has a different orientation to the orientation of the ends of the two linear structures.
At step S34 it is determined if there are more pairs of linear structures for which a probability of a connection has not been determined. If it is determined that there are more pairs of linear structures then processing returns to step S32 where a further pair is selected for processing. If it is determined that all possible pairs of linear structures s, t have been processed to determine a probability Λs t then at step S35 a set π of the most likely connection for each linear object γs is determined by determining a linear structure γt for which the value Λs t is a maximum for the linear object γs and adding the connection (s,t) to the set π.
At step S36 the minimum energy es t associated with a connection between each of the pairs of linear structures (s,t) in π is determined according to equation (12) below:
where:
us,t is the connecting line between pixels qs and qt described above (that is pixels in γsi γ, between which the Euclidean distance is minimal) and includes the pixels qs and qt, I is a pixel in the connecting line us t, and ψ, = ^e'""1 as described above. The value es,t is the energy required to make a direct connection between pixels qs and q,. At step S37 the pair of linear structures (s,t) in π for which the value eS|t is minimal is determined and at step S38 a check is performed to determine if the value es t is smaller than a threshold ethresh. If it is determined that the value es t is not smaller than e^resh then it is determined that no further connections can be made in the image β and at step S39 the image β is output.
If it is determined at step S38 that es t is smaller than ethresh then at step S40 the image β is updated to include the connection us, and the two linear structures s, t become a single linear object. At step S41 the set of linear structures I" is updated based upon the updated image β and joined linear structures s, t. Processing passes from step S41 to step S31 and the processing to determine connections in the image is repeated based upon the updated model.
The image β output from step S39 (updated with any connections whose energy does not exceed the threshold ethresh) is used to determine nerve fibers in the area of the eye represented by the CCM image received at step S1. Areas with a value '1 ' are determined to be nerve fibre and areas with a value O' are determined to be background.
It has been described above that pixels are classified as nerve fiber or not nerve fiber based upon an image in which linear structures are enhanced. In an alternative embodiment (schematically illustrated in Figure 8A), an input image 2 may be processed to generate a plurality of scaled images 12. The input image 2 and each of the scaled images 12 may each be processed using the two models of equations (9) and (10). The model of equation (9) is applied at a plurality of different orientations to each of the input image 2 and each of the scaled images 12 to generate a respective plurality of images indicated by one of S^x.y) to Sn(X, y). A vector of pixel values 13 may be determined for each pixel 14 (corresponding to one of the pixels of the input image 2). The vector of pixel values 13 comprises values taken from each of the images in each of the plurality of images Si(x,y) to Sn(X, y). The vector of pixel values 13 for each pixel may be processed using a trained classifier 15 to provide an output 16 indicating whether the pixel is part of a linear structure and indicating the orientation of the linear structure of which the pixel is a part.
In more detail, and with reference to Figure 8B at step S51 an input image is received. At step S52 the input image is smoothed, for example using a Gaussian smoothing operation and at step S53 the image is sub-sampled such that the image comprises fewer pixels than the input image. Sub-sampling may be carried out in any
convenient way, for example by taking an average of pixel values which are used to form the basis of a pixel in the sub-sampled image. The processing of steps S52 and S53 is repeated a plurality of times to generate a plurality of images, each generated image being based upon a previous smoothed and sub-sampled image. At step S54 it is determined whether a predetermined number of iterations of steps S52 to S54 have been carried out to generate respective smoothed and sub-sampled images. If the predetermined number of iterations has not been carried out then processing continues at steps S52 and S53 where a further smoothed and sub-sampled image is generated based upon the previous smoothed and sub-sampled image. Otherwise at step S55 the input image and each smoothed and sub-sampled image output from step S53 is processed to generate a plurality of images S(x,y) as described above. Each of the plurality of images S(x,y) for each image is generated using a respective one of a predetermined number of orientations φ for the foreground model h and background model g. For example, eight orientations may be used to generate eight images.
At step S56 pixel values are determined for each pixel in the original image based upon each of the images S(x,y) generated at step S55 (where each image S(x,y) has a respective scale, and orientation associated with the model h). Given that the images output from step S53 are sub-sampled and contain fewer pixels than the input image, and correspondingly the images S(x,y) contain fewer pixels than the input image, values for pixel locations in the original image are determined based upon values for pixels in the image S(x,y) that represent the pixel location of the pixel in the original image. Pixel locations that are represented by a pixel in the image S(x,y) may be determined based upon the sub-sampling of step S53.
The image value at a particular pixel in the output image Sn(x,y), that has been generated by smoothing and sub-sampling at steps S52 and S53, will consist of contributions from a plurality of pixels in the input image. The number of pixels in the input image that contribute to a particular pixel value in the generation of Sn(x,y) at step S55 will increase as n increases, i.e. as more smoothing is applied to the input image. Correspondingly, since the image Sn(x,y) provides a value for each pixel location in the input image, at step S56 a plurality of pixel locations in the input image
take a value based upon a particular pixel in the image output from step S53, with a pixel in an image Sn(X, y) that is generated based upon an image that has been generated by smoothing and sub-sampling the input image a large number of times providing a value to a relatively large number of pixel locations in the input image relative to an image Si(x,y) that is generated based upon the input image (or an image that has been generated by smoothing and sub-sampling the input image a relatively small number of times).
At step S57 a feature vector is generated for each pixel in the input image, where values of the feature vector are the values determined at step S56 from each of the images S(x,y). At step S58 each feature vector comprising values for a pixel location in the original image is processed using a trained classifier which provides an indication of whether a pixel represents a linear structure in the image, together with an indication of an orientation of any linear structure represented by the pixel. A set of linear structures F can be determined from the output of the classifier, based upon classifications of pixels in the input image determined by the classifier.
For each of the methods of identifying linear structures in an image set out above, properties of nerve fibers in the input image can be determined based upon the set of identified linear structures F.
The following criteria are determined from the image β in order to determine an indication of the presence or absence of disease in the patient from whom the corneal image was captured:
fiber length;
fiber density;
fiber branch density;
fiber width; and
fiber tortuosity.
Each of the parameters can be determined in any convenient way from the nerve fibers identified by the processing of Figure 3. For example fiber length may be determined by calculating the total number of pixels in image β. Fiber density may be determined by calculating a count of the number of fibers in the image or a maximum
fiber density in an area of a predetermined size. Fiber branches may be identified based upon locations at which fibers coincide. The number of such locations may be summed to give a total number of branches per image. Fiber tortuosity may be determined according to the method described in Corneal Nerve Tortuosity in Diabetic Patients with Neuropathy, Kallinikos et. al., Investigative Ophthalmology & Visual Science, February 2004, Vol. 45, No. 2, or in any other suitable way.
The measure of fiber width may be generated directly from the image 2 of Figure 1 or from any other suitable image generated from the image 2, for example from the pre- processed image P or the binary image T. Additionally, nerve fibers may be identified in one image, such as the skeletonised image β, and the width of identified nerve fibers may be measured in another image, such as the image 2 of Figure 1 or an enhanced image based upon image 2. Measuring nerve fibers in image 2, or an enhanced image based upon image 2, may result in a more accurate indication of nerve fiber width.
A measure of a width of a nerve fiber at a particular point on the nerve fiber may be determined in any convenient way. For example, width of a nerve fiber may be determined using the estimate of local orientation of structures Θ. Figure 9 shows part of an image 2 in which a width of a nerve fiber 17 at a particular point q is indicated by cross section 18. The direction in which to measure the width, indicated by cross section 17, is selected so as to be perpendicular to the direction indicated by the local orientation of structures 19 of the point q. A number of ways of determining a measure of fiber width at the cross section 18 will now be described, although it will be appreciated that width of a nerve fiber may be determined in any convenient way.
One suitable measure of fiber width at a particular point on a nerve fiber is a number of sequential pixels along the cross section 18 whose pixel value each exceed a predetermined threshold. The number of sequential pixels may be determined by thresholding the image S in which linear structures have been enhanced such that all values in the image S having a value less than the threshold are set to a constant value (for example 0). The number of pixels along the cross section 18 which have a
value different to the constant value may be counted so as to provide a measure of fiber width.
Alternatively, an indication of width of a nerve fiber at a particular point on a nerve fiber may be determined by processing values along cross section 18 in the input image 2 or an image generated from the input image 2 (for example, a Gaussian smoothed image generated from the input image 2) to generate second derivative values of values along the cross section 18. Edges of the nerve fiber along the cross section 18 are located at points having maximal gradient (that is points at which first derivative values are maximal), and as such are indicated as zero values in the second derivative values. A value indicating a distance between points having second derivative values of zero is therefore also indicative of width of the nerve fiber along the cross section 18 and can be used as an alternative to the number of pixels along the cross section 18 described above.
The measures of width of nerve fiber described above (number of pixels exceeding a threshold and distance between points having second derivative values of zero) can be used to determine an average width of a nerve fiber by determining the width of the nerve fiber at each possible point along the nerve fiber according to one of the methods (each possible point being determined, for example, by a pixel in the skeletonised image β), or by determining the width of the nerve fiber at a plurality of intervals along the nerve fiber, and averaging the determined widths.
An indication of an average fiber width in the image may be generated, determined by processing a plurality of widths, each width being determined from a respective nerve fiber identified in the image. Alternatively, an average width of a nerve fiber may be determined by determining the length of the nerve fiber In, for example by determining the number of pixels in the skeletonisation of the nerve fiber in the image β, determining the total number of pixels which exceed a predetermined threshold in the nerve fiber tn in a non-skeletonised image (for example the image P or the binary image T) and dividing tn by In. The value Vl„ provides an indication of the average width, in pixels, across the length of the nerve fiber. Determining a total length of all
fibers in the image and a total number of pixels of all fibers in the image allows an average fiber width across an image to be determined in a similar way.
Alternatively, the measure of fiber width may be a measure indicative of fiber width distribution such as, for example, a histogram indicating the distribution of measured widths in the image. The histogram may indicate average widths determined for each nerve fiber or may indicate a plurality of measured widths determined from each of a plurality of nerve fibers. The distribution of the measured widths of nerve fibers in the histogram may be used to determine an indication of the presence or absence of disease in the patient from whom the corneal image was captured, for example by training a classifier on a training set of images to classify fiber width distributions as indicative of the presence or absence of disease. In general terms, low fiber width values are indicative of nerve damage or repair and therefore may be associated with disease or recovery from disease, respectively. Additionally, where nerve fiber width distribution indicates that a disproportionately large number of nerve fiber widths are relatively low (as indicated by a distribution which is asymmetric, with a greater number of fiber widths having a width lower than an average fiber width than fiber widths having a width higher than an average fiber width), this may be indicative of nerve fiber beading and may be indicative of neuropathy.
In an alternate embodiment the measure of fiber width may be based upon pixel intensities determined in a direction transverse the nerve fiber. The direction transverse the nerve fiber may be determined based upon the estimate of local orientation of structures Θ at the point at which the measure of fiber width is determined. For example, a sum of pixel values along cross section 18 of Figure 9 may be generated and used as an indication of fiber width at a particular point on a nerve fiber. The distribution of sums of pixel intensities along a plurality of cross sections 18 at a plurality of points in the image may be classified using a classifier trained on a training set of images to classify pixel intensity distributions as indicative of the presence or absence of disease. Distribution of fiber width based upon pixel intensities may again be used to indicate, for example, nerve fiber beading.
The classifier may further use pixel intensity distribution in a direction along the path of the nerve fiber, as indicated by the estimate of local orientation of structures Θ, and may classify images based upon a comparison of variation in pixel intensity across a nerve fiber and along a nerve fiber.
Alternatively, pixel intensities in a direction transverse the nerve fiber may be processed to generate a plot of pixel values along the cross section 18. The plot of pixel values may be integrated to determine a value indicative of an area under the plot and the value indicative of an area may be used as a value indicative of width.
In each of the methods described above for determining a width, it will be appreciated that any suitable image can be used, including, without limitation, the input image, and images generated from the input image by or during the processing set out above.
In general terms each of short fiber length, low fiber density, low fiber branch density and high fiber tortuosity are indicators of neuropathy and these indicators are combined to generate the indication of the presence or absence of disease. The parameters may be combined in any convenient way. For example a manually classified training set of images may be processed to generate a set of weights for the criteria and the generated set of weights applied to values generated from an image captured from a patient to output a value indicating a confidence that the patient has neuropathy. The confidence may be used as a measure of the severity of neuropathy of the patient and used to assess regeneration of the nerve and improvement of neuropathy in the patient, for example in response to a treatment.
In one embodiment a classifier may be trained on a training set of images to classify images as indicative of the presence or absence of disease based upon some or all of the measures indicative of fiber length, fiber density, fiber branch density, fiber width and fiber tortuosity, determined from a previously unseen image.
The processing of Figure 3 has been described with reference to a single image. It will be appreciated however that the presence or absence of disease may be
determined with reference to a plurality of images captured from a patient. For example an average of each of the measures described above may be calculated across all valid images captured from a patient. Additionally each of the images may have an associated weight applied to the values determined from the image. Weights for images may be determined based upon a quantitative indicator of image validity.
Although specific embodiments of the invention have been described above, it will be appreciated that various modifications can be made to the described embodiments without departing from the spirit and scope of the present invention. That is, the described embodiments are to be considered in all respects exemplary and non- limiting. In particular, where a particular form has been described for particular processing, it will be appreciated that such processing may be carried out in any suitable form arranged to provide suitable output data.
Claims
1. A method of generating data providing a quantitative indication of the presence or absence of disease from a corneal image of a patient, the method comprising:
processing said image to identify at least one linear structure;
processing said at least one identified linear structure to generate data indicating properties of said at least one identified linear structure; and
processing said data indicating properties of said at least one identified linear structure to generate said data providing a quantitative indication of the presence or absence of disease.
2. A method according to claim 1, wherein said generated data indicating properties of said at least one identified linear structure comprises a plurality of data values and processing said data indicating properties of said at least one identified linear structure comprises combining said data values.
3. A method according to claim 2, wherein each of said data values has an associated weight, and said data values are combined in accordance with the respective associated weight.
4. A method according to claim 1 , wherein said generated data indicating properties of said at least one identified linear structure comprises a plurality of data values and processing said data indicating properties of said at least one identified linear structure comprises processing said data values with a trained classifier to generate said quantitative indication of the presence or absence of disease.
5. A method according to any preceding claim, wherein said quantitative indication is a confidence of the presence or absence of disease.
6. A method according to any preceding claim, wherein said image is a corneal confocal microscopy image.
7. A method according to any preceding claim, wherein said disease is a neuropathy.
8. A method according to claim 7, wherein said neuropathy is a peripheral neuropathy.
9. A method according to any preceding claim, wherein processing said image to identify at least one linear structure comprises:
processing said image to generate an enhanced image; and
processing said enhanced image to identify said at least one linear structure.
10. A method according to any preceding claim, wherein processing said image to generate an enhanced image comprises:
generating a respective first value for each of a plurality of points in said image;
generating a respective second value for each of a plurality of points in said image, and
combining said first values and said second values to generate said enhanced image.
11. A method according to claim 10, wherein generating a respective first value for each of a plurality of points in said image comprises applying a first model to said image and generating a respective second value for each of a plurality of points in said image comprises applying a second model to said image.
12. A method according to any one of claims 1 to 11 , wherein processing said image to identify at least one linear structure comprises:
processing said image at a plurality of scales to generate a plurality of values for each of a plurality of image locations;
for each of said plurality of image locations, processing said plurality of values to generate an indication of whether the image location represents a linear structure in said image; and processing said generated indications of whether the image location represents a linear structure in said image to identify at least one linear structure.
13. A method according to any preceding claim, wherein the identified linear structure represents a nerve fiber.
14. A method according to claim 13, wherein said properties are properties of said linear structure indicative of properties of said nerve fiber selected from the group consisting of: fiber length, fiber density, fiber branch density, fiber width and fiber tortuosity.
15. A computer program comprising computer readable instructions configured to cause a computer to carry out a method according to any one of claims 1 to 14.
16. A computer readable medium carrying a computer program according to claim 15.
17. A computer apparatus for generating data providing a quantitative indication of the presence or absence of disease from an image of a patient comprising:
a memory storing processor readable instructions; and
a processor arranged to read and execute instructions stored in said memory; wherein said processor readable instructions comprise instructions arranged to control the computer to carry out a method according to any one of claims 1 to 14.
18. Apparatus for generating data providing a quantitative indication of the presence or absence of disease from an image of a patient, the apparatus comprising:
means for processing said image to identify at least one linear structure; means for processing said at least one identified linear structure to generate data indicating properties of said at least one identified linear structure; and
means for processing said data indicating properties of said at least one identified linear structure to generate said data providing a quantitative indication of the presence or absence of disease.
19. A computer implemented method for screening a population for disease, the method comprising, at the computer:
receiving a plurality of images, each image being taken from a member of said population; and
processing each of said plurality of images according to any preceding claim to determine a quantitative indication of the presence or absence of disease from each member of said population.
20. A method of generating output data providing an indication of the presence or absence of disease from an image of a patient, the method comprising:
generating a respective first value for each of a plurality of points in said image;
generating a respective second value for each of a plurality of points in said image; and
combining said first values and said second values to generate an enhanced image;
processing said enhanced image to identify at least one linear structure in said enhanced image;
processing said at least one identified linear structure to generate data indicating properties of said at least one identified linear structure;
generating said output data providing an indication of the presence or absence of disease based upon said data indicating properties of said at least one identified linear structure.
21. A method according to claim 20, wherein generating a respective first value for each of a plurality of points comprises applying a first model to said image.
22. A method according to claim 20 or 21 , wherein generating a respective second value for each of a plurality of points comprises applying a second model to said image.
23. A method according to any one of claims 20, 21 or 22, wherein said image is a corneal image.
24. A method according to any one of claims 20 to 23, wherein, for a particular point in said enhanced image, if said first value for the particular point and said second value for the particular point satisfy a predetermined criterion, said point in said enhanced image takes a greater value than if said predetermined criterion is not satisfied.
25. A method according to any one of claims 20 to 24, wherein combining said first values and said second values to generate said enhanced image comprises, for each of a plurality of points in the image:
determining whether a first value for a respective point and a second value for a respective point satisfy a predetermined criterion;
if said first value and said second value satisfy said predetermined criterion, setting said point to a first output value; and
if said first value and said second value do not satisfy said predetermined criterion, setting said point to a second output value.
26. A method according to claim 25, wherein said first output value is based upon said first value and said second value.
27. A method according to claim 25 or 26, wherein said second output value is a constant value.
28. A method according to any one of claims 20 to 27, wherein for points of said image representing linear structures, said first value is greater than said second value.
29. A method according to any one of claims 20 to 28, wherein for points of said image not representing linear structures, said first value is less than said second value.
30. A method according to claim 21 or any claim dependent thereon, wherein said first model is based upon an estimate of orientation of structures in said image.
31. A method according to claim 30, wherein said first model provides a respective value for each of a plurality of points in said image, and each value is based upon an estimate of orientation at that point in said image.
32. A method according to claim 30 or 31 , further comprising determining an estimate of orientation of structures of said image by applying a least-squares estimation operation to said image to generate a local orientation estimate for each of a plurality of points in said image.
33. A method according to claim 32, wherein determining said estimate of orientation of structures of said image further comprises applying a smoothing operation to said local orientation estimate.
34. A method according to claim 21 or any claim dependent thereon, wherein said first model is a Gabor model.
35. A method according to any one of claims 20 to 34, wherein processing said enhanced image to identify at least one linear structure in said enhanced image comprises:
applying a threshold to said enhanced image to generate a thresholded image; and
wherein said at least one linear structure is a connected area in said thresholded image.
36. A method according to any one of claims 20 to 35, wherein processing said enhanced image to identify at least one linear structure in said enhanced image comprises:
identifying a plurality of linear structures in said enhanced image; and processing said identified plurality of linear structures to form a single linear structure from said identified plurality of linear structures.
37. A method according to claim 36, wherein processing said identified plurality of linear structures to form a single linear structure comprises:
selecting a pair of linear structures;
for the selected pair of linear structures, determining a cost associated with a connection between said pair of linear structures;
processing the cost associated with the selected pair of linear structures with respect to a predetermined threshold; and
determining whether to form a single linear structure from said pair of linear structures based upon said processing.
38. A method according to claim 37, wherein selecting a pair of linear structures comprises:
selecting a plurality of pairs of linear structures from said identified plurality of linear structures;
for each of said pairs of linear structures, determining a cost associated with a connection between the pair of linear structures; and
selecting a pair associated with an optimal cost.
39. A method according to claim 38, wherein selecting said plurality of pairs comprises:
determining a probability associated with each of a plurality of pairs of linear structures, said probability indicating a probability that said pair represents a single linear structure; and
selecting said plurality of pairs based upon said determined probabilities.
40. A method according to claim 39, wherein said probability is determined based upon at least one property of the linear structures said at least one property being selected from the group consisting of: intensity of the linear structures, size of the linear structures, distance between the linear structures, orientation of the linear structures, and orientation of a connection between the linear structures.
41. A method according to any one of claims 20 to 41 , wherein said disease is a nerve disorder.
42. A method according to claim 41 , wherein said nerve disorder is a neuropathy.
43. A method according to claim 42, wherein said neuropathy is a peripheral neuropathy.
44. A method for processing images to assess the condition of a patient, the method comprising:
processing a first image obtained from said patient at a first time using a method according to any one of claims 20 to 43;
processing a second image obtained from said patient at a second time using a method according to any one of claims 20 to 43;
comparing results of processing said first image and processing said second image to assess a change in the condition of the patient between said first time and said second time.
45. A method according to claim 44, wherein said second image is obtained after said patient has been subjected to therapy.
46. A method according to claim 45, wherein said therapy comprises a medicament.
47. A computer program comprising computer readable instructions configured to cause a computer to carry out a method according to any one of claims 20 to 46.
48. A computer readable medium carrying a computer program according to claim 47.
49. A computer apparatus for generating output data providing an indication of the presence or absence of disease from a image of a patient comprising.
a memory storing processor readable instructions; and a processor arranged to read and execute instructions stored in said memory; wherein said processor readable instructions comprise instructions arranged to control the computer to carry out a method according to any one of claims 20 to 46.
50. Apparatus for generating output data providing an indication of the presence or absence of disease from a image of a patient, the apparatus comprising:
means for generating a respective first value for each of a plurality of points in said image;
means for generating a respective second value for each of a plurality of points in said image;
means for combining said first values and said second values to generate an enhanced image;
means for processing said enhanced image to identify at least one linear structure in said enhanced image;
means for processing said at least one identified linear structure to generate data indicating properties of said at least one identified linear structure;
means for generating said output data providing an indication of the presence or absence of disease based upon said data indicating properties of said at least one identified linear structure.
51. A method of enhancing a corneal image, the method comprising:
generating an estimate of orientation of structures of said corneal image; and enhancing linear structures in said corneal image based upon said estimate of orientation of structures.
52. A method according to claim 51 , wherein enhancing linear structures in said corneal image comprises:
applying a first model to said corneal image to generate a respective first value for each of a plurality of points in said corneal image;
applying a second model to said corneal image to generate a respective second value for each of a plurality of points in said corneal image; and
combining said first values and said second values to generate said enhanced corneal image.
53. A method according to claim 51 or 52, wherein generating said estimate of orientation of structures of said image comprises applying a least-squares estimation operation to said corneal image to generate a local orientation estimate for each of a plurality of points in said corneal image.
54. A method according to claim 53, wherein determining said estimate of orientation of structures of said image further comprises applying a smoothing operation to said local orientation estimate.
55. A method according to claim 54 as dependent upon claim 52, wherein said first model is applied to said corneal image based upon said estimate of orientation of structures.
56. A computer program comprising computer readable instructions configured to cause a computer to carry out a method according to any one of claims 51 to 55.
57. A computer readable medium carrying a computer program according to claim 56.
58. A computer apparatus for enhancing a corneal image comprising:
a memory storing processor readable instructions; and
a processor arranged to read and execute instructions stored in said memory; wherein said processor readable instructions comprise instructions arranged to control the computer to carry out a method according to any one of claims 51 to 55.
59. Apparatus for enhancing a corneal image, the apparatus comprising:
means for generating an estimation of orientation of structures of said corneal image; and
means for enhancing linear structures in said corneal image based upon said estimation of orientation of structures.
60. A method of generating output data providing an indication of the presence or absence of disease from a corneal image, the method comprising:
identifying an elongate structure representing a nerve fiber in the corneal image, the elongate structure defining a path;
generating data based upon a part of said elongate structure, said data indicating a property of said elongate structure in a direction transverse to the path defined by said part of said elongate structure; and
generating said output data providing an indication of the presence or absence of disease based upon said generated data.
61. A method according to claim 60, wherein generating data based upon said part of said elongate structure comprises determining a width of said part of said elongate structure.
62. A method according to claim 60, wherein generating data based upon said part of said elongate structure comprises determining an intensity at a plurality of points of said elongate structure along said direction transverse to the path defined by said part of said elongate structure.
63. A method according to any one of claims 60 to 62, wherein said direction transverse to the path defined by said part of said elongate structure is based upon an estimate of orientation of said part of said elongate structure.
64. A method according to claim 63, wherein said direction transverse to said part of said path is substantially perpendicular to a direction indicated by said estimate of orientation of said part of said elongate structure.
65. A method according to any one of claims 60 to 64, further comprising:
generating respective data based upon each of a plurality of parts of said elongate structure, each of said respective data indicating said property of said elongate structure in a direction transverse to the path defined by a respective one of said plurality of parts of said elongate structure;
wherein said output data is based upon an average of said respective data.
66. A method according to claim 60 or 61 or claim 63, 64 or 65 as dependent upon claim 60 or 61 , wherein generating data based upon a part of said elongate structure comprises:
determining a length of said part of said elongate structure;
determining an area of said part of said elongate structure;
wherein said data is based upon said determined length and said determined area.
67. A method according to any one of claims 60 to 66, further comprising: identifying a plurality of elongate structures, each elongate structure representing a nerve fibre in the corneal image;
generating respective data based upon a part of each of said elongate structures, said respective data indicating a property of a respective elongate structure in a direction transverse to the path of said part of said respective elongate structure; and
generating said output data based upon said generated respective data.
68. A method according to claim 67, wherein generating said output data comprises generating data indicative of a distribution of said generated respective data.
69. A method according to any one of claims 60 to 68, wherein generating said output data based upon said generated data comprises:
generating further data based upon said part of said elongate structure, said further data indicating a property of said elongate structure in a direction along the path of said part of said elongate structure; and
comparing said data indicating a property of said elongate structure and said further data.
70. A method according to any one of claims 60 to 69, wherein identifying said linear structures comprises enhancing said corneal image using a method according to any one of claims 31 to 35.
71. A method according to any one of claims 60 to claim 70, wherein said disease is a neuropathy.
72. A computer program comprising computer readable instructions configured to cause a computer to carry out a method according to any one of claims 60 to 71.
73. A computer readable medium carrying a computer program according to claim 72.
74. A computer apparatus for generating output data providing an indication of the presence or absence of disease from a corneal image comprising:
a memory storing processor readable instructions; and
a processor arranged to read and execute instructions stored in said memory; wherein said processor readable instructions comprise instructions arranged to control the computer to carry out a method according to any one of claims 60 to 71.
75. Apparatus for generating output data providing an indication of the presence or absence of disease from a corneal image, the apparatus comprising:
means for identifying an elongate structure representing a nerve fibre in the corneal image, the elongate structure defining a path;
means for generating data based upon a part of said elongate structure, said data indicating a property of said elongate structure in a direction transverse to the path of said part of said elongate structure; and
means for generating said output data providing an indication of the presence or absence of disease based upon said generated data.
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB0912154A GB0912154D0 (en) | 2009-07-13 | 2009-07-13 | Disease determination |
| GB0912154.2 | 2009-07-13 | ||
| GBGB1005905.3A GB201005905D0 (en) | 2010-04-09 | 2010-04-09 | Disease determination |
| GB1005905.3 | 2010-04-09 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| WO2011007126A2 true WO2011007126A2 (en) | 2011-01-20 |
| WO2011007126A3 WO2011007126A3 (en) | 2011-06-30 |
Family
ID=43449895
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/GB2010/001330 Ceased WO2011007126A2 (en) | 2009-07-13 | 2010-07-13 | Disease determination |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2011007126A2 (en) |
-
2010
- 2010-07-13 WO PCT/GB2010/001330 patent/WO2011007126A2/en not_active Ceased
Non-Patent Citations (2)
| Title |
|---|
| RUGGERI A ET AL: "Analysis of corneal images for the recognition of nerve structures", ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY, 2006. EMBS '06. 28TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE, IEEE, PISCATAWAY, NJ, USA, 30 August 2006 (2006-08-30), pages 4739-4742, XP031390557, ISBN: 978-1-4244-0032-4 * |
| SCARPA F. ET AL: "Automatic Recognition of Corneal Nerve Structures in Images from Confocal Microscopy", INVESTIGATIVE OPHTHALMOLOGY AND VISUAL SCIENCE, vol. 49, no. 11, 9 July 2008 (2008-07-09), pages 4801-4807, XP002613044, DOI: 10.1167/iovs.08-2061 * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2011007126A3 (en) | 2011-06-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN119963681B (en) | MRI medical image correction method, system and computer readable storage medium based on convolutional neural network | |
| Niemeijer et al. | Comparative study of retinal vessel segmentation methods on a new publicly available database | |
| Soomro et al. | Computerised approaches for the detection of diabetic retinopathy using retinal fundus images: a survey | |
| US8737703B2 (en) | Systems and methods for detecting retinal abnormalities | |
| CN116630762B (en) | A multi-modal medical image fusion method based on deep learning | |
| CN101127117A (en) | A Method for Segmenting Vascular Data Using Sequential Digital Subtraction Angiography Images | |
| JP5740403B2 (en) | System and method for detecting retinal abnormalities | |
| WO2008063604A2 (en) | Computer aided tube and tip detection | |
| Moraru et al. | Dempster-shafer fusion for effective retinal vessels’ diameter measurement | |
| CN120318105A (en) | Method and system for identifying and detecting craniocerebral disease regions based on MRI images | |
| CN117952962B (en) | Bone mineral density detection image processing method and system | |
| JP5632920B2 (en) | System and method for determining blur characteristics in a blurred image | |
| Yedidya et al. | Tracking of blood vessels in retinal images using Kalman filter | |
| Niemeijer et al. | Automated localization of the optic disc and the fovea | |
| EP3364854A1 (en) | System and method for the selection of optical coherence tomography slices | |
| Jung et al. | Automated detection of circular marker particles in synchrotron phase contrast X-ray images of live mouse nasal airways for mucociliary transit assessment | |
| Ahmed et al. | Retina based biometric authentication using phase congruency | |
| Kamble et al. | Automatic blood vessel extraction technique using phase stretch transform in retinal images | |
| WO2011007126A2 (en) | Disease determination | |
| Dash et al. | Analysis of vessel segmentation based on various enhancement techniques for improvement of vessel intensity profile | |
| Iacoviello et al. | Parametric characterization of the form of the human pupil from blurred noisy images | |
| EP1910998A2 (en) | System and method for automatic segmentation of vessels in breast mr sequences | |
| Mehta et al. | An enhanced segmentation technique for blood vessel in retinal images | |
| Devi et al. | Improved edge detection methods in OCT images using a hybrid framework based on CGWO algorithm | |
| Liu et al. | Segmentation refinement of small-size juxta-pleural lung nodules in CT scans |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 10765825 Country of ref document: EP Kind code of ref document: A2 |
|
| NENP | Non-entry into the national phase in: |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 10765825 Country of ref document: EP Kind code of ref document: A2 |








