EP1787156A1

EP1787156A1 - Automated diagnosis of malaria and other infections

Info

Publication number: EP1787156A1
Application number: EP05762647A
Authority: EP
Inventors: Nicholas Etienne Ross; David Milton Rubin; Charles John Pritchard
Original assignee: Individual
Current assignee: Individual
Priority date: 2004-06-11
Filing date: 2005-06-10
Publication date: 2007-05-23
Also published as: WO2005121863A1

Abstract

A process and apparatus for automated diagnosis of infections, using an image acquisition platform that captures microscopic images from a sample slide in an automated, systematic manner and then processes these images on a computing platform using image classification algorithms.

Description

FIELD OF INVENTION

This invention lies in the field of diagnosis and identification of micro-organisms and other objects. The invention has particular application in the field of identification of infections and of pathological conditions, including by the automated visual identification of micro-organisms. The invention is applicable over a range of organisms.

BACKGROUND

Infectious diseases, such as malaria, tuberculosis and sexually transmitted infections (STI's), are a major global health problem, causing millions of deaths annually. The economic effect of diseases such as malaria is also significant. The control of these diseases requires a rapid and accurate diagnosis that facilitates prompt treatment.

Despite advances in diagnostic techniques for malaria and many other diseases, light microscopy remains the most widely and commonly used method. This entails examining thick and thin blood smears for the presence of Plasmodia. It is the most efficient and reliable diagnostic technique, with very high sensitivies and specificities. Microscopy also makes it possible to differentiate between species, quantify parasitaemia and observe asexual stages of the parasites. Low material costs mean that the marginal costs of tests are very low. Microscopy is labour intensive and time consuming and the accuracy of the final diagnosis ultimately depends on the skill and experience of the technician and the time spent studying each slide. Compared with expert microscopy, standard laboratory microscopy has a sensitivity of approximately 90 %, a figure that drops dramatically in the field. Variable smear quality and slide degeneration with time are also problematic.

Antigen and immunoassay based technologies have also been developed to provide simple 'dip stick' diagnoses for diseases such as malaria and chlamydia. These technologies are extremely portable and require very low levels of training, but have high per test costs and other limitations, such as the inability of malaria testing kits to simultaneously diagnose all species or quantify parasitaemia. In addition, a different test is also required for every disease for which a diagnosis is attempted.

Some novel diagnostic tests have been developed that use, for example in the case of malaria, a cell counter to perform the diagnosis. None of these methods are known to be in use commercially, and the requirement to have a cell counter or other similarly complicated equipment available makes them unsuitable for use in any but the largest medical laboratories. THE INVENTION

This invention describes a system to improve upon infectious disease diagnosis by microscopy by removing its most serious limitation, viz. reliance on diagnosis by a human operator. This is achieved by developing a digital image processing system to automate the examination of samples such as (but not limited to) blood, sputum or cervical smears. To be successful, the system must provide at least the same diagnostic information as would normally be provided by a human operator. For example, in the case of malaria, it should give a positive or negative diagnosis of malaria with similar sensitivity and specificity to normal microscopy, differentiate parasites by species and quantify the degree of parasitaemia.

Further examples of conditions that the invention is well adapted to be applied to are Chlamydia, gonorrhea, syphilis or tuberculosis, or parasites such as malaria, bilharzias, leishmaniasis and African trypanosomiasis. The term "germ" will be used herein to encompass any micro-organism or object that the invention can be applied to.

A process in accordance with this invention includes the steps of : - capturing images of a germ from suitably prepared sample slides using an automated microscope; and analyzing the images using specially designed algorithms on a computing platform, whereby images are: pre-processed using filters such as median and morphological filters; analysed for relevant information, such as using granulometry to determine object radii; segmented using various novel threshold selection methods; characterized using features generated from first and second order statistical properties of the objects; and classified using these features by means of a suitable classifier such as back propagation feed forward neural networks in a binary tree topology.

The system of this invention enables the automatic identification and diagnosis of said diseases and the yielding of other diagnostic information, such as parasite load and species in the case of malaria.

The capturing of images preferably uses a novel, simple auto configuration of microscope. The characterising step may also include novel features that quantify criteria examined by human technicians during manual analysis.

Apparatus in accordance with this invention incorporates an image acquisition platform that captures microscopic images from a sample slide in an automated, systematic manner and then processes these images on a computing platform using image classification algorithms. Rather than simply being a diagnostic aid to a skilled technician by semi- automating the process and identifying areas of interest, the system is able to replace the technician and completely perform the diagnosis. The system also benefits from a novel, simplified configuration that allows for tailored optics, improved ease of use and a more compact physical unit. In addition to the new physical configurations, the system utilizes novel image classification algorithms.

The ability to integrate a communications system into the invention also makes available a large number of other possibilities such as health reports; remote maintenance and software upgrades; and the remote collection of image samples for scientific purposes. Among other things this will also assist health authorities with, for instance, disease incidence monitoring and optimal allocation of resources.

The communications module can utilize a range of current or future communications technologies, such as without limitation: GPRS, satellite communications, GSM, Ethernet or WiFi. By incorporating a modular design, the technology used can be customized to best fit the communications infrastructure of any particular area of development.

The input to the system is a suitably stained slide for the targeted disease. For example, for malaria the slides used are thin blood films, which are routinely stained using the quickdiff protocol, although alternative staining protocols such as the more conventional Giemsa stain are also possible. In the example of chlamydia, a Gram stain or a commercially available immunoassay such as the MikroTrak Chlamydia trachomatis Direct Specimen Test (Syva Co.) could be used, and for tuberculosis a suitable stain could be a sputum sample stained using, for example, the Ziehl-Neelsen or Auramine stains.

Since the input to the system - a suitably stained slide for the target disease - is the same as a technician using a light microscope would use, it can be seamlessly integrated into any existing diagnostic process that is based on light microscopy. This allows a very simple operating process to be used, whereby a sample is prepared as normal for the diagnosis desired, the slide is then inserted into the device and a diagnostic algorithm selected. It is a significant change from current practice that this device performs the diagnosis rather than a technician using a microscope. This makes the invention very flexible (unlike light microscopy), a trained technician is not required) with a wide scope for use (suitable for any diagnosis where the target can be identified on a stained sample fixed to a slide). If desired, a medical practitioner could review the diagnosis whether remotely (such as over the internet or other communications network) or at the place of use by means of a user interface, such as a PC-based graphical user interface (GUI). The innovations disclosed enable vastly increased efficiency, greater robustness, ease of use and accuracy when compared to human diagnosis, and are more cost-effective. The invention also drastically reduces the manpower requirements required for the diagnostic process, leading to savings and releasing capacity from existing diagnostic operations, as well as making it possible to have accurate, point-of-care diagnosis in many areas that previously have lacked the necessary manpower resources.

BRIEF DESCRIPTION OF THE DRAWINGS:

Figure 1 shows a possible configuration of a system embodying the elements of this invention.

Figure 2 is a flow chart showing the operation of the image acquisition hardware.

Figure 3 shows a comparison of the image classification algorithms for the general case, malaria and chlamydia.

Figure 4 shows a design concept that is a possible embodiment of the invention in a robust, portable unit.

Figure 5 shows the results from some of the steps of applying the malaria algorithm to a sample image. DETAILED DESCRIPTION OF THE INVENTION

REFERRING TO FIGURE 1 : Microscopic images from a suitably stained slide 7 for a target disease are automatically captured by the image acquisition hardware. The hardware consists of a simplified microscope (a tube 5 and lenses 4 with suitable magnification) connected to an image capture device 6; and a stage 1 moveable in the XY-plane to move the microscope field around the slide, and moveable along the Z-axis to allow for autofocus, onto which the slide is loaded. Control software controls the movement of the stage and the operation of the image capture device. A light source 8 and filter 9 is also provided. For example, without limitation, a possible configuration could be a charge-couple device (CCD) camera connected to a microscope with 1000X magnification, a halogen lamp light source and a stage moveable along the X-, Y- and Z-axes.

Although this kind of automated microscope is known, this particular configuration is novel in its simplicity: the specification of the image acquisition hardware is just to capture a sequence of digital images from the slide at the designed magnification. This means that simpler optics could be used (using only one objective) than in regular automated microscopes. By contrast to other approaches where, for example, images are precisely captured and stitched together to create a digital map of an entire sample, the location and neighbourhood of images captured in this system are not important, since each image is independently assessed by the selected diagnostic algorithm and all images are equally valid. This means that in theory the movement in the XY-plane to the next field-of-view could be arbitrary, but in practice a systematic method would be used to ensure that no fields are repeated so that every image adds new information and the efficiency of the system is maximized. The important consequence of this is that XY-plane movements need not be as precise as current systems require, since a slight overlapping or slight break between adjacent fields is not important, which allows a simpler, cheaper design for the automated stage. The simple design makes it possible to integrate the hardware into a compact design that can be used in a variety of locations.

The process steps implemented in the apparatus are indicated on the tabulated format at 10.

The inclusion of wide-area communications capacity as well as storage and a novel user interface means that the device can become extended from a diagnostic device to also be used for patient management and remote health reporting. RFERRING TO FIGURE 2 :

The operation of the image acquisition software is such that a series of images 11 is captured from the slide 7 and passed to the relevant diagnostic algorithm 12 until the algorithm indicates that a diagnosis has been made and no more information is required.

For example, in malaria, a positive diagnosis could be signaled once several plasmodia have been identified and their species determined, and the level of parasitaemia ascertained. A negative diagnosis could be signaled when, for example, ten thousand fields have been examined without finding any parasites. As another example, in chlamydia the presence of as few as ten elementary bodies (EB's) could signal a positive sample, while again a negative sample would require that a large number of fields have been investigated without detecting pathogens.

The ability to scan and process images sequentially means that the number of fields examined can be increased to as many as is desired, limited only by the area of the sample on the slide. This allows the device to improve upon current diagnostic methods. For example, by scanning a large number of fields in malaria, the sensitivity of the overall diagnosis, which is a function of the sensitivity of the diagnostic algorithm and the number of fields examined, can be made high enough to detect infections even at very low parasite loads. In current manual practice the entire slide is almost never examined.

The innovation detailed herein also allows microscope technology to be extended to diseases in which it is not commonly used. In the diagnosis of chlamydia, for example, it is considered too tedious for a technician to search through a slide for the presence of stained or fluorescent EB's. Under the scheme described in this patent, this becomes possible in an automated or semi-automated manner.

The speed of diagnosis is also increased, since the system is not constrained by factors such as operator fatigue. This also improves consistency of performance.

REFERRING TO FIGURE 3 :

The images, once captured, are passed to an image classification algorithm shown in the figure that processes them sequentially. The general structure of the image classification algorithm is in the form of a pattern recognition and classification system. This consists of a number of stages: image acquisition (implemented by the hardware platform); pre-processing 13 and image analysis 14; image segmentation 15; feature generation 16; and classification 17. Although based on tested pattern recognition and classification techniques, each algorithm is unique and specific to a target disease, but at the same time can all be used on the hardware platform since their input is simply a sequence of images of the sample slide. The actual implementation of the image processing algorithms can differ for different diseases. For malaria (Figure 3a), for example, the algorithm identifies erythrocytes (red blood cells) and malaria parasites present in the images, and from this information makes a diagnosis as to whether or not malaria is present, and if present, determines the species of the malaria and calculates the level of parasitaemia.

The desired effect, automated diagnosis of a disease or infection, is achieved by the software using, without limitation, a variety of filtering, analysis, segmentation, marking and classification steps as described below in more detail:

The first stage 17 of the malaria algorithm, pre-processing, involves the use of a median filter to remove noise from the image. A morphological area closing filter is also used to generate an image with less detail that is used for certain operations such as erythrocyte segmentation. For chlamydia (Figure 3b), as another example, the pre-processing would also involve using suitable median, morphological or other filter implementations to generate an image that can be used for segmentation.

The following step, image analysis, is not necessarily required for all diseases (for example chlamydia), but is required for diseases such as malaria since the morphological methods used for image segmentation depend on the size and shape of the objects in the image. The size and eccentricity of the erythrocytes is also an important feature for differentiating malaria parasite species. Granulometry is used to generate a pattern spectrum that indicates the size distribution of circular objects present in the image, from which the mean erythrocyte radius can be determined.

The average eccentricity of erythrocytes is determined by the ratio of major to minor axes of cells that are adjudged to be free-standing by virtue of their area. An upper limit on the area of an erythrocyte in order for it to be considered free- standing can be calculated from the pattern spectrum, which indicates the mean and standard deviation of the radii of erythrocytes present in an image.

The granulometry and eccentricity measures are carried out on binary masks. A threshold that separates the two principal modes - background and foreground - that are present in the image is used to obtain the mask.

REFERRING ALSO TO FIGURE 5 :

Figure 5a shows a sample image, with malaria parasites that are to be automatically identified circled. The next stage of the algorithm is image segmentation. For chlamydia, for example, this might only involve a threshold selection method to generate a mask of EB's present, based on the staining of the EB's, and a morphological filteY to ^' remove any objects of incorrect size. For the example of malaria, this step is more complicated and requires that potential parasites and erythrocytes are identified and segmented from the image, so that the infected erythrocytes can be extracted. Erythrocytes are identified using a threshold that separates the two principal modes - the background and the erythrocytes - that are present in the image histogram (Figure 5b). Morphological filters are then applied to remove small artifacts and smooth the objects present in the mask (Figure 5c). Parasites are identified using another threshold technique (Figure 5d), derived from the histogram of the green colour component of the image.

Continuing the malaria example, the mask of possible parasites is sued as a marker (figure 5e) to eliminate all erythrocytes from the erythrocyte mask that are not suspected of infection (Figure 5c). The infected objects often consist of clumps of overlapping erythrocytes, and it is necessary to separate them. To do this, a morphological gradient of the greyscale, morphologically filtered green component of the image is constructed by finding the difference between the dilation and erosion of the image. An intersection of the morphological gradient image and the dilated infected cell cluster is used to generate a mask of infected erythrocytes (Figure 5f).

The next step of the algorithm is to generate features that can be used to correctly classify the objects segmented from the image. Our novel approach is to generate first and second order features from characteristics such as texture and colour, and to also generate features that quantify details that a human technician would assess when analyzing a sample. These could include, for example, the relative size of infected cells, the shape and size of possible pathogens and even the spacing between objects that have been identified.

In the case of chlamydia, for example, relevant features include, amongst others, the colour, shape, size and relative position of objects to each other.

In the case of malaria, features generated from the infected erythrocytes include but are not limited to first and second order statistical measures that provide quantitative measures of the colour and texture of the objects. Other features are generated that are similar to the specific indicators used by a technician to determine parasite species, such as the size of the parasite, the size of the infected erythrocyte and the position of the parasite in the infected erythrocyte. These features are largely generated from binary masks of the plasmodia and infected erythrocytes, and are novel compared to most other feature generation methods in that rather than making statistical measures of image properties, they quantify measures that are consciously assessed by a human technician when making a diagnosis.

The final step in the image classification is the classification of the objects based on the feature vectors. For the example of chlamydia, a single neural network classifier would be a suitable solution. For malaria, since there are two classifications occurring, without limitation two neural networks in a binary tree structure are used. The first neural network uses predominantly colour and texture information to separate out false positives (Figure 5h), and the second neural network classifies the plasmodia according to species.

A number of embodiments of the described invention can be realized. These include, but are not limited to, use as a diagnostic tool, where the device makes an independent diagnosis from the given information, and as a diagnostic aid, where the device makes an assessment that is reviewed by a medical practitioner.

REFERRING TO FIGURE 4 :

One possible realization of the device, embodying the invention described, is shown in Figure 4. As shown in this realization, the invention can be integrated into a robust, portable design that allows it to be used in a wide variety of locations. Since the skill level to prepare slides is far lower than that required to analyse slides and make a diagnosis, the diagnostic process using light microscopy can be extended to many locations where it has previously been unfeasible.

The result from the diagnosis may be conveyed using a user interface. For example, this could take the form of a graphical user interface (GUI) on a liquid crystal display (LCD). An embodiment of the device could also include a communications module, as indicated in Figure 1 , which allows the invention to form the basis for a wider health monitoring network, for example disease incidence management by health authorities. The ability to store results and the development of novel user interfaces can extend an additional embodiment of the device to include patient management. Parasite load can be provided as further diagnostic information and species as for example in the case of malaria.

Claims

CLAIMS:

1. A process of automated diagnosis, which includes the steps of : - - capturing images of a germ from suitably prepared sample slides using a automated microscope; and analyzing the images using specially designed algorithms on a computing platform.

2. A process of automated diagnosis as claimed in claim 1 , in which the images are pre-processed using filters such as median and morphological filters.

3. A process of automated diagnosis as claimed in either one of claim 1 or 2, in which the images are analysed for relevant information using granulometry to determine object radii.

4. A process of automated diagnosis as claimed in any one of claims 1 to 3, in which the images are segmented using various novel threshold selection methods.

5. A process of automated diagnosis as claimed in any one of claims 1 to 4, in which the images are characterized using features generated from first and second order statistical properties of the objects.

6. A process of automated diagnosis as claimed in any one of claims 1 to 5, in which the images are classified using these features by means of a suitable classifier such as back propagation feed forward neural networks in a binary tree topology.

7. A process as claimed in any one of claims 1 to 6, which provides a yield of further diagnostic information, including parasite load and species, in applicable cases.

8. A process as claimed in any one of claims 1 to 7, which stores information generated by the process to aid in patient management and to be communicated to required destinations.

9. A process of automated diagnosis as claimed in any one of claims 1 to 8, in which eccentricity is determined in the images captured by the ratio of minor axis to major axis of cells that are adjudged to be free standing by virtue of their area.

10. A process of automated diagnosis as claimed in any one of claims 3 to 9, in which granulometry measures are carried out on binary masks in the images captured.

11. A process of automated diagnosis as claimed in either one of claims 9 or 10, in which eccentricity measures are carried out on binary masks in the images captured.

12. A process of automated diagnosis as claimed in any one of claims 1 to 11 , in which eccentricity is determined in the images captured, by the ratio of major to minor axes of cells that are adjudged to be free-standing by virtue of their area.

13. A process of automated diagnosis as claimed in any one of claims 1 to 12, in which granulometry measures are carried out on binary masks in the images captured.

14. A process of automated diagnosis as claimed in any one of claims 1 to 13, in which eccentricity measures are carried out on binary masks in the images captured.

15. A process of automated diagnosis as claimed in any one of claims 1 to 14, in which an automatic calculation whether an object is free standing by assessment of the pattern spectrum, indicating the mean and standard deviation of the radii of objects present in images captured, is carried out.

16. A process of automated diagnosis as claimed in any one of claims 1 to 15, in which image segmentation techniques, including threshold selection methods, are used to identify target objects.

17. A process of automated diagnosis as claimed in claim 16, in which morphological filters and the like are used to remove any objects of incorrect size in the images captured.

18. A process of automated diagnosis as claimed in any one of claims 1 to 17, in which morphological filters are used to remove artefacts and objects present in the mask.

19. A process of automated diagnosis as claimed in any one of claims 1 to 18, in which malaria parasites are identified using another threshold technique, derived from the histogram of the green colour component of the image. '

20. A process of automated diagnosis as claimed in any one of claims 1 to 19, in which separation of clumps of infected objects is achieved using a morphological gradient of the greyscale.

21. A process of automated diagnosis as claimed in claim 20, in which, the morphologically filtered green component of the image is constructed by finding the difference between the dilation and erosion of the image.

22. A process of automated diagnosis as claimed in claim 21 , in which an intersection of the morphological gradient image and the dilated infected cell cluster is used to generate a mask of infected erythrocytes.

23. A process of automated diagnosis as claimed in any one of claims 1 to 22, in which features are generated to correctly classify the objects segmented from the image, the features including, the colour, shape, size, texture and relative position of objects to each other.

24. A process of automated diagnosis as claimed in claim 23, in which objects are classified based on the feature vectors.

25. A process of automated diagnosis as claimed in claim 24, in which in chlamydia, a single neural network classifier is used.

26. A process of automated diagnosis as claimed in claim 24, in which for malaria, two neural networks in a binary tree structure are used, the first neural network using predominantly colour and texture information to separate out false positives, and the second neural network classifying the plasmodia according to species.

27. A process of automated diagnosis as claimed in any one of claims 1 to 26, in which patient details and results are stored.

28. A process of automated diagnosis as claimed in any one of claims 1 to 27, in which a custom user interface is used for patient management.

29. A process of automated diagnosis as claimed in any one of claims 1 to 28, in which a communications module is integrated.

30. A process of automated diagnosis as claimed in any one of claims 1 to 29, in which diagnostic results are communicated to a central authority to facilitate health reporting and disease incidence management.

31. A process of automated diagnosis as claimed in any one of claims 1 to 30, in which software updates are performed remotely and system operation is checked and assessed remotely.

32. A process of automated diagnosis as claimed in any one of claims 1 to 31 , in which images are collected remotely for scientific and other purposes.

33. A process as herein described.

34. Apparatus for carrying out the process as claimed in any one of claims 1 to 32, which incorporates an image acquisition platform that captures microscopic images from a sample slide in an automated, systematic manner and then processes these images on a computing platform using image classification algorithms.

35. Apparatus as herein described and as illustrated in the drawings.