WO2008131225A1

WO2008131225A1 - Method, system and apparatus for capturing digital microscopic medical images for remote analysis via machine vision or distributed panels

Info

Publication number: WO2008131225A1
Application number: PCT/US2008/060831
Authority: WO
Inventors: Brian Sroub; Konstantinos Veropoulos; Howard Fein; Manzoor Mohideen; James Uhlir
Original assignee: Interscopic Analysis, Llc
Priority date: 2007-04-20
Filing date: 2008-04-18
Publication date: 2008-10-30

Abstract

A digital microscope includes a base, a digital camera, a LED light source, a light source power supply, a USB connector, a fine focus, an objective lens, a slide stage, a condenser lens, a condenser, a condenser adjuster, and a diaphragm adjuster, wherein the slide stage is independently controllable, the objective lens is independently controllable, the condenser lens, condenser adjuster, and the condenser are independently controllable, there is no viewing lens, and the light source is powered via USB.

Description

METHOD, SYSTEM AND APARATUS FOR CAPTURING DIGITAL

MICROSCOPIC MEDICAL IMAGES FOR REMOTE ANALYSIS VIA

MACHINE VISION OR DISTRIBUTED PANELS

[0001] This application claims priority to a provisional patent application, Serial No. 60/913,266, filed April 20, 2007, entitled Method, System And Apparatus For Capturing Digital Microscopic Medical Images For Remote Analysis Via Machine Vision Or Distributed Panels, which is hereby incorporated by reference. BACKGROUND OF THE INVENTION [0002] Microscopes have been used for over 100 years to detect tuberculosis (TB) bacilli in sputum samples. In fact, of all the TB tests available on the market, physicians still consider the isolation of a TB bacillus under a microscope to be the most convincing form of proof ("the gold standard") that a patient is infected with TB. [0003] Still, it is difficult to accurately find TB bacilli using a microscope, and numerous medical studies suggest the test inaccuracies are in the 40% to 80% range (mostly due to human error as a result of the tedious task of going through a large number of slides). Tuberculosis bacilli are approximately 1 - 3 microns in size. There are two popular stains that preferentially attach to these bacilli: 1) Ziehl Neelsen stain, which stains the bacteria pink and the background blue; and 2) Auramine/Rhodamine stain, which is a fluorescent stain. Fluorescents are generally easier to see, but they require special lighting, are more expensive, and less specific.

[0004] In most labs in the developing world, the technicians will stain the specimen with Ziehl Neelsen stain and then examine the slide at 100Ox magnification using oil immersion and a halogen-lit bright field. These techniques result in severe technician fatigue which is a leading cause of the test's inaccuracies. The use of oil immersion also covers the lab equipment with oil creating generally untidy conditions and a cross contamination risk, which also causes inaccuracies. [0005] For these reasons, labs in more affluent countries have moved away from microscopy as their primary method for TB screening toward more convenient - but more expensive - methods, such as cultures, PCR (polymerase chain reaction) detection, and molecular based approaches. These methods are not economically practical in the developing world where a large percentage of TB and other infectious diseases occur. Some better-resourced labs in those regions have attempted to ameliorate microscopy problems through the use of fluorescent stains, but those stains add a variety of costs to the supply chain and the overwhelming majority of slides are stained with Ziehl Neelsen. Additionally, the stain may fade over time and the slides may become unusable. This, in combination with manual screening presents an additional difficulty in revisiting the slides for a secondary diagnosis or treatment monitoring. A solution to this would be the digitization of slides the first time they are placed under the microscope. Keeping digital records of stained samples, not only allows preservation of the evidence that supports the diagnosis, but also represents a visual record for the patient and provides technicians and researchers the ability to apply more advanced image processing and analysis methods that can further support the medical decision. SUMMARY OF THE INVENTION

[0006] This application relates to the method, system, and apparatus used to capture reliable digital microscopic images suitable for screening by remote technicians on a computer screen and/or machine vision, especially in resource constrained environments. In this invention, microscope will defined as "An optical instrument that uses a lens or a combination of lenses to produce magnified images of small objects, especially of objects too small to be seen by the unaided eye."

[0007] Several aspects of the invention include: 1) the digital microscope apparatus which is much simpler to install and operate than most image enabled digital microscopes today. Additionally, these simplifications allow a lower cost structure for the microscope; 2) the nature of the image is significantly different. Namely, it is a digital image taken at much lower magnification but with the required resolution to reliably discern and identify the bacilli of interest. In one embodiment, the magnification is 20Ox (established by a 2Ox objective lens and another 1Ox lens). This allows the originating lab to generate much larger sample coverage per image, increase coverage (and thereby sensitivity), and reduce the amount of time required in image capture; 3) an analysis tool that allows the technician pool to easily identify the TB bacteria using panning, digital zooms, and color enhancement that significantly reduce eye strain; 4) an analysis tool that takes the input from the technicians and uses it to refine the machine vision technology to teach it to see TB; and 5) the analysis tool will also generate a report output that shows the TB bacilli found in the sample in magnified form. It is to be understood that the present invention applies not just to TB, but to any analytes chosen using sound medical, chemical, and business judgment. BRIEF DESCRIPTION OF DRAWINGS

[0008] The invention may take physical form in certain parts and arrangement of parts, a preferred embodiment of which will be described in detail in this specification and illustrated in the accompanying drawings which form a part hereof and wherein: [0009] FIGURE 1 shows a schematic of the system overview describing the interrelationship of the system elements;

[0010] FIGURE 2 shows one embodiment of the imaging apparatus with a description of elements; [0011] FIGURE 3 shows a screen view of the analysis tool (current embodiment is called

"TB Mark");

[0012] FIGURE 4 shows the report output generated by the system; [0013] FIGURE 5 shows another embodiment of the imaging apparatus; [0014] FIGURE 6 shows an xy actuator; and, [0015] FIGURE 7 shows a z actuator.

DETAILED DESCRIPTION

[0016] Referring now to the drawings wherein the showings are for purposes of illustrating embodiments of the invention only and not for purposes of limiting the same, FIGURE 1 presents overview schematics of the image capture and analysis system. Images are first captured on the digital microscope (1). The images are numbered and categorized by specimen, compressed and transmitted out of the lab using routines identified in (2). The files may or may not be deleted at the time of transmission. It is to be understood that the images may be sent uncompressed as well, and that any means of compression, chosen using sound engineering judgment, can be used. The software

program called ICI (3) incorporates the control of the camera in the Interscope, the numbering, compression, and transmission of the images. The current version is written using the Matlab high-level language, but the code may also be written in faster languages such as C++, or any other language chosen using sound engineering judgment. The images are sent over a global computer network (4) and are received by the ICI (8) software module. ICI sorts the images and assign the work to the various technician pool members, based either on human judgment or algorithms. For the work coming to individual technicians or doctors, they receive files in their work directories are notified by email, or other communication methods, including but not limited to page, instant messenger, or SMS message. The technicians (5) then open the files using the program called TB Mark and conduct their analysis, sending it to the report consolidator and database modules (7). In addition to being an easy tool for technicians to use, the TB Mark program (5) also teaches the machine vision programs (6) the morphological characteristics of the analyte from the tech pool's output, images flagged for machine vision analysis flow directly to the program (6) which identifies the bacilli and sends the results to the report consolidator (7). Then the consolidated reports are sent back over the global computer network (4) to the customer in the originating lab. These reports may be output directly to a printer and/or an FTP site for further inspection. The report consolidator (7) generates a visual output that, in one embodiment, includes image numbers, customer number, and sample title. [0017] With reference now to FIGURES 2, and 5-7, the image capturing apparatus is called, alternately, digital microscope and Interscope. The image capturing apparatus (28) has an objective lens (10), a condenser (11), a condenser lens (12), a fine focus (13), a slide stage (14), a light source (15), a diaphragm adjuster (16), a digital camera (17), a power source and data bus (18), and a base (19). In one embodiment, the camera (17) has a sensor with 1280 x 1024 resolution, with 1 lmm measurement in the diagonal, and a pixel pitch of 6.7 micrometers. It is to be understood, however, that the invention is not limited to those specifications. The Interscope (28) does not have an eye piece. Rather, the only way to see the images captured by the Interscope (28) is via the computer attached via the data bus (1). This ensures the operator will focus on the digital image, which reduces cost. In one embodiment, the Interscope is outfitted with a single objective lens (10) selected for the target tasks (e.g. TB bacilli detection). Because we have a single objective lens, the condenser (11) and the condenser lens (12) require little or no adjustment once the Interscope (28) has been properly set up. Z axis variations do exist between specimens so a fine focus (13) is available to the operator. The stage (14) has fine control in x, y, and z directions. The three dimensional Cartesian coordinate system provides the three physical dimensions of space — length, width, and height. The three Cartesian axes defining the system are perpendicular to each other. The relevant coordinates are of the form (x,y,z). The x-, y-, and z-coordinates of a point can also be taken as the distances from the yz-plane, xz-plane, and xy-plane respectively. The x- coordinate is also known as abscissa, the y-coordinate is also known as the ordinate of the point, and the z-coordinate is also called applicate. In this embodiment, the stage (14) is rigid, wherein the stage does not bend more than approximately four or five micrometers. The stage (14) mates with an x,y actuator (30) and a z actuator (32). The actuators (30, 32) have a graduated scale to allow for fine movement along all three axes. Superior image quality is achieved when microscope components are optimized for viewing objects less than 3 microns long at 200 times magnification. This includes precise spacing between the objective lens, the telan lens and the camera. This has been achieved in prototypes through an iterative process. Additionally, the lighting and method of conditioning the light by projecting the light through a collimator and then reflecting the light off a precision mirror through a condenser located at a precise location beneath the specimen slide, which maximizes the amount of light illuminating the specimen in order to provide sufficient contrast to properly resolve the objects of interest. [0018] With continuing reference to FIGURE 2, the operator places a prepared slide on the stage (14), positions it toward areas of interest, using the fine control in the x, y, and z directions, and then captures the image using the ICI software (3). Additionally the slide may be placed in a slide holder (not pictured) which allows the operator free movement of the specimen on the stage, but protects the objective lens from contamination. The specimen is lit using LED lamps (15) which have the characteristics of low power usage, long life (-100,000 hours versus 2,000 hours for a halogen lamp), and small variation in lighting intensity. In one embodiment, the lamp is a fluorescent LED lamp. The apparatus also contains a diaphragm adjuster (16) to be set at time of installation and then left alone (although it is to be understood that the diaphragm adjuster can be adjusted after installation if needed). In one embodiment, the digital camera (17) is sourced from PixeLINK (it is to be understood that the invention is not limited to that digital camera, but any digital camera, chosen using sound engineering judgment, can be used). It is a 1.3 mega pixel camera with a 6.7 micron pixel pitch. The apparatus is connected to a local computer via a data bus port (18). In this embodiment of the invention, the data bus also powers the apparatus (18). USB or Firewire maybe used for this connectivity. The apparatus is held together with structural components or base (19). In one embodiment, the tolerance of these parts is controllable to fewer than ten microns to preserve Interscope image integrity. In the prototype version shown here, machined parts are used. Injection molded chassis are also envisioned.

[0019] With reference now to FIGURE 3, a screen shot of the software used by the technician pool to conduct the analysis is shown. The image is presented center screen (20) where it may be panned, zoomed, and color adjusted. The software also contains a magnification feature (21) that allows popup examination (shown) as well as window-in- window magnification where the technician can easily see how much of the image they are covering. When in Marker mode, cross hairs may follow the mouse around the image (22). Pressing the left click in this mode marks the pixels under the mouse as those being part of a TB bacillus and a number is superimposed on the image. The location and characteristics of these bacilli are recorded in a table (23) for transmission and future reference. They also are used to train the machine vision to automatically identify the TB bacilli, hi one embodiment, machine vision reads the digital image, which consists of pixels. The machine vision views the pixels one at a time, and compares each pixel with the next one. The machine vision uses probabilities to match shape, size, and color to a pre-set catalog of analytes and characteristics. The machine vision can eliminate objects that cannot possibly match what is being tested for. The imaging apparatus can be connected to a computer monitor or television screen, and the images can be viewed on the monitor or television screen.

[0020] With reference now to FIGURE 4, the initial report output that is sent back to the originating lab (customer) is shown. The report has customer specific information on the top (24) including the identification number, when it was collected, and the lab information. It also shows the written results provided by the technician or the machine vision routine (25). The whole image, with numbers, marking the bacilli (26) is displayed so the originating physician can audit the work. To further facilitate this audit, the first five bacilli marked (27) are displayed in a magnified format. These images may further be used to counsel patients regarding their condition to increase treatment compliance. [0021] The embodiments have been described, hereinabove. It will be apparent to those skilled in the art that the above methods and apparatuses may incorporate changes and modifications without departing from the general scope of this invention. It is intended to include all such modifications and alterations in so far as they come within the scope of the appended claims or the equivalents thereof.

[0022] Having thus described the invention, it is now claimed:

Claims

I/WE CLAIM:

1. A digital microscope characterized by: a base; a digital camera; a LED light source; a light source power supply; a USB connector; a fine focus; an objective lens; a slide stage; a condenser lens; a condenser; a condenser adjuster; and, a diaphragm adjuster, wherein the slide stage is independently controllable in x, y, and z axes, the objective lens is independently controllable, the condenser lens, condenser adjuster, and the condenser are independently controllable, there is no viewing lens, and the light source is powered via USB.

2. A digital microscope characterized by: a base; a digital camera; an LED light source; a light source power supply; at least one lens; and, a slide stage, wherein the slide stage is independently controllable.

3. The digital microscope of claim 2, wherein the microscope is further characterized by: the microscope having no viewing lens.

4. The microscope of claim 3, wherein the slide stage has at least three directional controls, wherein the each directional control is independently controllable.

5. The microscope of claim 4, wherein the microscope is further characterized by: an objective lens; a condenser lens; a condenser; and, a condenser adjuster, wherein the objective lens is independently controllable, and the condenser lens, condenser adjuster, and the condenser are independently controllable.

6. The microscope of claim 5, wherein the microscope is further characterized by: a USB connector, wherein the light source is powered by the USB connector.

7. The microscope of claim 2, wherein the light source is a fluorescent LED light source.

8. A method for capturing digital images for remote analysis, the method characterized by the steps of: imaging at least one specimen using a digital microscope; compressing the images; transmitting the compressed images via a global computer network; receiving and assigning the images; notifying at least one technician of the receipt and assignment of the at least one compressed image; identifying an analyte; sending a report of the analyte to a report consolidator; consolidating the report; and, transmitting the report via the global computer network to an originating laboratory.

9. The method of claim 8, wherein identifying the analyte is further characterized by the steps of: examining the image via software, wherein the software provides: a magnification feature; window-in- window magnification; and, report generation; and, generating at least one report.

10. A method for capturing digital images for remote analysis, the method characterized by the steps of: imaging at least one specimen using a digital microscope; transmitting the images via a global computer network; receiving and assigning the images; analyzing the image using machine vision; comparing the machine vision analysis against a pre-set database; sending a report of the analysis to a report consolidator; and, transmitting the report via the global computer network to an originating laboratory.