CA2086786A1 - Trainable automated imaging device - Google Patents

Trainable automated imaging device

Info

Publication number
CA2086786A1
CA2086786A1 CA 2086786 CA2086786A CA2086786A1 CA 2086786 A1 CA2086786 A1 CA 2086786A1 CA 2086786 CA2086786 CA 2086786 CA 2086786 A CA2086786 A CA 2086786A CA 2086786 A1 CA2086786 A1 CA 2086786A1
Authority
CA
Canada
Prior art keywords
images
class
operator
objects
features
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
CA 2086786
Other languages
French (fr)
Inventor
Calum Macaulay
Branko Palcic
David Michael Garner
Alan Harrison
Bruno Jaggi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Oncometrics Imaging Corp
Original Assignee
Xillix Technologies Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xillix Technologies Corp filed Critical Xillix Technologies Corp
Publication of CA2086786A1 publication Critical patent/CA2086786A1/en
Abandoned legal-status Critical Current

Links

Abstract

ABSTRACT

A TRAINABLE AUTOMATED IMAGE CYTOMETRY DEVICE
FOR FULLY AUTOMATED SEARCH OF DESIRABLE OBJECTS

An apparatus and a method for training an automated image cytometer consists of acquiring in focus images of individual objects, analyzing the features of each object and recording the resulting data in a storage medium. An operator then identifies a selection of images which are to be treated as being of the same class. A computer then performs multivariate analysis on the features of the identified images and uses the results to identify other potential members of the class. The apparatus may be further trained by displaying the potential members of the class and having the operator identify those members which are not properly within the class. Further multivariate analysis is then performed to refine the class identification.

Description

T~TI.E OF THE INV~NTION

A TRAINABL~: AUq~OMATED IMAGE CYTO~TRY DlNICE
FOR FULI.Y AUTOMATI~D 8EARCH OF DE8I~ABL15 OE~JECT~

FIEI.D OF THIS INV~NTION

The invention relates to automated imaging and analyzing devices which can be trained by human operators to capture images of object~ and to recognize objects with certain characteristics.

BACKGROUND OF TH15 INV~NTIOII

Over the past five decades, ~everal microscopes have been developed which can be used for quantitative measurement~
of objects such as cells, bacteria, crystals, wood particles, etc. Basically all guantitative microscopes are composed of a microscope equipped with a light-transducer, often a video camera. To capture images, a frame-grabber is usually provided which include~ an analog-to-digital converter. Often these are combined on an imaging board with some proces~ing capabilities.

The availability of such devices enables accurate measurement of characteristic features of images of the objects. Such devices are commonly used in science, industry and in medical diagnosis.

Some such systems have been equipped with motorized ~tages under computer control and have been provided with algorithms ~uch that the systems can bo taught to recognize certain objects having identified characterist~cs ~uch as ~ize, .,.., , :
,, . ~ , shape, amount of material, distribution of material over the object, etc. after an operator has "trained~ the ~y~tem by pointing out a certain number of such ob~ects. Multivariate analysis of features (eg. discr;m;nant function analysis, decision tree process, neural network~) ~8 sometimes uset to enable the system to recognize other objects falling within the same class as those identified by the operator. Such sy~tems are relatively crude and are dedicated to the task of segmenting objects for independent imaging and for later analysis.

Automation of the analysis and classification of objects (for example the diaqnosi~ of disease in cells) has been a more recent objective of certain segments of the scientific co" unity. 13fforts at achieving ~uch automated classification have uniformly been directed at a fully automated system capable of cla~sifying each image individually and having false classification error rates as low a~ possible. Such a fully automated approach requires an accurate analysis and cataloguing of discriminant features of the class of objects to be identified. Such prior analysis and cataloguing involves preparing a large training set of images of characteristic object~ whose features are carefully and precisely categorized for later use in the analysis of unclassified objects. This takes a significant amount of expert time in relation to every specific application or class of objects to be categorized. For example, for each independent feature to be meaningfully used in discriminant function analysis, typically 20 to 50 representative ob~ects must be analyzed. Thu~, if 20 indepondent featuros aro ''' . .
,:. ' .
:: . . .
.. . . .. . .

.
, to be u~ed, approximately 1000 image~ of the same clas~ muet be eelected by human experts, featuree extracted and input into discriminant function analyeie.

This iB a very time-coneuming process, often involving several experts in mathematice, phyeics, electronics, biology and medicine, and taking months to achieve the goal. Such a process is prohibitively expensive as well as lengthy although a system must be trainable in minutee or hours if it is to be used for practical applications in medicine, biotechnology, the paper and wood industry or environmental control. Even after cataloguing features, the usefulness of the exercise will be limited by variations in sample preparation and character at the user level.

An additional problem is that in mo~t such cases, the imnges to be analyzed and catalogued must be taken in precise focus, with exact segmentation (i.e. defining a mask outlining the boundaries of the object~ in the image) if feature values are to be reliably obtained for use in diecriminant analysis. This is usually done manually or semi-manually by an expert. In addition to being time consuming, the results are not very reproducible due to human error and the inherent difficulty for humans to distinguish subtle differences in focus and other optical variables.
.

In view of the above, automated analysis and classification of microscopic objects has heretofore been a tedious and time consumlng exerc~eo, l~ted to particular ' ` ' , `

.~

applications for which extensive prior analysis and cataloguing has been undertaken.

It is an object of thie invention is to provide apparatus and a method for the imaging, analy~is and classification of microscopic objects wherein the apparatus can be taught to discriminate between classes of objects with relatively little operator input without the need for separate prior analysis and cataloguing of features characteristic of the desired class.

It i8 a further object of this invention to provide such apparatus wherein the operator can interactively define classes or analysis parameters for the application at hand by identifying objects to be classified and wherein the apparatus thereafter has the ability to define the cla~s, to retrieve images of other objects in that class for display and to use cla~s parameters in the acqui3ition of images of other objects.

It iB a further object of this invention to provide such apparatus wherein the operator can interactively redefine cla~ses or analysis parameters, display clas~ data and physically retrieve and view ~pecific objects from the class in the ocular of the microscope.

.. ,. ,, -8UM~R~ OF TBIS INVI~NTIOII

Our approach to providing a fast and reliable automated imaging and classification system relies on departing from the application-specific cataloguing of the prior art and relying on the abilities of the operator of the device to recognize objects of a class. We have found that an operator can effectively train our automated system for the task of classifying objects which would require extensive feature analysis and cataloguing in co~pletely automated systems. While the initial training may sometimes result in lower accuracy than prior art systems based on previous cataloguing of features, we have found that a few additional training steps will result in similar accuracy to that achieved in the prior art, with orders of magnitude of improvement in time, and with the additional advantage of flexibility in application.

According to one of its aspects, the invention consists of an apparatus for imaging, proce~sing and analyzing objects comprising a microscope, a motorized microscope stage for receiving slides containing objects to be analyzed, a controller for controlling the movement of the motorized stage along the x, y and z axes, imaging means optically connected to the microscope, a computer, software means for operating the controller and the imaging means to acquire segmented images of objects on the slides in precise focus, ~oftware means for analyzing the features of the images, visual display means for displaying the images of the ob~ects, means enabling the operator ~. '. ' . , :

20~6786 to identify images which are members of a class, software meane for performing user activated automatic multivariate analysis on the features of the images identified by the operator, and software means by which the results of the multivariate analysis are utilized to identify other objects belonging to the class.

According to another of its aspects, the invention i8 a method of analyzing microscopic objects compri~ing the steps of acquiring focused images of individual objects, analyzing the features of each image and recording the re~ulting data in a storage medium, an operator identifying a plurality of individual images to be treated as being of the same class, and performing user-activated automatic multivariate analy~is of the features of the identified images and utilizing the results to identify other images which are potential members of the cla~s defined by the operator and recording the identity of such potential members in a storage medium.

DESAIL~D DE8CRIPTION OF THE PREF~RRED EMBODIMENT

~ he invention may be more fully appreciated by reference to the following detailed description of a preferred embodiment in conjunction with the as~ociated drawing in which:

Figure 1 is a schematic diagram of the preferred embodiment of the present invention.

The preferred embodiment of the invention i~ described in relation to a ~ystem for analyzing cells on slides such a~
might be delivered to a cl~nical lab for diagnosi~, and in particular for analyzing cell nuclei.

Referring to Figure 1, there is illustrated a microscope 10 having a digital camera 12, an ocular 14, and objective lens 16.

A motorized stage 18 i8 provided with a step motor having elements 20x, 20y and 20z for effecting movement of the motorized stage 18 in the x, y, and z axes. The operation of the motor is controlled via the computer 22 through the x, y, z controller 24. An automated slide loader 26 is provided which i8 capable of receiving a set of slides and delivering individual slides to the motorized stage 18 of the microscope 10. The automated slide loader 26 is linked to the controller 24 to coordinate the positioning of a slide on the motorized stage 18 and the retrieval from storage of particular Rlides to be reviewed.

The digital camera 12 is preferably of high spatial and photometric resolution equipped with a ~cientific CCD detector tsuch a~ the MicroImager 1400 from Xillix Technologies Corp.) and i~ mounted on the camera port 28 of the microscope 10.

, : . . ".

,.

^` 2086786 An imaging board 30 is provided for enabling the images viewed by the camera 12 to be stored in the memory 32 of computer 22.

A high resolution video monitor 34 i8 provided to enable the display of the digitized images. The computer ~8 also provided with software for analyzing the images, for enabling the display of multiple images simultaneously in a matrix on the monitor, and for displaying graphical data derived from the images. A mouse 36 iB provided to enable an operator of the apparatus to interact with the displayed images or with the graphical displays on the monitor.

The operation of the system as described below is aontrolled by appropriate software as will be appreciated by those skilled in the art.

Cells whose nuclei are to be examined according to this embodiment would be delivered to the system with appropriate staining to highliqht the cell nuclei.

The system is programmed to move to a fra~e on a microscope slide, examine if there are any nuclei in the frame, and if 80, find first an approximate general focus. The system then determines a basic threshold based on a preselected feature such as intensity to identify stained nuclei. Once the basic threshold is established, the imaging board captures all images of nuclei abiding by preselected criter~a ~uch a~ size, opt~cal :' .

.,~. , .

.

density or compactness. The po~ition of all nuclear imagee iB
recorded in the computer memory 32 by reference to information received from the stage controller 24.

In the next step, u~ing the z-drive, many different images of the selected frame are taken at different z-positions of the average focal plane. Features of the image under the ma~k of each nucleus are continuously measured at incremental z-values. The increment in Z-values is chosen such that it corresponds roughly to depth of field of the selected objective.

Thereafter, only those images of the nuclei which are in focus are kept in the computer memory. A new segmentation mask is created to correspond to the precisely in-focus image.
Features of the images are then calculated and the new mask as well as the features are stored in computer memory.

This is a fast process and in a few minutes several i hundred in focus images can be collected from a typical i microscope slide depending on the density of the nuclei on the slide ~e.g. 20 - 200 nuclei per frame).
,! ~
The images are displayed on the video monitor for viewing the segmented im~ges of the nuclei. Over 100 nuclel can be displayed at a time as an ~m~ge matrix on the monitor and the operator is able to flip through all pages of images by pressing a function key on the computer.

,~ ~
;

The image matrix of the monitor is keyed to the mouse and to the stage controller by software such that when the operator selects a particular image or ob~ect of interest from the monitor and operates a function key on the computer, the controller will operate the motorized stage to position the slide under the objective 80 as to enable the operator to examine the object of interest in precise focus.

The user activated classification of nuclei proceeds as follows. Using the mouse, the operator points to a few nuclei in the image matrix which are to be treated as being within a class. The computer records the images 80 identified.

When a human operator has pointed to a few characteristic nuclear images of a selected class, (for example, cell nuclei of intermediate ectocervical cells which are intermixed with superficial and basal ectocervical cells, metaplastic cells, leucocytes, debris, etc.) a process which takes one or two minutes, a computer program establishes the mean value for each feature identified and calculates the standard deviation. Using a combination of the means and standard deviations of all features, new minima thre~holds and maxima thresholds of all features for the desired class are calculated by the computer program. Thereafter, only images of those nuclei which meet the new criteria are displayed on the video monitor.

The operator can adjust the combination of means and standard deviations for each individual feature or u~e a default -: , - : .

-` 20~6786 value, e.g. mean +3x St. Dev. to e~tabli~h the new minima and maxima for all features. After this process, the sy~tem i8 substantially trained, as very few nuclei other than tho~e targeted will pass the new ~filtern.

An alternative approach to identifying typical class members i~ by the operator controlling the microscope stage, for example by using a joystick 38, to bring a desired nucleus under the crosshair of the ocular and pressing an appropriate function key on a computer keyboard 40.

Aiternatively, the same selection of nuclei can be accomplished by displaying selected features on histograms or two-dimensional feature plots. The operator can then use the mouse to graphically select tho6e objects represented in the feature space that are to define a class.

The above process may sometimes result in an accuracy or a filtering that is less than is desired for a particular application or a particular operator. For further training of the system a different procedure is used. The nuclei selected by the first filter are predominantly the desired nuclei.
However, a significant proportion (e.g. typically from 5 - 20~) of undesirable images escape such simple filters even when over 100 features are used. In the ~econd step of training the ;m~ges of nuclei retained by the computer as being within the desired class are displayed on the monitor. The operator points out the undesirable nuclel image~ on tho computor screen (or , . .: .
:. : , .

alternatively the operator identifies points on the histogram and/or scatter plots by using the mouse). In this way the operator quickly establishes two classes of images: "right~ and ~wrong ones. To point out even several hundred wrong images by a pointer i8 a very fast procedure taking generally not more than a few minutes even if most objects are examinet under the microscope. Each nucleus selected from the computer screen i8 automatically brought under the crosshair of the microscope for the operator~ observation if 80 desired. This is accomplished by the computer program which draws upon the recorded position of the in focus image to direct the motorized stage controller.
Nultivariate analysis (discriminant function analysis) and decision tree processes are programmed in the computer such that the best discriminating features are used to characterize ~wrong~
nuclei and to further separate the wrong nuclei from the de~ired (right) ones.

The system can be trained in the ~econd phase in a single session, or it can be retrained several times. The best operator is an experienced expert ~e.g. a biologist or pathologist in the case of cells rather than a computer expert (mathematician, physicist, electrical engineer, etc.) as the teaching of the system involves only pointing at the ~right~ type of cells.

A significant feature of this system is that it can be trained as described above simultaneously on one or more selected classe~. Once the ~ystem has been ~o trained and ha~ committed ' ' ` ' ` ~ ' to memory the results of the foregoing classification analysie it can apply this training while scanning new slide~ of similar material prepared under identical procedures, ~e.g. stain, deposition, etc.). That is, the system will have been ~trained~
durinq the prior session 80 that it can now decide during image capture and feature extraction, which objects belong to each class thereby obviating the need for the operator to retrain the system with each new set of slides.

It will be appreciated by those skilled in the art that a number of aspects of the preferred embodiment described herein are not essential to the invention and suitable alternatives or substitutions may be made without departing from the principles of the invention.

... .
.:, . : ' . ' ., ~ -' .

Claims (9)

1. Apparatus for imaging, processing and analyzing objects comprising:

a microscope;

a motorized microscope stage for receiving slides containing objects to be analyzed;

a controller for controlling the movement of the motorized stage along the x, y and z axes;

imaging means optically connected to the microscope;

a computer;

software means for operating the controller and the imaging means to acquire segmented images of objects on the slides in precise focus;

software means for analyzing the features of the images;

visual display means for displaying the images of the objects;

means enabling an operator to identify images which are members of a class;

software means for performing user activated automatic multivariate analysis on the features of the images identified by the operator; and, software means by which the results of the multivariate analysis are utilized to identify other objects belonging to the class.
2. Apparatus as in claim 1 further comprising means for displaying a plot of features of the images and means whereby the operator can identify a plurality of images to be treated as being of the same class by identifying points in the plot.
3. Apparatus as in claim 1 further comprising software means for operating the controller to bring into the field of view of the microscope an object whose features have been previously analyzed when an operator selects the image from said visual display means.
4. A method of analyzing microscopic objects comprising the steps of:

(a) acquiring focused images of individual objects;

(b) analyzing the features of each image and recording the resulting data in a storage medium;

(c) an operator identifying a plurality of individual images to be treated as being of the same class;
and, (d) performing user activated automatic multivariate analysis on the features of the identified images and utilizing the results to identify other images which are potential members of the class defined by the operator and recording the identity of such potential members in a storage medium.
5. A method as in claim 4 comprising the additional steps of:

(e) identifying to the operator which images have been identified as potential members of the class in step d;

(f) the operator identifying a plurality of images which are not properly within the class;

(g) performing multivariate analysis of the features of the images identified in step f and identifying other images which are not properly in the class; and, (h) recording the identity of the images which are properly in the class according to steps c to g.
6. A method as in claim 4 wherein the operator identifies a plurality of images to be treated as being of the same class by identifying points in a plot of features of the images.
7. A method as in claim 5 or 6 wherein the operator identifies a plurality of images which are not properly within the class by identifying points in a plot of features of the objects.
8. A method for analyzing microscopic objects comprising the steps of:

(a) performing the method of claim 4 using the apparatus of claim 1;

(b) providing a new set of objects to be examined by the said apparatus;

(c) the apparatus automatically examines images of the objects and determines whether each image is within the class defined by the method of claim 4 at the time the features of the images are analyzed.
9. Apparatus as in claim 2 further comprising software means for operating the controller to bring into the field of view of the microscope an object whose features have been previously analyzed when an operator selects points from the plot.
CA 2086786 1992-10-14 1993-01-06 Trainable automated imaging device Abandoned CA2086786A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US96127392A 1992-10-14 1992-10-14
US07/961,273 1992-10-14

Publications (1)

Publication Number Publication Date
CA2086786A1 true CA2086786A1 (en) 1994-04-15

Family

ID=25504268

Family Applications (1)

Application Number Title Priority Date Filing Date
CA 2086786 Abandoned CA2086786A1 (en) 1992-10-14 1993-01-06 Trainable automated imaging device

Country Status (1)

Country Link
CA (1) CA2086786A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5930732A (en) * 1995-09-15 1999-07-27 Accumed International, Inc. System for simplifying the implementation of specified functions
US6091842A (en) * 1996-10-25 2000-07-18 Accumed International, Inc. Cytological specimen analysis system with slide mapping and generation of viewing path information
US6118581A (en) * 1995-09-15 2000-09-12 Accumed International, Inc. Multifunctional control unit for a microscope
US6148096A (en) * 1995-09-15 2000-11-14 Accumed International, Inc. Specimen preview and inspection system

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5930732A (en) * 1995-09-15 1999-07-27 Accumed International, Inc. System for simplifying the implementation of specified functions
US6118581A (en) * 1995-09-15 2000-09-12 Accumed International, Inc. Multifunctional control unit for a microscope
US6148096A (en) * 1995-09-15 2000-11-14 Accumed International, Inc. Specimen preview and inspection system
US6091842A (en) * 1996-10-25 2000-07-18 Accumed International, Inc. Cytological specimen analysis system with slide mapping and generation of viewing path information

Similar Documents

Publication Publication Date Title
US4700298A (en) Dynamic microscope image processing scanner
US5333207A (en) Inspection apparatus and method with inspection auditing for images presented on a display
US5933519A (en) Cytological slide scoring apparatus
US5939278A (en) Automated histological specimen classification system and method
US5655029A (en) Device and method for facilitating inspection of a specimen
CA2219432C (en) Cytological specimen analysis system with prescreening and generation of viewing path information
JP6453298B2 (en) System and method for observing and analyzing cytological specimens
US20070031043A1 (en) System for and method of intelligently directed segmentation analysis for automated microscope systems
DE3313789A1 (en) SELF-ACTING MICROSCOPIC EXAMINATION DEVICE AND METHOD FOR DETERMINING AND RECOVERING OBJECTS IN AN IMAGE
WO2007084722A2 (en) Automated microscopic sperm identification
US6535626B1 (en) Inspection system with specimen preview
CA2086786A1 (en) Trainable automated imaging device
DE19801400C2 (en) Process for the automatic detection, description of properties and interpretation of Hep-2 cell patterns
KR101962869B1 (en) Bone marrow interpretation supporting apparatus based on image analysis
Bonton et al. Colour image in 2D and 3D microscopy for the automation of pollen rate measurement
JPH08315144A (en) Device and method for pattern classification
WO2023031622A1 (en) System and method for identifying and counting biological species
DE19637741B4 (en) Automatic cytology sample analysis system using individualized patient data
JP3654835B2 (en) Test system with sample preparation function
Smit et al. A commercially available interactive pattern recognition system for the characterization of blood cells: description of the system, extraction and evaluation of simple geometrical parameters of normal white cells
JPH03123860A (en) Chromosome inspecting device
EP0774734A2 (en) System and method for computerized archiving
Rosen ANNOTATION: AUTOMATED MICROSCOPY
WO2023112002A1 (en) Photomicrographic image-processing method for automatic scanning, detection and classification of particles
Brugal Bone marrow cell image analysis by color cytophotometry

Legal Events

Date Code Title Description
EEER Examination request
FZDE Dead