CA2297119A1 - Inspection system with specimen preprocessing - Google Patents

Abstract

A clinical laboratory integrated screening and diagnostic system for preprocessing and inspection of cytological samples or other specimens. A processor receives digital images of each specimen and, using statistical analysis, estimates probabilities of atypia for objects or regions contained in the specimen. Based at least in part on these probabilities, the procesor then estimates a probability of atypia for the specimen as a whole. A specimen having high enough estimated probability of atypia is automatically categorized as suspicious or abnormal and is in turn screened by a human technician with the assistance of a preview system and a computer-assisted microscope workstation. A sample with sufficiently low or no probability of atypia is automatically categorized as apparently within normal limits and is then either screened by a human technician for quality assurance or automatically reported as within normal limits. After screening by the human technician, samples that the technician deems to be suspicious or abnormal are reviewed by an expert for final diagnosis.

Description

INSPECTION SYSTEM WITH SPECIMEN PREPROCESSING
1. Field of the Invention The present invention relates to specimen or sample inspection systems and more particularly to systems in which human operators inspect a substantial number of individual S specimens to locate a particular subset such as "suspicious," irregular or abnormal specimens.
As used herein, the term "specimen" is not necessarily limited to a medical or biological specimen but may more generally extend to any sample item or portion of a group as a whole.
The present invention may thus find particular use in a variety of contexts such as, for example, examining histological specimens (i.e., tissue-section-based as in anatomic or surgical pathology}, examining cytological specimens (i.e., cellular samples (such as those samples prepared from specimens taken from body cavity fluids, voided urine, sputum, and gynecological tract) as analyzed by cytotechnologists and cytopathologists, cytogeneticists, hematologists, neuroscientists, microbiologists, cell biologists, etc.), examining silicon wafers in an integrated electronic circuit manufacturing process in the semiconductor industry, and other materials inspection processes.

2. Description of the Related Art In a typical scenario, a human inspector must inspect and analyze a substantial number of specimens each day to determine whether the specimens deviate from some predetermined normal range. Abnormal specimens are identified and are subject to further, more detailed review and analyses. The subsequent, more detailed review may require a reviewer with additional expertise, such as a pathologist in the case of the cervical Pap test.
In a usual case, for example, of screening samples from asymptomatic women, most of the specimens are considered "normal," or "within normal limits" ("WNL") and therefore need not be subjected to additional scrutiny. Depending on the detail and scope of this inspection and analysis, this additional scrutiny can unfortunately be a very slow, painstaking, error-prone and costly process.
For purposes of illustration, the present invention will be described in the context of cytological specimen analysis, such as cervical Pap smear or Pap test analysis. Pap smears, which are routinely taken from women, facilitate the detection of pre-cancerous changes and/or the early stages of cancer, thus reducing the chances of any cancer or related abnormal condition spreading or advancing in clinical staging with the resultant negative impact on the prognosis for the patient. A Pap smear is prepared by first collecting a vaginal, cervical and endocervical tissue sample from a patient. The sample is then fixed to a slide, for instance by alcohol fixation, and appropriately stained (in a manner similar to the Papanicolaon (Pap) procedure) to enable microscope-based visualization and analysis. Figure 9 illustrates a conventional Pap smear.
The conventional Pap smear process is limited, because it produces large numbers of cells (50,000 to 300,000 cells per Pap smear, typically) that are often obscured by inflammatory and other materials that make accurate and sensitive diagnoses more difficult in some cases. The Pap smear process is also limited because it deposits the cells in a spatially non-uniform manner that is difficult and time-consuming to analyze visually, as shown, for example, by Figure 9.
Several alternative monolayer sample or liquid-based preparation (LBP) approaches have therefore been developed. One method depends upon centrifugation to separate cells before deposition onto a glass microscope slide, for example. Another method relies upon physical filtration of a specimen through a filter with a controlled pore-size distribution. An example of the latter approach is the Cytyc ThinPrep° instrument and process.
The Cytyc ThinPrep~' sample preparation generally comprises straining a sample through a filter having pores smaller than the average sample cell diameter but sufficiently large to allow passage of cellular fragments and other debris. In one research-use embodiment, the LBP device pore diameter is about 40-50 micrometers. The filter preferably has no rough edges or other features that would rupture the sample cells. Upon filtration, sample cells remain on the filter surface while the debris passes through, resulting in a filter surface enriched in sample cells and depleted in debris. The filter surface is next pressed onto a microscope slide to transfer the sample cells from the filter surface to the slide. This results in the slide having a more uniform distribution (i.e., monolayer) of cells and, by employing a filter having a smaller area than the slide, a higher concentration per unit area of sample cells, substantially free of cellular and other debris that could interfere with a cytotechnologist's ability to detect cellular events and analysis of a slide.
The Cytyc ThinPrep~ sample preparation thus provides a more spatially uniform distribution of cells within the cellular disk (CD), as shown, for instance, by Figure 10.
In practice, after the sample is prepared from the specimen, the slide is then screened by a highly skilled technician ("cytotechnologist"), in an effort to determine the specimen adequacy and to identify possible cellular abnormalities in the specimen. The cytotechnologist generates notes regarding each specimen deemed to have possible abnormalities. The cytotechnologist then provides the specimen slide, together with these WO 99/04244 PCTlUS98114719 paper-based or electronic notes of his or her findings, to an expert cytopathologist (i.e., specialized physician) for further review and final specimen diagnosis.
To screen a Pap smear specimen, the cytotechnologist generally views the Pap smear slide containing the Pap smear through a conventional optical microscope to detect the presence of potentially rare cancer cells or cells exhibiting other abnormal conditions.
Because a cancerous cell may appear in only one of thousands of locations in an otherwise normal-appearing specimen, however, the cytotechnologist must generally examine every area of the slide in order to make a valid (i.e., accurate) determination.
Overlooking any area could potentially result in a false negative (FN) diagnosis. Further, many portions of the specimen slide may contain no cells at all, but the cytotechnologist must examine even those areas to at least determine the absence of pertinent (i.e., diagnostically significant) material.
Of course, this process of thoroughly screening a specimen for the presence of cancerous or abnormal cells is often laborious, tedious, error-prone and costly. Still, cytotechnologists have been known to examine more than 20,000 slides annually in an effort to classify specimens as within normal limits and to identify abnormalities and enable pathologists to diagnose Pap smear specimens. In many cases, this specimen review rate is driven in part by financial concerns such as competition based on the number of specimens analyzed. The dilemma faced by laboratories is that they need to increase their specimen throughput rate for economic reasons and simultaneously to reduce their specimen throughput rate to lower their false negative error rate.
Based on the cytotechnologist's primary review (i.e., screening) of the specimen, the cytotechnologist determines that the specimen is unsatisfactory, satisfactory but limited, or satisfactory for analysis. If satisfactory, then the cytotechnologist determines either that the specimen contains suspicious material such as pre-cancerous or cancer cells, or that the specimen is apparently within normal limits. Typically, statistically speaking, "suspicious"
and "abnormal" specimens may account for approximately 5% to 10% of Pap smears in the United States, in laboratories that are screening asymptomatic women. The remaining statistical 95% to 90% of the cases in turn are classified by the cytotechnologist as appalrently normal.
If a specimen contains even a single well-preserved and well-stained cancer cell out of tens or hundreds of thousands of cells, the cytotechnologist should find the specimen to be suspicious, or atypical or abnormal. Failure to identify a specimen properly as abnormal during this screening process may be disastrous, as it may leave a cancer undetected and untreated, and may ultimately lead to the death of the patient.
The cytotechnologist forwards all "suspicious," or "atypical" or "abnormal,"
specimens to a pathologist for detailed review and final diagnosis and "sign-out" in light of the cytotechnologist's notes and findings. One of the pathologist's goals is to analyze the specimen at issue and determine based on medical expertise whether the specimen contains cancerous or pre-cancerous cells. In doing so, the pathologist must strive to minimize both false negative diagnoses and false positive diagnoses, as false negative diagnoses could leave cancer undetected, while false positive diagnoses could result in unnecessary or inappropriate, harmful and costly cancer treatment such as surgery, chemotherapy or the like.
Most of the specimens that the cytotechnologist deems to be "apparently normal" are classified as "within normal limits," (WNL) and the analysis of those specimens is completed.
However, to minimize the possibility of false negatives in the screening process and to identify cytotechnologists that may have screening quality performance problems, at least some of those specimens should be subject to a secondary screening, or "re-screening," by a cytotechnologist. In the United States, at least 10% of these "apparently normal" specimens must be randomly selected (along with known high-risk cases based on prior information) and re-screened for quality assurance by a different cytotechnologist.
In addition, to further minimize false negatives during the Pap smear screening process, cytotechnologists must spend sufficient time screening each specimen slide. For this reason, legal regulations in some states in America restrict individual cytotechnologists to screening no more than 100 Pap smear slides in a single day. Other states provide even stricter limitations, such as a maximum of 80 slides per day. Assuming an average 7 hour work day, these regulations would have a typical cytotechnoiogist screening and classifying an average Pap smear slide in no more than 4.20 to 5.25 minutes.
Notwithstanding these maximum limitations, the average number of Pap smear slides screened per day by cytotechnologists in the United States is on the order of only 50 to 60, corresponding to cytotechnologists typically spending less than 7 to 8 minutes reviewing each slide in order to carefully determine whether any abnormal cells are present.
Of course, as cytotechnologists spend more time screening each slide, they will theoretically make fewer false negative errors. At the same time, however, as cytotechnologists spend more time screening each slide, they will screen fewer slides each day, and the labor and cost of specimen screening will consequently rise. This is a difficult scenario for laboratories, in the fW.l. YV:\~L:.1 A".lll.La\<.IIL:.V IIaJ ..:V- U-J.J . ~1 ~JU . -r m'*J U.J .V :JJ'Y'ltJ.l ~ » .J
AU~-TU-88 15:03 Ff0I1- CA 02297119 2000-O1-14 T-8fl P.05/14 F-Z64 United States for exataple, since the third-party health insucancc reimbursement rafts arc typically so low that Pap leafs arc nominally or not profitable for many, if not most, clinical diagnostic cytology laboratories.
A need therefore exists for a more e$cient specim~ sv.~xeeniug system that minimizes the presence of false negative specimen classification errors while reducing the tune required to analyze specimens accurately and to coutpilc useful informarion about suspicious, atypical, or abnormal specimens for reference by diagnostic ocpcrts_ 3. Iueorporation by Aeferetrce The following U.S. patent applications arc expressly incotporatcd hereixi by reference:
(i) U.S. Patent Application Serial No. 081529,220, filed September 15, 1995;
(ii) U.S. Patent Application Serial No. a8~529,198, filed September 15, 1995;
and (iii) U.S. Patent Application Serial No. 08/736,790, filed October 25, 1996.
4. Background References U.S. Patent No. 4,S 13,438, to Crraham et al., describes a tncthod and system for locating and re-locating objects-of interest in an imago. T~he objects-of interest tray be biological cells on a microscope slide. According to Graham, a detector scans the image and detects objects, and a feature extractor identifies objects of interest such as by comparison to a stored rcfcza><ce vector of features- Au object marker then generates a marker signal for the object-of interest, including features of the objoet as well as distance and oricntatiott of neighboring objects, and the market signal is stored. Iu turn, if the microscope -slide is displaced, a conzparator may re-locate objects of iztte~rest by reference to the rnar~er signals, and the system tray notify an operator that the object has been re-located.
Crraham discloses that a specimen can be automatically analyzed to identify ubjects of interest, and, if the spxitnen appears abnormal, the system can indicate that the specimen should be reviewed and can automatically re-locate selected areas of the slide.
U.S. Patent No. S,62S,705, to Recht, describes a mothod for classifying objects in a specimen as likely to °be premaligtant or malignant cells, basod on as inteasiry texture analysis. According to Rccht, as automated microscope systean first scans a specimen slide at low resolution, an image processor identifies areas that tnay contain biological matter, and the image of the area is stored together with an indication of its focal place. The system then scans the specimen at a higher resolution and conducts a gray scale analysis of each identified area to determine (c.g., by size, gray scale density or surface intensity texture characteristics) if the area contains a pretnalignant or malignant cell. If so, au area of pixels encompassing -S-'.»..n. ~a",o~
AMENDED SHEET
700 SOUTh w~CK111 Ouwt dae~4o. r.r.asweoe TEILY110wt (3121 lli.m01 m..v. vv:v~w n-.rm.m.n.m.v wu .LV- u-u:u . ..:.i~.~c7 ~ ~ 't'Y.7 OJ
.:..~:7:Tt'tUJ~N c Aua'EO'88 i::03 From' CA 02297119 2000-01-14 T-8T1 P.U6/14 F'Z64 the cell is generated, and the system re-scarfs the area at a high resolution to produce a high resolution color image tile ("net image") of the cell and its surroundings.
For each such tilt, the system, then detenuines a likelihood ("act value") that the object is a premaligtiant or malignant cell. Finally, the system geueratcs and displays an 8 x 8 suznmary screen of the tiles having the bxghcst net values, which a cytoteclmician eau scan in search of an object having attributes of the call type for which elassifieanori is being performed.
International Patent Application Publiearion No. WO 96!03709 A1, to Neuromedieal Systems, inc., describes a method far facilitating review of :~ selected speciram or area of interest with a microscope. An automated microscope scans a specimen image, and an image processor mntpholo,gically filters the image to identify possibly malignant or prcmaligaant cells. After additional processing, images and their coordinates arc stored for display on a tnosiitor. The 64 highest ranked cells arc rhea presented to a cytotechnician in an 8 x 8 mania;, or other sequence (c.g., simultaneously or separately) or arrangement such as four screens of a 4 x 4 matrix of tiles. In order w ensuxe that the cytotechnician looks at each of the tilts, the system then requires the cycotechaieian m position a cursor on each itnagt for a set period of time determined to be sufficieully long for inspection of a tile.
European Patent Application Publication No. Q 647 $44 A2, to ~ioffman-La .Rochc, describes a system for expedited handling of cytology samples, involving intezactioa between a cytotechnologist and automated machine analysis. A system autamaticatly analyzes cells in the specimen and makes a diagnosis based on this azixlysis. The system then generates a gallery of cells and displays the gallery to a cytologist. The cytologist t~eviews the gallery, makes an indcpendern determination whether any of the ells arc abnormal (without yet !mowing the systeta's diagnosis), and inputs the detMrnination into the system. The system then wmpares its review with the cytologist's review. If both believe the slide to be normal, a "normal" diagnosis is reported; if either find abnormalities, the slide is forwarded to another cytologist for full revidw. Alternatively, if the cytologist finds the slide to be normal but the computer does not, the gallery eau be reviewed by a supervisor err re-reviewed by the cytologist.
International Patent Application Publication Na. WU 95/02204, to CourpuCyte Corporation, describes a coxaputerized slide encoder. The encoder provides for establishing a marker that indicates microscope viewing areas corresponding to pixel locations reviewed.
The system may, for instance, establish a u~ark at a location every set time period, so that the - Sa -rcoo~u eoE~u~
~u~ws s wicr~or~
70D lOUTn ~wCtilt M~
U.G~p. uuw0li 10f0i ~~E~a~n9W 000 AMENDED S;~EE~

~~ ,. ~ wr.W .r roJ~t~YVa~. m r ~Aua-ZO-si9~~ 15:04~V~~~~ Frog- ~v ~ '~CA~ 02297119. 2000-01-14 ~ T-971 ?.Uf/14 F-Z64 density of marks may indicate how lor<g a technician has spent reviewing particular areas of the specimen. The technician may also record an audio record about areas reviewed.
lnterttarional Patent Application Publication Ir'o. WQ 97!25678, also to Co><npuCyte Corporation, describes a network system for review and analysis of microscope slides and specimens that have been previously examined. The system provides for at least two separate microscope stations in a network that ate individually linked to a cotaputer that can recall stored movements and location iuforn>u;stioa from a storage tneditun, for review at a location remote from the microscogc used in the origi>AaI exa»tinauoa. In turn, the specimen slide can be placed on the microscope, and the microscope relocates to slide images st designated sites.
Faeh rnicraseope station may further have access to patient background (e.g., demographics medical history) inforu>atiorr as wdl as an on-line library of call type images for comparison with the slide being reviewed, to aid in diagzsostic decisions.
U.S. Patent No. 4,812,909, to Yokobayaslzi et al., desczbes a1a image display apparatus that cart simultaneously display an image of a blood sataple alad data regarding the blood sample, so that au operator can observe the combined images on a single CRT. A
switch ertablcs selective display of a picture scene only, a character scene only, or an overlapping scene of a cbatacter scene and a picttue scene.
woOppWfw IOEwwfw 1~f i ~r~OrF
7W IWUT. wwCaf7t01wf ~ r~ ~ ~l~f-, C
CrrCwtb. uun0~61(~i0i I..y~ t J,~y . ,.,,'~;~
nm.,o~a u,s~ s,s.aoo, SUMMARY OF THE INVENTION
The present invention provides an integrated system that improves efficiency in specimen inspection and analysis. For purposes of example, and without limitation; the invention will be described herein is a clinical diagnostic cytology system.
In one aspect, the invention may be a process for inspecting multiple samples each prepared from cytological specimens, to facilitate classification of said specimens as either apparently within nairmal limits ("apparently-WNL") or apparently not within normal limits ("apparently-not-WNL") The invention may, for instance, include the following functions:
(1) Acquiring into a machine a set of digital images of the regions of the sample;
(2) The machine analyzing the digital images of the regions;
(3) The machine identifying cellular objects in a subset of the regions;
(4) The machine assessing whether the sample is satisfactory (e.g., adequate) to facilitate classification;

(5) The machine estimating a probability of cellular atypia for cellular objects identified in the sample;

(6) The machine rank ordering cellular objects in the sample according to their respective estimated probabilities of cellular atypia, a first subset of the cellular objects in the sample defining those cellular objects having the highest estimated probabilities of cellular atypia in the sample;

(7) The machine deriving an estimated probability of specimen atypia of the corresponding specimen, based at least in part on the estimated probabilities of cellular atypia of cellular objects in the sample;

(8) The machine interpreting the corresponding specimen as apparently-WNL if the estimated probability of specimen atypia falls within a predetermined probability range;

(9) The machine interpreting the corresponding specimen as apparently-not-WNL if the estimated probability of specimen atypia does not fall within said predetermined probability range; and (10) If the machine interprets the corresponding specimen as apparently-not-WNL, (i) displaying for preview by a human observer a set of biasing-information about the sample, the biasing information including a subset of the cellular objects having the highest probabilities of cellular atypia in the sample, whereby subsequent screening of the sample by the observer may be biased by the observer's preview of the biasing-information, and (ii) subsequently displaying for screening by the observer the subset of regions of the sample.
Other features and advantages of the invention will become apparent to those skilled in the art by reading the following description, with reference to drawings where appropriate.

BRIEF DESCRIPTION OF THE DRAWINGS
A preferred embodiment of the present invention is described herein with reference to the accompanying drawings, in which:
Figure 1 is a block diagram illustrating the process flow in a preliminary embodiment of the present invention;
Figure 2 is a flow diagram illustrating components employed and functions performed by a prescreening and screening system in the present invention;
Figure 3 is an illustration of a automated microscope-based screening station that may be employed in the present invention;
Figure 4 is a flow chart illustrating the process flow in a preferred embodiment of the present invention;
Figure 5 is a functional block diagram of the process flow in a preferred embodiment of the present invention;
Figures 6a and 6b are illustrations of multiple information windows displayed in preview monitor in accordance with a preferred embodiment of the present invention;
Figure 7 is an illustration of discrete cell images in an electronic monolayer preparation (EMP) as displayed in a preferred embodiment of the present invention;
Figure 8 is an illustration of a microscope field-of view containing cellular matter of interest within a preferred embodiment of the present invention;
Figure 9 is a graphical representation of a conventional "Pap Smear" cervical cytology slide;
Figure i 0 is a graphical representation of a Cytyc ThinPrep'R~ cervical cytology slide;
Figure 11 is an illustration of areas within a cellular disk (CD) that are occupied by cells or other light-absorbing objects, with circles overlaid on the CD to show the relative sizes, counts and positions of the typical fields-of views; and Figure 12 is a graphical representation showing coordinates and fields-of view within a ThinPrep~~ cellular disk that identify suspicious and abnormal cellular material.
_g_ DETAILED DESCRIPTON
OF THE PREFERRED EMBODIMENTS
Referring to the drawings, Figure 1 illustrates a block diagram of the system flow in a preliminary embodiment of the present invention. At block 12, a specimen is collected and a sample such as a Pap smear or LBP is prepared from the specimen, for instance on a 1 x 3 inch slide with a 1 x 2 inch coverslipped area adjacent to a 1 x 1 inch typical slide label for patient and slide identification. At blocks 14 and 16, the sample is then subjected to a cytological screening system, which includes automated or interactive machine prescreening and visual screening by a cytotechnologist. Based on the screening process, the cytotechnologist determines either that the specimen is suspicious or that the specimen appears to be within normal limits. All specimens that are deemed suspicious are forwarded to an expert for review and diagnosis, at block 18. Of the specimens that are deemed to be apparently within normal limits, at least 10% are re-screened at block 16 for quality control and particularly to reduce the possibility of false negatives.
Figure 2 shows, by way of example, some of the functions that may be performed in the prescreening and screening stages 14, 16 of this invention. In combination, these stages are adapted for use in a clinical laboratory or similar facility, and preferably include an image capture apparatus 100, a mapper 104 and an automated microscope-based screening station 110.
The image capture apparatus 100 preferably takes the form of a camera and a frame grabber. The camera is preferably a CCD (charge coupled device) scientific grade type camera with a 1 K x 1 K or larger format, and a 3 class or better sensor. Such a camera is available commercially under the trade name ES-1 from Kodak Corporation, Rochester, New York, and is also available from Pulnix America, Sunnyvale, California. Such a camera is characterized by an active sensor area of 9mm x 9mm or larger with a pixel spacing of 9 microns or finer and can capture, or scan, images at a rate of at least 30 frames/second and provide a digital output at a minimum rate of 30 MHz. The optical system is configured to provide an effective pixel resolution of approximately 2.4 microns at the sample. While such a resolution is appropriate for the preferred embodiment described herein, it may be changed for other applications. The specifications stated herein are illustrative of a particular preferred embodiment and may be altered. As an example, a camera with a format larger than 1 K x 1 K would reduce the number of images to be captured, because each captured image would contain a larger portion of the slide. As another example, a pixel spacing of finer than 9 microns would result in higher spatial resolution, subject to optical physics limitations.
The camera provides its digital output to a frame grabber, which operates to store the digital data received from the camera. The frame grabber preferably employs a PCI type S interface and is characterized by a data transfer rate of at least 50 MHz.
In addition, the frame grabber preferably also employs digital signal processing for optical shade correction and blob finding. A preferred frame grabber takes the form of a Data Raptor type frame grabber available from Bit Flow Corp., Woburn, Massachusetts. In an alternative embodiment, the frame grabber may perform certain image analysis and enhancement functions by way of specialized hardware devices, to provide a speed increase over performing such functions in software. For instance, the frame grabber may be configured with specialized hardware, such as digital signal processing circuitry, to perform some of the functions described below as being performed by software.
Image capture of a sample on the slide 102 is preferably performed by subdividing the 1 S slide into a plurality of equally sized regions, illustrated by the dotted lines in the slide 102, and individually capturing digital images of the sample, region-by-region. The digital image of each region is stored in a memory once captured and is analyzed by the mapper 104. The regions of the slide shown in Figure 2 are simplified for sake of illustration. In practice, a slide will typically have far more regions than shown in Figure 2. For example, a typical slide that measures approximately 75mm x 25mm, with an area of roughly SOmm x 25mm being occupied by a sample. Such a slide will contain approximately 200 non-overlapping regions of approximately 2.Smm x 2.Smm.
In the preferred embodiment, the mapper 104 is implemented as a software program stored in a semiconductor, magnetic, optical or other similar type of storage device and executed by a general purpose digital computer. One such slide-mapping system is the TRACCELI.~ system available from AccuMed International, Inc., of Chicago, Illinois. The mapper 104 performs automated image analysis of the captured digital images.
For example, the mapper may operate to automatically analyze each region for the presence of cytological material. If any cytological material is detected, the region is designated by the mapper as a "screenable" region. In addition, the mapper may identify and exclude from subsequent analysis normal squamous and epithilial cells. As another example, the TRACCELL~ system may be configured to make preliminary determinations about the sample as a whole, such as whether the sample is satisfactory (e.g., adequate) for analysis.
Unsatisfactory samples may rv v u.J ~ .~l ~UU
T'YaJ o7 '~J.TYYUJ.x O
~ ~AUWZO-9a~ I~5:04~~~~y~~Ffom- CA 02297119 2000-01-14 ' T-8T1 P.O6/14 F-Zti4 thrn be identified and returned without further analysis. The basis for as "unsatisfactory"
specimen may inc3ude unacceptable low cell coutats and an umia-stained or ova.-stained sample.
Once al! regions of the slide 102 have been c»ptuted and analyzed as indicated at S block 106, the mapper 104 gaaerates a plurality of tiles as indicated st blor.Ic 107. For simplified illustration, these tiles are shown as circles within the slide 102 at the screening station 11 U. finch of the tiles ~y correspond to a field-of veew selected for review by the cytotechnologist using the microscope at the screddztg station. Collectively, the tiles surround all of the cytological material determined by the mapper to be re4uired for viewing by the cytotechnologist. For this reason, as those of ordinary skill in the art will apprraciate, other tiling shapes and configurations, such as hexagons, array alternatively be employed to further improve screening ef$cieacy.
The mapper 104 assigns spatial slide coordiszates (including a focal plant coordinate) to each tile or sale region of interest and develops a routing function defusing an optimal 1 S route for nucroscopic display of the des;gnated areas of the sample. The mapper then transmits the coordinates to the screening station 110.
The screening station includes a nucroscope with a motorized stage and focus drive assanbly, each of which may be operated by computrr control or by an operator employing an ergonomic input device, or by a cotabixvation of coiuputtr and human control. The screening station is coupled tn the tuapptr 104 via a data conununication link and, upon receiving a series of coordinaxes from the mapper, displays microscopic fields-of view of the areas designated by the trapper in accordance with the muting fuzzcdaal, err muting pattern, developed by the ataagpcr.
A preferred sct~eeuing station is the ArCE~L~ specimen saeruitag station produced by AccuMed International; Tnc., of Chicago, IUiaois_ Figure 3 illusarates an example of this station 110, which includes an automated clectranic sad optical imaging micmscope (or video microscope) 210;"to which a motorized stage 21~, motorized focus driver (not shown) and motorized tuaet 220 have been f ttcd. The autotnatcd microscope 210 may be an Olympus $X-40 microscope, available from Olympus Optical Corporation of Tokyo, Japan and preferably includes a set of lenses 216 individually seletaable by a anotorixed control.
The screening station 110 includes a slide ataagaziue 218, a slide holder 219, a bar code reader and painter 221, and a light sourcx 222. The motorized stage 214 cuovcs along as axis dcsigaratcd as the Y-axis in Figure 3. In twn, slide holder 219 is connected to the motoriza3 "w ~w m.rrar.rue~rof:
7_0i ~1/1h ~_ON1K
tl4VIl0~ yt~ 10001 AMENDED S~iEET

WO 99/04244 PCT/US98/147~9 stage 214 and is itself motorized to move along an axis designated as the X-axis in Figure 3.
A controller board within station 110 receives external control signals to control the operation and movement of the motorized stage and slide holder, thereby providing autorniated movement of the specimen slide 102 in two dimensions relative to the microscope lens 216.
S In a preferred embodiment, the camera of the image capture apparatus 100 is affaxed to a video port on the microscope 210, in order to capture cell images and avoid havirxg to .
move the slide 102 between the microscope and the camera. Alternatively, another embodiment excludes the intervening video port entirely and integrates the image capture apparatus with the microscope. The mapper 104 is in turn coupled to the screening station by direct physical data links or by way of a data network such as a local area network. V~Phile neither the physical structure of the mapper, image capture apparatus and screening station nor the manner of coupling the mapper to the screening station is critical, such an arrangement allows the mapper to be physically separate from the screening station and allows the mapper to exchange information with a plurality of screening stations. Alternative arrangements of the manner in which the mapper and screening station are coupled, such as by way of example, a direct serial link, will be apparent to those of ordinary skill in the art in view of the present disclosure.
A cytotechnologist wishing to use the screening station 110 to view a slide inserts the slide or a group of slides into a slide Garner, which is then inserted into the slide magazine 218. The system extracts a slide from the magazine and scans a bar code on the slide using the bar code reader 221. The identity of the slide, as determined by the scanned bar code, is used by the system to retrieve coordinates from the mapper 104. The slide is then transported from the magazine onto the stage and positioned in accordance with the coordinates received from the mapper 104.
The cytotechnologist may set the speed at which he or she reviews these fields-of view presented at the screening station 110. The cytotechnologist may, for instance, accelerate, decelerate or stop the automated review process. The mode of automated review can also be changed at will by the user. Such modes include, for example, step, stop-,and-repeat screening, continuous screening, and slow-mode screening, among others.
Additionally, the cytotechnologist may at any time elect to switch to a manual review mode, for instance, in order to review surrounding areas on the slide at issue without being limited ' to the established routing pattern. Beneficially, the screening station 110 then enables the cytotechnologist to return to the automated routing pattern at the point that the WO 99/04244 PC'T/LJS98/14719 cytotechnologist began to wander away from the established path. In this way, the station 110 helps to ensure that the cytotechnologist does not miss any areas of the specimen, including those that may be potentially critical to accurate diagnosis.
In a preferred embodiment, the screening system defined in part by the mapper and the screening station 110 beneficially may be coupled to a database management system (DMS), for storage and display of information resulting from the screening process and other a priori information (such as patient demographics and medical history data) and in turn to facilitate passing pertinent findings to the expert pathologist for aid in diagnosis. The DMS
preferably takes the form of a programmed general purpose desktop computer that has sufficient storage and processing capability to run, for example, a Microsoft Windows operating environment and an advanced database application such as Microsoft Aceess.
When screening a sample, the cytotechnologist may, for instance, enter notes about an area of the sample, and those notes may be stored in the DMS together (in a database relationship) with the spatial coordinates of the area of interest, as provided by the mapper.
1 S During subsequent diagnosis, the reviewing pathologist may conveniently access the notes corresponding to a specified slide or area of a slide by, for instance, scanning a bar code or other identifying code associated with the slide, to access the corresponding information stored in the DMS. In this way, once the sample is passed to the pathologist for expert diagnosis, the pathologist may refer to the cytotechnologist's notes, various a priori information such as patient demographics and medical history, and the corresponding sample region or regions of interest, which may be simultaneously or subsequently visualized and reviewed with the benefit of simultaneously reviewing the patient demographics and patient history data.
In an improved embodiment of the present invention, an enhanced "preprocessor"
system and "expedited screening" system is introduced. Figure 4 illustrates in general a process flow according to this improved embodiment. Figure 5, in turn, illustrates in greater detail a functional block diagram of the improved embodiment.
Referring first to Figure 4, at step 20, a cytological specimen is collected and a sample is prepared from the specimen. At step 22, the sample is optically scanned, to acquire a set of image data into a preprocessor machine. At step 24, the preprocessor analyzes the digital images) of the sample and identifies cellular objects in the sample, such as normal and atypical intact cells, well stained or poorly-stained cells or cellular components, well preserved or poorly preserved cells or cellular components, subcellular organelle such as nuclei and nucleoli, regions of cytoplasm, cellular fragments, cellular debris, adjoining or overlapping cells appearing in clusters or clumps, and multiple cells appearing together as a tissue fragment. The preprocessor then eliminates areas of the slide that do not contain cellular matter, thereby identifying "screenable regions" of the slide as described above.
At step 26, the preprocessor estimates for each cellular object a probability that the cellular object is atypical ("probability of cellular atypic" or, more generally, "probability of object atypic"). At step 28, the preprocessor estimates a probability that the sample as a whole, and therefore the underlying specimen, is atypical ("probability of specimen atypic"), based at least in part on the estimated probabilities of cellular atypic.
At step 30, the preprocessor determines whether the estimated probability of specimen atypic falls within a range that would suggest the specimen is "suspicious" or "abnormal."
These ranges may be determined by training the preprocessor to mimic the classification performance of expert cytopathologists on large number of training and test slides or other control data. The ranges may correspond to diagnostic categories such as "within normal limits," "pre-cancerous" and "cancer." If the probability of specimen atypic falls within this range, then, at step 32, the preprocessor categorizes the specimen as "suspicious" or "abnormal." In turn, at step 34, the preprocessor identifies the "suspicious"
and "atypical"
cellular objects in the specimen, based on their estimated probabilities of cellular atypic.
At step 36, the preprocessor then presents those "suspicious" and "atypical"
cellular objects to a cytotechnologist for screening at a screening station such as the TRACCELL~-guided AcCELL~ workstation described above. In addition, this screening process is aided by a "preview" system. According to the preview system, the preprocessor or other data management system compiles and presents to the cytotechnologist, prior to actual screening, a set of "biasing-information" (or a priori information) about the specimen.
The biasing-information, which preferably includes discrete images of the most suspect cellular objects in the sample or the relocation of the cells in the microscope for human review prior to screening, is designed to enable the cytotechnologist to quickly form an educated opinion as to whether the specimen at issue is likely to be normal or is likely to be suspect. In turn, with knowledge of this a priori information, the cytotechnologist may efficiently spend more or less time actually screening the sample.
If, at step 30, the preprocessor's estimated probability of specimen atypic does not fall within the range suggesting that the specimen is "suspicious" or "abnormal", then, at step 38, the preprocessor categorizes the specimen as "apparently-WNL." Specimens categorized as - ~ua-20-99~ ~ 5:U4 ~..._..F'~ --,. ., .. , , r~ u.r - VvIJ-fTtlV . m v CA 02297119.2000-O1-14 T-8fl P.U9/14 F-T64 apparcatly-WNL by the preprocessor are ttteu either automatically classified as WNL, at step 40, or presented to a cytatechnolagist for xretning, at step 42.
Referring stow to Figure 5, the improved embodauent of the present invention takes the form of as integrated laboratory diagnostic system.. In Figwe 5, at block 44, a cytological speciuttn is fast collected &orn a patient, atu3 a sample sack as a Pap smear is prepared froth the spceimeu. At block 46, the samtple is then optically scatutod and analyzed by the preprocessor.
In the prefexrai anbodimertt, the pt~rocessor tray consist of one or mote machines and preferably includes at least one computer processor and a set of software routines for caxrying out the various fllaCtYDnS described below. In addition, the prepmccsser is preferably in~ta~fa~CCd with or connected to a file server, pravidiag as electronic gateway to relevant infomtation about specimens as will be described in greater detail below.
The preprocessing functions of the preferred embodiment ttu3y be etzarely automated and carried out by the preprocessor machine(s). Altcrrtatively, as indicated by block 47 in Figure S, the preprocessing functions may ittchule both fully autotaatad atachine prac~tg and human "processing" or interaction. The machine and htttnan processing may be independently pexfottned or interactive in, for ~xar~rla, a closely coupled intcractivc operating cnviraaunrnt with feedback from the machine to the hutttan and from the luuuaa to the ~.
In addition, the preprocessing fus:crions of the present invention may be tattled aui in either one pass or multiple passes_ For instance, the invaxaott may involve fit scanning a sarxsplc at low spatial and optical reaolutiorl and atutlyxing the low resolution ixttaga(s) to identify screenable regions and to eliminate artifacts. Similarly, in a fixbt pass, the preprocessing image analysis may be conducted in black and white. In tutu, for instance, the inve~ation may involve scanning the sample at a higher resolution, and poss~ly in full color, and analyzing the ixnage(s) to categorize types of cellular matter in the satrtple (such as idaati.fying "suspicious" or "abnounal" cellular objprts or regions-of intarst).
As shows by fimrdonal block 48 is Figure 5, the preprocessor serves is part as a specimen pro-scn;ener.'°This pte~screrna pteferably performs the fiutepaus of the TRwcC~~.°°
system described above, such as detecting and trtapping r~giat~s of the slide and eliminating unsatisfaetoo~y samples horn Earths ptneessiug. In addition, as shown by block SO in Figure S, the preprocessor serves in part as a "suspicious and atypical event detector and analyzer"
(also refexred to as as "atypic analyzer"). As as atypic analyzer, the preprocessor identifies suspicious arid abnorraal cellular objects is the sauiple sad, based on a statistical analysis of .~ma.~.,,~.o~r.~s"
,..~. ~~.~
AMEtvD>:C SHEET

WO 99/04244 PCT/US98/14y19 the level of atypic of these cellular objects, automatically categorizes the specimen as either (i) "apparently-WNL" or (iii) "suspicious or abnormal" (apparently-not-WNL).
To identify and evaluate cellular objects in the sample, the preprocessor preferably applies statistical and hierarchical pattern recognition techniques and thereby estimates for each cellular object a probability that the object is atypical (i.e., a "probability of cellular atypic"). More particularly, the preprocessor preferably analyzes the digitized images of sample regions or the discrete cellular objects already identified in the pre-screening stage, and compares features found in those images to certain morphological, photometric, spectral, and other features that, individually or collectively, have known meaning.
These features may include, for example, specified size, shape, color, optical density range (gray level range or contrast), optical density distribution (texture), and topology (architecture relative to other cells) and combinations of such parameters.
The more a given cellular object matches or diverges from various known features or combinations of features, the more the preprocessor may be able to draw conclusions about the probability of atypic of the object. For instance, if a particular cell nucleus shape is known to be associated with cancer, the preprocessor may conclude that a cellular object very close to that shape is very likely to be problematic -- or is very suspicious.
The preprocessor may therefore estimate a high probability of atypic for such an object.
Conversely, divergence of a cellular object from that shape may indicate to the preprocessor that the cellular object is less likely to be problematic. The preprocessor rnay estimate a lower probability of atypic for such an object.
It will be understood that the information (such as known features or "normal"
ranges) referenced by the preprocessor in evaluating cellular atypic may be population-based and/or specimen-based. Population-based information may be statistical information, such as averages, standard deviations and statistical moments of inertia, derived from large samples of patients. Such information may indicate generally that a given shaped cell is likely to have a given meaning. Population-based information may also include information established from a control specimen known to be normal for a given population, and/or a control specimen known to be abnormal for the given population. On the other hand, specimen-based information may be statistical information derived from the cells of the specimen at issue, thereby providing a personal or individualized baseline for the patient from whom the specimen was collected.

In addition, the preprocessor may also consider other factors related to the specimen at issue, in estimating probabilities of cellular atypic. These factors may include; for instance, data regarding patient medical history and demographics, such as an indication that the patient from whom the sample was drawn is particularly at risk for cancer or other diseases or has a history of abnormal Pap smears. Such information may automatically signal to the preprocessor that certain cellular objects that would otherwise be of little interest to a cytotechnologist may be more likely to be of interest. Conversely, such information may indicate that objects having generally atypical traits may in fact be normal for the particular patient. For example, it is known that many early indicators of cancer are, morphologically speaking, essentially identical to normally occurring cytological changes associated with cell repair, therapeutic treatments (such as radiation therapy) and/or various demographic factors.
Therefore, the preprocessor may adjust its estimates of cellular atypic based on such a priori information.
In the preferred embodiment, the preprocessor assigns an estimated probability of cellular atypic to each cellular object of the sample. For instance, a probability of 1.0 may represent the most atypical object (such as a clearly cancerous cell), and a probability of 0.0 may represent the least atypical or the most normal object (such as a healthy cell). The preprocessor then stores in a buffer the coordinates of cellular objects (or fields-of view containing cellular objects) that have varying estimated probabilities of cellular atypicality.
In addition, the preprocessor preferably ranks these objects in descending order of probability of atypic, thus providing a ready indication of the most suspect objects of the sample. In addition, the preprocessor may compute and store an indication of its level of confidence in each of its probability estimates, such as, for instance, an indication that it is 80% certain that an area is of interest (e.g., atypical or complex), or that it is only 20%
confident in its finding.
Based on its estimates of probabilities of cellular atypic for the cellular objects in the sample, the preprocessor then preferably derives a statistical atypic distribution for the sample as a whole. This statistical atypic distribution will serve as the basis for a quantitative an evaluation or estimate of the probability that the sample as a whole (and in tulxi the underlying specimen) is atypical (or "probability of specimen atypic"). For example, the x-axis of the distribution may be the estimated probability of cellular atypic, and the y-axis of the distribution may be the number of cellular objects in the sample having that estimated probability. The preprocessor may additionally weigh, or adjust, this statistical distribution based on a knowledge of various factors that may affect the estimate, to the extent the preprocessor did not already consider those factors in estimating individual probabilities of cellular atypia. For instance, the preprocessor might adjust its estimated probability of specimen atypia based on information about patient medical history or demographics.
Based on the estimated probability of specimen atypia, as represented by a statistical atypia distribution, the preprocessor may decide whether or not the specimen is apparently-WNL. For instance, if all of the cellular objects in the sample have zero probability of cellular atypia, then the statistical atypia distribution for the sample will indicate a zero probability of specimen atypia. On the other hand, if the distribution resembles a Poisson distribution peaking at around 0.9 probability of atypia with the tail of the distribution going toward 0.0, then the preprocessor may well conclude that the specimen is abnormal or at least suspicious. Similarly, if the atypia distribution is skewed toward higher probabilities of atypia, even though it peaks at a lower probability of atypic, the preprocessor may conclude that the specimen is apparently-not-WNL and is likely to be suspicious or abnormal.
Path "A" in Figure 5 represents the specimens that the preprocessor categorized as apparently-WNL. As shown at block 52, the preprocessor or other machine (or person) may then automatically classify and report some or all of these "apparently-WNL"
specimens as WNL. Alternatively, as shown at block 54, some or all of these "apparently-WNL"
specimens may be presented to a cytotechnologist for quality control screening. In this regard, while the U.S. Food and Drug Administration now permits fully automated classification of a Pap smear specimens as WNL, in limited cases and with restrictions, it may nevertheless be desirable to have cytotechnologists screen at least some portion of the specimens that the preprocessor found to be apparently-WNL, in order to reduce the possibility of false negative findings.
Advantageously, the cytotechnologist may conduct this quality control screening of "apparently-WNL" samples with an automated microscope workstation such as the ACCELL~
workstation described above. In this regard, the TRACCELL~-hke portion of the preprocessor may control the movement of a motorized microscope stage at the workstation, by transmitting the coordinates of specimen regions to the workstation, thereby efficiently guiding the cytotechnologist through screenable regions of the specimen.
Further, knowing that the preprocessor has already studied the sample and has found the sample to be apparently-WNL, the cytotechnologist may more quickly screen the sample than would otherwise have been possible absent the preprocessor of the present invention.

Based on his or her screening of the apparently-WNL sample at block 54, the cytotechnologist determines that the specimen is either (i) suspicious or abnormal, as shown at block 58, or (ii) WNL, as shown at block 60. All apparently-WNL specimens that are deemed by the cytotechnologist to be suspicious or abnormal are then forwarded to an expert pathologist for review and final diagnosis. In contrast, those apparently-WNL
specimens that are deemed by the cytotechnologist to be WNL are classified and reported as WNL. As shown by path "C" in Figure 5, however, at least 10% of the apparently-WNL
samples that are deemed by the cytotechnologist to be WNL are preferably re-screened for added quality control.
According to the preferred embodiment, all specimens that the preprocessor categorized as "suspicious" or "abnormal" are processed by the atypia analyzer of block 50, to distinguish "suspicious" and "abnormal" cells from "normal cells" and to exclude "normal"
cells from further processing. For instance, as to each cellular object, the preprocessor may determine whether the estimated probability of cellular atypia of the object is greater than a 1 S probability level predetermined at particular thresholds based upon statistical discrimination experiments that maximize the classification accuracy of the machine as compared to expert human diagnosis (i.e., the gold standard). If so, the preprocessor may designate the particular cellular object as "suspicious" or "abnormal." If not, however, the preprocessor may designate the particular cellular object as "normal." In this way, the preprocessor enriches the cell population at issue in each sample, focusing the cytotechnologist's subsequent analysis on only those cellular objects that are most likely to be of interest.
There may be some objects in the sample that the preprocessor cannot recognize or that the preprocessor is otherwise unable to automatically categorize as "suspicious or abnormal" or "within normal limits." To reduce the chances of false negatives, the preprocessor may designate such unknown matter or regions-of interest as "suspicious."
Alternatively, however, the preprocessor may present the unrecognizable matter to a human and interact with the human as shown by block 47 in Figure 5. For example, the preprocessor may display the object in question on a color-image monitor for consideration by a cytotechnologist, and, based on a review of the object, the cytotechnologist may input to the preprocessor a suggested designation of the object as either "normal" or "suspicious or abnormal." The preprocessor may then use the information provided by the cytotechnologist to categorize the object.

- .L~... . ~.~ . .. . ., Aus-ZO-98 15:05 Frog- CA 02297119 2000-01-14 T-8I1 ~P.10/14 F-Z64 Path "H" in Figure S represents the specituens that the preprocessor categorized as "suspicious" err "abuarrnai." As shown at block SC, the enriched popttlarion of "suspicious"
and "abnormal" cellular objects in these samples are then presented to a cytotechnologist for screeping. As with the appateady WNL specimens discussod above, the cytotecbnologist may conduct this screening with the assistance of an automated ttticrosoope worksraaon, such as the TxwcC~L~.~-guided AcCEt.t,a' workstation. In this regard, since the preprocessor bas ahnady identified the most suspect cellular objxts in the sample, the prcpracessor may etlxcieatiy guide the cytotechnalogist to saeeu only the "worst-cast" objects, thereby ncpcditing flat scracuing process.
In addition, as notod above, the cytotec4nologist's scr~ning of each sample that the preprocessor categorizod as "suspicious cu sbnotrnal° is preferably aided by a "preview" stage.
For this puiposc, the prepu~ocessor or other clefs lnaaagetaeat machine carapiles and pres~ats to the cyrotechnologist a varisry of a priori infamaation about the speoizneu, and the cytotechnologist prrvirws this a priori informariaa before actteahy screening the specimen. This preview is arranged to channel the cytotecbt~logist's attendee toward significant specitnar-related information while allowing tbc cytoteclmologist tn focus less on insiguificaat infosmatiaxt or "noise" that does not bear on whether the specimen is normal or ab~rmal. A
object of this preview stage is to minimize the possibility of a false negative diagnosis dtuing the cytotechnologigt's subse~ucnt sassing, by biasing the cytbtechaologist's attention toward diagnostically significant ipfarmatint~ An additiolnal object of this preview stage is to cnab~lc the cytotcclmalogist to couipile tuore readily the relevant information about the sQocimcn for review by a diagnosing pad~oiogist in relation to spocifie areas of the spacinzea.
Duriztg the preview stage, the preprocessor beneficially provides the cytotechuologist with a variety of useful infnrmstion for cor~sidcration by the cytotechnologist. This information may be providod to the cytotechaologist in any convatient fashion and in any form. Generally spea>ang, a file server may store some or all infottnatson patinmt to specitueats being scteeried at a gives cytology laboravory or at a remote laboratory and may serve one or more "client" prsview workstations at which peniaent data is displayed. In one embodiment, these preview display workstations may be incorporated in the same waits that are used as the screening stations, such as the AcCu.~~ workstations.
Whether coafigurai as a statZdalone unit or incorporated as pare of the scrceztiag workstation, the preview wot9cstation coatetaplated by tht present invenTian preferably includes at least one computer or video display, or other iueehanistu for conveying in an ~~w ~~
~aJWtf aP61G~a1F
70D iOJIh w~Gl~ OfIVE ~ ~,,~-GrrcrCO ~r~051CioU ~ '.!~h~~1~~ ~;'~'~' ll~Ir~E p~s)ysaCW r',:

observer pertinent information about a specimen. The workstation is human controlled, providing mechanisms to enable the cytotechnologist to flag information displayed in the preview stage that appears to be particularly pertinent. As the cytotechnologist reviews the preview information, for instance, he or she may operate a mouse or other selection device at the preview workstation, to flag pieces of information that appear to bear on whether the specimen at issue is normal, suspicious, or abnormal. As the cytotechnologist flags pieces of information, or as the preprocessor generates pertinent information about the specimen at issue as will be described below, the information may be automatically appended to the electronic database record associated with the specimen, for convenient review by the pathologist.
Additionally, the workstation preferably includes a bar code reader and mechanism for scanning a bar code or other indicia, to provide a cytotechnologist with the preview data associated with a particular specimen. For this purpose, in the event the preview workstation is incorporated as part of the screening station such as the AcCELt,°
station discussed above, the bar code reader 221 of the AcCEt,L° screening station may serve to initiate a preview of data pertinent to the specimen under analysis by reading a bar code affixed to the specimen slide.
In use, the preprocessor may obtain a portion of its preview information from external sources such as external insurance company, hospital, or physician or laboratory databases or direct data entry, and the preprocessor may generate other preview information based on its direct analysis of the specimen at issue. Regardless of its origin, some or all of this information may be displayed for viewing by the cytotechnologist during the preview stage, in order to bias the cytotechnologist's attention toward more diagnostically significant aspects of the specimen. Therefore, all of this information may be referred to as "biasing information."
As examples, and without limitation, the biasing-information provided to the cytotechnologist by the invention may fall into categories such as (i) patient demographics information, (ii) patient history information such as current or previous test results, (iii) the slide at issue (such as specimen adequacy information (e.g., cellularity and staining adequacy)), and (iv) images of the specimen, each of which will be described in more detail below. This information may be selectively displayed in a single window on the preview display or may, alternatively, be displayed in multiple windows for consideration in combination by the cytotechnologist, as depicted, for instance, in Figures 6a and 6b. Further, this information may take any of a variety of formats, including, for instance, narrative descriptions, tables, charts, plots, digitized (electronic) images, and microscope fields-of view, as weal as enhanced images, annotated images, and comparison displays of combinations of various data types.
S The preprocessor may compile some of the a priori information about the specimen as it searches for and identifies cellular objects of interest in the sample. As described above, for instance, the preprocessor preferably rank orders the cellular objects in the sample according to their probabilities of cellular atypic, and the preprocessor stores images of the identified cellular objects.
In addition, the preprocessor may also isolate each image of an atypical or suspicious cell or field-of view apart from any background images, to facilitate preview display of the diagnostically significant images. The preprocessor may, for example, identify the location of a cellular object or field-of view in a given digital image and, through automated image processing, eliminate the background around the area of interest and enhance the edges of the object or field-of view image. The preprocessor may then combine these electronically isolated images together into a visual mosaic image, thereby forming an artificial or virtual specimen or composite field-of view that consists of cellular objects from the sample without background images or "noise." This synthesized image simulating a liquid-based preparation may be referred to as an "Electronic Monolayer Preparation" (EMP).
Alternatively, the preprocessor may shade the background area, in order to usefully retain a visual context of the cellular object or field-of view while highlighting the area specifically of interest. This process may require one or more passes through the stored digital images of the specimen.
As another example, the preprocessor may use pertinent a priori information about the spatial distribution of certain types of samples when identifying and ranking the probabilities of atypic of cellular objects in those samples. For example, if the sample is a liquid-based preparation such as a ThinPrep° slide (e.g., as shown in Figure 9), the geometry of slide will contain areas of high cellularlity (such as the cellular disk), areas of medium cellularity (such as the bleed zone), areas of low cellularity (such as the annular ring), and areas with ultra-low cellularity (such as areas outside the boundary imprint). It is thus known, for instance, that objects in the annular ring of such a preparation are typically more likely to be degenerate (e.g., artifacts) than objects in the cellular disk. As a result, if the preprocessor finds objects with substantially the same probabilities of cellular atypic in the cellular disk and in the annular ring, the preprocessor may fairly conclude that the object in the cellular disk is actually more likely to be abnormal.
In addition, the preprocessor may be configured to automatically flag certain other pieces of information as likely to be pertinent to the cytotechnologist's analysis. For example, the preprocessor may be configured to identify specimens that may be unsatisfactory for one reason or another, such as due to questionable collection, fixation or staining. Rather than rejecting such specimens outright, the preprocessor may set a flag indicating that collection may have been unsatisfactory. As another example, the preprocessor may be configured to specifically identify clusters of cells in the specimen and to flag such areas of the specimen as likely to be pertinent.
Once the preprocessor has completed its initial processing of the specimen images and data pertinent to the specimen under analysis, the preview workstation displays biasing-information for viewing by the cytotechnologist. As indicated above, one category of such information may be images of the specimen. In this regard, the preprocessor preferably 1 S displays at the preview workstation a discrete set of the apparently "most atypical," "most suspect," or "most complex" regions (cellular objects or fields-of view) for consideration by the cytotechnologist. For instance, these may be the stored images of cellular objects that the preprocessor ranked with highest probabilities of atypia.
As depicted in Figures 6 and 7, for example, the preprocessor may display a grid or, alternatively, a visual mosaic (EMP) of a predetermined number of the most suspect objects or fields. These images may depict either the individual cells or fields-of view with background images removed or shaded as discussed above. Additionally, the preprocessor may highlight some of the images in this display (for instance, with color, texture or shading) based on the preprocessor's automated findings, so as to focus the cytotechnologist's attention on particular matters. Similarly, in a preferred embodiment, the preprocessor may display in conjunction with various discrete images, or in relation to the specimen as a whole, a graphical scale, text or other indicia indicating the preprocessor's degree of confidence in its findings that particular aspects of the specimen are likely to be of interest.
During the preview stage, in the event the cytotechnologist identifies any of these discrete objects as suspicious, the cytotechnologist may flag the object, for instance, by clicking a mouse pointer on the discrete image. The cytotechnologist may further choose to move the specimen physically under manual or computer control to visually review the flagged objects or regions-of interest. In a preferred embodiment, the cytotechnologist may also request the preprocessor to specifically characterize a particular cell.
Additionally, the cytotechnologist may manually or automatically input into the specimen record notes or associated information keyed to the discrete cell image, for later review by the diagnosing expert.
As an additional convenience, as the cytotechnologist is examining these discrete cell or field-of view images, the cytotechnologist may click on or otherwise select any of these images in order to see an actual microscopic field-of view or a magnified digital image of the specimen area that includes the cell, as illustrated for instance by Figure 6.
To provide this function, the preprocessor may communicate with or serve as the specimen-mapping system (such as the TRACCELL~ system) to direct the screening station (such as the AcCEr.L~
screening station) to display the associated field-of view. Alternatively or additionally, this actual microscopic field-of view or selected regions-of interest from that field-of view may be displayed directly on the same monitor that serves as the preview display.
In this way, the cytotechnologist may quickly view the actual context of any cell that the cytotechnologist 1 S sees as possibly suspect.
In addition, as suggested above, the preprocessor displays for preview by the cytotechnologist a series of other pertinent biasing-information, obtained by the preprocessor from external data sources and/or based on its own automated analysis. This information may be displayed on the same or a different display than the discrete images of the cells or fields-of view. In the preferred embodiment, however, this information is displayed on the same display as the discrete specimen images, so that the cytotechnologist can consider the other information in the context of the specimen images.
One area of biasing information may relate to the patient from whom the specimen was drawn and may include, for example, epidimiologic risk factors and abnormal prior physical examinations or laboratory test results. Employing the preview system of the present invention in the context of a lung cancer test (i.e., sputum screening), rather than a cervical Pap smear test, for example, the preprocessor may usefully display an indication of the number of packs of cigarettes per year that the patient has smoked. If the patient has smoked more than a designated number of pack-years, for instance, the technician may wish to flag this information, as the information may bear significantly on whether or not the specimen is from a patient at high risk to develop lung cancer. As another example, in the context of a Pap smear test, information about abnormal prior test results may include the results of cellular DNA tests previously conducted on patient specimens. Still additionally, patient information may include, for example, other patient medical records, family medical history, and patient demographics. For instance, this information may include specific patient risk factors for particular diseases based on family history data.
Another area of biasing-information may relate to the results of other tests that have been conducted on the same specimen being analyzed, for instance, from samples derived from aliquots of the same specimen, possibly included on a discrete area of the same slide.
As an example, if the specimen has undergone an HPV or a cellular DNA ploidy test b~ the cytology laboratory conducting the Pap test screening, the results of this test may be usefully displayed at the preview workstation for convenient examination by the cytotechnologist.
Figure 6b shows an example of a computer display of a discrete set of nuclear images 'that appear to be of most interest in a given specimen, together with a DNA
histogram and scatterplot display for the same specimen.
Still a further area of biasing-information may relate to the slide at issue.
This information may include information flagged or noted by the preprocessor based on its analysis of the specimen images. For instance, the preprocessor may include on the preview display an indication that a given area of the sample contains a cell cluster and is therefore more likely to be of interest. Information related to the slide at issue may also concern how the slide has been handled or mishandled in the cytology screening laboratory or whether the specimen is satisfactory in accordance with standards such as the Bethesda Classification Code for gynecological specimens. In this regard, pertinent information may concern specimen collection, fixation and/or staining.
With regard to specimen collection, for example, the specimen that was drawn from the patient may contain an inadequate vaginal, cervical or endocervical compoxxent.
Alternatively, the sample may contain an insufficient number of cells and therefore be viewed as unsatisfactory. To make these determinations, the preprocessor may automatically analyze the stored digital images of the sample to determine whether the sample lacks cells that would be expected to be present in complete samples.
With regard to specimen fixation, those of ordinary skill in the art appreciate that a specimen taken for a Pap smear test must typically be dipped into or sprayed with alcmhol immediately after being drawn, in order to preserve the specimen. If the specimen is not properly dipped into or sprayed with alcohol, air drying may rupture the nuclear envelope of the cells or alter the chromatin structure and distribution, creating a blurry effect on the Pap smear slide and decreasing the diagnostic value of the sample. The preprocessor system hiay be arranged to analyze automatically the stored digital images of the sample to identify the presence of air-drying artifacts, which would reflect poor fixation techniques.
With regard to staining, the preprocessor may employ automated digital image analysis techniques to determine that the sample was overstained or understained in that it was subjected to too much or too little hematoxylin, for example.
Alternatively, the system may determine that the sample was understained in that it was not subjected to enough hematoxylin. In either case, the preprocessor may display for examination by the cytotechnologist information identifying the adequacy of staining. The operator may flag such information and thus determine that the specimen at issue should not undergo an accelerated screening process.
By displaying information about unsatisfactory collection, fixation or staining, the cytotechnologist may conveniently identify and note poorly prepared samples and may flag the significant information for pathologist review. Additionally, in the event the cytotechnologist determines, based on this information, that the sample at issue is unsatisfactory for further analysis, he or she may either tag the sample to be returned without further analysis or immediately forward the sample to the expert pathologist for diagnosis.
Still further, as the cytotechnologist is examining the biasing-information displayed at the preview workstation, and particularly as he or she is previewing the grid of discrete cell or field-of view images, the preprocessor preferably provides the cytotechnologist with access to a referential database to help analyze and place the specimen in context (e.g, with use of AccuMed International, Inc.'s RELATIONAL CYTOPATHOLOGY REFERENCE
GUIDE~M
software). The preprocessor may include or may be locally or remotely interconnected to a database containing information about other specimens. This database may associate particular cellular characteristics with certain circumstantial information similar to the information provided to the cytotechnologist for preview. As the cytotechnologist notes a discrete specimen image of interest, the cytotechnologist may query the relational database for information about other similar cells, or the preprocessor may be arranged to automatically display pertinent information from the database. In doing so, the preprocessor may conveniently form a search filter based on the information currently flagged by the cytotechnologist. The preprocessor may thereby efficiently obtain database information about similar cells with similar background information.
During the preview process, the preprocessor and human technician interact and learn from each other, each gaining additional information that may aid in the cytotechnologist's subsequent screening of the specimen and perhaps ultimately in a pathologist's diagnosis.
Principally, the cytotechnologist benefits from viewing the biasing-information displayed by the preprocessor, because this information enables the cytotechnologist to focus attention on diagnostically significant aspects of the specimen. As a result, if the cytotechnologist has not detected or flagged anything suspicious or noteworthy about the specimen after examining the information provided by the preview preprocessor, then the cytotechnologist does not need to spend a significant amount of time looking at the slide. The cytotechnologist may instead assume that the specimen is probably one of the 90% to 95% that are normal, and the cytotechnologist may more rapidly screen the entire slide for any cellular abnormalities.
Alternatively, if the cytotechnologist detects some possible abnormalities during this prescreening process or has flagged information that may suggest the presence of abnormalities, then the cytotechnologist may properly spend more than an average amount of time screening this case with a greater than average probability of being abnormal. In this way, the present invention beneficially channels the cytotechnologist's attention during actual screening on specimens that are most likely to be suspicious or abnormal. On the other hand, the invention enables the cytotechnologist to avoid spending unnecessary excess time screening specimens that are likely to be within normal limits.
In addition, the preprocessor may learn significant information about the specimen at issue from actions or behavior of the cytotechnologist, and the preprocessor may use this information -- in addition to other information that it gleans from the specimen and/or from external data -- to prepare for efficient screening by the cytotechnologist.
At one level, for instance, this information may be as simple as the fact that the cytotechnologist requested an exploded view of a specific specimen region or requested referential database information in comparison to a specific specimen region. Knowing that the cytotechnologist took such action regarding the specific specimen region may signal to the preprocessor that the region is of significance to the cytotechnologist. This may cause the computer to flag an area for expert cytopathologist review even if the screening cytotechnologist did not mark this specific region-of interest.
At another level, the preprocessor may acquire information about potentially significant areas of the specimen at issue by monitoring the behavioral patterns of the cytotechnologist during the preview process. In this regard, it has been determined that some of the reactions of the cytotechnologist, even if subconscious, may convey information about signif cant aspects of the specimen at issue. These cytotechnologist reactions may include, WO 99104244 PC'TNS98/14719 for instance, the movement patterns of the cytotechnologist's eyes viewing the preview screen, the amount of time that the cytotechnologist's eyes focussed on particular pieces of biasing-information, and the cytotechnologist's pupil dilation. As an example, if the cytotechnologist's eyes suddenly move to or focus on a particular image of a specimen region, the region of new focus may be a diagnostically significant area of the specimen.
In a preferred embodiment of the present invention, based on the information that the preprocessor gleans from its automated preprocessing as well as during the preview by the cytotechnologist, the preprocessor next generates a routing function to facilitate automated microscopic display at the screening station of the fields-of view that contain the "suspicious"
or "abnormal" cellular objects (those fields-of view or regions being considered to have the probabilities of atypic of the cellular objects that they contain). As described above, such a routing function, or routing pattern, is keyed to the spatial coordinates on the specimen slide that were recorded during prescreening. In the preferred embodiment, the preprocessor may base the routing pattern on any of a variety of criteria. Such criteria may include, for example, the descending order of atypic or complexity previously established, the regions that the cytotechnologist flagged as being of interest during preview, and/or the regions that the preprocessor determined to be suspicious such as those regions that contain cellular fragments or that are overstained or understained.
Additionally, the preprocessor may configure the routing pattern for most efficient physical display at the screening station. As those of ordinary skill in the art will appreciate, microscopically displaying regions of a specimen in an order based on level of interest (probability of atypic) could result in inefficient movement around the slide from one region to another. To avoid this result, the preprocessor may configure the routing function to group specimen regions first by level of interest and then by location on the slide.
Alternatively, the system could display the most atypical cell images as shown in Figure 7, using an EMP or alternative display approach.
For instance, assuming the preprocessor has selected 100 regions that appear to be most likely to be "most atypical," "most suspect," or "most complex," the preprocessor may order the top 25 of those regions by location, the next 25 by location, and so forth. In this way, the screening station will require less movement about the slide to enable screening of the portion of regions designated by the preprocessor.
As another example, if the sample is a liquid-based preparation such as a ThinPrep~
slide, the preprocessor can use a priori information about the spatial distribution of the m. . a.a .. wa_. vw m... vv ...v u- uu . ~, . i.._. . -s t~~:.7 O~
.Ga3:.lJ9~'bb:.~ ~ iF 1 1 ~AUi-ZD-99 15:05 Fr011- CA 02297119 2000-01-14 T-8l1 P.il/14 F-Z64 sample to find the most atypical or suspicious cells ou the slide, such as those in the fields-of view illustrated by Figures 1 l and 12, and present those areas to the observer, first. Them the present invention can guide the observer through the remaining fields-of view with the other (less suspicious) cellular material. 'his suadficd approach enables the screener to review the cellular material on a liquid-based preparation in a sequence such that the most abnormal.
cells are flcely to be encountered earlier in the scr~ning process.
Provided with the biasing-information from the preview stage, the cytotechnologist next conducts actual screalizeg of "suspicious" or "abnoruaal~ cellular objects in the sample.
At this stage, the screenizig station preferably displays microscopic fields-of news of containing the "suspicious" or "abuorrnal" objects, according to the routing function dcvelopai by the preprocessor and at a most e~ciGnt rats. However, ultimate control over the rnarincr of display, such as speed and mute, reuiains in the hands of the cytotechnalagist, for instance through a control panel, keyboard or other input device provided a: the screening station. In addition to display of these fields-of view through a microscope, the invention tray also extend to display these regions-of interest for review on a computer monitor.
la a preferred embodiment, for instance, the routing function may call for the screening station to display microscopically regions of the sptxixrien in descending order of probability that the regions are of interest. Recogttiziag that probabilities of iatcrest arc 111Ce1y t0 dlVCt$i tnOTC betwCen IhC SpCCItnCI1 reg10I1S that 8rC IIlOrC
"aCyplcal," "SUSpGCt" Or "complex" than those that ate ranked Tower by the preprocessor, the screening station may be prrset to route mart slowly through regions with higher probabilities and quicker through other regions. The increase in speed as less interesting regions are displayed far screening may be continuous throughout the screening process. Alternatively, the screening station may, for instance, dispiny a fast group of speciraea regions at one rate and then another group of regions at another rate. Natwitbstanding ttte automated display in accordance with tbc routing fraction, however, the cytotachnologist may at any time elect to stop or manually alter the screening process and custom-adjust the screcaing times slid rates at will.
Similarly, recognizing that the cytotechnologist is likely to be more izitercsted in those specimen regions ranked with higher levels of suspicion by the preprocessor, the screening station may be set to display far a minimum period of lime regions of the specimen that have been deterrnincd to have at a predctcrmined interest level. For instance, the screening station tnacy be set to display for at least 3 seconds each of the top 10 "most atypical," "most suSpCCt," or "must complex" regions. The xreening station may then display other regions wrapttilNlG'OR ..tC. j. ' you souse w~" onnrt . ~~ y ;~eit ~~~ '~ ~ ~ _ _ .
GwwwOD raWOW eWm . .. .
TE~r~E Ate 9ra~

relatively more quickly, again always allowing the cytotechnoiogist to interrupt the automated screening process and proceed manually. The system can literally modulate the time allotted per each field-of view based upon its probability of containing abnormal cells, the complexity of the field-of view, its cellularity, the staining adequacy and many other parameters.
The cytotechnologist may also, or alternatively, set the screening station to route through areas of the specimen in a desired order or at a desired rate. For instance, the cytotechnologist may set the screening station to highlight or stop the screening process at each field-of view that contains one of the "most atypical," "most suspect,"
or "most complex" cells, objects, or fields-of view that was displayed during the preview stage. In this way, the cytotechnologist may automatically see these specimen regions in context. As another example, the cytotechnologist may conveniently set the screening workstation to stop automatically at every field-of view containing one or more of the suspicious cells or at each complex or otherwise flagged field-of view. Still additionally, the cytotechnologist may set the screening station to stop at only those "most suspicious" cells that the cytotechnologist flagged during the preview stage or to stop at only those fields-of view that meet specified criteria, such as those containing fragments or those that were understained or overstained.
The system can be implemented so that individual laboratories or users can set the operating characteristics individually or laboratory-wide for customization. Of course, as those of ordinary skill in the art will appreciate, the screening station may alternatively be prearranged to automatically route through one or more of these fields-of view with or without direct input from the cytotechnologist.
As still another variation, the screening station may be preset or configured by input from the cytotechnologist with a maximum total screening time. This is a significant requirement if this system is being used to re-screen all or many of the apparently WNL
specimens for quality assurance. Provided with the routing function developed by the preprocessor, for instance, the screening station can then display specimen regions at a rate designed to not exceed the preset maximum time. Again, however, the cytotechnologist is preferably provided with the ability to interrupt such automated screening at any time, or to proceed with automated screening without the maximum time constraint.
The present invention further contemplates that information gleaned during the preview stage may be presented to the cytotechnologist during the screening stage, in conjunction with specimen images being screened. For instance, the screening station rnay display pertinent information about a given field-of view on a monitor as text or graphics next to the actual field-of view or overlapped over the field of view.
Additionally, the screening station may display or present indicia such as a sliding bar scale, indicating of the preprocessor's degree of confidence in ranking of probability of interest, similar to that described above in the context of the preview stage.
Based on the cytotechnologist's screening of the "suspicious" or "abnormal"
specimen at block 56, the cytotechnologist determines that the specimen is either (i) suspicious or abnormal, as shown at block 58, or (ii) WNL, as shown at block 62. All of such specimens that the cytotechnologist deems to be suspicious or abnonmai are then forwarded to an expert pathologist for review and final diagnosis. In contrast, as shown by path "D"
in Figure 5, 100% of such specimens that the cytotechnologist deems to be WNL are preferably re-screened by the same or another cytotechnologist to ensure quality control, particularly since the preprocessor had found some reason to designate the specimens as "suspicious" or "abnormal." After this re-screening, if the cytotechnologist still deems such a specimen to be WNL, then the specimen is classified and reported as WNL.
Preferred embodiments of the present invention have been illustrated and described.
It will be understood, however, that changes and modifications may be made to the invention without deviating from the spirit and scope of the invention, as defined by the following claims.

Claims (16)

What we claim is:

1. A process for inspecting a plurality of specimens, said process comprising, for each specimen:
(a) acquiring into a machine a set of digital data representing an image of said specimen, said specimen defining a plurality of objects;
(b) said machine conducting as analysis of said digital data and thereby determining per object whether or not said object is suspicious or abnormal;
(c) based on said analysis, said machine making a determination of whether said specimen does not contain any suspicious or abnormal objects;
(d) is response to a determination that said specimen does not contain any suspicious or abnormal objects, shipping the following steps, whereby said specimen may be excluded from further processing; and (e) in response to a determination that said specimen contains any suspicious or abnormal objects, performing the following steps with respect to only those objects determined by said machine to be suspicious or abnormal:
(i) said machine presenting to an observer a set of biasing-information including at least objects that said machine identifies as most suspicious or abnormal, whereby subsequent screening of sand specimen by said observer may be biased by said observer's preview of said biasing-information;
(ii) said machine generating a muting function defining a sequence of regions of said specimen containing suspicious or abnormal objects to be presented to said observer for screening, said routing function defining for each of at least a plurality of said regions a timing parameter indicative of a duration of presentation of said region, wherein said timing parameter may vary among said regions based at least in part on levels of interest established by said machine for matter within said regions; and (iii) thereafter automatically presenting said regions to said observer according to said routing function.

2. A process as claimed in claim 1, further comprising:
said machine receiving inpur from said observer concerning said biasing information;
and said machine basing said routing function at least in part on said input.

3. A process as claimed is claim 2, wherein receiving input from said observer comprises monitoring a behavioral pattern of the observer as the observer observes said biasing-information.

4. A process as claimed in claim 3, wherein said behavioral pattern is selected from the group consisting of movement of the observer's eyes, focus of the observer's eyes on a particular piece of biasing information, and dilation of the observer's eyes, whereby said behavioral pattern may indicate an object of interest to the observer.

5. A process as claimed in claim 3, wherein said behavioral pattern comprises said observer requesting additional information about an object, whereby a request for additional information about an object may signal to said machine that said object is of interest to the observer.

6. A process as claimed in claim 5, wherein said additional information comprises an exploded view of said object.

7. A process as claimed in claim 5, wherein said additional information comprises referential database information for comparison to said object.

8. A process as claimed in claim 1, wherein said specimens comprise cytological specimens, and said objects comprise cellular objects.

9. A process as claimed is claim 1, wherein said biasing-information further includes information selected from the group consisting of patient-specific information, related-test information, slide-handling information, and specimen-adequacy information.

10. A process as claimed in claim 1, wherein said biasing-information further includes historical specimen data provided by a referential database.

11. A process as claimed in claim 1, wherein said specimen comprises a cytological specimen and said matter comprises cellular matter; and wherein, presenting to an observer a set of biasing-information including objects that said machine identifies as most suspicious or abnormal comprises displaying a mosaic of a plurality of cellular objects isolated from background images.

12. A process as claimed in claim 1, wherein automatically presenting said regions to said observer according to said routing function comprises said machine controlling an automated microscope stage and sequentially bringing said regions into view through a microscope lens assembly.

13. A process as claimed in claim 1, wherein said specimen comprises a Pap sample collected from a patient.

14. A process as claimed in claim 13, wherein said Pap sample is selected from the group consisting of a Pap smear and a liquid based preparation.

15. A process as claimed is claim 1, further comprising said machine automatically classifying as within-normal-limits a subset of said specimens determined to not contain any suspicious or abnormal objects.

16. A process as claimed in claim 1, further including permitting said observer to modify said sequence and durations as said regions are automatically presented to said observer according to said routing function.