JP2003504627A - Automatic detection of objects in biological samples - Google Patents

Abstract

(57) [Summary] A method, a system (FIG. 2), and an apparatus for use in an automatic light microscope (FIGS. 2, 31) for detecting a protein associated with a cell proliferation disorder are provided.

Description

Detailed Description of the Invention

[0001]

[Priority claim]

This application is US provisional patent application No. 60 / 143,823 filed on July 13, 1999.
No. 35 USC § 119 (e) (1), claiming priority benefit, and the present application is US Provisional Patent Application No. 60/0, filed Nov. 30, 1995.
It is a continuation application of U.S. patent application Ser. No. 08 / 758,436 filed Nov. 27, 1996, claiming the benefits of No. 26,805.

[0002]

TECHNICAL FIELD OF THE INVENTION

The present invention relates generally to light microscopy, and more particularly to automated light microscopy and devices for detecting proteins associated with cell proliferative disorders.

[0003]

[Prior art]

In the field of medical diagnostics, including oncology, the detection, identification, quantification and characterization of cells of interest, eg cancer cells, by testing biological samples is an important aspect of diagnosis. Generally, biological samples (eg bone marrow, lymph nodes, peripheral blood,
Cerebrospinal fluid, urine, exudates, fine needle aspirates, peripheral blood exfoliates or other materials)
It is prepared by staining the sample to fix the cells of interest. One method of preparing a cell sample is to react the sample with a specific probe that is a monoclonal antibody, polyclonal antibody or nucleic acid reactive with components of the cells of interest, eg, tumor cells. The reaction is detected by utilizing an enzymatic reaction that uses an enzyme such as alkaline phosphatase or glucose oxidase or peroxidase to convert a colorless soluble substrate into a colored insoluble precipitate, or by directly conjugating a dye to the probe. be able to. Conventional testing of biological samples has been performed manually by laboratory technicians or pathologists. In the conventional manual method, a slide glass prepared by mounting a biological sample is observed with a low-magnification microscope to visually confirm the position of a candidate cell of interest. Next, the region of the glass slide where the position of the target cell is known is observed at high magnification, and those objects are confirmed as the target cell, for example, a tumor or cancer cell. This manual method is time consuming and prone to error, including losing sight of the area of the glass slide. Automated cell analysis systems have been developed to improve the speed and accuracy of the test method. One known interactive system includes a single high power microscope objective for scanning a rack of glass slides, a portion of which is pre-identified for operator testing. ing. In that system, the operator first scans each glass slide at a low magnification similar to the manual method and then records the points of interest on the glass slide for analysis. The operator then stores the recorded location address and associated function in a data file. Once the operator has identified and stored the location of the point of interest, the glass slide is then loaded into an automated analyzer which captures an image of the marked point on the glass slide and captures the image. Perform an analysis.

Many types of cellular proteins are involved in cell proliferation and cell signaling.
Many of these proteins are important for normal cell growth. For example,
HER2 (neu) is a growth factor receptor, but when found in tumor cells,
It becomes a tumor that grows invasively. Studies have confirmed that overexpression of HER2 significantly reduced patient asymptomatic or overall survival. It is desirable to perform an immunohistochemical (IHC) assessment of HER2 / neu before the oncologist directs anti-HER2 / neu therapy. Since the utility value of treatment is high, HE
There is an increasing demand for standard methods of assessing R2 / neu expression.

Three general methods are currently available for detecting HER2 / neu. Namely, genetic detection, protein expression, and protein activity. The in situ hybridization method is HER2 /
It is commonly used to detect the neu gene. Immunohistochemical methods have been utilized to assess the expression of the HER2 / neu protein.

[0006]

A problem with the above automated systems is that the operator must continue to enter in order to first find a cell object for analysis. This continued reliance on manual input results in errors, including missing cells of interest. Such an error is particularly significant in so-called rare event assays, for example when finding one tumor cell in a cell population of one million normal cells. Furthermore, manual methods are extremely time consuming and require a high degree of training to identify and / or quantify cells. This applies not only to the detection of tumor cells, but also to other applications in a range that covers neutrophil alkaline phosphatase assays, reticulocyte counting and thermogenesis assessment. The associated manual labor, in addition to the errors that can result from lengthy simple and tedious manual inspections, adds high expense to such a scheme. Therefore, there is a need for an improved automated cell analysis system that can quickly and accurately scan large quantities of biological material on glass slides. Accordingly, the present invention provides a method and apparatus for automatically analyzing cells that eliminates the need for operator input to locate the cellular object to be analyzed.

[0007]

[Means for Solving the Problems]

According to the present invention, the slides prepared with biological sample and reagents are placed in a slide glass carrier, which preferably holds four slides. The slide glass carrier is loaded into the input hopper of the automatic system. The operator then
Enter data confirming the size, shape and position of the scanning area on each glass slide, or, preferably, the system automatically scans the scanning area on each glass slide while processing the glass slide. Find out. When the system is activated, the glass slide carrier is placed on the XY stage of the optical system. next,
The barcode used to identify the glass slide is read and stored for each glass slide in the carrier. The entire glass slide is rapidly scanned, typically at a low magnification of 10X. A low-magnification image of each scanning position is obtained,
It is then processed to detect candidate objects of interest. It is preferred to utilize color, size and shape to immobilize the object of interest. The position of each candidate object of interest is stored.

When the low level scanning of each slide glass in the carrier on the stage is completed,
The optics are adjusted to a high magnification, such as 40X or 60X, and then an XY stage is placed at the stored position for the candidate object of interest on each glass slide in the carrier. A high magnification image is acquired for each candidate object of interest and then a series of image processing steps are performed to confirm the results of the analysis performed at the low magnification. A high magnification image is stored for each identified object of interest.

Further, background staining is confirmed using control glass slides, including positive and negative controls. For example, the delta for these controls
In order to confirm, test the positive control slide glass for a specific staining method,
Then, a negative control test is performed. The next scan on the target object may identify the target object based on its color or intensity above the delta.

These images are then available to a pathologist or cell technician to search for closer examination to make the final diagnostic evaluation. When the position of each target object is stored, a mosaic of candidate target objects is generated and stored on the slide glass. The pathologist or cytotechnologist may view the mosaic or directly at the glass slide at the location of the object of interest in the mosaic for further evaluation. The mosaic can be stored on magnetic media for future reference, or sent to a remote location for review and / or storage. The entire process required to inspect a single glass slide can take anywhere from 2 to 15 minutes, depending on the size of the scan area and the number of candidate objects to be detected.

The present invention is useful in the field of oncology for the early detection of minimal residual tumor (“micrometastases”). Other useful applications include prenatal diagnosis in the field of fetal cells and infections in maternal blood to confirm pathogen and viral load, alkaline phosphatase evaluation, reticulocyte counting. The processing of the image obtained by the automatic scanning method of the present invention preferably includes the following steps. That is, transforming the image into a different color space; filtering the transformed image with a low pass filter;
Dynamically thresholding the pixels of the filtered image to suppress background material; perform morphological functions to remove artifacts from the thresholded image; The generated image to confirm that there is more than one region of connected pixels having the same color; then classify any region with a size greater than the minimum size as a candidate object of interest. The steps to do are included.

According to another aspect of the invention, the scanning area is scanned over a glass slide; obtaining an image of the position of each glass slide; analyzing the information of the texture of each image, Edge is detected; then the position corresponding to the detected edge is stored and automatically determined by defining the scan area. According to yet another aspect of the invention, autofocusing of the optical system is accomplished by first determining a focal plane from an array of points or positions within the scan area. Its induced (de
The rived) focal plane allows the following rapid autofocus for low magnification scanning operations. The focal plane is the proper focal position (focal posi) across the array of positions.
position), and then determined by performing an analysis such as a least squares method of the array of focal positions to obtain a focal plane across the array. Preferably, the focal position of each position is a two-stage position with a fixed number of coarse and precise interactions (co
It is confirmed by an increase in arse and fine interaction). At each interaction, an image is obtained and then an optical parameter, such as the pixel fractional value of the average value of the pixels of the resulting image, is calculated to form a set of fractional value data. The least squares method is performed on the fractional value data according to a known function. The least squares curve peak value is selected as an estimate of the best focus position.

In another aspect of the present invention, another high magnification focus position method is for focusing on a candidate object of interest within a found glass slide during analysis of a low magnification image. Find the area. The region of interest is preferably n columns wide, where n is a power of 2. Pixels in this region are then processed using the Fast Fourier Transform to produce a spectrum of component frequencies and a corresponding complex magnitude for each frequency component. The magnitudes of the frequency components within the range of 25% to 75% of the maximum frequency component are squared and summed to obtain the total magnification of the target area. This method is repeated for other Z positions, and the Z position corresponding to the maximum total magnification for the target area is selected as the best focus position.

According to yet another aspect of the invention, a method and apparatus for automatically manipulating a glass slide is provided. One glass slide is mounted side by side on a glass slide carrier along with several other glass slides. The slide glass carrier is placed in the input feeder along with other slide glass carriers,
This facilitates automatic analysis of a batch of slides. The slide glass carrier is mounted on the XY stage of the optical system, and the slide glass on it is analyzed. The first glass slide carrier is then lowered into the output feeder after automatic image analysis has been performed and the next carrier is automatically loaded.

In a specific embodiment, the present invention provides a disorder of cell proliferation, eg HER in tissue.
An automated system is provided for quantifying proteins associated with 2 / neu expression. This invention is useful for identifying HER2 overexpression in tissues, especially breast tissues.

The above and other features of the invention, including various novel details of construction and combination of parts, will be more particularly described with reference to the accompanying drawings and pointed out in the claims. It will be appreciated that the particular device embodying the invention is shown by way of illustration only and not as a limitation of the invention. The principles and features of this invention may be employed in numerous and various embodiments without departing from the scope of the invention.

[0017]

DETAILED DESCRIPTION OF THE INVENTION Overview The present invention provides an automated tissue image analysis method for detecting cell proliferative disorders, such as those associated with HER2 / neu. Also provided are other automated histological image analysis methods, such as immunohistochemical analysis of estrogen and progesterone receptors. Other methods described below are also within the scope of the invention. For example, MM / MRD analysis of tissues, MIB-I, analysis of microvessel density, oncogene p53,
Immunophenotyping, hematoxylin / eosin (
H / E) morphological analysis, undifferentiated tumor classification, antibody titration, nucleolar bulge,
Diagnostic methods utilizing HIV p24, human papilloma virus (HPV: for cervical biopsy), and mitotic index, which generally utilize staining methods, are within the scope of the invention. The present invention is useful in the field of oncology for early detection of minimal residual tumor (“micrometastases”).

When using for HER2 / neu, Automated Cellular Imaging System (A
CIS ™ analyzes at least two sub-samples, H / E conditioned sample glass slides and immunohistochemically conditioned sample glass slides. Sub-samples are morphologically reconstructed, displayed and used to perform the appropriate tests. The measuring device automatically selects the region of interest and uses the transformation of the color space (and morphological filtering if necessary) to select the region of tissue from the slides prepared as immunohistochemistry samples. The expression of HER2 / neu inside is confirmed and quantified. Areas of interest selected from both slides as well as the test results of the quantification analysis will be included in the patient's report.

The following acronyms and terminology are used throughout this application. Work list (work lis
t) is a group of samples adjusted for the same staining method and generally analyzed in the same automated manner. The sample group is composed of cases of one or more patients. A histologically reconstructed image is an image of the entire sample mounted on a glass slide. This image is created by combining two or more fields of view of the objective lens. Unless defined otherwise, technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

Although the same or equivalent methods and materials described herein can be used to practice the invention and to carry out the tests of the invention, suitable methods and materials are described below.

All publications, patent applications, patents, and other references cited herein are hereby incorporated by reference in their entirety. In case of conflict, the present application, including definitions, will control.
Furthermore, the materials and methods described herein are illustrative only and not limiting. Other features and advantages of the invention will be apparent from the following detailed description, drawings, and claims.

Preferred Workflow The preferred workflow is outlined below. 1. The sample arrives at the laboratory. These samples are labeled according to standard hospital practices to ensure chain of storage. 2. Preparation of sample 2.1. Two to three tissue pieces per patient case are preferred for this study. 2.2. A piece of tissue to be stained with H / E is attached to a glass slide. H /
The E staining protocol is compatible with HER2 / neu immunohistochemical staining. 2.3. A second piece of tissue to be stained with the anti-HER2 immunohistochemical staining system is mounted on a glass slide. 2.4. If necessary, a third antibody (ie patient negative control) and anti-H
A third piece of tissue to be stained with the ER2 staining system is mounted on a third glass slide. 2.5. In order to avoid the mechanical limitations of ACIS ™, the mounting of the sample
Follow the specifications. The sample should be 30 mm below and 65 mm above the top edge, with the frosted glass end of the slide as the top. The sample may reach both lateral edges of the glass slide. 2.6. If there are other patient samples, repeat section 2. 2.7. Insert two stained control slides. 3. Staining of sample 3.1. All sample slides prepared for H / E are stained according to the H / E staining protocol. 3.2. All slides of immunohistochemistry samples are stained with anti-HER2 staining system. 3.3. If necessary, slides of negative immunohistochemistry samples are stained with a third antibody (ie patient negative control) and anti-HER2 staining system. 3.4. Positive and negative control glass slides are stained with anti-HER2 staining system and appropriate antibodies. 4. Barcode all prepared slides into the open slot of the slide carrier. 5. Manually control the positive and negative control slides to verify controlled staining. Manually inspect the slides of the patient's positive and negative immunohistochemistry samples to verify that the patient's Bagland staining is normal. 6. Input of work list 6.1. Enter the information that matches your worklist. 6.1.1. Specify the worklist identification number. 6.1.2. Specify the protocol / application (HER2 / neu). 6.1.3. Specify barcodes for positive and negative control glass slides. 6.2. Enter information relevant to each patient's case. 6.2.1. Accession number and patient identification number (ID). 6.2.2. Designate 2/3 barcodes of the above case samples (H / E, immunohistochemistry, patient negative control samples). 6.2.3. Patient specific information: age (integer), gender (M / F), race (form-predefined option), diagnostic result (form-256), date and time of diagnosis (
Date format), date and time of last treatment (date format) and description of last treatment (Form-5)
12). 7. If the measuring device has not been calibrated daily, calibrate before starting the worklist (one 5 minute calibration is required per day). This calibration consists of radiometric, geometric and color standard calibrations. 8. The carrier containing the control glass slide and the patient's case are loaded into the input hopper. 9. Start automatic worklist analysis. 9.1. If the staining control does not have a large enough delta Δ, the instrument skips the remaining slides in the worklist. 10. After the analysis is complete, examine the worklist test results and generate an appropriate report online. For each case, the low-magnification H / E image is the patient ID,
It appears automatically by selecting the accession number or sample identification number. 10.1. While examining the low-magnification H / E histology reconstructed image (HR), the user selects: 10.1.1. Select the H / E HR region to zoom on. 10.1.2. Choose to proceed to low magnification immunohistochemistry HR. 10.1.3. Select the area to be saved in the form of the report prepared in advance. 10.2. While examining the high magnification H / E HR, the user selects: 10.2.1. Select to return to low magnification H / E HR. 10.3. While examining low magnification immunohistochemistry HR, the user chooses: 10.3.1. Select the area of immunohistochemistry HR to be zoomed on. 10.3.2. Select a region of low magnification immunohistochemistry HR and then query for the following quantification test results. The above quantification test results are associated with positive staining controls and negative immunohistochemistry glass slide scoring of patients. 10.3.2.1. Overexpression within selected regions is reported as 0, 1+, 2+, or 3+. 10.3.2.2. Percentage of positive cells within the selected area. 10.4. While examining high-magnification immunohistochemistry HR, the user chooses to: 10.4.1. Select areas of high magnification immunohistochemistry HR and then query for the following quantification test results. The quantification test results are associated with positive staining controls and negative immunohistochemical glass slide scoring of patients. 10.4.1.1. Overexpression within selected regions is reported as 0, 1+, 2+ or 3+. 10.4.1.2. Percentage of positive cells within the selected area. 10.4.2. Choose to return to low magnification immunohistochemistry HR. 10.5. Are there other samples to look at? 10.5.1. If there are other samples, repeat section 8. 10.6. If the tissue analysis has not been scanned, reload the glass slide, open the application, and adjust the scan area to the location of the tissue, if necessary. If the scan area is appropriate but the focus is not the tissue area but the failed focus, then change the focus inset to a value close to 1 in the application Make sure you are centered on the area of your organization. Run the application. Step 1
0.1. ~ 10.5. repeat. 10.7. Manually record the analysis results. 11. Check of quality assurance in measuring device. 11.1. When the analysis of both positive and negative control slides is complete, the Δ between the scores is calculated. If this Δ is not large enough, the analysis results for all slides in this worklist will therefore be flagged and the measuring device
Warn the user with an audible alarm. 11.2. If the Δ between the patient's positive and negative immunohistochemistry glass slides is not large enough, then the patient is flagged. 11.3. Montage of tissue sections from both positive and negative controls. Five tissue sections from each control are sufficient. 12. The above report will be reviewed online and modified as needed. 13. Send those reports to a printer or to the requesting clinician via the internet. The outputs of the worklist are stained positive control glass slides, stained negative control glass slides, two stained glass slides per patient case (one H / E and one IHC). staining),
And a comprehensive electronic report. This report includes selected image sections,
Includes quantification test results, reference materials and other relevant data.

Staining of Samples A cell sample (“sample”) is divided into two or more sub-samples. The term “sample” includes cellular material derived from a subject. Such samples include, but are not limited to, hair, skin samples, tissue samples, cultured cells, cultured cell media and biofluids. The term "tissue" refers to a mass of connected cells of human or other animal origin (eg, CNS tissue, neural tissue, or ocular tissue) and includes connecting materials and liquid materials associated with the cells. The term "biofluid" means a liquid material of human or other animal origin. Such "biological fluids" include, but are not limited to, blood, plasma,
There are serum, serum derivatives, bile, sputum, saliva, sweat, amniotic fluid, and cerebrospinal fluid (CSF) such as lumbar CSF and ventricular CSF. The term "sample" also includes medium containing separated cells. The amount of sample required to achieve a reaction can be determined by one of ordinary skill in the art by standard laboratory methods. The optimum amount of sample can be determined by the serial dilution method. The HER2 / neu method is an anti-HER2 / neu staining system such as DA
It is performed using a commercially available kit such as the kit provided by KO (Carpentaria, Calif., USA). The steps of the immunohistochemistry protocol are as follows. That is, (1) a washing buffer solution is prepared. (2) Deparaffinize the sample and then rehydrate. (3) Retrieve the epitope. Incubate in water at 95 ° C for 40 minutes. Cool the slides at room temperature for 20 minutes. (4) Add a peroxidase blocker. Incubate for 5 minutes. (5) Add primary antibody or negative control reagent. Incubate at room temperature for 30 ± 1 minutes. Rinse with wash solution. Place in wash solution bath. (6) Add peroxidase standardized polymer. Incubate at room temperature for 30 ± 1 minutes. Rinse in wash solution. Place in wash solution bath. (7) Prepare a DAB substrate chromagen solution. (8) Add the substrate chromagen solution (DAB). Incubate for 5-10 minutes. Rinse with distilled water.
(9) Counterstain. (10) Place the cover glass. The glass slide contains a cover-slip medium to protect the sample and to make optical corrections consistent with the requirements of the microscope objective. The cover glass covers the entire prepared sample. By placing the cover glass, air bubbles that would smear the stained sample do not enter. This cover glass can carry a DAKO Ultramount medium having a thickness of 1 to 1/2. (11) For each work list
Make a set of stained control slides. The set includes positive and negative controls. The positive control is stained with anti-HER2 antibody and the negative control is stained with another antibody. Both slides are identified by a unique barcode. When the measuring device reads the bar code, it recognizes the glass slide as part of the control set and performs the appropriate application. There are one or two applications for dye controls. (12) A set of measuring device calibration slides includes a slide that is used to calibrate the focus and color balance. (13) Use a dedicated carrier for one-touch calibration. Upon successful completion of this calibration process, the measuring device reports that it has itself been calibrated. Upon successful completion of the creation of standard slides, the user will be asked whether the measuring devices are in standard condition and how reproducible the test results are between and within the measuring devices. can do. Hematoxylin / eosin (H / E) glass slides are conditioned with standard H / E protocol. The standard solution contains: That is, (1) Gills
Hematoxylin (6.0 g of hematoxylin, 4.2 g of aluminum sulfate, 1.4 g of citric acid
, Sodium iodide 0.6g, ethylene glycol 269ml, distilled water 680ml), (2) eosin (eosin yellowish 1.0g, distilled water 100ml), (3) lithium carbonate 1
% (Lithium carbonate 1 g, distilled water 100 g), (4) acid-containing alcohol 1% 70% (alcohol 99 ml, concentrated hydrochloric acid 1 ml), and (5) Scott's tap water (Scott's tap wate
r) is included. To a beaker containing 1 L of distilled water, add 20 g of sodium bicarbonate and 3.5 g of magnesium sulfate. Add a magnetic stirrer and stir well to dissolve the salts. Using a filter funnel, pour the solution into a labeled bottle. The staining procedure is as follows. That is, (1) water is applied to the sample section, (2)
Place the section in hematoxylin for 5 minutes, (3) wash with tap water, (4)
"Perform blue coloring" in lithium carbonate or Scott tap water, (
5) Wash with tap water, (6) Place the section in alcohol containing 1% acid for several seconds,
(7) Wash with tap water, (8) Place the section in eosin for 5 minutes, wash with (9) tap water, then (10) dehydrate and clear. Attach the obtained section. As a result of staining with H / E, in the cells, the nucleus was stained in blue, the cytoplasm was stained in pink of various shades, the muscle fibers were stained in deep red with pink, and the fibrin was dark pink. , And red blood cells are stained orange-red. The present invention also provides a method for automated analysis of estrogen receptor and progesterone receptor. Estrogen and progesterone receptors, like receptors for other steroid hormones, play a role in target cell developmental processes and maintenance of hormone sensitivity. From a molecular point of view, the interaction of estrogen and progesterone receptors with target genes is most important for maintaining normal cell function and is also involved in the regulation of mammalian tumor cell function. There is. Expression of progesterone and estrogen receptors in breast tumors is a useful indicator for subsequent hormone therapy. Anti-estrogen receptor antibodies label breast cancer epithelial cells expressing the estrogen receptor. Immunohistochemical assays for estrogen receptors include anti-estrogen receptor antibodies, such as the well characterized clone and the method of Pertchuk et al. (Cancer 77: 2514-2519 1996) or DAKO (Carpentaria, Calif., USA). ) Provided by DAKO LSAB2
It is performed using a commercially available immunohistochemistry system such as the Immunostaining System. In breast cancer cells, immunohistochemical immunostaining of progesterone receptors has been demonstrated in the nuclei of cells from various histological subtypes. Anti-progesterone receptor antibodies label breast cancer epithelial cells that express the progesterone receptor. Immunohistochemical assays for progesterone receptors include anti-estrogen receptor antibodies such as the well characterized 1A6 clone, and the method of Pertchuk et al.
2514-2519 1996).

Still other automated analytical methods are included within the scope of the present invention, including the following methods. Micrometastasis / metastatic recurrent disease (MM / MRD) metastasis is a biological process by which cancer spreads from its original site to distant parts of the body. Micrometastases are the presence of small numbers of tumor cells, especially in the lymph nodes and bone marrow. Metastatic recurrent disease is similar to micrometastases, but is detected after treatment rather than before cancer treatment. MM / MR
The immunohistochemical assay of D is performed using a monoclonal antibody that reacts with the antigen found on cancers of the bladder, prostate and breast (metastatic-specific mucin).

MIB-1 MIB-1 is an antibody that can be used in immunohistochemical assays for the antigen Ki-67. The clinical stage in the first presentation is related to the proliferative index measured for Ki-67. High Ki-67 index values are clearly associated with metastases, neoplastic death, low asymptomatic survival, and low overall survival.

[0026] The analysis results of the analysis microvascular density of microvascular density is a measure of the formation of new blood vessels (angiogenesis). Angiogenesis is a hallmark of growing tumors. The microvessel density in a tumor can be evaluated by anti-CD34 immunostaining.

[0027] Overexpression of p53 oncogene p53 oncogene, human malignancies there is a change in the most common gene upon development. Research reports on various malignancies, including neoplasms of the breast, colon, ovary, lung, mesenchyme, bladder and bone marrow, have been published in p.
It is suggested that mutation of 53 genes is involved. The highest frequencies of occurrence have been identified in breast, colon and ovarian tumors. A wide variety of normal cells express wild-type p53, but generally in limited amounts. Overexpression and mutations of p53 have not been found in benign tumors or normal tissues. Immunohistochemical assays for p53 are performed using anti-p53 antibodies, such as the well characterized DO-7 clone.

Immunophenotyping Classification of undifferentiated tumors Antibody titering Raised nucleoli The nucleoli are organelles within the cell nucleus. Cervical malformations are pre-malignant abnormalities of normal and metaplastic epithelium (nuclear hyperstaining, nuclear enlargement and irregular contours, increased nuclear / cytoplasmic ratio, increased nucleolar ridge). And replacement with atypical epithelial cells having the cytological feature of chromosomal abnormalities. These changes seen in dysplastic cells are the same type of change, but to a lesser extent than clearly malignant cells. In addition, there are dysplasias of mild, moderate and severe.

HIV p24 Protein Human immunodeficiency virus (HIV) p24 antigen levels are measured by immunohistochemistry using an anti-HIV p24 antigen. HI
Testing for Vp24 protein is used to detect HIV virus. This test can be used to detect HIV, measure the amount of virus and examine the genetic tissue of HIV.

Human Papillomavirus (HPV) HPV is a sexually transmitted virus that causes warts. HPV is an important health concern because it causes cervical cancer. HP
V immunohistochemical assays are performed using antibodies associated with cervical cancer. Several serum reactions, in particular HPV16 epitopes LI: 13, E2: 9 and E
IgG responses to 7: 5 and IgA responses to HPV18E2-derived antigens are strongly associated with cervical cancer.

Incorporation of the mitotic viral genome into nuclear genetic material may promote uncontrolled growth of host cell malignancies.

Operation of the measuring device During operation of the measuring device, the user interacts with several screens. Such screens include, but are not limited to, patient entry, study data, construct report, manual controls and user preferences. The interface for patient data entry, rather than the entry of information about the patient's case,
It is organized as work list information composed of a large number of patient cases. For example, the worklist for HER2 / neu is one of two methods: H
It consists of tissue samples prepared by ER2 / neu immunohistochemistry or H / E methods. The slides in the HER2 / neu worklist are positive control slides (adjusted with immunohistochemical method and HER2 antibody), negative control slides (adjusted with immunohistochemical method and another antibody). ), And two slides of one or more patient cases. Two slides are prepared for each patient case. The first slide glass is HER
The 2 / neu immunohistochemical method is prepared and the second slide is prepared by the H / E method for general histological analysis.

Data related to HER2 / neu worklists include (1) worklist name or protocol identifier (eg, HER2 / neu); positive control slide glass barcode; and negative control slide glass bar. Is the code. The data related to the patient's case are (4) patient ID; (5) trust number; (6) corporate guarantee number; (7) related hospital (referring hospital); (8) related doctor (referrin).
g physician); (9) Immunohistochemically adjusted slide glass barcode; and (10) H / E adjusted slide glass barcode.

The user initiates the analysis with a batch button and then the glass slide carrier is loaded into the input hopper. The input hopper consists of immunohistochemical slides from patient cases, all H / E slides from patient cases, positive control slides, and negative control slides. During the analysis, the measuring device reads the barcode on the glass slide, confirms one of three situations, and then three suitable computer applications: (1)
The slides are positive control slides and run HER2-positive applications; (2) the slides are negative control slides,
Run the HER2-negative application; (3) the slide is an immunohistochemical slide or an H / E patient slide and run the HER2 application;

The system reads industry standard bar codes. The barcode is related to the type of protocol the measuring device uses to analyze the glass slide. The type of data stored for each glass slide is determined by the stained sample and the protocol used to analyze it. Therefore, a series of stores is preserved during the transportation of the glass slides through the system. If the bar code is unreadable or not found in one defined worklist, the action taken by the measuring device is to ignore the glass slide and send it to the output hopper.

The HER2-positive application will (1) find the sample, (2) go to five locations within the sample, (3) store the image from each location, and (4) complete the color space conversion. Then, the staining intensity at all five places is obtained, and then (5) the average value of the staining intensity is stored in the database (DB). This value is used as the normalized expression value for the HER2 quantification analysis. This value is also used to measure the Δ between the positive and negative control slides and to verify that Δ is sufficiently large.

A HER2-negative application can (1) find a sample, (2) go to five locations within the sample, (3) store images at each location, (4) complete color space conversion, and
Obtain the staining intensity for all five locations and then (5) store the average value of the staining intensity in the database. This value is used to calculate the Δ between the positive and negative control slides.
And confirm that Δ is sufficiently large.

The application of HER2 is (1) finding the sample, (2) scanning the glass slide at 10 × magnification to construct a histologically reconstructed object, and then (3) the object. Are stored in the database. Before analyzing the slides,
The measuring device checks to see if the space on the hard drive meets the specific capacity to store the data. If the capacity is met, the user is prompted to save before continuing with the device analysis. Before storing the target data in the database, it is necessary to perform compression.

After all slides have been processed in the worklist, the histologically reconstructed subject is analyzed by a histologist or physician.

The slide glass display interface displays a list of slides analyzed or a list of patient cases. Whenever the tissue is analyzed, for each patient, there will be at least two glass slides that the measurement device will analyze. This set of patient slides is the logical unit. Therefore, a histologist or doctor can select patient cases by patient ID or accession number.

When a pathologist selects a patient case in a user interface that displays a list of patients to be examined / analyzed, the H / E image is always displayed first. The pathologist uses this H / E image to determine if the cancer is invasive or not, and if immunohistochemistry (IHC) glass slide analysis is required.

When selecting a patient case, a low-magnification H / E image is displayed. This image can be scaled down to display the entire cross-section of the tissue within the monitor of the system (at least 1024 x 760 resolution).

The histologist has characteristics that can be used to select more than one patient case to examine and can move from one patient case to the next with the following characteristics. For each patient case, a reduced, low-magnification H / E image always appears first.

While viewing the reduced low-magnification H / E image, the pathologist can do the following: That is, (1) the area of the low-magnification H / E image is zoomed to high magnification; (2)
Choose to proceed to reduced low-magnification immunohistochemistry imaging of current patient cases; (3)
Select areas to capture in the patient's report (mark sections of the image); (4) choose to proceed to the next patient case (if selected in the previous user interface); and (5) choose to proceed to the previous patient case;

While viewing the zoomed H / E image, the pathologist can do the following: That is, (1) return to a reduced low-magnification H / E image; (2) choose to proceed to a reduced low-magnification immunohistochemistry image of the current patient case; (3) select regions to be included in the patient report ( Mark image section); (4) choose to go to the next patient case (if selected in the previous user interface); and (5) choose to go to the previous patient case ;be able to.

A pathologist can perform the following while viewing the reduced low-magnification immunohistochemical image. That is, (1) zoom the region of the low-magnification immunohistochemical image at a high magnification; (2) select to return to the reduced H / E image of the current patient case; (3) select the region of the reduced immunohistochemical image. Select and query quantification test results; (4)
Select the areas to be included in the patient report (mark the image section); (5) Select the quantification test results to be included in the patient report [save the quantification test results] (6) choose to proceed to the next patient case (if selected in the previous user interface); and (7) choose to proceed to the previous patient case (if applicable); You can

A pathologist can perform the following while viewing the zoomed immunohistochemistry (IHC) image. That is, (1) return to the reduced low-magnification IHC image; (2) choose to return to the reduced H / E image of the current patient case; (3) select a region of the zoomed immunohistochemistry image and quantify Inquire about the results of the mobilization test; (4) select the areas to be included in the patient's report (mark the image section);
(5) Select the quantified test results to be included in the patient's report (store the quantified test results); (6) Select to proceed to the next patient case (selected in the previous user interface) (If present); and (7) choose to proceed to the previous patient's case (if applicable);

After the pathologist has completed the analysis and investigation of all patient cases, a patient report can be prepared. The user interface tabulates all patient cases that the pathologist has already examined and analyzed. The pathologist can select one or more tabulated patient cases for which the patient report should be viewed electronically. HER2 / of patient case
The neu report can include the following information: That is, (1) patient information and demographics entered in the worklist information entry; (2) relevant doctors and hospitals entered in the worklist information entry; (3) pathologist and laboratory information; (
4) Sections of the H / E image marked for examination / analysis for inclusion in the report; (5) Immunohistochemistry images marked for examination / analysis for inclusion in the report. Section; (6) Include quantified analysis results stored during research / analysis for inclusion in the report; and (7) HER2 / neu application and scoring analysis reference material. it can.

The following functions are included in the interface for displaying the outline of the patient report. (1) print; (2) send (connect to e-mail if available and prompt user to where to send report); (3) report next patient case Handwriting (if more than one case is selected from the previous interface)
(4) previous patient case report (if applicable); and (5) preservation / application (
Save / apply) (when additional comment area is changed) function is included. Network connectivity is site-specific. Site-specific means sending a report
Desirable to send a report over the Internet without having to send the report to another host. It is preferable to be able to send directly.

The analyzer has at least two user modes. The first user mode is a fixed protocol consistent with FDA regulations. The second user mode can be changed by the user. The second user mode includes access to the following parameters of the application. Its parameters are (
1) Definition of area, center and height of scan; (2) Toggle switch to decide whether or not to use the find phase; (3) Focus type (focu
s type); (4) focus threshold; and (5) focus inset.

Automated System Referring now to the drawings, an automated cell analyzer for biological samples is designated by the reference numeral 10 and is shown in perspective view in FIG. 1 and in block diagram in FIG. The device 10 comprises a microscope subsystem 32 housed within a housing 12. Its housing 1
2 includes a slide glass carrier input hopper 16 and a slide glass carrier output hopper 18. The door 14 of the housing 12 protects the microscope subsystem from the outside environment. A computer 2 in which a computer subsystem has a system processor 23, an image processor 25, and a communication modem 29.
Equipped with 2. The computer subsystem further comprises a computer monitor 26, an image monitor 27 and other external peripheral devices including a storage device 21, a pointing device 30, a keyboard 28 and a color printer 35. An external power supply 24 that powers the system is also shown. The viewing eyepiece 20 of the microscope subsystem projects from the housing 12 and is used by the operator for viewing. The device 10 further comprises a CCD camera 42 used to acquire images through the microscope subsystem 32. A microscope controller 31 under the control of the system processor 23 controls the functions of a number of microscope subsystems which will be described in more detail. The slide glass automatic feed mechanism, together with the XY stage 38,
The slide glass is automatically operated in the apparatus 10. The illumination light source 48 emits light XY
Projection onto a stage 38, whereupon an image is formed through microscope subsystem 32, and CCD camera 42 causes image processor 25
An image is obtained for processing. A Z stage, or focus stage 46, under the control of the microscope controller 31, displaces the microscope subsystem in the Z plane for focus adjustment. The microscope subsystem 32 further comprises an objective lens turret 44 with a motor for selecting the objective lens.

The purpose of the device 10 is to automatically and unmannedly scan a calibrated microscope slide to detect and count candidate objects of interest such as normal cells and abnormal cells such as tumor cells. A preferred embodiment is the rare case where there is only one candidate object of interest per hundreds of thousands of normal cells, eg 1 to 5 candidate objects of interest per 2 cm ² area of glass slide. It is used to detect various events. The device 10 automatically locates and counts the candidate objects of interest,
And the normal cells present in the biological sample are estimated based on the characteristics of color, size and morphology. A number of dyes can be used to preferentially stain a candidate object of interest and normal cells to different colors to distinguish such cells from one another.

As mentioned in the prior art section of the present invention, biological samples can be prepared with reagents to obtain a colored, insoluble precipitate. Using the device of the present invention, this precipitation is detected as a candidate object of interest. During operation of device 10, a pathologist or laboratory technician loads the conditioned slides into a slide carrier. The slide glass carrier 60 is shown in FIG. 8 and is described further below. Each glass slide carrier holds up to four glass slides. Next, up to 25 glass slide carriers are loaded into the input hopper 16. The operator can specify the size, shape and location of the area to be scanned, or the system can automatically locate this area. The operator then enters the system through a graphical user 25 interface,
Command to start automatic scanning of slide glass. The unmanned scanning starts by automatically loading the first carrier and the slide glass on the XY stage 38 with a precision motor. The barcode label attached to the slide glass is read by the barcode reader 33 during the loading operation. Next, each slide glass
Scanned at a low user selected microscope magnification, eg 10 ×, the candidate object of interest is identified based on its color, size and morphological characteristics. The XY positions of the candidate cells are stored until the scan is complete.

After the low magnification scan is completed, the device automatically returns to each candidate object of interest and reimages and refocuses at a higher magnification, eg 40 ×, and then further analyzed. , Confirm that the object is a candidate object. The device stores an image of the object for later examination by a pathologist. All test results and images can be stored on a storage device 21, such as a removable hard drive or DAT tape, or sent to a remote location for review or storage. The stored image for each glass slide can be viewed and examined further during the mosaic of images. Further pathologists or operators can view the detected objects directly by microscope with the eyepiece 20 installed or on an image monitor.

Having described the overall operation of the device 10 from a high level, the device will now be described in further detail. Turning to FIG. 3, the microscope controller 31 is shown in more detail. The microscope controller 31 comprises several subsystems connected by a system bus. The system processor 102 controls these subsystems and is controlled by the device system processor 23 through the RS232 controller 110. The system processor 102 controls an input feeder and a set of motor controller subsystems 114-124 that control the output feeder, the motorized turret 44, the XY stage 38 and the Z stage 46 (FIG. 2). Histogram processor 108 receives input from CCD camera 42 and calculates variance data during the focusing operation as further described herein. The system processor 102 further controls the illumination controller 106 to control the sub-stage illumination source 48. The light output from the halogen bulb that illuminates the system changes over time as the bulb ages and factors such as optical alignment change. Furthermore, "overstained" glass slides can reduce camera exposure to unacceptable levels. An illumination light source controller 106 is provided to compensate for these effects. This controller is used with light control software to compensate for changes in light level. The light control software samples the output from the camera at timed intervals (eg, the time interval between loading glass slide carriers) and causes the light controller to adjust the light level to the desired level. Order. As such, the light controller is automatic and transparent to the user, and requires substantially no additional time to operate the system. The system processor 23 is a system, for example, a dual parallel Intel Pentium (registered trademark) 90 MHz device, a T.M. I
． Any processor capable of executing the operations of the C80 processor and the processor capable of performing 8-32 bits in the NT system may be used. The image processor 25 is preferably a Matrox Imaging Series 640 model.
The system processor 102 of the microscope controller is an Advanced Micro Devices AMD 29K.
It is a device.

4 and 5 show the device 10 in more detail. FIG. 4 shows a plan view of the device 10 with the housing 12 removed. A portion of the glass slide auto-feed mechanism 37 is shown to the left of the microscope subsystem 32 and works with the glass slide carrier output hopper 18 to remove the glass slide carrier that has been analyzed. It includes a load assembly 34 and an unloading platform 36. A vibration isolation mount 40, shown in more detail in FIG. 5, is provided to insulate the microscope subsystem 32 from mechanical shock and vibration that may occur in a typical laboratory environment. In addition to an external source of vibration, the high speed vibrations of the XY stage 38 may induce vibrations in the microscope subsystem 32. Such sources of vibration can be isolated from the electro-optical subsystem to avoid undesired plotting to image quality. The insulating mount 40 comprises a spring 40a and a piston 40b submerged in a high viscosity silicone gel encapsulated in an elastomeric membrane bonded to the casing to achieve a damping coefficient on the order of 17-20%. There is.

The features of automatic operation of the slide glass of the present invention will now be described. The glass slide automation subsystem operates once on a single glass slide carrier. 6A and 6B show a top view and a bottom view of the glass slide carrier 60, respectively. The slide carrier 60 is a slide glass 7 attached with an adhesive tape 62.
Holds up to four 0s. The carrier 60 includes ears 64 for suspending the carrier in the output hopper 18. Undercut 66 and pitch rack 68
Is formed on the top edge of the slide glass carrier 60 for mechanically operating the slide glass carrier. A keyway cutout 65 is formed on one side of the carrier 60 to help align the carriers. The adjusted slide glass 72 attached to the slide glass carrier 60 includes a sample area 72a and a barcode labeling area 72b. FIG. 7A shows a top view of a glass slide manipulation subsystem including a glass slide input module 15, a glass slide output module 17, and an XY stage drive belt 50. 7B is a partial cross-sectional view taken along the line AA of FIG. 7A. The glass slide carrier input module 15 comprises a glass slide input hopper 16, a loading platform 52 and a glass slide carrier loading subassembly 54. The input hopper 16 is arranged in a stack above the loading platform 52, with a series of glass slide carriers 60 (
6A and 6B). A guide key 57 protrudes from the side surface of the input hopper 16, and the guide key 57 has a key groove cutout 65 (FIG. 6A) in the carrier.
It fits in and a proper 15 alignment is achieved. The input module 15 further comprises a rotary indexing cam 56 and a switch 90 mounted on the loading platform 52, the operation of which will be described further below. The carrier loading subassembly 54 comprises a feed drive belt 59 driven by a motor 86. The feed drive belt 59 comprises a pusher tab 58 for pushing the slide glass carrier horizontally towards the 20X-Y stage 38 when driven. The homing switch 95 senses the pusher tab 58 while the belt 59 is rotating.

Looking specifically at FIG. 7A, an XY stage 38 is shown, which is controlled by the microscope controller 31 (FIG. 3) and is therefore not considered part of the glass slide manipulation subsystem. An X position motor 96 and a Y position motor 97 are provided.
The XY stage 38 further includes an opening 55 for allowing the illumination to reach the slide glass. A switch 91 is mounted adjacent to the opening 55 and immediately energizes the motor 87 to drive the stage drive belt 50 (FIG. 7B) upon sensing contact with the carrier. The drive belt 50 is a double-sided toothed belt having teeth that engage the pitch rack 68 of the carrier 60 (FIG. 6B).

The slide glass output module 17 is a slide glass carrier output hopper 1.
8, unloading platform 16 and slide glass carrier unloading subassembly 34. The unloading assembly 34 includes a motor 89 that is used to rotate the unloading platform 36 about a shaft 98 during unloading operations, which are described further below. The feed gear 93 driven by the motor 88 is
The carrier 60 is rotatably engaged with the pitch rack 68 (FIG. 6B) to transport the carrier and a rest position that presses against the switch 92. Spring holddown (
The spring loaded hold-down mechanism holds the carrier in place on the unloading platform 36.

The operation of operating the slide glass will now be described. Turning to FIG. 8, a series of glass slide carriers 60 are shown stacked in the input hopper 16 with their top edges 60a aligned. When the slide glass operation starts,
The indexing cam 56 driven by the motor 85 advances one revolution, and only one slide glass carrier is moved toward the bottom of the hopper 16 toward the loading platform 52.
Drop it up.

8A-8D show the operation of the cam in more detail. The cam 56 has an upper portion 56b
And a hub 56a to which the lower leaf 56c is attached. Leaf 56
b and 56 are semi-circular protrusions which are arranged facing each other and are vertically spaced apart. In the first state shown in FIG. 8A, the upper leaf 56b supports the carrier located at the bottom with the undercut portion 66. The cam 56 shown in FIG.
In the 20 state rotated by 80 °, the upper leaf 56b no longer supports the carrier,
Instead, the carrier descends slightly and is supported by the lower leaf 56c.
FIG. 8C shows the cam 56 rotated 270 °, in which the upper leaf 56b is rotated sufficiently to begin engaging the undercut 66 of the next slide glass carrier, but face to face. The lower leaf 56c still holds the carrier in the bottom position. After a full 360 ° rotation as shown in FIG. 8D, the lower leaf 56c has rotated opposite to the carrier stack and no longer supports the bottom position carrier, which now rests on the loading platform 52. There is.
In the same state, the upper leaf 56b supports the next carrier and repeats the cycle.

Referring again to FIGS. 7A and 7B, when the carrier descends to the loading platform 52, contact closes the switch 90 which energizes the motors 86 and 87. The motor 86 causes the pusher tab 58 to come into contact with the carrier and move the carrier in the XY direction.
The feed-in drive belt 59 is driven until it is pushed onto the stage drive belt 50. Stage drive belt 50 advances the carrier until it contacts switch 91, which is closed and the glass slide scanning process described further herein begins. When the scanning process is complete, the XY stage 38 is moved to the unloading position and motors 87 and 88 are energized to use the stage drive belt 50 to carry the carrier to the unloading platform 36. As previously mentioned, the motor 88 drives the delivery gear 93 to engage the carrier pitch rack 68 (FIG. 6B) of the carrier 60 until it contacts the switch 92. When switch 92 is closed, motor 89 is energized and unload platform 36 rotates.

The unloading operation is shown in more detail in the end views of the output module 17 (FIGS. 9A-9D). FIG. 9A shows an unloading platform 3 supporting a slide glass carrier 60.
6 is shown in a horizontal position. A hold-down mechanism 94 fixes the carrier 60 at one end. FIG. 9B shows the output module 17 after the motor 89 has rotated the unloading platform 36 to a vertical position, at which time the spring loaded holddown mechanism 94 ejects the glass slide carrier 60 into the output hopper 18. The carrier 60 is held within the output hopper 18 by ears 64 (FIGS. 6A and 6B). FIG. 9C shows the unloading platform 36 being rotated back to the 20 level. When the platform 36 rotates upward, it contacts the stacked carriers 60, and the upward movement of the platforms pushes the carriers forward of the output hopper 18. FIG. 9D illustrates the original horizontal unloading platform 36 after outputting a series of slide glass carriers 60 to the output hopper 18.

Having described the features of the entire system and automatic glass slides, aspects of the apparatus 10 relating to scanning, focusing and image processing will now be described in further detail.

In some cases, the operator knows where the scan area of interest is on the glass slide earlier than expected. A typical specimen of a slide glass for inspection can be placed at a known position by repeating the sample on the slide glass. Therefore,
The operator can configure the system to always scan the same area at the same location on every glass slide adjusted in this manner. However, at times, the area of interest may not be known, for example when the slides are manually prepared by the well-known smear technique. One feature of the invention is the automatic determination of scan areas using a texture analysis method. Figure 1
0 is a flow chart illustrating a process related to automatic position determination of the scanning area. Figure 10
As shown in, the basic method is a method in which the entire area of the slide glass is pre-scanned to confirm the texture feature indicating the presence of the smear, and the area is distinguished from an artifact such as stain.

At each position of this raster scan, for example, the image shown in FIG. 12 is obtained, and in steps 204 and 206, the texture information is analyzed. Since it is desirable to find the edge of the smear in a given image, texture analysis is performed over a region 78 called the window. This window area is smaller than the entire image as shown in FIG. This process is step 20.
Repeat scans across the glass slide at 8, 210, 212 and 214.

To increase speed, the texture analysis step is preferably performed with a 4 × objective at low magnification. One reason to operate at low magnification is to form an image of the largest glass slide area at one time. Cells are not required to be disassembled and viewed at this stage of the overall image analysis, so 4x magnification is preferred. In a general slide glass as shown in FIG. 11, a part 72b of the end of the slide glass 72 is stored for marking identification information. Except for this marker area, the entire slide glass is scanned by raster scanning method 76 to obtain several adjacent images. The texture value for each window includes the pixel variance across the window, the difference between the maximum and minimum pixel values within the window, and other indicators. The presence of the smear raises the texture value compared to the blank area.

From the viewpoint of confirming the position of the smear, one problem associated with the smear is that its thickness and texture are non-uniform. For example, the smear appears to be relatively thin at the edges and thicker towards the center due to the nature of the smearing process. To address this non-uniformity, texture analysis provides a texture value for each analyzed area. The texture values tend to increase gradually as the scan progresses across the smear from thin regions to thick regions, reaching a peak and then dropping to lower values again when reaching thin regions at the edges. The problem then is to determine the start and end or edges of the smear from a series of texture values. The texture value has a sharp start and an end (sharp begi
Since it does not have nnings and endings), it fits the waveform of a square wave.

After performing this scan and the texture evaluation scan, it must be determined which areas of high texture value show the desired smear 74 and which areas of high texture value show undesired artifacts. . This is accomplished in step 216 by fitting the function of steps to the texture values on a line-by-line basis. This function resembles a single square wave across a smear that begins at one end and ends at the other, and its amplitude provides the means of discrimination. The amplitude of the best-fitting step function is used to determine if smear or smear is present, as the higher values indicate smear. If a smear is determined to be present, the start and end coordinates of this pattern are noted until all lines have been processed and the smear area is defined at 218.

After the first focusing operation described further herein, the scan area of interest is scanned to obtain an image for image analysis. A preferred method of operation is to perform a complete scan of the glass slide at low magnification first to identify and locate the candidate object of interest, and subsequently to identify the candidate object as a cell of the candidate object of interest. This is a method of further performing image analysis at high magnification. An alternative method of operation is after the object is confirmed at low magnification,
This is a method of immediately performing high-magnification image analysis of each target candidate object. The low magnification scan then resumes the search for additional candidate objects of interest. Since it takes a few seconds to change the objective lens, this alternative operating method takes longer to complete.

The operator can preselect the level of magnification used to perform the scan. A lower magnification using a 10X objective is preferred to perform the scan because a larger area can be analyzed first for each acquired scan image. All methods of detecting cells involve a combination of decisions made at both low (10X) and high (40X) levels. The effective range of decision making at 10X magnification is wide. That is, an object that loosely fits the relevant color, size and morphological characteristics is 10X.
Confirmed at the level.

Analysis at the 40X magnification level then refines the decision making and begins to identify the object as likely a cell of interest or a candidate object. For example, at the 40X level, it is not uncommon to find that some objects identified at 10X are artifacts that the analysis process rejects. In addition, closely packed objects of interest visible at 10X are separated at the 40X level.

In the situation where cells span or overlap adjacent image fields, the cells are rejected or undetected as a result of image analysis of the individual adjacent image fields. To avoid such missing cells, the scanning actuation compensates by overlapping adjacent image fields in both the X and Y directions. Overlap sizes greater than 1/2 the average cell diameter are preferred. In the preferred embodiment, the overlap is specified as a percentage of the image field in the X and Y directions.

The time to complete the image analysis can vary depending on the size of the scan area and the number of identified candidate cells or objects of interest. In one embodiment of the preferred embodiment, a complete image analysis of a 2 cm ² scan area in which 50 objects of interest are identified can be performed in about 12-15 minutes. In this example, focus adjustment,
Not only is scanning and image analysis included, but storage of the 40X image as a mosaic on the hard drive 21 (FIG. 2) is included. Consider the utility of the present invention in a "rare event" application where there are one, two or very few cells of interest somewhere on a glass slide. The nature of the problem
To illustrate by analogy, when the slide glass is expanded to the size of a football field, for example, one tumor cell is about the size of one bottle cap. The problem in this case is to have a high certainty that the football field will be quickly searched to find caps for only a few bottles and that these caps have not been missed at all.

No matter how the scan area is defined, the first focusing operation must be performed for each glass slide before scanning. This is necessary because slides generally differ in their placement within the carrier. These differences include slight (but significant) variations in the tilt of the glass slide within its carrier. The tilt of each glass slide must be measured because it must remain in focus during scanning. This was achieved in the first focusing operation that measured the exact tilt, resulting in
Focus is maintained automatically during the scan.

The first focusing operation and other focusing operations described below utilize a focusing method based on the processing of the image acquired by the system. This method was chosen because it uses an IR beam reflected from the surface of the glass slide and is simpler than other methods that use mechanical gauges. Also, these methods do not work properly when the sample is protected with a cover glass. The preferred method described above reduces the cost of the system and improves reliability because there is no need to install additional components to perform the focus adjustment. FIG. 13A provides a flow chart illustrating the “focus” method. The basic method relies on the fact that the fractional value (i.e. standard deviation) of the mean of the pixel values is maximum at the best focus. The "brute-force" method utilizes a computer-controlled Z or focus stage to easily step through the focus, calculate the fractional value of the pixel at each step, and then calculate the maximum fractional value. Return to the focus position provided. Such a method is too time consuming. Therefore, additional features were added as shown in Figure 13A.

These features provide a quick means of determining pixel fraction values at a relatively coarse number of focus positions and then fitting a curve to the data to determine the optimum focus. Is included. This basic method applies to two steps, a coarse step and a fine step. In the coarse steps 220-230, the Z stage steps over the focus position of the user-specified range, and the size of the step is also specified by the user. It was found that these data fit the Gaussian function closely in the case of coarse focusing. Therefore, this first set of fractional value vs. focus position data is fitted at 228 with a least squares fit to a Gaussian function. The position of the peak of this Gaussian curve determines the initial or coarse estimate of the focus position for the input to step 232.

Following this, a second stepping operation 232-242 is performed utilizing smaller steps over a smaller focus range centered on the coarse focus position. Experience has shown that the data obtained over this smaller range generally best fits a quadratic polynomial. When this least squares fit is performed at 240, the quadratic peak at 244 provides the precise focus position.

FIG. 14 shows the procedure of how the above adjustment method is used to confirm the orientation of the glass slide in the carrier. As shown in FIG. 14, the focus position is confirmed at 264, as described above, for a 3 × 3 grid of points centered on the scan area. One or more of these points are outside the scan area,
If so, the method senses this at 266 by a low pixel variance value. In this case, the additional points are selected near the center of the scan area. FIG. 15 shows an initial array of points 80 and new points 82 selected near the center of the scan area. Once this array of focus positions is identified at 268, the least squares plane is fitted at 270 to this data. Foci that are too far above or below this best-fit plane (which may result from, for example, a dirty cover glass on the scan area) are discarded at 272 and the least-squares fit of the data is re-performed. This plane at 274 is the desired Z for maintaining focus during the scan.
Provides location information.

After identifying the best-fit focal plane, the scan area is scanned with an X raster scan of the scan area, as described above. During scanning, the X stage is placed at the starting point of the scan area, the focus (Z) stage is placed in the best-fit focal plane, the image is acquired and then processed as described herein, and the process is Repeat for all points on the scan area. In this way, focus can be maintained automatically without having to refocus at points that are time consuming during scanning. Before confirming the cellular objects at the 40X or 60X level, refocusing operations are required as this higher magnification requires more precise focus than the best-fit plane provides. FIG. 16 shows a flow chart of this process. As can be seen in FIG. 16, this process is similar to the fine focus method described above in that the object of interest is the object that maximizes the variance of the image pixels. It steps through a range of focal positions on the Z stage at 276, 278, calculates the image variance at each position at 278, fits a quadratic polynomial to these data at 282, then 284, At 286, the peaks of the curve are calculated to obtain the estimated best focus position. This last focus adjustment step is preceded by the fact that this magnification requires the focus setting to be in the range of 0.5 microns or better, so the range of focus and the size of the focus step are small. Different from the method described in. It should be noted that some combination of cell staining properties can be used to obtain improved focus by numerically selecting the focus position that provides the maximum variance value, which is quite different from selecting polynomial peaks. Is. In such cases, the polynomial is used to estimate the best focus, and the final step selects the actual Z position that gives the highest pixel variance value. At any point during the focusing process at 40X or 60X, if the parameter indicates that the focus position is inadequate, then the system proceeds as described above with reference to Figure 13A. It should also be noted that it automatically returns to the coarse focusing process. This guarantees rapid adaptation to variations in sample thickness. In the case of some biological samples and dyes, the previously devised focusing method does not give optimum focusing results. For example, certain white blood cells, known as neutrophils, are labeled with the well-known dye Fast R.
Stained with ed, alkaline phosphatase in the cytoplasm of the cells can be confirmed. To further confirm these cells and substances within these cells, the sample can be counterstained with hematoxylin to confirm the nuclei of the cells. In cells treated in this way, the cytoplasm with alkaline phosphatase turns red with a shade proportional to the amount of alkaline phosphatase in it, and the nucleus turns blue. However, when the cytoplasm and nucleus overlap, the cell turns purple. Obviously, such a color hinders finding the focused Z position using the focus method described above.

To find the best focus position at high magnification, a focus method such as the method shown in FIG. 13B can be used. The method selects pixels near the center of the candidate object of interest (
Beginning with block 248), a region of interest centered on the selected pixel is then defined (block 250). The width of the target area is preferably 2
The number of columns that are powers of. This choice of width results from subsequent processing of the region of interest, preferably using a one-dimensional Fast Fourier Transform (FFT) method. As is well known in the art, processing a pixel value column using the FFT method is facilitated by setting the number of columns to be processed to a power of two. In the preferred embodiment, the height of the region of interest is also a power of 2, but this is not necessary unless the region of interest is processed using a two-dimensional FFT.

After selecting the region of interest, the column of pixel values is processed using a preferred one-dimensional FFT to identify the spectrum of frequency components of the region of interest (block 252).
). Its frequency spectrum is in the range from DC to almost the highest frequency component. For each frequency component, the complex magnitude (complex magn
itude) is calculated. Preferably, the complex magnitudes of the frequency components in the range of about 25% to about 75% of the highest component are squared and summed to determine the total power of the region of interest (block 254). Alternatively, the region of interest can be processed with a smoothing window such as a Hanning window to reduce spurious high frequency components that occur when the pixel values of the region of interest are processed with FFT. This pre-treatment of the region of interest allows all complex magnitudes over the complete frequency range to be squared and summed. After calculating and storing the power of the region (block 256), a new focus position is selected, the focus is adjusted (blocks 258, 260), and the process is repeated. After evaluating each focus position, the focus position with the largest power factor is selected as the best focus position (block 262).

Described below are image processing methods utilized to determine during the scanning process whether candidate objects of interest, such as stained tumor cells, are present in a given image or field. The candidate object of interest detected during the scan has a high magnification (4
The image is re-imaged at 0X or 60X) to confirm the decision and then the area of interest for this cell is saved for later examination by a pathologist. The image processing includes color space conversion, low-pass filtering, background suppression, artifact suppression, morpheme processing, and plug analysis. One or more of these steps may be optionally removed. The operator is provided with the option of configuring the system to perform any or all of these steps and performing certain steps in succession more than once or several times. It should also be noted that the order of steps can be varied and thus optimized for a particular reagent or combination of reagents. However, the order described herein is preferred. It should be noted that the image processing steps of low pass filtering, thresholding, morphological processing, and blob analysis are commonly known building blocks of image processing.

An overview of the preferred process is shown in FIG. 17A. A preferred method of identifying and positioning candidate objects of interest in a stained biological sample on a glass slide begins with the acquisition of an image obtained by scanning the glass slide at low magnification (block 288). Each image is then converted from the first color space to the second color space (block 290) and the color converted image is then low pass filtered (block 292). The pixels of the low-pass filtered image are then compared to a threshold (block 294), and those pixels having a value equal to or greater than the threshold are then preferably determined as the target pixel. Recognized as a candidate object,
Pixels with values below the threshold are determined to be artifact or background pixels. The candidate object of the target pixel is then morphologically processed to recognize a group of candidate objects of the target pixel as candidate objects of interest (block 296).
These candidate objects of interest are then compared to the parameters of the blob analysis (block 2
98), further identify candidate objects of interest from those that do not warrant further processing because they do not fit the parameters of the blob analysis. The position of the target candidate object is stored before confirmation at high magnification. The process continues by determining if a candidate object of interest has been identified (block 300). If the candidate object of interest is not identified, the optical system is set to high magnification (block 302) and an image of the slide glass at the position corresponding to the identified candidate object of interest in the low magnification image is acquired. (Block 288). These images are then color converted (block 290).
), Low pass filtered (block 292) and compared to a threshold (block 2)
94), morphological processing is performed (block 296) and then compared to the parameters of the blob analysis (block 298) to identify which target candidate object found in the low-magnification image is the target object. Next, the coordinates of the object of interest are stored for future reference (block 303).

Candidate objects of interest, eg, tumor cells, are generally detected based on a combination of properties including size, morphology and color. The chain of decision-making based on these properties preferably begins with the process of color space conversion. CC connected to microscope subsystem
A D camera outputs a color image containing a matrix of 640 × 480 pixels. Each pixel has red, green, and blue (RGB) signal values.

The difference between the candidate objects of interest and their background, eg the difference between tumor cells and normal cells, can be seen in their respective colors, so transforming the matrix of RGB values into different color spaces. Is desirable. The samples are generally stained with one or more of the industry standard dyes “red” in color (eg DAB, New Fuchsix, AEC). The candidate object of interest appears red as it retains more dye, but normal cells remain unstained. The sample is also counterstained with hematoxylin so that the nuclei of normal cells or cells that do not contain the object of interest appear blue. In addition to these objects, stains and debris may appear black or gray, or may be tinted red or blue, depending on the dyeing method used. The fluid such as residual plasma that is also present in the smear has some color.

In the operation of color conversion, a means is provided for identifying the color information by creating a ratio of two RGB signal values. There are nine ratios, namely R / R, R / G, R / B, G / G, G / B, G / R, B / B, and B / G, depending on the three signal values for each pixel.
, B / R can be created. The optimum ratio to choose depends on the range of color information expected on the glass slide sample. As mentioned above, common dyes used to detect candidate objects of interest such as tumor cells are predominantly red rather than predominantly green or blue. Thus, the pixel of the stained cell of interest contains more red elements than the green or blue pixels. The ratio of red divided by blue (R / B) provides a value greater than 1 for tumor cells, but approximately 1 for transparent or white areas on glass slides. The remaining cells, ie, normal cells, are generally stained blue, so that the R / B ratio of pixels of these normal cells is 1
It becomes a smaller value. The R / B ratio is preferable in order to clearly distinguish the general color information in these applications.

FIG. 17B shows a flow chart for this conversion. Considering the processing speed,
The conversion is performed by a look-up table. The use of lookup tables for color conversion accomplishes the following three functions. That is, (1) perform a division operation; (2) scale the result for processing as an image with pixel values in the range 0-255.
e); (3) Define an object with low pixel values in each color band (R, G, B) as "black" to avoid infinite proportions (i.e. proportions when divided by zero);
It is a function. These "black" objects are generally dyed artifacts or may be the edges of bubbles created by gluing a coverslip over the sample.

At 304, if a look-up table was assembled for a particular color ratio (ie, tumor cell and nucleated cell dye selection range), each pixel of the original RGB image was 308
Convert with to generate output. Since it is important to distinguish red stained tumor cells from blue stained normal cells, the color ratio values are scaled by a user-specified factor. As an example, the factor 128 and (red pixel value) /
For the ratio of (blue pixel value), the transparent area on the glass slide has a ratio of 1 scaled by 128 to a final X value of 128. Pixels in tumor cells dyed in red have an X value of more than 128, but nuclei of normal cells dyed in blue have an X value of less than 128. In this way, the desired target object can be identified numerically. The obtained 640 × 480 pixel matrix (referred to as X image) is 0 to
2 is a grayscale image with values in the range of 255.

There are other ways to identify color information. One classical method transforms RGB color information into another color space, for example HSI (hue, saturation, intensity) space. In such a space, distinct hues such as red, blue, green and yellow can be easily distinguished. Moreover, relatively lightly stained objects can be distinguished from more strongly stained objects by varying the degree of saturation. However, converting RGB space to HSI space requires more complex calculations. Conversion to color ratio is faster. For example, a complete image can be converted in about 30 ms with the ratio method of the present invention, while it takes several seconds with the HSI conversion method.

In yet another way, color information can be obtained by taking a single color channel from the camera. As an example, consider the blue channel. In the blue channel, red objects are relatively dark. Blue or white objects are relatively light in the blue channel. As a general rule, take a single color channel and use anything darker than a certain threshold as a candidate object of interest, such as a tumor cell (because the cell is red, it is dark in the channel being examined). The threshold that can be classified can be easily set. However, this single channel method
Problems arise when the illumination is not uniform. Non-uniform illumination results in non-uniform overall pixel values in the color channel, eg, pixel values tend to peak in the center of the image, and at the edges where the illumination decreases. Thresholding this non-uniform color information presents a problem because the edges sometimes fall below the threshold. Therefore, it becomes more difficult to select an appropriate threshold level. However, in the case of the ratio method of the present invention, when the value of the red channel decreases from the center to the edge, the value of the blue channel also decreases from the center to the edge, resulting in a uniform ratio. Therefore, the ratio method is less sensitive to lighting.

As mentioned earlier, the color conversion method is relatively insensitive to color balance, ie changes in the relative output of the red, green and blue channels. However, some control is needed to avoid underexposure of either camera saturation or color band. This color balancing is performed automatically using a calibrating glass slide consisting of transparent areas and "dark" areas of known optical transmission or optical density. The system acquires an image from its transparent and "dark" areas and calculates the "white" and "black" adjustments to the image processor 25 to provide the correct color balance.

In addition to the color balance control described above, certain mechanical alignments are automated in this process. The center point of the field of view of various microscope objectives measured on glass slides can vary by a few microns (or a few tenths of a micron). This is a result of factors such as small variations in the position of the microscope objective lens 44a determined by the turret 44 (FIG. 4), ie small variations in the alignment of the objective lens with respect to the optical axis of the optical system. Since it is desirable for each microscope objective to be centered on the same point, such mechanical deviations must be measured and automatically corrected.

This is accomplished by forming an image of the test glass slide with recognizable features or marks. An image of this pattern is obtained by the system with a given objective and the position of the mark to be measured. The optics then rotates the turret to the next objective to acquire an image of the test object,
Reconfirm its position. A clear change in the position of the test mark is recorded for this objective lens. This process continues for all objectives. These spatial shifts, once measured, are automatically corrected by moving the stage 38 by a magnitude equal to the magnitude of the shift (but in the opposite direction) while changing the objective lens. It Thus, when selecting different objectives, there is no apparent change in the center point or field of view. The process of low pass filtering is done before thresholding. The purpose of thresholding is to obtain a pixel image matrix that contains candidate objects of interest, such as tumor cells above a threshold level, all remaining objects below the threshold level. However, the image actually acquired contains noise. The noise is in several forms, including white noise and artifacts. Microscope glass slides may have small debris that picks up the color during the dyeing process, and these debris are known as artifacts. These artifacts are generally small and spread over regions of the order of a few pixels and exceed the threshold. The purpose of low pass filtering is essentially to blur or smear the entire color transformed image. The low-pass filtering process blurs artifacts over larger objects of interest such as tumor cells, eliminating or reducing the number of artifacts that pass thresholding. The result is a more transparent, thresholded image downstream. In this low pass filter process, a 3x3 matrix of coefficients is added to each pixel in the 640x480 X image. The preferred coefficient matrix is:

[0095] 1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9

At each pixel location, a 3 × 3 matrix containing the pixel of interest and its neighbors is multiplied by the coefficient matrix and summed to give a single value for the pixel of interest. The output of this spatial convolution process is again a 640x480 matrix. As an example, consider the case where the center pixel and only its center pixel have a value of 255, and its neighbors, ie, top left, top, top right, etc., each have a value of zero.

The case of this singular white pixel corresponds to a small object. The result of matrix multiplication and addition using the coefficient matrix is 1/9 (255) or 28 for the central pixel, which is less than the reference threshold 128. Now consider another case where all pixels have a value of 255, which corresponds to a large object. In this case, performing a low-pass filtering operation on the 3x3 matrix gives a value of 255 for the central pixel. Thus, large objects retain their values, while smaller objects have less or no amplitude. In the preferred method of operation, the low pass filtering process is performed on the X image twice in succession.

In order to distinguish the target object in the X-image, eg tumor cells, from other objects and the background, we set the pixels in the target cell to a value of 255 and set all other regions to zero. Perform thresholding operations designed for. Thresholding ideally produces an image in which the cells of interest are white and the remaining images are black.
The problem faced with thresholding is where to set the threshold level. It cannot easily be considered that the cell of interest is represented by a pixel value above the reference threshold of 128. A typical image forming system uses an incandescent halogen lamp as a light source. As the bulb ages, the associated amount of red and blue output changes. As light bulbs age, blue tends to fall far below red and green. In order to absorb this source variation over time, a dynamic thresholding method is used to dynamically adjust the threshold for each acquired image.
Thus, for each 640 × 480 image, a single threshold is obtained that is unique to that image.

As shown in FIG. 18, the basic method is to calculate the average value of X values and the standard deviation of the average value for each field in 312. Then 3
At 14, a threshold is set to the average plus a value defined by the product of the (user specific) factor and the standard deviation of the color transformed pixel values.
Its standard deviation is related to the structure and number of objects in the image. The user-specific factor is preferably in the range of approximately 1.5 to 2.5. The factor is selected in the case of a glass slide in which the dye remains mainly in the cell boundary, a value at the end of the lower value range of the factor is selected, and in the case of a glass slide in which the dye is present throughout the glass slide. , The value at the end of the higher range of factors is selected. Thus, when the area encounters a greater or lesser background intensity on the glass slide, the threshold is raised or lowered to help reduce background objects. According to this method, the threshold is aged by aging of the light source during the step, so that the effects of aging are offset. At 316, the image matrix resulting from the thresholding step is
It is a binary image of black (0) 5 and white (255) pixels.

As is often the case with thresholding operations as described above, some undesired areas exceed the threshold due to noise, small stained cell debris and extra artifacts. Has a value. These artifacts are
Exclusion is desirable and possible due to its small size relative to the legitimate cells of interest. Morphological processing is used to perform this function.

Morphological processing is similar to the low pass filter convolution processing described above, except that it is applied to binary images. Morphological processing is performed pixel-by-pixel across the input image matrix, similar to spatial convolution, and the processed pixels are placed in the output matrix. The low-pass convolution process does not calculate the weighted sum of neighboring pixels as m, but uses set theory operations to combine neighboring pixels in a non-linear fashion.

Erosion is the process by which a single pixel layer is removed from the edge of an object. Extension (dilati
on) is the reverse process of adding a single pixel layer to the edge of the object. The morphological processing power is the power to further distinguish and remove small objects that have survived the thresholding process but may not be tumor cells. Morphological open (morphologi
The erosion and expansion processes that create the cal open) preferably eliminate small substances but leave large objects. For morphological processing of binary images, GA
Baxes, Digital Image Processing, 127-137, John Wiley & Sons, 1994.

FIG. 19 shows a flow chart of this processing. As shown in Figure 19, the morphological "open" process implements this suppression. The single morphological open consists of a single morphological erosion 320 followed by a single morphological extension 322. plural"
An "open" consists of multiple erosions followed by multiple extensions. In the preferred embodiment, one or two morphological opens have been found suitable. At this point in the processing chain, the processed image is a thresholded object of interest, such as a tumor cell (if present, was present in the original image), and too large to It contains some remaining artifacts that could not be eliminated by the processes of.

FIG. 20 is a flow chart showing a blob analysis performed to confirm the number, size and position of objects in a thresholded image. Brab is 2 in this case
It is defined as the area of connected pixels having the same "color" with a value of 55. A process for checking the number of such regions in 324 is performed for the entire image, and then 32
In step 6, a process of confirming the area of each detected blob and the x and y coordinates is performed.
At 328, for tumor cells, the size of each blob can be compared to a known minimum area to more accurately determine which objects are objects of interest, such as tumor cells and which objects are artifacts. The position (x, y coordinates) of the object identified as the cell of interest at this stage is saved for the final step of reimaging at 40X, described below. Objects that do not pass the above size test are discarded as artifacts.

The above processing chain identifies the object as a candidate cell of interest at its scanning magnification. As shown in FIG. 21, once the scan is complete, the system will 40 × at 330 ×.
Switching to a magnification objective lens, then at 332, each candidate is re-imaged to confirm said confirmation. Each 40 × image is reprocessed at 334 using the same steps as previously described, but with test parameters (eg, area) modified appropriately for high magnification. At 336, the area of interest centered on each identified cell is saved to a hard drive for histologist to investigate.

As mentioned above, the mosaic of stored images is made available to the pathologist for examination. As shown in FIG. 22, a mosaic 150 shows a series of images of cells confirmed by image analysis. The pathologist can then visually inspect these images to determine if each detail image should be accepted (152) or rejected (153). Five such decisions can be recorded and saved along with the mosaic of images to produce a printed report.

In addition to saving an image of the area with the cells, the coordinates of the cells are saved if the pathologist wishes to view the cells through an eyepiece or directly on an image monitor.
In this case, the pathologist reloads the glass slide carrier, selects a glass slide with cells from the mosaic of cell images for examination, and the system automatically places the cells under a microscope for observation. Place it in a proper manner.

It is known that the number of normal cells whose nuclei are stained with hematoxylin is very large, often as many as several thousand per 10 × image. Since these cells are so numerous and prone to agglutination, counting each individual nucleated cell adds undue processing burden at the expense of speed, and due to aggregation, is not always accurate. No coefficient is available. The instrument performs an estimation method by measuring the total area of each field stained blue with hematoxylin and then dividing this area by the average size of nucleated cells. FIG. 23 shows the outline of this method. In this method, a single color band (the red channel provides the best contrast for blue-stained nucleated cells) is calculated at 342 the average pixel value for each field.
There are two thresholds (high threshold and low threshold) as indicated by 44 and 346.
And then processed at 348 by counting the number of pixels between these two values. In the absence of opaque debris such as dirt, this provides the total number of mostly blue pixels. At 350, this value is divided by the average area of nucleated cells and then looped over all fields at 352 to get an approximate cell count. According to preliminary tests of this method, the accuracy is ±
15%. It should be noted that for certain glass slide preparation methods, the size of the nucleated cells may be significantly larger than the typical size. The operator can select an appropriate nucleated cell size to compensate for these characteristic values.

As with imaging systems, there is some loss in modulation transfer (ie, contrast) due to the characteristics of the modulation transfer function (MTF) of elements such as imaging lenses, cameras, electronics, etc. It is desirable to compensate for these MTF losses as it is desirable to preserve a "high quality" image of the cells of interest for the pathologist's investigation and storage. The MTF compensation or MTFC is implemented as a digital process applied to the acquired digital image. A digital filter is used to restore the high spatial frequency content of the image during storage while maintaining a low noise level. The MTFC technique enhances or restores image quality by using digital processing techniques rather than the traditional oil-based and other hardware-based methods. For MTFC, see "Tha Image Orocessin" by JCRues.
g Handbook, pp. 225-237: 1995, further described in CRC Press.

FIG. 24 shows the available functions of the user interface of the device 10. From the user interface provided graphically on the computer monitor 26,
The operator can select from among the capabilities of the device, including acquisition 402, analysis 404, and system configuration 406. At the acquisition level 402, the operator can select between a manual 408 mode and an automatic 410 mode of operation. In the manual mode, the operator is presented with a manual operation 409. Patient information regarding the assay can be entered at 412. At analysis level 404, the functionality of survey 416 and report 418 is available. At survey level 416, the operator has a montage function 4
You can select 20. At this montage level, pathologists will conduct diagnostic studies that include visiting an image, cell acceptance / rejection 424, nucleated cell count 426, cell count acceptance / rejection 428 and page preservation at 430. The function can be performed. Report level 418 allows the operator to create a patient report 432. At configuration level 406, the operator can select to configure preferences at 434, enter operator information 437 at 436, create a system log at 438 and toggle the menu panel at 440. Configuration preferences include scanning region selection functions at 442, 452; montage specifications at 444; bar code handling at 446; default cell counting at 448; dye selection at 450; and scanning objective at 454. Includes a selection of.

Computer Implementation Aspects of the invention may be implemented in hardware or software, or a combination of both. However, preferably, the algorithms and processes of the present invention each include at least one processor, at least one data storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one input device. It is implemented by one or more computer programs implemented on a programmed computer having one output device. Program code is applied to input data, perform functions described herein and generate output information. The output information is applied to one or more output devices in a known manner.

Each program can be executed in a desired computer language (including machine language, assembly language, high level procedural language or object oriented programming language) to communicate with a computer system. In any case, the computer language is a translated or interpreted language.

Each such computer program is preferably stored on a general purpose or special purpose programmed computer readable storage medium or device (eg, ROM, CD-ROM, tape or magnetic diskette). When the storage medium or storage device is read by a computer, the computer is configured to operate and the means described in the present application is implemented. It is also conceivable that the system of the present invention is used as a computer-readable storage medium that is configured by a computer program, and the storage medium configured in this way allows the computer to use a specific predefined method. To perform the functions described herein.

A number of embodiments of the invention have been described. However, various modifications may be made without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specifically exemplified embodiments but is limited by the claims of the present application.

[Brief description of drawings]

FIG. 1 is a perspective view of an automatic cell analyzer for carrying out the present invention.

FIG. 2 is a block diagram of the device shown in FIG.

3 is a block diagram of the microscope controller shown in FIG. 1. FIG.

FIG. 4 is a plan view of the device shown in FIG. 1 with the housing removed.

5 is a side view of the microscope subsystem of the apparatus shown in FIG.

6A is a top view of a slide glass carrier used in the apparatus shown in FIG. 1. FIG.

6B is a top view of the slide glass carrier shown in FIG. 6A. FIG.

7A is a top view of a slide glass automatic operation subsystem of the apparatus shown in FIG. 1. FIG.

7B is a partial cross-sectional view taken along the line AA of the slide glass self-propelled operating subsystem shown in FIG. 7A.

FIG. 8 is an end view of an input module of a slide glass automatic operation subsystem.

FIG. 8A shows an input operation of a slide glass automatic operation subsystem.

FIG. 8B shows an input operation of the slide glass automatic operation subsystem.

FIG. 8C shows an input operation of the slide glass automatic operation subsystem.

FIG. 8D shows an input operation of the slide glass automatic operation subsystem.

FIG. 9A shows an output operation of a slide glass automatic operation subsystem.

FIG. 9B shows an output operation of the slide glass automatic operation subsystem.

FIG. 9C shows an output operation of the slide glass automatic operation subsystem.

FIG. 9D shows an output operation of the slide glass automatic operation subsystem.

FIG. 10 is a flowchart of a procedure for automatically determining a scanning area.

11 shows a scanning path on a conditioned glass slide for the procedure shown in FIG.

12 shows an image of a field acquired by the procedure shown in FIG.

FIG. 13A is a flow chart of a preferred procedure for relocating a focal point.

FIG. 13B is a flow chart of a preferred procedure for determining focal position for neutrophils stained with Fast Red and counterstained with hematoxylin.

FIG. 14 is a flow chart of a procedure for automatically determining an initial focus.

15 shows an array of slide glass positions used in the procedure shown in FIG.

FIG. 16 is a flow chart of a procedure for automatic focus adjustment at high magnification.

FIG. 17A is a flow diagram of the overview of a preferred process for locating and confirming the location of objects of interest in a stained biological sample on a glass slide.

FIG. 17B is a flowchart of a procedure of color space conversion.

FIG. 18 is a flowchart of a background suppression procedure by dynamic threshold processing.

FIG. 19 is a flowchart of a morpheme processing procedure.

FIG. 20 is a flowchart of a plug analysis procedure.

FIG. 21 is a flowchart of the procedure of image processing at high magnification.

FIG. 22 shows a mosaic of cell images created by the device.

FIG. 23 is a flowchart of a procedure for estimating the number of nucleated cells in the field of view of an objective lens.

FIG. 24A illustrates the device functionality available at the device user interface.

FIG. 24B illustrates device functionality available at the user interface of the device.

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) G01N 33/48 G01N 33/483 C 5B047 33/483 G02B 21/24 5B057 G02B 21/24 21/34 21 / 34 21/36 21/36 G06T 1/00 295 G06T 1/00 295 400B 400 430F 430 450B 450 A61B 5/14 310 (81) Designated country EP (AT, BE, CH, CY, DE, DK, ES, FI) , FR, GB, GR, IE, IT, LU, MC, NL, PT, SE), OA (BF, BJ, CF, CG, CI, CM, GA, GN, GW, ML, MR, NE, SN , TD, TG), AP (GH, GM, KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZW) EA (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), AE, AL, AM, AT, AU, AZ, BA, BB, BG, BR, BY, CA, CH, CN, CR, CU, CZ, DE, DK, DM, EE, ES, FI, GB, GD, GE, GH, GM, HR, HU, ID, IL, IN, IS, JP, KE, KG, KP, KR , KZ, LC, LK, LR, LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, MX, NO, NZ, PL, PT, RO, RU, SD, SE, SG, S I, SK, SL, TJ, TM, TR, TT, TZ, UA, UG, US, UZ, VN, YU, ZA, ZW (72) Inventor Eris Bob USA California 92677 Laguna Niguel No. 22 Bee Clubhouse Drive 30902 F term (reference) 2G045 AA26 BA14 BB24 CB01 FA16 FA19 FB03 2G054 AA08 CA23 CE10 EA06 FA17 JA00 2G059 AA05 AA06 BB12 BB13 CC16 DD13 EE13 FF03 KK04 MM01 MM10 PP01 2H052 AB05 AC05 AC28 AC30 AD04 AD09 AD15 AD19 AD20 AD26 AD33 AE04 AE10 AF01 AF02 AF03 AF07 AF13 AF14 AF16 AF23 AF25 4C038 KK00 KL01 KL07 KM00 KM01 KX01 KY03 KY04 5B047 AA15 AA17 AB04 BB04 BC05 BC16 BC23 CA01 CA17 CB09 CB10 CB22 DC09 5B057 AA07 BA02 BA17 DA06 DB02 DB06 DB09 DC04 DC09 DC25

Claims (13)

[Claims] 1. A method for automatically detecting a cell proliferation disorder, which comprises: providing a sample on a glass slide, wherein the sample is stained with an antibody for a protein associated with the cell proliferation disorder; and treating the stained sample. Confirm the parameters; Scan multiple areas of the stained sample at low magnification on the optical system; Acquire low-magnification low-magnification images of each part of the scanning area; Process each low-magnification image Detect the candidate object of interest; store the stage coordinates of each part of each candidate object of interest; adjust the optical system to a high magnification; reposition the stage to the part of each candidate object of interest; Acquiring a higher-magnification image of each of the candidate objects of; and then storing each higher-magnification image of the candidate object. 2.   The method of claim 1, wherein the cell proliferation disorder is a neoplasm. 3.   The method according to claim 1, wherein the cell proliferation disorder is breast cancer. 4. The method according to claim 1, wherein the antibody is selected from the group consisting of an anti-HER2 / neu antibody, an anti-estrogen receptor antibody, an anti-progesterone receptor antibody, an anti-p53 antibody and an anti-cyclin D1 antibody. 5. The method of claim 1, wherein the treatment parameters are selected from the group consisting of positive controls, negative controls or test samples. 6. The detection of candidate objects of interest is performed by size, color or morphology.
The method described in. 7. An automated system for quantifying cell proliferative disorders in a sample; a slide glass configured to receive the sample, wherein the sample is stained with antibodies for proteins associated with the cell proliferative disorders. A slide glass; a code attached to the slide glass for identifying the processing parameters of the slide glass; a lens for enlarging the sample on the slide glass; a stage configured to hold the slide glass, which is attached to the lens. An optically coupled stage; a camera optically coupled to the lens and configured to capture an image from the lens; a computer program residing on a computer readable medium, the computer program comprising: Perform processing parameters of the code and target candidates based on color, morphology or size Select the position to confirm the body on the stage, acquire a low-magnification image of the target candidate object, store the position of the target candidate object, adjust the lens magnification, and move the stage to the target candidate object. A computer program for relocating to a position, acquiring a higher magnification image of the candidate object of interest, storing the higher magnification image, and retrieving the acquired image; and observing the image An automated system comprising a monitor configured for. 8.   The system of claim 7, wherein the code is a bar code. 9.   The system of claim 7, wherein the lens is a microscope objective. 10.   The system of claim 7, wherein the camera is a CCD camera. 11. A computer program residing on a computer readable medium, the computer performing the following: processing parameters of a glass slide, selecting a position on a stage of an optical system, Check the target candidate object by color, shape or size, obtain a low-magnification image of the target candidate object, store the position of the target candidate object, adjust the magnification of the optical system, and move the stage to the target. To obtain a higher magnification image of the candidate object of interest, store the higher magnification image, and enable retrieval of the acquired image. A computer program that comprises instructions. 12. An apparatus comprising a computer-readable storage medium for explicitly implementing program instructions for quantifying cell proliferative disorders, the program instructions comprising: That is, the processing parameters of the slide glass are performed, the position on the stage of the optical system is selected, the target object of interest is confirmed by color, shape or size, and a low-magnification image of the target object of interest is acquired, The position of the target candidate object is stored, the magnification of the optical system is adjusted, the stage is repositioned at the position of the target candidate object, and a higher-magnification image of the target candidate object is acquired. A device for invoking an operable instruction that causes an image to be stored and to be able to retrieve the acquired image. 13.   13. The device according to claim 12, wherein the image is a digital image.