WO2005114578A1 - Procede et systeme de quantification automatisee d'une analyse d'image numerique d'un jeu ordonne de microechantillons de tissu (tma) - Google Patents

Procede et systeme de quantification automatisee d'une analyse d'image numerique d'un jeu ordonne de microechantillons de tissu (tma) Download PDF

Info

Publication number
WO2005114578A1
WO2005114578A1 PCT/US2005/017481 US2005017481W WO2005114578A1 WO 2005114578 A1 WO2005114578 A1 WO 2005114578A1 US 2005017481 W US2005017481 W US 2005017481W WO 2005114578 A1 WO2005114578 A1 WO 2005114578A1
Authority
WO
WIPO (PCT)
Prior art keywords
digital image
tma
interest
objects
contrast
Prior art date
Application number
PCT/US2005/017481
Other languages
English (en)
Inventor
Abhijeet S. Gholap
Gauri A. Gholap
Prithviraj Jadhav
M. D. Sanford H. Barsky
Phd C.K.V. Rao
Madhura Vipra
Original Assignee
Bioimagene, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US10/938,314 external-priority patent/US20050136509A1/en
Priority claimed from US10/966,071 external-priority patent/US20050136549A1/en
Application filed by Bioimagene, Inc. filed Critical Bioimagene, Inc.
Publication of WO2005114578A1 publication Critical patent/WO2005114578A1/fr

Links

Classifications

    • C—CHEMISTRY; METALLURGY
    • C12—BIOCHEMISTRY; BEER; SPIRITS; WINE; VINEGAR; MICROBIOLOGY; ENZYMOLOGY; MUTATION OR GENETIC ENGINEERING
    • C12Q—MEASURING OR TESTING PROCESSES INVOLVING ENZYMES, NUCLEIC ACIDS OR MICROORGANISMS; COMPOSITIONS OR TEST PAPERS THEREFOR; PROCESSES OF PREPARING SUCH COMPOSITIONS; CONDITION-RESPONSIVE CONTROL IN MICROBIOLOGICAL OR ENZYMOLOGICAL PROCESSES
    • C12Q1/00—Measuring or testing processes involving enzymes, nucleic acids or microorganisms; Compositions therefor; Processes of preparing such compositions
    • C12Q1/68—Measuring or testing processes involving enzymes, nucleic acids or microorganisms; Compositions therefor; Processes of preparing such compositions involving nucleic acids
    • C12Q1/6876—Nucleic acid products used in the analysis of nucleic acids, e.g. primers or probes
    • C12Q1/6883—Nucleic acid products used in the analysis of nucleic acids, e.g. primers or probes for diseases caused by alterations of genetic material
    • C12Q1/6886—Nucleic acid products used in the analysis of nucleic acids, e.g. primers or probes for diseases caused by alterations of genetic material for cancer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/40—Image enhancement or restoration using histogram techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G06T7/0014—Biomedical image inspection using an image reference approach
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/77—Determining position or orientation of objects or cameras using statistical methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30072—Microarray; Biochip, DNA array; Well plate

Definitions

  • This invention relates to digital image processing. More specifically, it relates to a method and system for automated quantitation of tissue micro-array (TMA) image analysis.
  • TMA tissue micro-array
  • Tissue micro-arrays are multiple specimen slides that contain hundreds of individual tissues for one or multiple different biological specimens.
  • TMA allows staining (e.g., with Haematoxylin and Eosin (H/E) stain, etc.) and analysis of hundreds of samples on a slide over traditional one per slide.
  • Tissues from multiple patients or blocks are relocated from conventional histologic paraffin blocks on the same slide. This is done by using a needle to biopsy a standard histologic sections and placing the core into an array on a recipient paraffin block. This technique was originally described by in 1987 by Wan, Fortuna and Furmanski in Journal of Immunological Methods. They prepared of "cores" of paraffin- embedded tissue from standard histology blocks.
  • tissue micro-array TMA
  • TMA technology allows rapid visualization of molecular targets in thousands of tissue specimens at a time, at the DNA, RNA, protein levels, etc. Moreover, this technique requires less tissue for analysis and offers consistency in reporting results. Additionally with serial sections of the master block, investigators can analyze numerous biomarkers over essentially identical samples. Configuration of TMA depends on the end use. There could be samples of every organ in a particular animal's or human's body, or a variety of common cancers like breast and colon carcinomas with normal controls, or rare or obscure cases, such as an array of salivary gland tumors. An array of tissues from different knockout mice or a single, specific tissue (e.g., from cultured cells) could also be assayed. These slides with TMAs are treated like other individual histological section, using in situ hybridization to detect gene expression or identify chromosomal abnormalities, or employing immunohistochemistry (IHC) to localize protein expression.
  • IHC immunohistochemistry
  • TMAs to validate potential drug targets identified with DNA TMAs.
  • Scientists typically construct TMAs in paraffin blocks.
  • Each tissue core in the array is collected as a "punch” generally 0.6 millimeters (mm) to 2.0 mm in diameter, at a spacing of about 0.7 mm to 0.8 mm from a donor block of paraffin-embedded tissue, using a needle.
  • the surface area of each sample is about 0.282 mm 2 , or in pathologists' terms, about the size of 2-3 high power fields.
  • a second, slightly smaller needle is used to create a hole in the recipient block.
  • the tissue cores are then arrayed in the recipient block to produce a master block, from which researchers can obtain around 200 individual 5 micrometer ( ⁇ m) slices.
  • TMA technology Most of the applications of the TMA technology have come from the field of cancer research. Examples include analysis of the frequency of molecular alterations in large tumor materials, exploration of tumor progression, and identification of predictive or prognostic factors and validation of newly discovered genes as diagnostic and therapeutic targets.
  • a standard histologic section is about 3-5 mm thick, with variation depending on the submitting pathologist or tech. After use for primary diagnosis, the sections can be cut 50-100 times depending on the care and skill of the sectioning technician. Thus, on average, each archived block might yield material for a maximum of 200 assays.
  • this same block is processed for optimal TMA construction it could routinely be needle biopsied 200-300 times or more depending on the size of the tumor in the original block (Theoretically it could be biopsied lOOO's of times based on calculations of area, but empirically, 200-300 is selected as a conservative estimation).
  • TMAs Once TMAs are constructed, they can be judiciously sectioned in order to maximize the number of sections cut from an array.
  • the sectioning process uses a tape-based sectioning aid that allows cutting of thinner sections.
  • Optimal sectioning of arrays is obtained with about 2-3 ⁇ m sections.
  • TMA techniques instead of 50-100 conventional sections or samples for analysis from one tissue biopsy, TMA techniques produce material for 500,000 assays (assuming 250 biopsies per section times 2000 2.5 ⁇ m sections per 5mm array block) represented as 0.6 mm disks of tissue. TMA techniques essentially amplifies (up to 10,000 fold) from a limited tissue resource.
  • TMA micro liter
  • Another significant advantage is that only a very small (a few micro liter ( ⁇ l)) amount of reagent is required to analyze an entire TMA.
  • This advantage raises the possibility of use of TMA in screening procedures (for example in hybridoma screening), a protocol that is impossible using conventional sections. TMAs also save money when reagents are costly.
  • TMAs also save money when reagents are costly.
  • the original block of tissue must be returned to the patient or donating institution. In these cases the tissue block may be cored a few times without destroying the block. Then upon subsequent sectioning, it is still possible to make a diagnosis because tissue has been taken for TMA-based studies.
  • this type of research helps clinicians make better diagnoses and better decisions about patient care.
  • TMA applications include studies that attempt to link gene expression data with stages of tumor progression, screening and validation of drug targets, and quality control for molecular detection methods.
  • Example applications of tissue micro-arrays in cancer research including analyzing the frequency of a molecular alteration in different tumor types, to evaluate prognostic markers, to test potential diagnostic markers and optimize antibody-staining conditions.
  • tissue micro-arrays per se, were developed by researchers at the National Cancer Institute, it is not surprising that early adopters of this technology are using them in oncology. Future market growth will be driven by adoption of tissue micro-arrays in other areas of research, such as neurobiology and infectious disease, as well as their increased utilization in high-throughput analysis of tissue sections, validation of DNA micro-array data and biomarker discovery.
  • TMA is an informatics challenge.
  • a software system for image archiving allowing a user to examine digital images of individual histological specimens, such as tissue cores from a TMA; evaluate and score them; and store all the data in a relational database is essential for TMA.
  • Tissue scoring is inherently subjective and imprecise. It is nonquantitative based on a manual score using a four-point scale: negative, weak positive, strong positive, or no data. It calls for an automated image analysis process that can localize and quantify the biomarkers in the given set of array. It can assist pathologist in more objective analysis. Quantitative measurements ultimately will allow predictions about patient outcomes and their response to therapy. But for most, the promise of TMAs remains unfulfilled, because scientists lack user friendly methods of high-speed automated quantitative.
  • TMA analysis A number of companies have developed a variety of hardware and software solutions for TMA analysis. For example, Bacus Laboratories' BLISS system uses a tiling approach that scans a TMA piecemeal and then stitches together all the tiles to produce a single composite image.
  • Aperio Technologies' ScanScope digitizes an entire TMA array slide by applying linear detector technology used in fax machines. Trestle, with its MedScan product employs area scanning. Applied Imaging's Ariol imaging and analysis system can image both colorimetric and fluorescently labeled samples.
  • TissueAnalytics Array ⁇ the software from Tissueinformatics Inc., gives information about the subcellular location of staining and can detect the presence of rare events, proteins expressed at low levels.
  • Profiler portal.path.med.umich.edu
  • TMA tissue cores from a TMA
  • evaluate and score them and store all the data in a relational database.
  • TMA-Deconvoluter is a series of Excel macros that helps researchers get TMA data into a format that can be read by conventional data analysis tools like Cluster and Tree View (rana.lbl.gov).
  • Cluster runs a hierarchical cluster analysis on the TMA data, helping users to interpret the highly complex datasets obtained from TMAs stained with large numbers of antibodies, and TreeView allows researchers to browse the clustered data.
  • Stainfinder is a Web interface that links the clustered TMA data to an online image database, allowing scientists to rapidly reevaluate the data and compare different stains on the same core.
  • TMA tissue micro-array
  • the method and system may improve automated analysis of digital images including biological samples such as tissue samples from digital images of a tissue micro-array (TMA) and aids automated diagnosis of diseases (e.g., cancer).
  • TMA tissue micro-array
  • the method and system provides reliable automatic TMA core gridding and automated TMA core boundary detection.
  • FIG. 1 is a block diagram illustrating an exemplary automated biological sample analysis processing system
  • FIG. 2 is a flow diagram illustrating a method for locating objects of interest a digital image of a tissue sample from a tissue micro-array (TMA);
  • TMA tissue micro-array
  • FIG. 3 is a flow diagram illustrating a method for locating objects of interest a digital image of a tissue sample from a tissue micro-array (TMA);
  • TMA tissue micro-array
  • FIG. 4A is a block diagram illustrating an original digital image of a
  • FIG. 4B is a block diagram illustrating a contrast adjusted digital image of the original digital image of FIG. 4 A;
  • FIG. 4C is a block diagram 62 illustrating a plural centers located in the contrast adjusted digital image of FIG. 4B;
  • FIG. 5 is a block diagram illustrating a Gausssian kernel
  • FIG. 6A is a block diagram illustrating a fragmented TMA core
  • FIG. 6B is a block diagram illustrating a TMA core spread into an irregular shape
  • FIG. 7 is a block diagram illustrating plural TMA core boundaries in the contrast adjusted digital image of FIG. 4C.
  • FIG. 8 is a block diagram illustrating an exemplary flow of data in the automated biological sample processing system. DETAILED DESCRIPTION OF THE INVENTION Exemplary automated biological sample analysis system
  • FIG. 1 is a block diagram illustrating an exemplary automated biological sample processing system 10.
  • the exemplary system 10 includes one or more computers 12 with a computer display 14 (one of which is illustrated).
  • the computer display 14 presents a windowed graphical user interface ("GUI") 16 with multiple windows to a user.
  • GUI windowed graphical user interface
  • the system 10 may optionally include a microscope or other magnifying device (not illustrated in FIG. 1).
  • the system 10 further includes a digital camera 18 (or analog camera) used to provide plural digital images 20 in various digital images or digital data formats.
  • One or more databases 22 include biological sample information in various digital images or digital data formats.
  • the databases 22 may be integral to a memory system on the computer 12 or in secondary storage such as a hard disk, floppy disk, optical disk, or other non- volatile mass storage devices.
  • the computer 12 and the databases 22 may also be connected to an accessible via one or more communications networks 24.
  • the one or more computers 12 may be replaced with client terminals in communications with one or more servers, or with personal digital/data assistants (PDA), laptop computers, mobile computers, Internet appliances, one or two-way pagers, mobile phones, or other similar desktop, mobile or hand-held electronic devices.
  • PDA personal digital/data assistants
  • the communications network 24 includes, but is not limited to, the Internet, an intranet, a wired Local Area Network (LAN), a wireless LAN (WiLAN), a Wide Area Network (WAN), a Metropolitan Area Network (MAN), Public Switched Telephone Network (PSTN) and other types of communications networks 24.
  • LAN Local Area Network
  • WiLAN wireless LAN
  • WAN Wide Area Network
  • MAN Metropolitan Area Network
  • PSTN Public Switched Telephone Network
  • the communications network 24 may include one or more gateways, routers, or bridges. As is known in the art, a gateway connects computer networks using different network protocols and/or operating at different transmission capacities. A router receives transmitted messages and forwards them to their correct destinations over the most efficient available route. A bridge is a device that connects networks using the same communications protocols so that information can be passed from one network device to another. [0047]
  • the communications network 24 may include one or more servers and one or more web-sites accessible by users to send and receive information useable by the one or more computers 12. The one ore more servers, may also include one or more associated databases for storing electronic information.
  • the communications network 24 includes, but is not limited to, data networks using the Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Internet Protocol (IP) and other data protocols.
  • TCP Transmission Control Protocol
  • UDP User Datagram Protocol
  • IP Internet Protocol
  • TCP provides a connection-oriented, end-to-end reliable protocol designed to fit into a layered hierarchy of protocols which support multi-network applications.
  • TCP provides for reliable inter-process communication between pairs of processes in network devices attached to distinct but interconnected networks.
  • IPF Internet Engineering Task Force
  • RRC Request For Comments
  • UDP provides a connectionless mode of communications with datagrams in an interconnected set of computer networks.
  • UDP provides a transaction oriented datagram protocol, where delivery and duplicate packet protection are not guaranteed.
  • IETF RFC- 768 the contents of which incorporated herein by reference.
  • IP is an addressing protocol designed to route traffic within a network or between networks. IP is described in IETF Request For Comments (RFC)-791, the contents of which are incorporated herein by reference. However, more fewer or other protocols can also be used on the communications network 19 and the present invention is not limited to TCP/UDP/IP.
  • the one or more database 22 include plural digital images 20 of biological samples taken with a camera such as a digital camera and stored in a variety of digital image formats including, bit-mapped, joint pictures expert group (JPEG), graphics interchange format (GIF), etc.
  • JPEG joint pictures expert group
  • GIF graphics interchange format
  • the present invention is not limited to these digital image formats and other digital image or digital data formats can also be used to practice the invention.
  • the digital images 20 are typically obtained by magnifying the biological samples with a microscope or other magnifying device and capturing a digital image of the magnified biological sample (e.g., groupings of plural magnified tissue samples, etc.).
  • An operating environment for the devices of the exemplary system 10 include a processing system with one or more high speed Central Processing Unit(s) (“CPU”), processors and one or more memories.
  • CPU Central Processing Unit
  • processors and one or more memories.
  • CPU Central Processing Unit
  • memories one or more memories.
  • acts and symbolically represented operations or instructions include the manipulation of electrical signals by the CPU or processor.
  • An electrical system represents data bits which cause a resulting transformation or reduction of the electrical signals or biological signals, and the maintenance of data bits at memory locations in a memory system to thereby reconfigure or otherwise alter the CPU's or processor's operation, as well as other processing of signals.
  • the memory locations where data bits are maintained are physical locations that have particular electrical, magnetic, optical, or organic properties corresponding to the data bits.
  • the data bits may also be maintained on a computer readable medium including magnetic disks, optical disks, organic memory, and any other volatile (e.g., Random Access Memory (“RAM”)) or non-volatile (e.g., Read-Only Memory (“ROM”), flash memory, etc.) mass storage system readable by the CPU.
  • RAM Random Access Memory
  • ROM Read-Only Memory
  • the computer readable medium includes cooperating or interconnected computer readable medium, which exist exclusively on the processing system or can be distributed among multiple interconnected processing systems that may be local or remote to the processing system.
  • sample includes cellular material derived from a biological organism. Such samples include but are not limited to hair, skin samples, tissue samples, cultured cells, cultured cell media, and biological fluids.
  • tissue refers to a mass of connected cells (e.g., central nervous system (CNS) tissue, neural tissue, or eye tissue) derived from a human or other animal and includes the connecting material and the liquid material in association with the cells.
  • biological fluid refers to liquid material derived from a human or other animal.
  • sample also includes media containing isolated cells.
  • One skilled in the art may determine the quantity of sample required to obtain a reaction by standard laboratory techniques. The optimal quantity of sample may be determined by serial dilution.
  • biological component include, but not limited to nucleus, cytoplasm, membrane, epithelium, nucleolus and stromal.
  • medical diagnosis includes analysis and interpretation of the state of tissue material in a biological fluid. The interpretation includes classification of tissue sample as “benign tumor cell” or “malignant tumor cell”. Interpretation also includes quantification of malignancy.
  • a digital image 20 typically includes an array, usually a rectangular matrix, of pixels.
  • Each "pixel" is one picture element and is a digital quantity that is a value that represents some property of the image at a location in the array corresponding to a particular location in the image.
  • the pixel values typically represent a gray scale value.
  • Pixel values for a digital image 20 typically conform to a specified range.
  • each array element may be one byte (i.e., eight bits). With one-byte pixels, pixel values range from zero to 255. In a gray scale image a 255 may represent absolute white and zero total black (or visa-versa).
  • Color images consist of three color planes, generally corresponding to Red, Green, and Blue (RGB). For a particular pixel, there is one value for each of these color planes, (i.e., a value representing the red component, a value representing the green component, and a value representing the blue component). By varying the intensity of these three components, all colors in the color spectrum typically may be created.
  • One type of commonly examined two-dimensional digital images 20 are digital images made from biological samples including cells, tissue samples, etc. Such digital images are commonly used to analyze biological samples including a determination of certain know medical conditions for humans and animals. For example, digital images are used to determine cell proliferate disorders such as cancers, etc. in humans and animals.
  • TISSUE MICRO-ARRAYS TISSUE MICRO-ARRAYS
  • TMA tissue micro-array
  • the method and system described herein automatically analyze a digital image of a TMA created using a needle or other means to biopsy standard histologic sections and placing the resulting needle core or other core into a micro- array.
  • FIG. 2 is a flow diagram illustrating automated Method 26 for processing digital images of tissue micro-arrays (TMA).
  • TMA tissue micro-arrays
  • Step 28 plural of objects of interest are differentiated from a background portion of a digital image of a tissue micro- array (TMA) of a tissue sample to which a chemical compound has been applied by adjusting a contrast of the digital image to create a contrast adjusted digital image.
  • Step 30 plural boundaries of plural individual TMA cores are identified in the differentiated plural objects of interest in the contrast adjusted digital image based on a plural predetermined factors.
  • a medical conclusion is formulated using the identified plural TMA cores in the contrast adjusted digital image.
  • Method 26 is illustrated with one exemplary embodiment. However, the present invention is not limited to such an embodiment and other embodiments can also be used to practice the invention.
  • plural of objects of interest including plural TMA cores are automatically differentiated from a background portion of a digital image of a TMA of a tissue sample to which a chemical compound has been applied by adjusting a contrast of the digital image to create a contrast adjusted digital image.
  • Step 28 includes making the plural potential TMA cores darker and the background portion lighter in the digital image by adjusting a contrast of the digital image to create a contrast adjusted digital image using determined maximum and minimum pixel values obtained from the digital image. Other pixel values within plural potential TMA cores and the background portion are mapped into a range including the maximum and minimum pixel values.
  • the present invention is not limited to this embodiment and other embodiments can be used to practice the invention at Step 28.
  • the chemical compound includes a Haematoxylin and Eosin (H/E) stain.
  • H/E staining is used so the red and blue color planes are used to determine stained pixeal in potential TMA core and non-stained background portion pixels. If a biological tissue sample was treated with a chemical compound other than H/E stain, stained and non-stained pixels in the digital image 20 would appear as a different colors and thus other color planes would be used to practice the invention and determined stained and unstained pixels.
  • plural boundaries of plural TMA cores are automatically identified in the differentiated potential TMA cores of interest in the contrast adjusted digital image based on a plural predetermined factors.
  • the predetermined factors include, but are not limited to, size, shape, length, width, core boundary characteristics, overlapping core areas, core grid position and pixel intensity of potential TMA cores.
  • a medical conclusion is automatically formulated using the identified plural TMA cores in the contrast adjusted digital image.
  • the medical conclusion includes a medical diagnosis or medical prognosis for a human cancer.
  • the human cancer includes a human breast cancer, prostrate cancer or other human cancers.
  • the medical diagnosis may also be made for animals.
  • graphical lines are drawn around individual identified TMA cores to make them easier to identify (e.g., see FIG. 7).
  • the displayed TMA cores are graphically displayed on a GUI on display 14.
  • FIG. 3 is a flow diagram illustrating automated Method 34 for processing digital images of tissue micro-arrays (TMA).
  • TMA tissue micro-arrays
  • Step 36 plural objects of interests in a digital image of a tissue sample from a tissue micro-array (TMA) are differentiated from a background portion of the digital image by adjusting a contrast of the digital image to create a contrast adjusted digital image.
  • Step 38 plural centers of plural differentiated objects of interest are located in the contrast adjusted digital image.
  • a digital filter is applied to the located plural centers of the plural objects of interest to remove unwanted objects.
  • plural areas of interest are expanded around the filtered plural centers of the plural objects of interest.
  • Step 44 overlapping objects, if any, are determined from the plural expanded areas of interest.
  • Step 46 plural boundaries of the plural expanded areas of interest are determined for plural TMA cores to allow a medical conclusion to be formulated.
  • Method 34 is illustrated with one exemplary embodiment. However, the present invention is not limited to such an embodiment and other embodiments can also be used to practice the invention.
  • FIG. 4A is a block diagram illustrating an original digital image 50 of a TMA.
  • FIG. 4A illustrates plural exemplary TMA cores 52 and a background portion 54 in the original digital image 50.
  • digital image 50 adjustment is done through histogram modification.
  • the present invention is not limited to such an embodiment and other digital image adjustment methods can be used to practice the invention.
  • Histogram modification helps ensure that the digital image 50 is insensitive to variations in staining intensity, image capturing device sensitivity, optical microscope lighting conditions.
  • contrast of the Red, Green and Blue (RGB) color planes of image are stretched based on image statistics, namely mean and standard deviation calculated based on pixel intensity values.
  • pixel values are computed using the Equation (1).
  • Modified pixel intensity Conl * (Pixel Value)/ (P max - P in), (1)
  • Conl is a first constant with a maximum value in the enhanced range or 255.
  • the present invention is not limited to this constant value and or contrast adjusting equation other constant values and other contrast adjusting equations can also be used to practice the invention.
  • Color values at a given pixel are independently computed from Red, Green and Blue components of the digital image 50.
  • a determination of an active range of original intensities in each of the colors is made by computing histograms of color planes (i.e., R, G and B) of the digital image 50.
  • the histograms are used to compute a minimum intensity such that, starting from lowest intensity, cumulative pixels up to minimum intensity is equal to about 0.5% to 5% of a total number pixels in the digital image.
  • An original active range is mapped to an enhanced range of intensity value (zero, 255). All pixels with value less than minimum intensity are also set to a value of zero.
  • the present invention is not limited to this embodiment and other percentages and active ranges of intensities can also be used to practice the invention.
  • These histograms are used to compute a minimum intensity such that, starting from lowest intensity, the cumulative pixels up to minimum intensity is equal to pre-defined percentage "P m in,” and a maximum intensity such that, starting from lowest intensity, the cumulative pixels up to maximum intensity is equal to a predefined percentage "P ma ⁇ -" Pixels in the active range, that is, in between minimum intensity and maximum intensity value are later mapped to an enhanced range (e.g., zero to 255). Equation(l) is used for modifying pixel intensities.
  • a pre-defined percentage of 2% is used for "P m m," for determining a minimum intensity in each color plane in the current embodiment.
  • P m m a pre-defined percentage of 2%
  • the present invention is not limited to such a pre-defined percentage and other pre-defined percentages can also be used to practice the invention.
  • a pre-defined percentage of 90% is used for "P max ,” for determining a maximum intensity in each color plane in the current embodiment.
  • P max a pre-defined percentage of 90%
  • the present invention is not limited to such a pre-defined percentage and other pre-defined percentages can also be used to practice the invention.
  • FIG. 4B is a block diagram illustrating a contrast adjusted digital image 56 of the original digital image 50 of FIG. 4A.
  • FIG. 4B illustrates the plural TMA cores 58 are darker in color than the brighter background portion 60 of the contrast adjusted digital image after automated processing at Step 36.
  • plural centers of plural differentiated objects of interest are located in the contrast adjusted digital image 56.
  • a given TMA there could be several hundred cores, some of these cores are placed away from the center of an ideal grid used for analysis.
  • cores at some of the grid positions for a TMA might be missing.
  • locating centers of cores present in a TMA is done using Gaussian kernel.
  • the present invention is not limited to such an embodiment and other embodiments can also be used to practice the invention.
  • FIG. 5 is a block diagram illustrating a Gausssian kernel 66.
  • a Gaussian kernel of sigma three is used as is illustrated in Equation (2).
  • the present invention is not limited to this embodiment another other Gaussian kernels and other equations to find a center of a TMA core can also be used to practice the invention.
  • Gaussian kernel f x) power (e - constantG*x*x / (Sigma*Sigma)) / (Sigma * sqrt (ConC*pi)), (2)
  • Equation (3) A Gaussian kernel is used for convolution with a modified image as is illustrated in Equation (3).
  • G ⁇ f( ⁇ )*i ⁇ , (3) ⁇ - ⁇ ( e ⁇ nelsize/2)
  • G is a Gaussian value at a color position
  • kernel size 1 + 2 * ceiling (2.5 * Sigma)
  • Ix is a pixel value at x. Pixels that are on a curve of symmetry of epithelial cell or epithelial area are marked. Typically there will be two curves of symmetry, one parallel to X-axis and the other parallel to Y-axis. Pixels belonging to an area of interest are selected based on the intensity. Pixels with intensity value less than (i.e., Mean + Standard Deviation) of the image are selected as pixels belonging to an area of interest.
  • the present invention is not limited to using the Gaussian kernel illustrated in Equation (3) and other equations can also be used to practice the invention.
  • a selected pixel is considered to be on the curve of symmetry (i.e., horizontal) only if the pixel's intensity value is less than five neighboring pixels intensity values in a upper direction and five neighboring pixel intensity values in a lower direction.
  • Table 1 illustrates selection of pixel "F".
  • the intensity value of Pixel F should be less than or equal to the intensity values pixels A, B, C, D, E, G, H, I ,J and K.
  • a selected pixel is considered to be on the curve of symmetry (i.e., vertical) only if a pixel intensity value is less than five neighboring pixels in first (e.g., left of) direction and five neighboring pixels intensity value in a second direction (e.g., right of). That is, in a row of eleven pixels, the intensity value of pixel F should be less than or equal to the intensity values pixels A, B, C, D, E, G, H, I, J and K as is illustrated in Table 2.
  • TMA core areas 58 are identified as a set of X-connected pixels that satisfy above conditions.
  • X 8.
  • the present invention is not limited to such an embodiment and other values can be used for X.
  • FIG. 4C is a block diagram 62 illustrating a plural centers 64 located in the contrast adjusted digital image 56 of FIG. 4B after the automatic execution of Step 40.
  • a digital filter is applied to the located plural centers of the plural objects in the contrast adjusted digital image. It is observed that digital images of TMAs have artifacts, dust particles and other objects of non- tissue material. Removing such objects with a digital filter increases the accuracy of calculating a distance between adjacent grid points in the TMA.
  • a digital filter based on an expected size of a TMA core 58 is used.
  • a normal size of a TMA core is about 0.6 millimeter (mm) to 2.0 mm (i.e., "normal size") in diameter.
  • the present invention is not limited to such an embodiment and other types of digital filters can also be used to practice the invention.
  • TMA cores that are of very small size (e.g., less than about 0.6 mm) and/or a very large size (greater than about 2.0 mm) are filtered out from further consideration. Only cores of a "normal size" are used to avoid errors in later automated calculations of average TMA core width and height.
  • FIG. 6 A is a block diagram illustrating a fragmented TMA core 68.
  • FIG. 6B is a block diagram illustrating a TMA core spread into an irregular shape 70.
  • an initial estimate of a TMA core boundary is arrived upon based on a threshold computed from contrast adjusted digital image statistics.
  • a lateral-X and lateral-Y histograms around left right and top-bottom of each TMA core 64 is used to extend an area of interest.
  • a lateral histogram in the X direction gives a measure of the number of pixels belonging to tissue in a column in the area of interest. If this count is very low (ideally it should be zero) and there is no difference between adjacent rows for three columns the current row is the edge of a TMA core 64.
  • a lateral histogram in Y direction gives a measure of a number of pixels belonging to tissue in a row in the area of interest. If this count is very low (ideally should be zero) and there is no difference between adjacent columns for three rows then the current column is an edge of the core.
  • TMA cores 64 Dilating a thresholded image followed by eroding the dilated image is used to remove noise (i.e., very small objects). Removal of small objects eliminates dust and artifacts. Treating these as potential TMA cores 64 leads to incorrect gridding, as these very small objects may be scattered on the slide.
  • the dimensions of the detected TMA cores 64, including length and breadth are used to eliminate non-core tissue parts. If a tissue part is having height or width less than the minimum height or width of the normal TMA cores then it is deleted from the rest of the process. Average core size is computed.
  • overlapping objects, if any, from the located plural objects are determined. It is observed that many TMA cores 64 in a TMA are far from ideal grid positions. An extent (i.e., spread) of a TMA core 64 might be off from a center of a grid by a huge margin. This might lead to a situation where several cores 64 touch each other or significantly overlap. Therefore it is necessary to detect touching or overlapped area of interest for separating individual TMA cores 64 for accurate automated analysis of TMA cores 64. The four corners of the area of interest, which is a smallest rectangle around a TMA core 64 are used to check if these corners fall within another TMA core's area of interest or not.
  • boundaries for the expanded areas of interest are determined to delineate individual TMA cores 64 and allow a medical conclusion to be formulated.
  • An average width, average height of a grid based on a distance between cores is computed.
  • Separated cores are indexed and identified separating individual TMA cores 64.
  • the present invention is not limited to such an embodiment and other embodiments can also be used to practice the invention.
  • a first TMA core 64 is searched for from left to right and top to bottom. Once a first TMA core is detected, other TMA cores on a same row are detected by searching for other cores center within a range of a predetermined incremental distance (e.g., 5 to 8 pixels). The pre-determined incremental distance is an average grid parameter. A check on the row and column size is done using the digital image dimensions.
  • a predetermined incremental distance e.g., 5 to 8 pixels.
  • the pre-determined incremental distance is an average grid parameter.
  • a check on the row and column size is done using the digital image dimensions.
  • the present invention is not limited to this searching method and other methods can also be used to practice the invention,
  • FIG. 7 is a block diagram 72 illustrating plural TMA cores 74 in the contrast adjusted digital image 62 of FIG. 4C after the automatic execution of Step 46.
  • FIG. 7 illustrates boundaries 74 determined for plural TMA cores and is enlarged to 200% to highlight the boundary determination results. Note how boundaries of plural TMA cores have been automatically determined even though the TMA cores 74 were not precisely placed on the original slide, include fragmented portions of TMA cores (e.g., TMA cores numbered 16 and 28) and there are very small or no separations (see FIG. 4A) between some TMA cores 74 (e.g., TMA cores numbered 28 and 29).
  • graphical lines are drawn around individual TMA cores 74 to make them easy to identify.
  • the displayed TMA cores 74 are graphically displayed on a GUI on display 14.
  • the boundaries for the determined TMA cores are used to determine a medical conclusion.
  • the medical conclusion includes a medical diagnosis or medical prognosis for a human cancer.
  • the human cancer includes a human breast cancer, prostrate cancer or other human cancer.
  • FIG. 8 is a block diagram illustrating an exemplary flow of data 76 in the automated biological sample processing system 10.
  • Pixel values from a digital image of a TMA are captured 78 as raw digital images 80.
  • the raw digital images are stored in raw image format in one or more image databases 22.
  • TMA cores in the TMA are analyzed on the digital image and modifications made to the raw digital images 80 are used to create new biological knowledge 82 using the methods described herein.
  • the new biological knowledge is stored in a knowledge database 84.
  • Peer review of the digital image analysis and life science and biotechnology experiment results is completed 86.
  • a reference digital image database 88 facilitates access of reference images from previous records of life science and biotechnology experiments at the time of peer review.
  • Report generation 92 allows configurable fields and layout of the report. New medical knowledge is automatically created and stored in the knowledge database 84.
  • ANN Artificial Neural Networks
  • an ANN based on FIG. 8 is used for training and classifying cells from automated TMA analysis over a pre-determined period of time.
  • the present invention is not limited to such an embodiment and other embodiments can also be used to practice the invention.
  • the invention can be practiced without used of an ANN.
  • the present invention is implemented in software.
  • the invention may be also be implemented in firmware, hardware, or a combination thereof, including software. However, there is no special hardware or software required to use the proposed invention.
  • the method and system described herein provide at least: (1) two or more different levels of automated processing of TMAs, one for an entire TMA and the other at an individual TMA core level.
  • This two level processing approach ensures that variations at a TMA global level as well as a local TMA level are compensated for; (2) use of image statistics to estimate distances between adjacent TMA cores and remove unwanted objects and artifacts and determine boundaries of TMA cores; and (3) methods to locate and refine an extent or boundary of each TMA core automatically including detection and handling of overlapping and spread out TMA cores.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Proteomics, Peptides & Aminoacids (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Organic Chemistry (AREA)
  • Wood Science & Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Pathology (AREA)
  • Analytical Chemistry (AREA)
  • Zoology (AREA)
  • Genetics & Genomics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Immunology (AREA)
  • Hospice & Palliative Care (AREA)
  • Oncology (AREA)
  • Quality & Reliability (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Medical Informatics (AREA)
  • Biophysics (AREA)
  • Biotechnology (AREA)
  • Microbiology (AREA)
  • Molecular Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Biochemistry (AREA)
  • General Engineering & Computer Science (AREA)
  • Investigating Or Analysing Biological Materials (AREA)

Abstract

L'invention concerne un procédé et un système de quantification automatisée d'une analyse d'image numérique d'un jeu ordonné de microéchantillons de tissu (TMA). Le procédé et le système analysent automatiquement une image numérique d'un TMA présentant plusieurs noyaux TMA créés à l'aide d'une aiguille de biopsie ou autres techniques pour créer des sections histologiques normales et placer les noyaux d'aiguille résultants dans le TMA. L'analyse automatisée permet de déterminer automatiquement une conclusion médicale, telle qu'un diagnostic médical ou un pronostic médical (p. ex. d'un cancer humain). Le procédé et le système de l'invention assurent un maillage automatique et fiable du noyau TMA, et une détection automatique des limtes du noyau TMA comprenant la détection de noyaux TMA superposés ou accolés sur un maillage.
PCT/US2005/017481 2004-05-21 2005-05-19 Procede et systeme de quantification automatisee d'une analyse d'image numerique d'un jeu ordonne de microechantillons de tissu (tma) WO2005114578A1 (fr)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US57326204P 2004-05-21 2004-05-21
US60/573,262 2004-05-21
US10/938,314 2004-09-10
US10/938,314 US20050136509A1 (en) 2003-09-10 2004-09-10 Method and system for quantitatively analyzing biological samples
US10/966,071 US20050136549A1 (en) 2003-10-30 2004-10-15 Method and system for automatically determining diagnostic saliency of digital images
US10/966,071 2004-10-15

Publications (1)

Publication Number Publication Date
WO2005114578A1 true WO2005114578A1 (fr) 2005-12-01

Family

ID=34972108

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2005/017481 WO2005114578A1 (fr) 2004-05-21 2005-05-19 Procede et systeme de quantification automatisee d'une analyse d'image numerique d'un jeu ordonne de microechantillons de tissu (tma)

Country Status (1)

Country Link
WO (1) WO2005114578A1 (fr)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8027030B2 (en) 2007-06-15 2011-09-27 Historx, Inc. Method and system for standardizing microscope instruments
US8417015B2 (en) 2007-08-06 2013-04-09 Historx, Inc. Methods and system for validating sample images for quantitative immunoassays
US8655037B2 (en) 2007-05-14 2014-02-18 Historx, Inc. Compartment segregation by pixel characterization using image data clustering
CN104021316A (zh) * 2014-06-27 2014-09-03 中国科学院自动化研究所 基于基因空间融合的矩阵分解对老药预测新适应症的方法
US9240043B2 (en) 2008-09-16 2016-01-19 Novartis Ag Reproducible quantification of biomarker expression
AU2022279500B1 (en) * 2022-12-01 2023-11-23 Provectus IP Pty Ltd Automated cell analyser and method

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001042796A1 (fr) * 1999-12-13 2001-06-14 The Government Of The United States Of America, As Represented By The Secretary, Department Of Health & Human Services, The National Institutes Of Health Technologie de micro-matrice de tissus a haut debit et applications
US20020177149A1 (en) * 2001-04-20 2002-11-28 Rimm David L. Systems and methods for automated analysis of cells and tissues
US20030118222A1 (en) * 2000-11-30 2003-06-26 Foran David J. Systems for analyzing microtissue arrays

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001042796A1 (fr) * 1999-12-13 2001-06-14 The Government Of The United States Of America, As Represented By The Secretary, Department Of Health & Human Services, The National Institutes Of Health Technologie de micro-matrice de tissus a haut debit et applications
US20030118222A1 (en) * 2000-11-30 2003-06-26 Foran David J. Systems for analyzing microtissue arrays
US20020177149A1 (en) * 2001-04-20 2002-11-28 Rimm David L. Systems and methods for automated analysis of cells and tissues

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
BHANDARKAR SUCHENDRA M ET AL: "Automated analysis of DNA hybridization images for high-throughput genomics", MACH VISION APPL; MACHINE VISION AND APPLICATIONS JULY 2004, vol. 15, no. 3, February 2004 (2004-02-01), pages 121 - 138, XP002344161 *
BLEKAS K ET AL: "An unsupervised artifact correction approach for the analysis of dna microarray images", PROCEEDINGS 2003 INTERNATIONAL CONFERENCE ON IMAGE PROCESSING. ICIP-2003. BARCELONA, SPAIN, SEPT. 14 - 17, 2003, INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, NEW YORK, NY : IEEE, US, vol. VOL. 2 OF 3, 14 September 2003 (2003-09-14), pages 165 - 168, XP010670547, ISBN: 0-7803-7750-8 *
BRANDLE NORBERT ET AL: "Robust DNA microarray image analysis", MACH VISION APPL; MACHINE VISION AND APPLICATIONS OCTOBER 2003, vol. 15, no. 1, October 2003 (2003-10-01), pages 11 - 28, XP002344162 *
JAIN AJAY N ET AL: "Fully automatic quantification of microarray image data", GENOME RESEARCH, vol. 12, no. 2, February 2002 (2002-02-01), pages 325 - 332, XP002344159, ISSN: 1088-9051 *
STEINFATH MATTHIAS ET AL: "Automated image analysis for array hybridization experiments", BIOINFORMATICS (OXFORD), vol. 17, no. 7, July 2001 (2001-07-01), pages 634 - 641, XP002344160, ISSN: 1367-4803 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8655037B2 (en) 2007-05-14 2014-02-18 Historx, Inc. Compartment segregation by pixel characterization using image data clustering
US8027030B2 (en) 2007-06-15 2011-09-27 Historx, Inc. Method and system for standardizing microscope instruments
US8120768B2 (en) 2007-06-15 2012-02-21 Historx, Inc. Method and system for standardizing microscope instruments
US8417015B2 (en) 2007-08-06 2013-04-09 Historx, Inc. Methods and system for validating sample images for quantitative immunoassays
US9240043B2 (en) 2008-09-16 2016-01-19 Novartis Ag Reproducible quantification of biomarker expression
CN104021316A (zh) * 2014-06-27 2014-09-03 中国科学院自动化研究所 基于基因空间融合的矩阵分解对老药预测新适应症的方法
CN104021316B (zh) * 2014-06-27 2017-04-05 中国科学院自动化研究所 基于基因空间融合的矩阵分解对老药预测新适应症的方法
AU2022279500B1 (en) * 2022-12-01 2023-11-23 Provectus IP Pty Ltd Automated cell analyser and method

Similar Documents

Publication Publication Date Title
US8428887B2 (en) Method for automated processing of digital images of tissue micro-arrays (TMA)
JP6816196B2 (ja) 包括的なマルチアッセイ組織分析のためのシステムおよび方法
US10275880B2 (en) Image processing method and system for analyzing a multi-channel image obtained from a biological tissue sample being stained by multiple stains
US7760927B2 (en) Method and system for digital image based tissue independent simultaneous nucleus cytoplasm and membrane quantitation
US8515683B2 (en) Method and system for automated detection of immunohistochemical (IHC) patterns
EP3345161B1 (fr) Systèmes de traitement d'images, et procédés d'affichage de plusieurs images d'un échantillon biologique
Kothari et al. Pathology imaging informatics for quantitative analysis of whole-slide images
US20050265588A1 (en) Method and system for digital image based flourescent in situ hybridization (FISH) analysis
US20050136549A1 (en) Method and system for automatically determining diagnostic saliency of digital images
JP2023030033A (ja) デジタル病理学分析結果の格納および読み出し方法
WO2007024264A2 (fr) Procede et systeme de quantification simultanee, a base d'images numeriques et independante du tissu du noyau, du cytoplasme et de la membrane
WO2005114578A1 (fr) Procede et systeme de quantification automatisee d'une analyse d'image numerique d'un jeu ordonne de microechantillons de tissu (tma)
EP4260324A1 (fr) Systèmes et procédés de génération d'ensembles de données d'entraînement d'image d'histologie de modèles d'apprentissage machine
WO2005076216A2 (fr) Procede et systeme pour l'analyse automatique d'hybridation in situ par fluorescence a base d'image numerique
WO2005096225A1 (fr) Procede et systeme de detection automatises de profils immunohistochimiques (ihc)
WO2000072258A2 (fr) Procede et systeme pour analyser de maniere polyvalente des donnees experimentales

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KM KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NG NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

WWW Wipo information: withdrawn in national office

Country of ref document: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 69(1) EPC, EPO FORM 1205A, RESENT ON 30/03/07.

122 Ep: pct application non-entry in european phase

Ref document number: 05751018

Country of ref document: EP

Kind code of ref document: A1