EP1138019A2 - Procede et systeme pour analyser de maniere polyvalente des donnees experimentales - Google Patents

Procede et systeme pour analyser de maniere polyvalente des donnees experimentales

Info

Publication number
EP1138019A2
EP1138019A2 EP00937712A EP00937712A EP1138019A2 EP 1138019 A2 EP1138019 A2 EP 1138019A2 EP 00937712 A EP00937712 A EP 00937712A EP 00937712 A EP00937712 A EP 00937712A EP 1138019 A2 EP1138019 A2 EP 1138019A2
Authority
EP
European Patent Office
Prior art keywords
assay
features
intensity
images
aggregate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP00937712A
Other languages
German (de)
English (en)
Inventor
Albert H. Gough
Oleg P. Lapets
Gary Bright
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Cellomics Inc
Original Assignee
Cellomics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Cellomics Inc filed Critical Cellomics Inc
Publication of EP1138019A2 publication Critical patent/EP1138019A2/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro

Definitions

  • This invention relates to analyzing experimental data. More specifically, it relates to methods and system for general purpose analysis of images from experimental data collected with automated feature-rich, high-throughput experimental data collection systems.
  • feature-rich data includes data wherein one or more individual features of an object of interest (e.g., a cell) can be collected.
  • object of interest e.g., a cell
  • Identification, selection, and validation of targets for the screening of new drug compounds is often completed at a nucleotide level using sequences of Deoxyribonucleic Acid ("DNA”), Ribonucleic Acid (“RNA”) or other nucleotides.
  • DNA Deoxyribonucleic Acid
  • RNA Ribonucleic Acid
  • Genes are regions of DNA
  • proteins are the products of genes. The existence and concentration of protein molecules typically helps determine if a gene is “expressed” or “repressed” in a given situation.
  • Responses to natural and artificial compounds as indicated by changes in gene expression are typically used to improve existing drugs, and develop new drugs. Changes in binding between proteins are also used to screen compounds for biological activity. However, it is often more appropriate to determine the effect of a new compound on a cellular level instead of a nucleotide or protein level.
  • Cells are the basic units of life and integrate information from DNA, RNA, proteins, metabolites, ions and other cellular components. New compounds that may look promising at a nucleotide or protein level may be toxic at a cellular or organism level. Florescence-based reagents can be applied to cells to determine ion concentrations, membrane potentials, enzyme activities, gene expression, as well as the presence of metabolites, proteins, lipids, carbohydrates, and other cellular components. There are two types of cell screening methods that are typically used: (1) fixed cell screening; and (2) live cell screening. For fixed cell screening, initially living cells are treated with experimental compounds being tested. After application of a desired compound the cells are incubated for a given time and then "fixed" to preserve a final cell state for later analysis.
  • Live cell screening usually requires environmental control of the cells (e.g., temperature, humidity, gases, etc.) since before, during and after application of a desired compound, the cells are kept in a controlled environment until data collection is complete.
  • Fixed cell assays allow spatial measurements to be acquired, but only at one point in time. Live cell assays allow both spatial and temporal measurements to be acquired.
  • a "cell assay” is a specific implementation of image processing methods used to analyze images of cells and return results related to the biological processes being examined.
  • a "cell protocol" specifies a series of system settings including a type of analysis instrument, a cell assay, dyes used to measure biological markers in cells, cell identification parameters and other general image processing parameters used to collect cell data.
  • the spatial and temporal frequency of chemical and molecular information present within cells makes it possible to extract feature-rich cell information from populations of cells. For example, multiple molecular and biochemical interactions, cell kinetics, changes in sub-cellular distributions, changes in cellular morphology, changes in individual cell subtypes in mixed populations, changes and sub-cellular molecular activity, changes in cell communication, and other types of cell information can be acquired.
  • Photographic images are typically collected using a digital camera, but can also be generated by scanning systems such as confocal light microscope systems.
  • a single photographic image may take up as much as 512 Kilobytes ("KB") or more of storage space as is explained below. Collecting and storing a large number of photographic images ads to the data problems encountered when using high throughput systems.
  • KB Kilobytes
  • Such automated feature-rich cell screening systems and other systems known in the art typically include microplate scanning hardware, fluorescence excitation of cells, fluorescence emission optics, a microscope with a camera, data collection, data storage and data display capabilities.
  • feature-rich cell screening see "High content fluorescence-based screening,” by Kenneth A. Guiliano, et al., Journal of Biomolecular Screening, Vol. 2, No. 4, pp. 249-259, Winter 1997, ISSN 1087-0571, "PTH receptor internalization,” Bruce R. Conway, et al., Journal of Biomolecular Screening, Vol. 4, No. 2, pp.
  • An automated feature-rich cell screening system typically automatically scans a microplate with multiple wells and acquires multi-color fluorescence data of cells at one or more instances of time at a pre-determined spatial resolution.
  • Automated feature-rich cell screening systems typically support multiple channels of fluorescence to collect multi-color fluorescence data and may also provide the ability to collect cell feature information on a cell-by-cell basis including such features as the brightness, size and shape of cells and sub-cellar measurements of organelles within a cell.
  • the collection of data from high throughput screening systems typically produces a very large quantity of data and presents a number of bioinformatics problems.
  • bioinformatic techniques are used to address problems related to the collection, processing, storage, retrieval and analysis of biological information including cellular information. Bioinformatics is defined as the systematic development and application of information technologies and data processing techniques for collecting, analyzing and displaying data acquired by experiments, modeling, database searching, and instrumentation to make observations about biological processes.
  • the need for efficient data management is not limited to feature-rich cell screening systems or to cell based arrays.
  • Virtually any instrument that runs High Throughput Screening ("HTS") assays also generate large amounts of data.
  • HTS High Throughput Screening
  • a "bio-chip” is a stratum with hundreds or thousands of absorbent micro-wells on its surface.
  • a micro-well includes a specific point of attachment that may or may not have any depth.
  • a single bio-chip may contain 10,000 or more micro-gels.
  • each micro-well on a bio-chip is like a micro-test tube or a well in a microplate.
  • a bio-chip provides a medium for analyzing known and unknown biological (e.g., nucleotides, cells, etc.) samples in an automated, high-throughput screening system.
  • Collecting feature-rich cell data from a microplate plate used for feature-rich screening typically includes 96 to 1536 individual wells.
  • a microplate is a flat, shallow dish that stores multiple samples for analysis.
  • a "well” is a small area in a microplate used to contain an individual sample for analysis.
  • Each well may be divided into multiple fields.
  • a "field” is a sub-region of a well that represents a field of vision (i.e., a zoom level) for a photographic microscope.
  • Each well is typically divided into one to sixteen fields, or more
  • Each field typically will have between one and six photographic images taken of it, each using a different light filter to capture a different wavelength of light for a different fluorescence response for desired cell components.
  • a pre- determined number of cells are selected to analyze. The number of cells will vary (e.g., between one and one hundred or more). For each cell, multiple cell features are
  • the cell features may include features such as size, shape, brightness,
  • a biologist may desire to develop two or more different cell assays run at the same time to focus on different cell information. For example, for a first cell assay it may be necessary to collect cell feature data including cell shape, cell size and cell diameter data for a desired experiment by analyzing cell image data. For a second cell assay, it may be desirable to collect skewness and kurtosis for a desired cell feature by analyzing cell image data.
  • analysis tools known in the art do not allow a biologist to select his/her own image processing techniques to create a cell assay outside of a fixed list of image processing techniques available with the analysis tool. That is, a biologist may desire to analyze skewness and kurtosis, but his/her analysis tool may only provide image processing techniques for analyzing cell shape, and cell size.
  • Another problem is that even if image processing packages known in the art are used, a biologist or other scientist, has to select not only image processing routines to accomplish an assay feature measurement, but also choose from a large number of image processing options for the image processing routines. This may create additional confusion or frustration on the part of the biologist as the biologist may not know what image processing options are the most appropriate for a give assay feature.
  • the general purpose tool should provide image processing techniques for a cell assay created by a biologist, without requiring the biologist, other scientist or analyst have any in-depth knowledge of image processing techniques.
  • One aspect of the invention includes a method for presenting assay features associated with a pre-determined set of image processing routines for analyzing experimental data including images.
  • the pre-determined set of image processing routines includes only a limited set of options available for processing an image.
  • Another aspect of the invention includes a method for analyzing experimental data including images using a set of selected assay features selected from a set of predetermined assay features to help analyze image data.
  • the set of selected assay features are processed in a pre-determined order appropriate for analysis of image data.
  • a pre-determined set of general assay features is presented.
  • An assay feature includes one or more measurements for an object in a digital photographic image acquired from the experimental data.
  • the set of general assay features includes object features, aggregate features and general purpose image processing features.
  • a set of desired assay features is selected from the pre-determined set of general assay features.
  • a set of images is processed using the desired assay features from the selected set of general assay features.
  • Such general assay features e.g., length, width, height, etc.
  • the general assay features presented typically include only a few of the many possible image processing options that could be used to take such measurements from a digital image, thereby helping to reduce confusion associated selecting such image processing options.
  • the methods and system may help provide a general purpose assay development tool.
  • the methods and system may allow a biologist, other scientist or lab technician not trained in image processing techniques to quickly and easily design protocols and assays to analyze images acquired from experimental data (e.g., cells).
  • the methods and system may improve the identification, selection, validation and screening of new experimental compounds (e.g., drug compounds).
  • the methods and system may also be used to provide new bioinformatic techniques used to make observations about experimental data including multiple digital photographic images.
  • FIG. 1 A is a block diagram illustrating an exemplary experimental data storage system
  • FIG. IB is a block diagram illustrating an exemplary experimental data storage system
  • FIG. 2 is a block diagram illustrating an exemplary array scan module
  • FIG. 3 is a flow diagram illustrating a method for selecting assay features for experimental data.
  • FIG. 4 is a flow diagram illustrating a method for selecting assay features for images acquired from experimental data
  • FIG. 5 is a block diagram illustrating an exemplary graphical user interface for selecting object features
  • FIG. 6 is a block diagram illustrating an exemplary graphical user interface for selecting general image processing operations
  • FIG. 7 is a block diagram illustrating a screen display for graphically displaying images processed using a desired set of assay features.
  • FIG. 1A illustrates an exemplary data storage system 10 for preferred embodiments of the present invention.
  • the exemplary data storage system 10 includes an analysis instrument 12, connected to a client computer 18, a shared database 24 and a data store archive 30 with a computer network 40.
  • the analysis instrument 12 includes any scanning instrument capable of collecting feature-rich experimental data, such as nucleotide, protein, cell or other experimental data, or any analysis instrument capable of analyzing feature-rich experimental data.
  • feature-rich data includes data wherein one or more individual features of an object of interest (e.g., a cell) can be collected.
  • the client computer 18 is any conventional computer including a display application that is used to lead a scientist or lab technician through data analysis.
  • the shared database 24 is a multi-user, multi-view relational database that stores data from the analysis instrument 12.
  • the data archive 30 is used to provide virtually unlimited amounts of "virtual" disk space with a multi-layer hierarchical storage management system.
  • the computer network 40 is any fast Local Area Network ("LAN") (e.g., capable of data rates of 100 Mega-bit per second or faster).
  • LAN Local Area Network
  • Data storage system 10 can be used for virtually any system capable of collecting and/or analyzing feature-rich experimental data from biological and non-biological experiments.
  • FIG. IB illustrates an exemplary data storage system 10 ' for one preferred embodiment of the present invention with specific components.
  • the data storage system 10 ' includes one or more analysis instruments 12, 14, 16, for collecting and/or analyzing feature-rich experimental data, one or more data client computers, 18, 20, 22, a shared database 24, a data store server 26, and a shared database file server 28.
  • a data store archive 30 includes any of a disk archive 32, an optical jukebox 34 or a tape drive 36.
  • the data store archive 30 can be used to provide virtually unlimited amounts of "virtual" disk space with a multi-layer hierarchical storage management system without changing the design of any databases used to stored collected experimental data as is explained below.
  • the data store archive 30 can be managed by an optional data archive server 38.
  • Data storage system 10' components are connected by a computer network 40. However, more or fewer data store components can also be used and the present invention is not limited to the data storage system 10' components illustrated in FIG. IB. In one exemplary preferred embodiment of the present invention, data storage system 10' includes the following specific components. However, the present invention is not limited to these specific components and other similar or equivalent components may also be used.
  • Analysis instruments 12, 14, 16, comprise a feature- rich array scanning system capable of collecting and/or analyzing experimental data such as cell experimental data from microplates, DNA arrays or other chip-based or bio-chip based arrays.
  • Bio-chips include any of those provided by Motorola Corporation of Schaumburg, Illinois, Packard Instrument, a subsidiary of Packard BioScience Co. of Meriden, Connecticut, Genometrix, Inc. of Woodlands, Texas, and others.
  • Analysis instruments 12, 14, 16 include any of those provided by Cellomics,
  • the one or more data client computers, 18, 20, 22, are conventional personal computers that include a display application that provides a Graphical User Interface ("GUI") to a local hard disk, the shared database 24, the data store server 26 and/or the data store archive 30.
  • GUI Graphical User Interface
  • the GUI display application is used to lead a scientist or lab technician through standard analyses, and supports custom and query viewing capabilities.
  • the display application GUI also supports data exported into standard desktop tools such as spreadsheets, graphics packages, and word processors.
  • the data client computers 18, 20, 22 connect to the store server 26 through an Open Data Base Connectivity ("ODBC") connection over network 40.
  • ODBC Open Data Base Connectivity
  • computer network 40 is a 100 Mega-bit (“Mbit”) per second or faster Ethernet, Local Area Network (“LAN”).
  • Mbit Mega-bit
  • LAN Local Area Network
  • other types of LANs could also be used (e.g., optical or coaxial cable networks).
  • the present invention is not limited to these specific components and other similar components may also be used.
  • OBDC is an interface providing a common language for applications to gain access to databases on a computer network.
  • the store server 26 controls the storage based routines plus an underlying Database Management System ("DBMS").
  • DBMS Database Management System
  • the shared database 24 is a multi-user, multi-view relational database that stores summary data from the one or more analysis instruments 12, 14, 16.
  • the shared database 24 uses standard relational database tools and structures.
  • the data store archive 30 is a library of image and feature database files.
  • the data store archive 30 uses Hierarchical Storage Management ("HSM”) techniques to automatically manage disk space of analysis instruments 12, 14, 16 and the provide a multi-layer hierarchical storage management system.
  • HSM Hierarchical Storage Management
  • 10 ' for preferred embodiments of the present invention include a processing system
  • CPU Central Processing Unit
  • acts and symbolically represented operations or instructions include the manipulation of electrical signals by the CPU.
  • An electrical system represents data bits which cause a resulting transformation or reduction of the electrical signals, and the maintenance of data bits at memory locations in a memory system to thereby reconfigure or otherwise alter the CPU's operation, as well as other processing of signals.
  • the memory locations where data bits are maintained are physical locations that have particular electrical, magnetic, optical, or organic
  • the data bits may also be maintained on a computer readable medium
  • RAM Random Access Memory
  • non-volatile e.g., Read-Only Memory
  • ROM read only memory
  • FIG. 2 is a block diagram illustrating an exemplary array scan module 42 architecture.
  • the array scan module 42 such as one associated with analysis instrument 12, 14, 16 (FIG. IB) includes software/hardware that is divided into four functional groups or modules. However, more of fewer functional modules can also be used and the present invention is not limited to four functional modules.
  • the Acquisition Module 44 controls a robotic microscope and digital camera, acquires images and sends the images to the Assay Module 46.
  • the Assay Module 46 "reads" the images, creates graphic overlays, interprets the images collects feature data and returns the new images and feature data extracted from the images back to the Acquisition Module 44.
  • the Acquisition Module 44 passes the image and interpreted feature data to the Data Base Storage Module 48.
  • the Data Base Storage Module 48 saves the image and feature information in a combination of image files and relational database records.
  • the client computers 18, 20, 22 use the Data Base Storage Module 48 to access feature data and images for presentation and data analysis by the Presentation Module 50.
  • the Presentation Module 50 includes a display application with a GUI as was discussed above.
  • FIG. 3 is a flow diagram illustrating a Method 52 for selecting assay features for experimental data.
  • multiple pre-determined assay features for analyzing images acquired from experimental data are presented.
  • An assay feature includes one or more measurements for an object in an image acquired from the experimental data.
  • a set of desired assay features selected from the multiple presented assay features are received.
  • one or more image processing routines from a library of image processing routines are selected for an assay feature from the set of desired assay features. The one or more image processing routines are used to accomplish the selected assay feature.
  • the one or more image processing routines are associated with the assay feature.
  • a loop is entered to repeat steps 58 and 60 for assay features in the set of selected assay features.
  • Method 52 is illustrated with one specific embodiment of the present invention. However, the present invention is not limited to such an embodiment and other embodiments can also be used.
  • multiple pre-determined assay features for analyzing digital photographic images (hereinafter "images") acquired from experimental data for an assay are presented by analysis instruments 12, 14, 16 (FIG. IB) or by client computers 18, 20, 22 (FIG. IB).
  • the multiple pre-determined assay features include object features (See, e.g., FIG. 5).
  • An "object” feature operates on an individual object (e.g., a cell) or an object component (e.g., cell membrane, cell nucleus, etc.)
  • the multiple pre-determined assay features include object features and aggregate features.
  • An “aggregate” feature includes assay features that operate on multiple objects (e.g., number of objects, average value of a feature, standard deviation value of a feature, etc.).
  • the multiple pre-determined assay features include only aggregate features.
  • the multiple predetermined assay features presented at Step 54 include general assay features that can be used by virtually any biologist, other scientist or analyst to analyze measurements from objects (e.g., cells) in images collected from experimental data.
  • Such general assay features e.g., length, width, height, etc.
  • the general assay features presented typically include only a few of the many possible image processing options that could be used to take measurements from a digital image.
  • an assay feature for a simple measurement such as determining an object's length
  • may include multiple different types of image processing thresholds e.g., a number of pixels, types of pixels, type of object components in around a desired object, etc. to be included for the object to determine its length.
  • two image processing thresholds e.g., a minimum and a maximum
  • Other image processing thresholds are handled internally without presenting such information to a user.
  • the general assay features and limited image processing options for the general assay features presented allow a biologist, other scientist or analyst without much image processing experience to easily and quickly create assays and protocols. Since general assay features and limited image processing options are presented, instead of specific assay features with many different options, a user with limited image processing experience is less likely to get confused when he/she is creating an assay or protocol.
  • the general assay features associated with image processing options are presented in a specific ordering.
  • the present invention is not limited to such an embodiment with such a specific ordering.
  • This specific ordering may also help a user with limited knowledge of image processing select the appropriate options for a desired assay or protocol.
  • an assay will include two or more channels.
  • a "channel” is a specific configuration of optical filters and channel specific parameters that are used to acquire an image.
  • different fluorescent dyes are used to label different cell structures. The fluorescent dyes emit light at different wavelengths. Channels are used to acquire photographic images for different dye emission wavelengths.
  • the first phase is typically called “image segmentation” or “object isolation,” in which a desired object is isolated from the rest of the image.
  • the second phase is typically called “feature extraction,” wherein measurements of the objects are calculated.
  • a feature is typically a function of one or more measurements, calculated so that it quantifies a significant characteristic of an object. Typical object measurements include size, shape, intensity, texture, location, and others.
  • the "size” of an object can be represented by its area, perimeter, boundary definition, length, width, etc.
  • the "shape" of an object can be represented by its rectangularity (e.g., length and width aspect ratio), circularity (e.g., perimeter squared divided by area, bounding box, etc.), moment of inertia, differential chain code, Fourier descriptors, etc.
  • the "intensity” of an object can be represented by a summed average, maximum or minimum grey levels of pixels in an object, etc.
  • Texture of an object quantifies a characteristic of grey-level variation within an object and can be represented by statistical features including standard deviation, variance, skewness, kurtosis and by spectral and structural features, etc.
  • the "location" of an object can be represented by an object's center of mass, horizontal and vertical extents, etc. with respect to a pre-determined grid system.
  • Method 52 is used to analyze cell image data and cell feature data from "wells" in a "microplate.” In another preferred embodiment of the present invention, Method 52 is used to analyze cell image and cell feature data from micro-gels in a bio-chip.
  • a "microplate” is a flat, shallow dish that stores multiple samples for analysis and typically includes 96 to 1536 individual wells.
  • a "well” is a small area in a microplate used to contain an individual sample for analysis.
  • Each well may be divided into multiple fields.
  • a "field" is a sub-region of a well that represents a field of vision (i.e., a zoom level) for a photographic microscope.
  • Each well is typically divided into one to sixteen fields, or more.
  • Each field typically will have between one and six photographic images taken of it, each using a different light filter to capture a different wavelength of light for a different fluorescence response for desired cell components.
  • the present invention is not limited to such an embodiment, and other containers (e.g., varieties of biological chips, such as DNA chips, micro-arrays, and other containers with multiple sub- containers), sub-containers can also be used to collect image data and feature data from other than cells.
  • Step 54 includes presenting a set of static assay features in a uniform manner on a graphical user interface for every user. In such an embodiment, the set of static assay features cannot be modified by a user.
  • Step 54 is optionally split into two sub-steps. In a first sub-step, a user first selects a desired set of assay feature names from a list of assay features. In a second sub-step the desired set of assay feature names is dynamically presented on graphical user interface specifically for the user. In such an embodiment, a user can dynamically modify the set of assay features that will actually be presented and used instead of receiving a set of static assay features that cannot be modified by a user.
  • an assay feature includes one or more measurements for an object in an image acquired from experimental data.
  • objects in the images acquired from experimental data include, but are not limited to, cells.
  • Exemplary object features for cells are illustrated in Table 1. However, other object features and can also be used and the present invention is not limited to the cell features illustrated in Table 1. Virtually any object feature can be presented at Step 54.
  • Step 54 also includes presenting aggregate features.
  • Aggregate features are features associated with a collection of objects such as a population of cells.
  • the aggregate features include, but are not limited to, any of the well summary data for a microplate including cells illustrated in Table 2.
  • the present invention is not limited to presenting aggregate features for the well summary data illustrated in Table 2. Virtually any summary data for aggregate features can be presented.
  • a "SPOT" indicates a small region of fluorescent response intensity as a measure of biological activity.
  • the aggregate features can also include, but are not limited to, microplate summary data for cells illustrated in Table 3.
  • MEAN indicates a statistical mean
  • STDEV indicates a statistical standard deviation, known in the art
  • SPOT indicates a small region of fluorescent response intensity as a measure of biological activity.
  • a set of assay features selected from the presented assay features are received on the analysis instruments 12, 14, 16 or client computers 18, 20, 22.
  • set of assay features selected from the multiple presented assay features may include object features for "cell perimeter,” “cell width” and “cell length.” (e.g., from Table 1).
  • one or more image processing routines from a library of image processing routines are selected for an assay feature from the set of selected assay features.
  • the one or more image processing routines are used to accomplish the selected assay feature.
  • one or more image processing routines are called from a library of image processing routines to accomplish the "cell length” feature.
  • image processing routines including "select_object( ),” "object boundingbox ( ),” “object rotatel ⁇ O ( ),” and “object_longest_side ( )” (e.g., see length feature in Table 6) may be selected from a library of image processing.
  • the one or more image processing routines are associated with the selected feature.
  • the "cell length” feature is associated with the image processing routines “select_object( ),” “object_boundingbox ( ),” “object_rotatel80 ( ),” and “object longest side ( )” (e.g., see length feature in Table 6).
  • Step 62 a loop is entered to repeat steps 58 and 60 for assay features in the selected set of assay features. For example, after the cell length feature is associated with the image processing routines, the cell width and cell perimeter features are also associated with image processing routines by repeating steps 58 and 60.
  • Method 52 allows a biologist, other scientist or analyst not trained in image processing to create assays and protocols to analyze experimental data.
  • Method 52 can be used to analyze images collected from feature-rich cell experimental data generated by HTS systems.
  • FIG. 4 is a flow diagram illustrating a Method 64 for selecting assay features for images acquired from experimental data.
  • a set of images is acquired from experimental data on an analysis device.
  • a set of assay features is selected from a set of multiple presented assay features to analyze the set of images.
  • An assay feature includes one or more measurements for an object in an image acquired from the experimental data.
  • a presented assay feature is associated with one or more image processing routines from a library of image processing routines to accomplish the assay feature.
  • processing of the set of images using the selected set of assay features is requested.
  • results are received from the processing of the set of images using the selected set of assay features.
  • Method 64 is illustrated with one specific embodiment of the present invention. However, the present invention is not limited to such an embodiment and other embodiments can also be used.
  • a set of images (e.g., for cells or components of cells acquired from cell experimental data) is acquired on analysis instruments 12, 14, 16 or client computers 18, 20, 22 (e.g., FIGS. 1 A and IB).
  • Images are acquired automatically from a feature rich array scanning system
  • Images are acquired from stored images sets after a desired experiment has been run by a feature rich array scanning system and the results have been saved in a shared database 24 or a store archive 30, or local hard drive.
  • FIG. 5 is a block diagram illustrating an exemplary graphical user interface 74 presented on the analysis instruments 12, 14, 16 or client computers 18, 20, 22 for selecting object features at Step 68.
  • the graphical user interface 74 includes graphical entities such as graphical check boxes or graphical buttons to select object features.
  • FIG. 5 illustrates, for example, graphical check boxes to select object features including size, shape, intensity, texture, location, area, perimeter, shape factor, equivalent diameter, length, width, integrated fluorescence intensity, mean fluorescence intensity, variance, skewness, kurtosis, minimum fluorescence intensity, maximum fluorescence intensity, geometric center, x-coordinate of a geometric center or y-coordinate of a geometric center.
  • FIG. 5 illustrates a set including some of the
  • FIG. 5 also illustrates graphical user interface
  • Step 68 a set of assay features is selected from
  • Step 68 includes creating a protocol for an assay by selecting multiple pre-determined assay features (e.g., selecting multiple graphical buttons from FIG. 5).
  • a "protocol” specifies a series of system settings including a type of analysis instrument, an assay, dyes used to measure biological markers, cell identification parameters and other general image processing parameters used to collect data.
  • An "assay” is a specific selection of image processing methods used to analyze images and return results related to biological processes being examined. For more information on the image processing methods used in cell assays targeted to specific biological processes, see co-pending applications 09/031,217 and 09/352,171,
  • FIG. 5 illustrates selection of
  • Radio button for DYE-0 84 is illustrated as
  • assay-X would include obtaining object measurements for
  • the assay features presented at Step 68 are associated with one or more image processing routines from a library of image processing routines to accomplish the assay feature measurement (e.g., at Step 60 of Method 52, FIG. 3).
  • a user selecting the assay features presented at Step 68 does not have to understand how the assay feature is accomplished, but only how to choose desired assay features of interest to accomplish his/her own desired analysis (e.g., for a desired assay). If a new library of image processing routines was used, the assay features presented at Step 68 typically would not change, even though a whole new set of image processing routines might be used to accomplish an assay feature measurement.
  • Step 70 processing of the set of images using the selected set of assay features is requested.
  • Step 70 includes selecting a series of general image processing operations in addition to selecting object and/or aggregate features.
  • the image processing operations are applied before receiving the results at Step 78.
  • the image processing operations may include filtering, object segmentation or mask modification (See, FIG. 6).
  • processing of the set of images at Step 70 includes applying general image processing routines to an image acquired from experimental data in a pre-determined order using a set of desired assay features selected from a graphical user interface (e.g., FIG. 6).
  • a graphical user interface e.g., FIG. 6
  • pre-determining the order of applying the general image processing routines relieves a user of another image processing detail when he/she is creating an assay or protocol.
  • Assay features are presented on a graphical user interface (e.g., FIG. 6) in the order that they are processed. For example, before segmenting an image, it is usually important to filter the image to improve the efficiency of the segmentation. The filters may smooth and sharpen an image.
  • Providing a pre-determined order helps make the creation of an assay or protocol simpler than if a user had to also determine a processing order himself/herself.
  • the pre-determined processing order may also help a user more easily compare his/her results between or among several different experiments.
  • processing the set of images at Step 70 with selected object and aggregate features may include both independent and dependent processing of fluorescence channels.
  • Independent processing refers to the creation of "independent masks” for each of the fluorescence channels.
  • a “mask” is one or more binary values used to selectively screen out or let through certain bits in a data value. Masking is typically performed by using a logical operator (AND, OR, XOR, NOT) to combine the mask and the data value.
  • Dependent processing refers to the use of a mask from one channel to derive a mask for analysis in another channel.
  • This "derived mask” may be a simple copy of the parent mask or further processing may be applied to the parent mask.
  • Feature extraction in the second channel occurs based on the derived mask.
  • an approach to analyzing the cytoplasm-to-nucleus translocation of a transcription factor in a cell can be performed using derived masks.
  • labeled nuclei are used to establish a mask.
  • a Transcription Factor (“TF") channel is setup to use a derived mask.
  • the TF channel is defined as dependent on the nucleus channel. This copies the nuclei mask to the TF channel.
  • the mask can be applied directly to measure a mean nuclear intensity of the TF, which is proportional to the amount of TF in the nucleus.
  • the mask is dilated a number of times and the binary exclusive OR/XOR function applied to the pair of masks.
  • images from selected fluorescent channels are typically processed through a series of general image processing operations before analysis. Such general image processing steps are used to remove noise and help improve feature interpretation.
  • the general image processing steps may include filtering, segmentation, etc. as is discussed below. Table 4 illustrates independent general image processing operations.
  • Filtering The ability to perform smoothing, noise reduction, or local contrast adjustment such as edge enhancement processing on the images as a preliminary step to segmentation, depending on the image quality and the task
  • Sharpening The sharpening method is based on a common, high pass 3 X 3 kernel Segmentation - Segmentation allows separation of an image into separate objects
  • Threshold (Fixed) - A single user specified threshold can be used for images with very stable backgrounds and relatively good SNR This is an alternative to the Separate Grey operation
  • the output of this method is a binary mask • Threshold (Auto) - A histogram-based method where the minimum intensity between two peaks can be determined automatically and then optionally corrected before applying
  • the output of this method is a binary image
  • Threshold - Threshold is setup interactively via a slider or by typing in a threshold value
  • the threshold value will be applied throughout the scan
  • the auto threshold is computed for the current image and the correction coefficient is determined to make it match the one set manually This coefficient will be applied to every threshold value determined during the scan
  • Mask Modification - Masks from the segmentation process may be modified by multiple cycles of erosion and dilation This is useful for smoothing the outlines of the masks as well as creating masks that may be impractical from just the segmentation methods
  • the sequence of erode and dilate, or dilate and erode, helps to remove noise from a mask outline
  • Erode - Masks may be reduced in size by binary erosion for any number of cycles Each erosion is a reduction in the size of the mask by removing perimeter pixels
  • Dilate -Masks may be expanded in size by binary dilation for any number of cycles Each dilation ads an additional outline of 1 pixel in width
  • Table 4 illustrates general image processing operations that are useful to apply to a dependent mask
  • other image processing operations can be used and the present invention is not limited to the image processing operations illustrated in Table 5.
  • Erode - Masks may be reduced in size by binary erosion for any number of cycles.
  • Dilate- Masks may be expanded in size by binary dilation for any number of cycles
  • • XOR- Masks can be combined by application of the exclusive OR binary operation Thus creating a ring around an original nuclear mask The ring can be expanded or contracted relative to the original nuclear mask while the width of the ring stays unchanged.
  • FIG. 6 is a block diagram illustrating an exemplary graphical user interface 86 for selecting general image processing operations. These operations, illustrated in Tables 4 and 5, are selected by inputting a number in the graphical box displayed, or by checking a graphical check box. If a graphical box has a value of zero, or a graphical check box is not checked, the general image processing operation is not executed. For example, as is illustrated in FIG. 6, no filtering is requested. However, grey scale segmentation 88 is selected, a value of 50 is used for the grey scale threshold 90. In addition, an independent mask is selected for dilating the mask for 2 cycles 92, and the XOR operation 94 is selected for a dependent mask.
  • processing at Step 70 includes obtaining measurements for selected object and aggregate features.
  • Table 6 illustrates one possible implementation of the object features from Table 1 using the independent masks from operations in Table 4.
  • the present invention is not limited to this implementation and other implementations can also be used.
  • dependent masks are more limiting than the set for independent masks.
  • One reason for this is that dependent masks are not necessarily related to a form of a signal in a dependent channel.
  • a perimeter or shape of a derived mask is typically more related to a primary channel rather than the dependent channel.
  • Table 7 illustrates one implementation of object features for dependent masks created using the aggregate operations from Table 5.
  • a primary mask is applied and desired object features are extracted, a derived mask is applied and aggregate features are extracted.
  • object features represent cell data and aggregate features represent the well- level or microplate level data for a population of cells in a well.
  • aggregate features for other types experimental data can also be used.
  • object and aggregate features are calculated and constrained by settings of aggregate "feature gates.”
  • Feature gates are provided to define sub-set of an object population that will contribute to an object or aggregate feature set.
  • the feature gates include selection of a range including a lower and upper limit on the range. For example a feature gate for the object feature area may be set with a lower limit of zero and an upper limit of 2000. Thus, only objects (e.g., cells) that have an area between zero and 2000 pixels will be included.
  • results are received from the processing of the set of images using the selected set of assay features.
  • the results are written to a local database associated with the analysis instruments 12, 14, 16 or client computers 18, 20, 22.
  • the results may also be propagated to the shared database 24 and/or the store archive 30.
  • results may be displayed using one of three display options illustrated in Table 8.
  • the present invention is not limited to three display options and more or fewer display options can also be used.
  • Method 64 can be used in an automatic manner.
  • a protocol is created to automatically accomplish the steps of Method 64 and store results in a database for later analysis.
  • Such a very specific embodiment may used in conjunction with a HTS system.
  • a protocol may be automatically initiated and used to automatically accomplish the steps of Method 64.
  • FIG. 7 is a block diagram illustrating an exemplary screen display 96 for graphically displaying information acquired from images processed using a desired set of assay features.
  • the present invention is not limited to this screen display and other screen displays, and more or less information can also be displayed, and the information can be displayed in different formats.
  • the screen display 96 includes a portion of an image of interest 98 for an object (i.e., a cell) acquired from an image 100 including multiple objects (i.e., a population of cells).
  • the screen display 96 includes object feature data 102 measured from the image of interest 98, and aggregate data 104 and 106 measured from image 100 and nine other images (not displayed).
  • the object feature data 102 and the aggregate data 104 and 106 displayed includes object and aggregate features selected at Step 68 of Method 64 (FIG. 4).
  • the image of interest 98 includes a magnified image of an individual cell identified by 98' in the image 100 including multiple objects.
  • Screen display 96 illustrates exemplary assay feature data only for well A-3 illustrated by the blacked well 108 in the graphical illustration of a microplate 1 10 including 1536 wells.
  • These methods and system described herein may allow experimental data from high-throughput data collection/analysis systems including images to be analyzed.
  • the methods and system can be used for, but is not limited to analyzing cell image data and cell feature data collected from microplates including multiple wells or bio- chips including multiple micro-gels in which an experimental compound has been applied to a population of cells. If bio-chips are used, any references to microplates herein, can be replaced with bio-chips, and references to wells in a microplate can be replaced with micro-gels on a bio-chip and used with the methods and system described.
  • the methods and system help provide a general purpose assay development tool.
  • the methods and system allow a biologist, other scientist, or lab technician not trained in image processing techniques to quickly and easily design protocols and assays to analyze images acquired from experimental data (e.g., cells).
  • the methods and system may improve the identification, selection, validation and screening of new drug compounds that have been applied to populations of cells.
  • the methods and system may also be used to provide new bioinformatic techniques to manipulate experimental data including multiple digital photographic images.
  • steps of the flow diagrams may be taken in sequences other than those described, and more or fewer elements may be used in the block diagrams. While various elements of the preferred embodiments have been described as being implemented in software, in other embodiments in hardware or firmware implementations may alternatively be used, and vice-versa.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Multimedia (AREA)
  • Investigating Or Analysing Biological Materials (AREA)
  • Image Processing (AREA)
  • Apparatus Associated With Microorganisms And Enzymes (AREA)
  • Measuring Or Testing Involving Enzymes Or Micro-Organisms (AREA)

Abstract

L'invention concerne des procédés et un système pour analyser de manière polyvalente des images acquises à partir de données expérimentales recueillies au moyen de systèmes automatisés de collecte de données expérimentales présentant de nombreuses caractéristiques et à productivité élevée. L'invention porte également sur une série de caractéristiques d'analyse générales prédéterminées. Une caractéristique d'analyse comprend une ou plusieurs mesures pour un objet situé dans une image photographique numérisée obtenue au moyen des données expérimentales. La série de caractéristiques d'analyse générales prédéterminées comprend les caractéristiques d'objet, d'agrégat et de traitement d'images polyvalent. Une série de caractéristiques d'analyse désirées est choisie parmi la série de caractéristiques. Une série d'images est traitée au moyen des caractéristiques d'analyse désirées choisies parmi la série de caractéristiques d'analyse. Les procédés et le système permettent d'obtenir un outil polyvalent de mise au point d'analyses. Les procédés et le système permettent à un biologiste, à d'autres scientifiques ou laborantins non qualifiés en matière de techniques de traitement d'images de concevoir rapidement et facilement des protocoles et des analyses permettant d'analyser des images obtenues au moyen des données expérimentales (par exemple, les cellules). Les procédés et le système peuvent améliorer l'identification, la sélection, la validation et le criblage de nouveaux composés pharmaceutiques qui ont été appliqués à des populations de cellules. Ils peuvent également fournir de nouvelles techniques bio-informatiques pour manipuler des données expérimentales, y compris de nombreuses images photographiques numérisées.
EP00937712A 1999-05-24 2000-05-24 Procede et systeme pour analyser de maniere polyvalente des donnees experimentales Withdrawn EP1138019A2 (fr)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US13548199P 1999-05-24 1999-05-24
US135481P 1999-05-24
US14006199P 1999-06-21 1999-06-21
US140061P 1999-06-21
PCT/US2000/014246 WO2000072258A2 (fr) 1999-05-24 2000-05-24 Procede et systeme pour analyser de maniere polyvalente des donnees experimentales

Publications (1)

Publication Number Publication Date
EP1138019A2 true EP1138019A2 (fr) 2001-10-04

Family

ID=26833371

Family Applications (1)

Application Number Title Priority Date Filing Date
EP00937712A Withdrawn EP1138019A2 (fr) 1999-05-24 2000-05-24 Procede et systeme pour analyser de maniere polyvalente des donnees experimentales

Country Status (6)

Country Link
EP (1) EP1138019A2 (fr)
JP (1) JP2003500664A (fr)
AU (1) AU768732B2 (fr)
CA (1) CA2359467A1 (fr)
IL (1) IL144365A0 (fr)
WO (1) WO2000072258A2 (fr)

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002041064A1 (fr) 2000-11-17 2002-05-23 Universal Imaging Corporation Diviseur de faisceau dichroique a changement rapide
DE60103428T2 (de) 2000-12-22 2005-06-02 Cellomics, Inc. Identifizierung von zellen während kinetischer versuchsreihen
GB2389692B (en) * 2001-02-20 2005-06-29 Cytokinetics Inc Method and apparatus for automated cellular bioinformatics
ES2185493B1 (es) * 2001-07-03 2004-06-16 Madrid Genetics, S.L. Metodo para evaluar in vitro en condiciones fisiologicas o patologicas relevantes la actividad biologica de compuestos a gran escala.
JP4454998B2 (ja) 2003-09-29 2010-04-21 株式会社ニコン 細胞観察装置および細胞観察方法
US8045782B2 (en) 2004-12-07 2011-10-25 Ge Healthcare Niagara, Inc. Method of, and apparatus and computer software for, implementing image analysis protocols
JP4649188B2 (ja) * 2004-12-09 2011-03-09 シスメックス株式会社 測定装置の測定結果管理方法、測定システム、測定装置用データ処理装置、及びコンピュータプログラム
US20080279441A1 (en) * 2005-03-29 2008-11-13 Yuichiro Matsuo Cell-Image Analysis Method, Cell-Image Analysis Program, Cell-Image Analysis Apparatus, Screening Method, and Screening Apparatus
JP4868207B2 (ja) * 2005-07-14 2012-02-01 オリンパス株式会社 スクリーニング方法およびスクリーニング装置
WO2012176785A1 (fr) 2011-06-20 2012-12-27 株式会社ニコン Dispositif de traitement d'image, procédé de traitement d'image et programme
JP2013137635A (ja) * 2011-12-28 2013-07-11 Dainippon Screen Mfg Co Ltd 画像表示装置および画像表示方法
US11645859B2 (en) * 2016-06-30 2023-05-09 Nikon Corporation Analysis device, analysis method, analysis program and display device
CN110398925B (zh) * 2019-08-19 2024-01-30 上海应用技术大学 一种基于Zigbee技术的化学实验智能车系统

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5235522A (en) * 1990-10-10 1993-08-10 Cell Analysis Systems, Inc. Method and apparatus for automated analysis of biological specimens

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
GUNTHER T. ET AL: "VIRIM: A massively parallel processor for real-time volume visualization in medicine", COMPUTERS & GRAPHICS, vol. 19, no. 5, 1 September 1995 (1995-09-01), AMSTERDAM, pages 705 - 710, XP004000244 *

Also Published As

Publication number Publication date
IL144365A0 (en) 2002-05-23
CA2359467A1 (fr) 2000-11-30
AU768732B2 (en) 2004-01-08
JP2003500664A (ja) 2003-01-07
AU5284800A (en) 2000-12-12
WO2000072258A3 (fr) 2001-05-31
WO2000072258A2 (fr) 2000-11-30

Similar Documents

Publication Publication Date Title
EP1145149B1 (fr) Procedes et systeme de collection et de stockage efficaces de donnees experimentales
US11661619B2 (en) Analysis and screening of cell secretion profiles
US8068988B2 (en) Method for automated processing of digital images of tissue micro-arrays (TMA)
AU2007352394B2 (en) Quantitative, multispectral image analysis of tissue specimens stained with quantum dots
EP1922695B1 (fr) Procede, appareil et logiciel informatique utilises pour effectuer un traitement d'images
AU768732B2 (en) Method and system for general purpose analysis of experimental data
EP3922980B1 (fr) Procédé mis en oeuvre par ordinateur, produit de programme informatique et système d'analyse de données
WO2005114578A1 (fr) Procede et systeme de quantification automatisee d'une analyse d'image numerique d'un jeu ordonne de microechantillons de tissu (tma)
EP1533720A2 (fr) Méthodes et système d'acquisition efficace et de stockage de données expérimentales

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20010719

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

AX Request for extension of the european patent

Free format text: AL;LT;LV;MK;RO;SI

17Q First examination report despatched

Effective date: 20011126

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20061201