WO2000052646A2

WO2000052646A2 - A method and apparatus for automation of laboratory experimentation

Info

Publication number: WO2000052646A2
Application number: PCT/US2000/005656
Authority: WO
Inventors: Seth Taylor; Ngon D. Dao; Randy L. Milbert
Original assignee: Molecularware, Inc.
Priority date: 1999-03-03
Filing date: 2000-03-03
Publication date: 2000-09-08
Also published as: WO2000052646B1; WO2000052646A3; AU3394700A

Abstract

An automated laboratory management system is provided. This system is particularly suited to management of high-throughput screening experimentation. The system provides for design of experiments, review of experimental design, and data analysis, storage, retrieval, and editing. Graphical aids are provided for efficient association of experimental material sources with destination material carriers and for association of experimental procedures with the experimental materials. The system includes a method for locating periodic image data in a noisy image, in part through determining the placement of a grid on an image.

Description

A METHOD AND APPARATUS FOR AUTOMATION OF LABORATORY

EXPERIMENTATION

Related Applications

This application claims priority to U.S. Provisional Patent Applications: Serial Number 60/122,636, filed March 3, 1999; and Serial Number 60/154,417, filed September 17, 1999; and Serial Number 60/170,677, filed on December 14, 1999. The entirety of these three U.S. Provisional Patent Applications are incorporated herein by reference.

FIELD OF THE INVENTION

The invention relates to the field of laboratory management and more specifically to the field of automation of high-throughput screening research.

BACKGROUND OF THE INVENTION

Laboratory researches must design and plan their experiments, perform the experiments, store and analyze experimental data, and generate reports based on raw and processed data. These experimental tasks are particularly challenging in fields that entail high volumes of samples or data. For example, biologists working in the fields of genetics and drug-discovery often employ high-throughput screening ("HTS") where microassay samples have volumes in a range of picoliters to milliliters. In analogy to biological experiments done in test tubes, experiments done in microassays require liquid-handling steps to mix reagents in strict quantities and specified orders. Commonly, these HTS liquid-handling robots are employed to automate the preparation of thousands of microassay samples, often using microarrays, microplates, or DN A Chips. Data is often collected in the form of images for the thousands of samples, further increasing the difficult of managing experiments. Sample tracking is the first challenge in data management, followed by data collection, analysis and reporting. The quantity and complexity of experiments achievable by these HTS research techniques far exceed the laboratory worker's ability to efficiently manually manage experimental information. This presents significant challenges in experimental design, sample tracking, and experimental data collecting, tracking and analysis.

What is needed is the means to readily design and perform experiments as well as quickly and efficiently store, analyze, annotate, and search information derived from HTS research. What is further needed is the ability to interface an experimental design and data management system to sample handling robots and data collection devices. What is further needed is the ability to perform these management tasks in a unified and intuitive manner.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 is a schematic diagram of an embodiment of laboratory facility, including an automated laboratory system.

Fig. 2 is a block diagram of an embodiment of the automated laboratory system of Fig. 1 that comprises five subsystems;

Fig. 3 is a block diagram of an embodiment of the subsystems in Fig. 2; Fig. 4 is a flowchart that depicts an example embodiment of message flow and processing in the automated laboratory system of Fig. 2;

Fig. 5 is a depiction of an embodiment of a visual display for sample mapping and experimental design;

Fig. 6 is a depiction of an embodiment of a visual display for experiment design and review of selected procedures;

Fig. 7 is a depiction of an embodiment of a grid overlay;

Fig. 8 is a flowchart of an embodiment of a method for orienting a grid on an image with periodic intensity spots; and Fig. 9 is a block diagram of an off-site database system in support of the automated laboratory system of Fig. 1.

DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring to Fig. 1 , the preferred embodiment provides a computer hardware and software-based automated laboratory system 10 that serves to manage research in a laboratory facility 1 , such as an HTS facility. The preferred embodiment is described in the following at increasing levels of detail, commencing with an overview of the interaction of the automated laboratory system 10 and other laboratory facility 1 components.

A laboratory facility Referring to Fig. 1, an embodiment of a laboratory facility 1 is depicted. Such a system would typically be employed in high-throughput screening ("HTS") laboratory such as that employed by research biologists, for example drug discovery scientists. The laboratory facility 1 is comprised of an automated laboratory system 10, a database 20, one or more sample handling robot drivers 30, one or more sample handling robots 40, experimental material sources 50 and experimental material carriers 60.

The robots 40 typically possess several modules to perform various operations, including: moving containers, such as sources 50 and carriers 60; dispensing solutions onto the destination material carriers 60; vacuum aspiration to collect source material 51 from the material sources 50; shaking samples; and heating and cooling samples. The individual mixtures of destination material 61 are typically present in the form of an array on the material carrier 60. In such arrays, the destination material 61 is often distributed as microarrays of droplets or in arrays of wells on microplates. More generally, the destination material carriers 60 can be comprised of: racks, holders, slides, microplates, microarrays, circuit boards, arrays of electric elements, arrays of microgel elements, etc. The moving of sample material from sources 50 to destination carriers 60 is preferably performed by a liquid handling robot that includes dispensing tips for preparing microplates and microarrays, for example the BioChip Arrayer™ (Packard Instrument Company,

800 Research Parkway, Meriden, CT 06450.) Various dispensing tips can utilized, such as those that employ pipettes, pin tools, piezoelectric elements, solenoids and cavro pumps.

Contact or non-contact techniques can serve to dispense molecules onto carriers 60. Contact techniques rely on the use of a surface that produces a drop of solution at one end that is transferred to the substrate in a position dependent manner by direct contact. A pipeting robot, such as that available from Nffymetrix, Inc. (3380 Central Expressway, Santa Clara, California 95051), can provide this capability. Molecules can be deposited through non-contact means by dispensing a drop of solution with a defined volume onto a specific position of a carrier 60. Νon-contact tools employ piezoelectric or solenoid driven pipeting devices.

Experimental data is typically in the form of images of the microplates and microarrays, the images being stored, along with other experimental data, in the database 20. The database 20 can comprise a commercial database, such as SQL Server ™ (Microsoft Coporation, One Microsoft Way, Redmond, Washington 98052-6399). The robots 40 typically provide automated control of the image collection instruments, for example, scanning laser confocal microscopes. Due to the distribution of destination material 61 in array form, each experimental image typically comprises an array of sub-images, each sub-image corresponding to one site of destination material 61.

Management of the laboratory facility 1 is the responsibility of the automated laboratory system 10. The system 10 comprises software running on a computer system, such as computer that supports the Microsoft Windows® operating system (Microsoft Corporation, One Microsoft Way, Redmond, Washington 98052-6399). The system 10 permits management of a full range of laboratory functions, including: design of experiments; control of the robots 40; tracking of source and destination materials 51 and 61; data collection, storage, and access via the database 20; data analysis; data modifications; and linking sample tracking data with related experimental image data.

The Automated Laboratory System

Referring to Fig. 2, an embodiment of the automated laboratory system 10 is depicted in a top-level, block diagram form. In this embodiment, five subsystems comprise the system 10: the data manager subsystem 100; the designer subsystem 200; the annotator subsystem 300; the analyzer subsystem 400; and the main subsystem 500. The data manager 100 maintains the databases for the system 10, including databases internal to the system 10 and databases external to the system 10 such as the database 20, and mediates all communications within the system 10. That is, all messages, comprising data and commands, that are passed between the various subsystems 100, 200, 300, 400, and 500, pass through the data manager 100. The data manager 100, further, has the task of notifying subsystems 200, 300, and 400 of changes in system 10 status by sending notification messages to all subsystems 200, 300, and 400 upon occurrence of such changes.. The system 10 is preferably implemented through use of object-oriented design methods and an object-oriented computer software language, such as C++ or Java. The system can be applied to a variety of data management needs. The preferred embodiment is tailored to management of microarray research in the field of biology. The system 10 enables experimental design, tracking and mapping of the mixing of reagents and biological molecules from sources 50, typically from wells on microplates, to destination material 61 locations, typically wells on microplates or sites on microarrays.

Subsystem Design and Function

The preferred embodiment uses a logical division of tasks between the various subsystems 100, 200, 300, and 400. This enhances the ability to implement the system 10, improves user interactions with the system 10, and provides for addition of features to the system 10, as desired. The preferred embodiment of the data manager 100 centralizes mediation of communications between subsystems 100, 200, 300, 400, and 500 and with the database 20 along with responsibility for data management. In the preferred embodiment, the designer 200 has responsibility for user interactions that entail design and review of experiments. In the preferred embodiment, the annotator 300 is, in essence, a data editor. Alphanumeric data can be viewed and edited, while also viewing experimental images. In the preferred embodiment, the analyzer 400 is responsible for calculations and user interactions that are involved in image and data analyses. In the preferred embodiment, the subsystem 100, 200, 300, and 400 share some implementation. For, example, since the user will wish to view images while annotating data and while analyzing data, the annotator 100 and the analyzer 400 can share image viewing classes that support on-screen display of images. The main subsystem 500 comprises the operating system portion of the system 10.

Embodiments of the subsystems 200, 300, and 400 utilize the Ul-model-mutator design pattern (UI meaning user interface), this design pattern known in the prior art, in the organization of subsystem 200, 300 and 400 classes. As depicted in Fig. 3, a subsystem 600 comprises a set of UI classes 620, a set of model classes 630, a set of mutator classes 640, and a facade class 610. Further employed is the object-oriented observer-listener message handling technique, known in the prior are, via addition of observer 660 and listener 670 classes. The facade class 610 mediates all messages passing to or from the subsystem 600. Hence, only subsystem facade classes 610 are registered with the data manager class for receipt of update notifications. UI classes 620 contribute to a graphical user interface including support for the windows that are associated with the particular subsystem 200, 300, and 400. Model classes 630 handle data storage and are associated with real-world objects, such as microplates and microarrays. Mutator classes 640 handle data manipulation. The number and relationships of UI, model, and mutator classes 620, 630 and 640 vary with the requirements of the particular subsystem 200, 300, and 400. Each subsystem 200, 300, and 400 possesses the implementation associated with the roll for the particular subsystem 200, 300, and 400, though, as noted above, some sharing of code is employed for efficiency of implementation.

The Data Manager - The data manager ("DM") subsystem 100 mediates all communications for the system 10. Hence, the flow of data and commands between subsystems 200, 300, 400, and 500 passes through the DM 100. The DM 100 further manages data storage for the system 10 and in particular is responsible for all data that is shared across the system 10. The facade 610 of the DM 100 is also referred to as the "librarian".

The DM 100 includes model classes that, when instantiated, correspond to various experimental design objects. Such model classes are given descriptive names, for example: DeckModel, which holds descriptive data for decks, e.g. sets of microplates, and RackModel, which holds descriptive data for a rack, e.g. a microplate. In the preferred embodiment, models provide all data associated with the object they represent as with as methods to access that information. For example, the RackModel class includes the following data items: name, width, height, and well depth. In the preferred embodiment, a model class is not implemented for wells. Since the number of wells is typically in the tens of thousands or more, the number of instantiations of well model class objects would be equally huge, leading to undesirable overhead.

Attributes are the various data items that are associated with a particular well. Other experimental objects can also have attributes, such as images and spots. In the preferred embodiment, objects such as racks and wells have attributes, i.e. associated data, that are managed via instantiations of an AttributeList class and an associated ByteBucket class. The naming of the classes is chosen for its descriptiveness. The AttributeList provides a solution to the problem of storing data fields when the exact number and type of data fields are not specified at compile time. Because such a high number of AttributeList objects can be instantiated, the internal data structures must be very memory space conservative. In the preferred embodiment, the AttributeList is structured to be runtime flexible and conservative of random access memory

("RAM") usage.

Each instantiation of an AttributeList contains an instantiation of a ByteBucket object. In the preferred embodiment, the ByteBucket is designed to be a flexible array of bytes whose memory size can be resized automatically. Preferably, a CArray is not used because it contains the added memory overhead associated with the CObject class from which it inherits. A CList also is undesirable because its implementation would take up even more RAM than would use of CNrray. A static byte array is unsuitable because its size is fixed at compile time. To achieve a compact design, the preferred embodiment use no pointers in the internal data structure of the AttributeList. The ByteBucket simply contains all the data fields, one after another encoded in the byte array. Such a data structure can lead to greater data access times. With the large number of AttributeLists instantiated at any one time, it is desirable to provide for lookup of data as fast as possible. For this purpose, each AttributeList contains a ByteBucket object that stores the raw data. The AttributeList handles interpretation of the data held within the ByteBucket, without use of conventional pointers. Hence, both RAM usage and data access speed efficiencies are obtained.

In the preferred embodiment, the internal data structures of the AttributeList comprise the ByteBucket, i.e. a resizable byte array and a, descriptively named, RoleCall. The RoleCall comprises an array of bits. Each bit represents a possible field in the AttributeList. If a particular bit has the value ' 1 ', then that field is present. The AttributeList does not possess information concerning the data type of the field. The ByteBucket data structure contains two sections. The first part of the ByteBucket, a pointer-containing section, contains 16-bit pointers to the actual data fields contained in the second section of the ByteBucket, a data-containing section. Each pointer is an absolute pointer from the first byte of the ByteBucket. There is a 16- bit pointer for each bit value of 1 ' in the RoleCall. The 16-bit pointers are arranged in the pointer section of the ByteBucket in the same order as the bits in the RoleCall. For example, if the first two bits of the RoleCall are ' 1 ', then there exist two pointers in the pointer section that correspond to data fields in the data section. As a result, if the data field is not present, no space is allocated for that field in the data section. Even when a particular data field is present, the pointer to the data field requires only 16-bits of space, rather than the 32 bits of a true 32-bit pointer. This simplification limits the total size of an attribute list to 64 Kbytes, however, this limitation is desirable given the extremely large number of AttributeList instantiations that can exist. In the preferred embodiment, the DM 100 possesses command classes that support command generation for the system 10. Commands are generally comprised of two types: read and write. Commands are transmitted between subsystems as messages that possess data which may include logic for operating on data. Write commands typically alter data while read commands typically collect and return data to the requesting subsystem 100, 200, 300, 400, or 500. In the preferred embodiment, the command classes are implemented following the approach of the command design pattern, know in the prior art. This design patterns provides for encapsulation of commands as objects in a manner that permits the queuing or logging of command requests and the support of undoable operations. An example of the interaction of commands and the sending of messages in the system 10 is given in the section below, entitled "The Messaging System".

The Designer Subsystem - The designer subsystem 200 has primary responsibility for the design and review of experimental plans. Methods are provided for mapping of sample sources 50 to destination carriers 60 and for associating experimental procedures with those associations, as described in a section below, entitled "Manipulation and Analysis of Images". The designer subsystem 200 further provides for review and modification to existing mappings and procedures.

The preferred embodiment of the designer subsystem 200 includes relatively large numbers of UI classes 620, model classes 630, and mutator classes 640. In the preferred embodiment, UI classes 620 support, for example, the display of decks, racks, and experimental procedures. Further, there is window support for manipulation of displays, such as a rack display, via zooming and translating display depictions. Similarly, model classes 630 are provided to hold experimental design data, such as procedures to be used and the microplates to be processed. Mutator classes 640 support the UI 620 and model classes 630 by providing the methods required to manipulate data while, for example, zooming a display or editing experimental design data.

In the preferred embodiment, there is sharing of some designer 200 classes with the annotator 300 and analyzer 400 subsystems since a user will desire some similar functionality while performing tasks via the different subsystems 200, 300, and 400. For example, the user will wish to observe racks while designing an experiment, while interacting with the designer subsystem 200, and later may wish to edit data associated with those racks while interacting with the annotator subsystem. 300. So, graphical support for rack viewing will be desirable from within the designer subsystem 200 and the annotator subsystem 300.

The Annotator Subsystem - In the preferred embodiment, the annotator subsystem 300 has responsibility for mediating user interaction with data stored in the system 10. The annotator 300 is essentially an edit and search tool. The annotator 300 provides for user interaction when the user wishes to examine alphanumeric data and add to, delete, or edit that data. The annotator 300 further provides facilities for a user to search for data, both alphanumeric files and images files. In the preferred embodiment of the system 10, the annotator 300. In the preferred embodiment, the annotator 300 comprises several UI classes 620 that provide graphical, i.e. windows, support to permit the user to search, select, and modify data.

In the preferred embodiment, the data displayed by the UI classes 620 of the annotator 300, are obtained by the facade 610 for the annotator 300 via the DM 100. The annotator 300 facilitates the automation of experimental management. The UI classes 620 of the annotator 300 accept changes as input while the facade 610 of the annotator 300 communicates the changes to the remainder of the system 10 via the DM 100. Hence, the database 20 can be updated and current data displayed when the designer 200 or analyzer 400 are accessed by the user. For example, a user can input values and annotations for experimental objects: images, wells, racks, procedures.

The annotator subsystem 300 in conjunction with the analyzer subsystem 400 provide for manipulation of experimental images. Image manipulation is described below, in a section entitled "Manipulation and Analysis of Images".

The Analyzer Subsystem - In the preferred embodiment, the analyzer subsystem 400 has primary responsibility for mediating user interaction in data analysis. The analyzer includes UI classes 620 and model classes 630 that support the display and manipulation of images for image processing. Aspects of processing and analysis of experimental images are described in the section below, entitled "Manipulation and Processing of Images".

In an alternate embodiment, the analyzer 400 exists as a stand-alone application for analysis of existing images.

The Main Subsystem - The main subsystem 500, encompasses the functionality and GUI support provided by the operating system. This includes support for windows, menus, and toolbars. The Microsoft Corporation Windows® operating system (Microsoft Corporation, One Microsoft Way, Redmond, Washington 98052-6399) is employed in the preferred embodiment. Messaging system

Referring to Fig. 4, the flow of messages through the system 10 is illustrated by the processing of a user selection of a particular well while working with the designer 200. The user initially selects (Step 1) a well, from a screen image, via mouse clicks. The selection is received by a UI class object 620 of the designer subsystem 200. The UI class object 620 in turn passes (Step 2) a request for coordinate information to a mutator class object 640 of the designer 200, the mutator class object 640 generating a selection request. The set selection request is passed to the designer facade 210 (Step 3), to be forwarded (Step 4) to the librarian 110 for a data update. The librarian notifies (Step 5) all subsystems 200, 300, 400, and 500 of the new set selection, e.g. notification that the user has made a new well selection. Since the annotator 300 supports display of current data, the annotator must respond to the notice in order to obtain the new data for display. Upon receiving the notification, the annotator facade 310 passes the notification to a UI class object 620 of the annotator 300 (Step 6). The UI class object 620 of the annotator 300 creates a get selection request, to obtain the updated user selection data, and passes it to the facade 310 (Step 7). Similarly, the get selection request is processed, by forwarding it to the librarian 110, to obtain current data for the selected well and transfer that data to the annotator 300 for display to the user.

The User Interface

Overview - Referring to Fig. 5., the system 10 includes provisions for an intuitive graphical user interface to assist in experimental design, data analysis, and sample tracking. In the preferred embodiment, while interacting with the designer 200, the system 10 can visually depict the relationships between source sample sites 51 and destination sample sites 61. For example, Fig. 5 shows a source microplate 1010 and a destination microarray 1020. Lines 1011 link source well sites on the microplate 110 to destination sample sites on the microarray 1020. More complex mappings can be achieved and displayed than depicted here. For example, multiple source sample sites 51 can contribute to multiple destination sample sites 61. Figure 5 further shows an image 1030 for a portion of the sample sites on the microarray 1020. Again, lines 1021 are used to indicate the association of the image with the particular sites on the microarray 1020. One or more on-screen boxes 1041 and 1042 can be used to display alphanumeric data, for example, lists of: microplate identifiers; microarray identifiers; experimental procedures; image identifiers; and image analysis data for particular image sites. Boxes can display lists of sources 50 and destination carriers 60 for selection during design of experiments. Further, lists of sample carriers, such as microplates, can be provided for the user to scroll through. The user can then select particular microplates to be employed as source 50 or destination 60 carriers. A procedure box can be displayed that supports the graphical depiction described above by displaying the x-y coordinates of source sites 50 and destination sites 61. Descriptions of procedural routines can be listed for sample sites 61, for example, shaking, heating, or cooling routines. Windows, such as dialog boxes, can be used to apply annotations to experimental components and procedures, either through the annotator 300 or the designer 200. In the preferred embodiment, keywords are used to describe various parameters, such as the purpose of a procedure, the number of times it will be repeated, and the liquid volume associated with the procedure. In the preferred embodiment, the system 10 provides for design and display of procedures, via interaction with the designer 200, through a series of linked procedures, known as a procedure tree. A procedure tree outlines all procedures that are used when experimental material 51 is moved between a material source 50 and a destination material site 61. Hence, the procedure tree outlines the various steps performed by a robot 40, including moving material and containers, and shaking, heating and cooling of samples. The procedure tree also displays source 50 to destination 61 mappings. Hence, a complete experiment can be created, reviewed, and edited via the procedure tree. In the preferred embodiment, defined sets of procedures are grouped together into graphical objects called loops, utilizing the object-oriented programming technique of inheritance. By utilizing on-screen boxes 1041 and 1042 that display procedure trees and source 50 and destination 61 sample locations, procedures and associated sample locations for source 50 and destination 61 sites can be selected and linked. Visual display eases the selection and review process. For example, selecting a procedural step with a mouse-click causes the listed locations of the associated sample sites 50 and 61 to be highlighted. Distinguishing coloring schemes are employed in the preferred embodiment to aid quick localization of correspondence between source 50 and destination 61 site locations. To further aid experimental design, the preferred embodiment provides drag-and-drop features to aid in establishing relationships between source 50 and destination 61 sites. For example, the cursor is employed to highlight a set of source sites 50, for example on a microplate. The highlighted section is then dragged to a display of, for example, a destination microplate and released when aligned with the desired set of destination 61 sites. Selected source sample 50 sites can be modified with further mouse clicks on the original selected set of source sites 50. To implement large sample mappings, the user can indicate multiplication factors to be applied to the dragged set of source 50 sites to include a proportionally larger set of sites from, for example, a microplate.

Linking of source samples, destination samples, and procedures - Referring to Fig. 6, one of the display options 2000 of the preferred embodiment supported by the designer 200 is shown. A source microplate 2010 and destination microplate 2020 are schematically depicted in two Windows, while a third Window displays a procedure tree 2030. In this display 2000, the individual sample sites are depicted as small squares, either source sites 2011 or destination sites 2021. The user can annotate each procedure in the tree with descriptive keywords. In the preferred embodiment, the ability to annotate procedures is also supported during user interaction with the annotator 300. The tree treats each procedural step as an object-oriented component where each object possesses attributes to describe the state and behavior of the object. The state of a procedure can include keywords that describe, for example, its purpose, the number of times the procedure is repeated, or the liquid volume associated with the procedure. Similarly, the behavior of a procedure describes information related to the actual procedural action.

The procedure tree 2030 is actively linked to source and destination locations 201 1 and 2021 in the sample carrier depictions 2010 and 2020. Highlighting a procedure automatically highlights the associated sample sites 2011 and 2021.

The procedure tree 2030 uses loop nodes, end nodes, and various procedure nodes. A procedure tree 2030 represents the process of mapping. Loop nodes allow the user to specify redundant mappings by overlay. End nodes allow the user to insert other nodes without confusion about what relationship the new node will have to existing nodes (e.g. is the new node a sibling of an existing node or a child?)

To select source wells to match the aspirate and dispense heads of the system 1, a ghosting scheme is used. The scheme shows a ghost of the head as an overlay on top of the source 2010 and destination 2020. For example, a ghost image similar in appearance to four sample sites 2011 or 2021 is used to represent how individual tips on a dispense head are positioned relative to each source carrier 2010 and destination carrier 2020. In this way, the user can visually determine the location of pipette heads relative to wells in microplates.

Using drag-and-drop techniques, a user can specify the moving and mixing of reagents from source carriers 2010 to destination carriers 2020. The preferred embodiment includes two methods for mapping the movement of reagents: 1) the ghosting method described above; and 2) - lo using block allocation via specification of x-y coordinates, i.e. where the tip allocations are made transparent to the user. The source and destination carriers 2010 and 2020 employ a coloring scheme to show how source and destination sites 2011 and 2021 are related. The preferred embodiment further allows for replicate block mapping via vertical and horizontal expansions. That is, large areas of source and destination carriers 2010 and 2020 can be mapped by specifying mappings of multiple smaller areas.

Zooming - The preferred embodiment provides three toolbar buttons for selection of the desired zooming mode: arrow, zoom in, and zoom out. Zooming support is provided, for example, by the analyzer 400 for examination of experimental images. One button may be selected at any one time. When the cursor is held over a carrier, for example the carriers 2010 and 2020 in Fig. 6, the cursor depicts an icon that indicates the current zoom mode. For zoom in, the image will expand and become centered on the site 2011 or 2021 selected by the cursor during a click. The image will similarly shrink when in zoom out mode. In arrow mode, the user can use the arrow to select sample sites 2011 or 2021. In another embodiment, the toolbar provides no arrow button. Here, zoom in and zoom out do not recenter the image; the image simply zooms in response to toolbar button clicks. The cursor never changes and can always be used to select sample sites 2011 and 2021.

Selection - As discussed above, the designer 200 supports user selections experimental design. Selection of source sites 2011 can be performed in a number of ways. A user can click and drag to create a rectangle about desired sites 2011. Upon releasing the mouse, the selected sites 2011 are highlighted. Destination sites 2021 can similarly be selected. The user is notified if the numbers of rows and columns for selected source and destination sites 2011 and 2021 do not match. In another embodiment, the rectangle that is displayed during click and drag for destination sites 2021 is automatically confined to a configuration that matches the previously selected source sites 2011. In another embodiment, the rectangle that is displayed during click and drag for destination sites 2021 is displayed as a ghost rectangle that follows movements of the cursor. The cursor can be used to click-and-drag to change the size of the ghost rectangle. In another embodiment, a click-and-drag creates a rectangle about desired source sites 2011 and, upon release of the mouse, the selected sites 2011 are highlighted. The user then specifies expansion parameters to be applied to the selected sites 2011 for mapping onto destination sites 2021. The user then places the selected rectangle at a desired location on the destination carrier 2020.

The embodiments described above are preferably provided with toolbar support to enable the user to select a mode for mapping of source sites 2011 to destination sites 2021. One embodiment employs three toolbar buttons: arrow, ghost, and box, to permit the user to select a preferred option. In a further aspect of this embodiment, there are three available procedure nodes: aspirate node, dispense node, and end aspirate node. Aspirate nodes have five parameters: source rack, source well tip N, source well tip B, source well tip C, and source well tip D. Dispense nodes have four parameters: source rack source well, destination rack, destination well. End aspirate nodes have no parameters. For example, a user can click a source site 2011 (e.g. a well) with the box button selected and draw a rectangle to create many aspirate nodes. The selected source sites 2011 (e.g. selected rectangle of wells) can be dragged and dropped onto destination sites 2021 (e.g. destination wells) to create dispense nodes. Alternatively, the above discussed selection procedures can be supported by a set of seven toolbar buttons: redundancy by overlay, redundancy by next available, replicate by column, replicate by row, replicate by box, duplicate, and custom mapping.

Linking of Procedures with Site Selections - Procedures can be associated with source and destination sites 2011 and 2021 during the selection methods discussed above. For example, the "box" button is first clicked. Then, an aspirate node in the procedure tree Window 2030 is clicked. The following selection of source sites 2010 causes the selections to be identified as aspirate sites. To add a loop to the procedure tree Window 2030, a node is first clicked. Next, an "add loop" toolbar button is clicked. This causes a loop to be inserted above the selected node. Nodes can be deleted by first selecting a node and then clicking a "delete node" toolbar button. Deleting and Adding Racks - Various embodiments can provide for the addition or deletion of source and destination carriers 50 and 60 for a particular experiment. In one embodiment of the designer 200, the user makes an "add rack" selection, in a menu associated with a deck Window, which causes an "add rack" dialog box to appear. To add a rack, the user selects a rack from a drop-down menu, identifies it as a source or destination rack, and clicks an "add" button. The user can add more than one rack at a time. The user clicks a "done" button, when finished. A similar approach is used to delete racks.

Coloring of Wells - To aid identification of the correspondence between source and destination sites 2011 and 2021, various coloring schemes can be employed. In one embodiment for the designer 200, random colors are used to paint mapped source and destination wells. Unmapped sites are depicted in gray tones. Alternatively, every source well is painted with random colors while only mapped destination wells are painted. In the preferred embodiment, gradient colors are used for mapped source and destination wells while unmapped wells are depicted in gray tones.

Display of Well Specific Data - Among options for graphical aid in display of well specific data, as supported by the analyzer 400 and annotator 300, are: 1) hold cursor over a well of interest and observe data in a tool tip; and 2) hold cursor over a well of interest and observe data on the status bar.

Well to Well Mapping Details - Additional features, in embodiments of the designer 200 that support review of experimental design, provide for display of well associations. In one embodiment, a click on a source well causes lines to appear that connect the source well with associated destination wells. The converse is true for clicks on destination wells. In another embodiment, a click on a source well causes a popup Window to provide a list of destination wells that the source well maps to. The converse is true for clicks on destination wells. The popup Windows can display additional data, such as well and rack identification numbers. In yet another embodiment, a click on a well causes associated wells to become highlighted. Manipulation and Analysis of Images

Interaction with images can be divided into three categories of work: selection of images for display; associating a grid overlay with a selected image; and the image pixel calculations. Such interaction, in the preferred embodiment, is supported by the analyzer 200. For convenience, grids are discussed first in the remainder of this section.

Grids - Referring to Fig. 7, a template overlay ("template") 3000 is schematically depicted. Templates 3000 are employed in analysis of images, that is, in the extraction of pixel data from images. The template 3000 is an object on the image display that shows which areas are used to segment a set of wells, i.e. destination samples 61. A template 3000 is comprised of a grid 3010 and individual well templates 3020. The well template 3020 is a collection of well segmentation lines. These lines show the boundary of what, in an image, is considered to be an image of a well 61. In a particular embodiment, well templates 3020 can be circles or squares or designed to fit the particular shape of a sample site 61 image, such as a well image or a microarray droplet site image. A template 3000 is manipulated for a selected image so that, when superimposed on the image, line elements of the grid 3010 are interposed between well 61 images and surround the area to be analyzed while the well templates 3020 individually surround the image sites that correspond to sample sites 61. Once properly prepared, a template 3000 is used in extraction of intensity data from its associated image.

In the preferred embodiment, a user simply perceives a template 3000 as an interface that returns values that can be associated with a rack. The template 3000 behaves as a pseudo-rack with which the user interacts. The template 3000 has functionality that is almost identical to a rack object in the designer 200, with tool-tip dialogs and full annotation capabilities. In reality, the template overlay 3000 is merely a template that is used to generate data from a selected image that will be associated with the image. Certain associations must be maintained for interaction with the designer 200 and the annotator 300. These are: 1) the association of the image with a deck and 2) the associations of the circles or squares in the template 3020 with the individual well 61 images.

When displaying many images at once, it may be difficult to check each image to insure that the template 3000 is placed correctly. A "spot check" Window is supported by the analyzer 400 to mitigate this problem. The spot check Window provides two views of an image: one zoomed-out and one zoomed-in on a few well sites 61. This is generally sufficient to ascertain template accuracy.

In the preferred embodiment, once an image is initialized, a template 3000 is automatically generated and matched against the image. The analyzer 400 calculates a best guess as to placement of the template 3000, as discussed below. To fine-tune template 3000 placement, the grid 3010 or individual well templates 3020 or groups of well templates 3020 can manually be moved. Further, the scale of the template 3000 relative to the image can be modified. Once template 3000 placement is satisfactory, the image can be analyzed.

Several options exist for user interaction with the grid 3010 and well templates 3020. In the preferred embodiment, the user can choose between "show image alone", "show image with template 3000", "show image with grid 3010", or "show image with wells 3020".

In greater detail, several options exist for template 3000, i.e. grid 3010 and well template 3020, setup. In one embodiment, the setup process is conducted through a wizard. The wizard acts on all selected images in parallel. There are several stages to the process: setting template 3000 parameters for each image; automatic detection of the template 3000; manual correction of the template 3000; and proceed to analysis. In another embodiment, a tool properties window provides tools dedicated to template 3000 setup. In another embodiment, no utilities are specifically dedicated to template 3000 manipulation. This provides for a simpler user interface.

Information about the template 3000 is entered via the annotator 300. One image data field accessible via the annotator 300 is the image spot size. An initial setting for the spot size that approximates the size of a well 61 image aids in the process of automatic template 3000 setup. Since this parameter can vary considerably from image to image, manual setting of the spot size can be desirable. Alternatively, the spot size can initially be set at half the grid 3010 line spacings. This generally provides for a satisfactory automated result. In another embodiment, preliminary image analysis is performed to obtain an estimate of spot size.

A method for locating periodic image data in a noisy image - To assist automation of grid 3010 placement while avoiding confusion due to image artifacts, a method is employed by the system 10 for calculating grid 3010 placement. Method steps are depicted in the flow chart in Fig. 8. The system 10 creates a grid 3010 with line spacings determined from the nominal spacings of individual spots in an image. Alternatively, line spacings are derived from knowledge of the actual physical site 61 spacings on a sample carrier 60 from which the image was obtained. The grid 3010 is generally setup with a number of rows and columns chosen to correspond to that found in the particular image. First, a grid 3010 is placed (Step 10) at an initial, arbitrary, location relative to an image. Initial displacement values, in the preferred

embodiment Δx and Δy, are chosen (Step 20). Values for Δx and Δy of ! the grid 3010 spacing

work satisfactorily. A weighted intensity sum for all pixels in the grid 3010 is obtained(Step 30). The weighting function can vary, however it must give greatest weight to the origin pixel, i.e. a pixel where grid 3010 lines intersect, and to pixels at periodic distances from the origin, i.e. at grid 3010 intersections. The weighting must fall off for pixels with increasing distance from those most heavily weighted pixels. This weighted sum gives a current intensity value. An

intensity sum is obtained for each of eight possible +/-Δx and +/-Δy combinations (Step 40): the

grid 3010 is shifted and a weighted sum is obtained for each of the eight displaced positions. It should be noted that altering the exact number and location of displaced positions can still suit the goals of the method. Next, (Step 60), the sum for the current position calculated at Step 30, is compared to the sums for each of the eight positions, calculated at Step 40. If the sum for the current position is not the greatest sum, the grid 3010 is moved (Step 70) to the location for the

displacement that gave the greatest sum at Step 40. A new Δx and Δy are selected (Step 75) and

a satisfactory new Δx and Δy are determined by taking the x-axis and the y-axis components of

^lλ of the vector connecting the new grid 3010 location with the prior grid 3010 location. The method then continues by returning to Step 30.

If, at Step 60, the sum for the current position is greater than the sum for each of the displaced positions, a vector sum is calculated for the eight displaced location (Step 95), each component of the sum being weighted by the intensity sum for the associated grid 3010 location. Various additional weighting factors can be applied, though preferably a weighting factor that limits the resultant vector to a magnitude no more than half a grid 3010 spacing is desirable. If the vector sum has a magnitude below some threshold value, i.e. near zero, the desired grid 3010 location has been found (Step 100). If the vector sum is greater than the threshold value, the grid 3010 is displaced by the vector sum (Step 105). The method then continues by returning to Step 30. Method steps continue as described above until the desired threshold is achieved at Step 100.

Image Pixel Data - Once the template 3000 has been properly aligned with an image, the image can be analyzed by the user, as supported by the analyzer 400, to extract spot intensity data and statistics, such as: average intensity of each image spot, i.e. each well site 61 image; peak intensity of each spot; and the total pixel intensity for all pixels at a spot. As described earlier, the annotator 300 can be used to examine and annotate image data.

Displaying and Annotating Images - The analyzer 300 provides for selection and viewing of stored images. In one embodiment, selection of an image causes the image to be displayed along with an "annotator Window" that displays image data. If more than one image is selected, the annotator Window displays common properties of all the selected images. Fields for which the values differ will be left blank. Some fields can be edited while others can be set to read-only. When multiple images are selected, data entered into a field is associated with all selected images. In one embodiment, two tabs are available for annotation of image data: "comments" and "properties." The properties tab lists essential properties of the selected images and the templates that will be overlaid on them for analysis. In the preferred embodiment, properties must be filled in before an image can be initialized, that is, associated with a destination carrier 60. Further, an image must be initialized before it can be analyzed. The comments tab can be used to store any information about images that does not neatly fit into a any category under the properties tab.

Many properties are associated with an image, such as: rack name, if a deck is loaded; template; image height; image width; number of horizontal well groupings; number of vertical well groupings; horizontal spacing between sets of wells; vertical spacing between sets of wells; number of rows per set; number of columns per set; spot geometry (i.e. well image, either circle or square); spot size; micron-to-pixel conversion; image background intensity subtraction; image brightness; and image contrast.

In the preferred embodiment, each displayed image is displayed in its own Window with its name on the title bar. For initialized images, a grid envelope (that is, a general outline that contains all the wells in the image) and a grid template (that is, the individual well templates) are displayed with the image. Both the grid template and the individual wells can be moved to new positions. If the image is yet to be initialized, the no grid can be placed until more parameters are entered via the annotator 300.

In another embodiment, of images that have been selected and displayed, those that have not been initialized are gray. Displayed images cascade one on top of another. The last displayed image is the active one. This embodiment provides for display of any number of images.

In a further embodiment, selected initialized images are grouped in one folder while selected uninitialized images are grouped in a second folder. These folders can be presented in a tree structure. When an image in the uninitialized image folder becomes initialized, the initialized image is moved to the initialized image folder.

Since images are linked to annotations and grouped in logical folders, the user is greatly assisted in locating experimental images. The use can search terms or examine trees to locate an image.

Method and Apparatus Support of Image Databases Referring to Fig. 9, a database support database system 4000 that can further enhance the utility of the system 10 is depicted. The system 10 is linked to database system 4000. Software to manage the database system 4000 resides, preferably, on a computer 4100. A database 4200 can reside on the computer 4100 or on independent hardware. The database 4200 stores images and data files. The database system 4000 can reside at a user's location, but preferably would reside off-site. The database system 4000, in one embodiment, is operated as an independently run service, offered to a user. As such, the database system 4000 provides backup and management of critical data, freeing the user to focus on research efforts.

In a further embodiment, data processing features are provided by the system 4000. In one aspect, the user would do a preliminary review of images, selecting image locations of interest, such as well 61 images,. A data packet, containing the user's selection, is then sent to the database system 4000. The database system 4000 then retrieves the images from the database 4200. The database system can then generate a composite image from the selected sites.

In a further embodiment, additional imaging processing and storage services can be provide by the database system, further support the system 10 user's research efforts. Given the huge numbers of images and related demands on image and data processing and image and data storage, as encountered in HTS research, the database system can greatly increase the efficiency of a researcher.

Method for Accommodating a Variable Table Size Requirement In some circumstances, a software program may need to accommodate a variable size requirement. For example, one may need to dynamically vary the number of columns available for data storage. This can be accommodated by implementing a table with an extra column.

When additional columns are needed for data storage, the extra column can be loaded with pointers to additional tables, or other identifiers for access of an additional table or tables. This creates, in effect, a larger, i.e. a virtual, table.

Method for Operation on Data in a Table via Database Access Commands

In a software program that stores data in tables, for example in spread sheet form, it can be desirable to provide for manipulation, for example mathematical manipulation, of data in the tables in a manner external to the tables. This can be provided for through capabilities accessible via database query commands. For example, in the case of a relational database, SQL commands can provide for operations on the accessed data.

While the invention has been particularly shown and described with reference to specific preferred embodiments, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

We claim: L A method for coordinating a grid with an image, the image including periodic intensity levels, comprising the steps of: a) Determining the nominal spacing of the periodic data; b) Constructing a grid, the grid comprising grid elements of periodicity equal to that of the periodic data; c) Sizing the grid so that the grid has a number of rows of elements and a number of columns of elements to correspond to a number of rows and number of columns of periodic data elements; d) Placing the grid at an initial location relative to the image; e) Selecting an initial grid shift value; f) Adding the intensity of all image pixels, wherein each pixel intensity is weighted by a spatially variable factor with periodicity that of the grid; g) Calculating a weighted intensity for eight alternate locations for the grid that are associated with translation vectors for the eight possible sets of shift values; h) Comparing the intensity value for the initial location with the intensity value for each of the alternate locations; i) Defining a total translation vector, if the intensity value for the initial location is not greater than the intensity value for each of the alternate locations, as a vector addition of the 8 translation vectors and translating the grid by the total translation vector, wherein steps i and j are repeated.