EP1173818A1 - Cell-based binning methods and cell coverage system for molecule selection - Google Patents

Abstract

A cell-based binning method ('Data Driven' binning method) allows the inclusion of all molecules, generates a high percentage of occupied cells, and provides adequate division of the molecules in the low-dimensional subspaces (typically all one-dimension (1-D), two-dimension (2-D), and three-dimension (3-D) subspaces). A chemical space coverage criterion ('Uniform Cell Coverage (UCC)' criterion) measures the uniformity of coverage of the molecules selected. A fast exchange design algorithm ('fast exchange UCC' algorithm) that minimizes the number of searches of the candidate points while maximizing the number of exchanges during each pass through the candidate points. This method is many times faster than previous exchange algorithms and generates designs with good coverage properties.

Description

CELL BASED BINNING METHODS AND CELL COVERAGE SYSTEM FOR

MOLECULE SELECTION

Cross-Reference to Related Application

This application claims priority from U.S. Provisional Application No.60/132,516, entitled "CELL BASED BINNING METHODS AND CELL COVERAGE SYSTEM FOR MOLECULE SELECTION", by Raymond L.H. Lam, William J. Welch and S. Stanley Young, filed on May 4, 1999, which U.S. Provisional Application No. 60/132,516 is fully incorporated herein by reference.

Field of Invention

The present invention is related to drug research technology. More particularly, the present invention provides a system and method for enhancing successful discovery of drugs using high throughput biological screening technology by providing a representative set of compounds to screen. This provides a general method for selecting representative objects.

Background of the Invention

The use of robotics and miniaturization is now allowing researchers to quickly screen thousands of compounds for biological activity. Combinatorial chemistry provides the logistics of mass production of compounds and a wide range of molecular diversity for drug discovery. The automation of biological assays, High Throughput Screening (HTS), allows for investigation of thousands of compounds against biological targets per week. While this brute-force approach to lead generation certainly has its place in the field of drug discovery, it is not practical to adopt HTS for every new target of potential biological importance, given the size of today's chemical libraries (e.g., hundreds of thousands to millions of compounds).

Various molecular descriptors (explanatory variables) can be readily computed to describe the chemical properties of every molecule in the database. When there is no prior model relating biological response to these descriptors, the generally accepted procedure is to screen (test) a diverse subset of the overall database, and then examine further compounds that are structurally similar to any promising leads. Measures of "diversity" and "similarity" are based on the numerical descriptors. The assumption here is that similar objects are more likely to have similar biological responses. Thus, if an initial subset is to be selected, the subset should "fill" or "cover" the numerical space in some sense. Ideally, selected objects should be as dissimilar as oss le aud any candidate not selected should be near a molecule in the experimental design.

To measure the "coverage" of a descriptor space, the space is divided into cells. A good experimental design will ideally have at least one molecule in every cell. When this condition is met, the space is said to he covered- h a conventional cell-based method, the range for each of the k descriptors is subdivided into m bins of equal si∑e, yielding m¹ cells. With even moderate values of m and k, a huge number of cells are generated, most of which are empty even for the candidate set of all molecules in the database. Any subset selected has even poorer coverage of the cells, making comparison of potential experimental designs difficult

Figure 1 shows the univariate and pairwise plots (100 and 105) of the six descriptors for the NCI candidate molecules (described later), with the distributions of the NCI molecules in one-dimension ( -D) and two-dimension (2-D) projections for all 6 descriptors. It is clear that much of the space is empty. Either the collection is missing chemicals or it is not possible to make compounds with certain combinations αf descriptors. In more than two dimensions, this problem will be even worse. Consequently, to deal with a problem of practical importance, P. R. Menard et al., "Chemistry Space Metrics in Diversity Analysis, Library Design, and Compound Selection " Journal of Chemical Information and Computer Sciences, 38, 1204-1213, (1998), restricted the number of descriptors to 3 to 6 and the number of bins per descriptor to 4 to 7 and excluded a large number of candidate points by treating them as outlying observations. Even with these restrictions, over 80% of cells were empty in an example that they presented with 6⁶ cells.

A discussion is now made of the various existing methods for selecting an experimental design to cover a space of explanatory variables and the deficiencies in these existing methods. The most common designs for selecting diverse molecules are random designs, distance designs, and cell-based binning designs.

The simplest designs are based on random sampling. In fact, most new leads have been discovered through random screening, in which large numbers of compounds are tested for a specific biological activity, and the active compounds are then selected for optimization. S.S. Young et al., "Random Versus Rational - Which is Better for General Compound Screening?", Network Science, wwwjtetsci.org/Science/Screening fearure09 itml (1996), used a constant radius hyperspbere around each randomly selected compound to measure the coverage of the descriptor space. They concluded that, unless a very large number of compounds are used to fill space, randomly selected compounds will cover as much space as carefully selected compounds. On the other hand, if the important dimensions for a particular problem are identified, and if a focused set of compounds is desired, then rational selection should be more effective than random designs.

There are three main types of distance based designs for selecting molecules from chemical databases. R.E. Higgs et al., "Experimental Designs for Selecting Molecules from Large Chemical Databases," Journal of Chemical Information and Computer Sciences, 37, 861-870 (1997), refer to these as "Edge", "Spread" and "Coverage" designs. These methods first define a descriptor distance metric (e.g., Euclidean or Manhattan distance) to measure the similarities or dissimilarities of the molecules, and then find the optimal coverage of the space based on some distance criterion. Edge (D-optimal) designs identify molecules at the edge of the descriptor space that produce minimum variance estimators for parameters in a regression model that is linear in the descriptors (explanatory variables). Spread designs (see, e.g., R.W. Kennard et al., "Computer Aided Design of Experiments," Technoraetrics 11, 137- 148 (1969)) identify a subset of molecules that are maximally dissimilar with respect to each other. Coverage designs (see, e.g., P.J. Zemroch, "Cluster Analysis as an Experimental Design Generator, With Application to Gasoline Blending Experiments," Technometrics 28, 39-49 (1986)) select a subset of molecules that are maximally similar to the candidate set of molecules. The following references provide more detailed descriptions of these designs: RE. Higgs et al., "Experimental Designs for Selecting Molecules from Large Chemical Databases," Journal of Che ical Information and Computer Sciences, 37, 861-870 (1997); M.E. Johnson et al., "Minirnax and Maxirnin Distance Designs," Journal of Statistical Planning and Inference 26. 131-148 (1990); and R. Tobias, SAS QC Software. Volume 1: Usage and Reference, SAS Institute Inc., Cary, N.C., 657-728 (1995).

There arc ihree problems with distance-based designs. First, in general these designs try to find a subset with optimal coverage of the entire descriptor space but pay little attention to the coverage in lower-dimensional subspaces. The low-dimensional coverage (i.e., 1-D.2-D and 3-D) can be quite poor. The following references addressed this issue by incorporating 1-D coverage into their spread designs: M-D. Morris et al., "Bayesian Design and Analysis of Computer Experiments: Use of Derivatives in Surface Prediction,'^* Technometrics 35, 243- 255 (1993); and MJD. Morris et al., "Exploratory Designs for Computational experiments," Journal of Statistical Plarining and Inference, 43, 381-402 (1995). Secondly, descriptors that are unrelated to target activity can have a significant impact on the distribution of the molecules in the space, and can ake the "optimality" of a design irrelevant. Without proper selection of descriptors, these optimal designs are not expected to improve the quality of rational sampling over that of random sampling. Thirdly, the presence of relatively few outlying observations can have significant impact on these designs. Very often this requires removal of many outlying molecules to come to a sensible design.

In the conventional cell-based method, each of the k numerical descriptors is subdivided into m bins of equal size, yielding ra^L cells or bypercubes, and the experimental design chooses at least one molecule from every cell. This method is attractive because it is easy to divide the descriptor space into cells and allocating even a very large dataset to these cells is straightforward. Missing diversity (i.e., empty cells) can easily be identified. The following references disclose cell-based binning methods to compare the relative diversity of molecular databases and to select diverse subsets of molecules: D.J. Cummins et al., "Molecular Diversity in Chemical Databases: Comparison of Medicinal Chemistry Knowledge Bases and Databases of Commercially Available Compounds," Journal of Chemical Information and Computer Sciences, 36, 750-763 (1996); and P-R. Menard et al., "Chemistry Space Metrics in Diversity Analysis, Library Design, and Compound Selection," Journal of Chemical Information and Computer Sciences, 38, 1204-1213 (1998). A problem with many existing cell-based binning methods is that they generate too many cells and many of the cells are empty. Even when k and ra are relatively small, the number of empty cells is often more than the number of nonempty cells for chemistry problems. To reduce the number of empty cells, C i rains et al. (1996) and Menard et al. (1998) restricted the number of descriptors and the number of bins per descriptor. They also excluded many outlying candidate points. Even with these compromises, they reported a large proportion of empty cells. Indeed, a very low cell occupancy is expected by Menard et al. (1998) — they recommended a targeted occupancy of 12-15%.

Two popular space-filling designs, currently only applied to more regular sampling spaces, are Latin hypercube designs (see, e.g., M J). McKay et al., "A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code," Technometrics 21, 239-245 (1979)) and uniform shell designs (see, e.g., D Ji. Doehlert, Uniform Shell Designs," Applied Statistics, 19, 231-239 (1970)). Latin hypercubes have excellent 1-D coverage and are very popular in experiments with computer models. The main problem in applying these methods to compound selection is that for chemical compounds only certain combinations of descriptor values exist.

Some of the other problems related to design and analysis of molecular data include:

1. The model is vague. In many drug design proble s, it is not clear which model is appropriate to relate biological response to molecular properties beyond the assertion that similar objects are more likely to respond similarly. Thus if a collection of chemical objects are described numerically and if a subset is to be selected, then the subset should "fill" the numerical space in some sense — selected objects should be as dissimilar as possible or any candidate not selected should be near a selected object.

2. There is more than one response. Several biological screenings, each designed to detect a specific biological activity, may be in operation within the research division of a pharmaceutical company at any given time. It is hoped that the collective output of these screens will provide enough leads to contribute to the discovery process in a meaningful way. . The samples to be chosen from are often a collection of restricted sampling points and their descriptor values are dependent A standard experimental design procedure assumes that the descriptor space can be represented as a region bounded by a k-dimensional hypercube with any point in the cube being a candidate point. The standard design is not possible for compound selection problems because, even though the space is continuous, the possible molecules are discrete. It is not possible to place a compound at certain positions in the space. One is restricted to the compounds that one has or can make.

4. The number of candidate points (NJ and the number of design points (n<) are large. The number of possible combinations of the samples to be chosen from is so large that it becomes computationally impossible to consider every possible combination in the experimental design - it may take days or weeks or even months to compare every combination. In theory, to identify the optimal design, one needs to examine all possible subsets of size n from the N_c candidate points, thus performing N_c-chose-π,, subset evaluations. In practice, the magnitudes of N and n„ prohibit a full scale optimization. For example, even to choose a small design of 100 points from a very small candidate set of 1000 molecules, there are 6.4xl0¹³⁹ possible subsets. For moderate or large datasets, search algorithms are usually applied to find a very good design, as it is impossible to find the globally best design.

5. The number of descriptors can be very large. Tens to hundreds to thousands of molecule descriptors are possible. This implies a high dimensional problem.

Thus, what is needed is a fast method to select representative objects while providing optimal (or near optimal) space coverage.

Summary of Invention

The present invention provides an efficient and effective system and method for enhancing successful discovery of drugs using high throughput biological screening technology by providing a representative set of compounds to screen.

Embodiments in accordance with the present invention include the following: 1. A cell-based binning method (hereinafter referred to as the "Data Driven" binning method) allows the inclusion of all molecules, generates a high percentage of occupied cells for the candidate set, and provides adequate selection of the molecules to cover the low-dimensional subspaces (typically all one-dimension (1-D), two-dimension (2-D), and three-dimension (3-D) subspaces).

2. A chemical space coverage criterion (hereinafter referred to as the "Uniform Cell Coverage (UCC)" criterion) measures the urήformity of coverage of the molecules selected.

3. A fast exchange design algorithm (hereinafter referred to as the "fast exchange UCC^* algorithm) that minimizes the number of searches of the candidate points and points from the provisional design for exchange, while maximizing the number of exchange-; during each pass through the candidate points. This method is many times faster than previous exchange algorithms and generates designs (representative sets) with good low dimensional coverage properties.

These methods will be described in the detailed description. While this invention will be described with reference to drug research, the methods disclosed herein are also adaptable to many other applications, e.g., selection of patients for clinical trials.

For convenience, the terms "UCC" and "LWY" designs are used interchangeably.

Brief Description of the Drawings

Figure 1 illustrates diagrams of univariate and pairwise plots of the six descriptors for the NCI molecules;

Figure 2 illustrates block diagrams of equal number of cells in 1-D and 2-D subspaces and four types of space filling designs, with nine 1-D cells in the 1-D subspaces and nine 2-D cells (3 by 3) in the 2-D subspaces;

Figure 3 illustrates block diagrams of an Equal Width (EW) bin, an Equal Frequency (EF) bin, and a hybrid bin in accordance with an embodiment of the present invention;

Figure 4 are charts Ulustrating applications of the EW bins and the Hybrid bins that lead to, respectively, many empty bins and zero empty bins;

Figure 5 illustrates diagrams of examples of plots of 1-D and 2-D non-empty cells for the NCI candidate points;

Figure 6 illustrates charts comparing the UCC design to 100 random designs;

Figure 7 illustrates plots of l-D filled cells for the random and the UCC designs

Figure 8 illustrates diagrams comparing 2-D coverage between the random design and the UCC design;

Figure 9 illustrates a flowchart of a method of dividing low dimensions of a high mathematical space into a finite number of bins;

Figure 10 is a flowchart illustrating a method of identifying "good" candidates, in accordance with an embodiment of the present invention;

Figure 11 is a flowchart illustrating a method of identifying design points for exchange, in accordance with an embodiment of the present invention; and

Figure 12 illustrates a block diagram of a computer that can implement the present invention-

Detailed Description Those of ordinary skill in the art will realize that the following description of the preferred embodiments is illustrative only and not in any way limiting. Other embodiments of the invention will readily suggest themselves to those skilled in the art.

Terminology Used and Notation

The following terms and terminology will be used in the description. A brief definition follows each of which is meant to instruct, but not limit, the use of the term as provided.

A Candidate Point is an object that can be tested, for example, a molecule or a patient. A "good" candidate point is one that has different characteristics from those that have already been selected. For distance-based designs, the "best" candidate point is one that is maximally dissimilar to those that have been selected; for cell-based designs, the "best" candidate point is the one that fills the most unfilled low-D cells.

An optimal design or a locally optimal design: if exchanging the "best" candidate and design points yields no further improvement to the design.

Molecule, compound and structure: For simplicity and brevity, the terms "molecule", "compound" and "structure" are used interchangeably. Technically, a compound is a chemical substance with two or more elements. A molecule is the simplest structural unit that displays the characteristic physical and chemical properties of a compound, A structure is the graphical presentation of the bonds and atoms of a compound. In drug discovery, a compound is considered a chemical substance not yet defined, while a molecule is a defined compound. Graphically, a molecule is referred to a chemical structure-

NCI molecules: Molecules from a National Cancer Institute (NCI) chemical database, with the NCI database having over 29,000 molecules and having data in the public domain. The NCI chemical database can be obtained from, for example, the following web site: http://dtp.nci.nih.gov/docs/aids aids data.html. In May 1999, there were 32,110 compounds in the NCI DTP AIDS antiviral screen database. Some of these molecules were removed as their descriptors could not be computed, leaving 29,217 unique molecules. They are described by six continuous variables (descriptors), which measure molecular bonding patterns, and atomic properties such as surface area, charge, hydrogen-bond donor and acceptor ability. The descriptors, called BCUT values, are based on the work by F.R- Burden, "Molecular Identification Number for Substructure Searches," Journal of Chemical Information and Computer Sciences, 29, 225-227 (1989). It was found that structurally similar compounds usually have very sirnilar-BCUT values.

Core98 molecules: Molecules from a proprietary molecular database called Core98. There are 23,056 unique molecules in the Core98 database, and these are described by six BCUT continuous variables.

In general, denote the k continuous descriptors by x j_r, ... . x*_, and let X,. be a candidate set with N_c candidate points. The objective is to choose a representative set of n,, design points, u , to cover the descriptor space occupied by the candidate set

Within the full k-dimensional descriptor space, p-dimensional (p-D) subspace is defined by p of the k descriptors (1 < p ≤ k). For convenience, these subspaces will be referred to as Xi for 1-D subspaces, Xij for 2-D subspaces, and Xijl for 3-D subspaces. For example, XI is a 1-D subspace defined by x,, and X12 is a 2-D subspace formed by x, and x₂.

Cell-Based Approach

We use a number of techniques to keep the cells small, yet limit their number, and to ensure that relatively few cells are empty in the candidate set. First, we focus attention on low- dimensional subspaces, typically all 1-D, 2-D, and 3-D subspaces. By considering no more than three variables at a time, fewer cells are required to represent a subspace. Selecting a design with good coverage of all low-dimensional subspaces is analogous to a two-level fractional factorial design of Resolution IV. Such a design is a complete factorial for any subset of three or fewer variables (see, e.g., Box et al., Statistics for Experimenters, New York: Wiley (1978) p.388) and can estimate all interaction effects if only three factors are found to be important. Secondly, we keep the number of cells constant over all subspaces, avoiding the exponential increase with dimension. Thirdly, to avoid empty cells caused by the sparsity of molecules towards the limits of a descriptor's range, we adopt a data-driven hybrid binning method that makes bins larger towards the extremes.

A descriptor bin is a segment of a descriptor range. Cells are formed by combining descriptor bins. Larger descriptor bins are used for higher-dimensional subspaces in order to keep the number of cells in each subspace constant.

For example, for any 1-D subspace a single descriptor's range could be divided into 729 distinct (or disjoint) intervals to form 729 descriptor bins, (It will be explained below why 729 is a convenient number). As only one descriptor is involved for a 1-D subspace, the 729 bins immediately become the 729 cells for the subspace. For a 2-D subspace, each of the two descriptor ranges could be divided into 27 bins. The 2-D subspace will have 27 x 27 - 729 cells formed by combining the 1-D bins. Similarly, for any 3-D subspace, 9 bins for each descriptor range would give 9 x 9 x 9 = 729 cells.

The number 729 is convenient as it has an integer square and cube root In general, there would typically be ra cells per subspace. For 1-D.2-D, and 3-D subspaces, there would typically be m, m"³, and m^{, s} bins per descriptor, respectively. Thus, convenient values of m are, for example, 2^ = 64, 3^ω - 729 and 4^ω =^» 4096, 5^ = 15625, etc.

For example, suppose we have two descriptors and we want to select nine design points. Figure 2 illustrates how the l-D and 2-D subspaces can each be divided into nine cells. Each of the two 1-D subspaces for descriptors 1 and 2 are divided into nine l-D cells and the 2-D subspace is divided into nine 2-D cells (i.e., 3 by 3). Therefore, there are nine 1-D cells in each 1-D subspace and nine 2-D cells in the 2-D subspace. To illustrate the binning method, four example designs are included in Figure 2 with each of the nine design points denoted by "X". The max-roin design (2^Q0), representing both the Edge and Spread designs, gives very poor 1-D projections (i.e., it covers only three of the nine 1-D cells in each dimension). The random design (205) gives better l-D projections but still does not cover all nine cells in each 1-D subspace and all nine cells in the 2-D subspace. The "good 1-D projections" design (210) covers all nine cells in each 1 -D subspace but covers only three of the 2-D cells. Finally, the "good 1-D/2-D projections" design (215) gives good coverage in all three π O 00/67165

subspaces: Every cell is occupied.

With k descriptors, there are

, +

(!) V 6 6

1-D, 2-D, and 3-D subspaces in total. When k⁵^, for example, there are 7 subspaces: XI, X2. X3, X12, X13, X23 and X123. When k=6, there are 41 subspaces and when k=10, there are 175 subspaces. For larger k, it might be necessary for computational reasons to reduce the number of subspaces by focusing on, for example, only l-D and 2-D subspaces.

Including subspaces of 4-D and higher will usually not be practicaL Chemists generally believe that two molecules must have fairly close values of all critical descriptors for similar biological activity (see, e.g., J.W. McFarland et aL, "On the Significance of Clusters in the Graphical Display of Structure-Activity Data," Journal of Medicinal Chemistry, 29, 505-514 (1986)). This indicates that bins have to be small if one molecule from a bin is to represent the rest. Yet, even with 10 bins per dimension, which is probably too few, there are 10,000 cells per 4-D subspace. Clearly, we would need to choose at least this many molecules if the experimental design is to cover every cell. Thus, it is not possible to give dense coverage of a 4-D subspace with a modest subset of molecules. For analysis, this implies that interaction effects are hopefully limited to no more than three factors.

How big should ra be? Even with the data-driven binning method described next, there will be some cells with no molecules. The proportion of empty cells, which varies from subspace to subspace, will tend to increase with m. In addition, if n_a design points are to be selected, U_o nonempty cells per subspace are typically chosen, so that a good space-filling design can cover distinct nonempty cells. These two considerations suggest that m should be approximately equal to n,, or a little larger.

In an embodiment of the invention, we use a hybrid of two simple-to-implemeni methods, Equal Width (EW) and Equal Frequency (EF), to construct bins. The EW method simply divides a descriptor's range into equal-width intervals. Alternatively, EF bins have cut-points chosen to make the frequency of candidate points approximately equal in each bin. The present invention provides a data-driven, hybrid method that combines the best features of EW and EF bins.

In regions where there is a reasonable density of descriptor values, EW bins are compelling. When a molecule is chosen to represent a bin (and hence a cell), it is the size of the bin that determines the quality of coverage in the descriptor space, not the number of molecules in a bin. Another way of looking at this is that EF bins are very small where there is a high density of candidate molecules. Such regions will be over-represented in an experimental design, to the detriment of coverage in regions where candidates are sparse and bins are wide.

On the other band, outlying or extreme descriptor values may inflate a descriptor's range, making many EW bins empty towards the extremes. This problem is compounded in multiple dimensions: Unless every bin forrning a cell is nonempty, the cell is empty. To avoid this, extreme candidates are sometimes removed from consideration (see, e.g., Cummins et aL (1996) and Menard et al., (1998)). By definition, the EF method has candidate points in every 1-D bin and hence none are empty. Empty cells in 2-D or 3-D subspaces can still arise, but EF bins will tend to have fewer empty cells.

To combine the best features of EW and EF bins, we use a data-driven, hybrid method- EF bins are constructed for the extreme values. For example, the first percent of a descriptor's values can be placed in one bin, with a similar bin for the last one percent. E W bins are then used between these extreme bins. Thus, EW bins predorninate, while the EF method for the extreme values avoids empty bins.

The concept of EF (300) versus EW (305) versus Hybrid (310) bins is illustrated in Figure 3. The nine design points show how the space may be covered using the various binning approaches. Figure 4 similarly compares the distribution of molecules into EW (400) or hybrid (405) bins for the XI subspace of the NCI database. In both cases, 150 bins are used. It is clear that many of the EW bins in the top histogram are empty. In contrast, in the bottom histogram there is one EF bin (the first percentile), 148 EW bins, and then one EF bin (the last percentile), which generates no empty bins. Criteria For Evaluating Coverage

In a conventional cell-based design, there is one set of cells based on all k descriptors. Simply picking a point from each occupied cell guarantees a good coverage design. However, as mentioned, this can generate too many empty cells. Previously, we defined cells based on low-dimensional subspaces to overcome this problem. With more than one subspace, it is no longer straightforward to select a set of candidate points to give good coverage simultaneously in many subspaces. If, say, one point is chosen from each cell in a particular subspace, these points may be unevenly distributed in other subspaces. We now describe two measures of the quality of coverage; the second measure will be used below (Fast Exchange Algorithm section) as an optimization criterion to drive the numerical search for a good experimental design.

We first need some definitions and notation. Let X denote a set of points (molecules) in The descriptor space; X will typically be the entire set of candidate points, X^, or a trial experimental design, X The set X is said to cover cell i in subspace s if at least one of the points falls in that cell. Mathematically, we set up indicator variables c_β(X) taking the value 1 if cell i in subspace s is covered and 0 otherwise.

The first experimental design criterion simply computes the percentage of cells that are covered by a design, averaged over all subspaces. Some cells are not covered by the candidate set, , and so cannot be covered by any choice of design; these cells are eliminated from consideration when computing the criterion.

In subspace s, the percentage of cells covered by a design X_d is:

∑^c-⁽ «>

where the summation is over all cells in the subspace. We can then define the average percentage coverage over, for example, all i-D subspaces as:

where S, is the set of all 1-D subspaces and |S, | is the number of such subspaces. For 2-D subspaces we define P,._D analogously, and so on For example, the average percentage coverage over all 2-D subspaces is:

We can then obtain the average percentage coverage, P. For example, if 1-D, 2-D, and 3-D subspaces are being considered, the P can be calculated as follows:

I _ V\-D ^* -D +-* 3-j )

(4)

The average could also be weighted, for example, by giving more weight to 1-D subspaces. One deficiency of this criterion is that it ignores the distribution of design points in the covered cells. For instance, consider two very different space filling designs: one design has two points in each of 50 cells and the other design has 1 point in 49 of these cells and 51 points in the remaining cell. With respect to these 50 cells, the coverage is 100% for both designs, yet we would prefer the first design as the distribution of points is more uniform. Thus the criterion P is modified to select a design that takes the uniformity of coverage into consideration.

Uniform Cell Coverage (UCC)

Suppose design X_« places njXJ points in cell i of subspace s. If the candidate set X_c does not cover this cell (i.e., c„(χ.) = 0), then n„(X ) has to be 0. For cells that are covered by X,. (i.e., c^Xj = 1), it is preferred that nJ J counts are approximately 1. Thus, ideally, n,-(Xj = c„(XJ for every cell. In subspace s, then, a measure of lack of uniformity is:

Again, these quantities can be accumulated over subspaces. The total lack of uniformity for 1-D subspaces, for example, is:

and U_JJ, can be defined analogously, and so on. For example, the total lack of unifoπnity for 2-D subspaces is:

By averaging with weights across, for example, the 1-D, 2-D, and 3-D subspaces, the uniform cell coverage (UCC) criterion can be calculated as U:

where w„ w₂, and w₃ are weights. In the examples below (Results section), equal weights arc used.

Minimizing U in equation (8) discourages uncovered cells in the design and tends to avoid having more than one design point per cell. This is the criterion used by the optimization method in accordance with an embodiment of the present invention.

The indicator variables c„(X_c) in equation (5) provide the target numbers of points per cell in the UCC criterion. With a simple modification to these targets, a generalized UCC is obtained. For example, suppose that we want two design points in each cell. We can set the target for a cell to 0, 1, or 2 if there are no candidate points, one point, or at least two points, respectively. In the examples of this disclosure, we use equation (5) without modification-

Figure 9 illustrates a flowchart of a method of dividing low dimensions of a high mathematical space into a finite number of bins. The process is as follows:

1. Dctemiine the low-D subspaces to be examined (e.g., 1-D/2-D/3-D subspaces) (step 900).

2. Choose the number of cells in a subspace, say m cells per subspace (e.g., m=64, 729 or 4096) (step 905).

3. Decide the number of extreme points to be included in the first and last bins (e.g., 1% of points in each bin) (step 910).

4. Divide each descriptor's range into m, ^w and m^w bins for, respectively, 1-D, 2-D and 3- D subspaces (i.e., if dimension = l-D, then divide a descriptor's range into m bins; if dimension = 2-D, then divide the range into m"² bins; if dimension - 3-D, then divide the range into n ⁷³ bins) as follows (step 915).

5. For each 1-D subspace, do the following:

5a. Determine the cut points of the first and last bins for each descriptor, say cl and c2 (step

920).

5b. The width of the remaining (m-2) bins are then equal to (c2-cl) (m-2). The cut point for bini is equal to cl + (i-l)*(c2-cl)7(m-2), i = 2, ... , m-1. These generatem 1-D hybrid bins and thus m 1-D cells per ID subspace (step 925).

6. For each 2-D subspace, do the following:

The first ra"² l-D bins form the first 2-D bin; the second ra"* i-D bins form the second 2-D bin, and so on (step 930). These generate m^{, z} 2-D bins and thus m^{, 2}x m"²2-D cells per 2-D subspace (step 935).

7. For each 3-D subspace, do the following:

The first m"³ 1-D bins form the first 3-D bin, the second m^w 1-D bins form the second 3-D bin, and so on (step 940). These generate m"³ 3-D bins and ro^lΛx m"³ x m^l" 3-D cells per 3-D subspace (step 945). Fast Exchange Algorithm

With conventional cell-based binning, which focuses on the one space of all descriptors, choosing a subset of molecules is straightforward: One or more molecules is randomly sampled from each nonempty cell. This strategy does not work for the UCC criterion above, however. Randomly sampling in the nonempty cells of one subspace might give a poor distribution of molecules in another subspace. Thus, an optimization algorithm is needed; we modify the exchange algorithm as disclosed in H.P. Wynn, "Results in the theory and construction of D-optiraal experimental designs," Journal of the Royal Statistical Society B, 34, 133-147 (1972). The adaptations greatly reduce the computational effort, especially when dealing with very large candidate sets.

Wynn's algorithm starts with a random subset of n,, points (an initial design) from the _£ candidates. The optimization criterion is then sequentially improved by a series of exchanges. (Wynn worked with a criterion called D optimality, but we will use UCC.) In each exchange, a point in the candidate set replaces a point in the current design. An exchange is broken down into two steps. First, a point in the candidate Ust is found to add to the current design. The point added from the candidate list is the one with the best value of the design criterion for the modified design of n_«,+ I points. Second, a point in the new design of r^+ 1 points is removed; mis point is chosen to give the best criterion value for the new design of ^points amongst those that are subsets of the .._<,+ 1 points available. These exchanges continue until the criterion cannot be improved.

For moderate to large lists of candidate molecules, Wynn's algorithm can be made much faster. Moreover, our algorithm tends to find a better value of the design criterion. We now describe the adaptations.

Identifying Good Candidates for Exchange

Wynn's algorithm is computationally inefficient for large candidate lists, as it loops through the whole candidate list, X_^, to find only one candidate to add. Moreover, many of the initial n, points will have to be replaced, requiring many loops if n,, is moderately large. The adaptations we first describe are aimed at obtaining many exchanges per X,. loop, to minimize the number of X. loops required. Every time a candidate is visited, we note the improvement

is in the criterion if it were added to the design. Hence, an approximation to the distribution of improvements can also be maintained. As we pass through the candidates, whenever a candidate's change is in the upper tail of this distribution, it is deemed "good" and considered for an exchange. (A similar process will be described below to search for a design point to complete the exchange.) Thus, each X_c loop considers many "good" candidates for exchanges.

Specifically, let δ, denote the improvement (i.e., reduction) in the UCC criterion in Equation (8) if candidate j were added to the current n,, design points to give n„+ 1 points. The algorithm for identifying good candidates, with some explanation in parentheses, is as follows (and described with reference to Figure 10):

1. (Step 1100 in Figure 10) Initialization of the δ distribution. Randomly select a set of candidate points (e.g., 100 candidate points). Compute their δ values, and denote the sorted values by δ_UJ > ... ≥ δ_tJ0Q) Set λ=n_β N_c and δ* - δ_lψI where q = lOOλ. (In Step

1105, if candidate j has δ, > δ*, it will be considered for an exchange. This rule will try approximately n_<, of the N_ς candidates during the first X^ loop, because all n_a initial design points may have to be replaced.)

2. (Step 1105) Loop through the candidates. For j=l,.... N_s do the following steps:

• Compute δ_; and note the value for later use in updating δ* (Step 1105a).

• If δ_j ≥ δ* (Step 1105b), then:

• Try exchanging candidate j with one of the current design points (Step 1105c)).

• If candidate j was exchanged, then

• Set δ_j - -100. (As candidate j is now in the design, introducing it again is undesirable). (Step 11054)

Proceed to step 1105g (j = j + 1) and begin step 1105 again, else

* Replace δ* by δ* + 10λ (step 1 l05e). (A failed exchange suggests that δ* is allowing poor candidates to be considered, i.e., δ* is too small).

If δ_j > δ* (Step 1105b) is not satisfied, then proceed to step 1105g and begin step 1105 again. . (Step 1110) Determine if there is any improvement in the complete loop. If there was no improvement in the criterion in the last X^ loop, then stop.

If there is any improvement in the complete loop, then proceed to step 1105g and begin step 1105 again.

4. (Step 1115) If j=N_c, then update δ* for the next X_c loop. Sort the δ_j values from the last X_c loop and denote them by δ_tI) ≥ ... > δ_(N(.,. Set λ to half the previous value and δ* = δ_(ll), where q - max(lQ, N_< )- G to Step 1105. (Decreasing λ reduces the number of exchanges considered, because fewer exchanges are likely to improve the criterion with successive passes through the list. We always want to consider at least 1Q promising candidates in the next X^ loop, however, to be conservative about teπnination.)

Note that when a good candidate is found in Step 1105, we do not re-start the X_t loop at the beginning. Rather we continue with the next candidate. These "floating" loops allow many exchanges in one X,. loop.

Identifying Design Points for Exchange

Whenever a "good" candidate for inclusion in the design is identified by the rules in the above method, a design point must also be removed if an exchange is to take place. We evaluate the design points and identify a "bad" point, i.e., one that should be removed, using similar rules.

For a fixed candidate j under consideration for inclusion, let Δ, denote the overall improvement in the UCC criterion in Equation (8) if design point i of the n„ current design points is replaced by candidate j. Thus, Δ, includes the δ contribution from adding candidate j. A distribution of Δ, values is maintained, and we implement an exchange as soon as a "good" Δj value is found, rather than search all 11 design points. The details are as follows (and described with reference to Figure 11):

1. (Step 1200 in Figure i 1) Initialization of the Δ distribution. If this is the first search of the design list, then:

• Randomly select a set of design points (e.g.. 100 design points), compute their Δ₍ values, and denote the sorted values by Δtυ - • ■• ≥ Δ ₀₀,

• Set Δ* = Δ_(q), where q = lOOλ, using the λ value in effect for searching the candidate list (Exchanges with Δ, Δ^¥ will be implemented.)

• Set i=l . (Start at the top of the design-point list.)

2. (Step 1205) Compute Δ, and note the value for later use in updating Δ*.

3. (step 1210) If Δ, ≥ A* (see step 1210a), then

• Implement the exchange of design point i with candidate j (step 1210b). Proceed to step 1220. else if all design points have been tried (step 1210c), then

• Let Δ^be the maximum Δ value over all the design points. If A^ ≥ 0 (step 12l0d), then

• Implement the exchange of the design point giving Δ-^with candidate j. (Step I2l0e). Proceed to step 1220. If -. < 0 (step I210d), then proceed to step 1220. If all design points have not been tried (step 1210c), then proceed to step 1215.

4. (Step 1215) If -=11^ then

• Update Δ*. Sort the Δj values from the last X,, loop and denote them by Δ _(l) > ... > A_iτtύy Set Δ* = max(0.01 , Δ ₍₍₎₎), where q = n,,λ , using the λ value in effect for searching the candidate list. (Step 1215a).

• Set i^l (step 1215b); proceed to step 1205. else

• Set i to i+1 (step 1215c). Proceed to step 1205.

5. (Step 1220) If an exchange occurred in steps 1210b and I2l0e or all design points had been tried in step 1210c, then

• Return to searching for the next "good" candidate to add. The next search for a "bad" design point to remove will start at Step 1205 with the current value of i. else

• Go to Step 1205.

Note that in Step 1210, an exchange can occur with Δ,._^ = 0, i.e., it does not change the criterion. Allowing "neutral" exchanges of this type may be useful to break away from a design that is only locally optimal. Updating the UCC criterion

Finally, we describe how the criterion can be efficiently updated when only one point is changed, either when adding a candidate or when removing a design point.

When a point is added to or removed from the design, it will affect only one of the m cells in each subspace. Let z, be the number of design points in the affected cell in subspace s. If we are adding a point, then z, becomes z, -t- 1, and the change to U, in (5) is:

Note that c_Λ(XJ in equation (5) must equal 1 , as a cell must be covered by at least one candidate if a point is to be added (or removed). Similarly, when a design point is removed, the change to U_£ in equation (5) is:

RESULTS

We now apply the above data-driven binning method and above fast design algorithm to select 729 molecules from the 29,217 NCI molecules.

Forming Cells

The distributions of the NCI molecules in 1-D (500) and 2-D (505) projections for all six descriptors are shown in Figure I. To apply the hybrid binning method described above, the first and last percentiles are assigned to EF bins, with EW bins between. There are six l-D, fifteen 2-D, and twenty 3-D subspaces, and each of the 41 subspaces (1-D, 2-D, and 3-D) is divided into 729 cells. Over the 41 subspaces, on average there are 82.3% nonempty cells; the worst subspace X246 is with 63.8% nonempty cells.

Figure 5 shows the locations of the nonempty cells for all 1-D and 2-D subspaces. Inspection of the ID plots reveals that adding a few extra EF bins to x could further increase the proportion of nonempty cells in subspaces formed by x„ but the current binning is acceptable and we do not pursue this. UCC Design Versus Random Designs

As a benchmark, we generate 100 random designs by randomly selecting 729 points from the 29,217 candidates. Their values of the average percentage coverage and uniform cell coverage criteria, P and U in Equations (4) and (8), respectively, are shown in Figure 6. These random designs, on average, cover 45.6% of the cells occupied by the candidate points. The mean U value is 3108.1.

Starting from the first random design, our fast exchange algorithm described above, leads to an improved coverage. The arrows (600 and 605) in Figure 6 show the resulting P and U values. Based on these criteria, the design made possible by the invention (UCC design) has much better 1-D, 2-D, and 3-D coverage than any of the 100 random designs. In terms of U (the smaller the better), the best random design has a value of 2905 compared with 591 (see arrow 600) for the UCC design. In terms of P (maximum 100%), the best random design has 47.1% coverage while the UCC design has 74.1% (see arrow 605).

Figure 7 compares the cell frequencies in 1-D projections for the starting random design (first of the 100) (plots 700 to 705) and the UCC design (plots 730 to 755). Most of the cells for UCC design have one or two design points. The analogous 2-D projections of the design points are shown in Figure 8, where a dot is plotted in a cell if there is at least one design point. (Plot 800 is the 2-D coverage for the random design, while plot 805 is the 2-D coverage for the UCC design). It is obvious from these figures that the UCC design has superior coverage in 1-D and 2-D projections. Similar plots for the 3-D projections show the same pattern.

UCC Design Versus Wynn's Algorithm

Table 1 compares the coverage and number of loops through the candidate set (proportional to computation time) for the UCC design algorithm with those for Wynn's original exchange algorithm (see, H.P. Wynn, "Results in the theory and construction of D-optimal experimental designs," Journal of the Royal Statistical Society B, 34, 133-147 (1972)). It is seen that the fast exchange algorithm produces better values of P and U. Although the UCC design algorithm makes more exchanges, it makes far fewer passes through the candidate and design points. In terras of computer CPU time, the UCC design algorithm took less than approximately one hour while Wynn's method took about 21 hours. These timings relate to implementations in SAS PROC IML on a Pentium 11400MHz computer with 64MB RAM. Faster speeds are undoubtedly possible by converting to, say, C++. To make it feasible to run Wynn's exchange algorithm, we incorporated the fast UCC update described above.

Table I. Coverage and Running Time for a Random Design, the Wynn Exchange Algorithm, and the Fast Exchange Algorithm (UCC).

Design No. of Coverage No. of loops

Algorithm Exchanges U P X^ X„

Random 0 2988.2 45.5 0.0 0.0

Wynn 759 605.3 74.1 759.0 759.0

UCC 1814 591.1 74.6 19.4 446.4

visits to the candidate set and one X„ loop=729 visits to the design set.

The UCC algorithm makes most of the improvement in the first few loops through the candidates. Indeed, for the NCI database, it gave a better U value after five X,. loops (less than about 16 minutes) than that of the final design from the Wynn algorithm. Similar results are obtained with Core98 molecules.

Our design problem is somewhat special for the following reasons. The candidate set of possible explanatory variable combinations is discrete, because only certain compounds can be made. Moreover, the set of discrete points can be large and highly irregular (see for example, Figure 1). It is believed that two compounds must have very similar values of all critical descriptors to have similar properties. Thus, the design needs to cover the space densely. It is clearly impossible to achieve dense coverage in more than three dimensions at a time without an extraordinarily large design. Hence, we have proposed designs that aim for uniform coverage in all l-D, 2-D, and 3-D projections.

The aim of such experimental designs is not just to discover highly active compounds, but to find several structurally different chemical classes. These provide options for further optimization of activity, physical properties, distribution, half-life, toxicity, etc. By covering the descriptor space uniformly, there is more chance of discovering multiple classes.

The design algorithm described above can efficiently deal with tens of thousands of compounds in the candidate set. Much larger sets of compounds will be of interest as technology advances. The algorithm may, for example, be also implemented with multiple processors.

Additional Results

It has been observed that careful selection of screening compounds using existing methods does not increase the hit rate over random selection (see, e.g., S.S. Young, Random Versus Rational - Which is Better for General Compound Screening? Network Science, www.netsci.org/science screenin^/feature09.html ( 1996)). However, if active compounds are uniformly distributed in the descriptor space, which is arguably the case for a new assay, then, the uniform coverage design in accordance with the invention can lead to a higher hit rate than random selection which is expected to over sample compound dense regions. Here we are saying that one can get a higher hit rate because the active areas for a new assay are rather uniform although the compounds are not. Compounds are usually not uniformly distributed as they are made either because of ease of synthesis or following biological activity that will lead to dense clusters and gaps. For example, when a good compound for a particular assay is found, similar analogs are made where only a few of the selected atoms of the compound are changed.

If the active molecules are very evenly distributed over the molecular descriptor space, then the uniform coverage design in accordance with the invention has a better chance of selecting active molecules than a random selection. For example, suppose that there is one active molecule in each region in the space; some regions may have over 1000 molecules and some have few molecules. The UCC design has a better chance of selecting active molecules in the regions with few molecules.

So, does empiricism support this theory? Does selection of an initial screening set by the UCC design lead to a higher hit rate? To assess this we examined the hit rates for the 15 assays performed on the 23,056 Core98 compounds. The results show that the subsets gave slightly higher hit rates than random selection on 86% to 100% of the 14 assays. These positive results support the proposition that uniform coverage designs generate higher hit rates than random designs. These results are consistent with the proposition that regions of high potency are uniformly distributed through chemical space.

Our uniform coverage design algorithm selects diverse subsets of compounds for biological screenings. Using the UCC design algorithm, we selected a subset of 4096 compounds from 23056 Core98 compounds. We repeated the selection 5 times using different starting sets for the design selection. Each started with a different random selection. The biological activity ratings for 15 assays on the 23056 compounds were available from previous work. We examined the hit rates for the 15 assays on each of the 5 sets of 4096 compounds selected and noted the number of times the design gave a higher hit rate than a design selected at random. Random designs will approximate the hit rate for the entire sample so for our comparisons we use the bit rate for the entire sample.

The results are now described. For the 15 assays, the number of active compounds found (Le., compounds with an activity value ≥ 50) and the corresponding hit rates are summarized in Table 2. We will refer to these hit rates as the population or random hit rates. The hit rates for the 5 sets of compounds selected are also included in Table 2. The ?' columns next to the Ηϊt rate' columns indicate whether the hit rate is higher (1) or not (0).

Table 2. Population Hit Rates for 15 Assays on 23056 Compounds and the Corresponding Hit Rates for the 5 Designs.

population 5 S( 5tS Of 4091 5 C(ampound: 3 va* HqaiDe UCC 1 OCC 2 OCC 3 OCC 4 DCC 5

Y ASKa »γ>-S0 Hit race Kiτratu ? : WXrate ? I Kitrare ? ] Hitra e ? ' Hitrate yi assay! 46 0.200 0.317 1 0.293 1 0.317 1 0.293 1 0.293 1 y2 assay.! 34 0.147 0.171 1 0.171 1 0.195 1 0.171 I 0.195 1

*3 assays 33 0.143 0.171 1 U.U24 0 0.146 1 0.073 0 0.098 0 y< assays 60 0.260 0.464 1 0.415 1 0.439 1 0.391 1 0.439 1 ys 4»«ay5 S3 0.230 0.342 1 0.317 1 0.391 1 0.342 I 0.391 1 yδ asβayβ 14 0.061 O.073 1 0.073 1 0.073 1 0,073 1 0,073 1 y assay7 14 0.061 0.122 1 0.122 1 0.122 1 0.122 1 0.122 1 y8 aεsayδ 0 0.000

»9 assay9 13 0.056 0.122 1 0.122 1 0.122 1 0.122 I 0.122 1 ylO -o-sdyio 10 0.043 0.049 1 0.024 0 0.049 1 0.024 0 0.Q49 1 yii assa 11 103 0.447 0.806 l 0.806 1 0.732 1 0.757 1 0.830 1 yl2 a3sάyl2 so 0.217 0.317 I 0.317 1 0.317 1 0.342 I 0.366 1 yl3 αβsayl3 507 2.199 2.930 1 2.856 1 2.905 1 2.881 1 2.856 1 yi4 assayl4 517 2.242 3.0S2 1 3.198 1 3.052 I 3.198 1 3.247 1 yl5 assaylS 10 0.043 0.04S 1 0.O«9 1 0.049 1 0.049 1 0.073 1

Hua-oar of times better 14 12 14 12 13

% out of 14 better 100 86 100 86 93

For assay 8, all compounds had activity values less than 50. This assay is not useful for evaluating the hit rate. Of the remaining 14 assays, the UCC designs have higher hit rates than the hit rate of the entire sample (the expected hit rate of a random design) for 86 to 100% of the assays. The UCC designs consistently led to higher hit rates on 12 of the 14 assays. For Assays Y3 and YlO, the UCC designs yield both higher and lower hit rates.

Small, but consistent, increased hit rates for the UCC designs over random designs were observed. These positive results support the proposition that uniform coverage designs generate higher hit rates than random designs. These results are consistent with the proposition that regions of high potency are uniformly distributed through chemical space.

In summary, the differences between the system and method in accordance with various embodiments of the invention and the "classic" system and method are as follows: 1. Classic space filling designs keep track of the distances between all the design points or between each candidate point and its nearest design point. This becomes computationally expensive as the design gets large and if the number of candidate points is large.

2. Classic space filling designs attempt to fill all dimensions of the space.

3. Classic exchange algorithms typically search the entire candidate hst to find the best point to exchange. They find one exchange point in one loop through the candidate set. The fast update feature significantly reduces the computation cost for measuring the coverage of a design when an exchange is made. Without this feature, for the large candidate and design sizes we consider here, the Wynn algorithm would have taken weeks to months to find a (near) optimal design.

4. Classic designs typically exclude many objects because hey use a mathematical model to infer what will happen in the regions without design points.

5. Classic space filling designs often have to remove outlying observations in an attempt to avoid having many empty bins, or, if clustering is used, having many, very small clusters.

6. Classic designs typically have poor low dimensional coverage.

Application and Implementation of the UCC algorithm

In a preferred embodiment, the method of the present invention is implemented on a digital computer. More specifically, the method is implemented in computer code that provides the necessary instructions for a processor performing the data manipulations necessary to run or execute the method. While the computer having memory and a processor is the preferred device in the invented system, other apparatus may also be used as is well-known by those skilled in the art.

In one embodiment, the present invention is implemented on a computer 1600 as illustrated in Figure 12. The computer 1600 comprises a central processing unit (CPU) 1602 for performing the calculations of the various methods described above, a storage device 1608 for storing data and files that can be retrieved by the processor 1602, an input device 1604 for enabling user interaction with the computer 1600, a display device 1606, and a memory 1619 for storing one or more programs during execution, such as a program that performs at least one of the above methods and or a conventional program GUI OS 1620. Alternatively, the present invention may be implemented, for example, across a network of computers, enabling programs to be run by multiple processors at separate physical locations. Thus, a computer program was created that implemented the method and the software was installed on a computer.

It is also within the scope of the present invention to implement a program or code that can be stored in an electronically-readable medium to permit a computer to perform any of the methods described above.

Thus, while the present invention has been described herein with reference to particular embodiments thereof, a latitude of modification, various changes and substitutions are intended in the foregoing disclosure, and it will be appreciated that in some instances some features of the invention will be employed without a corresponding use of other features without departing from the scope of the invention as set forth.

3P

Claims

What is claimed is:

1. A method of selecting objects from a collection of numerically characterized, candidate objects to cover low dimension projections of a high dimensional mathematical space, the method comprising:

(a) dividing each subspace into a finite number of bins',

(b) selecting a provisional design;

(c) measuring the cell coverage for the provisional design;

(d) finding a candidate point that will cover many uncovered low-dimensional bins in the provisional design;

(e) finding a point in the provisional design wherein the point is in cells covered by other design points;

(f) tracking the distributions of possible changes in both the candidate set and in the developing design, with an exchange being made if a candidate point is in the tail of the distributions;

(g) moving through the entire collection of candidate points to determine improvement of the coverage of the developing design, and finding a point in the provisional design for removal;

(n) moving through the provisional design to determine the effect on the coverage of the developing design upon removal of a point, and evaluating a candidate point for a possible exchange and performing an exchange for a good candidate point; (i) updating the cell counts that change in the provisional design; and (j) performing steps (b) to (i) and selecting the best resulting design.

2. The method of claim 1 wherein the dividing step (a) comprises: dividing each 1-D subspace into N bins.

3. The method of claim I wherein the dividing step (a) comprises: dividing each 2-D subspace into N ~ N* x N^Λ rectangular bins.

4. The method of claim I wherein the dividing step (a) comprises: dividing each 3-D subspace is into N = N"³ x N^ιa x N^l/Ϊ cubic bins.

5. The method of claim 1 wherein the selecting step (b) comprises: selecting N objects randomly from the candidate objects.

6. The method of claim 1 wherein the measuring step (c) comprises: counting in each low dimensional space the numbers of cells that can be occupied and the numbers of objects in each occupied ceil.

7. The method of claim 1 wherein the finding step (d) includes using a floating loop to loop through the candidate set to allow many exchanges in one pass through the floating loop.

8. The method of claim I wherein the finding step (e) includes using a floating loop to loop through the list of design points in the developing design to allow many exchanges in one pass through the floating loop.

9. An electronically-readable medium storing a program for permitting a computer to enable a method of selecting objects from a collection of numerically characterized, candidate objects to cover low dimension projections of a high dimensional mathematical space, the method comprising:

(a) dividing each subspace into a finite number of bins;

(b) selecting a provisional design;

(c) measuring the cell coverage for the provisional design;

(d) finding a candidate point that will cover many uncovered low-dimensional bins in the provisional design;

(e) finding a point in the provisional design wherein the point is in cells covered by other design points;

(f) tracking the distributions of possible changes in both the candidate set and in the developing design, with an exchange being made if a candidate point is in the tail of the distributions;

(g) moving through the entire collection of candidate points to determine improvement of the coverage of the developing design, and finding a point in the provisional design for removal;

(h) moving through the provisional design to determine the effect on the coverage of the developing design upon removal of a point, and evaluating a candidate point for a possible exchange and performing an exchange for a good candidate point; (i) updating the cell counts that change in the provisional design; and (j) performing steps (b) to (i) and selecting the best resulting design.

0. A computer system for enabling a method of selecting objects from a collection of numerically characterized, candidate objects to cover low dimension projections of a high dimensional mathematical space, the computer system comprising: a processor; a memory coupled to the processor and storing therein a program executable by the processor for peiφrrmng the steps of:

(a) dividing each subspace into a finite number of bins;

(b) selecting a provisional design;

(c) measuring the cell coverage for the provisional design;

(d) finding a candidate point that will cover many uncovered low-dimensional bins in the provisional design;

(e) finding a point in the provisional design wherein the point is in cells covered by other design points;

(f) tracking the distributions of possible changes in both the candidate set and in the developing design, with an exchange being made if a candidate point is in the tail of the distributions;

(g) moving through the entire collection of candidate points to determine improvement of the coverage of the developing design, and finding a point in the provisional design for removal;

(h) moving through the provisional design to determine the effect on the coverage of the developing design upon removal of a point, and evaluating a candidate point for a possible exchange and performing an exchange for a good candidate point; (i) updating the cell counts that change in the provisional design; and (j) performing steps (b) to (i) and selecting the best resulting design.

U . A cell-based binning method for enabling design points to cover all low-dimensional subspaces of a descriptor space, the method comprising: constructing an Equal Frequency bin at extreme values of a descriptor, and constructing Equal Width bins in between the Equal Frequency bins.

12. An electronically-readable medium storing a program for permitting a computer to enable a method for enabling design points to cover all low-dimensional subspaces of a descriptor space, the method comprising: constructing an Equal Frequency bin at extreme values of a descriptor, and constructing Equal Width bins in between the Equal Frequency bins.

13. A computer system for enabling a method for enabling design points to cover all low- dimensional subspaces of a descriptor space, the computer system comprising: a processor; a memory coupled to the processor and storing therein a program executable by the processor for performing the steps of: constructing an Equal Frequency bin at extreme values of a descriptor; and constructing Equal Width bins in between the Equal Frequency bins.

14. A method of selecting objects from a collection of numerically characterized, candidate objects to cover low dimension projections of a high dimensional mathematical space, the method comprising: determining the low-D subspaces to be examined; choosing the number (m) of cejls in a subspace; deterrnining the number of extreme points to be included in the first bin and last bin; dividing each descriptor's range into ra, ra"² and m^w bins for i-D, 2-D and 3-D subspaces, respectively; for each 1-D subspace, determining the cut points of the first bin and last bin for each descriptor, say cl and c2 (step 920) and generating m 1-D bins and m 1-D cells per ID subspace; for each 2-D subspace, forming a first 2-D bin from the first m ² 1-D bins; forming a second 2-D bin from the second ra"² l-D bins, and generating m"²2-D bins and m^w x m^,/22- D cells per 2-D subspace; for each 3-D subspace, forming a first 3-D bin from the first ^w 1-D bins, forming a second 3-D bin from the second m"³ 1-D bins, and generating m"³ 3-D bins and ra"³ x m^ιn x ra'^Λ 3-D cells per 3-D subspace.

15. An electronically-readable medium storing a program for permitting a computer to enable a method of selecting objects from a collection of numerically characterized, candidate objects to cover low dimension projections of a high dimensional mathematical space, the method comprising: determining the low-D subspaces to be examined; choosing the number (ra) of cells in a subspace; determining the number of extreme points to be included in the first bin and last bin; dividing each descriptor's range into m, ra"² and m^w bins for 1-D, 2-D and 3-D subspaces, respectively; for each 1-D subspace, determining the cut points of the first bin and last bin for each descriptor, say cl and c2 (step 920) and generating m 1-D bins and m 1-D cells per ID subspace; for each 2-D subspace, forming a first 2-D bin from the first ra"² 1-D bins; forming a second 2-D bin from the second ra^w 1-D bins, and generating m"³2-D bins and ra^{, 2} in"²2- D cells per 2-D subspace; for each 3-D subspace, forming a first 3-D bin from the first m"³ 1-D bins, forming a second 3-D bin from the second m"⁵ 1-D bins, and generating ra^{, J} 3-D bins and ra"³ x m^w x m"¹ 3-D cells per 3-D subspace.

16. A computer system for enabling a method of selecting objects from a collection of numerically characterized, candidate objects to cover low dimension projections of a high dimensional mathematical space, the computer system comprising: a processor; a memory coupled to the processor and storing therein a program executable by the processor for performing the steps of: determining the low-D subspaces to be examined; choosing the number (ra) of cells in a subspace; deteπmning the number of extreme points to be included in the first bin and last bin; dividing each descriptor's range into ra, m^w and m¹* bins for 1-D, 2-D and 3-D subspaces, respectively; for each 1-D subspace, determining the cut points of the first bin and last bin for each descriptor, say cl and c2 (step 920) and generating ra 1-D bins and m 1-D cells per ID subspace; for each 2-D subspace, forming a first 2-D bin from the first m"² 1-D bins; forming a second 2-D bin from the second m^{1 2} i-D bins, and generating m"¹2-D bins and m"¹ x m^,/22- D cells per 2-D subspace; for each 3-D subspace, forming a first 3-D bin from the first ra" 1-D bins, forming a second 3-D bin from the second m^IΛ 1-D bins, and generating in"³3-D bins and ra"³ x m^,/J x m^lfl 3-D cells per 3-D subspace.

17. A method of measuring uniformity of coverage of selected molecules, comprising: measuring a lack of uniformity in each subspace; and calculating an average of the lack of uniformity in each subspace.

18, The method of claim 17 wherein the lack of uniformity for 1-D subspaces is:

U ^ui~P - ^ i ,

where U_s is a measure of lack of uniformity is in subspace s, U, - [n,_;(X_d ) - c_tl (X_e jf and

# where design X_a places 0 if the candidate set X^ does not cover cell i, cJ c) ⁼ I for cell i covered by candidate set X^, Si is the set of all 1-D subspaces, and I S,| is the number of 1-D subspaces.

19. The method of claim 18 where the lack of uniformity for 2-D subspace is:

_{Tj =} s eS__

where S, is the set of all 2-D subspaces I Sj is the number of 2-D subspaces.

0. The method of claim 19 where the lack of uniformity for 3-D subspace is:

where S₃ is the set of all 3-D subspaces | Sj is the number of 3-D subspaces.

21. The method of claim 20 wherein the weighted average of the lack of uniformity over all subspaces is: r_j _ føt/i-o + w,£/₂_o + ψ -o)

where w„ Wj, and _s are weights.

22. An electronically-readable medium storing a program for permitting a computer to enable a method of measuring uniformity of coverage of selected molecules, the method comprising: measuring a lack of uniformity in each subspace; and calculating an average of the lack of uniformity in each subspace.

23. A computer system for enabling a method of measuring uniformity of coverage of selected molecules, the computer system comprising: a processor; a memory coupled to the processor and storing therein a program executable by the processor for performing the steps of: measuring a lack of uniformity in each subspace; and calculating an average of the lack of uniformity in each subspace.

24. A method of rairuhuzing the number of searches of candidate points and maximizing the number of exchanges during each pass through the candidate points in a provisional design, the method comprising:

(a) identifying good candidate points for exchange by keeping track of the distribution of improvement of coverage by adding a point to the developing design and noting if a point is in the tail of the distribution; (b) identifying design points for exchange by keeping track of the distribution of improvement of coverage by exchanging a candidate point with a point in the developing design and noting if the improvement of the design is in the tail of this distribution; and

(c) updating a uniform cell coverage criterion when a candidate point is added or when a design point is removed from the provisional design.

25. The method of claim 24 wherein the identifying step (a) includes using a floating loop to loop through the candidate set to allow many exchanges in one pass through the candidate set.

26. The method of claim 24 wherein the identifying step (b) includes using a floating loop to loop through the design points to allow many exchanges in one pass through the design points.

27. An electronically-readable medium storing a program for perrmtting a computer to enable a method of minimizing the number of searches of candidate points and maximizing the number of exchanges during each pass through the candidate points in a provisional design, the method comprising: identifying good candidate points for exchange by keeping track of the distribution of improvement of coverage by adding a point to the developing design and noting if a point is in die tail of the distribution; identifying design points for exchange by keeping track of the distribution of improvement of coverage by exchanging a candidate point with a point in the developing design and noting if the improvement of the design is in the tail of this distribution; and updating a uniform cell coverage criterion when a candidate point is added or when a design point is removed from the provisional design.

28. A computer system for enabling a method of measuring uniformity of coverage of selected molecules, the computer system comprising: a processor, a memory coupled to the processor and storing therein a program executable by the processor for performing the steps of: identifying good candidate points for exchange by keeping track of the distribution of improvement of coverage by adding a point to the developing design and noting if a point is in the tail of the distribution; identifying design points for exchange by keeping track of the distribution of improvement of coverage by exchanging a candidate point with a point in the developing design and noting if the improvement of the design is in the tail of this distribution; and upOating a uniform cell coverage criterion when a candidate point is added or when a design point is removed from the provisional design.

29. The method of claim I wherein the selecting step (b) wherein the selecting step (b) comprises fixing specific candidate points in th* provisional design and selecting the remaining points at random.

30. The method of claim 1 wherein the step (f) comprises fixing specific points in the provisional design and not allowing those points to be exchanged out of the design.

31. The method of claim 1 wherein the steps (a) to (j) are computer processed and distributed over multiple processors.

32. The method of claim 11 wherein the steps are computer processed and distributed over multiple processors.

33. The method of claim 14 wherein the steps are computer processed and distributed over multiple processors.

34. The method of claim 16 wherein the steps are computer processed and distributed over multiple processors.

35. The method of claim 17 wherein the steps are computer processed and distributed over multiple processors.

36. The method of claim 24 wherein the steps are computer processed and distributed over multiple processors.