Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N20/00—Machine learning
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N5/00—Computing arrangements using knowledge-based models
  • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N7/00—Computing arrangements based on specific mathematical models
  • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks

Definitions

• the field of this invention is the application of classification tree models incorporating Bayesian analysis to the statistical prediction of binary outcomes where the binary outcome is estrogen receptor status.
• Bayes's law which states that the posterior probability of a parameter p is proportional to the prior probability of parameter p multiplied by the likelihood of p derived from the data collected.
• This increasingly popular methodology represents an alternative to the traditional (or frequentist probability) approach: whereas the latter attempts to establish confidence intervals around parameters, and/or falsify a-priori null-hypotheses, the Bayesian approach attempts to keep track of how a-priori expectations about some phenomenon of interest can be refined, and how observed data can be integrated with such a-priori beliefs, to arrive at updated posterior expectations about the phenomenon.
• Bayesian analysis have been applied to numerous statistical models to predict outcomes of events based on available data. These include standard regression models, e.g.
• binary regression models as well as to more complex models that are applicable to multi-variate and essentially non-linear data.
• Another such model is commonly known as the tree model which is essentially based on a decision tree.
• Decision trees can be used in clarification, prediction and regression.
• a decision tree model is built starting with a root mode, and training data partitioned to what are essentially the "children" modes using a splitting rule. For instance, for clarification, training data contains sample vectors that have one or more measurement variables and one variable that determines that class ofthe sample. Narious splitting rules have been used; however, the success of the predictive ability varies considerably as data sets become larger.
• This invention discusses the generation and exploration of classification tree models, with particular interest in problems involving many predictors. Problems involving multiple predictors arise in situations where the prediction of an outcome is dependent on the interaction of numerous factors (predictors), such as the prediction of clinical or physiological states using various forms of molecular data.
• predictors such as the prediction of clinical or physiological states using various forms of molecular data.
• One motivating application is molecular phenotyping using gene expression and other forms of molecular data as predictors of a clinical or physiological state.
• the invention addresses the specific context of a binary response Z and many predictors xi; in which the data arises via case-control design, i.e., the numbers of 0/1 values in the response data are fixed by design.
• case-control design i.e., the numbers of 0/1 values in the response data are fixed by design.
• the invention elaborates on a Bayesian analysis of this particular binary context, with several key innovations.
• the analysis of this invention addresses and incorporates case-control design issues in the assessment of association between predictors and outcome with nodes of a tree. With categorical or continuous covariates, this is based on an underlying non-parametric model for the conditional distribution of predictor values given outcomes, consistent with the case-control design.
• An innovative element ofthe invention is the implementation of a tree-spawning method to generate multiple trees with the aim of finding classes of trees with high marginal likelihoods, and where the prediction is based on model averaging, i.e., weighting predictions of trees by their implied posterior probabilities.
• the advantage of the Bayesian approach is that rather than identifying a single "best" tree, a score is attached to all possible trees and those trees which are very unlikely are excluded. Posterior and predictive distributions are evaluated at each node and at the leaves of each tree, and feed into both the evaluation and inte ⁇ retation tree by tree, and the averaging of predictions across trees for future cases to be predicted.
• an embodiments that concerns gene expression profiling using DNA microarray data as predictors of a clinical states in breast cancer.
• the clinical state is estrogen receptor ("ER") prediction.
• ER estrogen receptor
• the example of ER status prediction demonstrates not only predictive value but also the utility of the tree modeling framework in aiding exploratory analysis that identify multiple, related aspects of gene expression patterns related to a binary outcome, with some interesting interpretation and insights.
• This embodiment also illustrates the use of metagene factors - multiple, aggregate measures of complex gene expression patterns - in a predictive modeling context.
• model sensitivity to changes in selected subsets of predictors are ameliorated though the generation of multiple trees, and relevant, data-weighted averaging over multiple trees in prediction.
• the development of formal, simulation-based analyses of such models provides ways of dealing with the issues of high collinearity among multiple subsets of predictors, and challenging computational issues.
• Figure 1 Three ER related metagenes in 49 primary breast tumors. Samples are denoted by blue (ER negative) and red (ER positive), with training data represented by filled circles and validation data by open circles.
• Figure 2 Three ER related metagenes in 49 primary breast tumors. All samples are represented by index number in 1-78. Training data are denoted by blue (ER negative) and red (ER positive), and validation data by cyan (ER negative) and magenta (ER positive).
• Figure 2 Honest predictions of ER status of breast tumors. Predictive probabilities are indicated, for each tumor, by the index number on the vertical probability scale, together with an approximate 90% uncertainty interval about the estimated probability. All probabilities are referenced to a notional initial probability (incidence rate) of 0.5 for comparison. Training data are denoted by blue (ER negative) and red (ER positive), and validation data by cyan (ER negative) and magenta (ER positive).
• Figure 4 Table of 491 ER metagenes in initial (random) order.
• Figure 5 Table of 491 ER metagenes ordered in terms of nonlinear association with ER status.
• Model Context and Methodology Data ⁇ Zi, x, ⁇ (i ⁇ 1, . . ., ) are available on a binary response variable Z and ap — dimensional covariate vector x: The 0/1 response totals are fixed by design.
• Each predictor variable x j could be binary, discrete or continuous.
• Bayes ' factor measures of association At the heart of a classification tree is the assessment of association between each predictor and the response in subsamples, and we first consider this at a general level in the full sample. For any chosen single predictor x; a specified threshold _ on the levels of x organizes the data into the 2 x2 table.
• Bayes' factor As a Bayes' factor, this is calibrated to a likelihood ratio scale. In contrast to more traditional significance tests and also likelihood ratio approaches, the Bayes' factor will tend to provide more conservative assessments of significance, consistent with the general conservative properties of proper Bayesian tests of null hypotheses (See Sellke, T., Bayarri, M.J. and Berger, J.O., Calibration of p_values for testing precise null hypotheses, The American Statistician, 55, 62- 71, (2001) and references therein).
• each probability ⁇ T is a non-decreasing function of T, a constraint that must be formally represented in the model.
• the key point is that the beta prior specification must formally reflect this.
• the sequence of beta priors, Be(a 7 , b ⁇ ) as T varies, represents a set of marginal prior distributions for the corresponding set of values ofthe cdfs.
• the threshold-specific beta priors are consistent, and the resulting sets of Bayes' factors comparable as T varies, under a Dirichlet process prior with the betas as margins.
• the required constraint is that the prior mean values m ⁇ are themselves values of a cumulative distribution function on the range of ⁇ , one that defines the prior mean of each ⁇ ⁇ as a function.
• Bayes' factors of 2.2,2.9,3.7 and 5.3 correspond, approximately, to probabilities of .9, .95, .99 and .995, respectively.
• This guides the choice of threshold, which may be specified as a single value for each level ofthe tree.
• the Bayes' factor measure will always generate less extreme values than corresponding generalized likelihood ratio tests (for example), and this can be especially marked when the sample sizes 0 and i are low.
• the propensity to split nodes is always generally lower than with traditional testing methods, especially with lower samples sizes, and hence the approach tends to be more conservative in extending existing trees.
• Any node in the tree is labelled numerically according to its "parent" node; that is, a node , splits into two children, namely the (left, right) children (2 + 1; 2j + 2):
• the candidates nodes are, from left to right, as 2 m _ 1; 2 m ; : : : ; 2 m+1 - 2.
• Inference and prediction involves computations for branch probabilities and the predictive probabilities for new cases that these underlie. We detail this for a specific path down the tree, i.e., a sequence of nodes from the root node to a specified terminal node.
• These are uncertain parameters and, following the development of Section 2.1, have specified beta priors, now also indexed by parent node j, i.e., Be(a ⁇ , j , b ⁇ , j ). Assuming the node is split, the two sample Bernoulli setup implies conditional posterior distributions for these branch probability parameters: they are independent with posterior beta distributions
• the predictor profile of this new case is such that the implied path traverses nodes 0, 1, 4, 9, terminating at node 9.
• This path is based on a (predictor, threshold) pair ( ⁇ o, ⁇ 0 ) that defines the split ofthe root node, ( ⁇ i, T ⁇ )that defines the split of node 1, and ( ⁇ 4 , ⁇ 4 ) that defines the split of node 4.
• the new case follows this path as a result of its predictor values, in sequence:
• Pr(Z * 0)
• Prediction follows by estimating 7T* based on the sequence of conditionally independent posterior distributions for the branch probabilities that define it. For example, simply "plugging-in" the conditional posterior means of each ⁇ . will lead to a plug-in estimate of ⁇ * and hence it*.
• the full posterior for 7T* is defined implicitly as it is a function ofthe ⁇ .. Since the branch probabilities follow beta posteriors, it is trivial to draw Monte Carlo samples ofthe ⁇ . and then simply compute the corresponding values of ⁇ * and hence 7r* to generate a posterior sample for summarization. This way, we can evaluate simulation-based posterior means and uncertainty intervals for ⁇ * that represent predictions of the binary outcome for the new case.
• the tree generation can spawn multiple copies ofthe "current" tree, and then each will split the current node based on a different threshold for this predictor.
• multiple trees may be spawned this way with the modification that they may involve different predictors.
• the forward generation process allows easily for the computation ofthe resulting relative likelihood values for trees, and hence to relevant weighting of trees in prediction.
• the overall marginal likelihood function for the tree is then the product of component marginal likelihoods, one component from each of these split nodes.
• the overall marginal likelihood value is the product of these terms over all nodes j that define branches in the tree. This provides the relative likelihood values for all trees within the set of trees generated. As a first reference analysis, we may simply normalise these values to provide relative posterior probabilities over trees based on an assumed uniform prior. This provides a reference weighting that can be used to both assess trees and as posterior probabilities with which to weight and average predictions for future cases.
• Example 1 Metagene Expression Profiling to Predict Estrogen Receptor Status of Breast Cancer Tumors
• This example illustrates not only predictive utility but also exploratory use ofthe tree analysis framework in exploring data structure.
• the tree analysis is used to predict estrogen receptor ("ER") status of breast tumors using gene expression data.
• Prior analyses of such data involved binary regression models which utilized Bayesian generalized shrinkage approaches to factor regression.
• prior statistical models involved the use of probit linear regression linking principal components of selected subsets of genes to the binary (ER positive/negative) outcomes. See West, M., Blanchette, C, Dressman, H., Ishida, S., Spang, R., Zuzan, H., Marks, J.R. and Nevins, J.R. Utilization of gene expression profiles to predict the clinical status of human breast cancer. Proc. Natl. Acad.
• Metagenes are useful aggregate, summary measures of gene expression profiles.
• the evaluation and summarization of large-scale gene expression data in terms of lower dimensional factors of some form is utilized for two main pu ⁇ oses: first, to reduce dimension from typically several thousand, or tens of thousands of genes to a more practical dimension; second, to identify multiple underlying "patterns" of variation across samples that small subsets of genes share, and that characterize the diversity of patterns evidenced in the full sample.
• a cluster-factor approach is used here to define empirical metagenes. This defines the predictor variables x utilized in the tree model.
• Metagenes can be obtained by combining clustering with empirical factor methods.
• the metagene summaries used in the ER example in this disclosure, are based on the following steps. Assume a sample of n profiles of p genes; Screen genes to reduce the number by eliminating genes that show limited variation across samples or that are evidently expressed at low levels that are not detectable at the resolution ofthe gene expression technology used to measure levels. This removes noise and reduces the dimension ofthe predictor variable; Cluster the genes using kjneans, correlated-based clustering. Any standard statistical package may be used. This analysis uses the xcluster software created by Gavin Sherlock (http://genomewww.stanford.edu/sherlock/cluster.html).
• a large number of clusters are targeted so as to capture multiple, correlated patterns of variation across samples, and generally small numbers of genes within clusters; Extract the dominant singular factor (principal component) from each ofthe resulting clusters.
• any standard statistical or numerical software package may be used for this; this analysis uses the efficient, reduced singular value decomposition function ("SVD") in the Matlab software environment (http://www.mathworks.com/products/matlab).
• the original data was developed using Affymetrix arrays with 7129 sequences, of which 7070 were used (following removal of Affymetrix controls from the data.).
• the expression estimates used were log2 values ofthe signal intensity measures computed using the dChip software for post-processing Affymetrix output data (See Li, C. and Wong, W.H. Model-based analysis of oligonucleotide arrays: Expression index computation and outlier detection. Proc. Natl. Acad. Sci, 98, 31-36 (2001), and the software site http://www.biostat.harvard.edu/complab/dchip/).
• the corresponding p metagenes were then evaluated as the dominant singular factors of each of these cluster, as referenced above. See Figures 4-5 that provide tables detailing the 491 metagenes.
• the data comprised 40 training samples and 9 validation cases. Among the latter, 3 were initial training samples that presented conflicting laboratory tests ofthe ER protein levels, so casting into question their actual ER status; these were therefore placed in the validation sample to be predicted, along with an initial 6 validation cases selected at random. These three cases are numbers 14, 31 and 33.
• the color coding in the graphs is based on the first laboratory test (immunohistochemistry).
• the current tree model identifies several metagene patterns that together combine to define an ER profile of tumors, and that when displayed as in Figures 1 and 2 isolate these three cases as quite clearly consistent with their designated ER negative status in some aspects, yet conflicting and much more in agreement with the ER positive patterns on others.
• Metagene 347 is the dominant ER signature; the genes involved in defining this metagene include two representations ofthe ER gene, and several other genes that are coregulated with, or regulated by, the ER gene. Many of these genes appeared in the dominant factor in the regression prediction. This metagene strongly discriminates the ER 11 negatives from positives, with several samples in the mid-range.
• this metagene shows up as defining root node splits in many high-likelihood trees.
• This metagene also clearly defines these three cases - 16, 40 and 43 - as appropriately ER negative.
• a second ER associated metagene, number 352 also defines a significant discrimination. In this dimension, however, it is clear that the three cases in question are very evidently much more consistent with ER positives; a number of genes, including the ER regulated PS2 protein and androgen receptors, play roles in this metagene, as they did in the factor regression; it is this second genomic pattern that, when combined together with the first as is implicit in the factor regression model, breeds the conflicting information that fed through to ambivalent predictions with high uncertainty.
• the free model analysis here identifies multiple interacting patterns and allows easy access to displays such as those shown in Figures 1 to 3 that provide insights into the interactions, and hence to inte ⁇ retation of individual cases.
• predictions based on averaging multiple frees are in fact dominated by the root level splits on metagene 347, with all frees generated extending to two levels where additional metagenes define subsidiary branches. Due to the dominance of metagene 347, the three interesting cases noted above are perfectly in accord with ER negative status, and so are well predicted, even though they exhibit additional, subsidiary patterns of ER associated behaviour identified in the figures.
• Figure 6 displays summary predictions.
• the 9 validation cases are predicted based on the analysis ofthe full set of 40 training cases.
• Predictions are represented in terms of point predictions of ER positive status with accompanying, approximate 90% intervals from the average of multiple free models.
• the training cases are each predicted in an honest, cross-validation sense: each tumor is removed from the data set, the tree model is then refitted completely to the remaining 39 training cases only, and the hold-out case is predicted, i.e., treated as a validation sample. Excellent predictive performance is observed for both these one-at-a-time honest predictions of training samples and for the out of sample predictions ofthe 9 validation cases.
• One ER negative, sample 31 is firmly predicted as having metagene expression patterns completely consistent with ER positive status. This is in fact one ofthe three cases for which the two laboratory tests conflicted.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• Software Systems (AREA)
• General Physics & Mathematics (AREA)
• Artificial Intelligence (AREA)
• Data Mining & Analysis (AREA)
• Computing Systems (AREA)
• General Engineering & Computer Science (AREA)
• Mathematical Physics (AREA)
• Evolutionary Computation (AREA)
• Probability & Statistics with Applications (AREA)
• Computational Linguistics (AREA)
• Algebra (AREA)
• Computational Mathematics (AREA)
• Mathematical Analysis (AREA)
• Mathematical Optimization (AREA)
• Pure & Applied Mathematics (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Medical Informatics (AREA)
• Measuring Or Testing Involving Enzymes Or Micro-Organisms (AREA)

Applications Claiming Priority (3)

Publications (1)

Family

ID=32312860

Family Applications (1)

Country Status (3)

• 2002
  • 2002-11-12 WO PCT/US2002/038216 patent/WO2004044839A2/en not_active Ceased
  • 2002-11-12 EP EP02808139A patent/EP1567981A1/de not_active Withdrawn
  • 2002-11-12 AU AU2002368346A patent/AU2002368346A1/en not_active Abandoned

Also Published As

Similar Documents

Legal Events