WO2005106757A2

WO2005106757A2 - Selection of differentially expressed genes

Info

Publication number: WO2005106757A2
Application number: PCT/GB2005/001615
Authority: WO
Inventors: Sachi Nandan Mukherjee
Original assignee: Oxford University Innovation Ltd
Current assignee: Oxford University Innovation Ltd
Priority date: 2004-04-28
Filing date: 2005-04-27
Publication date: 2005-11-10
Anticipated expiration: 2006-10-28
Also published as: GB0409505D0; WO2005106757A3

Abstract

Selection of differentially expressed genes uses an algorithm performed on an overall dataset comprising plural sets of microarray data representing expression levels of a plurality of genes to rank the genes by relevance on the basis of a statistical measure of the differential between the expression levels for two types of biological sample. An optimal algorithm is selected from a plurality of algorithms without making assumptions about the nature of the data by creating a plurality of trial datasets drawn from the overall dataset, performing each algorithm on the trial datasets and calculating the consistency between the trial results sets to select the algorithm providing the highest consistency. The selected algorithm is applied to the overall dataset to select a number of genes. Such selection of the optimal algorithm improves the selection of relevant genes which may then be made the subject of further, detailed experimental study.

Description

Selection of Differentially Expressed Genes The present invention relates to, in the field of bioinformatics, the selection of differentially expressed genes from microarray data. The present invention may also be applied in other fields to the selection of objects from data representing sampled values for those objects. Information coded and stored in genes is utilised in a dynamic manner by living cells. The series of steps from the information represented by the gene to an active product in the form of a protein includes an intermediary molecule messenger RNA (mRNA). Microarrays are a recently developed technology by which it is possible to experimentally derive data representing expression levels of a large number of mRNAs and hence the expression levels of a plurality of the corresponding genes in a biological sample. These expression levels effectively correspond to the activity of the gene in the sample. Microarray experiments can therefore help to identify genes which are relevant to a biological phenomenon such as a disease or a physiological process. In particular, one can compare the expression level of each gene to two types of biological sample in which the phenomenon is different, for example one sample of diseased tissue and another sample of healthy tissue. The gene which is relevant to the phenomenon being studied is generally expected to have a different level of activity for the two types of biological sample. Such genes which have a differential between the expression levels for the two types of biological sample are said to be differentially expressed. However, the analysis of microarray data is a very challenging task. Firstly, this is because of the high noise levels in the experimentally derived microarray data which arises from the inherent variability in the experimental process and underlying biological systems. Secondly, this is because of the relatively small number of sets of microarray data forming the dataset in practice, these being limited by the time and cost of the experimental procedure. Thirdly, there is massive dimensionality, i.e. a large number of genes. Accordingly, as a basis for gene selection, algorithms are performed on the dataset of microarray data to rank the genes on the basis of a statistical measure of the differential between the expression levels for the two types of biological sample.

A group of genes ranked as most relevant are selected. The genes selected using the algorithm are then experimentally studied using other, detailed techniques. Such studies are expensive and time consuming, so it is important for the gene selection to be as good as possible. A large number of such algorithms employing different statistical and computational techniques have been proposed. Different algorithms are known to be optimal in different cases, depending on the statistical properties of the biological system being examined. Conversely, no algorithm is optimal in all cases, so it is necessary to select an algorithm for a given dataset. Much work has been done on the properties of individual algorithms, which can in principle assist in selection of an algorithm based on any knowledge of statistical properties of biological system being examined. However, in general, these statistical properties are partially or totally unknown a priori, so in practice it is unclear what algorithm is optimal. Assumptions about the dataset may be incorrect, resulting in use of an inappropriate algorithm. For example, a common algorithm is to log-transform the microarray data and to apply a t-test. Under some conditions, this procedure is effective, but under certain other conditions this procedure can fail very badly and other algorithms can give far better results. Yet in practice it is difficult or impossible to know whether or not these very conditions are present for a particular experimentally derived dataset. Theoretical research has shown that selection of a sub-optimal algorithm for a given dataset can have a severe impact on error rates in gene selection. Such errors carry with them identifiable costs. That is, false positives lead to resources being wasted in the further, detailed experimental study of irrelevant genes. Conversely, false negatives may lead to missed opportunities by abandonment of further, detailed experimental study of relevant genes. It would therefore be desired to improve the selection of algorithms for use in performing gene selection. Problems of the same nature occur in other areas of the field of bioinformatics and in particular genomic data analysis, and also in other fields such as market research. According to the present invention, there is provided a computer- implemented method of selecting differentially expressed genes from an overall dataset comprising plural sets of microarray data experimentally derived in respect of two types of biological sample, each set representing expression levels of the same predetermined plurality of genes, the method comprising: (a) in respect of each a plurality of algorithms capable of being performed on the overall dataset to produce a results set ranking the genes by relevance on the basis of a statistical measure of the differential between the expression levels for the two types of biological sample, performing the steps of: creating a plurality of trial datasets by drawing from said overall dataset a plural number of sets of microarray data to form each trial dataset; performing the given algorithm on each of the trial datasets to produce a plural number of trial results sets; and calculating a measure of similarity between the trial results sets in at least one group of genes ranked in the trial results sets as most relevant; and (b) selecting the algorithm in respect of which said calculated measure of similarity between the trial results sets is highest; (c) performing the selected algorithm on the overall dataset to produce a final results set; and (d) selecting a number of genes ranked in the final results set as most relevant. The measure of similarity provides a good measure of the ability of an algorithm in question to successfully select relevant genes from the dataset in question. The algorithm having the best measure of similarity is the algorithm which most closely matches the nature of the data in the dataset and so is most likely to be optimal for gene selection from the underlying biological system. Thus that algorithm is selected and used for gene selection. The selected genes may then be experimentally studied further. As the algorithm most likely to be optimal is used, the selected genes are those most likely to be relevant, ie false positives and false negatives are reduced. This produces real benefits in terms of reducing waste of resources by studying genes which are in fact irrelevant (false positives) and also in terms of reducing failure to study unselected genes which are in fact relevant (false negatives). Importantly, this selection of the appropriate algorithm may be achieved based solely on the actual dataset, without the need for any additional information or assumptions. Thus, the method is not dependent on knowledge or assumptions, which are possibly incorrect, about the dataset or the underlying biological system. The reason why the measure of similarity provides a measure of the ability of an algorithm to correctly rank genes by relevance can be understood intuitively. That is, the algorithm which best matches the original dataset is most likely to produce similar or consistent results for each of the trial datasets drawn from the original dataset. However, it can also be demonstrated mathematically that, under the conditions typically experienced in gene selection, in respect of two given algorithms, the algorithm having the higher absolute value of the measure of similarity has the higher true probability of success in selecting relevant genes. The calculated measure of similarity is a statistical random variable because it depends on the data which is itself a statistical random variable. However, the calculated measure of similarity is an estimate of the absolute measure of similarity and thus may be used as the basis for selecting the algorithm having the best probability of success. In practice, the method is robust to difficult statistical conditions. The preferred measure of similarity is an average number of genes in the at least one group of genes in common between pairs of the trial results sets, which may be expressed as a proportion of the number of genes in the at least one group. Herein, the number of genes in a group of genes ranked in a results set as most relevant in common between a pair of the trial results sets is defined as consistency (although this term is also used to mean something different in the context of estimation in statistics). Consistency is the preferred measure of similarity because it is simple to implement and simple to analyse mathematically, as consistency may calculated merely by counting the number of genes in common. However, other measures of similarity between the trial results sets could alternatively be used. The method of the present invention may be applied to other areas in the field of bioinformatics and in particular genomic data analysis, and also to other fields where it is desired to select a relatively small number of relevant objects from a large number of irrelevant objects based on noisy measurements for the objects, giving values for the objects sampled from noisy distributions. Examples are: econometric data; financial data; market research data; consumer records data; information retrieved (e.g. of content such as text, images or audio from databases); or datamining. In the context of this general case, references herein to a gene maybe generalised to objects, references herein to an expression level of a gene may be generalised to sampled values for the objects and references herein to the differential between expression levels of a gene for two types of biological sample may be generalised to any characteristic of data for the objects. To allow better understanding, an embodiment of the present invention will now be described by way of non-limitative example with reference to the accompanying drawings, in which: Fig. 1 is a flow chart of a method of selecting and studying genes; Fig. 2 is a flow chart of the computer-implemented gene selection steps of the method of Fig. 1; and Fig. 3 is a view of an optical disk. A method of implementing the present invention is illustrated in the flow charts of Figs. 1 and 2. The overall method is shown in Fig. 1. In step 1, an overall microarray dataset 2 is experimentally derived. Step 1 may be performed using conventional microarray techniques which are well known to the skilled person and so will not be described in detail herein, but in general these techniques involve the following. DNA probes are arranged on microarray. Plural microarrays, which may be identical or may have common DNA probes, are prepared. Each DNA probe may be unique and corresponds to all or part of a gene. Two types of biological samples for studying a biological phenomenon are prepared. For example, to study a disease, the two types of biological sample may be diseased tissue and healthy tissue. mRNAs or corresponding cDNAs derived from the two types of biological sample are brought into contact with the DNA probes on the plural microarrays. Hybridisation of mRNAs in the samples with the corresponding DNA probes occurs. The hybridisation at each DNA probe which corresponds to the activity of the corresponding gene in the sample is experimentally measured, for example by detecting the resultant electroluminescence. The measurements are stored as microarray data representing the expression level of the genes in the sample. Each microarray produces a set of microarray data representing expression levels for the same plurality of genes. The plural sets of microarray data together constitute the overall dataset 2 in respect of the two types of biological sample. Thus, the overall dataset 2 comprises plural sets of microarray data in respect of one type of biological sample and plural sets of microarray data in respect of the other type of biological sample where each of the sets of microarray data is a set of expression levels of the same predetermined plurality of genes, that is corresponding to some or all of the DNA probes on the microarrays. As an alternative, the present method may be applied to an existing overall dataset 2, in which case step 1 is omitted, hi practice, this alternative is common because the experimental procedure in step 1 for producing the overall dataset 2 is time-consuming and expensive. For analytical purposes, the method may also be performed on an overall dataset 2 which is artificially generated. Unfortunately, the experimental procedures in step 1 and the underlying biological system are very noisy. As a result, the individual expression levels in each of the sets of microarray data effectively constitutes a random sample from a statistical distribution. However, the distribution is unknown and depends on the biological system of the two types of biological sample which is being studied. In general, the distribution may involve both a shape for a particular gene and a correlation between different genes, and may vary across the population of genes. A gene is said to be differentially expressed if the distribution for that gene differs between the two types of biological sample. Normally this is considered as a difference between the means of the distribution, but in general this can be taken as a difference in any respect between the two distributions. As the individual expression levels are random samples from the distribution, the detection of a differentially expressed gene must be performed on the basis of a statistical analysis of the plural sets of microarray data in the overall dataset 2. Such a statistical analysis maybe performed by an algorithm performed on the overall dataset 2 to produce a results set which ranks the genes by relevance on the basis of a statistical measure of the differential between the expression levels for the two types of biological samples. The algorithm may produce a ranking value for each gene on the basis of the statistical measure which ranking values are then used to rank the genes, hi general, such algorithms may be used for gene selection by selecting a group of genes ranked in the results set as being most relevant. A plurality of different gene selection algorithms of this type are employed in step 3. A large plurality of such algorithms are known, as disclosed in Sebastiani, Gussoni, ohane, & Ramoni, (2003), "Statistical Challenges in Functional Genomics", Statistical Science, 18 (1), 33-70 and Baldi,& Hatfield, (2002) "DNA microarrays and gene expression", Cambridge University Press. Such algorithms include, but not exclusively: - conventional two-sample hypothesis tests, for example as disclosed in Pan, (2002), "A comparative review of statistical methods for discovering differentially expressed genes in replicated microarray experiments", Bioinformatics, 18(4), 546- 554. Such two-sample hypothesis tests include, but not exclusively a t-test as described in standard text books of statistics (e.g. "Probability and Statistics", 3rd Edition, DeGroot & Schervish) and disclosed in the context of microarray analysis in Arfin, Long, Ito, Tolleri, Riehle, Paegle, & Hatfield, (2000), "Global gene expression profiling in Escherichia coli K12: the effects of integration host factor", J. Biol. Chem., 275 (38), 29672-29684 and Tanaka, Jaradat, Lim, Kargul, Wang, Grahovac, Pantano, Sano, Piao, Nagaraja, Doi, Wood HI, Becker, & Ko, (2000), "Genome-wide expression profiling of mid-gestation placenta and embryo using a 15,000 mouse developmental cDNA microarray", Proc. Natl. Acad. Sci. USA, 97 (16), 9127-9132; - non-parametric two-sample hypothesis tests, for example a rank-sum test or the tests disclosed in Troyanskaya, Garber, Brown, Botstein, & Airman, (2002), "Nonparametric methods for identifying differentially expressed genes in microarray data", Bioinformatics, 18 (11), 1454-1461 and Tusher, Tibshirani, & Chu, (2001), "Significance analysis of microarrays applied to the ionizing radiation response", Proc. Natl. Acad. Sci. USA, 98 (9), 5116-5121; - Bayesian algorithms, for example as disclosed in Long, Mangalam, Chan,

Hatfield, & Baldi, (2001), "Improved statistical inference from DNA microarray data using analysis of variance and a Bayesian statistical framework. Analysis of global gene expression in Escherichia coli K12", J. Biol. Chem., 276 (23), 19937-19944; and - information-theoretic algorithms, for example as disclosed in Amir Ben-

Dor, Nir Friedman & Zohar Yakhini, "Scoring Genes for Relevance", Technical Report No. AGL-2000-13, Agilent Technologies. The plurality of algorithms used in step 3 may include any combination of known algorithms and may equally include newly developed algorithms. The precise choice of algorithm is not critical to the present invention, although preferably the algorithms will include algorithms with a wide variety of different properties. In step 3, one of the plurality of algorithms is selected and performed on the overall dataset 2. Step 3 and a display step 5 described further below are performed by a computer program executed on a computer system 6 illustrated schematically by a dotted line in Fig. 1. The computer system 6 may be any type of computer system but is typically a conventional personal computer executing a computer program written in any suitable language. The computer program may be stored on a computer-readable medium, which may be of any type, for example: a recording medium, such as a disk-shaped medium 10 as shown in Fig. 3, insertable into a drive of the computing system 6 and which may store information magnetically, optically or opto-magnetically; a fixed recording medium of the computer system 6 such as a hard drive; or a computer memory. The steps performed on the computer system 6 are shown in detail in Fig. 2. In particular, step 3 consists of a number of steps 31 to 41. In step 31 , one of the plurality of algorithms is chosen. Next, in step 32, a plurality of trial datasets 33 are created. Each trial dataset 33 is created by drawing a number of sets of microarray data from the overall dataset 2. There are a number of ways of so drawing sets of microarray data which will be described in more detail below. Next, in step 34, the algorithm chosen in step 31 is performed on each of the trial datasets 33. This produces a plural number of trial results sets 35, one for each of the trial datasets 33. Next, in step 36 a value of consistency is calculated. In this context, "consistency" means the following. Consistency is defined for any given pair of trial results sets and with reference to a group of genes ranked in the trial results sets as being most relevant. Consistency is the number of genes in that group in common between the pair of trial results sets, that is the number of genes which are present in the group of genes ranked as most relevant in both of the pair of trial results sets. Preferably, the group is of predetermined size, that is the group will consist of a number of genes which is predetermined in the sense of being fixed before performance of the method, although it may be selected by user input to the computer system 6. As an alternative, the group may be selected on the basis of a comparison of the ranking value for each gene with a predetermined threshold. In this case, the number of genes in the group varies depending on the nature of the actual microarray data in the trial datasets 33. In step 36, respective values of consistency are calculated in respect of plural pairs of trial results sets and the respective values are averaged to give an average consistency which is output from step 36. As to the choice of pairs of trial results sets for which consistency is calculated, this depends on how the trial datasets are drawn from the overall datasets and will be described in more detail below. Step 36 is preferably carried out with relevance to a single group of genes. As an alternative, step 36 may be carried out with reference to plural groups of genes of different sizes, hi this case, a value of consistency is calculated in respect of each group of genes and the values of consistency thus calculated are combined together in some manner to obtain final value of consistency which is output from step 36. The consistency calculated in step 36 is a measure of similarity between the trial results sets. As an alternative, other measures of similarity between the trial results sets could be applied. For example, rather than just counting the number of genes in common in the group of genes, the measure of similarity could take into account the correlation between the ranking of each gene within the group. Another possibility is for the measure of similarity to take into account the ranking value calculated for each gene by the algorithm. However, the measure of similarity is always calculated in respect of a group of genes ranked in the trial results sets as being most relevant, because the ranking of the less relevant genes is not significant to the accuracy of the algorithm in question. Use of consistency as the measure of similarity is preferred because it is simple to implement in practice and also is simple to analyse mathematically, because it is just a matter of counting the number of genes in common. Below, the efficacy of the present invention will be explained by reference to consistency, but the comments apply equally to alternative measures of similarity. In step 37, it is checked whether steps 32, 34 and 36 have been performed to calculate an average consistency in respect of each of the plurality of algorithms. If not, then the method returns to step 31 to chose a new algorithm and steps 32, 34 and 36 are repeated for the newly chosen. Once an average consistency has been calculated in respect of each of the plurality of algorithms, the method proceeds from step 37 to step 38. In step 38, the average consistencies calculated in step 36 are compared and the algorithm having the highest average consistency is selected. Next, in step 39 the algorithm selected in step 38 is performed on the overall dataset 2 to produce a final results set 40. In step 41, a number of genes ranks in the final results set 40 as being most relevant is selected. Preferably, the number is predetermined in the sense of being fixed before performance of the method, although it may be selected by user input to the computer system 6. As an alternative, the genes maybe selected on the basis of a comparison of the ranking value of each gene with a predetermined threshold, hi this case, the number of genes selected varies depending on the nature of the actual microarray data in the overall dataset 2. The group of genes with respect to which consistency is calculated in step 36 maybe equal to the number of genes selected in step 41. However, this is not essential. Even if they are different, the consistency calculated in step 36 remains a valid basis for selecting an algorithm. Data 4 representing the genes selected in step 41 is output. In step 5, information identifying the selected genes is displayed on a display of the computer system 6. Next, in step 7, the relevant genes for selected in step 3 are experimentally studied in detail experiment to analyse the involvement of the genes in the biological phenomenon under investigation. Such further experiments are in themselves of a conventional nature and so will not be described in detail, but examples are as follows. The selected genes can be further analysed to assess or identify the activity of the encoded protein, to identify other proteins which interact with the protein, for example to identify receptors or ligands for the protein. Further experiments can be carried out to identify agents which modulate the activity of the protein or which modulate expression of the relevant gene, with a view to identifying agents which are useful in the treatment of the condition associated with differential expression of the genes. Further experiments typically involve production of vectors for expression of the relevant gene and transformation of host cells in order to carry out further studies to assess activity of the protein or identify modulators of the protein activity or expression. The reason why consistency is useful as a basis for selection of one of the plurality of algorithms is as follows. As a preliminary point, it is noted that for each given algorithm one can consider the probability of success of the algorithm, that is the probability of the algorithm correctly selecting the relevant genes from the entire population of genes. In principle, the probability of success for a given algorithm, a given number of sets of microarray data (i.e. sample size) and a given number of genes selected, the probability of success is determinate from the population parameters, that is the number of genes (i.e. population size) and the statistical distributions for each gene. The statistical distributions may be defined by, for example, the shape, mean and variance of the distribution for each gene and the correlations of the distributions between each gene. The distributions inherently define the true numbers of differentially expressed and irrelevant genes. However, as discussed above, the distributions are not in practice known a priori and so the true probability of success of each algorithm cannot be determined. Despite this, it can be shown that consistency correlates with the probability of success of a particular algorithm under the conditions particularly encountered in a microarray dataset for gene selection. In particular, in such a dataset, the number of irrelevant genes far outweigh the number of relevant genes, typically by at least in order of magnitude. For example, in a typical dataset, the number of relevant genes is of the order of 10 to 100, but the total number of genes is at least a thousand and usually several thousand. Typically, the number of genes selected using the algorithm is of the order of 10 to 100. It can be seen intuitively that consistency is a good measure to select an algorithm from the observation that there are many more relevant genes than irrelevant genes. A simple example will clarify this basic idea. Consider a scenario where there are 50 truly relevant genes, 5000 irrelevant ones and 60 genes are being selected. Consider the case that a particular algorithm selects 30 relevant genes and 30 irrelevant genes. The 30 relevant genes can be thought of as having been taken from a pool of 50 relevant genes, whilst the 30 irrelevant genes can be thought of as having been taken from a pool of 5,000 irrelevant genes. Thus, the variation in the identifies of the irrelevant genes will far exceed the variation in the identities of the relevant genes. This means that when we compare selection results for different samples from the population, the level of agreement between results set, i.e. the similarity or consistency, will be dominated by the effects of the relevant genes. Therefore, an algorithm which picks a higher proportion of relevant genes also tends to have a higher consistency or similarity between trial results sets based on trial datasets drawn from the original datasets. A similar idea can also be demonstrated mathematically as follows. One can define absolute consistency as the average consistency between datasets drawn randomly from the distributions of the underlying biological system in the limit of an infinite number of datasets. In this case, the average is the average consistency between each possible pair of datasets. Although the consistency between any individual pair of datasets is a statistical random variable, the absolute consistency in the limit of an infinite number of datasets has a single value dependent on the distributions. If we define the true consistencies of algorithms A and B as C_A and C_B, respectively; the number of relevant genes as N_τ; the number of irrelevant genes as N_F; the number of genes selected as Z; and the probability of success of algorithms A and B as P_A and P_B, respectively, then it can be shown mathematically that for plausible algorithms (for which P_A and P_B is not zero or close to zero), that, if: ( P_A + P_B) . ( N_T+ N_F) > 2.Z (1) and Z >N_T (2) then C_A > C_B => P_A > P_B (3) Equations (1) and (2) normally hold for a typical gene selection method.

Equation (1) normally holds because ( N_τ + N_F ) is very high as compared to Z.

Equation (2) is merely needed to define the probability of success. When equations

(1) and (2) hold, then the algorithm A having the higher consistency C_A also has the higher probability of success P_A. Although absolute consistency is determinate based on the statistical properties of the biological system, the statistical properties are in practice unknown and so the absolute consistency is not determinable. However, in the present method the calculation of consistency in step 37 from trial results sets drawn from the overall dataset calculates an estimate of the absolute consistency. This estimate is itself a statistical random variable which tends to the absolute consistency in the limit.

Therefore, this estimated consistency is a good predictor for the absolute consistency which itself correlates with the probability of success of the different algorithms. The drawing of trial datasets in step 32 and the calculation of consistency in step 36 maybe performed in a number of different ways to derive an estimate of the absolute consistency. Two preferred techniques are as follows. The first technique involves dividing the overall dataset 2. In particular, in step 32, the overall dataset 2 is divided into two groups of sets of microarray data. Each group forms a trial dataset 33, so the division effectively draws two trial datasets 33. Such division is performed in step 32 a plurality of times to produce a plurality of pairs of trial datasets 33. Subsequently in step 36, the consistency is calculated between the trial datasets 33 of each pair created by a single division. Then, the average of the thus-calculated consistencies is calculated and taken as the final value of consistency. This first technique is very simple to implement and produces a reasonable estimate of the absolute consistency. A second technique is to use a bootstrap procedure. Bootstrap procedures are a well-established re-sampling technique which has been used in a great variety of statistical applications to estimate the value of a variable from a dataset. Using this technique, in step 32 trial datasets 33 are created by drawing sets of microarray data randomly with replacement from the overall dataset 2. The number of sets from microarray data drawn is equal to the number of sets on microarray data in the overall dataset 2. As the sets of microarray data are drawn randomly with replacement, it is possible to draw the same set of microarray data any number of times. Thus the trial datasets 33 may include repeated copies of any given set of microarray data in the original dataset 2. Subsequently in step 37 the consistency is calculated between each possible pair of trial datasets 33 within the plurality of trial datasets 33. Then, the average of the thus-calculated consistencies is calculated and taken as the final value of consistency. With such a bootstrap procedure the average consistency calculated in step 37 tends to an optimal estimate as the number of trial datasets 33 increases. Therefore, with this technique the number of trial datasets 33 is chosen to be relatively large compared to the number of sets of microarray data in the original dataset 2. The second technique has the advantage of providing a better estimate of the absolute consistency than the first technique, tending to an optimal estimate as the number of trial datasets 33 increases. On the other hand, the second technique has the disadvantage of requiring a greater amount of calculation than the first technique, i practice, the first technique is sufficient in many practical situations. Although the method shown in Fig. 2 involves drawing all the trial datasets 33 in step 32, before performing the algorithm in step 34 and calculating consistency in step 36, this is not essential and in principle it is possible to draw one or more trial datasets 33 and perform the algorithm on those trial datasets 33, before drawing further trial datasets 33. In the methods described above, the plurality of algorithms used in step 3 may be a finite number of predetermined algorithms. As an alternative, the plurality of algorithms may be selected from a family of algorithms defined generically by a predetermined mathematical definition having at least one variable. In this case, respective algorithms in the family correspond to different values of the at least one variable. In this case, in step 31 an algorithm is chosen by selecting particular values for the at least one variable in the predetermined definition. The choice in step 31 is based on any consistencies calculated previously in step 36 for previously chosen algorithms. In particular, the choice in step 31 uses a search strategy intended to identify the algorithm having the maximum consistency. Correspondingly, in step 37, it is decided, on the basis of the search strategy, whether the consistency has been so maximised. To demonstrate the efficacy of the present method, step 3 has been applied to some artificially generated datasets 2 and to some actual datasets 2 as follows, hi both cases, the plurality values consist of four algorithms, namely a t-test, a difference-of-means algorithm; a difference-of-medians algorithm and a rank-sum test. In both cases, the method was applied using the second technique for deriving the trial datasets in step 32. Firstly, step 3 was applied to three overall datasets 2, each generated using a Gaussian model but with different conditions. For the first overall dataset 2, the variance was the same for both types of biological sample and was fixed across the various genes. The consistency calculated in step 36 was found to be highest for the difference-of-means algorithm with the t- test and the rank-sum test also performing well. However, the consistency for the difference-of-medians test was rather worse, being about half that of the difference- of-means algorithm. For the second overall dataset 2, the variances were the same for each of the two types of biological sample, but were varied for the different genes. This example reflects, for example, the situation where the pooled variances for expression levels from genes involved in cancer are lower than those for non-cancer genes. In this case, the consistency calculated in step 36 for the t-test and the rank-sum test were both relatively high, as compared to the difference-of-means algorithm and the difference- of-medians algorithm. The third overall dataset 2 was generated by drawing variances for the relevant genes from a distribution having a high mean and drawing the variances for the irrelevant genes from a distribution having a low mean. This set-up reflects, for example, a situation where the pooled variance for genes involved in cancer turns out to be higher than that for other genes. In this case, the consistency calculated in step 36 was highest for the difference-of-means algorithm, with the difference-of-medians algorithm performing less well and with the t-test and the rank-sum test performing worse of all. These results demonstrate that changes in variance across genes can seriously impact the relative performance of different algorithms. In general terms any variation in the statistical distributions resulting from the biological system may heavily influence the accuracy of different gene selection algorithms, depending on both the precise nature of the variation in question and the nature of the statistical measure of the differential employed by the algorithms. However, such effects are difficult to detect, because the nature of the biological system is a priori unknown. However, the present method allows a selection of the algorithm which best matches the distribution of data within the overall dataset 2. Secondly, step 3 was applied to two well-known and widely analysed microarray datasets which are publicly available, namely the colorectal cancer data of Alon, Barkai, Notterman, Gish, Ybarra, Mack, & Levine, (1999), "Broad Patterns of Gene Expression Revealed by Clustering Analysis of Tumor and Normal Colon Tissues Probed by Oligonucleotide Arrays", Proc. Natl Acad. Sci. USA, 96 (12), 6745-6750 and the leukemia data of Golub, Slonim, Tamayo, Huard, Gaasenbeek, Mesirov, Coller, Loh, Downing, Caligiuri, Bloo field, & Lander, (1999), "Molecular classification of cancer: class discovery and class prediction by gene expression monitoring", Science, 286 (5439), 531-537. The present method was performed on these datasets as the overall datasets 2 after restriction to the 2000 genes with highest minimal intensity and after pre-processing according to current practice by changing from absolute to relative expression levels to remove within-slide location and by transformation of the data into a log space. In respect of both overall datasets 2, the consistency calculated in step 36 was highest for the difference-of-means algorithm with the difference-of-medians algorithm performing nearly as well. In contrast, the consistency calculated in step 36 for the t-test and the rank-sum test performed relatively poorly. Thus, the present method predicts that the difference-of-means algorithm will rank these particular overall datasets 2 best and this algorithm was selected in step 38 and used to select the relevant genes in step 41 identified for further, detailed experimental study in step 7.

Claims

1. A computer-implemented method of selecting differentially expressed genes from an overall dataset comprising plural sets of microarray data experimentally derived in respect of two types of biological sample, each set representing expression levels of the same predetermined plurality of genes, the method comprising: (a) in respect of each a plurality of algorithms capable of being performed on the overall dataset to produce a results set ranking the genes by relevance on the basis of a statistical measure of the differential between the expression levels for the two types of biological sample, performing the steps of: creating a plurality of trial datasets by drawing from said overall dataset a plural number of sets of microarray data to form each trial dataset; performing the given algorithm on each of the trial datasets to produce a plural number of trial results sets; and calculating a measure of similarity between the trial results sets in at least one group of genes ranked in the trial results sets as most relevant; and (b) selecting the algorithm in respect of which said calculated measure of similarity between the trial results sets is highest; (c) performing the selected algorithm on the overall dataset to produce a final results set; and (d) selecting a number of genes ranked in the final results set as most relevant.

2. A method according to claim 1, wherein said measure of similarity is an average number of genes in the at least one group of genes in common between pairs of the trial results sets.

3. A method according to claim 2, wherein said average number is expressed as a proportion of the number of genes in said at least one group.

4. A method according to any one of the preceding claims, wherein said step of creating a plurality of trial datasets comprises performing, at least once, the step of splitting the sets of microarray data in said overall dataset into two groups to form two trial datasets.

5. A method according to claim 4, wherein said step of splitting the sets of microarray data in said overall dataset into two groups is performed a plural number of times; said step of calculating a measure of similarity comprises calculating a respective measure of similarity between the two trial datasets formed each time the step of splitting is performed and calculating the average of said respective measures of similarity.

6. A method according to any one of claims 1 to 3, wherein said step (a) comprises a bootstrap procedure in which said step of creating a plurality of trial datasets comprises drawing randomly with replacement from said overall dataset a number of sets of microarray data equal to the number of sets of microarray data in said overall dataset to form each trial dataset; and said step of calculating a measure of similarity comprises calculating a respective measure of similarity between each possible pair of trial datasets within the plurality of trial datasets and calculating the average of said respective measures of similarity.

7. A method according to any one of the preceding claims, wherein said plurality of algorithms all meet a predetermined definition having at least one variable, respective algorithms corresponding to different values of the at least one variable, and said step (a) is performed iteratively for different algorithms meeting the predetermined algorithm using a search strategy to maximise the calculated measure of similarity.

8. A method according to any one of the preceding claims, wherein said at least one group comprises a group of genes equal in number to said number of genes selected in said step (d).

9. A method according to any one of the preceding claims, further comprising displaying information identifying the genes selected in said step (d).

10. A method of gene selection comprising: experimentally deriving plural sets of microarray data in respect of two types of biological sample, each set representing expression levels of the same predetermined plurality of genes, to form an overall dataset; and performing a method according to any one of the preceding claims on the overall dataset.

11. A method according to any one of the preceding claims, further comprising experimentally studying the genes selected in said step (d).

12. A computer-implemented method of selecting an object from an overall dataset comprising plural sets of data each representing sampled values for the same plurality of obj ects the method comprising: (a) in respect of each a plurality of algorithms capable of being performed on the overall dataset to produce a results set ranking the objects by relevance on the basis of a measure of a characteristic of the data, performing the steps of: creating a plurality of trial datasets by drawing from said overall dataset a plural number of sets of data to form each trial dataset; performing the given algorithm on each of the trial datasets to produce a plural number of trial results sets; and calculating a measure of similarity between the trial results sets in at least one group of objects ranked in the trial results sets as most relevant; and (b) selecting the algorithm in respect of which said calculated measure of similarity between the trial results sets is highest; (c) performing the selected algorithm on the overall dataset to produce a final results set; and (d) selecting a number of objects ranked in the final results set as most relevant.

13. A method according to claim 12, further comprising displaying information identifying the objects selected in said step (d).

14. A computer program capable being executed on a computer system and capable when so executed of causing the computer system to perform a method according to any one of claims 1 to 9, 12 or 13.

15. A computer-readable medium storing a computer program according to claim 14.