GB2408360A

GB2408360A - Performance Analysis

Info

Publication number: GB2408360A
Application number: GB0327148A
Authority: GB
Inventors: Timothy Adrian Lewis
Original assignee: Toshiba Research Europe Ltd
Current assignee: Toshiba Europe Ltd
Priority date: 2003-11-21
Filing date: 2003-11-21
Publication date: 2005-05-25
Anticipated expiration: 2023-11-21
Also published as: GB2408360B; GB0327148D0

Abstract

A method for analysing performance of a computer apparatus. The method comprises executing at a first time a plurality of first operations to obtain first performance data, and executing at a second time at least one second operation to obtain second performance data. A relationship between at least part of the second performance data and at least part of the first performance data is obtained. Said relationship and at least part of said first performance data, is used to compute third performance data indicative of performance at said at least one first operation at said second time.

Description

PERFORMANCE ANALYSIS

The present invention relates to methods for analysing the performance of a computer apparatus.

Computers are used in many aspects of modern life. These include not only desktop computers used in homes and offices, but also portable computers such as laptops, Personal Digital Assistants (PDAs), and computing functionality incorporated into modern mobile telephones.

There is a need to obtain data indicative of the performance of a computer. Preferably, this data should be available in a form such that a plurality of different computer apparatuses can be compared from the point of view of their performance. In order to satisfy this requirement, various organizations have developed so called "benchmarks" which comprise a series of operations which are run on a plurality of different computers. The performance exhibited by the different computers in executing this series of operations can then be compared. Typically, the time taken to execute the series of operations on the different computers is used to represent differences in performance of various computers.

In developing a benchmark it is important that tests included within the benchmark are representative of normal operation of the computers under consideration. Typically, a benchmark containing a large number of operations will provide more accurate performance data. Furthermore, executing a test of plurality of times and taking an average is likely to improve accuracy. However, although large benchmarks are typically more accurate, they take a longer time to execute, and consume more power in their execution. This is a particular concern in the case of battery powered devices such as laptops, mobile telephones, and PDAs. It is also a particular concern if a benchmark is to be run during operation of a device so as to obtain data indicative of the current "load" of the device.

Object oriented software development is now commonplace. A common object oriented programming language is Java_. Java_ source code is "compiled" to form Java_ byte code. Java_ byte code is interpreted by a Java_ Virtual Machine (JVM) which converts Java_ byte code into the native machine code of the computer on which the code is to execute. Naval byte code is a machine independent language, and therefore, when Java_ source code is compiled to form Java_ byte code it is portable between different platforms, providing only that a Java_ Virtual Machine exists for that platform.

Although this platform independence is an advantage of the Java_ programming language, it does incur a cost in terms of efficiency. Specifically, as indicated, Java_ is an interpreted programming language, meaning that interpreted Java applications execute more slowly than similar programs written in a traditional high level programming language such as C or C++ which are compiled to generate machine code.

In order to overcome this problem some JVM implementations include a so called "Just In Time" (JIT) compiler. JIT compilers translate certain Java,M byte code sequences into machine code for the particular hardware on which the JVM is implemented. This compilation occurs only once, and therefore offers benefits over interpretation of Java_ byte code.

US 6,118,940 (International Business Machines Corporation) describes a method and apparatus for creating benchmark programmes for the analysis of Java_ Virtual Machine implementations. The described method analyses byte code traces, selects frequently occurring operations for inclusion within a benchmark, and then creates an appropriate benchmark program. The method proposed by US 6,118,940 therefore allows different Java'M Virtual Machine implementations and their associated JIT compilers to be compared. GB 2367642 (International Business Machines Corporation) describes a further method for obtaining performance data from Java_ virtual machine implementations.

Although the prior art systems described above provide methods for obtaining performance data which can be used to compare different operating environments, none of these systems address the problem that relatively large benchmarks run for a relatively long time are required in order to obtain results truly indicative of system performance.

It is an object of the present invention to obviate or mitigate at least some of the

problems experienced by prior art systems.

According to the present invention there is provided a method for analysing performance of a computer apparatus, comprising executing at a first time a plurality of first operations to obtain first performance data, executing at a second time at least one second operation to obtain second performance data, determining a relationship between at least part of the second performance data and at least part of the first performance data and computing from said relationship and at least part of said first performance data, third performance data indicative of performance of said first operations at said second time.

Said computer apparatus may be under a first load condition at said first time and under a second load condition at said second time. The term "load condition" is used herein to mean the current workload of the computer apparatus.

Said at least one first operation is preferably a plurality of first operations. Said at least one second operation may be a subset of said plurality of first operations. The relationship between at least part of the second performance data and at least part of the first performance data is preferably a ratio of at least part of the second performance data to at least part of the first performance data.

The at least one second operation may be an addition operation, and more preferably an integer addition operation. Use of the integer addition operation is preferred because this operation executes reasonably quickly, gives a good indication of processor load and uses no other device resources such as a memory device or disk storage.

The method may be implemented in a machine independent programming language.

The machine independent programming language may be compiled to generate machine independent byte code. The machine independent programming language is preferably Java_.

Said first performance data may be obtained by executing the at least one first operation a plurality of times, and recording a time taken to execute said plurality of operations.

Said at least one first operation may be executed such that said first performance data is obtained with a predetermined accuracy. Said second performance data may be obtained by executing the at least one second operation a plurality of times, and recording a time taken to execute said plurality of operations. Said at least one second operation may be executed such that said first performance data is obtained with a predetermined accuracy.

The computer apparatus may be desktop computer, a laptop computer, a portable digital assistant, or a mobile telephone. The computer apparatus may be a microprocessor.

The computer apparatus may be a computer network.

The present invention also provides a method for selecting a computer apparatus from a plurality of interconnected computer apparatuses. The method may comprise obtaining performance data indicative of the performance of each of said plurality of interconnected computer apparatuses using a method as set out above, and selecting one of said plurality of computer apparatuses on the basis of the obtained performance data.

Said plurality of computer apparatuses may be connected by a computer network.

Performance data indicative of the performance of each of said plurality of interconnected computer apparatuses may be obtained by a server connected to said computer network. Said plurality of computer apparatuses may be microprocessors within a single device.

The invention also provides a data carrier carrying computer program code means to cause a computer to execute procedure as set out above.

The invention further provides a computer apparatus comprising a program memory containing processor readable instructions, and a processor for reading and executing the instructions contained in the program memory. Said processor readable instructions control the processor to carry out the method as set out above.

Embodiments of the present invention will now be described, by way of example, with reference to the accompanying drawings, in which: Figure I is a flow chart showing a process for creating a benchmark test used in an embodiment of the present invention; Figure 2 is a flow chart showing of a method for running a test created in accordance with the method illustrated in Figure 1 to obtain performance data of specified accuracy; Figure 3 is a graph showing the ratio of various benchmarks to the integer addition benchmark on a desktop, laptop and PDA; Figure 4 is a schematic illustration of an architecture for obtaining performance data in accordance with the present invention; Figure 5 is a flow chart illustrating a process for obtaining performance data using the architecture of Figure 4; and Figures 6 to 8 are graphs illustrating the accuracy obtained using a method for obtaining performance data in accordance with the present invention.

Methods for obtaining benchmark data in accordance with the invention are now described. The main purpose of many benchmarks is to obtain data indicative of the performance of various devices, which can be used to compare performance of the various devices. it is therefore important that benchmarks are created which are easily portable between different devices. The Java programming language provides a convenient way of creating platform independent applications, and preferred embodiments of the present invention implement benchmarks using the Java programming language.

Benchmark tests must be created so as to test a representative selection of operations. In general these operations comprise arithmetic operations such as multiplication, addition and division performed on a variety of data types (e.g. integer, double and long integer), file input and output operations, memory allocation operations, and memory reading and writing operations. Table I sets out tests which are used in a preferred embodiment of the present invention:

Test Name Description Example Java Code Fragment

Intadd, intmult, Arithmetic operations on intdiv,doubleadd, different types. x = y + z doublemult, doublediv WriteByte Writes ten bytes to a file on the local disk Testout.write("JJJJJJJJJJ"); eadByte Reads ten bytes from a local file into a character testin.read(ch,0,10); buffer WriteMem Writes a byte into an array Ch[5] = 115; ReadMem Reads a byte from an array val = ch[5]; ocalAssign Writes to a local variable val = 5; neKmalloc Allocates 1K of Memory Mem = new int[500]; DbjectSerial Serialises a vector Object containing five short stream. writeObject(vector); Strings.

:)bjectDeSerial Deserialise the object.

stream.readObject(); eturnlntFCal1 Call a simple method that returns an integer. returnInt(); ubstring Return the substring starting at character 3 of a a.substring(3); character string.

tringComp Find the index at which strings "Stringa" and a.indexOf(b); "Stringb" differ VectoraddElement Returns the second element of a 5 clement Vector.elementAt(2); Vector

TABLE 1

Table I includes a name for each test in a first column, a brief textual description in a second column, and an appropriate Java code fragment in a third column.

Benchmark tests should in general run as quickly as possible, although it is often the case that longer tests will yield more accurate results, a compromise is therefore necessary to obtain results of acceptable accuracy within a reasonable time. The need to get this balance right is exacerbated by the fact that it may be necessary to run benchmark tests concurrently with other processes, in order to obtain performance data under predetermined conditions. In these eases, the processing resources required by the benchmark (e.g. memory) are of particular concern.

Each test comprises a number of occurrences of the basic Java code fragment shown in Table 1. This number of occurrences must be chosen such that the time taken is reasonable, and such that accuracy is acceptable. A method for making this choice is now described.

Each test comprises m occurrences of the basic operation, enclosed within a loop which runs for n iterations. The value of m must be chosen such that it is not impracticably long, but must also be chosen such that the time taken to perform the operations of interest is not significantly affected by the execution overhead of the enclosing loop.

Furthermore, in the preferred Java implementation, it should be noted that an upper limit on the value of m is imposed by the JVM limit for member function byte code size.

Figure] illustrates a process for selecting m and n. m is selected at step S]. Tests have shown that setting m to have a value of fifty provides a reasonable compromise. For such a value the shorter operations (e.g. integer addition) give meaningful results, and the longer operations do not take an excessively long time to execute.

Having selected a value for m, a value for n is selected. At step S2 n is initialised to a value of one. A counter variable i is initilased to zero at step S3. The current system time is obtained at step S4 using a Java library function System.currentTimeMillis().

Step S4 sets a variable startTime to be equal to this obtained time. Step S5 checks that i is less than n. Initially, the condition is TRUE, and the m operations are therefore executed at step S6. The counter variable i is incremented at step S7, and control returns to step S5, where a check is executed to determine whether i is less than n. This determines whether further iterations of the loop are required. Given that n is initially set to one, the condition of step S5 returns FALSE, and control passes to step S8, where a variable end time is set to be the current time. Step S9 determines an actual time by setting an actualTime variable to be equal to endTime minus startTime. It may be the case the insufficient time has elapsed for startTime to differ from endTime given the resolution of the timer. This is checked at step S10, by ensuring that actutalTime is greater than zero. If this check returns FALSE, control passes to step Sl I where n is multiplied by two, and the process is repeated for the new value of n. This process continues until actualTime is set to a meaningful value by step S9. When this is the case, step S 10 returns TRUE, and a value of n is calculated at S 12 such that the test will take approximately one second to execute.

Having determined values of m and n as described with reference to Figure 1, a test is defined as comprising n occurrences of the m operations.

A predetermined number of tests are then executed to ensure that acceptable accuracy is achieved. If acceptable accuracy is not achieved the number of tests is doubled, and this process continues until acceptable accuracy is achieved. This process is illustrated in Figure 2.

A variable p represents the number of tests to be executed. This variable is initialised at step S13. A counter variablej is initialised to zero at step S14, and steps S14, Sl9 and S20 together ensure that steps S15, S16, S17 and S18 are executed p times. That is steps S 14, S 19 and S20 represent a for loop of the form: for (j=O; IMP; j++) i Step S15 obtains the current time using the System.currentTimeMillis() function mentioned above. Step S16 executes m operations n times as described, and step S17 obtains the time after execution of these operations. The current time is written to an element j of an array times at step S 18.

After execution of these steps p times, the condition of step S20 returns false, and step S21 is used to calculate the accuracy of the data stored in the array times. The acceptability of this accuracy is checked at step S22. If the accuracy is acceptable the obtained data is used for performance comparisons. If the accuracy is not acceptable (i.e. the condition of step S22 is FALSE), control passes to step S23 where p is multiplied by two, and the process is repeated. p continues to be increased in this way until the obtained accuracy is acceptable.

A method for analysing the accuracy of obtained data at step S21is now described.

As described above, p time values are obtained in the array times. These values can be used to compute a sample mean x and a sample variance 2 using known statistical techniques. The sample mean has a Normal distribution with a mean x and a variance -. The true mean lies within 2-of the sample mean ninety five percent of the

P time.

If it is desired to be within k percent of the true mean then: <key (1) x200 For example, when p=50, and k=5, the ratio of the sample variance to the sample mean must be below 0.177. k is considered to represent the accuracy of the benchmark, and it is therefore the value of k that is analysed at step S22 of Figure 2.

If k= 10 the time taken to run all the tests shown in Table 1 is shown in Table 2 for three different platforms.

T e Time/s

YP

Desktop 90 Laptop(Win2000) 1] 6 PDA] 076

TABLE 2

It can be seen from figure 2, that the time taken to run the tests so as to achieve an acceptable accuracy is relatively high, particularly for the PDA. It is therefore clear that running tests in this way at runtime to obtain a snapshot of current machine performance will not be viable. However, running tests taking these times just once (e.g. at startup or connection to a network) may well be acceptable.

Figure 3 is a graph showing the ratio of the integer addition test "intadd" to other tests across the different platforms shown in Figure 3. It can be seen that these ratios vary widely for different platforms. For example, the "writemem"test relating to writing data to memory takes a time comparable to integer addition on the PDA device, but over four times longer on both the desktop and laptop machines. Figure 3 shows that simply taking an integer benchmark for each platform and applying this to some predetermined ratios common to all platforms will not provide sufficiently accurate results.

One possible way of determining current performance from a complete set of benchmarks obtained at start up is to obtain load information using a function provided by the operating system. Java does not support this function directly, although a Java implementation could be realised by providing a class for each platform which includes a call to that platform's operating system function call. All these classes could implement a common interface to ease programming. Having obtained this data it is then necessary to estimate how much slower the assessed load will cause the benchmarks to operate. This estimation is not straightforward. Furthermore a class must be provided for each platform thereby removing some portability from the method. This approach is therefore not used in the present invention.

An architecture suitable for implementing the present invention is illustrated in Figure 4. A server I communicates with a PDA 2, a desktop computer 3 and a laptop computer 4. A process in accordance with the present invention which can be implemented using the architecture of Figure 4 is illustrated in Figure 5.

At step S23, the server I obtains a complete set of benchmark data from each of the computers 2, 3, 4. Data obtained in this way for each computer is that computer's "static profile" and represcuts performance when no other processes are executing.

When it is desired to obtain data indicative of the current performance of one of the computers 2, 3, 4, a command is issued by the server 1 to execute the benchmark test for the integer addition operation. The time taken to execute this test ( Ti') is returned to the server 1, and this represents the computer's "load profile" (step S24) A ratio is then calculated as follows (step S25): T..' Ti (2) where Tj'is the time taken to execute the integer addition benchmark under current load conditions (the load profile); and Ti is the time taken to execute the integer addition benchmark at startup (part of the static profile).

Having obtained this ratio, each test of the static profile is multiplied by the ratio, and the obtained data is considered to be indicative of current performance for each test in the set of benchmark tests (step S26) .

The present inventors have discovered that this method provides surprisingly accurate results as is now described, this is particularly surprising given that which is indicated by the graph of Figure 3. Results obtained using the method described above are now presented.

Tests were executed on three different computers, details of which are set out in Table 3 below.

OS Processor RAM (MB) Desktop Linux] 000 MHz Pentium Ill 256 Laptop Windows2000 747 MHz Pentium III 128 PDA Pocket PC Inte] XScale 64

TABLE 3

In order to simulate a loaded machine, a simple program containing an infinite loop was executed. When run on its own, this program utilised all CPU resources of each computer.

Table 4 shows how accuracy of estimates varies in dependence upon the time taken to obtain measurements. It can be seen that results of greater accuracy are obtained by using tests which execute over a longer period of time.

Accuracy/ Unloaded Loaded 1 Loaded2 Loaded3 % Time/s Time/s Time/s Time/s 10.0 0.1 3 12 33 5.00 0.1 7 26 68 1.00 4 34 101 277 0.50 8 100 210 558 0.20 34 279 0.10 69 0.05 279

TABLE 4

The values were taken using the desktop platform and the integer addition benchmark, but similar behaviour is seen for the other operations on other platforms. The missing values are timings either beyond what would be a practical time (an arbitrarily chosen 5 minutes) or for accuracy above a half of one percent (which is unlikely to be of any additional practical use). For the loaded benchmarks, the time is far greater than one would expect since the measurement process has to run a far larger number of iterations to obtain a sufficiently accurate estimate.

Figures 6 to 8 illustrate how estimate accuracy varies across differently loaded devices.

Figure 6 shows data for a desktop computer, Figure 7 shows data for a laptop computer and Figure 8 shows data for a PDA.

A full series of benchmarks was run on al] computers under three load conditions. This provided accurate data against which estimate data was measured. The estimation process illustrated in Figure 5 was then carried out for each device, so as to obtain integer addition values under the various load conditions which were then used to obtain a full set of estimated test data under the various load conditions. A] 0% accuracy level was used when running all tests.

The percentage differences between estimated and measured benchmarks is plotted for the respective computers in Figures 6 to 8.

It can be seen that many values have an accuracy within 10% of the real value. This is as good as could be hoped given that this is the level of benchmark accuracy.

An accuracy within 40% of the measured value is still useful for most performance predictions, and the majority of the plotted results are within this limit.

The estimation process clearly works best on the Desktop machine (Figure 6). This is possibly because it is a more powerful machine and would be expected to multitask more smoothly. Generally operations more similar to the integer addition operation (for example, double multiplication or String comparison) have better estimates. Generally speaking, heavily loaded situations such as the doubly and triply loaded ones would be less likely to occur in practice. Surprisingly, these heavily loaded estimates were, on a number of occasions, better than the more lightly loaded estimates. Operating system process scheduling policies may be responsible for this.

These results of Figures 6 to 8 suggest that the method provided by the present invention as illustrated in Figure 5 is appropriate, and will produce sufficiently accurate estimates.

As described above, the present invention provides a convenient method for obtaining dynamic performance data. One application of the present invention is now described.

A user has a persona] area network (PAN) comprising a plurality of different devices.

Each of these devices is under the control of the user. An application is to be implemented as a set of communicating component tasks which may operate on different devices of said plurality of devices. It is desirable to ensure that each task operates on the most suitable device. The present invention can be used to determine current performance data relevant to each device, and this data can then form the basis for device selection.

For example, a user may operate an email application which provides four different component tasks which may execute on different devices: 1. A user interface component receives requests ("Show me recent emails from X") from the user and displays the results on a nearby monitor.

2. An indexing/sorting component that extracts summary information from a database of emails.

3. A backup/synchronizing component that maintains copies of the email database on a variety of different devices.

4. A retrieval component that pulls down mails from a number of different mailboxes, filters duplicates and removes "spam" or other unwanted mails.

Each component requires certain properties from the device on which it runs. For example the indexing component could use much processing power when dealing with complex queries on a large database of emails. By identifying the operations which are key to the task performed by a given component, a performance value for that operation under the current load condition of each device can be obtained in accordance with the invention, and these values can be used to choose the best device for that component.

As devices enter and leave the PAN, the ideal device may change, and this may be handled to periodically running tests to obtain current performance data so as to ascertain the best device.

This invention may be realised as a set of portable Middleware components that can be installed in a large range of devices to predict code performance on other devices. Code running on one device could autonomously decide whether it would run effectively on another device. As a result it could instruct a code migration layer to move the component to the chosen device.

The preceding description has focused on implementation of the invention using the Java_ programming language, which is particularly advantageous because of the portability provided by the Java_ language. However it will be appreciated that the method proposed by the present invention is in no way restricted to a Java_ implementation.

The description set out above has been concerned with methods for obtaining performance data for operations execution on various computer devices (e.g. a desktop computer, a laptop computer, and a PDA). However it will be appreciated that the invention is not limited to such applications, but is instead widely applicable to all computer apparatuses, including, for example computer networks and micro processors.

The invention can be used for computer network performance monitoring. ]6

Claims

CLAIMS: I. A method for analysing performance of a computer apparatus,

comprising: executing at a first time a plurality of first operations to obtain first performance data; executing at a second time at least one second operation to obtain second performance data; determining a relationship between at least part of the second performance data and at least part of the first performance data; and computing from said relationship and at least part of said first performance data, third performance data indicative of performance of said first operations at said second time.
2. A method according to claim 1, wherein said computer apparatus is under a first load condition at said first time, and under a second load condition at said second time.
3. A method according to claim 1 or 2, wherein said at least one second operation is a subset of said plurality of first operations.
4. A method according to any preceding claim, wherein the relationship between at least part of the second performance data and at least part of the first performance data is a ratio of at least part of the second performance data to at least part of the first performance data.
5. A method according to any preceding claim, wherein the at least one second operation is an addition operation.
6. A method according to claim 5, wherein the at least one second operation is an integer addition operation.
7. A method according to any preceding claim, wherein the method is implemented in a machine independent programming language.
8. A method according to claim 7, wherein the machine independent programming language is compiled to generate machine independent byte code.
9. A method according to claim 8, wherein the method is implemented in JavarM.
10. A method according to any preceding claim, wherein obtaining said first performance data comprises executing the at least one first operation a plurality of times, and recording a time taken to execute said plurality of operations.
11. A method according to claim 10, wherein said at least one first operation is executed such that said first performance data is obtained with a predetermined accuracy.
12. A method according to any preceding claim, wherein obtaining said second performance data comprises executing the at least one second operation a plurality of times, and recording a time taken to execute said plurality of operations.
13. A method according to claim 12, wherein said at least one second operation is executed such that said first performance data is obtained with a predetermined accuracy.
14. A method according to any preceding claim, wherein the computer apparatus is a desktop computer, a laptop computer, a Portable Digital Assistant, or a mobile telephone.
15. A method according to any one of claim I to 13, wherein the computer apparatus is a microprocessor.
16. A method according to any one of claims 1 to 13, wherein the computer apparatus is a computer network.
17. A method for selecting a computer apparatus from a plurality of interconnected computer apparatuses, comprising: ]8 obtaining performance data indicative of the performance of each of said plurality of interconnected computer apparatuses using a method according to any one of claims 1 to 16; selecting one of said plurality of computer apparatus on the basis of the obtained performance data.
18. A method according to claim 17, wherein said plurality of computer apparatuses are connected by a computer network.
19. A method according to claim 17, wherein performance data indicative of the performance of each of said plurality of interconnected computer apparatuses is obtained by a server connected to said computer network.
20. A method according to claim 17, wherein said plurality of computer apparatuses are microprocessors within a single device.
21. A data carrier carrying computer program code means to cause a computer to execute procedure in accordance with the method of any one of claims 1 to 20.
22. A method for obtaining performance data substantially as hereinbefore described, with reference to the accompanying drawings.

22. A computer apparatus comprising: a program memory containing processor readable instructions; and a processor for reading and executing the instructions contained in the program memory; wherein said processor readable instructions control the processor to carry out the method of any one of claims 1 to 20.

23. A method for obtaining performance data substantially as hereinbefore described, with reference to the accompanying drawings.

Amendments to the claims have been filed as follows

A

CLAIMS: A method for ana]ysing performance of a computer apparatus, comprising: executing at a first time a plurality of first operations to obtain first performance data; executing at a second time at least one second operation to obtain second performance data; determining a relationship between at least part of the second performance data and at least part of the first performance data; and computing from said relationship and at least part of said first performance data, third performance data indicative of performance of said first operations at said second time.

2. A method according to claim 1, wherein said computer apparatus is under a first load condition at said first time, and under a second load condition at said second time.

3. A method according to claim or 2, wherein said at least one second operation is a subset of said plurality of first operations.

4. A method according to any preceding claim, wherein the relationship between at least part of the second performance data and at least part of the first performance data is a ratio of at least part of the second performance data to at least part of the first performance data.

5. A method according to any preceding claim, wherein the at least one second operation is an addition operation.

6. A method according to claim 5, wherein the at least one second operation is an integer addition operation.

7. A method according to any preceding claim, wherein the method is implemented in a machine independent programming language.

JO

8. A method according to claim 7, wherein the machine independent programming language is compiled to generate machine independent byte code.

9. A method according to any preceding claim, wherein obtaining said first performance data comprises executing said plurality of first operations and recording a time taken to execute said plurality of first operations.

lO. A method according to claim 9, wherein said plurality of first operations is executed such that said first performance data is obtained with a predetermined accuracy.

11. A method according to any preceding claim, wherein obtaining said second performance data comprises executing the at least one second operation a plurality of times, and recording a time taken to execute said plurality of operations.

12. A method according to claim l l, wherein said at least one second operation is executed such that said first performance data is obtained with a predetermined accuracy.

13. A method according to any preceding claim, wherein the computer apparatus is a desktop computer, a laptop computer, a Portable Digital Assistant, or a mobile telephone.

14. A method according to any one of claim 1 to 12, wherein the computer apparatus is a microprocessor.

15. A method according to any one of claims 1 to 12, wherein the computer apparatus is a computer network.

16. A method for selecting a computer apparatus from a plurality of interconnected computer apparatuses, comprising: ( , . I. obtaining performance data indicative of the performance of each of said plurality of interconnected computer apparatuses using a method according to any one of claims 1 to 15; selecting one of said plurality of computer apparatus on the basis of the obtained performance data.

17. A method according to claim 16, wherein said plurality of computer apparatuses are connected by a computer network.

18. A method according to claim 16, wherein performance data indicative of the performance of each of said plurality of interconnected computer apparatuses is obtained by a server connected to said computer network.

19. A method according to claim 16, wherein said plurality of computer apparatuses are microprocessors within a single device.

20. A data carrier carrying computer program code means to cause a computer to execute procedure in accordance with the method of any one of claims 1 to 19.

21. A computer apparatus comprising: a program memory containing processor readable instructions; and a processor for reading and executing the instructions contained in the program memory; wherein said processor readable instructions control the processor to carry out the method of any one of claims 1 to 19.