GB2406400A

GB2406400A - Method and apparatus for testing timing constraints

Info

Publication number: GB2406400A
Application number: GB0322242A
Authority: GB
Inventors: Paul Crispin Bristow; Clive Andrew Jervis; David Jonathan King
Original assignee: Motorola Inc
Current assignee: Motorola Solutions Inc
Priority date: 2003-09-23
Filing date: 2003-09-23
Publication date: 2005-03-30
Anticipated expiration: 2023-09-23
Also published as: GB2406400B; GB0322242D0

Abstract

A method (300) for generating an automatic test script for a system specification having one or more timing constraints comprises the steps of automatically mapping one or more automatic test scripts employing one or more timers to one or more timing constraints; and generating an automatic test script for a system specification there from. In this manner, an algorithm is able to augment, say, a tree representation of a test script using timers to test whether timing constraints on a notation, such as MSCs, are met. Thus, timing constraints are used to provide an indication that an event should (or should not) happen within a given window of time.

Description

a ec a ë . a a a 1 2406400

METHOD AND APPARATUS FOR TESTING TIMING CONSTRAINTS

Field of the Invention

This invention relates to a method and apparatus for testing timing constraints. The invention is applicable to, but not limited to, testing timing constraints in

test specifications for system specifications.

Background of the Invention

In recent years the use of automated testing techniques,

together with the introduction of formal system

specification languages, simulation and model checking tools, has greatly reduced 'product-to-market' cycle

time. A system specification includes system

requirements that are described in some formal notation such as Message Sequence Charts (MSCs). MSCs are often employed in systems for telecommunication processes, to generate test scripts automatically from the system specification. Basic MSCs and High-level MSCs (HMSCs) can be interpreted as a set of traces of events. HMSCs are basically directed graphs with nodes that are MSCs, hence allowing the graphical representation of combinations of MSCs, as shown in the simplified representation 100 in FIG. 1. In this case, a collection of individual test scripts in MSC form may be considered, in combination as an HMSC, as representing a test

specification for the telecommunication system

specification.

By analysing these test scripts, the functionality and operation of a practical telecommunication system may be c c . simulated. Advantageously, this enables flaws and discrepancies to be determined prior to manufacture and full system installation. Examples of automatic test script techniques can be found in "An Industrial Use of FP: A Tool for Generating Test Scripts from System Specifications", International Conference on Functional Programming, by P. Baker, C. Jervis, D. King, in Trends in Functional Programming, 2000, ISBN 18415002400, and "Design Tools for Requirements Engineering" by Gerard J. Holzmann, Doron A. Peled, and Margaret H. Redberg, Bell Labs Technical Journal, Winter 1997. In the context of automatically generating test scripts, a superficially simple specification can lead to a vast number of test scripts. For example, an elementary MSC with only 24 messages can generate 2,704,156 different tests! Tools such as "ptk" (a tool described in the above mentioned Baker et al. reference) may take the requirements specification in MSC form and automatically generate the complete set of test scripts that are required for conformance testing.

A next step in the evolution of automated testing techniques is to explore further opportunities for reducing cycle time, whilst still ensuring that the particular implementation exhibits all the behaviours

defined by its specification.

Tool vendors such as Telelogic produce test scripts from specifications such as MSC. However, these vendors' tools, such as Telelogic's Autolink, currently only allow the addition of timers. Therefore, timer events can be directly represented in additional test scripts to an . . e.

e * . . . C . . . . . . - - MSC, and are directly translated into code for those events. Thus, Autolink may provide the most basic support for timers, but it provides no support for timing constraints. Other automated test generation tools, such as, Teradyne's Testmaster, provide no support for timing issues in any way.

A need has therefore arisen for a mechanism to produce test scripts that accommodates timing constraints, to more accurately track the specification that is being represented. The inventive concepts herein described aim to advance the field of automated testing techniques by allowing the automatic generation of tests for timing constraints. This is a far more sophisticated method of expressing timing constraints than the current technique

of simply adding timers to a specification.

Summary of Invention

In a first aspect, the present invention provides a method for generating an automatic test script for a

system specification having one or more timing

constraints. The method comprises the steps of automatically mapping one or more test scripts employing one or more timing constraints to test scripts with one or more timers; and generating an automatic test script

for a system specification there from.

Preferably, the step of automatically mapping further comprises the steps of analysing each test script node, substantially in turn to: calculate what new timers to start to test timing constraints on that current node, obtaining timing information on a number of currently . . e . .

C .e..

running timers within the test script) generating a permutation tree of the currently running timers; and merging the permutation tree with the test script tree.

In this manner, timers within the final generated test script are used to test timing constraints in an iterative process until each and every node of the test script has been visited, and the timing constraints associated with each node/event been addressed.

In a second aspect, the present invention provides a method of testing a system specification, as claimed in Claim 20.

In a third aspect, the present invention provides a storage medium, as claimed in Claim 22.

In a fourth aspect, the present invention provides an apparatus, as claimed in claim 23.

Further aspects are as defined in the appended claims.

In summary, the present invention describes an algorithm for augmenting a tree representation of a test script using timers to test whether timing constraints on a notation, such as MSCs, are met. Preferably, the algorithm additionally allows the use of complex combinations of timing constraints via nesting and overlapping, as well as individual constraints. In this context, timing constraints provide an indication that an event should (or should not) happen within a given window of time. The preferred algorithm uses the starting, . . ...

c e . . . tee e. e stopping, and expiring of timers to implement various timing constraints.

A primary advantage provided by the inventive concept is the ability for a requirements writer to use high-level testing concepts (i.e. timing constraints) without being concerned how these would be implemented in an actual system of tests.

Brief Description of the Drawings

FIG. 1 shows an example of a known HMSC based on MSCs.

Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which: FIG. 2 illustrates an example of a message sequence chart having a timing constraint, namely a bounded constraint with an upper and lower bound, that can be used in the generation of an automatic test script in accordance with a preferred embodiment of the present invention; FIG. 3 illustrates a flowchart to generate an automatic test script having timing constraints in accordance with a preferred embodiment of the present invention) FIG. 4 illustrates an example tree diagram generated to show the tree permutations for a simple system of timers on a given test script node; FIG. 5 illustrates an example test script tree generated using an algorithm for a single bounded constraint.

. . . - . .. . e FIG. 6 illustrates a block diagram overview of a software tool system that is able to implement the algorithm according to the preferred embodiments of the present invention.

Description of Preferred embodiments

The preferred embodiment of the present invention proposes to add timing constraints to an MSC, and consequently to any HMSC employing such MSCs. The timing constraints augment the basic set of traces with special events representing a passage of time, thereby allowing time to be represented in a quantifiable manner. When generating tests that include timing constraints, tests are performed to ensure that timing constraints are met.

Notably, these tests make appropriate use of timers.

Timers are selected as the most appropriate vehicle for this purpose, as they are currently the only time-based operation available at the test script level. Time intervals are described in terms of two time points, which can be either relative to a given event or an absolute value. Analysis of the various types and timing of situations that may arise is required, in order to decide an appropriate use of timers within the system.

A system specification, i.e. a specification of the functional requirements of a system is often described in terms of a message sequence chart (MSC). However, a skilled artisan would recognise that this is not the only

form such a specification may take. The preferred

embodiment of the present invention is described with a a a c a a. a.' a ce a a reference to the MSC-based algorithm currently implemented by PTK_. However, it is within the contemplation of the present invention that the inventive concepts herein described are equally applicable to other

test specification algorithms and techniques for

describing the functional requirements of a system and are not limited to use in MSCs or with PTK.

In accordance with the preferred embodiment of the present invention, the proposed method describes time as being relative to any two messages/events rather than an absolute value. In this regard, the technique uses a textual mark-up in comment boxes attached to the events.

However, a skilled artisan will appreciate that it would be straightforward to convert absolute timing constraints to relative constraints with the addition of extra timers.

The preferred representation implements one or more (and preferably all of four types of timing constraint: (i) A bounded constraint with an upper and lower bound, as illustrated in FIG. 2; (ii) A maximal constraint, where two events may be separated by less than a maximum timed (iii) A minimal constraint, where two events must be separated by at least a minimum time; and (iv) A wait constraint, where a test script must wait until a timer expires before sending a message. An error will be flagged if the timer expires and the test script is not ready to send the message.

The timing constraints (i), (ii) and (iii) can be used when the final event in a timing constraint is a c. - . ' a a . ,,,,, . . * * a

_

receive' message. If the final event is a 'send' message, the fourth timing constraint can be used. It is noteworthy that 'receive' messages may have any number of timing constraints attached to them. However, any individual pair of events ending in a 'receive' message may not have more than one timing constraint. A 'send' message may have only one wait constraint attached, as there is direct control over sending events.

Although this representation has been used in PTK and as part of the input for the algorithm, it is envisaged that the algorithm would work equally well with any other representation of relative constraints.

Referring now to FIG. 3, a flowchart 300 illustrates an algorithm for using timers to test whether timing constraints are met, according to a preferred embodiment of the present invention. The algorithm takes as its input a test script tree, as shown in step 305. For the first iteration of the algorithm, when there is no current node, the head of the test tree is taken. The algorithm then proposes visiting each node in turn, in a top-down' fashion, adding new sub-trees as appropriate based on the current node. These sub-trees can vary widely in complexity from a couple of nodes representing a timer that expires, followed by a verdict, to an almost mirror image of the test tree below the current node.

The algorithm can be described as encompassing at least the following steps. For each test node: (i) Obtain all currently running timers, as in step 315; c e c 1 C 1 C ale ale '. I e c a C C C. (ii) Generate a permutation tree of the running timers, in step 325; and (iii) Merge the permutation tree with the test script tree and add 'start' instructions for any new timers, as shown in steps 330 and 335.

In accordance with a preferred embodiment of the present invention, the algorithm is able to handle a large number, as well as many permutations, of timers. When a number of timers are running concurrently, any, some, or all of them may 'time-out'. Therefore, the preferred algorithm must ensure that all time-outs are caught.

Hence, the algorithm has been developed to capture all timing possibilities.

Every path through a test script tree will have a leaf node; these leaf nodes are the final events in each path.

In a correctly formed test tree, the leaf nodes represent the test script returning information to the test execution system about the outcome of the test script.

The outcomes are known as 'verdicts' and typically have values such as 'pass', 'fail', or 'inconclusive'.

Furthermore, the complexity of the permutation tree generation in step 325 can be reduced by using knowledge of the timer types to halt execution early. For example, if timer-1 and timer-2 are the lower and upper bound timers respectively implementing a bounded constraint, the expiration of timer-2 prior to timer-1 would result in an 'inconsistent' verdict in the final test script.

Hence, any path in the permutation tree in which timer-2 appears before timer-1 need be expanded no further.

A:. . en: ë ce. :e e: : : : . .. . . During the merging process in step 330, extra verdicts will be added to the test script tree, for example a timer which expires will be followed by the cancelling of all other running timers and the output of a verdict.

These extra verdicts will all be either 'fail' or inconsistent' verdicts. For example: (i) A 'fail' verdict would arise when an upper bound timer on 'receive' times out before an event occurs; (ii) An 'inconsistent' verdict would arise in a similar case if the upper bound timer expired but the test script was still waiting for a previous constrained event to occur.

A simple rule can be applied generally for deciding between 'inconsistent' or 'fail' verdicts when non- conforming events occur. When a timer that should not expire expires, an 'inconsistent' verdict is returned if, and only if, there are any other running timers that are required to time-out or be stopped prior to it.

Otherwise a 'fail' verdict is returned.

For clarification purposes, a timer 'a' is required to finish (either by time-out or by being stopped) before a timer 'b' if: (i) 'a' is attached to an event preceding the event that 'b' is attached to; or (ii) If 'a' is a lower bound timer and 'b' is an upper bound timer in the same bounded constraint.

Once the timer permutation tree in step 330 has been created, the test and timer trees need to be merged, as shown in step 335. FIG. 4 illustrates an example tree e eve e e e e ee e e ae e e ee cce ec-se e ee ë e ë e se< e ea e diagram 400 generated to show the permutations for a simple system employing three timers '1', '2', and '3'.

By merging this timer tree in step 335, at each node in the test script tree, it is possible to create a new test script in which every possible combination of timer expiration is checked for. In many cases, a test script is unlikely to have many more than three timers running at a given time, resulting in a tree of manageable size as shown in FIG. 4.

The algorithm now checks to see whether there are any multiple receive instructions, as shown in step 340. The preferred procedure for handling multiple receives is described below. If there are no multiple receives in step 340, the algorithm moves to the immediate child node of the current test node, in step 310. The algorithm then moves to step 330, where the test and timer trees associated with the new child node are merged. The previous process then repeats in an iterative manner until all of the nodes have been tested.

The algorithm so far described makes the assumption that there is only one 'send' or 'receive' in any node list in the test script tree. This is true in PTK for send messages, when two possible sends occur at a point in an MSC, as two test scripts are generated, one for each possibility. However, this is not the case with receives', and without additional support the algorithm fails to generate all possible time-outs at any point for each receive node.

The algorithm of the preferred embodiment of the present invention has been developed to handle multiple receives, . . c.

e e c e e ce. see e e e see .e whilst merging the list of lists of test trees generated by add_constraints. If a node list in the original test tree contains more than one receive in step 340, the algorithm deletes all but one copy of the repeated time out nodes, as shown in step 345. The sub-trees of each of the deleted nodes (i.e. the deleted nodes' children) are then copied to the remaining node, i.e. the one node that was not deleted, in step 350.

The algorithm then performs a check to see if there is any repetition of these deleted nodes in the child nodes, in step 355. If there is any repetition of these deleted nodes in any of the child nodes, in step 355, the algorithm obtains the immediate child nodes of the remaining node, in step 360. If there is no repetition of these deleted nodes within any of the child nodes, in step 350, the algorithm obtains all new timers, in step 320.

Hence, the algorithm recursively removes any repetition of timers in the tree caused by the multiple receives.

These multiple receives can arise from a number of scenarios, such as MSCs with more than two instances and an appropriate combination of messages, co-regions, or alternative expressions.

In practice, the merging process in step 330 is a particularly complex function within the algorithm, as it must take account of differing types of constraints and messages. Therefore, the tree merging process is explained in greater detail below. e..

e e . e. e As mentioned previously, the merging process takes place on a per node basis for each node of the test script tree. Each node is augmented by a tree of timers, but is dependent upon rules for each message and constraint type. Example code for each message type is as follows: Each of the message rule sets given below preferably include two stages: (i) A sibling stage, which generates other nodes that will appear as alternatives to the current timer node that is being generated (i.e. at the same level in the tree); and (ii) A children stage, which generates a series of sub-trees that are child nodes beneath the current timer node that is being generated.

The two stages are combined, along with a time-out the current timer, as follows: Node (Time-out "current timer", children) :: siblings This code creates a new node list that is used to replace the test script node that is being augmented.

The preferred rule sets for augmenting each of the above message types is as follows. The algorithms are described in a mostly language-independent pseudo-code but make use of the Standard Mark-up Language (SML) list constructor ':', the list append operator '@', the empty list '[ ]', the list functionals 'map' and 'concat', and a tree data structure defined in SML as follows: datatype 'a tree = Node of 'a * ('a tree) list cee e e e a'. ec. ee e e .e e ee The algorithm employed in the preferred embodiment of the present invention applies the following definitions, relative to the current node in the test script tree, in order to interpret the above rules: (i) Free timer - a timer is free to expire at any point without creating an error (i.e. result in a 'fail' or 'inconclusive' verdict), such as a timer implementing the lower bound of a bounded constraint terminating on an event occurring after the current one.

(ii) Required timer - a timer which must expire before an event can occur, a timer implementing the lower bound of a bounded constraint terminating on the current event is one example of this.

(iii) Disallowed timer - a timer which must not expire before an event can occur, such as a timer implementing the upper bound of a bounded constraint terminating on the current event.

(iv) Wait timer - a timer implementing a wait constraint.

The preferred embodiment of the present invention also applies the following abbreviations for various data structures: (i) 'hpt' - the head of the timer permutation tree; (ii) 'tpt' - the tail of the timer permutation tree (the list of children below 'hpt')i (iii) 'tt' - the test script tree rooted at the current node; (iv) 'htt' - the head of the test script tree tt'; # . # # # # . . . # # # # # # ## # # # # e # -e e . (v) 'ttt' - the tail of the test script tree tt'i (vi) 'rl' - the list of running timers) (vii) 'sl_pre' - the list of timers started before the current node (due to new constraints)) (viii) 'sl_post' - the list of timers started after the current nodes (ix) 'stl_post' - the list of timers to be stopped after the current nodes (x) 'first_time' - a boolean flag indicating if this is the first time a given branch of the code has been reached for a call to add_constraintsi and (xi) 'pending_wait' - used to determine when there is a send node with a wait construct outstanding? This is used to halt recursive descent into receive node from a parent send node (see, for example, the 'receives with constraints' rule).

Most of these data structures are either given directly, in the case of the trees, or derived from the constraints on the test script trees and the list of running timers.

The exceptions are for 'first_time' and 'pending_wait', which are effectively trivial assignments.

The aforementioned data structures also assume the existence of the following functions: (i) Merge_timers_and_nodes - takes the current node, a list sl_pre, a list sl_post, a list stl_post, and a list of test script trees. It puts starts for each timer in sl_pre before the current node, starts for each element in sl_post after the current node, and puts the list of test script trees below the final start in sl_post.

. . . * . . . . . . . (ii) Add constraints - a function that takes a list of running timers, a list of test trees, and calls the appropriate rule from those below on each element in the list of test trees, merges the resulting list of list of test trees into a list of test trees and returns it.

An initial call to this function starts the merge process.

(iii) Error_verdict - a function to add in cancels for any running timers and calculate whether a verdict should be fail or inconclusive as a result of a given timer expiring. This uses the verdict rules described earlier.

(iv) Send/receive+/receive- - synonyms for the send, receive with constraints, and receive without constraints rules respectively. These all take the list of running timers and the test script tree 'tt' as arguments.

Each of the above rule sets, in addition to the current test script node and timer permutation tree, requires a list of the currently running timers. Lists of required, free, disallowed, and wait timers are also derived from this list of the currently running timers.

Example code employed in the preferred embodiment of the present invention is provided below for the different message types of 'send', 'receives with constraints' and receives without constraints', when applying the aforementioned two stage message rule sets and definitions.

eee e . . . . - e Be e -a Sends Sibling stage If not (required list = [ ]) then no siblings else if (wait timer (hpt)) then error_verdict else if (first_time) then add_constraints (rl + sl_pre) ttt else no siblings.

Children stage (if (free timer (hpt)) then concat (map (send (rl - hpt) tt) tpt) else t]) @ (if (not (free timer (hpt))) andalso (not (required list = t])) then error_verdict else if (required list = t]) then merge_timers_and_nodes htt sl_pre sl_post t] (add_constraints (rl - hpt + sl_pre + sl_post) ttt) else t]) Receives with constraints Sibling stage if (required list = [ ]) then if (first_time) then merge_timers_and_nodes htt [ ] sl_post stl_post @ . * - e * * * * *. * * * * * e - - (add_constraints (rl + sl_pre + sl_post stl_post) ttt) Children stage (if (disallowed hpt) then error_verdict else if (pending_wait) then concat (map (fn x => if (disallowed x) then error_verdict else nil) else concat (map (receive+ (rl - hpt) tt) tpt)) @ (if (required list = l]) then merge_timers_and_nodes htt sl_pre sl_post l] (add_constraints (rl - hpt + sl_pre + sl_post - stl_post) ttt) else [ ]) Receives without constraints Sibling stage if (first_time) then merge_timers_and_nodes htt l] sl_post stp_post (add_constraints rl ttt) Children stage (if (disallowed hpt) then error_verdict else concat (map (receive- (rl - hpt) tt) tpt)) @ (if (required list = [ ]) then . . . - . . merge_timers_and_nodes htt [ ] sl_post stp_post (add_constraints (rl - hpt) ttt) else [ ]) Thus, in this manner, a test tree and timer tree merging process can be performed within the algorithm, taking into account differing types of timer constraints and messages.

In an enhanced embodiment of the present invention, the algorithm additionally allows the use of complex combinations of timing constraints via nesting and overlapping, as well as individual constraints. In this context, nesting of timing constraints is when one timing constraint is entirely contained within another.

Overlapping of timing constraints is when either a start or an end of a timing constraint, but not both, occurs between the start and end of one or more other timing constraints. As will be appreciated, this can give rise to complex situations, for example a lower bound timer may expire. Hence, even though a given message may be received, a timer from a further timing constraint may also need to expire before the message can 'truly' be received. Therefore, when merging trees together the preferred embodiment of the present invention takes into account all running timers and not just the ones for the current node, as well as their types and associated semantics when creating the final tree.

Referring now to FIG. 5, an example test tree 500 shows a test script tree generated using the preferred algorithm for a single bounded constraint. Note that this is only an example test script tree, which does not include every '8 '8 8 8'a 86.'8 8 C 8 8 8 8 8 -81 8 aspect as the original example of FIG. 2 did not include every kind of constraint.

It is within the contemplation of the invention that configuring, or adapting, any suitable apparatus may implement the aforementioned embodiments. For example, a computer or other processing apparatus may be adapted.

Alternatively, a processor implementing processor- implementable instructions and/or stored on a suitable storage medium, such as computer memory, hard disk, floppy disk, read only memory (ROM), programmable ROM (PROM), random access memory (RAM), etc. may implement the processes/algorithm described. The processor may be a computer, a network of computers, or one or more dedicated processors.

It is also within the contemplation of the present invention that the inventive concepts described herein could be used with any description language that employs the following: (i) Has an equivalent concept of timing constraints, (ii) Has a timer operation to map on to, and (iii) Can be represented in a tree structure (noting that looping constructs cannot unless the loop is itself a single node).

The method is intended to be included in a software tool whose input is a test script tree, as described above.

The output of the software tool is designed to be a system simulation that incorporates timing constraints using timers. . . . . . . . c c c c c . . .

A block diagram example of a software tool/system 600 that can be adapted to implement the inventive concepts of the present invention is illustrated in FIG. 6. The

software tool/system 600 includes a specification

function 620 that is operably coupled to a man machine interface (MMI) 630. In accordance with the preferred embodiment of the present invention, the specification function 620 includes the system specification 622, for example a TErrestrial Trunked RAdio (TETRA) specification as defined by the European Telecommunications Standards

Institute (ETSI). The system specification 622 is

operably coupled to a machine operating system interface 626. The machine operating system interface 626 interfaces with the man machine interface (MMI) 630. The MMI 630 includes a computation process 632, which accesses one or more physical memory storage devices 634 in order to obtain data to assist the aforementioned methods. The computation process 632 is also operably coupled to a visual display 636 and a user input device 638 such as a keyboard.

The above-described software tool/system 600 is merely an example of a number of system/simulation configurations that can be adapted to employ the inventive concepts hereinbefore described. It is also within the contemplation of the present invention that the inventive concepts herein described are equally applicable to any asynchronous event driven scenario that can be represented by MSCs, e.g. telecommunications, distributed computing, database systems, etc. Furthermore, it is within the contemplation of the present invention that it may, in some instances, be implemented on any general purpose computer, for example the algorithm may run on a c . ë .. e6e ce c e . ee. e ë general computer emulating the above hardware or as part of a simulation that involves no specific hardware. In addition, it is envisaged that the algorithm may be employed in a mixed mode of operation, whereby it is used to test, say, a first half of a test script, when the remaining half has yet to be designed/built.

It will be understood that the algorithm to support timing constraints in automatic test scripts, as described above, provide at least one or more of the following advantages: (i) The inventive concepts of the algorithm allow a requirements writer to use high level concepts (i.e. timing constraints) without worrying how these would be implemented in an actual system of tests; and (ii) The inventive concepts of the algorithm enable system specifications that contain concurrent time-critical processes to be dealt with in an improved manner. This is an important advantage in the

telecommunications field where a large number of

processes operate in a concurrent fashion.

Whilst specific, and preferred, implementations of the present invention are described above, it is clear that one skilled in the art could readily apply variations and modifications of such inventive concepts.

Thus, a mechanism to support timing constraints in automatic test scripts has been provided where the disadvantages described with reference to prior art arrangements have been substantially alleviated.

Claims

a . ea.I.

Claims 1. A method (300) for generating an automatic test script for a system specification having one or more timing constraints, the method characterized by the steps of: automatically mapping one or more automatic test scripts employing one or more timers to one or more timing constraints; and generating an automatic test script for a system

specification there from.
2. A method (300) for generating an automatic test script according to Claim 1, wherein for a plurality of test nodes within the system specification, the step of automatically mapping comprises the steps of: analysing each test script node, substantially in turn, and identifying new timers to start to test timing constraints on each current test script node.
3. A method (300) for generating an automatic test script according to Claim 1 or Claim 2, the method further characterized by the steps of: obtaining (315) timing information on a number of currently running timers within the test script; generating (325) a permutation tree of the currently running timers; and merging (330) the permutation tree with a test script tree.

. . . . c . . . e.e
4. A method (300) for generating an automatic test script according to any preceding Claim, the method further characterized by the step of: testing one or more timing constraints using the one or more timers.
5. A method (300) for generating an automatic test script according to any of preceding Claims 2 to 4, wherein the number of currently running timers comprises substantially all of the currently running timers within the test script.
6. A method (300) for generating an automatic test script according to any of preceding Claims 2 to 5, wherein the method further comprises the steps of adding start times for any new timers (335) generated following a merging step.
7. The method (300) for generating an automatic test script according to any preceding Claim, wherein the one or more timers indicate time as being relative to any two messages or events.
8. The method (300) for generating an automatic test script according to any preceding Claim, wherein the one or more timing constraints comprise one or more of the following: (i) A bounded constraint with an upper and lower bound; (ii) A maximal constraint, where two events may be separated by less than a maximum time; (iii) A minimal constraint, where two events must be separated by at least a minimum time; Ace e e e e e .e ee..e e e ë. e. (iv) A wait constraint, where a test script waits until a timer expires before sending a message.
9. The method (300) for generating an automatic test script according to Claim 8 wherein the timing constraint (i), (ii) or (iii) is used when a final event in a timing constraint is a 'receive' message.
10. The method (300) for generating an automatic test script according to Claim 8, wherein timing constraint (iv) is used if the final event is a 'send' message.
11. The method (300) for generating an automatic test script according to any of preceding Claims 3 to 10, wherein the step of merging (330) comprises the step of: adding one or more outcomes from the generated test script to the test script tree.
12. The method (300) for generating an automatic test script according to any of preceding Claims 2 to 11, wherein a head of the test tree is taken for a first iteration of the method when there is no current node.
13. The method (300) for generating an automatic test script according to any of the preceding Claims, wherein the step of mapping comprises visiting each node of the test script in turn and, if appropriate, adding one or more new sub-tree(s) based on a current node.
14. The method (300) for generating an automatic test script according to any of the preceding Claims, further characterized by the step of: checking (340) for multiple receive instructions. e e

e e e e ece ee. ce e e e ear e ee e
15. The method (300) for generating an automatic test script according to Claim 14, wherein if there are no multiple receives, the step of automatically mapping one or more automatic test scripts to one or more timing constraints is performed on a child node of the current test node.
16. The method (300) for generating an automatic test script according to any of the preceding Claims, wherein the step of automatically mapping comprises applying one or more message rule sets.
17. The method (300) for generating an automatic test script according to Claim 16, wherein the one or more message rule sets comprises: (i) A sibling stage, which generates other nodes that will appear as alternatives to a current timer node that is being generated; and/or (ii) A children stage, which generates a series of sub-trees that are child nodes beneath a current timer node that is being generated.
18. The method (300) for generating an automatic test script according to any of preceding Claims 3 to 17, wherein the step of obtaining (315) timing information on a number of currently running timers comprises deriving a list of one or more of the following timers: required timer, free timer, disallowed timer, and/or wait timer.
19. The method (300) for generating an automatic test script according to any of the preceding Claims, wherein . . # eve ce. ce e e e ë..

the test script is generated using a representation of relative timing constraints in PTK.
20. A method of testing a system specification

characterized by the step of generating an automatic test script according to any of the preceding Claims.
21. The method according to any preceding claim, wherein the system specification is a telecommunication system.
22. A storage medium (634) storing processor- implementable instructions for controlling one or more processors to carry out the method of any of the preceding Claims.
23. Apparatus adapted to implement the method of any of the preceding Claims.
24. A method substantially as hereinbefore described with reference to, and/or as illustrated by, FIG. 3 of the accompanying drawings.
25. Apparatus substantially as hereinbefore described with reference to, and/or as illustrated by, FIG. 6 of the accompanying drawings.