GB2377515A

GB2377515A - Test Specifications for System Specifications

Info

Publication number: GB2377515A
Application number: GB0116699A
Authority: GB
Inventors: William Peter Robin Mitchell; David Jonathan King; David King
Original assignee: Motorola Inc
Current assignee: Motorola Solutions Inc
Priority date: 2001-07-09
Filing date: 2001-07-09
Publication date: 2003-01-15
Also published as: GB0116699D0

Abstract

A method of generating a test specification from a system specification, comprising generating the test specification as a finite state automaton (20). The system specification may be in the form of a message sequence chart (1). Testing using the test specification is also described. The test specification describes a collection of test scripts in the form of a single finite state automaton (20). System specifications containing concurrent iterating processes can be accommodated. This makes the invention suitable for use with telecommunication systems.

Description

TEST SPECIFICATIONS FOR SYSTEM SPECIFICATIONS Field of the Invention This invention relates to test specifications for system specifications, and testing using such test specifications. The invention is applicable to, but not limited to, system specifications in the form of message sequence charts.

Background of the Invention In the field of this invention it is known to generate tests scripts automatically from a system specification.

A system specification includes system requirements that are described in some formal notation such as Message Sequence Charts (MSCs). This is often employed in systems for telecommunications processes. In this case the collection of individual test scripts can be considered to represent a test specification for the system specification. Examples of these techniques can be found in"An Industrial Use of FP: A Tool for Generating Test Scripts from System Specifications", International Conference on Functional Programming, by Paul Baker, Clive Jervis, David King, in Trends in Functional Programming, 2000, ISBN 18415002400, and"Design Tools for Requirements Engineering"by Gerard J. Holzmann, Doron A. Peled, and Margaret H. Redberg, Bell Labs Technical Journal, Winter 1997.

A superficially simple specification can lead to a vast number of test scripts. For example an elementary MSC with only 24 messages can generate 2,704, 156 different tests! Tools such as"ptk" (a tool described in the above mentioned Baker et al. reference) can take the requirements specification in MSC form and automatically generate the complete set of test scripts that are required for conformance testing. However, ptk is limited as to the total number of test scripts it can generate.

Two particular problems arise when large numbers of test scripts derive from a system specification. The first problem is how to construct such a potentially huge number of test scripts. The second problem is how to analyse all these test scripts, since it may well be the case that many of them are not needed or are not important.

Another issue with conventional test specifications comprising multiple test scripts is that, if practitioners wish to simulate a system in SDL (ITU Recommendation Z. 100, Specification and Description Language), then they have to write the code by hand. This is time consuming and error prone. When used in simulation of a system in SDL, the present invention aids the finding of flaws in the design of a system specification, thus tending to improve system development cycle time.

Known prior art relating to the field of this invention is found in"Synthesizing ROOM Models from Message Sequence Chart Specifications"by Stefan Leue, Lars Mehrmann, Mohammad Rezai, obtainable at:

http ://fee. uwaterloo. ca/-sleue/rnsc. html#tr98-06 ; and"Interpreting Message Flow Graphs, Formal Aspects of Computing", by Stefan Leue, P. Ladkin, 7 (5) : 473- 509,1995 ; and"Model checking of message sequence charts"by R. Alur and M. Yannakakis, Proceedings of the Tenth International Conference on Concurrency Theory, Springer Verlag, 1999 obtainable at: http ://www. cis. upenn. edu/-alur/onlinepub. html.

These papers detail how to construct a finite state automaton from a simple MSC and how to perform"model checking analysis"of this. Performing model checking analysis does not constitute generating a test specification, because model checking is only concerned with investigating the inter-relationships of the states in the state machine. Moreover, the algorithms disclosed in these papers, although compatible with non-concurrent iteration, are not compatible with MSCs (or other system specifications) comprising concurrent iteration between processes.

It would therefore be desirable to generate a different type of test specification instead of or in addition to a

collection of test scripts, thereby alleviating the above mentioned problems.

Summary of Invention In a first aspect, the present invention provides a method of generating a test specification from a system specification, as claimed in claim 1.

In a second aspect, the present invention provides a method of testing a system specification, as claimed in claim 8.

In a third aspect, the present invention provides a method of testing a system, as claimed in claim 9.

In a fourth aspect, the present invention provides a storage medium storing processor-implementable instructions, as claimed in claim 11.

In a fifth aspect, the present invention provides apparatus, as claimed in claim 12.

Further aspects are as claimed in the dependent claims.

The present invention provides a test specification comprising a single finite state automaton. Thus, compared to known test specifications, an alternative representation is provided that in essence can capture all the test scripts in a single concise construct.

From this all the individual test scripts can be generated, if that is necessary or desirable. Also, the representation can be analysed to discover significant information about the requirements without having to generate the tests. For example it will be possible to implement the finite state automaton directly in the above mentioned SDL.

In summary, the present invention provides a method detailing how to represent a complete set of test scripts in the form of a finite state automaton (FSA). Moreover this is the smallest possible FSA that can represent the traces given by the requirements specification. The FSA is a very compact way of representing all of the possible test scripts. For example, considering the abovementioned example of an embodiment of an elementary MSC with 24 messages, the invention can describe the potential 2, 704, 156 test scripts with a finite state automaton (FSA) that uses only 144 states.

Brief Description of the Drawings Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which : FIG. 1 illustrates a message sequence chart ; FIG. 2 illustrates a simple finite state automaton ;

FIG. 3 is a flowchart showing the process steps of an embodiment of the present invention; and FIG. 4 illustrates a finite state automaton generated for the message sequence chart of FIG. 1.

Description of Preferred Embodiments A system specification, i. e. a specification of the functional requirements of a system, is often described in terms of an MSC, though this is not the only form such a specification may take. The purpose of such a specification is to describe a partial ordering of the events occurring in the system. It is then possible to regard a specification as a partial order on a set of events, for the purposes of describing the preferred embodiment of this invention. The partial ordering describes the relative order in which the events are to occur in practice as specified by the functional requirements.

Also for the purposes of describing the preferred embodiment of this invention, a conformance test derived from the specification as a total ordering, which extends the partial order of the extension, will be identified.

Assume that a functional requirement in the form of an MSC M, that does not contain any loop constructs, is provided. We will write x < y to denote that the functional requirement specification represented by M asserts that event x occurs before event y.

FIG. 1 shows a simple example of an MSC 1. This MSC 1 has two parts, i. e. a part 2 drawn on the left-hand side, and a part 4 drawn on the right-hand side.

Part 2 comprises, using conventional MSC terminology, two instances 6,8, with a message a (indicated by reference numeral 10) between the instant 6 and the instant 8. In FIG. 1, the expression ! a"signifies"send message a", and the expression ? a"signifies"receive message a".

Part 4 comprises two instances 14,16, with a message b (indicated by reference numeral 12) between the instant 14 and the instant 16. The expression ! b signifies "send message b", and the expression" ? b"signifies "receive message b". The instances 6,8 of the first part 2 do not communicate with the instances 14,16 of the second part 4.

Further suppose that various loop constructs are now added to M. A loop construct represents an iteration of the messages that are within its scope. Loop 18 in FIG. 1 is an example of such a loop. Loop 18 is a single loop introduced around the single message b. This loop 18 is specified as"loop < 0, infinity > , which is notation specified in International Telecommunication Union (ITU) recommendation Z. 120, causing message b to be repeated any finite number of times within the specification. Note also that the loop 18 only extends over the second part 4 part of the MSC 1, and does not extend over the first

part 2 of the MSC 1. A consequence of this will be explained later below.

The introduction of a loop (or loops) adds further constraints to the order in which events can occur. In particular it imposes a relationship on the repeated occurrences of events that occur within the scope of the loop.

If an MSC contains n loops let Ll to Ln denote the additional relationship that these loops impose on the events in M. Thus in the example of FIG. 1, we have only one loop and it imposes that a new ? b can occur before a ! b if the message is repeated. We write this as? b L1 ! b.

In general for the i-th loop construct, we write x Li y to denote that the loop has introduced an additional relationship of events x and y.

Define x Li y whenever y is any event which is minimal with respect to < within the region bounded by the loop, y < x, and x is maximal with respect to < within the region bounded by the loop. Here a'minimal'event within the loop is an event that has no predecessor within the loop. A'maximal'event within the loop is an event that has no successor within the loop.

The generation of a finite state automaton for the above described MSC 1 will now be described.

A finite state automaton (FSA) is a set of states together with a set of labelled transitions between the states. In addition, one of the states is termed the start state, and some of the states are referred to as accepting states. An FSA has a very simple visual form, considering the level of information contained therein.

As a simple example, shown in FIG. 2, let the set of states be A, B, C and take as the labelled transitions A < a > B, B < b > A and A < c > C. Here we write X < y > Z to denote there is a transition from state X to state Z which has label y. Let A be the start state and C be the only accepting state. View the states as nodes in a graph, and the transitions as labelled arrows between the nodes. So there are three nodes A, B and C, an edge from A to B labelled with a, an edge from B to A labelled with b, and an edge from A to C labelled with c.

There are infinitely many paths going from A to C in the graph. For example one path is A < a > B < b > A < a > B < b > A < c > C.

Given any path from A to C, we can write out the labels of the transitions that occur. For the example just given the sequence will be < a, b, a, b, c > . The set of all such sequences is defined to be the language accepted by the FSA.

A specification containing loops is an example of a recursive specification, for which it is much more complicated to determine what the causal relationship between events is.

The field of finite state automatons is well documented.

Further details may be found, for example, in an elementary text such as Introduction to Automata Theory, Hopcroft and Ullman. Also, note that a finite state automaton is sometimes referred to as a finite state machine.

The following notation will be used in the description of the finite state automaton (FSA) generated in this embodiment: First we need some notation to describe the FSA with. An FSA is a set of states S, and a set of transitions T.

Each transition is a triple < S1, a, S2 > , which denotes the transition of state Sl to state S2 with a transition labelled by a.

Let R denote any kind of relationship between events in M. Write (x R y) to denote that x is related to y by the relation R.

Define: n (R) (a) = the set of those events b where (a R b) and there is no c where (a R c) and (c R b).

For a set of events X, the set of minimal events according to R is the set: m (R) (X) = the set of events a in X where there is no b in X such that (b R a).

For a set of events U, let [U, R, u] be the union of the set m (R) (U- {u}) and the set n (R) (u). Define the set trns (R) (U) = the set of transitions < U, u, [U, R, u] > where u is any element of U.

For a set of binary relations SR, define the complete set of transitions as: set~trns (SR) (U) = the union of all the sets of transitions trns (R) (U) where R is one of the relations in the set SR.

If S is a set of sets, then define (concat S) = the union of the sets contained in S.

For a set of transitions Tr, define nx (Tr) = the set of sets V where < U, a, V > is one of the transitions in Tr.

In this embodiment, an FSA is constructed that describes the set of test traces that are defined by the specification M. FIG. 3 is a flowchart showing the process steps of this embodiment.

At step s2, initialise the sets of states SX = the set of all events in M. At step s4, also initialise the set of transitions TX to be the empty set. At step s6, an algorithm is executed to repeatedly modify these values until finally they define the desired FSA.

In this embodiment, the algorithm executed in step s6 is as follows:

Further define RM to be the set of relations Li (defined by the loop constructs in M) and the partial ordering < .

For this algorithm we require auxiliary variables nextStates and nextTrans to store intermediate values.

Initially set nextStates = SX and nextTrans = the empty set.

The FSA algorithm: Repeat the following steps until the set nextStates is empty: nextTrans = the function concat applied to the set of values set trns (RM) (U) where U is an element of nextStates nextStates = nx (nextTrans) SX = the union of the previous value of SX and nextStates TX = the union of the previous value of TX and nextTrans Once the above steps have completed the final values, SX defines the set of states for the finite state automaton, and TX defines the transitions of TX of the finite state automaton. The start state of the finite state automaton is the set of all events in M, and the only accepting state is the empty set.

In the case of the MSC 1 shown in FIG. 1, the above algorithm produces a finite state automaton 20 of the form shown in FIG. 4. The FSA 20 comprises nine states (nodes) 21-29, and fifteen transitions (arrows) 31-45.

The start state 21 i. e. { ! a,! b} is at the top of the diagram and the accepting state 29 i. e. {} is at the bottom of the diagram.

By moving from the start state to the accepting state, one produces a test script. As a first example, if the following sequence is considered: transition 31transition 35-transition 41-transition 44, then the following list of symbols is produced: ! a- ! b- ? a- ? b.

This list of symbols specifies one possible test for the MSC, namely a test sequence of the following scenario: send message a-send message b-receive message areceive message b.

As a second example (illustrating the effect of the loop 18) if the following sequence is considered: transition

31-transition 34-transition 40-transition 39transition 40-transition 39-transition 40- transition 44, then the following list of symbols is produced : ! a- ? a- ! b- ? b- ! b- ? b- ! b- ? b. This list of symbols specifies another possible test for the MSC, namely a test sequence of the following scenario: send message a-receive message a-send message b-receive message b-send message b-receive message b-send message b-receive message b.

Then, to perform the above tests for the message sequence chart 1, the identified sequence is implemented and the message sequence chart 1 is observed to see if it provides the correct response.

Additionally, in the case of a system that has actually been built or otherwise implemented according to the system specification represented by the message sequence chart 1, the above tests may be implemented in equivalent fashion for the system itself. The identified sequence is implemented in the actual system, and the system is observed to see if it provides the correct response. One type of system to which this can be applied is a telecommunication system.

Furthermore, individual test scripts can be generated from the finite state automaton if necessary or desirable. Also, the representation can be analysed to discover significant information about the requirements without having to generate the tests. For example it is possible to implement the finite state automaton directly in SDL.

It will be understood that the methods given above for describing a collection of test scripts in the form of a single finite state automaton enable specifications that contain concurrent iterating processes to be dealt with.

This is in contrast to prior art arrangements that can only cater for iteration where the processes iterate in a non-concurrent fashion. This is an important advantage, since in the telecommunications field almost all processes iterate in a concurrent fashion.

Furthermore, in the above description of the MSC 1, it was noted that the loop 18 only extends over the second

part 4 part of the MSC 1, and does not extend over the first part 2 of the MSC 1. Moreover part 2 does not communicate with part 1. From a conceptual viewpoint, this situation is unremarkable. One part of the MSC iterates its behaviour some arbitrary amount, whilst the rest of the MSC continues without any concern for what is occurring in the loop. The method given here can cope with this situation, and any other where the MSC contains loops which contain some, but not all, of the instances.

This is in contrast to prior art methods such as those disclosed in the two Leue et al. references cited in the introduction above, which are unable to cope with this situation.

The above embodiments may be implemented by configuring or adapting any suitable apparatus, for example a computer or other processing apparatus. Alternatively, the processes described may be implemented by a processor implementing processor-implementable instructions and/or stored on a suitable storage medium, such as computer memory, hard disk, floppy disk, ROM, PROM etc. The processor may be a computer, a network of computers, or one or more dedicated processors.

The examples given above are merely exemplary, and it will be appreciated that the invention may be applied to other examples of system specifications. Also, variation in the algorithms or notation are possible, in particular by using conventional finite state automaton generating options.

Thus a method for describing a collection of test scripts in the form of a single finite state automaton has been provided, wherein the abovementioned disadvantages associated with prior art arrangements have been substantially alleviated.

Claims

Claims 1. A method of generating a test specification from a system specification (1), characterised by the step of: generating the test specification as a finite state automaton (20).
2. A method according to claim 1, the method further characterised by the step of: representing by the system specification (1) a causal relationship between events.
3. A method according to claim 2, wherein the system specification (1) is a message sequence chart (1).
4. A method according to claim 2 or 3, wherein the system specification (1) comprises concurrent iteration between processes.
5. A method according to claim 3, wherein the message sequence chart comprises a loop (18) extending over only part (4) of the message sequence chart (1).
6. A method according to any of claims 3 to 5, comprising the steps of: (i) initialising sets of states to be equal to the set of all events in the message sequence chart (1); (ii) initialising sets of transitions to be the empty set in the message sequence chart (1); and (iii) modifying repeatedly the sets of states and sets of transitions values until the finite state automaton (20) is completed.

<Desc/Clms Page number 18>
7. A method according to claim 6, further comprising using auxiliary variables to store intermediate values of the sets of states and the sets of transitions.
8. A method of testing a system specification (1), characterised by performing testing using a test specification generated from the system specification using the method according to any of claims 1 to 7.
9. A method of testing a system, characterised by performing testing using a test specification generated from a system specification (1) of the system using the method according to any of claims 1 to 7.
10. A method according to any preceding claim, wherein the system is a telecommunication system.
11. A storage medium storing processor-implementable instructions for controlling one or more processors to carry out the method of any of claims 1 to 10.
12. Apparatus adapted to implement the method of any of claims 1 to 10.
13. A method substantially as hereinbefore described with reference to the accompanying drawings.
14. Apparatus substantially as hereinbefore described with reference to the accompanying drawings.