IE85627B1 - Testing of communication networks - Google Patents

Abstract

ABSTRACT A test system (1 ) is for testing a communication network. It comprises remote test devices (4) to transmit and receive communication signals via the network (5) under test. a central client system (10) receives and processes user test instructions and routes corresponding test commands to the remote test devices (94). The test devices (4) communicate with the central client system (10) including routing network under test responses to the central client system. The test device (4) comprises a speaker coupled with a microphone, and a modem (4) coupled with the microphone and arranged for communication via the network under test. The client system Comprises an application server (10) adapted to generate emulation screens and download them to a client terminal (1 1), to receive inputs made on the screens, and to interpret them as test commands for a test session. The screens include display controls emulating mobile or fixed telephone device user interfaces, including ntmibers to be dialled, and messaging inputs, and also displays of responses from the network under test. a device controller (35) manages a plurality ofthreads to a plurality oftest devices.

Description

Field of the Invention The invention relates to testing of telecommunication. networks.

Prior Art Discussion In the testing of telecommunication networks, test devices such as test phones and other telecommunications test apparatus are used to make and receive test calls and use other telecommunication services such as Short Message Service and Mobile Messaging Service. A test system is described in US6480576 (AT&T), which performs connection testing and there is a monitoring system.

The invention is directed towards improving efficiency in carrying out testing of networks in different locations.

SUMMARY OF THE INVENTION According to the invention, there is provided a test system for testing a communication network, the test system comprising: remote test devices adapted to transmit and receive communication signals via the network under test; a central client system adapted to receive and process user test instructions and toproute corresponding test commands to the remote test devices; and wherein the test devices are adapted to communicate with the central client system including routing network under test responses to the central client system; wherein each test device comprises a speaker coupled with a microphone, and a modem coupled with the microphone and arranged for communication via the network under test; modem arranged for communication via the network under test; wherein the central client system is adapted to transmit test commands over the Internet; wherein each test device is addressable; and wherein the system comprises a plurality of test devices and a representation function to logically represent at least some of the test devices as virtual devices and their status and generate user outputs representing the devices and their status.

In one embodiment, the test device automatically interfaces for audio communication without human intervention.

In another embodiment, the test device is adapted to capture a visual output from the network and relay it to the client system.

In one embodiment, the test device comprises a camera for collecting visual output.

In one embodiment, the test device is adapted to capture a digital stream representing visual output.

In one embodiment, the client system comprises an application server adapted to generate emulation screens and download them to a client terminal, to receive inputs made on the screens, and to interpret them as test commands for a test session.

In "a further embodiment, said screens include display controls emulating mobile or fixed telephone device user interfaces, including numbers to be dialled, and messaging inputs, and also displays of responses from the network under test.

In one embodiment, the application server comprises a queuing function adapted to buffer and queue test commands and to route them to a test device. - 3 _ In one embodiment, the system comprises a device controller adapted to manage a plurality of links to a plurality of test devices.

In one embodiment, the test devices comprises a device server comprising test control function and modems said test control functions interfacing with the application server device controller.

In one embodiment, the application server comprises a SIM manager for registering SIM details for use by the device controller.

In one embodiment, the representation fl.1I1CtlOI1 does not include information about location of the represented test device.

In one embodiment, the representation fimction automatically generates user outputs indicating the range of remaining operations for use of the represented test device in order to guide the user according to allowed test sessions.

In one embodiment, the representation fimction is adapted to assign represented devices to a user and to only permit test instructions for those devices.

In another embodiment, the representation fL11‘1Ctl0I‘l indicates as active a test device which retains two connections to a device controller, one to forward user test instructions to the device controller and another to receive asynchronous events from the device controller.

In one embodiment, the representation function maintains representation of the test devices’ states and correlates incoming events including user requests and device status changes to update the device representation.

In one embodiment, the representation function is adapted to maintain a thread of execution to manage each individual remote test device, and to take an overview of the collective state of all devices (4) within a session.

In one embodiment, the representation function is adapted to combine the states of a plurality of devices to generate information concerning the overall network state and provide a more . complete picture of network performance than possible_fi'om any individual device state.

According to another aspect, the invention provides a computer readable medium comprising software code adapted to perform operations of a test system as defined above in any embodiment when executing on a digital processor or processors.

DETAILED DESCRIPTION OF THE INVENTION Brief Description of the Drawings The invention will be more clearly understood from the following description of some embodiments thereof, given by way of example only with reference to the accompanying drawings in which:- Fig. 1 is a diagram illustrating a testing system of the invention; Fig. 2 is a block diagram illustrating the architecture of an application server and a device server of the system; Figs 3 and 4 are message sequence diagrams illustrating operation of the system; and V Fig. 5 is a diagram illustrating logical relationships between test devices and virtual device states.

Description of the Embodiments Referring to Fig. 1 a test system 1 comprises a device server 2 having a processor 3 and modems 4 for communication in a PLMN network 5 under test. The system also comprises an application server 10 for communication with client terminals 11.

The client terminals 11 reside at a location distant from the radio coverage provided by the network 5 under test, for example they might reside in different countries. The modems reside . within the radio coverage provided by the network 5 under test, and in this embodiment are GSM modems. The locations of the application server 10 and the device server 2 are less significant, however it would be common but not essential for the application server 10 to reside close to_ the client terminals and for the device server processor 3 to be co-located with the modems 4. One application server 10 can connect with many device servers 2 and manage many parallel sessions between client terminals 11 and device servers 2. A device server ,2 may be allocated at any time to one client terminal session. An advantageous feature of this architecture is that the application server 10 can also host other types of applications. The web server application can be replaced with an automation server which runs test actions based on an input script. In general, the application server serves to abstract test scenario drivers from the network test equipment.

In the example of a test environment based on an interactive Web interface, human testers use Web browsers running on the client terminals 11 to connect to a Web server running on the application server 10, by means of which they invoke remote testing actions, which are converted by the application server 10 into commands. These commands are transmitted to the device server 2, the processor 3 of which receives and processes them. The device server processor 3 will then invoke the requested actions on one or both of the attached modems 4— to cause them to carry out typical network transactions such as making calls, sending SMS messages, and transmitting data across the network. Two modems 4 are provided so that it is possible for one to act as the calling party and one to act as the called party for end-to-end testing. There may of course be a bank of more than two modems, for parallel test sessions and redundancy.

Referring to Fig. 2 the application server 10 comprises server components 20 including a Push server, a Web server, and proxy, load balancer, and static media server. The components 20 also include a file store. A message queue component 21 manages interaction with a device controller , which manages threads 26 linked with device servers 2. The device controller 25 has access to a’ SIM store and manager 27.

The servers 20 communicate with the ‘client terminals 11, allowing interactive, testing to be instructed by the user. Advantageously, they generate screens for emulation of telephone or any other GSM device user interfaces. For example, referring to Figs. 3 and 4 some of the inputs by the user (who is remote from the device server 2) are “pressing” call, number, and hang-up display controls. Hence, the servers 20 allow the user to log in to the system and also to then make inputs on the terminal in a marmer which emulates the inputs made on an actual GSM device. _ 6 _ The controller client program 21 acts as a buffer between the user test commands and the device controller 25 of the application server 10. It queues test commands for the device servers 2.

The server 10 also comprises a script-driven automation server 22 which provides scripts for use in tests.

As shown in Fig. 2, the device server 2 comprises device test control programs 3 which interface with modems 4. As this diagram shows, there may be any number of device servers 2 controlled by a single application server 10.

In the overall chain from user terminals 11 to the modems 4, the interaction between the terminal 11 browsers and the queuing function 21 is near real time, and from the queuing function 21 to the modems 4 it is fully real time. Management of the real time testing operations is driven by the queuing functions 21, the device controller functions 25, and the device server processing fimctions 3.

Operation of the device server 2 and the device controller 25 includes: i) Connecting a test microphone connection of one modem 4 to a speaker connection of the other modem 4. ii) Connecting a speaker connection to a microphone connection. iii) Connecting a client tenninal 11 to the PSTN 5 whereby the client 11 is given a dialable number or other addressable number. iv) Configuring a modem 4 to automatically answer all incoming calls received at an interface point. v) On answering of an incoming_cal1, connecting the audio paths from the incoming side of the call (PSTN or VoIP) to the modem microphone and receiver.

Test personnel can dial the number of the modem 4 and take full control of the audio paths of the test system 1.

The device server 2 may in some embodiments be connected to the visual output transmitted through the network under test, so that the results of test actions on the visual output could also be measured. The visual output can be captured for example by a Web camera or “screen- scraping” software running on the test apparatus. The device server 2 may. additionally log the - 7 _ test data it receives to, for example, store the incoming and outgoing audio information digitally in files. The device server 2 could include a facility to play out actions driven by a command script, which includes recorded audio for use in the test. This would be meaningfully combined with the previous point for automated testing.

Referring to Figs. 3 and 4 a device controller 25 runs on the application server 10 and this diagram shows how a series of commands are transmitted by a client 11, resulting in interaction between the modems 4. As shown in Fig. 3 the test user inputs are “pressing” of numbers and call icons to emulate GSM device user inputs. The servers 20 and queuing functions 21 create a device session and invoke a test action. The device controller 25 transmits commandsl-N to the device server 2, which interprets the commands and implements them using the modems 4. One modem 4 acts as an originating GSM device, and the other as the terminating device. For calls, the device server 2 generates audio for either real time voice spoken by the user at the terminal 11 or "from a text-to-voice function within the device server 2. The incoming voice in the terminating modem 4 is picked up by a microphone in the device server 2. The actual voice is transmitted back to the terminal 11, or the voice is analyzed locally by the device server 2 and a test result is routed back to the terminal.

Referring to Fig. 4 a sequence for a hang-up which is simulated at the user terminal is shown.

This demonstrates that the system 1 is capable of testing a wide range of GSM operations initiated by a user.

The test sessions driven by the user terminals 11 are particularly suited to testing of voice services such as Interactive Voice Response (IVR) and verifying voice paths.

Test scenarios The system 1 conducts rich end-to-end tests of network features under manual guidance, rather than simple low-level automatic testing or continuous monitoring.

Referring again to Fig. 2 to take an example: a tester first logs into a terminal 11 and connects via its Web browser to the Web server 20 within the application server 10 to initiate a test session via the device controller 25. Within a test session, a number of remote devices are “seized” and assigned to the exclusive use of that tester. The device controller 25 also has a _ 3 _ security function which restricts the use of dialled numbers to a range reserved for test purposes only, and protects against abuse of the system 1. Also, the device controller 25 is programmed to supervise the work and performance of the testers by logging the activities of the test session.

The tester than chooses a variety of user-level actions to access network services, for example: — dial a number to make a call between two remotely-controlled devices — answer a call on the second device — talk into one device and listen on the other to determine not only basic connectivity but also qualitative infonnation such as voice quality — Initiate a teleconference call using four devices to test quality of conferenced voice in the network — Send a text message — Activate a USSD session (GSM) — Activate a data session and access data services supported by the network.

In general, the system 1 provides a means to conduct complex test of network services, by orchestrating test steps across a variety of test devices, mediated by the device controller 25.

A test scenario can involve a large number of steps and does not have to follow a fixed path. The environment is fiilly interactive, allowing the tester freedom to choose which next step to make at each stage.

When the test session is concluded, the tester disconnects; the device controller 25 returns the remote devices 4 to a flee pool, where it is available to be assigned to another tester in a new session.

It is also envisaged that the device controller 25 will manage priority of access to devices 2. This means in certain cases, a tester with a higher priority will be able to “take” remote devices even" if they are assigned to another tester (at lower priority). In this case the device controller 25 would remove the remote devices 2 fiom the lower priority session and re-assign them to the higher priority session. This could be used when a serious network problem arose and it became desirable to conduct a set of tests immediately (perhaps automatically) without incurring the delay of contacting other testers to request that they voluntarily release their devices.

Complex Device State Referring to Fig. 5 to remotely manage a device 4 state, a complex virtual representation of the device 4 is maintained across the system 1. Several system components are co-ordinated to maintain a consistent state.

To take an example, within a test session, a number of remote devices 4 may be employed by the tester during the session. These are presented as independent devices, regardless of the physical location or arrangement of the devices within the network, and across device servers 2.

Within the Web browser environment, the tester may have a number of (virtually represented) devices active, and a number of other devices inactive on a “carouse ”. The tester may move any device to or from the carousel thereby deactivating or activating it. By “inactive” is meant only that the tester cannot interact with the user interface of the device; otherwise even devices in the carousel remain fully capable of reacting to network events, e.g. receiving calls or text messages.

To this end, active devices retain two connections to the device controller 25, one to forward user interactions to the device controller 25 (via the Web server 20) and another to receive asynchronous events from the device controller 25 (via the Web Push Server). Devices on the carousel only receive asynchronous events from the device controller 25. In both cases, some of the device state is captured in the user interface; various “keys” (icons) are made active or inactive according to the device state (e. g. the “Call” button may not be used while a call is in progress).

The device controller 25 fi1rther maintains a more complete representation of the remote devices’ states and correlates incoming events (user requests, device status changes) to update the virtual representation of the device state. It maintains a thread of execution to manage each individual remote device, but also takes an overview of the collective state of all devices within a session.

In particular it can use its combined knowledge of the states of multiple devices to draw conclusions about the overall network state and provide a more complete picture of network performance than possible fi-om any individual device state. Finally at the top level, the device controller manages N test sessions across a pool of M test devices, where the relationship between test sessions and remote test devices may continually change as sessions are started and ended.

A screenshot of an example test file used for an automated test scenario appears below.

*; If Ew.-:t:T 5 Q 5; {seat-nevus:-2 immense i 26$ 3 ¥:£AL~DE¥iCE-I Ecaawoos ,- Emir rr 2»; fig; REAL-UEWCE-2 fiaxswas .3139 353.3321 5 Table 1 Each line of the file corresponds to an action that the device server 2 will undertake. ‘ — The first column describes the required device (modem 4) name.

— The second column describes the action the device should undertake. Options include: o MTMODE: Tum on MT mode for answering a call 0 CALLMODE: Turn off MT mode to make a call o WAIT: The system waits for the number of seconds described in column 3 o ANSWER: The device will answer an incoming call T 0 HANGUP: The device will hang up the current call 0 DIAL: The device will dial the number in column 3 PRESS: The device will press the key described in column 3, for use in Voicemail or IVR testing o TEXT: The device will send a textmessage to the number in column 3, including the text in column 4 The third and fourth columns include information required by the action in the second column, such as phone numbers, delay times and text.

On completion of the test session, an email will be sent to the specified address. The email will include the overall status of the test session, as well as individual test results. An example email is below.

SW: Results of automated test script $&fit: Thu 13.'UEa‘.!2{H19 14:18 Error running job, error an line 31 : ERROR : TIMEOUT REAL-DEVICE-1 MIMBDE> OK HA1? 28 SK NkIT 5 CK R531-DEVICE-1 CfiLLMQDE OK Rifii-D£VICE-3 Difii BS?62fi""‘ 0K Efiif 28 0E REAL-DEVICE-1 HANGHP CIK REfli-DEVICE-1 flTflODE GK NfiIT 2% 0K REAL-fiEVICE-1 fifiSfiER ERRQR : TIfiEUU? Table 2 To create a new test a user clicks on the "Create new Tester“ link, completes all fields, and clicks "Create". The new tester will be available to select on the testers page.

Create new tester use:-name: Last name: E-mad address: Password: Pawn:-‘d ecmfii-masticm: Table 3 To edit test details the user clicks on a tester Username or email. A supervisor can assign a User A user can view the Call Detail Records (CDRS, see table below) of a testing team by clicking on a CDRs link in the menu, and by default all team records are shown. To filter by user, one selects the desired user from the drop down list of users, Userfalter: maI1:uaItester aug 2%? 13:18:15 Aug 21169 13:18:}. - fi.ug»20~Er9 13:13:35 - 1 ?1.ugv2009 13:13:35 Arsg- 20-{I9 13:12:53 DC 9855539 Devin:eF 68661589 SUCCESS SUCCESS l38?33 75 Si) E5555 e~.viceF immerse 05 0856589 087.23?S B87233 SUCCESS SLECESS , 21 Aug: 2009 2C 13212335 or: 8866589 in Table 5 To create a SIM in on the system 1, a user clicks "Create SIM" and enters all details in the required fields in an interface generated by the Web application component 20. It is not required to allocate a user and device at this time. Before a device can be allocated to a user for testing, it needs to be provisioned with a SIM card, and the details are stored in the SIM manager 10(c).

To create a device, a user click on a Create Device link, completes the fields and clicks "Create" to create the new device. Devices should only be created by advanced users, and the following details are entered: I Device Name: A name for the device. This field should not be changed.

Connection String: This field should not be changed.

Audio URL: This field should not be changed.

User: Select from a list of created users to allocate the new device to a tester.

Type: Select the device hardware type from a list. This field should not be changed.

To change the SIM in a device, complete the following steps.

Enter the new SIM details into the RTS Application server.

Unassign the device from its user and unassign any SIM's assigned to this device.

Power OFF the Mobile Device for which the SIM change is to be made.

Open the SIM drawer by pressing the yellow button on the SIM holder to remove the SIM tray Replace the SIM.

Close the SIM drawer by re-inserting the SIM tray.

Power ON the Mobile Device using the appropriate front panel switch.

Assign the new SIM to the device.

The invention is not limited to the embodiments described but may be varied in construction and detail.

Claims (4)

Claims

1. A test system for testing a communication network, the test system comprising: remote test devices (4) adapted to transmit and receive communication signals via the 5 network (5) under test; a central client system (10) adapted to receive and process user test instructions and to route corresponding test commands to the remote test devices; and wherein the test devices (4) are adapted to communicate with the central client system (10) including routing network under test responses to the central client system; 10 wherein each test device (4) comprises. a speaker coupled with a microphone, and a modem (4) coupled with the microphone and arranged for communication via the network under test; 15 wherein each test device comprises a microphone for coupling with a speaker of a modem (4) arranged for communication via the network (5) under test; wherein the central client system (10) is adapted to transmit test commands over the Internet; 20 wherein each test device (2) is addressable; and wherein the system comprises a plurality of test devices and a representation fiinction to logically represent at least some of the test devices as virtual devices and their status and 25 generate user outputs representing the devices and their status.

2. A test system as claimed in claim 1, wherein each test device (2) automatically interfaces for audio communication without human intervention. 30

3. A test system as claimed in claims 1 or 2, wherein each test device (2) is adapted to capture a visual output from the network and relay it to the client system.

4. A test system as claimed in claim 3, where each test device (2) comprises a camera for collecting visual output. A test system as claimed in claim 3, wherein each test device (2) is adapted to capture a digital stream representing visual output. A test system as claimed in any preceding claim, wherein the client system comprises an application server (10) adapted to generate emulation screens and download them to a client terminal (11), to receive inputs made on the screens, and to interpretthem as test commands for a test session. A test system as claimed in claim 6, wherein said screens include display controls emulating mobile or fixed telephone device user interfaces, including numbers to be dialled, and messaging inputs, and also displays of responses from the network under test. A test system as claimed in claims 6 or 7, wherein the application server (10) comprises a queuing function (21) adapted to buffer and queue test commands and to route them to a test device (4). A test system as claimed in any of claims 6 to 8, wherein. the system comprises a device controller-(25) adapted to manage a plurality of links to a plurality of test devices. A test system as claimed in claim 9, wherein each test device comprises a device server (2) comprising test control functions (3) and modems (4), said test control functions (3) interfacing with the application server (10) device controller (25). A test system as claimed in any of claims 6 to 10, wherein the application server comprises a SIM manager (26) for registering SIM details for use by the device controller (25). A test system as claimed in any preceding claim, wherein the representation function does not include information about location of the represented test device. A test system as claimed in claim 12, wherein the representation function automatically generates user outputs indicating the range of remaining operations for use of the represented test device in order to guide the user according to allowed test sessions. A test system as claimed in any preceding claim, wherein the representation fimction is adapted to assign represented devices to a user and to only permit test instructions for those devices. A test system as claimed in any preceding claim, wherein the representation function indicates as active a test device which retains two connections to a device controller (25), b one to forward user test instructions to the device controller (25) and another to receive asynchronous events from the device controller. A test system as claimed in any preceding claim, wherein the representation function maintains representation of the test devices’ states and correlates incoming events including user requests and device status changes to update the device representation. A test system as claimed in claim 16, wherein the representation function is adapted to maintain a thread of execution to manage each individual remote test device, and to take an overview of the collective state of all devices (4) within a session. A test system as claimed in claim 17, wherein the representation function is adapted to combine the states of a plurality of devices to generate information concerning the overall network state and provide a more complete picture of network performance than possible from any individual device state. A computer readable medium comprising software code adapted to perform operations of a test system of any preceding claim when executing on a digital processor or processors.