WO2015084140A1

WO2015084140A1 - A system and method for emulating multiple independent wireless client devices in the cloud

Info

Publication number: WO2015084140A1
Application number: PCT/MY2014/000147
Authority: WO
Inventors: Fairul Rizal Bin FAHRURAZI; Fook Ann LOO
Original assignee: Mimos Berhad
Priority date: 2013-12-03
Filing date: 2014-06-03
Publication date: 2015-06-11
Also published as: MY167175A

Abstract

Wireless network performance test is initiated and test results are gathered from multiple independent wireless client devices which simplify and assist in the process of analysis and generation of test reports. The system for emulating multiple independent wireless client device in the cloud comprising a cloud platform (104) consisting a plurality of virtual machines (110a-110n); a plurality of USB wireless adapters (106); and a device agent (102). The method for emulating multiple independent wireless client devices in the cloud comprising steps of setting up test environment by user (202); reading files by Device Agent and providing list of USB devices in cloud platform which can be used (204); specifying number of wireless client to be created by user (206); launching different virtual machines in the cloud platform by the Device Agent by using virtual machine images provided by the user based on settings in the configuration files (208); mapping and attaching physical USB devices to virtual machines in the cloud platform to create multiple independent wireless client device by the Device Agent (210); injecting test automation scripts into all virtual machines by the Device Agent and initiating test execution in all virtual machine in the cloud platform based on settings in the configuration files (212); and gathering results from all virtual machines by the Device Agent upon completion of test execution based on the settings in the configuration files (214); and analyzing test results received from all virtual machines and generating test report (216).

Description

A SYSTEM AND METHOD FOR EMULATING MULTIPLE INDEPENDENT WIRELESS

CLIENT DEVICES IN THE CLOUD FIELD OF INVENTION

The present invention relates to a system and method for emulating multiple independent wireless client devices in the cloud. In particular, the invention initiates wireless network performance test and gathers test results from multiple independent wireless client devices which simplifies and assists in the process of analyzing and reporting test results received from multiple independent wireless client devices.

BACKGROUND ART Currently available wireless network performance testing is expensive and test environment used in the performance tests need to be as close as possible to the real environment in order to generate a meaningful result. Further, the testing environment requires multiple wireless client devices to be set up such as computers and personnel are required during the test to generate user load that closely resembles the real environment (e.g. if we need to generate a user load of 100 wireless client devices, it is a waste of resources to call up 100 computers and personnel). The drawback of the current available situation is resolved in the present invention which proposes a system and method for emulating multiple independent wireless client devices in the cloud that can be used for wireless network testing that are able to reduce time and cost.

One example which provides a method and system for emulating a user population for wireless local area network access point performance, optimization and configuration testing is proposed in United States Patent No. US 7,277,395 B2 entitled "Method and Apparatus for Wireless Network Load Emulation" (hereinafter referred to as the US 395 Patent). In the US 395 Patent, all emulated client devices is provided in a load emulator which shares the same wireless interface in the Input Output Controller to connect to the SUT. In contrast, in the present invention, each emulated wireless client device is attached to one physical USB wireless adapter. All emulated client devices have exclusive access to its own USB wireless adapter to connect to the SUT. Further, the invention as disclosed in the US 395 Patent does not emulate independent wireless client devices as compared to the present invention which provides emulation of independent wireless client devices in the cloud.. All emulated wireless client devices in the invention as disclosed in the US 395 Patent shares the same physical wireless interface and sends network traffic to and from the SUT in Round-Robin fashion as compared to the present invention in which all emulated wireless client devices are able to use its own USB adapter to send network traffic to and from the SUT in parallel (i.e. concurrent).

Another example that provides a system that introduces virtual Ethernet and virtual Wireless Ethernet interfaces for client using a single host device is disclosed in United States Patent Publication No. US 20030202486 A1 entitled "Method and System for Simulating Multiple Independent Client Devices in a Wired or Wireless Network" (hereinafter referred to as the US 486 Publication). In the invention as disclosed in US 486 Publication, all emulated client devices shares two physical wireless network cards in the host machine to connect to the SUT as compared to the present invention wherein each emulated wireless client device is attached to one physical USB wireless adapter. Further, in the US 486 Publication, only one host system is utilized as compared to m the present invention which utilizes cloud platform and USB adapter to emulate multiple independent wireless client devices. The invention as disclosed in the US 486 Publication is not scalable as it utilizes only one host system as compared to the present invention which is scalable (i.e. by adding more commodity hardware if required).

Another mechanism which provides emulation of wireless application through a scalable network architecture is disclosed in an IEEE paper entitled "Scalable Emulation of Dynamic Multi-Hop Topologies" by Anders Nickelsen, Morten N. Jensen, Erling V. Matthiesen, Hans-Peter Schwefel; IEEE, 2008. In the said paper, emulation testbed is provided for wireless application wherein wireless link is replaced with wired links and a topology emulator is provided in which all end-nodes connected to a central network switch by wired links is provided. In contrast, in the present invention, each emulated wireless client device is attached to one separate physical USB wireless adapter. Further, in the said paper, all emulated wireless client devices is not provided to use its own USB adapter to send network traffic to and from the SUT in parallel (i.e. concurrently) as proposed in the present invention. SUMMARY OF INVENTION

One aspect of the present invention provides a system (100) for emulating multiple independent wireless client devices in the cloud comprising. The system comprising a cloud platform (104) consisting a plurality of virtual machines (110a-110n); a plurality of USB wireless adapters (106); and a device agent (102). The device agent (102) having means for reading configuration files sent by users; scanning and discovering the plurality of USB wireless adapters attached to the cloud platform; launching virtual machines in the cloud platform based on settings in configuration files and number of available USB wireless adapters; mapping USB wireless adapters to virtual machines by attaching USB wireless adapters to virtual machines in the cloud platform to create multiple independent wireless client devices; injecting test automation scripts into all virtual machines; initiating test execution in all virtual machines; gathering test results from all virtual machines; and generating report of test results.

Another aspect of the invention provides that configuration files sent by user further comprising vendor ID and product ID of the USB wireless adapters to be used in the system; list of base virtual machine images selected to be used in the system; list of test automation scripts selected to be used in the system; mapping of USB wireless adapters to virtual machine images; and number of emulated devices for each combination of virtual machine images and test automation scripts and USB wireless adapters.

A further aspect of the invention provides a method (200) for emulating multiple independent wireless client devices in the cloud. The method comprising steps of setting up test environment by user (202); reading files by Device Agent and providing list of USB devices in cloud platform which can be used (204); specifying number of wireless client to be created by user (206); launching different virtual machines in the cloud platform by the Device Agent by using virtual machine images provided by the user based on settings in the configuration files (208); mapping and attaching physical USB devices to virtual machines in the cloud platform to create multiple independent wireless client device by the Device Agent (210); injecting test automation scripts into all virtual machines by the Device Agent and initiating test execution in all virtual machine in the cloud platform based on settings in the configuration files (212); and gathering results from all virtual machines by the Device Agent upon completion of test execution based on the settings in the configuration files (214); and analyzing test results received from all virtual machines and generating test report (216). The step for mapping and attaching physical USB devices to virtual machines in the cloud platform to create multiple independent wireless client device by the Device Agent (210); each wireless client device of the multiple independent wireless client device is attached to one separate physical USB wireless adapter (106).

In another aspect of the invention there is provided that the step for setting up test environment by user further comprises setting vendor ID and product ID in Device Agent's configuration files.

In yet another aspect of the invention is the step for specifying the number of wireless client to be created by user which further comprises the step for providing automated test scripts and base virtual machine image to be used by the user.

In a further aspect of the invention there is provided that the step for injecting test automation scripts into all virtual machines by the Device Agent and initiating test execution in all virtual machines in the cloud platform based on settings in the configuration files further comprises monitoring test execution status in all virtual machines in the cloud platform based on the settings in the configuration files.

In still another aspect of the invention there is provided that said automated test scripts will be tagged with one unique ID and generated test report includes total number of clients SUCCESSFULLY completed GUI automation script; total number of clients FAILED to run GUI automation script; response time on each client; and throughput on each client. The present invention consists of features and a combination of parts hereinafter fully described and illustrated in the accompanying drawings, it being understood that various changes in the details may be made without departing from the scope of the invention or sacrificing any of the advantages of the present invention.

BRIEF DESCRIPTION OF ACCOMPANYING DRAWINGS

To further clarify various aspects of some embodiments of the present invention, a more particular description of the invention will be rendered by references to specific embodiments thereof, which are illustrated in the appended drawings. It is appreciated that these drawings depict only typical embodiments of the invention and are therefore not to be considered limiting of its scope. The invention will be described and explained with additional specificity and detail through the accompanying drawings in which: FIG. 1 illustrates the general architecture of the present invention.

FIG. 2 is a flowchart illustrating the general methodology of an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, this specification will describe the present invention according to the preferred embodiments. It is to be understood that limiting the description to the preferred embodiments of the invention is merely to facilitate discussion of the present invention and it is envisioned without departing from the scope of the appended claims.

Referring to FIG. 1.0, the system (100) for emulating multiple independent wireless client devices in the cloud according to the present invention is illustrated. The system (100) comprising a cloud platform (104) consisting a plurality of virtual machines (110a-110n); a plurality of USB wireless adapters (106); and a device agent (102). Servers in a cloud platform which can host virtual machines are known as "node" or "node controller". Device agent (102) will read configuration files sent by users; scans and discover the plurality of USB wireless adapters attached to the cloud platform; launch virtual machines in the cloud platform based on settings in configuration files and number of available USB wireless adapters; map USB wireless adapters to virtual machines by attaching USB wireless adapters to virtual machines in the cloud platform to create multiple independent wireless client devices; injects test automation scripts into all virtual machines; initiates test execution in all virtual machines; gathers test results from all virtual machines; and generates report of test results. The virtual machines (110a-110n) have exclusive access to its own USB wireless adapter (106) and the configuration files sent by user further comprising vendor ID and product ID of the USB wireless adapters to be used in the system; list of base virtual machine images selected to be used in the system; list of test automation scripts selected to be used in the system; mapping of USB wireless adapters to virtual machine images; and number of emulated devices for each combination of virtual machine images and test automation scripts and USB wireless adapters. The general methodology (200) of an embodiment of the present invention is as illustrated in FIG. 2.0. As illustrated in FIG. 2.0, the test environment is first set up by the user (202). Device Agent will read files (204) wherein device agent will find all nodes in a cloud platform using the cloud platform's API. Device agent will tag all nodes with a unique node ID. Example of node ID ((Node ID="0" - nodel , IP=10.1.2.101 ) and (Node ID="1" - node2, IP=10.1.2.102)). An example of cloud platform API's output (taken from PQRE cloud) is as follows.

Device agent will login to all nodes in the cloud platform 1-by-1. Device agent will scan the USB devices attached to a node in the cloud platform and said step is repeated for all nodes in the cloud. Thereafter, device agent will provide a list of USB devices in the cloud platform which can be used (204). An example of USB devices in 1 node is provided as follows.

Based on the results obtained, Device agent will create an array (a list) to represent the USB in which the number of wireless client to be created by the user will be specified (206) by providing automated test scripts and base virtual machine image to be used by the user. Each element in the created array contains the following data: arrayElementfn] = {Node ID, USB Bus ID, USB Device ID, USB Vendor ID, USB Product ID}

ArrayElement[0] = {0, 001 , 001 , 1d6d, 5370}

ArrayElement[1] = {0, 002, 003, 0951 , 1607}

ArrayElement[2] = {1 , 002, 001, 1d6d, 5370}

ArrayElement[50] = {6, 002, 004, 1d6d, 5370} a. "USB Bus ID + USB Device ID" represents the "address" that uniquely

identify a USB device in a host system b. "USB Vendor ID + USB Product ID" uniquely identify 1 type of USB

device. c. "Node ID + USB Bus ID + USB Device ID" uniquely identify a USB device attached to a node in the cloud platform.

The device agent will read the configuration file and obtain the types of USB adapters selected by the user. Device agent will tag all USB adapter types with a unique ID. For example, if a user intends to use the following USB devices in the cloud, it will be tag as follows.

a. TP-LINK TL-WN727N, tag ID=ID0

b. TP-LINK TL-WN722N, tag ID=ID1

c. TP-LINK TL-WN822N, tag ID=ID2

Accordingly, the device agent will generate an array which represents the USB devices in the cloud platform which matches the Vendor ID and Product ID set in the configuration file. The length of this array will be less than or equals to the first set of array generated because the USB devices which do not match the Vendor ID and Product ID will be filtered out.

ArrayElement[0] = {0, 001 , 001, 1d6d, 5370}

ArrayElement[1] = {0, 002, 003, 0951 , 1607}

(2) ArrayElement[2] = {1 , 002, 001 , 1d6d, 5370}

ArrayElement[30] = {6, 002, 004, 1d6d, 5370}

Based on the the second set of array obtained above, device agent will calculate and generate an array which represents the number of "each type of USB device" attached to "each of the nodes" in the cloud. Each element in the array represents the data for 1 node. The length of the array is equals to the number of nodes in the cloud.

arrayElement[0] = {Node ID 0, number of USB adapter ID0 attached to Node 0, number of USB adapter ID1 attached to Node 0, ...}

ArrayElement[0] = {0, 6, 6, 0, ...}

ArrayElement[1] = {1 , 4, 10, 0, ...} (3)

ArrayElement[6] = {6, 0, 6, 2, ...}

Thereafter, device agent will locate the available resources in all nodes in the cloud using the cloud platform's API. Device agent will create an array to represent the available resources of all nodes in the cloud platform. Each element in the array represents the specification of 1 node in the cloud.

arrayElementfO] = {Node ID, number of CPU Core, amount of RAM in MB, size of harddisk storage in GB}

ArrayElementfO] = {0, 8, 16000, 500}

ArrayElement[1] = {1 , 8, 16000, 500}

ArrayElement[6] = {6, 8, 16000, 500} rv (4) Device agent will combine the array in (3) and (4).

arrayElement[n] = {Node ID, number of CPU Core, amount of RAM in MB, size of harddisk storage in GB, number of USB adapter IDO attached to Node n, number of USB adapter ID1 attached to Node n, ...}

ArrayElement[0] = {0, 8, 16000, 500, 6, 6, 0, ...}

ArrayElement[1] = {1, 8, 16000, 500, 4, 10, 0, ...}

ArrayElement[6] = {6, 8, 16000, 500, 0, 6, 2, ...}

Device agent will read the configuration file and obtain

a. all combinations of "base virtual machine image type + USB wireless adapter type" that the user wants to use, and

b. the number of virtual machine for each combination type.

For example:

a. Combination 1 :

i. Base virtual machine image = CentOS-6.4-x86_64_ID1 ii. USB wireless adapter = TP-LINK TL-WN727N

iii. Number of virtual machine = 20 b. Combination 2:

iv. Base virtual machine image = ubuntu-13.04-desktop-i386_ID2 v. USB wireless adapter = TP-LINK TL-WN822N

vi. Number of virtual machine = 10

For each type of virtual machine requested by the user, the device agent will

a. check the array in (5) and locate a node which has the required

resources (USB adapter + CPU + RAM + hard disk)

b. Launch the selected VM type in the selected node. The VM will be tagged with a unique VM ID.

c. Device agent will check the array in [2] and create an array with the following format. Each element in the array represents 1 VM launched by the device agent and the mapping to its designated USB adapter. arrayElement[n] = {Node ID, VM ID, USB Bus ID, USB Device ID}

ArrayElement[0] = = {0, 1 , 1d6d, 5370}

ArrayElement[1] = = {0, 2, 0951 , 1607}

(6)

ArrayElement[2] = = {1. 3, 1d6d, 5370}

ArrayElement[6] = = {6, 30, 1d6d, 5370} d. Update the array in [5]. Hardware resources (USB adapter + CPU + RAM + harddisk) which have been used in .b] will be removed from the array.

Thereafter, the device agent will launch different virtual machine images in the cloud platform by using virtual machine images provided by the user based on settings in the configuration files (208). The device agent will generate an error message if there are not enough resources in the cloud. The error message will be saved in a variable which will be shown to the user at the end of the system task. After launching all virtual machines, the device agent will perform "USB passthrough" to attach the USB adapters directly to the designated VM in the cloud. The method used to perform "USB passthrough" depends on the types of hypervisor used in the cloud. Different hypervisor will use different methods to perform USB passthrough.

The following is an example to perform USB passthrough in a node which utilizes KVM hypervisor and is managed by libvirt. Device agent will check the array in (6) to locate the USB Bus ID and USB Device ID of a USB device which needs to be attached to a virtual machine. Device agent will prepare a XML file with the following format:

(7) oddress bus="001" device="0027>

</source>

</hostdev> By referring to the array in (6), the device agent will perform "USB passthrough" to attach the USB adapter directly to the designated virtual machine. In this example, device agent will utilize the libvirt tool's API and the XML file in (7). Device agent will repeat the step to perform USB passthrough in a node for all virtual machines in array (6). The device agent will login to all virtual machines in array (6) and the device agent will verify that all virtual machines have been attached with one USB adapter. Thereafter, the device agent will read the configuration file to obtain the list of test automation script selected by the users. All test automation scripts will be tagged with one unique ID. For example: Script 1 , tag ID=0 and Script 2, tag ID=1.

The device agent will update the array in (6) with the mapping to the designated test script. Each element in the array represents one virtual machine, its designated USB adapter and its designated test automation script.

The format of the array is as below:

arrayElement[n] = {Node ID, VM ID, USB Bus ID, USB Device ID, Test Script Tag ID}

ArrayElement[0] = {0, 1 , 1d6d, 5370, 0}

ArrayElement[1] = {0, 2, 0951 , 1607, 0}

ArrayElement[2] = {1 , 3, 1d6d, 5370, 1} (8)

ArrayElement[6] = {6, 30, 1d6d, 5370, 0}

Physical USB devices are mapped and attached to virtual machines in the cloud platform to create multiple independent wireless client devices by the device agent (210). By referring to the array in (8), device agent will inject the selected graphical user interface (GUI) test automation script into all virtual machines in the cloud platform based on settings in the configuration files (212). The protocol that can be used includes SCP, FTP, TFTP, SFTP, etc. Device agent will repeat said step of injecting test automation scripts for all virtual machines in array (8). Subsequently, device agent will initiate test execution in all virtual machines in array (8) (212). Device agent will check and monitor the test execution status in all virtual machines and upon completion or occurrence of timeout error in all virtual machines, the device agent will gather the test results from all virtual machines in array (8) based on the settings in the configuration files (214). Subsequently, the device agent will analyze the test results received from all virtual machines and generate a test report (216). The generated test report includes total number of clients SUCCESSFULLY completed GUI automation script; total number of clients FAILED to run GUI automation script; response time on each client; and throughput on each client.

The present invention simplifies and assists the process of running / initiating wireless network performance test and gathers the test results from multiple independent wireless client devices which further simplifies and assists in the process of analyzing and reporting test results received from multiple independent wireless client devices. Each wireless client device is attached to one separate physical USB wireless adapter (106) and the said wireless client devices is provided with exclusive access to its own USB wireless adapter to connect to the System Under Test (SUT) directly and is able to run its own operating system and applications in parallel (i.e. concurrently).

The present invention may be embodied in other specific forms without departing from its essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore indicated by the appended claims rather than by the foregoing description. All changes, which come within the meaning and range of equivalency of the claims, are to be embraced within their scope.

Claims

1. A system (100) for emulating multiple independent wireless client devices in the cloud comprising:

a cloud platform (104) consisting a plurality of virtual machines (1 10a- 11 On);

a plurality of USB wireless adapters (106); and

a device agent (102)

characterized in that

the device agent (102) having means for:

reading configuration files sent by users;

scanning and discovering the plurality of USB wireless adapters attached to the cloud platform;

launching virtual machines in the cloud platform based on settings in configuration files and number of available USB wireless adapters;

mapping USB wireless adapters to virtual machines by attaching USB wireless adapters to virtual machines in the cloud platform to create multiple independent wireless client devices;

injecting test automation scripts into all virtual machines;

initiating test execution in all virtual machines;

gathering test results from all virtual machines; and

generating report of test results.

2. A system (100) according to Claim 1 , wherein the plurality of virtual machines (1 10a-1 10n) have exclusive access to its own USB wireless adapter (106).

3. A system (100) according to Claim 1 , wherein configuration files sent by user further comprising:

vendor ID and product ID of the USB wireless adapters to be used in the system;

list of base virtual machine images selected to be used in the system; list of test automation scripts selected to be used in the system; mapping of USB wireless adapters to virtual machine images; and number of emulated devices for each combination of virtual machine images and test automation scripts and USB wireless adapters.

A method (200) for emulating multiple independent wireless client devices in the cloud comprising steps of:

setting up test environment by user (202);

reading files by Device Agent and providing list of USB devices in cloud platform which can be used (204);

specifying number of wireless client to be created by user (206);

launching different virtual machines in the cloud platform by the Device Agent by using virtual machine images provided by the user based on settings in the configuration files (208);

mapping and attaching physical USB devices to virtual machines in the cloud platform to create multiple independent wireless client device by the Device Agent (210);

injecting test automation scripts into all virtual machines by the Device Agent and initiating test execution in all virtual machine in the cloud platform based on settings in the configuration files (212); and

gathering results from all virtual machines by the Device Agent upon completion of test execution based on the settings in the configuration files (214); and

analyzing test results received from all virtual machines and generating test report (216)

characterized in that

mapping and attaching physical USB devices to virtual machines in the cloud platform to create multiple independent wireless client device by the Device Agent (210); each wireless client device of the multiple independent wireless client device is attached to one separate physical USB wireless adapter.

A method (200) according to Claim 4, wherein setting up test environment by user further comprises setting vendor ID and product ID in Device Agent's configuration files.

6. A method (200) according to Claim 4, wherein specifying number of wireless client to be created by user further comprises providing automated test scripts and base virtual machine image to be used by the user.

7. A method (200) according to Claim 4, wherein injecting test automation scripts into all virtual machines by the Device Agent and initiating test execution in all virtual machines in the cloud platform based on settings in the configuration files further comprises monitoring test execution status in all virtual machines in the cloud platform based on the settings in the configuration files.

8. A method (200) according to Claim 6, wherein said automated test scripts will be tagged with one unique ID.

9. A method (200) according to Claim 4, wherein generated test report includes:

total number of clients SUCCESSFULLY completed GUI automation script;

total number of clients FAILED to run GUI automation script;

response time on each client; and

throughput on each client.