WO2017020251A1

WO2017020251A1 - Load testing based on network virtualization using a container

Info

Publication number: WO2017020251A1
Application number: PCT/CN2015/086047
Authority: WO
Inventors: Yang Luo; Gal Shadeck; Lin Chen; Gencheng SHEN
Original assignee: Hewlett Packard Enterprise Development Lp
Priority date: 2015-08-04
Filing date: 2015-08-04
Publication date: 2017-02-09

Abstract

The present disclosure provides a method, apparatus and systems for load testing based on network virtualization. The method comprises receiving, by a manager, a load generator deployment request, wherein the request comprises a list of test groups inside a test scenario; creating, by the manager, containers for each test group through requests to a container engine; querying, by the manager, information about the created containers from the container engine; applying, by the manager, a rule to each container using a command; and generating and sending, by the manager, a load testing scenario file to a load testing tool for load testing, wherein the load testing scenario file contains group information and allocated container information.

Description

LOAD TESTING BASED ON NETWORK VIRTUALIZATION USING A CONTAINER

BACKGROUND

The fast adoption of mobile devices has had a significant impact on network applications. Applications connecting to networks via mobile connections may mean lower speed and bandwidth than an intranet or a broadband network. An application may spend increased time on processing a request in low bandwidth conditions meaning that all the resources， like network sockets， data connections to a database， and/or entries in cache， will all have to be kept alive for a longer time. Accordingly， the capability of an application will be quite different in mobile and desktop scenarios. Considering that load testing on real mobile devices may not scale， this means that Network Virtualization (NV) may be desirable in load testing to simulate real life cases more accurately.

At the same time， the emergence of cloud computing is also changing the software industry profoundly. Using a cloud resource as a service can simplify the process of software release and deployment， and reduce a customer′s cost. Nowadays， more and more software vendors may decide to utilize the benefit of a cloud platform， and cloud solution providers are also investing in cloud based load testing solutions to fit customers′ needs.

BRIEF DESCRIPTION OF THE DRAWINGS

The examples of the present application may be more fully appreciated in connection with the following detailed description taken in conjunction with the accompanying drawings， in which like reference characters refer to like parts throughout and in which：

FIG. 1 illustrates a diagram of a schematic structure of a running load test at server side in accordance with various examples of the present disclosure.

FIG. 2 illustrates a schematic diagram of an information flow for load testing based on network virtualization in accordance with various examples of the present disclosure.

FIG. 3 illustrates a flowchart of a method for load testing based on network virtualization in accordance with various examples of the present disclosure.

FIG. 4 illustrates a block diagram of a system for load testing based on network virtualization in accordance with various examples of the present disclosure.

FIG. 5 illustrates a diagram of a computer system for implementing various examples of the present disclosure.

FIG. 6 illustrates a diagram of results achieved by the solutions of the present disclosure， according to an example

DETAILED DESCRIPTION

In the following detailed description of examples of the disclosure， numerous specific details are set forth in order to provide a more thorough understanding of the disclosure. However， the disclosure may be practiced without these specific details. In other instances， well-known features have not been described in detail to avoid unnecessarily complicating the description.

Existing network virtualization solutions can be integrated into a load testing tool. The integrated load testing solution can also be delivered as a service. However， the network virtualization solution may work on a platform such as Windows. Compared to such a platform， a platform such as Linux is cheaper and more lightweight. For example， the price of a dual-core Windows image is about 1.9 times that of a Linux image for the same hardware. The size of a Windows 2008 server image is about 30 GB， while a Linux image is only 8 GB. The difference in image size leads to a different deployment time and a different user experience.

The present disclosure provides load testing based on network virtualization. A user configures the expected test group， i.e.， selects a testing script and sets the bandwidth， latency and packet loss for each group. Load generators are deployed and configured， and a scenario file is generated to run the test in the load testing tool. A process of the load testing may involve in a manager， a controller and a container engine which will be described below in detail.

Referring now to FIG. 1， a schematic structure 100 of a running load test at server side in accordance with various examples of the present disclosure is depicted.

Upon a container engine initiating， a virtual bridge， i.e.， a virtual Ethernet 101 as shown in FIG. 1， will be created automatically on a host operating system (OS) . The virtual Ethernet 101 may be created to forward traffic between network interfaces.

When a container is created， an interface for the container is created. Specifically， as shown in FIG. 1， three containers， i.e.， container 1， container 2 and container 3， are created while the respective bound interfaces for the containers， i.e.， virtual Ethernet interface 1， virtual Ethernet interface 2 and virtual Ethernet interface 3， are created. Further， as shown in FIG. 1， container 1 is associated with VUser 1 and VUser 2 on which the load testing is performed. Similarly， container 2 is associated with VUser 3 and VUser 4 on which the load testing is performed， and container 3 is associated with VUser 5 and VUser 6 on which the load testing is performed.

In this way， the host OS can communicate with the containers through the respective bound virtual Ethernet interface such that the network communication of each of container 1， container 2 and container 3 is isolated from each other.

Although three created containers are illustrated in FIG. 1， it should be understood that the present disclosure is not limited in this regard. Any number of containers and the respective virtual Ethernet interface thereof can be created dependent on the specific application.

Referring now to FIG. 2， a schematic diagram 200 of an information flow for load testing based on network virtualization in accordance with various examples of the present disclosure is depicted.

As shown in FIG. 2， a manager 201 may integrate various components for the load testing and manage the information flow to simplify multiple network emulation. The manager 201 may be implemented as hardware， computer software， circuit or the like or combinations of thereof. Whether the manager 201 is implemented as hardware， software or circuit depends upon the particular application.

At stage 1， the manager 201 receives a load generator deployment request， wherein the request may consist of a list of test groups inside a test scenario. As an example， each group may contain a test script and network virtualization requirements.

At stage 2， the manager 201 creates containers 203 for each test group through requests to the container engine 202. In an example， two parameters can be used to create the containers. The first parameter is a pre-made load generating image to be used to initialize the containers as shown at stage 2.a. The second parameter is a pair of automatically mapped ports from a host environment to the containers. This makes the containers accessible from a load testing tool 204. The container engine 202 returns the unique ID of the created containers in a response.

At stage 3， the manager 201 queries information， about the created containers 203 from the container engine 202. As an example， the manager 201 queries the information about the created containers from the container engine 202 through a RESTful application programming interface (API) using the returned container IDs at stage 2. In the example， a name of a virtual Ethernet interface device bound to the created container may be contained in the RESTful response.

As used in the present disclosure， Representational State Transfer (REST) is a software architecture style consisting of guidelines and best practices for creating scalable web services. REST is a coordinated set of constraints applied to the design of components in a distributed hypermedia system that can lead to high performance and a more maintainable architecture. Web service APIs that adhere to the REST architectural constraints are called RESTful APIs.

It should be noted that a container engine， in some examples， maynot return the virtual Ethernet interface device name. In this regard， the container engine in the present disclosure is compiled to add this support.

At stage 4， the manager 201 applies a rule to each of containers 203 using a command. As an example， a Traffic Control command ″tc qdisc add dev veth80cc root tbf rate 200kbit latency 50ms″ may be used to apply 200kbit bandwidth and 50ms latency to a virtual Ethernet interface whose name is specified as veth80cc， wherein the virtual Ethernet interface is bound to a container. As an example， the command can be sent from a controller. In this way， the traffic from the related containers will be filtered by the controller.

At stage 5， the manager 201 generates and returns a load testing scenario file which contains group information and allocated Container information. The allocated Container information comprises ″host： mapped_port″ information wherein ″host″ indicates a host operating system on which a virtual Ethernet is created upon the container engine initiating and ″mapped_port″ indicates a port mapped from the host operating system to the created containers 203 for the load testing tool 204 accessing the containers.

At stage 6， the load testing tool 204 starts a load testing for a respective container 203. The load testing tool 204 communicates to the container 203 using the ″host： mapped_port″ information returned from the manager 201. A command from the load testing tool 204 will be redirected to the container 203. The available bandwidth and latency of the load generator inside the container 203 will be set according to the rule specified at stage 4.

The above embodiment enables a load testing to simulate different network bandwidth status， so that the test can better measure the performance of and resource usage at server side. It does not require altering the existing load testing system， which makes it easier to be adopted.

Referring now to FIG. 3， a flowchart of a method 300 for load testing based on network virtualization in accordance with various examples of the present disclosure is depicted.

The method begins with block 301 which includes receiving， by a manager， a load generator deployment request from a user， wherein the request may consist of a list of test groups inside a test scenario. As an example， each group contains a test script and network virtualization requirements.

The method 300 further comprises creating， by the manager， containers for each test group through requests to a container engine at block 302. In an example， the containers can be created by a pre-made load generating image and a pair of automatically mapped ports from a host environment to the containers. The pre-made load generating image can be used to initialize the containers. The pair of automatically mapped ports makes the containers accessible from a load testing tool. The container engine can return information such as the unique ID of the created container through the ports. As an example， the information can be carried in a RESTful response.

The method 300 further comprises querying， by the manager， information about the created containers from the container engine at block 303. As an example， the manager can query the information through a RESTful API using the returned container IDs. In an example， the container engine used in the present disclosure is compiled such that a name of a virtual Ethernet device bound to the created container can be contained in the RESTful response.

The method 300 further comprises applying， by the manager， a rule to each container using a command at block 304. A Traffic Control command， for example， has a format such as ″tc qdisc add dev veth80cc root tbf rate 200kbit latency 50ms″ . By the example command， 200kbit bandwidth and 50ms latency are applied to the virtual Ethernet interface whose name is specified as veth80cc， wherein the virtual Ethernet interface is bound to a container. Therefore， the traffic from the related containers can be filtered.

The method 300 further comprises generating and returning， by the manager， a load testing scenario file which contains group information and allocated container information at block 305. In an example， the allocated container information comprises ″host： mapped_port″ information from， for example， the second parameter in the block 302.

The method 300 further comprises starting， by a load testing tool， the load testing for a respective container at block 306. In an example， the load testing tool can communicate to the container using the ″host： mapped_port″ information. A command from the load testing tool will be redirected to the container. The available bandwidth and latency of the load generator inside the container will be set according to the rule applied in block 304 for the load testing in the container.

The method 300 employs the container based virtualization technology. The load testing can be run inside the respective container. Each container has its own bound virtual Ethernet network interface (veth) . By the method 300， it can configure the applied filters on the virtual Ethernet network interface and set limits to the bandwidth， latency or the like. The test runtime system does not have to be altered during the process.

Referring now to FIG. 4， a block diagram of a system 400 for load testing based on network virtualization in accordance with various examples of the present disclosure is depicted.

The system 400 comprises a managing module 401， a load generating module 402 and a controlling module 403.

In the system 400， the load generating module 402 creates an instance of a load generator service running inside a created container. As an example， the load generating module 402 like Docker may deliver software on a Linux platform. It behaves like a virtual machine， but may be much lighter and faster. In the example， the load generating module 402 enables encapsulating all the necessary binaries and configurations of a runnable system and exporting it as an image. In the example， the load generating module 402 may create the instance of the load generator service quickly. In an example， for each test group inside the test scenario， the system 400 creates a container and runs a load generator service inside it. Each container has a bound virtual Ethernet interface. Through this virtual Ethernet interface the communication of each container is isolated from each other.

In the system 400， the controlling module 403 controls the transmission of packets in the system. As an example， a Traffic Control command can be used by the controlling module 403 to control the transmission. In the example， the controlling module 403 may be a set of tools to control and manipulate the transmission of packets in the system 400. In another example， the controlling module 403 also can virtualize the network conditions per container. The network conditions can include， but not be limited to， bandwidth and latency of the load generator service or the like.

In the system 400， the managing module 401 integrates the load generating module 402 with the controlling module 403 and provides a user-friendly access to the system 400. As an example， the managing module 401 can be a service portal， a server， an interface or the like to interact with a user. In an example， the managing module 401 creates containers. As described above， the instance of the load generator service created by the load generating module 402 can run inside the created containers. In another example， the managing module 401 applies network virtualization rules to each of the created containers， for example， using a command from the controlling module 403. As an example， the command from the controlling module 403 can be the Traffic Control command. The network conditions of the instance of the load generator service created by the load generating module 402 are set according to the network virtualization rules.

Although three modules are depicted in FIG. 4， it should be appreciated that the system 400 can include other modules such as a display module for displaying information to the user， a network interface module for receiving information from a remote device or the like. The disclosure is not limited in this regard.

Referring now to FIG. 5， a diagram of a computer system 500 for implementing various examples of the present disclosure is depicted. Examples of the present disclosure may be implemented on virtually any type of computer regardless of platforms being used. For example， as shown in FIG. 5， the computer system 500 includes a processor (s) (e.g.， central processing unit (CPU) 501) ， an associated memory 502 (e.g.， random access memory (RAM) ， cache memory， flash memory， etc. ) ， a storage device 503 (e.g.， a hard disk， an optical drive such as a compact disk drive or digital video disk (DVD) drive， a flash memory stick， etc. ) ， I/O devices 504 such as a keyboard， a mouse， a microphone (not shown) or a monitor， and a network interface 505， which are coupled to each other via a bus 506.

The memory 502 includes， but is not limited to， a load testing module 507 storing machine readable instructions， which， when executed by the processor 501， cause the processor 501 to perform operations. In an example， the operations include performing the method 300 as explained with respect to FIG. 3. In another example， the system 400 as shown in FIG. 4 may be implemented as the load testing module 507. Another example of the load testing module 507 includes instructions that cause the processor 501 to implement the system 400 as shown in FIG. 4 and the method 300 as illustrated in FIG. 3.

Referring now to FIG. 6， a diagram of results achieved by the solutions of the present disclosure is depicted， according to an example

In an example， the diagram illustrates the comparison of ″throughput″ monitoring results during load testing. The left graph presents the testing results after applying one of the examples of the present disclosure while the right graph presents the original result.

Y-axis represents the throughput (bytes/sec) in a test scenario and X-axis represents elapsed time in format of Hour： Min： Sec. It can be seen from the illustrations that the throughput value after applying one of the examples of the present disclosure is reduced to a much lower speed， which is achieved by successfully emulating a scenario with lower bandwidth.

The advantages of the present disclosure are in that it is based on container technology， which can make the best use of resources and simplify multiple network emulation with limited physical/virtual machines； the technology stack is based on free software， which reduces the cost of implementation； and the manager or managing module can be transplanted to major cloud platforms which support load generating such as Docker， which enhances the applicability and scalability of implementation.

While the disclosure has been described with respect to a limited number of examples， those skilled in the art， having benefit of the present disclosure， will appreciate that other example embodiments can be devised without departing from the scope of the disclosure as disclosed herein. Accordingly， the scope of the disclosure should be limited only by the attached claims.

Claims

A method for load testing based on network virtualization， comprising：

receiving， by a manager， a load generator deployment request， wherein the request comprises a list of test groups inside a test scenario；

creating， by the manager， containers for each test group through requests to a container engine；

querying， by the manager， information about the created containers from the container engine；

applying， by the manager， a rule to each container using a command； and

generating and sending， by the manager， a load testing scenario file to a load testing tool for load testing， wherein the load testing scenario file contains group information and allocated container information.
The method according to claim 1， wherein each group contains a test script and network virtual izati on requirements.
The method according to claim 1， wherein creating containers further comprises using two parameters to create the containers， wherein the first parameter is a pre-made load generating image to be used to initialize the containers and the second parameter is a pair of automatically mapped ports from a host environment to the containers.
The method according to claim 3， further comprising receiving， by the manager， a unique ID of the created container from the container engine.
The method according to claim 4， wherein querying information about the created containers further comprises querying the information about the created containers from the container engine using the received container IDs.
The method according to claim 4， wherein a name of a virtual Ethernet interface bound to the created container is contained in a response.
The method according to claim 1， wherein the rule is bound to the created container to filter traffic from the container.
The method according to claim 1， wherein the load testing further comprises communicating， by the load testing tool， to the created containers using the allocated container information from the manager.
The method according to claim 8， wherein the available bandwidth and latency of a load generator inside the containers are set according to the rule applied by the manager.
A system for load testing based on network virtualization， comprising：

a load generating module to create an instance of a load generator service run inside a container for the loading testing；

a controlling module to control transmission of packets in the system； and

a managing module to integrate the load generating module and the controlling module and to create containers for each test group through requests to a container engine.
The system according to claim 10， wherein the created instance of the load generator service is run inside the created container for load testing.
The system according to claim 10， wherein the controlling module is further to virtualize network conditions per created container.
The system according to claim 12， wherein the network conditions include at least bandwidth and latency of the created instance of the load generator service.
The system according to claim 10， wherein the managing module is further to apply network virtualization rules to each of the created container using a command from the controlling module.
An apparatus for load testing based on network virtualization， comprising：

a processor； and

a memory storing computer readable instructions executable by the processor to：

submit a load generator deployment request， wherein the request comprises a list of test groups inside a test scenario；

receive information about containers created in response to the load generator deployment request；

apply a rule to the containers；

generate a scenario file which contains group information and allocatedcontainer information； and

initiate the load testing for a respective container with the scenario file.