WO2012095193A1

WO2012095193A1 - Inferring shaping

Info

Publication number: WO2012095193A1
Application number: PCT/EP2011/063166
Authority: WO
Inventors: Udi WEINSBERG; Augustin Soule; Laurent Massoulie
Original assignee: Thomson Licensing
Priority date: 2011-01-14
Filing date: 2011-07-29
Publication date: 2012-07-19

Abstract

The present invention relates to the inference of shaping, in particular to the inference of shaping of packets transmitted along a network path. First and second packets are transmitted with a common time distribution. The respective time distributions in the arrival times of the first and second packets is measured after they have travelled along the path, and shaping is inferred to have occurred if the distribution of the first and second packets is different.

Description

Inferring shaping

The present invention relates to the inference of shaping, in particular to the inference of shaping of packets transmitted along a network path.

The ever increasing adoption of high speed Internet connections inside homes in recent years has resulted in a wide range of bandwidth hungry applications, such as video streaming, voice and video chats over IP and peer-to-peer file sharing. These applications can cause extreme stress on core network of residential Internet Service Providers (ISPs), inducing ISPs to protect their network infrastructure by restraining consumed bandwidth. This is usually achieved by deploying traffic shaping devices, enabling ISPs to have a fine-grained and differentiated, per application, control over consumed bandwidth. This form of traffic shaping is performed either with or without advertising the exact policy to users and has been referred to as "network neutrality" violation.

As traffic shaping becomes prominent in ISPs, there exists a need for better understanding the properties of traffic differentiation. For example, users that need to choose between several ISPs may wish to know about traffic shaping performed by the different ISPs. In addition, ISP operators may wish to better understand quality of service (QoS) policies of peering networks and those perceived by their own users. Furthermore, regulators may wish to evaluate the legitimacy and correctness of traffic shaping.

Several techniques have already been developed to detect traffic differentiation. One technique includes methods of targeting application-specific shaping policies. In these, the method can be specific to detect Bittorent throttling using a request to send (RST) packet inserted by the ISP.

Another technique passively collects traffic and uses a statistical method to infer whether all the classes have the same service envelope. This method can be tailored for different shaping policies, such as Weighted Fair Queuing or Strict Priority. However, the measurement point observes the traffic in order to find a shaper. Other methods detects variations on the perceived performance by passively measuring the Round Trip Time of TCP handshakes or changes in download speed. While these methods are running in an end hosts they can involve the concurrent use of an application in order to detect its throttling.

Yet another method actively sends traffic to detect the presence of shaper. An application is tested against another by sending two stream of traffic, one supposedly not shaped and the other copying the application tested. However, these methods can be cumbersome.

According to one aspect of the present invention there is provided a method of inferring whether transmitted packets have been subjected to shaping along a path, including: transmitting packets of a first type and packets of a second type along the path; receiving the packets after they have been transmitted along the path; measuring first time intervals and second time intervals, the first time intervals being the time intervals between the receipt of successive packets of the first type, and the second time intervals being the time intervals between the receipt of successive packets of the second type, the first and second time intervals each having a respective distribution of values; comparing the distribution of values for the first time intervals and the distribution of values for the second time intervals; and, on, the basis of the comparison, inferring whether one of the first and second types of packets have been subject to shaping relative to the other of the first and second types of packets .

Because the time interval between packets is observed, the received traffic is effectively looked at on a small time scale. This allows shaping to be inferred using fewer packets than would be the case otherwise. Furthermore, because fewer packets are needed, the time needed to determine shaping is reduced, thereby reducing the chance that extraneous factors or events will influence the inference.

Preferably, packets of the first type and of the second type are each transmitted such that that the transmission time intervals between successive packets of the first type and the transmission time intervals between successive packets of the second type each have a common distribution. This will reduce the need for the relationship between the distributions corresponding to the first and second types to be known. The packets of the first and second types may but need not be strictly interleaved with one another.

Preferably, packets of the first type are transmitted as a first flow, and packets of the second type are transmitted as a second flow. That is, packets will a given flow will preferably have a common source and destination internet protocol address, and yet more preferably, also a common source and destination port

According to another aspect of the invention, there is provided apparatus for inferring whether transmitted packets have been subjected to shaping along a path, including:

an input stage arranged to receive transmitted packets of a first type and transmitted packets of a second type, the input stage being arranged to receive the packets after they have been carried along a path;

a processor stage arranged in use to: measure first time intervals and second time intervals, the first time intervals being the time intervals between the receipt of successive packets of the first type, and the second time intervals being the time intervals between the receipt of successive packets of the second type, the first and second time intervals each having a respective distribution of values; compare the distribution of values for the first time intervals and the distribution of values for the second time intervals; and, in dependence on the comparison, infer whether one of the first and second types of packets have been subject to shaping relative to the other of the first and second types of packets .

Preferably, the processor is arranged in use to determine whether shaping has occurred by comparing the distribution of values of the first time intervals with the distribution of values of the second time intervals, and determining that the shaping has occurred if the comparison indicates that the two distributions are the same. In this way, the processor determines whether shaping has occurred on the basis that packets of the first and second types are transmitted such that the time intervals between the transmission of packets of the first type and time intervals between the transmission of packets of the second type each have the same distribution.

The apparatus may be distributed. In particular, the processor stage may be distributed: for example, there may be provided a receiving station for receiving packets and determining the arrival time difference between successive packets of the first type and the arrival time difference between successive packets of the second type. A remote station may be provided for analyzing the time intervals, the remote station being remote from the receiving station and connected thereto by a network link.

It will be understood that the term "packet" is intended to include various forms of data units, such as datagrams and data chunks .

The invention will now be described, by way of example, and with reference to the following drawings in which:

Figure 1 shows a :

Figure 2 shows a

Figure 3 shows in

Figure 4 shows a

Figure 5 is a

determine whether traffic shaping has occurred. Figure 6 shows (a) a packet sequence with inter arrival times listed, and (b) a corresponding table.

In Figure 1, a measurement server 10 and a probing client 12 are connected to network 14, the measurement server 10 being connected to the network at an entry point 15 through a backbone network (not shown) . In the present embodiment, the network 14 is provided by an ISP (internet service provider), and is connected to the backbone network at the entry point 15. The measurement server 10 is provided with a processor 16, memory 18 and associated hardware 19 which are together arranged to generate a series of packets. In particular, the measurement server is arranged to generate packets which correspond to a first application and to generate packets which correspond to a second application, which packets are referred to herein as first and second packets respectively. The probing client 12 likewise has a processor 13, memory 15 and other hardware 17.

The packets from the measurement server 10 are transmitted over the network 14 and to the probing client 12. The network 14 includes a plurality of routers 20 connected by links 22. At least one traffic shaper device 24 is provided in the network 14. The shaper device 24 is positioned along the path followed by the packets. The path, at least along a section thereof, will normally be common to packets from the first and second applications.

It is not always known whether the path followed by packets in the network 14 will encounter a shaper, and if so, whether the shaping is performed on the packets . In particular, if packets from different applications are generated by different servers but nevertheless enter the network 14 at the entry point 15, it is not always easy to distinguish between a situation in which shaping has occurred from a situation in which packets have been retarded before entering the network, for example because one of the servers is slow to respond or because packets from the different servers have followed different network paths before reaching entry point 15.

In overview, to provide an indication of whether shaping has occurred, in particular whether shaping has occurred in the network 14, the probing client 12 is arranged to receive and analyze the packets from the measurement server 10. In dependence on the relative arrival times of the first and second packets, and an assumption on or knowledge of the manner in which the first and second packets have been transmitted from the measurement server 10, the pc 12 is arranged to make a determination or inference as to whether the first or the second packets have been subject to shaping.

The shaper device 24 is shown in more detail in Figure 2. It comprises a plurality of buffers, here a first and second buffer 26,28, and a packet classifier 30 for determining which of the first and second applications the packets correspond to, and for channeling packets to one of the first and second buffers accordingly. The classifier is arranged to read at least part of the packets to determine an identifier or other characteristic indicative of the application responsible for a given packet. Here, packets of the first and second applications are directed to the first and second buffers respectively. A scheduler 32 is provided for selecting which of the first and second buffers packets are drawn from in order to be forwarded for onward transmission. Thus, a shaper device can be viewed as a device which, when placed on a path, applies a policy on packets, using information contained in the header or the payload of the packets so as to schedule which packet should be delivered next. A First-In First-Out policy may be applied, which case packets will be transmitted from the buffers according to their order of arrival. This correspond to the case where no shaping is applied. However a Priority scheduler will send packets from a given queue if it is not empty and select the other queue otherwise. A Weighted Fair Queuing (WFQ) scheduler associates a weight with each queue and selects packets using these weights. For instance a 2-1 WFQ picks the first queue twice as more as the second queue .

The probing client 12 of Figure 1 is shown in more detail in Figure 3. The probing client 12 has an input stage 34 for receiving packets from the network 14, which input stage is coupled to an analyzer stage 36. The analyzer stage is arranged to read at least part of the arriving packets in order to determine whether they are first packets or second packets. The analyzer stage 36 is coupled to a clock stage 38 for determining the time of arrival or other temporal characteristic of the first and second packets. In dependence on whether shaping is detected, an alert signal is generated. A user interface 40 is also provided, the user interface having a display 42 for displaying an indication of whether shaping has occurred, and input keys 44 for a user to enter commands. The analysis stage and clock stage are preferably implemented on the processor 13 and a memory 15.

A set of packet sequences (i) to (iv) is show in Figure 4. Sequence (i) is a packet stream as transmitted by the measurement server 10. Packets are indicated by squares and circles, depending on whether they are first packets or second packets. Sequence (i) represents one embodiment of the invention in which the first and second packets are interleaved, and the time between the transmission of two successive packets of a given type (shown by the greek symbol tau in Figure 4) is constant. Sequences (ii) to (iv) show the order of packets upon arrival at the probing client 12 in different situations. In each situation, the order of packets has changed, that is, first and second packets are no longer interleaved, and it appears that shaping has taken place. The packets of the first and second types are preferably transmitted according to the internet protocol, in which case they will be preferably transmitted as first and second flows respectively .

There may be situations in which shaping occurs despite the fact that the order of the first and second packets is maintained, or, conversely, first and second packets may no longer be interleaved without traffic shaping having been applied .

In order to determine if traffic shaping has occurred, the analyzer stage 36 is arranged to perform the steps show in Figure 5. A first step is to determine the inter-arrival time (IATs) of a packets of the same type. Accordingly, the analyzer stage is arranged to calculate the difference in arrival time of successive packets of a given type using signals from the clock stage 38. The analyzer stage 36 is then arranged rank the IATs; that is, to put the calculate IATs in order of increasing magnitude and to number each IAT with an increasing integer value. The result of this step can be viewed as a table which in one column is the row number, and another column has is an indication of whether the associated IAT for that row corresponds to a first packet or a second packet. To illustrate this, Figure 6a shows sequence (ii) from Figure 4 with the inter arrival times indicated as til, tl2 etc for first packets, and t21, t22 etc for second packets. A ranking table derived from these arrival times is shown in Figure 6b, where the inter arrival times are such that t22>t21>tll>tl3>tl4>tl2>t23>t24>t25.

Next, the sum of the ranks, that is, row numbers, of IATs of first packets is computed. Next, the values U, mu and sigma defined with reference to the equations below are calculated. Next, z=Abs ( (U-mu) /sigma) is obtained, and in dependence on the magnitude of the determined z, is inferred that shaping has occurred.

rr _ 71 1 . ( l-i — 1 ——

> ranks

9 _

nl*ⁿ2

Here ni is the number of packets received from the ith flow. That is, nl is the number of first packets received and n2 is the number of second packets received. Since U can be approximated by a normal distribution we use the normalized value z related to the probability that the two distributions are identical.

Using the normal sigma rule for z, it is considered that there is less than 1% likelihood that the two distributions are identical if z > 3. That is, shaping is inferred to have taken place if z > 3. However, a different threshold for z may be chosen, a lower threshold allowing shaping to be inferred with a lower degree of certainty. For example, a threshold of z > 2 may be chosen, in which case shaping can be inferred with a certainty value of 5% . In this way, a comparison of the distributions of the first and second packet IATs is obtained. The distribution of the first and second packets inter transmissions times is the same, so by determining if the distribution of inter arrival times is the same, it can be inferred whether shaping has occurred to one packet type relative to the other. Variation in inter arrival times due to network fluctuations or delays to both packet types is likely to have the same effect on both packet types on average, and is not likely to alter the difference in the distribution of the inter arrival types of the first and second packets .

By distribution, it is meant the number of IATs that for each value of time (each value of time being a time increment corresponding to the measurement granularity) . Thus, for illustration, if time is measured in milliseconds, the distribution will be the number of IAT that have a value of lms, the number that have a value of 2ms, the number that have a value of 3ms...etc. In this way, the distribution can be seen as the number of IATs as a function of time. As a result of the above steps, less than 1000, and even less than 500 packets are transmitted in order to infer that shaping has occurred. The packets are transmitted at a constant rate.

Figure 7 shows a system 46 according to a further embodiment of the invention. There is provided an experiment server 46; a measurement server 48; a spoofed server 50; and a probing client 52. The experiment server 46, measurement server 48 and the spoofed server 50 are connected by a first network referred to as the internet 54, and the spoofed server 50 is connected to the internet 54 though a residential ISP network 56.

The measurement server 48 is arranged to create a small number of packets that mimic the two application that need to be tested, that is, to determine whether packets of one application are subjected to shaping relative to those of another. These packets are then transmitted in an interleaved fashion, as indicated schematically in series (i) of Figure 2 where the square and round symbols indicate packets from a first and second application respectively. The measurement server is arranged to send the interleaved sequence preferably without using any congestion avoidance protocol, and at a high speed rate. In the special case of TCP, each packet is created by a program running on the measurement server program, avoiding any congestion control of an TCP/IP stack, if any. Clearly, the above described features may be applied to the measurement server of the first embodiment.

As in the first embodiment, the probing client 52 is arranged the receive test packets from the measurement server 48. However, in the present embodiment, the probing client is arranged measure the inter arrival times of the packets from each application, and to transmit the measurements to the experiment server 46.

The experiment server 46 is arranged to transmit the following information to the probing client: The Hostname or IP of the measurement server; The Hostname or IP where to send the measurement result; The unique ID for the measurement campaign; The unique ID of this experiment; Packet filter rules, expressed in the Berkeley Packet Filter format, to identify the probing flows; The Hostname or IP of the spoofed hosts, if any.

The experiment server is arranged to determine which pair of applications should be tested between the measurement server and the probing client. Once the experiment server receives the measurement results from the probing client it uses the set of algorithms explained with reference to Figure 5 in order to detect or infer the presence of a shaper in the path between the measurement server and the probing client.

The spoofed server is preferably out of the direct control the other servers in Figure 7. In particular, only the first handshake packets are exchanged with it, all subsequent communication happens between the measurement server (using the spoofed server's address) and the probing client.

The following steps may be performed in order to infer packet shaping: First, the probing client transmits a test requests to the experiment server (step 1), the test request indicating that the probing client wishes for an experiment or test to be carried out to determine whether traffic shaping is occurring. In response, the experiment server identifies the next experiment to be run at the client and sends back a set of commands to run at the probing client. If the test involves a third-party server, such as YouTube, the probing client opens a connection to this server (optional step 2) and send back the connection identifiers (such as TCP sequence number) to the experiment server. The experiment server then contacts the measurement server (step 3) with the connection identifiers and the test details. The measurement server then sends the test packets or probes as described above to the probing client (step 4), potentially impersonating the third party server.

The probing client is arranged to record the packets sent by the measurement server and send them back to the experiment server, closing the test. The probing client is then ready to start a new test.

In the present embodiment, the inter-departure time distributions are the same for the packets from each application. In order to obtain such a situation, it is helpful to dis-activate any congestion avoidance algorithm that may be in place. This may require a privileged access to the network stack. Since it is not always easy to have such access to servers, it is helpful to "emulate" the connection from them. This is done by using the spoofed server. Moreover in order to test the same path it useful to send the packets from each application from the same machine or starting point.

In summary, the shaper detection described above uses the inter-arrival time between each packets of an application. Each test leads to two inter-arrival series, one for each application. The distribution of these two series are then compared using a statistical technique. Ά shaper is assumed to be in the path if the two series differ with a large confidence. One test used to compare the two series is the Mann-Whitney U-Test. This shaper detection method has been validated using real-life experiments as well as controlled experiments inside a test bed. However, other statistical techniques may used instead of the Mann-Whitney test.

Claims

1. A method of inferring whether transmitted packets have been subjected to shaping along a path, including:

transmitting packets of a first type and packets of a second type along the path;

receiving the packets after they have been transmitted along the path;

measuring first time intervals and second time intervals, the first time intervals being the time intervals between the receipt of successive packets of the first type, and the second time intervals being the time intervals between the receipt of successive packets of the second type, the first and second time intervals each having a respective distribution of values;

comparing the distribution of values for the first time intervals and the distribution of values for the second time intervals; and,

on, the basis of the comparison, inferring whether one of the first and second types of packets have been subject to shaping relative to the other of the first and second types of packets .

2. A method as claimed in claim 1, wherein the packets are transmitted in a seguence such that packets of the first type are interleaved with packets of the second type.

3. A method as claimed in claim 2, wherein the packets are interleaved such that each packet of the first type has neighboring packets of the second type, and each packet of the second type has neighboring packets of the first type.

4. A method as claimed in any of the preceding claims, wherein packets of the first type and packets of the second type are each transmitted such that that the transmission time intervals between successive packets of the first type and the transmission time intervals between successive packets of the second type each have a common distribution.

5. A method as claimed in any of the preceding claims, wherein packets of the first type and packets of the second type are each transmitted at a constant rate.

6. A method as claimed in any of the preceding claims, wherein less than 1000, preferably less than 500 packets are transmitted in order to infer that shaping has occurred.

7. A method as claimed in any of the preceding claims, wherein the packets of the first type are transmitted as a first flow, and packets of the second type are transmitted as a second flow .

8. A method claimed in any of the preceding claims, wherein the packets of the first type are generated by a first application, and packets of the second type are generated by a second application.

9. A method as claimed in any of the preceding claims, wherein, to compare the distribution of values for the first time intervals and the distribution of values for the second time intervals, a measure of confidence that the two distributions are the same is obtained.

10. A method as claimed in claim 9, wherein the shaping is inferred if the measure of confidence is above a threshold value.

11. Apparatus for inferring whether transmitted packets have been subjected to shaping along a path, including:

12. Apparatus as claimed in claim 11 wherein the processor stage is arranged in use to infer that the shaping has occurred if the comparison indicates that the two distributions are the same.

13. Apparatus as claimed in claim 12 wherein the processor stage is arranged in use to determine whether shaping has occurred if the two distributions are the same to within a level of likelihood.

14. Apparatus as claimed in any of claims 11 to 13, wherein the apparatus is distributed.

15. Apparatus as claimed in claim 14, wherein the processor stage is distributed.