WO2018103825A1

WO2018103825A1 - System and method for low memory and low traffic overhead heavy-hitter detection

Info

Publication number: WO2018103825A1
Application number: PCT/EP2016/079921
Authority: WO
Inventors: Symeon CHOUVARDAS; Lorenzo MAGGI; Jeremie Leguay; Moez DRAIEF; Stefano PARIS
Original assignee: Huawei Technologies Co., Ltd.
Priority date: 2016-12-06
Filing date: 2016-12-06
Publication date: 2018-06-14
Also published as: CN109952743B; CN109952743A

Abstract

The present invention relates to a system and method for identifying in terms of identity and size a plurality of heavy-hitter (HH) flows amongst data packet flows in a communication network. At each routing device, the data packet flows are filtered at the ingress of an adjustable sketch-based table according to an adjustable filtering rule, and a sketch-based counter is provided to detect candidate HH flows amongst the filtered data packet flows, the candidate HH flows being the actual HH flows together with false positives. In combination, a forwarding plane counter is provided at each routing device to collect forwarding plane counters from aggregates of data packet flows according to an adjustable forwarding plane monitoring rule. A controller iteratively adjusts the respective rules monitoring rules and estimates the identification of each HH flow based on the candidate HH flows and the forwarding plane counters, until reaching a final estimation.

Description

SYSTEM AND METHOD FOR LOW MEMORY AND LOW TRAFFIC OVERHEAD HEAVY- HITTER DETECTION

TECHNICAL FIELD The present invention relates to a system and method for monitoring the largest data packet flows in a communication network, and in particular for detecting and identifying heavy-hitter flows.

BACKGROUND Network monitoring plays a crucial role in network management. It provides some information about the network traffic usually used for traffic accounting, traffic engineering, load balancing or anomaly detection.

As depicted in Fig. 1 , a conventional network monitoring system 100 can comprise probes, which locally collect statistics and are individually embedded into respective network devices such as routers, and a collector aggregating all the information reported by the different probes and analyzing or computing the statistics locally collected from the different probes for the network management system.

One of the goals of the network monitoring system is to detect and track the set of the largest data flows, often referred to as heavy-hitters (HH) in the literature. This set of data flows can, for example, be defined as the very first largest flows or as the flows having a throughput larger than a predetermined threshold. Identifying these data flows helps to understand which users, such as applications or end users, consume the most of resources and to decide whether a special treatment, such as the use of a load balancer or the use of dedicated routes, should be applied to themselves.

In its simplest implementation, the detection of the largest flows would require to monitor each flow and then to filter the largest ones. However, monitoring every flow individually is challenging, inefficient or even infeasible owing to exploding traffic volume and limited monitoring resources, such as the equipment memory and the capacity of the control plane between the probes and the collector. An impractical solution would consist in using a table with a counter for each flow and reporting the largest entries. Unfortunately, the access memory of the network devices is not fast enough to maintain such a data structure. Another impractical solution would be to duplicate the whole traffic towards the collector. However, due to the capacity limitation of the control plane between the routers and the collector, this volume of traffic cannot be exported in most of the cases.

Thus, low-complexity and low-memory stream sampling techniques have to be performed on the network devices themselves in order to overcome such resource limitations. The most standard technique for traffic monitoring is packet sampling. Packet sampling prescribes to report one out of P raw packets to the collector, which then computes traffic statistics. This technique generates a large amount of signaling traffic between the probes and the collector, which is in the order of the number of packets flowing during the monitoring time-window. In addition, this sampling technique is of low accuracy with respect to sketch- based approaches. Indeed, with a same accuracy, denoted by ε, the sketch-based algorithms taken singly use less memory (space of 0(1 /ε)) than any other packet sampling techniques (space of 0(1/ε²)) as found, for example, in: Graham Cormode, "Sketch techniques for approximate query processing", Synposes for Approximate Query Processing: Samples, Histograms, Wavelets and Sketches, Foundations and Trends in Databases, NOW publishers, 201 1.

As an alternative to the packet sampling, advanced streaming techniques, also referred to as sketch streaming algorithms, have been introduced to efficiently compute statistics over a stream of data. One of the sketch techniques, also called count-min sketch, particularly aims at keeping track of flow sizes with a limited memory (i.e., with a memory size much smaller than the total number of data packet flows), so that it can be used to detect HHs. The count- min sketch comprises a table whose cells contain aggregate flow sizes. It leverages hash functions to map the flows into the table cells, and uses only a sub-linear space unlike a simple counter table at the expense of overestimating flow sizes. For the detection of HHs, the count-min sketch is implemented inside each traffic probe. Thereby, the list of HHs is directly reported to the monitoring collector and a very low signaling traffic is generated over the control plane between the routers and the collector. Indeed, the count-min sketch generates an amount of signaling traffic in the order of the number of detected HHs.

Moreover, it needs a small memory as the table size is much smaller than the total amount of flows. The downside is that, by construction, it always overestimates the size of flows and often leads to false positives, i.e., to small flows wrongly detected as HHs. Nevertheless, for a given accuracy level, the sketch-based algorithms such as the count-min sketch always use less memory than the packet sampling techniques, as disclosed in: Graham Cormode, "Sketch techniques for approximate query processing", Synposes for Approximate Query Processing: Samples, Histograms, Wavelets and Sketches, Foundations and Trends in Databases, NOW publishers, 201 1. Another sketch-based algorithm to estimate HHs is the so called space-saving that keeps a limited size table to store the estimated large flows. On seeing a flow, if it is already stored in the table then its counter is incremented, otherwise the flow with the smallest counter is replaced and even in this case its counter is incremented, as disclosed in: Ahmed Metwally, Divyakant Agrawal, Amr El Abbadi, "Efficient computation of frequent and top-k elements in data streams", ICDTO5, Proceedings of the 10th International conference on database theory, pp. 398-412, Springer Berlin Heidelberg (2005).

Nevertheless, space-saving also systematically overestimates the size of flows.

Several flow-based monitoring techniques have been proposed in the literature, amongst which the flow sampling. These flow-based techniques compute a limited number of selected data packet flow aggregates and report accurate statistics about them to the collector.

However, as the number of flow aggregates to be monitored is limited, a large portion of the data packet flows is not taken into account in that coarse or small coverage. The data packets can be aggregated into flows using the monitoring protocol named IPFIX, standing for internet protocol flow information export, while the truncated data packets can be exported using the sampled flow (sFlow) protocol.

Another way of collecting statistics about the flow aggregates is to leverage existing counters, such as the ternary content-addressable memory (TCAM) counters, from the packet forwarding system in the routers. Indeed, each time when a packet hits an entry of a forwarding table, a counter is updated for this specific entry. Additional entries used for routing can be configured only to compute statistics on the flow aggregates. Similar to the streaming algorithms described above, this benefits from generating a very low signaling traffic to be reported to the collector. More specifically, the signaling traffic is in the order of the number of forward plane rules installed in the switches. Despite that, the computed statistics (i.e., the flow counters) are very accurate, but refer to the flow aggregates.

However, the forward plane rules are usually installed on power-hungry memories such as the TCAMs, which exhibit stringent limitations on their size. This creates the need to install rules that are each applicable to many different flows. Thereby, the collector has to solve a highly undetermined system to disaggregate flows and estimate HHs, which usually leads to very large estimation errors as disclosed in: Mehdi Malboubi et al., "Intelligent SDN based traffic (de) aggregation and measurement paradigm (iSTAMP)", IEEE INFOCOM 2014.

Another possibility can be to disaggregate the flows iteratively by assigning counters to sub- flows whenever a flow aggregate looks promising as found in Masoud Moshref et al., "DREAM: dynamic resource allocation for software-defined measurement", SIGCOMM 2014. Nevertheless, this approach is affected by a slow convergence rate and consumes a high amount of precious forward plane resources since new monitoring rules need to be installed at each time when a flow is disaggregated.

SUMMARY

It is therefore an object of the present invention to provide a system and method for identifying a plurality of heavy-hitter flows amongst a plurality of data packet flows in a communication network, by means of which an accurate heavy-hitter monitoring can be obtained using a small amount of resources in term of memory size on the routing devices and in term of control plane signaling capacity.

The object is achieved by the features of the independent claims. Further embodiments of the invention are apparent from the dependent claims, the description and the drawings of the figures.

According to a first aspect, the invention relates to a system for identifying a plurality of heavy-hitter (HH) flows amongst a plurality of data packet flows in a communication network. The system comprises a controller and at least one routing device. The controller is adapted to receive from an user a request for an identification of the plurality of HH flows, to perform iteratively an estimation of the identification of each HH flow amongst the plurality of HH flows and to send in response to the received request a final estimation of the identification of each HH flow amongst the plurality of HH flows towards the user. The at least one routing device is adapted to route the plurality of data packet flows along the communication network and to receive from the controller a request to initiate a counter collection in response to the request received from the user for the identification of the plurality of HH flows. Each routing device comprises a packet filter adapted to filter the plurality of data packet flows at the ingress of an adjustable sketch-based table according to an adjustable filtering rule, a sketch- based counter adapted to detect a plurality of candidate HH flows amongst the plurality of filtered data packet flows and a forwarding plane counter adapted to collect from aggregates of data packet flows amongst the plurality of data packet flows a plurality of forwarding plane counters according to an adjustable forwarding plane monitoring rule, wherein the controller estimates the identification of each HH flow amongst the plurality of HH flows based on both the plurality of candidate HH flows and the plurality of forwarding plane counters.

Thereby, the sketch entities and the forwarding plane counter can be jointly used to improve their respective accuracy when taken singly. In addition, the information transmitted from each routing device towards the controller is proportional to the number of candidate HH flows whose number is significantly smaller than the total number of data packet flows, which allows to improve the estimation accuracy and to use a small amount of resources in term of memory size. In terms of definition, the candidate HH flows may be considered the very first largest flows or the flows having a rate larger than a predetermined threshold, the sketch- based table may be considered a database, the size of which is much smaller than the set of data packet flows and which contains size estimation values for the detected candidate HH flows, and the aggregates of data packet flows may be considered the flows having the same forwarding rule installed on a switch. In this regard, it should be noted that the switch is unable to distinguish amongst the flows inside the same aggregate since there exists one counter per rule.

According to a first implementation of the system according to the first aspect, the controller adjusts the filtering rule and the forwarding plane monitoring rule based on the estimated identification of each HH flow amongst the plurality of HH flows.

Thereby, the iterative adjustment or updating process can be more accurately performed by the controller.

According to a second implementation of the system according to the first implementation of the first aspect, the adjustment of the filtering rule and the forwarding plane monitoring rule is iteratively performed by the controller after each estimation of the identification of each HH flow until reaching a predetermined number of iterations determined by the user, the final estimation of the identification of each HH flow amongst the plurality of HH flows being obtained after reaching the predetermined number of iterations. Thereby, the accuracy of the estimation of the identification of each HH flow can be controlled by the user through the setting of the maximum number of iterations and also improved with respect to a single measurement (i.e., without iteration) of estimation.

According to a third implementation of the system according to the first or second

implementation of the first aspect, the filtering rule is adjusted by applying a sketch-based algorithm during a predetermined time window determined by the user.

Thereby, the accuracy of the detection of each HH flow can be controlled algorithmically and also improved with respect to an instantaneous adjustment of the filtering rule.

According to a fourth implementation of the system according to any one of the preceding implementations of the first aspect, the forwarding plane monitoring rule is adjusted by splitting the plurality of data packet flows and/or by assigning forwarding plane counters to a selection of data packet flows.

Thereby, the monitoring granularity can be improved.

According to a fifth implementation of the system according to the first aspect or any one of the preceding implementations of the first aspect, the controller adjusts the sketch-based table through an adjustment of configuration parameters of the sketch-based table depending on traffic characteristics.

Thereby, an optimization of the sketch-based table can be obtained.

According to a sixth implementation of the system according to the fifth implementation of the first aspect, the configuration parameters of the sketch-based table are adjusted through an adjustment of the size of the sketch-based table based on an estimated skewness of the size distribution of the plurality of data packet flows.

Thereby, an enhanced optimization of the sketch-based table can be obtained. According to a seventh implementation of the system according to the first aspect or any one of the preceding implementations of the first aspect, a reduction of the plurality of candidate HH flows amongst the plurality of filtered data packet flows is performed using a trimmed or low-rank estimation.

Thereby, it is not needed to estimate the totality of the candidate HH flows, which allows to improve the accuracy of the detection of each HH flow and to save memory.

According to an eighth implementation of the system according to the first aspect or any one of the preceding implementations of the first aspect, the plurality of candidate HH flows is maintained by the at least one routing device.

Thereby, the complexity of the system can be reduced. In particular, the plurality of candidate HH flows can be maintained in a space-efficient way, for example, through a heap structure.

According to a ninth implementation of the system according to the first aspect or any one of the preceding implementations of the first aspect, the identification of each HH flow amongst the plurality of HH flows is related to their respective identity and size.

Thereby, the identification is not only about the identity but also about the size, which is a configurable parameter.

According to a tenth implementation of the system according to the first aspect or any one of the preceding implementations of the first aspect, each HH flow is defined either as a flow whose size is higher than the sum of the size of the plurality of data packet flows divided by a predetermined number determined by the user, as a flow belonging to the top-N of the largest flows amongst the plurality of data packet flows, N being a predetermined number determined by the user, or as a flow whose size is higher than a predetermined threshold determined by the user.

Thereby, the size of a data packet flow can be configured by the user to define a HH flow. According to an eleventh implementation of the system according to the first aspect or any one of the preceding implementations of the first aspect, each routing device is a router or a switch.

Thereby, the system can be implemented within a software-defined networking (SDN) architecture.

The above object is also solved in accordance with a second aspect.

According to the second aspect, the invention relates to a method for identifying a plurality of heavy-hitter (HH) flows amongst a plurality of data packet flows in a communication network. The method comprises receiving from a user a request for an identification of the plurality of HH flows, initiating a counter collection in response to the received request received for the identification of the plurality of HH flows, performing iteratively an estimation of the identification of each HH flow amongst the plurality of HH flows, and sending in response to the received request a final estimation of the identification of each HH flow towards the user. The step of initiating a counter collection comprises filtering the plurality of data packet flows according to an adjustable filtering rule, detecting a plurality of candidate HH flows amongst the plurality of filtered data packet flows, and collecting from aggregates of data packet flows amongst the plurality of data packet flows a plurality of forwarding plane counters according to an adjustable forwarding plane monitoring rule, and the step of performing an estimation of the identification of each HH flow amongst the plurality of HH flows is based on both the plurality of candidate HH flows and the plurality of forwarding plane counters.

According to a first implementation of the method according to the second aspect, the method comprises the step of adjusting iteratively the filtering rule and the forwarding plane monitoring rule after each estimation of the identification of each HH flow until reaching a predetermined number of iterations, the final estimation of the identification of each HH flow amongst the plurality of HH flows being obtained after reaching the predetermined number of iterations.

The above object is also solved in accordance with a third aspect. According to the third aspect, the invention relates to a computer program comprising a program code for performing the method according to the second aspect or the first implementations of the second aspect when executed on a computer.

Thereby, the method can be performed in an automatic and repeatable manner.

The computer program can be performed by the above system. The system can be programmably arranged to perform the computer program.

More specifically, it should be noted that the above apparatus may be implemented based on a discrete hardware circuitry with discrete hardware components, integrated chips or arrangements of chip modules, or based on a signal processing device or chip controlled by a software routine or program stored in a memory, written on a computer-readable medium or downloaded from a network such as the internet.

It shall further be understood that a preferred embodiment of the invention can also be any combination of the dependent claims or above embodiments with the respective independent claim.

These and other aspects of the invention will be apparent and elucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following detailed portion of the present disclosure, the invention will be explained in more detail with reference to the exemplary embodiments shown in the drawings, in which:

Fig. 1 shows a conventional block diagram of a network monitoring system 100 wherein the probes are embedded into the routers; Fig. 2 shows a schematic block diagram of a network monitoring system 200 according to an embodiment of the present invention;

Fig. 3 shows a schematic block diagram of a network monitoring system 300 in the illustrative case of a communication between the controller and an individual switch according to an embodiment of the present invention;

Fig. 4 shows a flow diagram for identifying a plurality of HH flows amongst a plurality of data packet flows according to an embodiment of the present invention;

Fig. 5 shows a schematic diagram illustrating the estimation of the skewness of the size distribution of the data packet flows according to an embodiment of the present invention;

Fig. 6 shows a schematic block diagram illustrating the signaling (Sg1 , C1 , C2) of the network monitoring system 300 according to an embodiment of the present invention.

Identical reference signs are used for identical or at least functionally equivalent features.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

Fig. 2 shows a schematic block diagram of a network monitoring system 200 according to an embodiment of the present invention. The network monitoring system 200 comprises a controller such as a software-defined networking (SDN) controller and at least one routing device (numbered from 1 to K) amongst the K routing devices. Each routing device can be a switch or a part of the switch such as a router. The controller receives a request from a user (e.g., an application or an end user) and sends its reply towards the user. Although Fig. 2 depicts a centralized controller architecture in which the controller communicates with each routing device and vice versa, it should be noted that the network monitoring system 200 of the present invention can also be extended to a distributed controller architecture in which each controller communicates with each respective routing device and vice versa. Fig. 3 shows a schematic block diagram of a network monitoring system 300 in the illustrative case of a communication between the controller and an individual switch according to an embodiment of the present invention. Thus, the network monitoring system 300 is directly derived from the network monitoring system 200 by taking K equal to unity, i.e., K=1 . As can be gathered from Fig. 3, the routing device comprises a data plane together with a forwarding plane. An application-specific traffic sampling module such as a sketch entity (e.g., the count- min sketch), which is based on a table referred to as a sketch-based table that is a database whose size is much smaller than the total number of data packet flows, is provided inside the data plane in order to provide a list of candidate heavy-hitter (HH) flows with false positives (the false positives being flows that are reported as HH flows when in fact they are not) at a low memory and a low control plane overhead cost. The forwarding plane, which is already implemented in a router or a switch, comprises a forwarding plane counter adapted to retrieve accurate statistics on aggregates of data packet flows, also referred to as forwarding plane counters (Y), through, for example, the implementation of a ternary content- addressable memory (TCAM). In response to a request for a HH flow identification from a user (e.g., an application or an end user), the controller is adapted to provide an estimated identification of each HH flow in terms of identity and size based on information exchanges between itself and the data and forwarding planes of the routing device.

In a first milestone, the present invention enables to compress any information about the data packet flows inside the data plane using a sketch-based scheme since, due to memory constraints and/or limitations, statistics related to each individual data packet flow cannot be all stored in memory. To that extent, any data packet flow arriving at the switch has its size updated by the sketch as it uses fewer memory units than the actual number of data packet flows. In spite of a lossy compression, the sketch has however the advantage of efficiently limiting the compression noise.

In a second milestone, the present invention allows the controller to exploit the information issuing from the sketch entity inside the switch, namely the information about a set of reported HHs, so as to define proper monitoring rules on the forwarding plane. In more details, the sketch entity comprises a sketch-based counter, which always overestimates the size of the data packet flows owing to the effect of lossy compression. By means of the sketch-based counter, the switch will thus detect the actual HH flows together with false positives, the false positives being flows that are reported as HH flows when in fact they are not. The switch sends both the forwarding plane counters (Y) collected by the forwarding plane counter and the detected HH flows with false positives towards the controller, which can then estimate the identification of the HH flows in terms of identity and size. Based on the resulting estimation, the controller sends newly adjusted monitoring rules stating which flows should be aggregated and/or which flows should be directly monitored towards the forwarding plane of the switch. In addition, depending on the traffic condition, the controller can send a feedback information towards the data plane of the switch in order to modify the configuration parameters (e.g., the size) of the sketch-based table. For example, the controller can decide to increase or decrease the size of the sketch-based table according to whether the skewness of the size distribution of the data packet flows is respectively small or high. Finally, the controller sends towards the switch newly adjusted filtering rules, which will filter out specific data packet flows from the ones being taken into account in the sketch computation. For example, if a data packet flow is a HH flow then it is filtered out from the sketch-based table and it is explicitly monitored by a proper adjustment of the forwarding plane. A skewness detector adapted to determine the skewness of the size distribution of the data packet flow may also be provided inside the data plane in order to improve the estimation of the identification of the HH flows. For example, the controller can decide to increase or decrease the size of the sketch-based table according to whether the skewness of the size distribution of the data packet flows is respectively small or high.

Fig. 4 shows a flow diagram for identifying a plurality of HH flows amongst a plurality of data packet flows according to an embodiment of the present invention.

At the step SO, the user (e.g., an application or an end user) sends a request for an identification of the plurality of HH flows towards the controller, the identification being related to the identity and the size. Each HH flow can be defined either as a flow whose size is higher than the sum of the size of the plurality of data packet flows divided by a

predetermined number (k) determined by the user, as a flow belonging to the top-N of the largest flows amongst the plurality of data packet flows, N being a predetermined number determined by the user, or as a flow whose size is higher than a predetermined threshold (S) determined by the user.

At the step S1 , the controller sends the received request towards each routing device (e.g., a switch or a router) in order to kick-off or initiate the sketch-based counter collection.

At the step S2, the plurality of data packet flows is filtered at the ingress of the sketch-based table according to filtering rules that are maintained and updated by the controller. More specifically, those filtering rules will filter out some specific data packet flows from the sketch- based table, namely flows that had been previously reported as HH flows and are explicitly monitored by a proper adjustment of the forwarding plane according to forwarding plane rules, such as TCAM rules, which are maintained and updated by the controller. Thus, these specific data packet flows will not participate in the construction of the sketch-based table, such as a count-min sketch table.

At the step S3, each routing device collects and sends towards the controller the forwarding plane counters (Y) along with the candidate HH flows extracted from the sketch-based table, which are HH flows with false positives. More specifically, each forwarding plane counter collects from aggregates of the data packet flows the forwarding plane counters (Y), which are stored in a vector Y, by applying the forwarding plane rules (e.g., the TCAM rules).

Moreover, each sketch-based counter collects sketch-based counters by applying a sketch- based algorithm, such as the count-min (CM) sketch algorithm, during a predetermined time window (e.g., during L seconds) determined by the user. The CM sketch is based on a table, i.e., a sketch-based table, whose size is much smaller than the total number of data packet flows. Each time when a data packet belonging to a flow (denoted f) arrives, a different hash function h,(f) is applied to f for the i-th row of the sketch-based table, and the corresponding counters denoted CM(i, h,(f)) are incremented by an amount equal to the packet size. The size of a data packet flow is then estimated as the minimum among the counters associated to the hash function, i.e., estimated as min CM(i, hi(f)). On top of this, a list of potential or

i

candidate HH flows (denoted: HH U ΔΊ) is maintained in each switch in the form of a heap structure. Afterwards, the switches send both the vector Y and the list of the candidate or potential HH flows (HH U ΔΊ) towards the controller.

At the step S4, the controller infers the HH flows (denoted: HH U Ai) and their estimated size (denoted: %, Vi e HH u A_t) based on the information coming from the step S3. More specifically, the controller estimates the identification of each HH flow by solving the system: Y = AX, where Y is a vector related to the aggregate measured flow size, A is a routing matrix and X is a vector related to the actual flow size. It should be noted that this problem is highly underdetermined since the length of the vector Y is significantly smaller than the length of the vector X. In order to improve the accuracy of the detection of the HH flows, the information coming from each sketch-based table will be exploited as well. In particular, the list of the candidate or potential HH flows reported by the CM sketch-based counter towards the controller comprises the actual HH flows along with some false positive occurrences (ΔΊ), i.e., HH U ΔΊ. Thus, it is possible to filter out all the flows that are not included in the set of flows HH U A . Therefore, instead of trying to estimate the whole vector X, the present invention restricts the estimation to the sub-vector X, which only contains the coordinates corresponding to the set of flows HH U A'_t, in order to solve the corresponding system: Ϋ = A X. Such a system can be efficiently solved using a trimmed or low-rank estimation, e.g., the trimmed least squares estimation. Thereby, a reduction of the candidate or potential HH flows can be carried out and an estimation of the identification of each HH flow in terms of identity and size can be achieved by the controller.

At the step S5, the controller sends towards each switch new forwarding plane monitoring rules and new filtering rules based on the HH flows (i.e., HH U At) inferred at the step S4. This adjustment process is an updating process that can be repeated several times after each estimation of the identification of each HH flow until reaching a predetermined number of iterations (T) determined by the user, the final estimation of the identification of each HH flow being obtained after reaching the predetermined number of iterations (T). More specifically, the controller is able to assign the forwarding plane rules, for example, the TCAM rules, to the aforementioned specific data packet flows, i.e., the flows that are previously reported as HH flows. In the case of large aggregates, it should be noted that the forwarding plane rules in TCAM could also be split as to improve the monitoring granularity. Another option would be that the controller applies filtering rules that would determine the flows capable of contributing to the CM sketch. Finally, depending on the traffic condition, the controller can decide to increase or decrease the size of the sketch-based table.

At the step S6, the controller sends the identity of the inferred HH flows along with their estimated size towards the user.

At the optional step S7, the skewness of the size distribution of the data packet flows can be estimated in order to improve the estimation of the identification of the HH flows made at the step S4 and to tune the configuration parameters of the sketch-based table at the step S3. For example, the controller can decide to increase or decrease the size of the sketch-based table according to whether the skewness of the size distribution of the data packet flows is respectively small or high. Referring to Fig. 5, the skewness estimation can be done by under-sampling the data packet flow with two different under-sampling factors ki and k₂, then by counting the number of distinct elements through a respective count-distinct module by means of a sketch algorithm as disclosed, for example, in: P. Flajolet et al. "Hyperloglog: the analysis of a near-optimal cardinality estimation algorithm", DMTCS Proceedings 1 , 2008. Finally, the ratio between each resulting number of elements D(Ti) and D(T₂) yields the estimated skewness of the size distribution of the data packet flows.

Fig. 6 shows a schematic block diagram illustrating the signaling (Sg1 , C1 , C2) of the network monitoring system 300 according to an embodiment of the present invention. As can be gathered therein and in conjunction with the flow diagram of Fig. 4, the controller (e.g., a SDN controller) can comprise an interface (e.g., an applications programming interface (API)), which allows to exchange control messages (Sg1 ) with the user at the steps SO and S6 and also command messages (C1 , C2) with the routing device at the steps S3 and S5. The interface can be implemented with, for example, the OpenFlow protocol or the simple network management protocol (SNMP).

Thus, the user communicates at the step SO with the controller through the interface (e.g., the northbound API) by sending a control signal (Sg1 ) requesting for an identification of the plurality of HH flows, whereas the controller communicates with the user at the step S6 by sending back the control signal (Sg1 ) informing about the identity of the inferred HH flows along with their estimated size through the interface such as the northbound API.

For its part, the routing device communicates at the step S3 with the controller through the interface (e.g., the southbound API) by sending both a first command signal (C1 ) informing about the forwarding plane counters (Y) and a second command signal (C2) informing about the candidate HH flows extracted from the sketch-based table. In response, the controller communicates at the step S5 with the routing device through the interface (e.g., the southbound API) by sending both the new forwarding plane monitoring rules through the first command signal (C1 ) and the new filtering rule through the second command signal (C2).

Finally, the combination of the sketch entities (i.e., packet filter, sketch-based counter, sketch-based table) and the forwarding plane counter inside each routing device as well as the updating or adjustment processes performed by the controller according to the present invention allow to accurately detect and identify the HH flows using a reduced amount of resources in term of memory on each routing device and in term of control plane signaling (Sg1 , C1 , C2) capacity. With respect to the conventional packet sampling, the present invention is beneficial by transmitting from each routing device an information, which is proportional to the number of HH flows, towards the controller. With respect to the sketch- based sampling taken singly, the present invention increases the estimation accuracy by decreasing the number of false positive occurrences owing to the combination of the sketch- based sampling and the forwarding plane counter process. With respect to the method exploiting the forwarding plane counter taken singly and in which the number of data packet flows is large (i.e., many unknown variables) and the number of aggregates of data packet flows is reduced (i.e., few equations) due to power and cost limitations, the present invention has the advantage of restricting the estimation to the set of HH flows with false positives whose number is significantly smaller than the total number of data packet flows, which allows to increase the estimation accuracy and to use a small amount of resources in term of memory size.

In summary, the present invention relates to a system and method for identifying in terms of identity and size a plurality of heavy-hitter (HH) flows amongst data packet flows in a communication network. At each routing device, the data packet flows are filtered at the ingress of an adjustable sketch-based table according to an adjustable filtering rule, and a sketch-based counter is provided to detect candidate HH flows amongst the filtered data packet flows, the candidate HH flows being the actual HH flows together with false positives. In combination, a forwarding plane counter is provided at each routing device to collect forwarding plane counters from aggregates of data packet flows according to an adjustable forwarding plane monitoring rule. A controller iteratively adjusts the respective rules monitoring rules and estimates the identification of each HH flow based on the candidate HH flows and the forwarding plane counters, until reaching a final estimation. Thus, through a combination of the sketch entities and the forwarding plane counter and an updating process performed by the controller, the present invention presents the benefits of accurately monitoring the HH flows using a small amount of resources in term of memory size on the routing devices and in term of control plane signaling capacity, thereby reducing the complexity.

While the invention has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. The invention is not limited to the disclosed embodiments. From reading the present disclosure, other modifications will be apparent to a person skilled in the art. Such modifications may involve other features, which are already known in the art and may be used instead of or in addition to features already described herein. The invention has been described in conjunction with various embodiments herein. However, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless

telecommunication systems.

Although the present invention has been described with reference to specific features and embodiments thereof, it is evident that various modifications and combinations can be made thereto without departing from the spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded simply as an illustration of the invention as defined by the appended claims, and are contemplated to cover any and all modifications, variations, combinations or equivalents that fall within the scope of the present invention.

Claims

1. A system for identifying a plurality of heavy-hitter (HH) flows amongst a plurality of data packet flows in a communication network, the system comprising: a controller adapted to receive from a user a request for an identification of the plurality of HH flows, to perform iteratively an estimation of the identification of each HH flow amongst the plurality of HH flows, and to send in response to the received request a final estimation of the identification of each HH flow amongst the plurality of HH flows towards the user; and at least one routing device adapted to route the plurality of data packet flows along the communication network and to receive from the controller a request to initiate a counter collection in response to the request received from the user for the identification of the plurality of HH flows, each routing device comprising: a packet filter adapted to filter the plurality of data packet flows at the ingress of an adjustable sketch-based table according to an adjustable filtering rule; - a sketch-based counter adapted to detect a plurality of candidate HH flows

amongst the plurality of filtered data packet flows; and a forwarding plane counter adapted to collect from aggregates of data packet flows amongst the plurality of data packet flows a plurality of forwarding plane counters (Y) according to an adjustable forwarding plane monitoring rule, wherein the controller estimates the identification of each HH flow amongst the plurality of HH flows based on both the plurality of candidate HH flows and the plurality of forwarding plane counters (Y).

2. The system of claim 1 , wherein the controller adjusts the filtering rule and the forwarding plane monitoring rule based on the estimated identification of each HH flow amongst the plurality of HH flows.

3. The system of claim 2, wherein the adjustment of the filtering rule and the forwarding plane monitoring rule is iteratively performed by the controller after each estimation of the identification of each HH flow until reaching a predetermined number of iterations (T) determined by the user, the final estimation of the identification of each HH flow amongst the plurality of HH flows being obtained after reaching the predetermined number of

iterations (T).

4. The system of claim 2 or 3, wherein the filtering rule is adjusted by applying a sketch- based algorithm during a predetermined time window (L) determined by the user.

5. The system of any one of claims 2 to 4, wherein the forwarding plane monitoring rule is adjusted by splitting the plurality of data packet flows and/or by assigning forwarding plane counters to a selection of data packet flows.

6. The system of any one of the preceding claims, wherein the controller adjusts the sketch-based table through an adjustment of configuration parameters of the sketch-based table depending on traffic characteristics.

7. The system of claim 6, wherein the configuration parameters of the sketch-based table are adjusted through an adjustment of the size of the sketch-based table based on an estimated skewness of the size distribution of the plurality of data packet flows.

8. The system of any one of the preceding claims, wherein a reduction of the plurality of candidate HH flows amongst the plurality of filtered data packet flows is performed using a trimmed or low-rank estimation.

9. The system of any one of the preceding claims, wherein the plurality of candidate HH flows is maintained by the at least one routing device.

10. The system of any one of the preceding claims, wherein the identification of each HH flow amongst the plurality of HH flows is related to their respective identity and size.

1 1 . The system of any one of the preceding claims, wherein each HH flow is defined either as a flow whose size is higher than the sum of the size of the plurality of data packet flows divided by a predetermined number (k) determined by the user, as a flow belonging to the top-N of the largest flows amongst the plurality of data packet flows, N being a predetermined number determined by the user, or as a flow whose size is higher than a predetermined threshold (S) determined by the user.

12. The system of any one of the preceding claims, wherein each routing device is a router or a switch.

13. A method for identifying a plurality of heavy-hitter (HH) flows amongst a plurality of data packet flows in a communication network, the method comprising: receiving from a user a request for an identification of the plurality of HH flows; initiating a counter collection in response to the received request received for the identification of the plurality of HH flows; performing iteratively an estimation of the identification of each HH flow amongst the plurality of HH flows; and sending in response to the received request a final estimation of the identification of each HH flow towards the user, wherein: initiating a counter collection comprises: - filtering the plurality of data packet flows according to an adjustable filtering rule; detecting a plurality of candidate HH flows amongst the plurality of filtered data packet flows; and collecting from aggregates of data packet flows amongst the plurality of data packet flows a plurality of forwarding plane counters (Y) according to an adjustable forwarding plane monitoring rule; performing an estimation of the identification of each HH flow amongst the plurality of HH flows is based on both the plurality of candidate HH flows and the plurality of forwarding plane counters (Y).

14. The method of claim 13 comprising: adjusting iteratively the filtering rule and the forwarding plane monitoring rule after each estimation of the identification of each HH flow until reaching a predetermined number of iterations (T), the final estimation of the identification of each HH flow amongst the plurality of HH flows being obtained after reaching the predetermined number of iterations (T).

5. A computer program comprising a program code for performing the method according any one of claims 13 to 14 when executed on a computer.