WO2017030515A1 - A method for estimating flow size distributions - Google Patents
A method for estimating flow size distributions Download PDFInfo
- Publication number
- WO2017030515A1 WO2017030515A1 PCT/TR2016/050112 TR2016050112W WO2017030515A1 WO 2017030515 A1 WO2017030515 A1 WO 2017030515A1 TR 2016050112 W TR2016050112 W TR 2016050112W WO 2017030515 A1 WO2017030515 A1 WO 2017030515A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- module
- flow size
- estimation
- pricing
- sampling
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000009826 distribution Methods 0.000 title claims abstract description 20
- 238000005070 sampling Methods 0.000 claims description 36
- 238000004458 analytical method Methods 0.000 claims description 10
- 238000012800 visualization Methods 0.000 claims description 9
- 230000001413 cellular effect Effects 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 4
- 230000006870 function Effects 0.000 description 9
- 230000005856 abnormality Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000036962 time dependent Effects 0.000 description 2
- 241000721662 Juniperus Species 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 238000012913 prioritisation Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- Marketing (AREA)
- Game Theory and Decision Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Development Economics (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention relates to a method for estimating flow size distributions which is performed by a telecommunication system enabling applicability of the dynamic customized pricing techn ques by providing accurate estimation of the service usage rates via flow size distributions (FSD) of the end users and by providing a correlation between the estimated FSDs and a particular ujtili8ty function representing the pricing method of the data services in telecommunication field.
Description
A METHOD FOR ESTIMATING FLOW SIZE DISTRIBUTIONS TECHNICAL FIELD
The invention relates to a method for estimating flow size distributions which is developed for operators in telecommunication field; which facilitates estimating the usage patterns; which can characterize the statistical information about traffic patterns, network resource usage and user behaviors; which can enable detect traffic/user abnormality and improve network performance; which offers solutions for evaluation and development of the pricing policy of the network operator; and which is performed by a telecommunication system enabling applicability of the dynamic customized pricing techniques by providing accurate estimation of the service usage rates via flow size distributions (FSD) of end users and by providing a correlation between the estimated FSDs and a particular utility function representing the pricing method of the data services in telecommunication field.
STATE OF ART Most of the studies for estimating flow size distribution focus on deducing from sampled traffic data. However, if it is taken into account that the general data size is very large, some new studies take the data flow into account. In order to solve this problem, sampling techniques should be used which provide a balance between the practical application limits and estimation accuracy.
Currently, two main solutions are generally used in telecommunication field for tracking flows: sFlow : It is based on the uniform sampling technology which meets the basic requirements for a network tracking solution (a packet is a sample per second (s)).
- NetFlow : It is a protocol collecting IP traffic data which was developed by Cisco Systems.
Most of the routers and switches support this feature. Also, routers such as Juniper jFlow support NetFlow solution as they can receive NetFlow data.
The drawbacks of the existing applications are listed as follows:
NetFlow and sFlow solutions may miss some tiny flows although they can identify estimation errors important only for FSD estimations. Tiny flows consist of few number of packets. When compared to large flows, it is more probable to miss these packets via uniform sampling.
- Another drawback of the existing approaches is that the time intervals in FSD variations cannot be tracked.
Existing FSD estimation algorithms do not constitute a utility function based on the methodology and pricing policies.
By using a given utility function, an integrated solution has not been found yet which enables the pricing policies to be estimated in a customized way.
The following patent applications can be given as examples for the known state of the art:
[1 ] EP 20050253187 (M.S. Kodialam and T.V. Lakshman. Per-flow traffic estimation, December 7 2005)
[2] EP 20050253184 (H. Fang, S.K. Muralidharan, and V.L. Tirunell. Accelerated per-flow traffic estimation, December 7 2005.)
[3] US 7,957,315 (N. Duffield, L.M. Breslau, C. Ee, A. Gerber, C. Lund, and S. Sen. System and method for sampling network traffic, June 7 201 1 .)
[4] US 8,064,359 (N. Duffield, L.M. Breslau, C. Ee, A. Gerber, C. Lund, and S. Sen. System and method for spatially consistent sampling of flow records at constrained, content-dependent rates, November 22 201 1 .)
[5] US 8,335,160 (A. Carvalho, Y. Teplitsky, S. Rahman, M. Tiwari, and V. Valluri. Flow sampling with top talkers, December 18 2012.)
[6] US 1 1 /258,444 (B.Krishnamurthy. Method and apparatus for data network sampling, January 25 2007)
[7] US 7,512,980 (J.A. Copeland and J. Jerrim. Packet sampling flow-based detection of network intru- sions, March 31 2009)
[8] US 13/197,402 (K.M. Worth. Detecting suspicious network activity using flow sampling, February 7 2013)
[9] US 13/780,941 (M. Chiang, C. Jae-Wong, S. Ha, and S. Sen. System and methods for time dependent internet pricing, August 29 2013.)
In [1 ], a method of estimating per-flow traffic by determining the sampling interval based on a desired level of accuracy is described. In [2], a method of splitting the incoming traffic stream into a number of sub-streams by a hash function for flow display which requires high memory is described. In [3], a hash function is defined in order to calculate the number of the sampled flows and the sampling possibility is chosen according to the real flow sizes. In [4], it is extended as to include a flow based prioritization. A sampling technique which is not proper for source usage of the users is described in [5]. A sampling according to the signatures designed in accordance with a certain application such as intervention is described in [6].
In [7], a technique for analyzing the sampled packets in order to detect the intrusions by assigning a concern index value for each flow is described. In [8], the flows are sampled in order to evaluate the security of the network and in order to detect any kind of suspicious activity. Moreover, in [9], systems and methods for implementing time-dependent pricing in wireless/broadband access networks are disclosed.
Consequently, because of the mentioned drawbacks and the inadequacies of the existing solutions, an improvement or development in the methods for estimating flow size distribution which enable the pricing policies to be estimated in a customized way different from the known methods is necessary.
OBJECTIVES OF THE INVENTION
The objective of the invention is to improve the estimation methods of the flow sizes by providing a new method in order to estimate the above mentioned flow size distribution and in order to overcome the disadvantages in user-specific pricing.
The objective of the invention is to provide a method which provides an applicable solution having high accuracy rate and which combines data transmission with packet sampling.
The objective of the invention is to facilitate the estimation procedure of the patterns in network data traffic which is critical for an operator by being different from the existing applications.
An objective of the invention is to provide information necessary for traffic planning, network management and efficient usage of available sources which are encountered in demand- oriented architecture structures.
An objective of the invention is to help the network operator by providing potential clues in order to characterize the statistical information on traffic patterns, network source usage and user behaviors; in order to detect the traffic/user abnormality; and in order to increase the network performance.
An objective of the invention is to provide an extensive point of view in order to evaluate and improve the pricing policy of the network operator thanks to the patterns.
An objective of the invention is to search, design and improve the tools which can determine the output of a particular pricing policy.
The objective of the invention is to search for the applicability of the dynamic customized pricing techniques in order to evaluate the general utility function for data services.
The objective of the invention is to provide a solution which comprises an accurate determination of the service usage rates via flow size distributions (FSDs) of the end users and correlation of the data services of the determined FSDs with a particular utility function representing the pricing method.
The invention is a method for estimating flow size distributions which is performed by a telecommunication system enabling applicability of the dynamic customized pricing techniques by an accurate determination of the service usage rates via FSDs of the end users and correlation of the determined FSDs with a particular utility function representing the pricing method of data services in telecommunication field.
In order to fulfill the above mentioned objects the method comprises the steps of:
sampling the packets from the Cellular Telecommunication Network Connection (1 ) according to the available sampling parameters by the Customizable Sampling Module
(2);
- transmitting the packet information obtained after sampling to the Flow Size Estimation and Estimation Module (3);
updating the flow size estimation according to the new information; if necessary, changing the sampling parameter and sending it to the Customizable Sampling Module (2) by the Flow Size Estimation and Estimation Module (3);
- sending the estimated flow size to the Clustering Algorithm (4) in order to cluster according to the day and time;
- analysis of the network usage by the Usage Analysis Module (5) by using the flow size estimation;
- providing an income account according to different pricing models by the Pricing and Income Account Module (6);
visualization of the information on the number of the packets and the sizes thereof in the flow by the Visualization Module (7).
DRAWINGS WHICH HELP UNDERSTANDING OF THE INVENTION
The structural and characteristic features and all of the advantages of the present invention can be understood more clearly thanks to the drawings and the detailed description given below with reference to the these drawings and therefore, the assessment should be performed by taking these drawings and detailed description into account.
Figure 1 is a general view of method operation of the present invention and the communication of the elements performing this method in the system with each other.
DEFINITIONS OF THE REFERENCES
1. Cellular Telecommunication Network Connection
2. Customizable Sampling Module
3. Flow Size Estimation and Estimation Module
4. Clustering Algorithm (Clustering)
5. Usage Analysis Module
6. Pricing and Income Account Module
7. Visualization Module
The drawings do not need to be scaled and unnecessary details may be omitted which are not needed for the invention to be understood. Moreover, the elements which are substantially identical or which have at least substantially identical functions are denoted with the same number.
DETAILED DESCRIPTION OF THE INVENTION
The preferred process steps of the Flow Size Distribution Estimation method of the present invention are only described in order to clarify the subject matter in this description.
Most of the studies on estimation of flow size distribution focus on deduction from sampled traffic data. However, when it is considered that the data size is very large, some recent studies take data flow into consideration. Because, in order to process these estimation algorithms high speed connections, a high CPU and memory are necessary. The flow size distribution estimation method of the present invention provides an applicable solution with high accuracy rate and combines data transmission with traditional packet sampling. Therefore the invention provide searching new sampling methods of data flow.
High connection speed and minimum source usage is necessary in order to process the FSD estimation algorithms. For this reason, the stability of the connection speed is very important. Moreover, it is known that the most accurate solution for FSD estimates enable estimation of the whole flow population distribution at the measurement point. Online detection of the traffic flow size distribution is provided by obtaining an online algorithm via using probabilistic methods which are suitable for network-wide distribution.
Another drawback of the existing approaches is that time intervals in FSD variations cannot be tracked. The method of the present invention enables the models to be developed by including time notion and enables understanding how the flow sizes vary based on time. Therefore, tracking of the traffic flow size distributions varying based on time can be performed.
The detected flow size distributions can be used for basic problems such as testing the pricing strategies.
The system performing the method of the present invention generally comprises:
- a Cellular Telecommunication Network Connection (1 ) provided by the operator;
- a Customizable Sampling Module (2) for obtaining a sample data value from the high speed data transmission;
a Flow Size Estimation and Estimation Module (3) used for estimation of the flow sizes belonging to the sampled data and for flow size estimations fulfilled;
- a Clustering Algorithm (4) which clusters the data sizes according to the day and time;
- a Usage Analysis Module (5) which analyzes the information obtained by FSD and
clustering process and data usage of the subscribers;
a Pricing and Income Account Module (6) which calculates the income from the subscriber whose usage and pricing are determined within the given input parameters of the chosen pricing model;
- a Visualization Module (7) which visualizes the result of the network usage analysis;
The Customizable Sampling Module (2) samples the packets from Cellular Telecommunication Network Connection (1 ) according to available sampling parameter. In this step, a part of the packets are processes for counting and some of them are ignored. The packet information obtained after sampling is transmitted to the Flow Size Estimation and Estimation Module (3). This module updates the flow size estimation according to the new information. At the same time, if necessary, it changes the sampling parameter and sends it to the Customizable Sampling Module (2). Estimated flow size is sent to the Clustering Algorithm (4) in order to be clustered based on day and time. The Usage Analysis Module (5) analyzes the network usage by using flow size estimation and the Pricing and Income Account Module (6) performs an income account according to different pricing models. All of the information and statistical results about the packet number and packet sizes in the flow are graphically visualized by the Visualization Module (7). The Visualization Module (7) can process real-time and offline data.
Claims
1. A method for estimating flow size distributions which is performed by a telecommunication system enabling applicability of the dynamic customized pricing techniques by an accurate determination of the service usage rates via flow size distributions (FSD) of end users and correlation of the determined FSDs with a particular utility function representing the pricing method of data services in telecommunication field, characterized in that the telecommunication system performing said method comprises: - a Cellular Telecommunication Network Connection (1 ) provided by the operator;
- a Customizable Sampling Module (2) for obtaining a sample data value from the high speed data transmission;
a Flow Size Estimation and Estimation Module (3) used for estimation of the flow sizes belonging to the sampled data and for flow size estimations fulfilled;
- a Clustering Algorithm (4) which clusters the data sizes according to the day and time;
- a Usage Analysis Module (5) which analyze the information obtained by FSD and
clustering process and data usage of the subscribers;
a Pricing and Income Account Module (6) which calculates the income from the subscriber whose usage and pricing are determined within the given input parameters of the chosen pricing model;
- a Visualization Module (7) which visualize the result of the network usage analysis; and characterized in comprising the steps of: - sampling the packets from the Cellular Telecommunication Network Connection (1 ) according to the available sampling parameters by the Customizable Sampling Module
(2);
- transmitting the packet information obtained after sampling to the Flow Size Estimation and Estimation Module (3);
- updating the flow size estimation according to the new information; if necessary, changing the sampling parameter and sending it to the Customizable Sampling Module (2) by the Flow Size Estimation and Estimation Module (3);
- sending the estimated flow size to the Clustering Algorithm (4) in order to cluster according to the day and time;
- analysis of the network usage by the Usage Analysis Module (5) by using the flow size estimation;
- providing an income account according to different pricing models by the Pricing and Income Account Module (6);
- visualization of the information on the number of the packets and the sizes thereof in the flow by the Visualization Module (7) .
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TR2015/10226 | 2015-08-18 | ||
| TR201510226 | 2015-08-18 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2017030515A1 true WO2017030515A1 (en) | 2017-02-23 |
Family
ID=56119733
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/TR2016/050112 WO2017030515A1 (en) | 2015-08-18 | 2016-04-14 | A method for estimating flow size distributions |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2017030515A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110177348A (en) * | 2019-04-22 | 2019-08-27 | 中国移动通信集团河北有限公司 | Flow authentication control method and device |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050210533A1 (en) * | 2001-11-30 | 2005-09-22 | Copeland John A | Packet Sampling Flow-Based Detection of Network Intrusions |
| EP1603274A1 (en) | 2004-06-04 | 2005-12-07 | Lucent Technologies Inc. | Per-flow traffic estimation |
| EP1603273A1 (en) | 2004-06-04 | 2005-12-07 | Lucent Technologies Inc. | Accelerated per-flow traffic estimation |
| US20070019548A1 (en) | 2005-07-22 | 2007-01-25 | Balachander Krishnamurthy | Method and apparatus for data network sampling |
| US20100034102A1 (en) * | 2008-08-05 | 2010-02-11 | At&T Intellectual Property I, Lp | Measurement-Based Validation of a Simple Model for Panoramic Profiling of Subnet-Level Network Data Traffic |
| US20100157809A1 (en) * | 2008-12-23 | 2010-06-24 | At&T Intellectual Property I, L.P. | System and method for spatially consistent sampling of flow records at constrained, content-dependent rates |
| US7957315B2 (en) | 2008-12-23 | 2011-06-07 | At&T Intellectual Property Ii, L.P. | System and method for sampling network traffic |
| EP2372953A2 (en) * | 2010-03-30 | 2011-10-05 | Telefonaktiebolaget L M Ericsson (Publ) | Flow sampling with top talkers |
| US20130036469A1 (en) | 2011-08-03 | 2013-02-07 | Worth Kevin M | Detecting suspicious network activity using flow sampling |
| US20130226669A1 (en) | 2012-02-29 | 2013-08-29 | The Trustees Of Princeton University | System and Methods for Time Dependent Internet Pricing |
-
2016
- 2016-04-14 WO PCT/TR2016/050112 patent/WO2017030515A1/en active Application Filing
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050210533A1 (en) * | 2001-11-30 | 2005-09-22 | Copeland John A | Packet Sampling Flow-Based Detection of Network Intrusions |
| US7512980B2 (en) | 2001-11-30 | 2009-03-31 | Lancope, Inc. | Packet sampling flow-based detection of network intrusions |
| EP1603274A1 (en) | 2004-06-04 | 2005-12-07 | Lucent Technologies Inc. | Per-flow traffic estimation |
| EP1603273A1 (en) | 2004-06-04 | 2005-12-07 | Lucent Technologies Inc. | Accelerated per-flow traffic estimation |
| US20070019548A1 (en) | 2005-07-22 | 2007-01-25 | Balachander Krishnamurthy | Method and apparatus for data network sampling |
| US20100034102A1 (en) * | 2008-08-05 | 2010-02-11 | At&T Intellectual Property I, Lp | Measurement-Based Validation of a Simple Model for Panoramic Profiling of Subnet-Level Network Data Traffic |
| US20100157809A1 (en) * | 2008-12-23 | 2010-06-24 | At&T Intellectual Property I, L.P. | System and method for spatially consistent sampling of flow records at constrained, content-dependent rates |
| US7957315B2 (en) | 2008-12-23 | 2011-06-07 | At&T Intellectual Property Ii, L.P. | System and method for sampling network traffic |
| US8064359B2 (en) | 2008-12-23 | 2011-11-22 | At&T Intellectual Property I, L.P. | System and method for spatially consistent sampling of flow records at constrained, content-dependent rates |
| EP2372953A2 (en) * | 2010-03-30 | 2011-10-05 | Telefonaktiebolaget L M Ericsson (Publ) | Flow sampling with top talkers |
| US8335160B2 (en) | 2010-03-30 | 2012-12-18 | Telefonaktiebolaget L M Ericsson (Publ) | Flow sampling with top talkers |
| US20130036469A1 (en) | 2011-08-03 | 2013-02-07 | Worth Kevin M | Detecting suspicious network activity using flow sampling |
| US20130226669A1 (en) | 2012-02-29 | 2013-08-29 | The Trustees Of Princeton University | System and Methods for Time Dependent Internet Pricing |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110177348A (en) * | 2019-04-22 | 2019-08-27 | 中国移动通信集团河北有限公司 | Flow authentication control method and device |
| CN110177348B (en) * | 2019-04-22 | 2021-03-19 | 中国移动通信集团河北有限公司 | Flow authorization control method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200322237A1 (en) | Traffic detection method and traffic detection device | |
| CN108965347B (en) | Distributed denial of service attack detection method, device and server | |
| US9503465B2 (en) | Methods and apparatus to identify malicious activity in a network | |
| JP5612696B2 (en) | Network management system and method for identifying and accessing quality of service results within a communication network | |
| CN110225037B (en) | DDoS attack detection method and device | |
| KR20080066653A (en) | Method and apparatus for whole-network anomaly diagnosis and methods to detect and classify network anomalies using traffic feature distributions | |
| CN113132180B (en) | Cooperative type large flow detection method facing programmable network | |
| CN105763387B (en) | network traffic monitoring method and device | |
| JP2008283621A (en) | Apparatus and method for monitoring network congestion state, and program | |
| JP4324189B2 (en) | Abnormal traffic detection method and apparatus and program thereof | |
| JP5801241B2 (en) | Network state change detection system, traffic information storage device, network state change detection method, and traffic information storage program | |
| CN109952743B (en) | System and method for low memory and low flow overhead high flow object detection | |
| JP2008042879A (en) | Congestion path classification method to classify congestion path based on packet delay fluctuation, management apparatus and program | |
| JP2007036839A (en) | Apparatus, system, and method for dividing quality deterioration in packet exchange network | |
| CN106789429B (en) | A kind of adaptive low-cost SDN network link utilization measurement method and system | |
| JP6325993B2 (en) | Service monitoring apparatus and service monitoring method | |
| WO2017030515A1 (en) | A method for estimating flow size distributions | |
| JP5180247B2 (en) | Packet sampling apparatus and method and program | |
| CN108063764B (en) | Network traffic processing method and device | |
| CN114697272B (en) | Traffic classification method, system and computer readable storage medium | |
| Ogino | Evaluation of machine learning method for intrusion detection system on Jubatus | |
| JP4814270B2 (en) | Traffic fluctuation amount estimation method, apparatus and program thereof | |
| JP4732386B2 (en) | Congestion path classification method, management apparatus, and program for classifying congestion paths generated in wide area network | |
| CN114640600A (en) | Network service quality analysis method, system and storage medium | |
| CN106330743B (en) | Method and device for measuring flow balance degree |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 16728755 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 16728755 Country of ref document: EP Kind code of ref document: A1 |