JP6714532B2 - Throughput measuring method, program and device - Google Patents

Description

The present invention relates to a throughput measuring method, a program, and a device for measuring the throughput of the communication from a device log that relays the communication or a log collected by a packet analysis device in an IP communication network.

There is known a technique of acquiring communication quality information of a network from a DPI (Deep Packet Inspection) device or a device log. Patent Document 1 discloses a technique for measuring the throughput of a network by an active method. Patent Document 2 discloses a technique in which end-to-end communication quality grouping is performed and the target communication group is used as a sample according to the user experience throughput calculation application.

JP 2016-116104 Special table 2012-531146

In passive network performance measurement that relies on packet observation and device logs, the expected throughput may not be obtained if communication is restricted due to poor client or server performance. As a result, the performance measurement result is down and reliability is low.

In patent document 2, in order to solve such a technical problem, the communication performance between a client and a server is stored. Then, for example, the estimation accuracy is improved by dividing into groups according to the high transfer rate and obtaining the measurement statistic value from the sample of only the group that can perform high-speed communication when measuring the network performance.

However, in measuring the end-to-end network performance from the log of the communication device or the DPI device, the accuracy of the method of Patent Document 2 is low to reject the unsuitable sample observation value, and the desired estimated value is obtained. I can't.

An object of the present invention is to solve the above technical problems and to provide a throughput measuring method, a program, and a device capable of improving a throughput estimation system based on communication flow information acquired from a DPI device or the like.

In order to achieve the above-mentioned object, the present invention is characterized in that a throughput measuring device for measuring the throughput of communication of interest based on a communication flow from an arbitrary source to a destination has the following configuration. There is.

(1) The means for acquiring the source information, the destination information, and the throughput for each communication flow, the means for classifying each communication flow based on the source information and/or the destination information, and the communication flow for each classification destination A means for screening based on the statistical value of the throughput of the classification destination and a means for measuring the throughput of the communication flow after the screening corresponding to the communication of interest are provided.

(2) The screening means excludes all communication flows of classification destinations in which the ratio of communication flows whose throughput exceeds a predetermined throughput threshold does not reach the predetermined ratio threshold.

(3) From the acquisition unit, the communication start time, end time, communication volume, and number of constituent packets of the TCP connection are acquired as the communication characteristic amount, and the packet size transmitted/received in the TCP connection is calculated. A means for obtaining an average throughput of a TCP connection exceeding a predetermined threshold is provided, and the average throughput represents the throughput of the TCP connection.

According to the present invention, the following effects are achieved.

(1) Throughput, which is a typical indicator of communication quality, depends not only on the network quality of the evaluation target, but also on the capabilities of the source node (terminal), destination node (server), domain (service), etc. According to the present invention, a communication flow having a source or a destination as a factor that may decrease throughput, or a communication flow with a domain that provides a service that may cause a drop in loopt is excluded from throughput evaluation targets. Throughput measurement that can accurately represent quality becomes possible.

(2) Since it becomes easier to screen for abnormal values that hinder network performance estimation, more precise network performance estimation becomes possible.

It is a functional block diagram showing the composition of the principal part of the throughput measurement system concerning one embodiment of the present invention. It is a histogram of a throughput distribution. It is the cumulative relative frequency of the throughput distribution. A diagram representatively showing an example in which the transmission source from which a communication flow is extracted in the same area (marked with a circle in the figure) and the transmission source of a screened communication flow change according to time and parameters of statistical processing (part 1) ). A diagram representatively showing an example in which the transmission source from which the communication flow is extracted in the same area (marked with a circle in the figure) and the transmission source of the screened communication flow change according to the time and the parameter of the statistical processing (part 2). ).

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a functional block diagram showing a configuration of a main part of a throughput measuring system according to an embodiment of the present invention, and a configuration unnecessary for the description of the present invention is omitted here.

The present invention is intended for end-to-end measurement of network performance, and in particular, throughput measurement of a communication flow in which a mobile terminal MN relays a wireless base station BS to access a server or an OTT service via a network NW. It is targeted. The transmission source is not limited to the mobile terminal wirelessly connected to the network, and may be a fixed terminal wiredly connected.

Such a throughput measurement system can be configured by mounting an application (program) that realizes each function described below on a general-purpose computer or server. Alternatively, it may be configured as a dedicated machine or a single-purpose machine in which a part of the application is implemented as hardware or ROM.

In the throughput measurement system 1, the average throughput calculation unit 101 acquires an observed throughput distribution from a DPI device or a communication relay device (hereinafter, represented by the DPI device 2) and stores it in the communication flow database (DB) 3. From the throughput determined by the DPI device 2, an average throughput suitable for network performance evaluation is calculated.

In this embodiment, a source IP address, a source port number, a base station identifier, and the like are acquired as source information for each communication flow from the DPI device 2, and destination IP address, destination port number, and destination are used as destination information. Get domain etc. Further, the throughput is acquired as the quality information, and further, the communication characteristic amount such as the communication amount and the packet number is acquired as necessary.

In the IP network represented by the Internet, the TCP throughput can be observed in the DPI device 2, but even the same TCP connection has a low throughput immediately after the connection is established. Therefore, when measuring network performance, it is desirable to observe and use the peak value of TCP throughput for evaluation.

However, if the DPI device 2 does not have the function of detecting the peak value, the average throughput of the TCP connection that is estimated to have reached the sufficient communication speed may be used instead. In this embodiment, for example, the communication start time TCP_BEGIN, the end time TCP_END, the communication amount TCP_VOL, and the number of constituent packets TCP_PKTS of the TCP connection are acquired from the DPI device 2. Then, using a separately set packet size threshold TH_PKT_SIZE, TCP_VOL/(TCP_END-TCP_BEGIN)*8 is adopted as an effective average throughput for TCP connections that satisfy TCP_VOL/TCP_PKTS>TH_PKT_SIZE.

This is because it is assumed that a TCP connection with a sufficiently large average packet size has performed continuous communication, and thus such average throughput can be estimated as a throughput closer to the peak value.

The upstream estimated throughput close to the peak value and the downlink estimated throughput close to the peak value calculated as described above are, as shown in FIG. 1, source IP address, destination IP address, source port, destination port, It is stored in the database 3 together with the upstream traffic of TCP connection, the downstream traffic of TCP connection, the number of upstream packets of TCP connection, the number of downstream packets of TCP connection, and the FQDN or domain estimation result by DNS.

The average throughput is used as calculation source data for estimating the terminal and network performance, and is also used for calculating a statistical value for discarding data that hinders these estimations.

The communication path is information such as a relay device and a VLAN, which is inferred from the point of observation of the communication. In the case of LTE communication, it is node information of SGW and PGW. The server can be identified by the destination IP address, for example.

The OTT service can be identified by domain estimation using DNS communication disclosed in JP-A-2014-230139. In this case, not only a specific server but also a service configured on a CDN (Contents Delivery Network) can be identified as the same service if the service domains are the same. The origination position can be identified from the geolocation of the source IP address, WHOIS registration information, and the information of the base station to which the communication originating user connects.

The classifying unit 102 classifies the records of the respective communication flows accumulated in the database 3 based on the transmission source clause and/or the transmission destination information set in advance or specified separately.

In the classification based on the transmission source information, each communication flow is classified based on the transmission source IP address, the transmission source port number, the accommodating base station identifier, and the like. In the classification based on source information, each communication flow is classified based on its destination IP address, destination port number, server identifier, OTT service identifier, and the like.

The screening unit 103 performs statistical processing using the throughput as an index for each communication flow group of the classification destination, and excludes all the communication flows of the classification destination that is not suitable for throughput measurement.

For example, if the communication flows are classified by the communication flow group that uses each OTT service, as shown in FIG. 2, the throughput distribution is obtained for each communication flow group of the classification destination corresponding to each OTT service, and the histogram is obtained. Turn into. As a screening condition designated in advance, for example, as shown in FIG. 3, an OTT service in which the 80% tile value of the throughput satisfies 60 Mbps, in other words, the OTT whose throughput of the upper 20% of communication flows is 60 Mbps or more. Only the communication flow groups of the classification destinations that can provide the service are selected, and the communication flow groups of the other classification destinations are excluded.

Similarly, when the communication services are classified by the destination Web server, the throughput distribution is obtained for each communication flow group of the classification destination corresponding to each Web server and is histogrammed. Then, only the communication flow groups of the classification destinations regarding the Web server that satisfies the screening condition designated in advance are selected, and the communication flow groups of the other classification destinations are all excluded.

It should be noted that a classification destination whose number of classified communication flows does not reach the reference number has low throughput reliability and may be excluded from the selection target.

The throughput of the communication flow group after being screened in this way can accurately represent the network performance without depending on the quality of the OTT service, the contract content, the capacity of the Web server, or the congestion state.

4 and 5 show the positions and numbers of transmission sources (marked with circles in the figure) from which communication flows are extracted in the same area, and the positions and numbers of transmission sources (marked with x in the figures) of screened communication flows as time. It is the figure which represented as an example an example which changes according to the parameter of statistical processing. As shown in the figure, the transmission source that can be the target of throughput has changed greatly, and the importance of the selection can be understood.

The throughput measuring unit 104 selects a communication flow group of interest from the screened communication flow group based on preset or separately specified transmission source information and/or transmission destination information, and Evaluate network performance based on the throughput of flows.

For example, if the network performance with a certain area as the transmission source is to be evaluated, the above-mentioned screened communication flow group is extracted from the database 3. Then, the communication flow group whose source is the area is selected based on the source IP address or the base station identifier, and the maximum value of the throughput is extracted from the selected communication flow group. It can be used as an index value of network performance related to the area.

In addition, if the user's perceived performance of communication quality deterioration is estimated with respect to the area, if the minimum value of the throughput is extracted from the communication flow group having the area as the transmission source, this can be used as the evaluation value. ..

When extracting the minimum value of the throughput in order to estimate the perceived performance of the user who has deteriorated communication quality, there is a risk that some communication with extremely low throughput may be dragged and the throughput lower than the actual perceived quality may be estimated. is there. Therefore, if such a possibility cannot be abandoned, it is desirable to make a correction by a statistical method.

For example, the observation throughput may be aggregated for each base station to create a histogram, and the qth percentile may be calculated and used as an estimated value. In this way, the throughput value corresponding to q% (for example, about 5%) can be minimized from the slowest one, which means that the throughput observed due to the abnormal value is actively rejected.

According to the present invention, in end-to-end network performance measurement, a communication slowdown that deteriorates performance due to a factor unrelated to network performance can be eliminated, so that accurate throughput measurement is possible.

Note that the classification criteria for each communication flow is not limited to the above, and as transmission source information, attention may be paid to APN (Access Point Name), model, OS, and version thereof, and these may be classified as indicators. .. The model, OS, and version can be used if the communication carrier is managed, but if not managed, it can be estimated from the HTTP User Agent.

Further, in the above embodiment, the attribute information of the transmission source and the transmission destination is described by focusing on various communication parameters such as an address and a port number, but the present invention is not limited to this, and the date, It may be classified by adding calendar information such as a day of the week, or temporal attribute information such as a season, a time, and a time zone.

1... Throughput measurement system, 2... DPI device, 3... Communication flow database, 101... Average throughput calculation unit, 102... Classification unit, 103... Screening unit, 104... Throughput measurement unit

Claims (10)

Based on the communication flow from any source to the destination, in the throughput measurement device that measures the throughput of the communication of interest,
Means for acquiring source information, destination information and throughput for each communication flow,
Means for classifying each communication flow based on source information and/or destination information;
A means for screening the communication flow of each classification destination based on the statistical value of the throughput of the classification destination,
A means for evaluating network performance based on the throughput of the screened communication flow corresponding to the communication of interest,
The means for acquiring the throughput acquires the throughput of the TCP connection in which the packet size obtained by dividing the communication traffic of the TCP connection in the predetermined period by the number of constituent packets in the predetermined period exceeds a predetermined threshold value. .. The throughput measuring apparatus according to claim 1, wherein the classifying unit classifies each communication flow based on a source IP address and/or a source port number when classifying each communication flow based on source information. apparatus. 3. The throughput measurement according to claim 1, wherein the classifying unit classifies each communication flow based on destination IP address and/or destination port number when classifying each communication flow based on destination information. apparatus. 4. The throughput measuring device according to claim 1, wherein the classifying unit classifies each communication flow based on a destination of the OTT service when classifying each communication flow based on destination information. .. 4. The throughput measuring apparatus according to claim 1, wherein the classifying unit classifies each communication flow based on destination server address when classifying each communication flow based on destination information. 4. The throughput measuring device according to claim 1, wherein the classifying unit classifies each communication flow based on destination domain name when classifying each communication flow based on destination information. 7. The throughput measuring device according to claim 1, wherein the communication flow is a TCP session. From the means for acquiring, the communication start time, end time, communication volume, and number of constituent packets of the TCP connection are acquired as the communication feature quantity, the packet size transmitted and received by the TCP connection is calculated, and the packet size is a predetermined threshold value. Further comprising means for determining the average throughput of TCP connections exceeding
The throughput measuring device according to claim 7, wherein the average throughput represents the throughput of the TCP connection. Based on the communication flow from any source to the destination, in the throughput measurement method in which the computer measures the throughput of the communication of interest,
Acquire source information, destination information, and throughput for each communication flow,
Classify each communication flow based on source information and/or destination information,
Screen the communication flow of each classification destination based on the statistical value of the throughput of the classification destination,
Evaluating network performance based on the throughput of the screened communication flow corresponding to the communication of interest,
When acquiring the throughput, a throughput measuring method characterized by acquiring the throughput of the TCP connection in which the packet size obtained by dividing the communication traffic of the TCP connection in the predetermined period by the number of constituent packets in the predetermined period exceeds a predetermined threshold value. .. In the throughput measurement program that measures the throughput of the communication of interest based on the communication flow from any source to the destination,
A procedure for acquiring source information, destination information, and throughput for each communication flow,
A procedure for classifying each communication flow based on source information and/or destination information,
A procedure for screening the communication flow of each classification destination based on the statistical value of the throughput of the classification destination,
Causing a computer to execute a procedure of evaluating network performance based on the throughput of the screened communication flow corresponding to the communication of interest,
In the procedure for acquiring the throughput, a throughput measuring program for acquiring the throughput of the TCP connection in which the packet size obtained by dividing the communication traffic of the TCP connection in the predetermined period by the number of constituent packets in the predetermined period exceeds a predetermined threshold.