JP2013157931A - Transmission source/destination organization specification device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To associate an organization with an IP address included in flow data to grasp a traffic state for each site.SOLUTION: In the present invention, DNS traffic data is collected, a destination IP address and flow data including the destination IP address, number of packets, flow size, and flow acquisition start time are collected, a DNS information database is created in which an inquiry transmission source domain name included in the DNS traffic data is determined as a first element, an IP address corresponding to the domain name is determined as a second element, an inquiry transmission source IP address is determined as a third element, inquiry time is determined as a fourth element, and the number of inquiry for the domain name is determined as a fifth element, and a flow database is generated on the basis of the acquired flow data. The DNS information database and the flow database are compared with each other, and a domain name is given to the destination IP address which appears in the flow database.

Description

  The present invention relates to a source / destination organization specifying apparatus, method, and program, and more particularly to specifying a source and destination organization of a certain traffic by associating DNS (Domain Name Service) traffic with flow data. The present invention relates to a transmission source / destination organization identification apparatus, method, and program.

・ Technology for understanding traffic conditions:
As a person who grasps the traffic situation, there is a person who uses NetFlow that can be collected by a device such as a router (for example, see Non-Patent Document 1). In this method, data flowing on a network is handled as a set of data called a flow that is recognized by a combination of a destination, a transmission source, a port number, and the like. By using this technology, it is possible to grasp how much traffic is flowing from which IP address to which IP address. However, NetFlow is a technology that can acquire only limited information such as IP address, port number, number of packets, and flow size. In order to collect all information, a method of collecting data in packet units using tcpdump or the like (for example, refer to Non-Patent Document 2) or a method of collecting a flow including application layer information using sflow (for example, Non-Patent Document 3).

・ Expand CDN (Contents Delivery Network) business:
As a recent Internet service, the expansion of the CDN business is remarkable. CDN is a mechanism for efficiently distributing content such as websites, music, and videos. The content of an organization that uses a CDN is placed on a server on the CDN, not the server that the organization owns and operates. The server may be owned by a CDN operator or may borrow a server from another operator.

・ Issues to grasp the traffic situation:
Due to the expansion of the above CDN business, it is becoming difficult to grasp the correct traffic situation using flow data. This is because content of various organizations is placed on the server on the CDN. For example, assume that the websites of organizations A and B use a CDN and are located on the same distribution server owned by the CDN operator. In this case, when a user browses the websites of organizations A and B, the destination IP address is the server of the CDN operator. Since the destination information included in the flow data is an IP address, it is impossible to determine whether the traffic is to the organization A or B even if only the flow data is viewed. An example of this is shown in FIG. In the example shown in FIG. 1, when both company A and company B have Web contents on the server (10.0.0.1), the same "10.0" Since you are accessing a server on a .0.1 "CDN, you can't tell which site you browsed by looking at just the IP address (192.168.0.1) of the user terminal.

  Also, when there is a distribution server in a certain ISP (Internet Service Provider), it cannot be determined whether the destination IP address is the ISP provider's server or the CDN provider's distribution server. As described above, the problem is that the IP address and the organization are not one-to-one, making it difficult to accurately grasp the traffic situation.

・ Related technologies:
There is research to clarify the traffic flowing on the Internet and the structure of the Internet (see Non-Patent Document 4, for example). These studies analyze the traffic volume between ASs and the Internet structure by analyzing flow data. In addition, there is a technique for analyzing the network structure of a Web content distributor using DNS data and route information (see Non-Patent Document 5, for example).

B. Claise, "Cisco Systems NetFlow Services Export Version9," IETF RFC3954, October 2004. http://www.ietf.org/rfc/rfc3954 TCPDUMP / LIBPCAP public repository, "http://www.tcpdump.org". P. Phaal, S. Panchen, and N. McKee, "InMon Corporation's sFlow: A Method for Monitoring Traffic in Switched and Routed Networks," IETF RFC3176, September 2001.http: //www.ietf.org/rfc/rfc3176 C. Labovitz, S. Iekel-Johnson, J. Oberheide, and F. Jahanian, "Internet Inter-Domain Traffic," In Proceedings of ACM SIGCOMM '10. B. Ager, W. Muhlbauer, G. Smaragdakis, and S. Uhlig, "Web Content Cartography," In Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference.

  However, in Non-Patent Document 4, as described above, it is difficult to accurately grasp the traffic flow and status of finer traffic using only data of the IP layer such as flow data. Further, since DNS data is used in Non-Patent Document 5, analysis at a fine granularity is performed, but it is difficult to grasp the traffic flow rate only by route information.

  The present invention has been made in view of the above points, and relates to a transmission source / destination organization identification apparatus, method, and program capable of associating an IP address included in flow data with an organization and grasping a traffic situation by site. The purpose is to provide.

In order to solve the above problems, the present invention (Claim 1) is a transmission source / destination organization specifying device for specifying a source and destination organization of a certain traffic by associating DNS traffic data and flow data. There,
DNS data collection means for collecting the DNS traffic data;
A flow data collection means for collecting the flow data including a destination IP address, a destination IP address, the number of packets, a flow size, and a flow acquisition start time;
The inquiry source domain name included in the DNS traffic data is the first element, the IP address corresponding to the domain name is the second element, the inquiry source IP address is the third element, the inquiry time is the fourth element, and the domain name Creating a DNS information database having the number of inquiries as a fifth element, and generating a flow database based on the acquired flow data;
Matching means for matching the DNS information database with the flow database and assigning a domain name to a destination IP address appearing in the flow database.

Moreover, the present invention (Claim 2) is the above-mentioned matching means,
The destination IP address of the flow database is the second element of the DNS information database, the source IP address of the flow database is the third element of the DNS information database, and the flow acquisition start time of the flow database is the flow acquisition start time Searching for a record that matches the fourth element of the DNS information database, and assigning the first element to the destination IP address;
When the first search means cannot find a matching record, the destination IP address of the flow database matches the second element of the DNS information database, and the source IP address of the flow database Searching for a record that matches the third element of the DNS information database, and assigning a domain name that is the first element of the record to the destination IP address;
When the matching record cannot be searched in the first search means and the second search means, the record in which the destination IP address of the flow database matches the second element of the DNS information database is searched. , Third search means for assigning the domain name as the first element of the record to the destination IP address;
When a matching record cannot be searched in the first search means, the second search means, and the third search means, the network unit of the destination IP address of the flow database is the DNS information database. And fourth search means for searching for a record that matches the network part of the second element and assigning the domain name that is the first element of the record to the destination IP address.

Further, the present invention (Claim 3) is the database generating means,
In the CNAME information included in the DNS traffic data response, a CNAME database is created having a record in which the alias given by the CNAME is the first element, the original domain name is the second element, and the number of occurrences of the CNAME is the third element. Including CNAME database generation means to
In the matching means,
When assigning a domain name to the destination IP address, a record in which the domain name matches the first element of the CNAME database is searched, and if an alias exists, the alias is assigned to the destination IP address. Including means.

Moreover, this invention (Claim 4) is the said matching means,
In the case where a plurality of records are obtained by the search, a means for apportioning the number of packets and the flow size included in the flow database according to the number of times of inquiries for each domain name is included.

  As described above, according to the present invention, it is possible to use a relatively lightweight and collectable data such as DNS or a flow to identify which traffic is sent to which organization and from which organization. Therefore, it is possible to grasp the exact traffic situation. By accurately grasping the traffic situation, it is possible to identify the cause of a sudden change in traffic, and to use it as basic data for communication network equipment design. Highly effective.

It is a conventional system example. It is a system configuration figure in a 1st embodiment of the present invention. It is an example of DNS information DB in the 1st Embodiment of this invention. It is an example of flow DB in the 1st Embodiment of this invention. It is a figure which shows the process of the matching part in the 1st Embodiment of this invention. It is a figure which shows the process of the matching part in the 2nd Embodiment of this invention. It is FIG. (1) which shows the process of the matching part in the 3rd Embodiment of this invention. It is FIG. (2) which shows the process of the matching part in the 3rd Embodiment of this invention. It is a block diagram of the transmission origin and destination organization identification device in the 4th Embodiment of this invention. It is an example of CNAME DB in the 4th Embodiment of this invention. It is a figure which shows the process of the matching part in the 4th Embodiment of this invention. It is a flowchart of the matching part in the 4th Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Prerequisites>
First, the preconditions of the present invention will be described.

  As input of the present invention, it is assumed that traffic data reaching the DNS cache server and flow data collected using NetFlow, which is a protocol for monitoring traffic flow information passing through the network, are given. DNS traffic data includes query domain name, response to the query, query source IP address, query time, etc. Flow data includes source IP address, destination IP address, number of packets, traffic volume, flow acquisition start Information such as time is included. Further, the collection point of DNS traffic data and flow data may not be the same NW.

[First Embodiment]
<Outline of the present embodiment>
In the present invention, first, a DNS information database including information such as an IP address and a domain name is generated from DNS data. Next, the database generated from the DNS data is searched for the IP address appearing in the flow data, and the corresponding domain name is linked.

<Configuration>
FIG. 2 shows a system configuration in the first embodiment of the present invention.

  The system shown in the figure includes a user terminal 1, a communication destination server 2, a DNS cache server 3, a router 4, an L2SW 5 that is a switch that does not have an IP address, and a transmission source / destination organization specifying apparatus 100.

  The source / destination organization identification device 100 includes a flow data collection unit 110 that collects flow data, a DNS data collection unit 120 that collects DSN traffic data, a DNS information database that inquires the DNS server 3 from the DNS traffic data and includes response information (DB) 150, a database generation unit 130 that generates a flow database (DB) 140, a flow DB 140 and a DNS information DB 150, a matching unit 160 that links an organization and traffic, and an output unit 170 that outputs a matching result Consists of

  As an input of the present invention, it is assumed that traffic data reaching the DNS cache server 3 is given to the DNS data collection unit 120 and flow data collected using NetFlow is given to the flow data collection unit 110 from the router 4. DNS traffic data includes query domain name, response to the query, query source IP address, query time, etc. Flow data includes source IP address, destination IP address, number of packets, traffic volume, flow acquisition start Information such as time is included. Note that the location where DNS traffic data and flow data are collected need not be the same network.

<Generation of DNS information database>
The DNS data given from the DNS data collection unit 120 includes an inquiry transmission source, time information, and response information. The response information includes which domain name is queried and what is the IP address corresponding to the domain name. Using this information, the database generation unit 130 uses the domain name as shown in FIG. 3 as the first element, the IP address corresponding to the domain name as the second element, the inquired source IP address as the third element, and the inquiry. A DNS information DB 150 is generated with time information as the fourth element and the number of inquiries for the domain name as the fifth element.

<Create a flow database>
When the source, destination IP address, number of packets, flow size (flow rate), acquisition start time of the flow, and the like are input from the flow data collection unit 110 as the flow data, the database generation unit 130, as shown in FIG. In addition, the flow DB 140 is generated with the source IP address as the first element, the destination IP address as the second element, the flow acquisition start time as the third element, the number of packets as the fourth element, and the flow rate as the fifth element.

<Matching IP address and domain name included in flow data>
The flow data includes a transmission source, a destination IP address, the number of packets, a flow size (flow rate), an acquisition start time of the flow, and the like. As shown in FIG. 5, in the matching unit 160, the destination IP address included in the flow data of the flow DB 140 is the second element (IP address corresponding to the domain name) of the DNS information DB 150, and the source IP address is the DNS information DB 150. The third element (inquiry source) and the record whose flow acquisition start time matches the fourth element (inquiry time) of the DNS information DB 150 are searched. Since the flow acquisition start time and the fourth element of the DNS information DB 150 may not be exactly the same time, it is conceivable to allow a difference within a certain time instead of a perfect match. Next, the domain name, which is the first element of the search result record, is assigned to the destination IP address. As shown in FIG. 5, the matching result is a destination domain name, the number of packets, and a flow rate.

  When a plurality of records are obtained as a search result, the record with the latest inquiry time is used.

[Second Embodiment]
<Database search issues>
Recent OS and Web browser implementations have the ability to save DNS information on the terminal. For this reason, the transmission source terminal does not always communicate with the DNS server when browsing a Web site, and there are cases in which it directly accesses the Web server. In such a case, there is a problem that the search of the time information in the DNS information database or the inquiry source IP address fails. In addition, there is a case where the search of the flow data transmission source fails when the collection points of the DNS data and the flow data are different. Another problem is that the collected DNS traffic data does not contain all existing domain names and corresponding IP address information. In such a case, the search for the destination IP address included in the flow data and the second element of the DNS information DB 150 will fail. The following describes how to deal with this problem.

<Matching method using past inquiry information>
In this embodiment, as a process when the search fails, the past DNS inquiry information of the transmission source or the DNS inquiry information of a different transmission source is used. Specifically, as shown in FIG. 6, a record in which the destination IP address included in the flow data of the flow DB 140 matches the second element of the DNS information DB 150 and the source IP address matches the third element of the DNS information DB 150. And the domain name that is the first element of the record is assigned to the destination IP address.

[Third Embodiment]
In the present embodiment, a second method for solving the problem that a search using a transmission source fails when the above-described DNS data and flow data collection locations are different will be described.

<Matching method using inquiry information of other users>
In the present embodiment, DNS inquiry information of other users is used. Specifically, as shown in FIG. 7, a record in which the destination IP address included in the flow data of the flow DB 140 matches the second element of the DNS information DB 150 is searched, and the domain name that is the first element of the record is determined. Assigned to the destination IP address.

  However, if all domain names and IP address information are not available, the search using the destination IP address will fail. As a countermeasure in this case, the network part of the destination IP address and the IP address of the second element of the DNS information DB 150 is used. This is based on the fact that IP addresses in the vicinity of a certain IP address are often of the same organization. Specifically, as shown in FIG. 8, a record in which the upper n bits of the destination IP address matches the upper n bits of the IP address of the second element of the DNS information database is searched and is the first element of the record. Give the domain name to the destination IP address. Alternatively, a record in which the destination IP address included in the flow data matches the second element of the DNS information DB 150 is searched, and the domain name that is the first element of the record is assigned to the destination IP address.

  In the matching method shown in FIGS. 7 and 8, there are cases where a plurality of records are obtained as a search result of the DNS information database. In this embodiment, a coping method at this time will be described.

  In the case of FIG. 7 and FIG. 8, since inquiry information of other users is used, it is difficult to determine which domain name communicated. Therefore, the flow is divided into a plurality of domain names obtained from the inquiry tendency of all users. Specifically, if the domain names obtained by the search are A, B, and C, and the number of inquiries is 10, 40, and 50, A is the number of packets in the flow and 10% of the flow size is B. Will be allocated 40% and C will be allocated 50%. When A is an alias for domain names A ′ and A ″, the allocated 10% packet number and flow size are prorated according to the number of appearances of CNAME.

[Fourth embodiment]
When assigning a domain name to an IP address according to the first to third embodiments described above, there is a case where a correct domain name is not assigned by CNAME. For example, when the CNAME of “www.example.com” is “www2.example.com”, the following information is stored as a cache.
1. The alias for www.example.com is www2.example.com
2. The IP address of www2.example.com is xxx.xxx.xxx.xxx
In the above 1 and 2, a cache retention period is set, but there are cases where the retention periods are not the same. For example, if the retention period of 2 is set to be shorter than the retention period of 1, when the retention period of 2 has passed and the terminal resolves the name "www.example.com", the information of 1 is stored as a cache Therefore, the terminal performs name resolution of “www2.example.com” instead of www.example.com. At this time, “www2.example.com” is recorded in the DNS information database instead of “www.example.com”. As a result, in the above matching method, “www2.example.com” is erroneously assigned to the IP address instead of “www.example.com”.

  In this embodiment, in order to solve this problem, the CNAME database is used to restore the original domain name.

  FIG. 9 shows a configuration of a transmission source / destination organization specifying apparatus according to the fourth embodiment of the present invention. The apparatus shown in the figure has a configuration in which a CNAME database 155 is added to the configuration shown in FIG.

  The database generation unit 130 generates CNAMEDB 155 by the following method.

  In the information included in the DNS response, there is a resource record for defining a domain name alias called CNAME. For example, when a query is issued to the DNS server to obtain the IP address of “www.example.co.jp”, the response includes the following CNAME record.

www.example.co.jp.129909 IN CNAME www.example.com.
www.example.com. 387328 IN CNAME www.2. example.com.
This means that the alias for "www.example.co.jp" is "www.example.com" and the alias for "www.example.com" is "www2.example.com" . In this embodiment, the domain name on the left of CNAME is defined as the original domain name, and the domain name on the right is defined as the alias. The database generation unit 130 generates the CNAME DB 155 from the DNS response information with the first element as shown in FIG. 10 as the original domain name, the second element as the alias, and the third element as the domain name and the number of occurrences of the alias.

  As shown in FIG. 11, the matching unit 160 in the present embodiment searches for a record in which the domain name obtained by the search matches the first element of the CNAME DB 155, and further obtains the second element obtained from the CNAME DB 155. Check recursively whether it matches one element. Search until no matching records exist, and assign the domain name to the IP address.

  FIG. 12 shows a flowchart of the operation in the present embodiment.

  First, when the source IP address (s), destination IP address (d), start time (t), number of packets (pc), and flow size (fs) are input to the matching unit 160 (step 101), DNS It is determined whether or not s, d, and t exist in the information DB 150 (step 102). If they exist, a set of the domain name (f) and the number of queries (qc) is output (step 106). If it does not exist, it is determined whether or not s and d exist in the DNS information DB 150 (step 103). If it does not exist, it is determined whether d exists in the DNS information DB 150. If it exists, the processing of step 106 is performed. If it does not exist, it is determined whether or not the network unit NW (d) of the destination IP address d exists in the DNS information DB 150. For example, when d = 192.168.0.1, it is determined whether NW (d) = 192.168.0.0 / 24 or 192.168.0.0/16 exists. If it exists, the process of step 106 is performed, and if it does not exist, the process ends (step 105).

In step 106, after outputting (f, qc), it is determined whether f exists in the CNAME DB 155 (step 107), and if present, the original domain name (q) and the appearance function (cc) of CNAME are output. (Step 108), (q, pc, cc) and (q, ps, cc) are output as functions that apportion the number of packets and the flow size to the domain name (Step 109). If f does not exist in the CNAME DB 155 (No at Step 107), (f, pc, qc) and (f, ps, qc) are output as functions that apportion the number of packets and the flow size to the domain name (Step 110). ).
The operations of the flow data collection unit 110, the DNS data collection unit 120, the database generation unit 130, the matching unit 160, and the output unit 170 of the transmission source / destination organization identification apparatus illustrated in FIGS. Can be constructed as a program and installed and executed on a computer used as a transmission source / destination organization specifying apparatus, or distributed via a network.

1 User terminal 2 Destination server 3 DNS server 4 Router 5 L2SW
100 Source / destination organization identification device 110 Flow data collection unit 120 DNS data collection unit 130 Database generation unit 140 Flow database (DB)
150 DNS information database (DB)
155 CNAME database (DB)
160 matching unit 170 output unit

Claims (8)

A source / destination organization identification device that identifies a source and destination organization of a certain traffic by associating DNS (Domain Name Service) traffic data with flow data,
DNS data collection means for collecting the DNS traffic data;
A flow data collection means for collecting the flow data including a destination IP address, a destination IP address, the number of packets, a flow size, and a flow acquisition start time;
The inquiry source domain name included in the DNS traffic data is the first element, the IP address corresponding to the domain name is the second element, the inquiry source IP address is the third element, the inquiry time is the fourth element, and the domain name Creating a DNS information database having the number of inquiries as a fifth element, and generating a flow database based on the acquired flow data;
Matching means for matching the DNS information database and the flow database, and assigning a domain name to a destination IP address appearing in the flow database;
A transmission source / destination organization identification device characterized by comprising: The matching means includes
The destination IP address of the flow database is the second element of the DNS information database, the source IP address of the flow database is the third element of the DNS information database, and the flow acquisition start time of the flow database is the flow acquisition start time Searching for a record that matches the fourth element of the DNS information database, and assigning the first element to the destination IP address;
When the first search means cannot find a matching record, the destination IP address of the flow database matches the second element of the DNS information database, and the source IP address of the flow database Searching for a record that matches the third element of the DNS information database, and assigning a domain name that is the first element of the record to the destination IP address;
When the matching record cannot be searched in the first search means and the second search means, the record in which the destination IP address of the flow database matches the second element of the DNS information database is searched. , Third search means for assigning the domain name as the first element of the record to the destination IP address;
When a matching record cannot be searched in the first search means, the second search means, and the third search means, the network unit of the destination IP address of the flow database is the DNS information database. A fourth search means for searching for a record that matches the network part of the second element, and assigning the domain name that is the first element of the record to the destination IP address;
The transmission source / destination organization specifying apparatus according to claim 1. The database generation means includes
In the CNAME information included in the DNS traffic data response, a CNAME database is created having a record in which the alias given by the CNAME is the first element, the original domain name is the second element, and the number of occurrences of the CNAME is the third element. Including CNAME database generation means to
The matching means includes
When assigning a domain name to the destination IP address, a record in which the domain name matches the first element of the CNAME database is searched, and if an alias exists, the alias is assigned to the destination IP address. The transmission source / destination organization specifying apparatus according to claim 1, further comprising means. The matching means includes
The method according to any one of claims 1 to 3, further comprising means for apportioning the number of packets included in the flow database and the flow size according to the number of inquiries for each domain name when a plurality of records are obtained by the search. The source / destination organization identification device described. A source / destination organization identification method for identifying a source and destination organization of a certain traffic by associating DNS traffic data with flow data,
DNS data collection means for collecting the DNS traffic data by a DNS data collection means;
A flow data collecting step for collecting the flow data including a destination IP address, a destination IP address, the number of packets, a flow size, and a flow acquisition start time;
The database generation means includes a query transmission source domain name included in the DNS traffic data as a first element, an IP address corresponding to the domain name as a second element, a query transmission source IP address as a third element, and a query time as a fourth element. A database generation step of creating a DNS information database having the element, the domain name inquiry count as a fifth element, and generating a flow database based on the acquired flow data;
A matching step of matching the DNS information database with the flow database and assigning a domain name to a destination IP address appearing in the flow database;
A source / destination organization specifying method characterized by: In the matching step,
The destination IP address of the flow database is the second element of the DNS information database, the source IP address of the flow database is the third element of the DNS information database, and the flow acquisition start time of the flow database is the flow acquisition start time Searching for a record that matches the fourth element of the DNS information database, and assigning the first element to the destination IP address;
When the matching record could not be searched in the first search step, the destination IP address of the flow database matches the second element of the DNS information database, and the source IP address of the flow database Searching for a record that matches the third element of the DNS information database, and assigning a domain name that is the first element of the record to the destination IP address;
When a record that matches in the first search step and the second search step cannot be searched, the record in which the destination IP address of the flow database matches the second element of the DNS information database is searched. , A third search step of assigning the domain name that is the first element of the record to the destination IP address;
When the matching record cannot be searched in the first search step, the second search step, and the third search step, the network part of the destination IP address of the flow database is stored in the DNS information database. A fourth search step of searching for a record that matches the network part of the second element, and assigning the domain name that is the first element of the record to the destination IP address;
The transmission source / destination organization specifying method according to claim 5. In the database generation step,
In the CNAME information included in the DNS traffic data response, a CNAME database is created having a record in which the alias given by the CNAME is the first element, the original domain name is the second element, and the number of occurrences of the CNAME is the third element. Including CNAME database generation step to
In the matching step,
When assigning a domain name to the destination IP address, a record in which the domain name matches the first element of the CNAME database is searched, and if an alias exists, the alias is assigned to the destination IP address. 6. The transmission source / destination organization identification method according to claim 5, further comprising a step. Computer
A transmission source / destination organization specifying program for causing each of the means of the transmission source / destination organization specifying device according to any one of claims 1 to 4 to function.

Images