JP7435744B2 - Identification method, identification device and identification program - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act IEICE Technical Report on Information and Communication Management (ICM) vol. 119 No. 438 ICM2019-51 pp. 55-60 Publication date February 24, 2020

The present invention relates to an identification method, an identification device, and an identification program.

When creating a classifier using supervised learning for application identification, a large amount of data and labels corresponding to each data point are required. Conventionally, there are techniques for adding labels to flow data using packet data and techniques for extracting features using packet data.

T. Karagiannis, K. Papagiannaki and M. Faloutsos, “BLINC: Multilevel Traffic Classification in the Dark”, Proceedings of the ACM SIGCOMM 2005 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, Philadelphia, Pennsylvania, USA, August 22-26, 2005 Z. Chen, K. He, J. Li and Y. Geng “Seq2Img: A Sequence-to-Image based Approach Towards IP Traffic Classification using Convolutional Neural Networks”, 2017 IEEE International Conference on Big Data (Big Data).

However, when using flow data to add application-level labels, there is a problem in that it is difficult to add labels and the accuracy is low because the flow data includes only simple information such as an IP address and a port number. Furthermore, when using packet data, the larger the target network, the higher the load on collection and analysis, making it difficult to add labels at the application level, making it difficult to apply to large-scale networks.

The present invention has been made in view of the above, and an object of the present invention is to provide an identification method, an identification device, and an identification program that can appropriately identify an application that generates traffic even in a large-scale network. shall be.

In order to solve the above-mentioned problems and achieve the purpose, an identification method according to the present invention is an identification method executed by an identification device that identifies an application, and includes packet data and first flow data that satisfy a predetermined rule. a signature generation step that analyzes the packet data and generates a signature that associates the application with the IP address; a flow data generation step that generates second flow data from the packet data; Calculation of calculating first feature amount information, which is a statistical feature amount for each IP address, for the data, and calculating second feature amount information, which is a statistical feature amount for each IP address, for the second flow data. a step of adding a label to the second feature information using the signature; and a learning step of causing the discriminator to learn application identification using the first feature information and the second feature information as learning data. It is characterized by including a process.

Further, the identification device according to the present invention is an identification device that identifies an application, and includes a collection unit that collects packet data and first flow data that satisfy a predetermined rule, and an application and IP address that analyze the packet data. a flow data generation unit that generates second flow data from packet data; and a first feature that is a statistical feature amount for each IP address for the first flow data. a feature amount calculation unit that calculates second feature amount information that is a statistical feature amount for each IP address for the second flow data; The present invention is characterized by comprising a label adding unit that adds a label, and a learning unit that causes a discriminator to learn application identification using the first feature information and the second feature information as learning data.

The identification program according to the present invention also includes a collection step of collecting packet data and first flow data that satisfy a predetermined rule, and a first step of analyzing the packet data to generate a signature that associates an application with an IP address. a generation step, a second generation step of generating second flow data from packet data, and calculating first feature information, which is a statistical feature for each IP address, for the first flow data; a calculation step of calculating second feature information, which is a statistical feature for each IP address, for the flow data of No. 2; an addition step of adding a label to the second feature information using a signature; and a discriminator. and a learning step of learning application identification using the first feature information and the second feature information as learning data.

According to the present invention, in a data search including spatio-temporal data, an application that generates traffic can be appropriately identified even in a large-scale network.

FIG. 1 is a block diagram showing an example of the configuration of a communication system in an embodiment. FIG. 2 is a flowchart showing the processing procedure of the learning process according to the embodiment. FIG. 3 is a flowchart showing the processing procedure of the identification process according to the embodiment. FIG. 4 is a diagram illustrating an application example of the identification device according to the embodiment. FIG. 5 is a diagram illustrating another application example of the identification device 10 according to the embodiment. FIG. 6 is a diagram illustrating an example of a computer that implements an identification device by executing a program.

Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are denoted by the same reference numerals.

[Embodiment]
FIG. 1 is a block diagram showing an example of the configuration of a communication system in an embodiment. As shown in FIG. 1, the communication system according to the embodiment includes small-scale network (NW) devices 2A and 2B, NW routers to be identified 3A and 3B, and an identification device 10. The plurality of small-scale NW devices 2A, 2B, the plurality of identification target NW routers 3A, 3B, and the identification device 10 communicate via a network. Although FIG. 1 shows a case in which there are a plurality of small-scale NW devices 2A, 2B and identification target NW routers 3A, 3B, each may be a single number.

The small-scale NW devices 2A and 2B transmit traffic data of the small-scale NW to the identification device 10 by performing traffic mirroring in the small-scale NW. The small-scale NW devices 2A and 2B transmit the small-scale NW packet data D1 to the identification device 10.

The identification target NW routers 3A and 3B are routers provided in the identification target NW of the application, and collect network flow data (flow data) D2 of the identification target NW using a flow collection function or the like in the identification target NW. , to the identification device 10.

The identification device 10 identifies an application (for example, a web application) that has generated traffic from the flow data in the identification target NW. The identification device 10 causes the discriminator to pre-train application identification using labeled learning data generated from data of a small-scale NW, and then uses Domain Adaptation to perform learning on unlabeled flow data of the NW to be identified. use. Thereby, the identification device 10 constructs a classifier that can identify applications even in flow data in a large-scale identification target NW.

[Identification device]
Next, the identification device 10 will be explained with reference to FIG. As shown in FIG. 1, the identification device 10 includes a collection unit 11, a signature generation unit 12, a flow data generation unit 13, a signature database (DB) 14, a feature calculation unit 15, a label addition unit 16, and a classifier learning unit 17. (learning unit), a learned classifier 18, an application identifying unit 19 (identifying unit), and an output unit 20.

Note that the identification device 10 is configured such that a predetermined program is read into a computer or the like including, for example, ROM (Read Only Memory), RAM (Random Access Memory), CPU (Central Processing Unit), etc., and the CPU executes the predetermined program. This is achieved by doing. The identification device 10 also has a communication interface for transmitting and receiving various information with other devices connected via a network or the like. For example, the identification device 10 includes a NIC (Network Interface Card) and the like, and communicates with other devices via a telecommunications line such as a LAN (Local Area Network) or the Internet.

The collection unit 11 collects packet data and flow data that satisfy predetermined rules. During learning, the collection unit 11 collects packet data D1 of the small-scale NW transmitted from the small-scale NW devices 2A and 2B, and flow data of the identification target NW, which is a large-scale NW, transmitted from the identification target NW routers 3A and 3B. D2 (first flow data). The small-scale NW packet data D1 is packet data of a small-scale NW that is of a certain size to which a label can be added through subsequent processing.

Then, during learning, the collection unit 11 outputs the packet data D1 of the small-scale NW to the signature generation unit 12 and the flow data generation unit 13. Furthermore, the collection unit 11 outputs the first flow data to the feature calculation unit 15 during learning. At the time of identification, the collection unit 11 collects flow data of the identification target NW to be identified, and outputs it to the feature value calculation unit 15.

The signature generation unit 12 analyzes the packet data D1 of the small-scale NW and generates a signature that associates applications with IP addresses. The signature generation unit 12 analyzes the packet data collected in the small-scale NW using a DPI device or the like, and generates a label (for example, the name of the application) indicating the application category that generated the packet data, the source IP address, and the transmission source IP address. A signature is created by associating the destination IP address, port number, and time at which the packet was recorded.

The flow data generation unit 13 generates second flow data from the packet data D1 of the small-scale NW.

The signature DB 14 associates and stores a label indicating an application category generated by the signature generation unit 12 and a set of a source IP address, a destination IP address, a port number, and the time at which the packet was recorded. do.

During learning, the feature calculation unit 15 calculates first feature information, which is a statistical feature for each IP address, for the first flow data, which is the flow data D2 of the identification target NW. During learning, the feature calculation unit 15 calculates second feature information, which is a statistical feature for each IP address, for the second flow data generated by the flow data generation unit 13 from the packet data D1 of the small-scale NW. calculate. Further, at the time of identification, the feature calculation unit 15 calculates identification feature information, which is a statistical feature for each IP address, for the flow data of the identification target NW that is the identification target.

The feature calculation unit 15 calculates a histogram of the number of packets, a histogram of the number of bytes, or a histogram of the number of bytes and packets from a set of flow data with a certain IP address as the source and/or destination per 24 hours. Calculate at least one of them. Specifically, the feature calculation unit 15 calculates statistics such as the average number of bytes per packet for each destination IP address and source IP address for the first flow data, and calculates the first feature. Extract as quantity information. The feature calculation unit 15 calculates statistics such as the average number of bytes per packet for each destination IP address and source IP address for the second flow data, and extracts it as second feature information. .

During learning, the label adding unit 16 uses the signature generated by the signature generating unit 12 to add a label to the second feature amount information.

The classifier learning unit 17 causes the classifier to learn application identification using the first feature amount information and the second feature amount information as learning data. The classifier learning unit 17 performs preliminary training of the classifier using the labeled second feature amount information generated by the label adding unit 16. Thereafter, the classifier learning unit 17 uses the first feature amount information and the unlabeled second feature amount information to perform learning of the classifier using the domain application technique. The classifier learning unit 17 performs classifier learning by Domain Adaptation using the classifier obtained through preliminary learning, the first feature information, and the unlabeled second feature information.

The trained discriminator 18 is a discriminator that has become capable of identifying an application corresponding to the IP address of the flow data to be identified through prior learning and learning in the discriminator learning unit 17. Specifically, the learned classifier 18 inputs the feature amount information of the flow data to be identified, and outputs the probability that the IP address of the flow data to be identified provides each application.

The application identifying unit 19 uses the learned classifier 18 to identify an application corresponding to the IP address of the flow data to be identified. At the time of identification, the application identification unit 19 inputs the identification feature information to the trained classifier 18, and based on the classification result output from the trained classifier 18, the application identification unit 19 inputs the identification feature information to the learned classifier 18, and uses the IP address of the flow data to be identified based on the classification result output from the learned classifier 18. Identify the corresponding application. The output unit 20 outputs the identification result obtained by the application identification unit 19 to, for example, an external device.

[Learning process]
Next, a learning process for the classifier executed by the classifier 10 shown in FIG. 1 will be described. FIG. 2 is a flowchart showing the processing procedure of the learning process according to the embodiment.

As shown in FIG. 2, the collection unit 11 performs a collection process to collect packet data D1 of the small-scale NW and flow data D2 (first flow data) of the identified NW (step S1).

Then, the signature generation unit 12 analyzes the packet data D1 of the small-scale NW and generates a signature that associates the application with the IP address (step S2). Furthermore, the flow data generation unit 13 generates second flow data from the packet data D1 of the small-scale NW (step S3).

The feature calculation unit 15 calculates second feature information, which is a statistical feature for each IP address, for the second flow data (step S4). During learning, the label adding unit 16 adds a label to the second feature amount information using the signature generated by the signature generating unit 12 (step S5). The classifier learning unit 17 performs preliminary training of the classifier using the labeled second feature amount information generated by the label adding unit 16 (step S6).

Further, the feature calculation unit 15 calculates first feature information, which is a statistical feature for each IP address, for the first flow data (step S7). The classifier learning unit 17 performs classifier learning by Domain Adaptation using the classifier obtained through preliminary learning, the first feature information, and the unlabeled second feature information (step S8). Then, the classifier learning unit 17 generates a trained classifier 18.

[Identification processing]
Next, an identification process for identifying an application corresponding to an IP address of flow data of an identification target NW, which is executed by the identification device 10 shown in FIG. 1, will be described. FIG. 3 is a flowchart showing the processing procedure of the identification process according to the embodiment.

As shown in FIG. 3, at the time of identification, the collection unit 11 collects flow data of the identification target NW, which is a large-scale NW to be identified (step S11). Subsequently, the feature calculation unit 15 calculates identification feature information, which is a statistical feature for each IP address, for the flow data of the identification target NW (step S12).

The application identifying unit 19 uses the learned classifier 18 to identify the application corresponding to the IP address of the flow data to be identified (step S13). The output unit 20 outputs the identification result obtained by the application identification unit 19 to, for example, an external device (step S14).

[Application example 1]
An application example of the identification device 10 will be described. FIG. 4 is a diagram illustrating an application example of the identification device 10 according to the embodiment.

As shown in FIG. 4, the network flow data collected in the ISP NW is identified by the identification device 10, and as a result of the identification, the probability that the IP address of the flow data of the ISP NW provides each application is visualized. This allows the network administrator to understand the detailed NW situation, and to understand which routes (for example, routes R1 and R2) should be invested heavily. In this way, by applying the identification device 10, it is possible to improve the efficiency of NW monitoring and equipment investment planning by visualizing the traffic of the ISP network.

[Application example 2]
FIG. 5 is a diagram illustrating another application example of the identification device 10 according to the embodiment. As shown in FIG. 5, the identification device 10 is applied when detecting a very small amount of malicious communication included in large-scale traffic data Dt.

Specifically, by performing identification processing in the identification device 10 on large-scale traffic data Dt and excluding normal traffic from the large-scale traffic data Dt in advance, the amount of traffic data Dm to be investigated can be determined. can be reduced. In this way, by applying the identification device 10, screening for malicious communication detection can be performed, and the burden placed on malicious communication detection can be reduced.

[Effects of embodiment]
In this way, the identification device 10 according to the present embodiment causes the classifier to learn using the labeled learning data generated from the data of the small-scale NW, and then uses the domain application technique to learn the unlabeled large-scale The flow data of the identification target NW, which is a large-scale NW, and the data of a small-scale NW without a label are learned.

As a result, the identification device 10 uses Domain Adaptation to use the unlabeled flow data of the identification target NW for learning. In this case, it is possible to construct a classifier that can more accurately identify the data of the NW to be identified.

As described above, according to the identification device 10, it is possible to identify applications that generate traffic not only for data from small-scale networks but also for flow data from large-scale networks, for which labeling has been difficult until now. , application-level traffic identification becomes possible even in large-scale networks.

[System configuration, etc.]
Each component of each device shown in the drawings is functionally conceptual, and does not necessarily need to be physically configured as shown in the drawings. In other words, the specific form of distributing and integrating each device is not limited to what is shown in the diagram, and all or part of the devices can be functionally or physically distributed or integrated in arbitrary units depending on various loads, usage conditions, etc. Can be integrated and configured. Furthermore, all or any part of each processing function performed by each device can be realized by a CPU and a program that is analyzed and executed by the CPU, or can be realized as hardware using wired logic.

Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed manually. All or part of this can also be done automatically using a known method. In addition, information including processing procedures, control procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed arbitrarily, unless otherwise specified.

[program]
FIG. 6 is a diagram showing an example of a computer that implements the identification device 10 by executing a program. Computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These parts are connected by a bus 1080.

The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). Hard disk drive interface 1030 is connected to hard disk drive 1090. Disk drive interface 1040 is connected to disk drive 1100. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into disk drive 1100. Serial port interface 1050 is connected to, for example, mouse 1110 and keyboard 1120. Video adapter 1060 is connected to display 1130, for example.

The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, application programs 1092, program modules 1093, and program data 1094. That is, a program that defines each process of the identification device 10 is implemented as a program module 1093 in which computer-executable code is written. Program module 1093 is stored in hard disk drive 1090, for example. For example, a program module 1093 for executing processing similar to the functional configuration of the identification device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

Furthermore, the setting data used in the processing of the embodiment described above is stored as program data 1094 in, for example, the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads out the program module 1093 and program data 1094 stored in the memory 1010 and the hard disk drive 1090 to the RAM 1012 as necessary and executes them.

Note that the program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in a removable storage medium, for example, and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, program module 1093 and program data 1094 may be stored in another computer connected via a network (LAN, WAN (Wide Area Network), etc.). The program module 1093 and program data 1094 may then be read by the CPU 1020 from another computer via the network interface 1070.

Although embodiments to which the invention made by the present inventor is applied have been described above, the present invention is not limited by the description and drawings that form part of the disclosure of the present invention by this embodiment. That is, all other embodiments, examples, operational techniques, etc. made by those skilled in the art based on this embodiment are included in the scope of the present invention.

2A, 2B Small network (NW) equipment 3A, 3B NW router to be identified 10 Identification device 11 Collection section 12 Signature generation section 13 Flow data generation section 14 Signature database (DB)
15 Feature calculation section 16 Label addition section 17 Discriminator learning section 18 Learned classifier 19 Application identification section 20 Output section

Claims (5)

An identification method performed by an identification device for identifying an application, the method comprising:
a collection step of collecting packet data from the small-scale network that satisfies predetermined rules and first flow data of the large-scale network to be identified ;
a signature generation step of analyzing packet data from the small network to generate a signature that associates an application with an IP address;
a flow data generation step of generating second flow data from packet data from the small-scale network ;
First feature information, which is a statistical feature for each IP address, is calculated for the first flow data of the large-scale network , and first feature information is a statistical feature for each IP address, for the second flow data. a calculation step of calculating second feature information;
an addition step of adding a label to the second feature amount information using the signature;
a learning step of causing a classifier to learn to identify the application using the first feature information and the second feature information as learning data;
including;
In the learning step, the classifier is trained in advance by using the second feature information to which the label is added as learning data, and the first feature information without a label and the second feature information without a label are trained in advance. An identification method characterized in that the discriminator is trained by a domain application technique using quantity information . further comprising an identification step of using the identifier to identify an application corresponding to an IP address of the flow data to be identified;
The collecting step collects the flow data to be identified,
The calculation step calculates identification feature amount information, which is a statistical feature amount for each IP address, for the flow data to be identified,
The identification step includes inputting the identification feature information to the classifier, and identifying an application corresponding to the IP address of the flow data to be identified based on the classification result output from the classifier. The identification method according to claim 1, characterized in that: The calculation step calculates at least one of a histogram of the number of packets, a histogram of the number of bytes, or a histogram of the number of bytes and packets from a set of flow data with a certain IP address as a source and/or destination per 24 hours. The identification method according to claim 1 or 2, characterized in that one of the following is calculated. An identification device for identifying an application, comprising:
a collection unit that collects packet data from the small-scale network that satisfies predetermined rules and first flow data of the large-scale network to be identified ;
a signature generation unit that analyzes packet data from the small-scale network and generates a signature that associates an application with an IP address;
a flow data generation unit that generates second flow data from packet data from the small-scale network ;
First feature information, which is a statistical feature for each IP address, is calculated for the first flow data of the large-scale network , and first feature information is a statistical feature for each IP address, for the second flow data. a feature calculation unit that calculates second feature information;
a label adding unit that adds a label to the second feature amount information using the signature;
a learning unit that causes a classifier to learn to identify the application using the first feature information and the second feature information as learning data;
has
The learning unit causes the discriminator to learn in advance the second feature information to which the label is added as learning data, and the first feature information without a label and the second feature without a label. An identification device characterized in that the discriminator is trained by a domain application technique using quantity information . a collection step of collecting packet data from the small network and first flow data of the large network to be identified that satisfy a predetermined rule;
a signature generation step of analyzing packet data from the small network to generate a signature that associates an application with an IP address;
a flow data generation step of generating second flow data from packet data from the small-scale network ;
First feature information, which is a statistical feature for each IP address, is calculated for the first flow data of the large-scale network , and first feature information is a statistical feature for each IP address, for the second flow data. a calculation step of calculating second feature information;
an adding step of adding a label to the second feature amount information using the signature;
a learning step of causing a classifier to learn application identification using the first feature information and the second feature information as learning data;
make the computer run
In the learning step, the classifier is trained in advance by using the second feature information to which the label is added as learning data, and the first feature information without a label and the second feature information without a label are trained in advance. An identification program that performs learning of the discriminator using domain application technology using quantity information .

Images