JP7366690B2 - Equipment type estimation system - Google Patents

Description

The present invention relates to a device type estimation system.

The entire manufacturing industry is being asked to take social responsibility regarding product quality such as safety. At corporate manufacturing sites, business improvements are progressing through the use of IoT devices, and a variety of devices are being introduced. On the other hand, these devices may not be properly managed. For example, if a device infected with malware (malicious software) is illegally connected to a system such as an industrial system at a factory without permission from the administrator, the factory's data and , there is a risk of falsification of product data produced at that factory. As a result, these situations may affect a company's operations and product quality.

In the above example, it is necessary to discover equipment that is illegally connected to the system. To this end, it is important for factory managers to know, or to some extent estimate, what type of connected equipment it is, in order for factory managers to quickly discover the equipment.

In an IT system to which general-purpose computers such as personal computers and servers are mainly connected, an administrator or user can easily manage the computers by installing management agent software on the computers.

On the other hand, in systems where installation of application software such as management agent software is not considered, for example, systems in which embedded devices are used, such as some industrial systems, some measures are required. In such a case, one known method for understanding the type of device is to collect and analyze communication traffic when the device communicates over a network.

Patent Document 1 discloses a technology that "supports efficiency in identifying the type of terminal from which a service is accessed." Specifically, ``Based on a set of access logs showing access to one of the multiple services by any one of the multiple access sources, a first calculation unit that calculates statistics regarding access; a second calculation unit that applies a clustering method to the statistics to calculate the degree of association between each access source and each service; and the degree of association. and a classification unit that classifies the plurality of access sources into any group, each of which includes one or more access sources, based on the access source classification device. Here, the term "service" refers to a service "provided by a server by a content service provider (CSP)."

Additionally, Patent Document 2 discloses a communication analysis device that "analyzes the flow of communication traffic and estimates applications and their usage states related to the user's quality of experience."

JP2019-36120A JP2017-139580A

As described above, depending on the system, the configuration may not be such that a terminal accesses a server that provides services of a content service provider. Additionally, the server device may not output the access log.

Patent Document 1 calculates statistics from access logs indicating accesses to services provided by a content service provider through a server, and calculates statistics based on the operating system (OS) and type of the terminal (desktop PC type, notebook PC type, mobile terminal type). This relates to technology for identifying types (types, etc.). However, the technology of Patent Document 1 may not be applicable in cases where the system does not access a server providing services of a content service provider from a terminal, or in cases where the server device does not output an access log. .

Further, Patent Document 2 relates to a technology for estimating an application used by a user. However, Patent Document 2 does not disclose a technique for grasping or estimating what type of device is communicating.

As described above, the techniques described in Patent Documents 1 and 2 may not be applicable to estimating the type of equipment connected to the system.
Therefore, there is a need for a technique that can estimate the type of connected equipment even when it cannot be estimated using conventional techniques.

The present invention provides a technique for estimating the type of equipment connected to a system, and a system, device, method, and program using the technique.
A device type estimation system according to one aspect of the present invention is a device type estimation system that analyzes communication traffic flowing through a communication network and estimates the type of a device connected to the communication network, and includes learning classification processing. A classification device that performs estimation classification processing, and learning data for the classification device to perform the learning classification processing based on device type information of the device that outputs the communication traffic and feature amount data of the communication traffic. a verification data set for verifying the classification accuracy of the learning classification process of the classification device, and estimation data for causing the classification device to estimate the type of the device that outputs the communication traffic. a preprocessing device for outputting, a verification device for verifying the classification accuracy of the learning classification process of the classification device, and an estimation for estimating the type of the device based on the classification result of the estimation classification process of the classification device. and an estimation device that performs processing.

According to one aspect of the present invention, the type of equipment connected to a system can be estimated.

FIG. 2 is a conceptual diagram for explaining a device type estimation system. FIG. 2 is a functional block diagram illustrating a configuration example of a feature calculation device of the device type estimation system. FIG. 2 is a system operation sequence diagram showing an example of processing contents for implementing a learning phase of the device type estimation system. FIG. 2 is a system operation sequence diagram illustrating an example of processing contents for implementing an estimation phase of the device type estimation system. 3 is a flowchart illustrating an example of the processing content of the feature value calculation processing device in the learning phase and the estimation phase. It is a flowchart which shows an example of the processing content of a preprocessing device in a learning phase. It is a flowchart which shows an example of the processing content of a classification device in a learning phase. It is a flowchart which shows an example of the processing content of a verification apparatus in a learning phase. It is a flowchart which shows an example of the processing content of a preprocessing device in an estimation phase. It is a flowchart which shows an example of the processing content of a classification device in an estimation phase. It is a flowchart which shows an example of the processing content of an estimation processing device in an estimation phase. FIG. 3 is a diagram illustrating a configuration example of feature amount data. It is a figure showing an example of composition of a device type list. FIG. 3 is a diagram illustrating a configuration example of model management information. It is a chart showing a composition example of data for learning, data for verification, and data for estimation. It is a chart showing an example of a structure of a learning label, a verification label, and classification result data. It is a figure which shows the example of a structure of classification accuracy data. It is a figure which shows the example of a structure of estimation result data. FIG. 2 is a configuration diagram showing an example of the hardware configuration of a device that constitutes the device type estimation system. FIG. 6 is a diagram illustrating an example of a process of generating learning data and a learning label in a preprocessing device in a learning phase. FIG. 6 is a diagram illustrating an example of processing based on classification accuracy data in a preprocessing device in a learning phase.

Examples will be described below with reference to the drawings.
The device type estimation system of the embodiment estimates the type of device connected to a target system through analysis.

(system components)
With reference to FIG. 1, the configuration of a device type estimation system according to an embodiment will be described.
The equipment type estimation system 3 can analyze an industrial system like the target system 1, for example. Target system 1 is, for example, an industrial control system that includes various devices related to manufacturing, air conditioning, electricity, water, gas, building equipment, communications, etc., and various devices that control and monitor these devices via a communication network. It is a system.
The device type estimation system 3 includes a feature calculation device 31 , a preprocessing device 33 , a classification device 35 , a verification device 37 , and an estimation device 39 .

The target system 1 includes one or more devices 11, one or more network devices 13, and a management device 15 that manages the device type, network address, etc. of the devices 11 as ledger data. The devices 11 are interconnected via a communication network such as a LAN (Local Area Network) via a network device 13 such as a network switch.

The communication monitoring device 51 is a device that monitors the communication network of the target system 1. The communication monitoring device 51 is connected to the network device 13 and acquires packet data of communication traffic flowing to the network device 13 . Here, the packet data is data that is obtained by dividing communication data into a certain length and adding communication management data (header) to the divided communication data to form a single data block. The communication monitoring device 51 organizes packet data in units of communication flows, generates one or more data sets of communication flow data 81, and uses the feature calculation device 31, which is one of the components of the device type estimation system 3. Send to.

Here, the communication flow is based on the information of 5 tuples (source IP address, destination IP address, source port number, destination port number, transport protocol number) stored in the header of each packet data. Identified by a combination of

The feature calculation device 31 is a device that analyzes a data set of communication flow data 81 received from the communication monitoring device 51 and calculates the feature amount of communication traffic output by the device 11 connected to the communication network.

The classification device 35 learns by associating the device type of the device 11 that outputs the communication traffic with the feature amount of the communication traffic, and further classifies the feature amount of another communication traffic based on the learned content. This is a device that outputs. The verification device 37 is a device that verifies the classification accuracy of the learned classification device 35 and outputs information on the classification accuracy.

The preprocessing device 33 is a device that outputs a training data set for learning the classification device 35 and a verification data set for verifying the classification accuracy of the learned classification device 35. Further, the preprocessing device 33 outputs an estimation data set for causing the classification device 35 to classify the device type of the device 11 that outputs the communication traffic. Further, the preprocessing device 33 improves the learning data set with poor accuracy.

The estimation device 39 is a device that aggregates the classification results output by the classification device 35 and estimates the device type of the device 11 from the results. The estimation device 39 may transmit the estimation result of the device type of the device 11 to the management device 15 and update the device type information of the device 11 in the ledger data of the management device 15.

Further, the estimation device 39 may transmit the estimation result of the device type of the device 11 to the terminal device 53. For example, upon receiving the estimation result of the device type of the device 11, the terminal device 53 may display the content on the screen and have the administrator of the target system 1 confirm the content.

The device type estimation system 3 includes different operation phases including a learning phase and an estimation phase. In the learning phase, the device type estimation system 3 learns the characteristics of communication traffic output by the device 11. Furthermore, in the estimation phase, the device type estimation system 3 classifies the feature amount of the communication traffic output by the device 11 based on the content learned in the learning phase, and outputs the result.

Switching of the operation phase in the device type estimation system 3 is normally performed manually by an administrator such as by inputting a command to each device. However, when a certain condition is satisfied, the switching may be automatically performed by a determination program installed in the device type estimation system 3 or another device. Alternatively, different operating phases may be performed simultaneously or alternately over short periods of time.

Hereinafter, details of the configuration and processing contents of each device constituting the device type estimation system 3 will be explained with reference to the drawings.

(Configuration of feature calculation device)
FIG. 2 is a functional block diagram showing an example of the configuration of the feature calculation device 31. As shown in FIG.
When the feature amount calculation device 31 receives the communication flow data 81 from the communication monitoring device 51, it analyzes the dataset of packet data for each communication flow included in the communication flow data 81, and calculates the feature amount for each communication flow. Then, feature amount data 83 is output.

The communication flow data 81 includes header information such as a source MAC address, a destination MAC address, a source IP address, a destination IP address, and a transport protocol number. In addition, since it includes packet data of a series of communication sequences, the communication flow data 81 can also be analyzed to determine whether the communication is successful (for example, if a response is returned to a connection request). It is possible to distinguish by this.

(Processing content of the feature calculation device)
The processing contents of the feature value calculation device 31 will be explained using FIG. 5. FIG. 5 is a flowchart showing feature amount calculation processing (S11) in the learning phase in the feature amount calculation processing device 31. The feature quantity calculation process (S21) in the estimation phase is the same as the feature quantity calculation process (S11) in the learning phase.

When the feature calculation device 31 receives the communication flow data 81 from the communication monitoring device 51 (step S111), it extracts header information for each communication flow included in the communication flow data 81 (step S112). The header information includes source and destination MAC addresses, source and destination IP addresses, and transport protocol numbers.

Here, the source and destination MAC addresses do not necessarily need to be extracted. Depending on the network configuration of the target network 1, the MAC address of the network device 13 instead of the MAC address of the device 11 may be included in the packet data. In that case, the MAC address cannot be used to identify communication flows. Therefore, the source and destination MAC addresses are not essential.

Furthermore, contrary to the above, the source and destination MAC addresses may be extracted instead of the source and destination IP addresses. For example, if the target system 1 does not employ IP (Internet Protocol), address information of the network interface layer is used. Further, if necessary, identification information of the virtual network such as a VLAN ID (Virtual LAN Identifier) may be included.

Furthermore, the feature calculation device 31 may analyze the communication state and extract the communication state flag. Examples of communication states include a "successful communication" state and a "no response" state. An example of a successful communication state is a state in which a response is returned from the destination device in response to a connection request from the source device to the destination device. An example of a no-response state is a state in which a response is not returned from the destination device in response to a connection request from the source device to the destination device.

Then, the feature calculation device 31 calculates a communication feature from the communication pattern for each communication flow (step S113). Here, as an example of how to calculate the communication feature amount, we will use statistical values (minimum value, 25% value, median value, 75% value, maximum value, average value, variance) of packet sizes of packets sent and received as flow statistics. There is a calculation method that calculates statistical values (minimum value, 25% value, median value, 75% value, maximum value, average value, variance value) of the packet arrival interval at a specific number of packets.

Then, the feature amount calculation device 31 generates feature amount data 83 including the header information extracted in step S112 and the communication feature amount calculated in step S113 for each communication flow (step S114). Then, the feature amount calculation device 31 transmits the feature amount data 83 to the preprocessing device 33 (step S115).

(Configuration of feature data)
An example of the structure of the feature data 83 is shown in FIG.
Each record of the feature data 83 includes a source MAC address value 831, a destination MAC address value 832, a source IP address value 833, a destination IP address value 834, or both (AND/OR), and a transceiver. It is configured to include a port protocol number 835, a communication status flag value 836, and a communication feature value 837. These pieces of information are values extracted or calculated in the feature amount calculation processing (S11 and S21) in the feature amount calculation processing device 31.

(Learning phase implementation flow)
FIG. 3 is a sequence diagram illustrating an example of system operation for implementing the learning phase of the device type estimation system 3.
First, the feature amount calculation device 31 receives the communication flow data 81 from the communication monitoring device 51, performs feature amount calculation processing (S11), and transmits the feature amount data 83 to the preprocessing device 33. The feature calculation device 31 periodically acquires communication flow data 81 from the communication monitoring device 51, such as at 5-minute intervals.

Here, the communication monitoring device 51 may periodically transmit the information to the feature calculation device 31. Alternatively, the timing of transmission or acquisition may be determined by command input by the administrator. Alternatively, the timing may be determined at a time when a condition is satisfied, such as when the communication flow data 81 is transmitted when the accumulated size of the communication flow data 81 in the communication monitoring device 51 exceeds a certain amount. Next, the preprocessing device 33 performs learning preprocessing (S13).

(Processing content of preprocessing device 33 in learning phase)
Here, the learning preprocessing (S13) in the preprocessing device 33 will be explained with reference to the drawings. FIG. 6 is a flowchart showing an example of the processing content of the preprocessing device 33 in the learning phase.

When the preprocessing device 33 receives the feature amount data 83 from the feature amount calculation device 31, it receives the device type list 85 from the management device 15. Further, the preprocessing device 33 receives classification accuracy data 93 from the verification device 37 (step S131). Here, if there is no classification accuracy data 93, it is assumed that the classification accuracy is 100%, and the processing proceeds with the assumption that the classification accuracy is 100%.

(Configuration of device type list)
Here, the configuration of the device type list 85 will be explained with reference to the drawing.
FIG. 13 is a configuration diagram showing an example of the configuration of the device type list 85.
The device type list 85 includes an IP address value 851 and label information 852. In the label information 852, device type information of the device 11 to which the IP address value 851 is set is set by the administrator or user of the device 11. Returning to FIG. 6, the explanation will be continued.

Next, the preprocessing device 33 extracts the source IP address and the destination IP address from the feature data 83 for each communication flow (step S132). Here, depending on the environment, a MAC address may be used instead of an IP address.
Next, the preprocessing device 33 searches for each of the source IP address and the destination IP address from the IP address value 851 in the device type list 85, and obtains matching device type information (label information) (step S133 ). However, if there are no matching records, you can skip it.
Next, the preprocessing device 33 divides the feature data 83 based on the data model management information 99 included in the preprocessing device 33 .

(Configuration of data model management information)
Here, the structure of the data model management information 99 will be explained with reference to the drawing.
FIG. 14 is a configuration diagram showing an example of the configuration of the model management information 99.
The model management information 99 is information for managing division conditions when the preprocessing device 33 divides the feature amount data 83 into a plurality of pieces. Further, the preprocessing device 33 filters the feature amount data 83 under certain conditions. The model management information 99 also manages the filtering conditions.

The data model management information 99 includes model ID information 991, transmission/reception type information 992, transport protocol number 993, and communication status flag condition 994.

Model ID information 991 is identification information of each record of model management information 99. The transmission/reception type information 992 is information indicating which device type of the device 11, the transmission source or the destination device, is to be learned in each communication flow of the feature amount data 83. The transport protocol number 993 stores the transport protocol number of the header information for each communication flow of the feature amount data 83. The communication state flag condition 994 stores communication state flag information that is a condition when the preprocessing device 33 filters the feature amount data 83. That is, only the data of the communication flow that satisfies the communication state flag condition 994 is used for analysis by the classification device 35.

Returning to FIG. 6, the explanation will be continued. The feature amount data 83 is divided according to the transport protocol number 993 and the communication state flag condition 994 in each record of the data model management information 99. If there is transmission/reception type information 992, the feature amount data 83 is duplicated for each transmission/reception type. Furthermore, the value of model ID 991 is given to each record (step S134).

The preprocessing device 33 generates label information corresponding to the IP address corresponding to the sending/receiving type 992 for each of the divided feature data 83 in the order of entries (generated in step S133), and creates a model ID 991. The value of is given to each record (step S135).

Here, if there is no corresponding label information, the record is deleted from each of the feature data 83 copied in step S134, and the label information is not generated. Further, the preprocessing device 33 has already received the classification accuracy data 93 from the verification device 37, and based on the classification accuracy data 93, the classification accuracy for the label acquired in step S133 for the model ID 931 is managed in advance. If the value is below the threshold set by the user, the record is deleted from each of the replicated feature data 83 in step S134, and the label information is not generated.

Here, an example of the classification accuracy data 93 is shown in FIG. As shown in FIG. 17(A), the classification accuracy data 93 includes a model ID 931, a correct label 932, and the number of output classification result labels 933. When the classification device 35 learns the learning data 86 and the learning labels 87 of the model ID 931, the number of classification result labels for the correct labels of the classification result data for the verification data 88 is stored in the classification result label output number 933. be done.

Here, as shown in FIG. 17(B), the classification accuracy data 93 includes a correct number 936 indicating the number of correctly classified items and a false positive number 936 indicating the number of incorrectly classified items using the correct label. It may also be configured to include the number 937 and the number of missed detections 938 indicating the number that could not be classified into the correct label.

Here, an image of the process of step S135 using the classification accuracy data 93 is shown using FIG. For example, in the classification accuracy data 93, if the classification accuracy of "E company F device" is about 59%, the preprocessing device 33 uses the learning data 86 and the learning label 87 to determine "E company F device". Delete the corresponding record. Through this processing, learning data with poor classification accuracy is deleted, and as a result, the learning accuracy in the classification device 91 can be improved.

Specifically, in model ID: 01, the learning of "E company F device" is regarded as a failure. In the subsequent learning phase, the preprocessing device 33 deletes the communication record corresponding to "E company F device" from the learning data and the learning label.
Next, the preprocessing device 33 divides each feature amount data 83 into learning data 86 and verification data 88 (step S136).

(Configuration of training data and verification data)
Here, an example of the configuration of the learning data 86 and the verification data 88 will be explained with reference to the drawings. FIG. 15 is a configuration diagram showing a configuration example of the learning data 86 and the verification data 88.

The structure of the learning data 86 is shown in FIG. 15(A), and the structure of the verification data 88 is shown in FIG. 15(B). Further, the configuration of the estimation data 90 is shown in FIG. 15(C). The learning data 86 includes a model ID 861 and a communication feature amount 862. The configuration of the verification data 88 is similar to the learning data 86, and includes a model ID 881 and a communication feature amount 882.

Note that the estimation data 90 is similarly configured to include a model ID 901 and a communication feature amount 902. Returning to FIG. 6, the explanation will be continued.

Next, the preprocessing device 33 divides each label information into a learning label 87 and a verification label 89 (step S137).

(Configuration of training labels and verification labels)
Here, an example of the configuration of the learning label 87 and the verification label 89 will be explained with reference to the drawings. FIG. 16 is a configuration diagram showing an example of the configuration of the learning label 87 and the verification label 89.
The configuration of the learning label 87 is shown in FIG. 16(A), and the configuration of the verification label 89 is shown in FIG. 16(B). Further, the configuration of the classification result data 91 is shown in FIG. 16(C). The learning label 87 includes a model ID 871, an IP address 872, and label information 873. The configuration of the verification label 89 is similar to that of the learning label 87, and includes a model ID 891, an IP address 892, and label information 893. Note that the classification result data 91 similarly includes a model ID 911, an IP address 912, and label information 913.

Returning to FIG. 6, the explanation will be continued. Next, the learning data 86, the learning label 87, and the verification data 88 are transmitted to the classification device 35 (step S138). Then, the verification label 89 is transmitted to the verification device 37 (step S139).

Here, FIG. 20 illustrates an example of the processing from steps S133 to S139. As a result of the processing from steps S133 to S139, the feature amount data 83 is copied according to the transport protocol number 993 and the communication state flag condition 994. As a result, the same number of learning data 86, learning labels 87, and verification data 88 as the number of IDs of the model ID 991 is generated. Here, the preprocessing device 33 associates the learning data 86 and the learning label 87 in the same record row. For example, the data in the first record of the learning data 86 and the data in the first record of the learning label 87 are associated.

Returning to FIG. 3, the explanation will be continued. The classification device 35 performs learning classification processing (S15).

(Processing content of the classification device 35 in the learning phase)
The processing contents of the classification device 35 in the learning phase will be explained using FIG. 7.
FIG. 7 is a flowchart showing an example of the processing contents of the classification device 35 in the learning phase.
First, the classification device 35 receives the learning data 86, the learning label 88, and the verification data 88 from the preprocessing device 33 (step S151).

Next, the classification device 35 uses a machine learning algorithm (such as SVM) to learn the communication feature amount 862 of the learning data 86 and the label information 873 of the learning label 87 for each model ID 861, and further uses the verification data 88 The communication feature amount 882 is classified (step S152).
Then, the classification device 35 outputs the classification results for each model ID 861 as classification result data 91 (step S153). Next, the verification device 37 performs verification processing (S17).

(Processing content of the verification device 37 in the learning phase)
The processing contents of the verification device 37 in the learning phase will be explained using FIG. 8.
The verification device 37 receives the classification result data 91 from the classification device 35, and receives the verification label 89 from the preprocessing device 35 (step S171).

Next, the verification device 37 calculates classification accuracy for each model ID 881 using the classification result data 91 and the verification label 89 (step S172).
Then, the verification device 37 outputs classification accuracy data 93 for each model ID 881 (step S173).

(Implementation flow of estimation phase)
FIG. 4 is a sequence diagram illustrating an example of system operation for implementing the estimation phase of the device type estimation system 3.
First, the feature amount calculation device 31 receives the communication flow data 81 from the communication monitoring device 51, performs feature amount calculation processing (S21), and transmits the feature amount data 83 to the preprocessing device 33. The feature calculation device 31 periodically acquires communication flow data 81 from the communication monitoring device 51, such as at 5-minute intervals.

Here, the communication monitoring device 51 may periodically transmit the information to the feature calculation device 31. Alternatively, the timing of transmission or acquisition may be determined by command input by the administrator. Alternatively, the timing may be determined at a time when a condition is satisfied, such as when the communication flow data 81 is transmitted when the accumulated size of the communication flow data 81 in the communication monitoring device 51 exceeds a certain amount. Next, the preprocessing device 33 performs estimation preprocessing (S23).

(Processing content of preprocessing device 33 in estimation phase)
Here, the estimation preprocessing (S23) in the preprocessing device 33 will be explained with reference to the drawings.
FIG. 9 is a flowchart showing an example of the processing content of the preprocessing device 33 in the estimation phase.
The preprocessing device 33 receives the feature amount data 83 from the feature amount calculation device 31 (step S231).

Next, the preprocessing device 33 extracts the source IP address and destination IP address from the feature amount data 83 (step S232). Here, depending on the environment, a MAC address may be used instead of an IP address.

Next, the feature amount data 83 is divided from the data model management information 99 according to the transport protocol number 993 and the communication state flag condition 994. If there is transmission/reception type information 992, the feature amount data 83 is duplicated for each transmission/reception type. Furthermore, a model ID 901 is assigned (step S234).

Next, the preprocessing device 33 transmits the estimation data 90 to the classification device 35 (step S235).

(Processing content of the classification device 35 in the estimation phase)
The processing contents of the classification device 35 in the estimation phase will be explained using FIG. 10.
First, the classification device 35 receives estimation data 90 from the preprocessing device 33 (step S251).

Next, the classification device 35 uses a machine learning algorithm (such as SVM) to learn the communication feature amount 862 of the learning data 86 and the label information 873 of the learning label 87 for each model ID 901, and further uses the estimation data 90 The communication feature amount 902 is classified (step S252).
Then, the classification device 35 outputs the classification results as classification result data 91 for each model ID 901 (step S253).

(Processing content of the estimation device 39 in the estimation phase)
Here, the processing contents of the estimation device 39 in the estimation phase will be explained with reference to the drawings.
FIG. 11 is a flowchart showing an example of the processing content of the estimation device 39 in the estimation phase.
The estimation device 39 receives the classification result data 91 from the classification device 35 (step S271).

Next, the estimation device 39 aggregates the label information 913 in units of IP addresses for each model ID 911 from the classification result data 91, and calculates the label that appears most frequently (step S272).
Next, the estimation device 39 outputs the set of the IP address and the calculated label as estimation result data 95 (step S273).

(Structure of estimation result data)
Here, the configuration of the estimation result data 95 will be explained with reference to the drawing.
FIG. 18 is a configuration diagram showing an example of the configuration of the estimation result data 95.
The configuration of the estimation result data 95 includes a model ID 951, an IP address 952, label information 953, and an estimated probability 954.

Returning to FIG. 11, the explanation will be continued. Furthermore, the estimation result data 95 is transmitted to the management device 15 (step S274).

(Device configuration)
FIG. 19 is a configuration diagram showing an example of the hardware configuration of a device that constitutes the device type estimation system. FIG. 19 shows the hardware configuration of the device 1000, including the feature calculation device 31, preprocessing device 33, classification device 35, verification device 37, and estimation device 39, which constitute the device type estimation system. These devices are general-purpose computer devices that include a CPU 1001, a main storage device 1002, an auxiliary storage device 1003, a network interface device 1004, an input device 1005, and an output device 1006 that are interconnected via an internal communication line 1009 such as a bus. It consists of

The CPU 1001 is a processor that controls the operation of the entire device 1000. Further, the main storage device 1002 is composed of, for example, a volatile semiconductor memory, and is used as a work memory of the CPU 1001. The auxiliary storage device 1003 is composed of a large-capacity nonvolatile storage device such as a hard disk device, an SSD (Solid State Drive), or a flash memory, and is used to retain various programs and data for a long period of time.

The executable program 1100 stored in the auxiliary storage device 1003 is loaded into the main storage device 1002 when the device 1000 is started or when necessary, and the CPU 1001 executes the executable program 1100 loaded into the main storage device 1002, thereby causing the device to run. Various processes for the entire 1000 are executed.

The network interface device 1004 is an interface device for connecting the device 1000 to each network in the system, and is composed of, for example, a NIC (Network Interface Card). The input device 1005 includes a keyboard, a pointing device such as a mouse, and is used by the user to input various instructions and information to the device 1000. The output device 1006 includes, for example, a display device such as a liquid crystal display or an organic EL (electro-luminescence) display, and an audio output device such as a speaker, and is used to present necessary information to the user when necessary.

As described above, according to the device type estimation system of the embodiment, it is possible to estimate the type of device connected to the target system.

1 Target system 3 Device type estimation system 11 Device 13 Network device 15 Management device 31 Feature calculation device 33 Preprocessing device 35 Classification device 37 Verification device,
39 Estimation device 51 Communication monitoring device 53 Terminal device

Claims (15)

A device type estimation system that analyzes communication traffic flowing through a communication network to estimate the type of device connected to the communication network,
a classification device that performs learning classification processing and estimation classification processing;
a learning data set for the classification device to perform the learning classification process based on device type information of the device that outputs the communication traffic and feature data of the communication traffic; and a learning dataset for the classification device to perform the learning classification process. a preprocessing device that outputs a verification data set for verifying classification accuracy of classification processing and estimation data for causing the classification device to estimate the type of the device that outputs the communication traffic;
a verification device that verifies the classification accuracy of the learning classification process of the classification device;
an estimation device that performs estimation processing to estimate the type of the device based on the classification result of the estimation classification processing of the classification device;
A device type estimation system characterized by having the following. The training data set includes training data and training labels,
The verification data set includes verification data and a verification label,
In the learning phase,
The pretreatment device includes:
Pre-processing for learning is performed based on the feature amount data and the device type information to generate the learning data, the learning label, the verification data, and the verification label, and the learning data and the learning label are generated. sending the label and the verification data to the classification device; sending the verification label to the verification device;
The classification device includes:
Performing the learning classification process based on the learning data, the learning label, and the verification data, generating first classification result data and sending it to the verification device;
The verification device includes:
Generating classification accuracy data based on the verification label and the first classification result data and sending it to the preprocessing device;
In the estimation phase,
The pretreatment device includes:
Performing pre-processing for estimation based on the feature data to generate estimation data and sending it to the classification device;
The classification device includes:
Performing the estimation classification process based on the estimation data, the learning data, and the learning label, generating second classification result data and sending it to the estimation device,
The estimation device includes:
The device type estimation system according to claim 1, wherein the estimation process is performed based on the second classification result data to generate estimation result data. The pretreatment device includes:
In the learning phase,
The learning data set and the verification data are generated based on the feature amount data of the communication traffic and a device type list that is list information that associates the address of the device with the device type information. The device type estimation system according to claim 2. The pretreatment device includes:
In the learning phase,
The apparatus according to claim 2, wherein the apparatus refers to the classification accuracy data and performs the learning preprocessing again to improve the learning data set and generate a new learning data set. Type estimation system. The pretreatment device includes:
In the learning phase,
A claim characterized in that the training dataset is generated by dividing it into a plurality of training datasets with different conditions with reference to the classification accuracy data, and the training dataset with low classification accuracy is improved. The device type estimation system according to item 4. The pretreatment device includes:
In the learning phase,
If the classification accuracy of the classification accuracy data is lower than a predetermined threshold, the training data set with low classification accuracy is improved by deleting the training data with low classification accuracy from the training data set. The device type estimation system according to claim 5. The pretreatment device includes:
3. The device type estimation system according to claim 2, wherein a result of classifying the plurality of verification data having different conditions is output for each condition. The pretreatment device includes:
In the estimation phase,
The estimation data is
The device according to claim 2, wherein the device is generated by extracting a source address and a destination address from the feature data and dividing the feature data according to flag conditions of a transport protocol number and a communication state. Type estimation system. The classification device includes:
2. The device type estimation system according to claim 1, wherein learning is performed by associating the device type information of the device that outputs the communication traffic with the feature amount data of the communication traffic. The classification device includes:
2. The device type estimation system according to claim 1, wherein the device type estimation system learns a plurality of the learning data sets having different conditions that are divided and generated by the preprocessing device. The verification device includes:
The results of classifying the feature amount data included in the plurality of verification data outputted by the preprocessing device under different conditions are classified into the devices included in the plurality of verification labels outputted by the preprocessing device under different conditions. The device type estimation system according to claim 2, wherein the classification accuracy data is calculated and output for each of the conditions using type information. The estimation device includes:
From the second classification result data output by the classification device, the device type information is aggregated in address units for each condition, the device type information that appears most often is calculated, and the relationship between the address and the device type information is calculated. The device type estimation system according to claim 2, wherein a set is output as the estimation result data. The device type estimation according to claim 1, further comprising a feature amount calculation device that calculates the feature amount data of the communication traffic outputted by the device and sends the feature amount data to the preprocessing device. system. The feature calculation device includes:
communication flow data of the communication traffic is input;
14. The device type estimation system according to claim 13, wherein the feature data is calculated from a communication pattern for each communication flow data. The device type estimation system according to claim 1, further comprising a terminal device connected to display the type of the device estimated by the estimation device on a screen.

Images