JP2018093432A - Determination system, determination method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress a risk of an abnormal state being determined erroneously.SOLUTION: A determination system comprises: a learning model acquisition unit for acquiring a learning model created by performing machine learning on a traffic amount of communication data in the past; a traffic amount reference value acquisition unit for acquiring a traffic amount reference value, a reference value of a traffic amount of communication data set on the basis of the traffic amount of communication data in the past; an abnormality determination threshold acquisition unit for acquiring an abnormality determination threshold, an outlier of the traffic amount reference value with respect to the learning model; a determination target traffic amount acquisition unit for acquiring a determination target traffic amount, a traffic amount of determination target communication data; an abnormality determination target value acquisition unit for acquiring an abnormality determination target value, an outlier of the determination target traffic amount with respect to the learning model; and a determination unit for determining whether a value is an abnormal state or not, on the basis of the abnormality determination threshold and abnormality determination target value.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a determination system, a determination method, and a program.

  For example, Patent Literature 1 discloses a traffic monitoring system that performs monitoring based on a lower threshold set for a traffic volume measurement location, and periodically acquires traffic information related to the traffic volume at the measurement location. And a determination unit that considers that a measurement point is in an alarm generation state when a calculated value calculated based on the traffic information acquired by the traffic information acquisition unit falls below a lower limit threshold. The system is described.

JP 2009-017393 A

By the way, in the system of Patent Document 1, the traffic fluctuation rate as the determination target value is compared with the determination threshold value, and when the traffic fluctuation rate falls below the determination threshold value, it is determined that the state is abnormal.
Therefore, in the system of Patent Document 1, for example, immediately after the change of the user's usage method, the traffic fluctuation rate after the change of the user's usage method is compared with the determination threshold set before the change of the user's usage method. It is determined whether or not there is an abnormal state. As a result, immediately after the change in the user's usage method, there is a possibility that it is erroneously determined to be an abnormal state despite the normal state.
Further, in the system of Patent Document 1, for example, immediately after execution of switching work in the network system, the traffic fluctuation rate after execution of switching work is compared with the determination threshold set before execution of switching work, and an abnormal state is detected. It is determined whether or not. As a result, immediately after execution of the switching operation, there is a possibility that it is erroneously determined that the state is abnormal although it is in the normal state.

  Some aspects of the present invention have an object to provide a determination system, a determination method, and a program capable of suppressing a possibility that the traffic amount of communication data is erroneously determined as being in an abnormal state. .

  Another object of another aspect of the present invention is to provide a determination system, a determination method, and a program that can achieve the effects described in the embodiments described later.

In order to solve the above-described problem, an aspect of the present invention provides a learning model acquisition unit that acquires a learning model created by machine learning of the traffic amount of past communication data, and the traffic of the past communication data. A traffic amount reference value acquisition unit that acquires a traffic amount reference value that is a reference value of the traffic amount of communication data set based on the amount, and an abnormality determination threshold that is an outlier of the traffic amount reference value for the learning model An abnormality determination threshold value acquisition unit to be acquired, a determination target traffic amount acquisition unit that acquires a determination target traffic amount that is a traffic amount of communication data to be determined, and an abnormality determination that is an outlier of the determination target traffic amount with respect to the learning model An abnormality determination target value acquisition unit for acquiring a target value, an abnormality condition based on the abnormality determination threshold value and the abnormality determination target value A determination section for determining whether or not it is a determination system comprising a.
Further, according to another aspect of the present invention, the determination system includes a first step of acquiring a learning model created by machine learning of the traffic volume of past communication data, and the traffic volume of the past communication data. A second step of acquiring a traffic amount reference value that is a reference value of the traffic amount of communication data set based on the third step; and a third step of acquiring an abnormality determination threshold value that is an outlier of the traffic amount reference value for the learning model. A fourth step of acquiring a determination target traffic amount that is a traffic amount of communication data to be determined; a fifth step of acquiring an abnormality determination target value that is an outlier of the determination target traffic amount with respect to the learning model; And a sixth step for determining whether or not an abnormal state is present based on the abnormality determination threshold value and the abnormality determination target value. It is a method.
Another aspect of the present invention is based on the first step of acquiring a learning model created by machine learning of the traffic volume of past communication data in a computer, and the traffic volume of the past communication data. A second step of acquiring a traffic amount reference value that is a reference value of the traffic amount of the communication data set in step 3, and a third step of acquiring an abnormality determination threshold value that is an outlier of the traffic amount reference value for the learning model; A fourth step of acquiring a determination target traffic amount that is a traffic amount of communication data to be determined; a fifth step of acquiring an abnormality determination target value that is an outlier of the determination target traffic amount with respect to the learning model; And a sixth step of determining whether or not an abnormal state is present based on the abnormality determination threshold value and the abnormality determination target value. It is because of the program.

  According to an embodiment of the present invention, it is possible to suppress a possibility that the traffic amount of communication data is erroneously determined as being in an abnormal state.

The figure which shows an example of the network system to which the determination system of 1st Embodiment was applied. The figure which shows an example of the network system which can apply the determination system of 1st Embodiment. The figure which shows the flow of the process in the determination system of 1st Embodiment, etc. The figure which shows an example of the relationship between a connection apparatus, VLAN, and a port. The figure which shows the general abnormality determination method of the traffic volume for determination. The figure which shows an example of a misjudgment. The figure which shows the other example of a misjudgment. The figure which shows the determination by the determination system of 1st Embodiment. The figure which shows the other example of the determination by the determination system of 1st Embodiment. The figure which shows the calculation load which calculates a traffic amount reference value, and the calculation load which produces a learning model. The figure which shows an abnormality determination target value. The figure which shows the method of determining whether an abnormality determination target value is abnormal. The figure which compared two abnormality determination threshold values. The figure which shows an example of the calculation method of traffic amount reference value. The figure which shows the other example of the calculation method of traffic amount reference value. The figure which shows the relationship between an abnormality determination target value and an abnormality determination threshold value. The figure which shows the other example of the network system which can apply the determination system of 1st Embodiment. The flowchart which shows the flow of the whole process by the determination system. The figure which shows an example of the network system to which the determination system of 2nd Embodiment was applied.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[First embodiment]
A first embodiment of the present invention will be described.
FIG. 1 is a diagram illustrating an example of a network system 200 to which the determination system 100 according to the first embodiment is applied.
In the example illustrated in FIG. 1, the determination system 100 includes a learning model acquisition unit 10, a traffic amount reference value acquisition unit 11, an abnormality determination threshold acquisition unit 12, a determination target traffic amount acquisition unit 13, and an abnormality determination target value acquisition. Unit 14, a traffic amount reference value calculation unit 20, a determination unit 30, a network analysis unit 40, and an alarm analysis unit 50.
The learning model acquisition unit 10 acquires a learning model created by machine learning of the traffic volume of past communication data. In the example illustrated in FIG. 1, the learning model creation unit 210 is provided outside the determination system 100. The learning model creation unit 210 creates a learning model by machine learning of the traffic volume of past communication data. The learning model acquisition unit 10 acquires a learning model from the learning model creation unit 210.

The traffic amount reference value acquisition unit 11 acquires a traffic amount reference value that is a reference value of the traffic amount of communication data set based on the traffic amount of past communication data. In the example illustrated in FIG. 1, the traffic amount reference value calculation unit 20 is provided inside the determination system 100. The traffic amount reference value calculation unit 20 calculates a traffic amount reference value based on a statistical value of the traffic amount of past communication data. The traffic amount reference value acquisition unit 11 acquires the traffic amount reference value from the traffic amount reference value calculation unit 20.
The abnormality determination threshold value acquisition unit 12 acquires an abnormality determination threshold value that is an outlier of the traffic amount reference value for the learning model. In the example illustrated in FIG. 1, the abnormality determination threshold value calculation unit 220 is provided outside the determination system 100. The abnormality determination threshold value calculation unit 220 calculates an abnormality determination threshold value. The abnormality determination threshold value acquisition unit 12 acquires the abnormality determination threshold value from the abnormality determination threshold value calculation unit 220.

The determination target traffic amount acquisition unit 13 acquires a determination target traffic amount that is a traffic amount of communication data to be determined.
In the example illustrated in FIG. 1, the determination target traffic volume is measured at each of the plurality of ports 270-1, 270-2,... 270-8 included in the network system 200. The network system 200 includes an aggregation switch 280. The determination target traffic volume of each of the plurality of ports 270-1, 270-2,... 270-8 is acquired by the determination target traffic volume acquisition unit 13 via the aggregation switch 280. The traffic collection server 290 stores the determination target traffic amounts of the plurality of ports 270-1, 270-2,... 270-8.
In the example shown in FIG. 1, the traffic amount of past communication data of each of the plurality of ports 270-1, 270-2,... 270-8 is changed by the learning model creation unit 210 via the aggregation switch 280. To be acquired. The learning model creation unit 210 performs machine learning on the traffic amount of past communication data of each of the plurality of ports 270-1, 270-2,... 270-8, and thereby the plurality of ports 270-1, 270-2. , ..., 270-8 are created.

The abnormality determination target value acquisition unit 14 acquires an abnormality determination target value that is an outlier of the determination target traffic amount with respect to the learning model. In the example illustrated in FIG. 1, the abnormality determination target value calculation unit 230 is provided outside the determination system 100. The abnormality determination target value calculation unit 230 calculates an abnormality determination target value. The abnormality determination target value acquisition unit 14 acquires the abnormality determination target value from the abnormality determination target value calculation unit 230.
The determination unit 30 determines whether or not it is an abnormal state based on the abnormality determination threshold acquired by the abnormality determination threshold acquisition unit 12 and the abnormality determination target value acquired by the abnormality determination target value acquisition unit 14. . Specifically, the determination unit 30 determines that the state is abnormal when the abnormality determination target value is larger than the abnormality determination threshold.
In the example illustrated in FIG. 1, the determination unit 30 determines whether or not the determination target traffic amounts of the plurality of ports 270-1, 270-2,.

The network analysis unit 40 analyzes the number of ports determined by the determination unit 30 to be in an abnormal state, the position, and the like. Further, the network analysis unit 40 analyzes the determination result indicating that the state is abnormal. The determination result indicating that the state is abnormal includes, for example, a determination result caused by the user. Further, the determination result indicating that the state is abnormal includes, for example, a determination result caused by the network system 200.
When the determination result indicating that the state is abnormal is a determination result caused by the user, the network analysis unit 40 analyzes that the user's usage method has changed or that a switching operation or the like has been performed in the network system 200. To do. That is, the network analysis unit 40 analyzes that the network system 200 itself is normal.
When the determination result indicating the abnormal state is a determination result caused by the network system 200, the network analysis unit 40 analyzes that there is a possibility that an abnormality has occurred in the network system 200. Moreover, the network analysis part 40 specifies the location (suspected location) where the abnormality occurred. The suspected place is identified based on the position of the port determined to be in an abnormal state, the position of the port determined to be in a normal state, the configuration of the network system 200, the flow of traffic of communication data, and the like.

The alarm analysis unit 50 collates alarm information (history of issued alarms). Faults include silent faults that cannot be alarmed and normal faults that can be alarmed. The alarm analysis unit 50 analyzes whether it is determined that the abnormal state is caused by the silent failure or whether it is determined that the abnormal state is caused by the normal failure.
In the example illustrated in FIG. 1, the network analysis unit 40 and the alarm analysis unit 50 are provided in the determination system 100. However, in other examples, the network analysis unit 40 and the alarm analysis unit 50 may be omitted, or the determination system may be omitted. It may be provided outside the 100.

FIG. 2 is a diagram illustrating an example of a network system 200 to which the determination system 100 according to the first embodiment can be applied. In the example illustrated in FIG. 2, the network system 200 includes a first network 200-1, a second network 200-2, and a third network 200-3. The first network 200-1, the second network 200-2, and the third network 200-3 are connected to each other. The first network 200-1 is the network of the first operator, the second network 200-2 is the network of the second operator, and the third network 200-3 is the network of the third operator.
The first network 200-1 includes a plurality of VLANs (Virtual Local Area Networks) and a connection device 260-1 that connects them. The second network 200-2 and the third network 200-3 are configured in the same manner as the first network 200-1. The second network 200-2 includes a connection device 260-2, and the third network 200-3 includes a connection device 260-3.
By applying the determination system 100 according to the first embodiment to the network system 200 illustrated in FIG. 2, the determination unit 30 has a determination target traffic volume of each of a plurality of ports in the network system 200 in an abnormal state. It can be determined whether or not.

FIG. 3 is a diagram illustrating a processing flow and the like in the determination system 100 according to the first embodiment. In the example shown in FIG. 3, the traffic amount of past communication data of each of the plurality of ports in the connection devices 260-1 and 260-2 is stored in the traffic collection server 290. Further, the determination target traffic volume is measured at a plurality of ports in the connection devices 260-1 and 260-2.
The determination system 100 first performs traffic collection. Specifically, the determination system 100 collects the traffic amount of past communication data stored in the traffic collection server 290. Further, the determination system 100 collects the determination target traffic amounts of a plurality of ports in the connection devices 260-1 and 260-2 via the aggregation switch 280 having the mirror port 280a.
The decision system 100 then performs traffic analysis. Specifically, the determination system 100 acquires a learning model created by machine learning of the traffic amount of past communication data. Further, the determination system 100 calculates a traffic amount reference value based on the traffic amount of past communication data. The determination system 100 also acquires an abnormality determination threshold value that is an outlier of the traffic amount reference value for the learning model. In addition, the determination system 100 acquires an abnormality determination target value that is an outlier of the determination target traffic amount with respect to the learning model. Further, the determination system 100 determines whether or not each of the plurality of ports in the connection devices 260-1 and 260-2 is in an abnormal state based on the abnormality determination threshold value and the abnormality determination target value.

In the example illustrated in FIG. 3, the determination system 100 then performs alarm analysis. Specifically, the determination system 100 analyzes whether it is determined that the abnormal state is caused by a silent failure or whether it is determined that the abnormal state is caused by a normal failure.
The decision system 100 then performs network analysis. Specifically, the determination system 100 identifies a location (suspected location) where an abnormality has occurred in the connection devices 260-1 and 260-2. In the example illustrated in FIG. 3, the determination system 100 then executes a process for notifying a silent failure.

4 shows a connection device 260, a plurality of VLANs 250-1, 250-2, 250-3 connected to the connection device 260, and a plurality of ports 270-1, 270-2, 270-3 in the connection device 260. FIG.
In the example shown in FIG. 4, the VLAN 250-1 is connected to the connection device 260 via the port 270-1. Similarly, the VLANs 250-2 and 250-3 are connected to the connection device 260 via ports 270-2 and 270-3, respectively. VLAN 250-1, VLAN 250-2, and VLAN 250-3 are used by users U1, U2, and U3, respectively.
In the example illustrated in FIG. 4, the determination target traffic amount in the VLAN 250-1 is measured at the port 270-1. Similarly, the determination target traffic amounts in the VLAN 250-2 and the VLAN 250-3 are measured at the ports 270-2 and 270-3, respectively.

FIG. 5 is a diagram illustrating a general abnormality determination method for the determination target traffic volume. Specifically, FIG. 5A is a diagram illustrating an example of the relationship between the traffic amount of past communication data and the conventional determination threshold value. The horizontal axis in FIG. 5A indicates time. Similarly, FIG. 5B, FIG. 5C, FIG. 5D, FIG. 6A, FIG. 6B, FIG. 7C, FIG. B), FIG. 8A, FIG. 8B, FIG. 9A, FIG. 9B, FIG. 10A, FIG. 10B, FIG. 10C, and FIG. 11B, FIG. 11C, FIG. 12, FIG. 13A, FIG. 13B, FIG. 13C, FIG. 14A, FIG. 15A, and FIG. ) And the horizontal axis of FIG. 16 also indicate time.
The vertical axis in FIG. 5A indicates the traffic volume. Similarly, FIG. 5B, FIG. 5C, FIG. 5D, FIG. 6A, FIG. 6B, FIG. 7C, FIG. B), FIG. 8A, FIG. 8B, FIG. 9A, FIG. 9B, FIG. 10A, FIG. 10B, FIG. 10C, and FIG. 11 (B), FIG. 11 (C), FIG. 12, FIG. 13 (A), FIG. 13 (B), FIG. 13 (C), FIG. 14 (A), FIG. 14 (B), FIG. ), The vertical axis of FIGS. 15B and 15C also indicates the traffic volume.
In the example shown in FIG. 5A, when the “traffic amount of past communication data” as shown in FIG. 5A is obtained, the general traffic data is based on the “traffic amount of past communication data”. The “conventional determination threshold” is calculated by a statistical method. The determination target traffic amount (not shown) measured at the port is compared with the “conventional determination threshold value” shown in FIG. When the determination target traffic amount is smaller than the “conventional determination threshold value” shown in FIG. 5A, it is determined that the determination target traffic amount is in an abnormal state.
FIG. 5B is a diagram showing a data amount of “traffic amount of past communication data” necessary for calculating the “conventional determination threshold value” shown in FIG. 5A.

FIG. 5C is a diagram illustrating another example of the relationship between the traffic amount of past communication data and the conventional determination threshold value. In the example shown in FIG. 5C, based on the “traffic amount of past communication data” shown in FIG. 5C, the “conventional determination threshold” shown in FIG. Is calculated. When the determination target traffic volume is smaller than the “conventional determination threshold value” shown in FIG. 5C, it is determined that the determination target traffic volume is in an abnormal state.
FIG. 5D is a diagram showing a data amount of “traffic amount of past communication data” necessary for calculating the “conventional determination threshold value” shown in FIG. 5C.

In the example shown in FIG. 5A, the value of the “conventional determination threshold” is changed at a short time interval such as one hour. Therefore, the number of “conventional determination threshold values” calculated is large. In addition, the data amount of the “traffic amount of past communication data” necessary for calculating the “conventional determination threshold” shown in FIG. 5A is large (in the example shown in FIG. 5B, the data extraction period ta ~ Tb is long).
On the other hand, in the example shown in FIG. 5C, the value of the “conventional determination threshold” is changed at a long time interval such as 8 hours. Therefore, the number of calculated “conventional determination threshold values” is small. In addition, the data amount of the “traffic amount of past communication data” required for the calculation of the “conventional determination threshold” shown in FIG. 5C is small (in the example shown in FIG. 5D, the data extraction period tc ~ Td is short).

FIG. 6 is a diagram illustrating an example of erroneous determination that may occur in the general abnormality determination method for the determination target traffic amount illustrated in FIGS. 5 (A) and 5 (B). FIG. 6A is a diagram showing the determination target traffic amount measured at the port 270-1 shown in FIG. Similarly, FIGS. 6B and 6C are diagrams illustrating the determination target traffic amounts measured at the ports 270-2 and 270-3, respectively.
In the example illustrated in FIG. 6A, the usage method of the user U1 illustrated in FIG. 4 is not changed from the previous one. Therefore, the “determination target traffic amount” does not become smaller than the “conventional determination threshold value”. As a result, it is determined as “not abnormal” by a general determination system. Also in the example illustrated in FIG. 6C, the usage method of the user U3 illustrated in FIG. 4 is not changed from the previous one. As a result, it is determined as “not abnormal” by a general determination system.

On the other hand, in the example shown in FIG. 6B, the usage method of the user U2 shown in FIG. 4 is changed from the previous one. Further, the “determination target traffic amount” measured at the port 270-2 after the usage method of the user U2 is changed and the “conventional judgment threshold value” calculated before the usage method of the user U2 is changed. Is done. Therefore, during the period t1 to t2, the “determination target traffic amount” becomes smaller than the “conventional determination threshold value”. As a result, it is determined by the general determination system that the state is “abnormal state” during the period t1 to t2.
In the example shown in FIG. 4 and FIG. 6, no abnormality has occurred in the connection device 260 itself during the period t1 to t2. Therefore, the determination of “abnormal state” in the period t1 to t2 is an erroneous determination.
Similar to the example shown in FIG. 6B, even when the traffic volume to be judged fluctuates in a short time regularly or irregularly, for example, during the lunch break, during the nighttime switching work in the network system. There is a risk of erroneous determination by a general determination system.

FIG. 7 is a diagram illustrating another example of erroneous determination. In the example shown in FIG. 7A, the switching work in the network system is not executed at time 23:00.
On the other hand, in the example shown in FIG. 7B, switching work in the network system is executed at time 23:00. Further, the “determination target traffic amount” during the switching operation is compared with the “conventional determination threshold value” calculated based on the traffic amount of the past communication data when the switching operation is not performed. Therefore, at the time 23:00, the “determination target traffic amount” becomes smaller than the “conventional determination threshold value”. As a result, it is determined as “abnormal state” at 23:00 by a general determination system.
In the example shown in FIG. 7B, no abnormality has occurred in the network system itself at time 23:00. For this reason, the determination of “abnormal state” at time 23:00 is an erroneous determination.
For example, even when the usage method of the user changes and the “determination target traffic amount” changes in the short term, there is a possibility that a general determination system may make an erroneous determination as in the example shown in FIG. .

FIG. 8 is a diagram illustrating determination by the determination system 100 according to the first embodiment. 8A and 8B, a “learning model” is a learning model created based on the traffic volume of past communication data after the user's usage method has changed. The “traffic amount reference value” is a traffic amount reference value calculated based on the traffic amount of past communication data before the usage method of the user is changed. The “determination target traffic volume at time t01” is the determination target traffic volume measured after the user usage method has changed. The “abnormality determination threshold at time t01” indicates an outlier of the “traffic amount reference value” with respect to the “learning model at time t01”. The “abnormality determination target value at time t01” indicates an outlier of “the determination target traffic amount at time t01” with respect to “the learning model at time t01”.
In the example shown in FIG. 8A, since “the abnormality determination target value at time t01” is smaller than “the abnormality determination threshold value at time t01”, the determination unit 30 of the determination system 100 determines “not in an abnormal state”.

As described above, in the general determination system illustrated in FIG. 6B, “in an abnormal state” during the period t <b> 1 to t <b> 2 during which the value of the “determination target traffic amount” drops as the user's usage method changes. Will be misjudged.
On the other hand, in the determination system 100 according to the first embodiment, as shown in FIG. 8A, “abnormality” is also detected at time t01 when the value of “determination target traffic volume” drops as the user's usage changes. It is determined that it is not in a state. Therefore, it is possible to suppress the possibility that the traffic amount of communication data is erroneously determined as being in an abnormal state.

FIG. 8B illustrates an example in which the determination target traffic volume at time t02 after the user usage method changes is as large as that before the user usage method changes.
If the network system is normal after the usage method of the user is changed, the value of “the amount of traffic to be determined at time t02” needs to drop, as in the “learning model”. However, in the example illustrated in FIG. 8B, the value of “the amount of traffic to be determined at time t <b> 02” after the user usage method has changed is as large as that before the user usage method has changed.
Specifically, “the abnormality determination target value at time t02” is larger than “the abnormality determination threshold value at time t02”. Therefore, the determination unit 30 of the determination system 100 determines that “it is in an abnormal state”.

FIG. 9 is a diagram illustrating another example of the determination performed by the determination system 100 according to the first embodiment. FIG. 9A shows an example before execution of switching work in the network system at time 23:00. FIG. 9B shows an example after execution of switching work in the network system at time 23:00.
The “learning model” in FIG. 9A is a learning model created based on the traffic volume of past communication data before the execution of the switching work. The “learning model” in FIG. 9B is a learning model created based on the traffic volume of past communication data after execution of the switching work. The “traffic amount reference value” in FIGS. 9A and 9B is a traffic amount reference value calculated based on the traffic amount of past communication data before execution of the switching operation.
The “23:00 determination target traffic amount” in FIG. 9A is the determination target traffic amount at time 23:00 before the switching operation is executed. The “23:00 determination target traffic amount” in FIG. 9B is the determination target traffic amount at time 23:00 after the switching operation is executed.
The “23:00 abnormality determination threshold” in FIG. 9A is the “traffic amount reference value” in FIG. 9A and FIG. 9B for the “23:00 learning model” in FIG. 9A. Indicates an outlier. The “23:00 abnormality determination threshold” in FIG. 9B is the “traffic amount reference value” in FIGS. 9A and 9B with respect to the “23:00 learning model” in FIG. 9B. Indicates an outlier.
The “23:00 abnormality determination target value” in FIG. 9A corresponds to the “23:00 determination target traffic amount” in FIG. 9A with respect to the “23:00 learning model” in FIG. 9A. Indicates an outlier. The “23:00 abnormality determination target value” in FIG. 9B corresponds to the “23:00 determination target traffic amount” in FIG. 9B with respect to the “23:00 learning model” in FIG. 9B. Indicates an outlier.

In the example shown in FIG. 9A, the “23:00 abnormality determination target value” in FIG. 9A is smaller than the “23:00 abnormality determination threshold” in FIG. The determination unit 30 determines that “not abnormal”. In the example shown in FIG. 9B, the “23:00 abnormality determination target value” in FIG. 9B is smaller than the “23:00 abnormality determination threshold” in FIG. The determination unit 30 determines that “not abnormal”.
As described above, in the general determination system shown in FIG. 7B, at the time 23:00 when the value of the “determination target traffic amount” drops with the execution of the switching operation, it is erroneously described as “abnormal state”. It will be judged.
On the other hand, in the determination system 100 according to the first embodiment, as shown in FIG. 9B, at the time 23:00 when the value of “the traffic volume to be determined at 23:00” drops with the execution of the switching operation. Is determined to be “not in an abnormal state”. Therefore, it is possible to suppress the possibility that the traffic amount of communication data is erroneously determined as being in an abnormal state.

FIG. 10 is a diagram showing a calculation load for calculating a traffic volume reference value based on the traffic volume of past communication data, and a calculation load for creating a learning model by machine learning of the traffic volume of past communication data. is there.
In the example shown in FIG. 10 to which the determination system 100 of the first embodiment is applied, the traffic amount reference value calculation unit 20 is based on the statistical value of “traffic amount of past communication data” in FIG. Then, the “traffic amount reference value” in FIG. 10A is calculated. Specifically, the traffic amount reference value calculation unit 20 performs statistics on the traffic amount of past communication data over the “data extraction period tA to tB” in FIG. Based on the value, the “traffic amount reference value” in FIG. 10A is calculated.
The learning model creation unit 210 performs machine learning on the “traffic amount of past communication data” in FIG. 10C, thereby “learning model at time t001” and “learning model at time t002” in FIG. 10C. Create Specifically, the learning model creation unit 210 has the past over the “data extraction period tC to tD” in FIG. 10D, which is shorter than the “period tA to tB”, such as the last two weeks to one month. A learning model is created by machine learning of traffic volume of communication data.

In the example shown in FIG. 10 to which the determination system 100 of the first embodiment is applied, the calculation load (number of calculations) of the traffic amount reference value calculation unit 20 calculates the “traffic amount reference value” in FIG. This is a value obtained by dividing “period tA to tB” in FIG. 10B by the sampling interval of the traffic amount of the past communication data to be performed. The calculation load (number of calculations) of the learning model creation unit 210 is a value obtained by dividing “period tC to tD” in FIG. 10D by the sampling interval of the traffic amount of past communication data for creating the learning model. is there. The sampling interval of the traffic amount of the past communication data for calculating the “traffic amount reference value” in FIG. 10A is larger than the sampling interval of the traffic amount of the past communication data for creating the learning model. Set to a value. Therefore, the calculation load (number of calculations) of the traffic amount reference value calculation unit 20 is smaller than the calculation load (number of calculations) of the learning model creation unit 210. As a result, the calculation load of the traffic amount reference value calculation unit 20 can be reduced. That is, the calculation load of the determination system 100 can be reduced.
Further, the traffic amount reference value calculation unit 20 performs the “traffic” in FIG. 10A in a time unit (for example, 8 hours) longer than the time unit (for example, 5 minutes) in which the learning model generation unit 210 generates the learning model. The “quantity reference value” is calculated. Specifically, since user traffic usually has a periodicity in units of one day, as described above, the traffic amount reference value calculation unit 20 may calculate the “traffic amount reference value” in a long time unit. .
When the time unit of the learning model is set to 5 minutes, the value of the learning model created by the learning model creation unit 210 is a constant value for 5 minutes. If, for example, N times of calculations are required to create a learning model that has a constant value for 5 minutes, the number of calculations per unit time for creating a learning model is (N times / 5 minutes).
On the other hand, when the time unit of the traffic volume reference value is set to 8 hours, the traffic volume reference value calculated by the traffic volume reference value calculation unit 20 is a constant value for 8 hours. For example, when N times of calculations are required to calculate a traffic amount reference value that is constant for 8 hours, the number of calculations per unit time for calculating the traffic amount reference value is (N times / 8 hours). )become.
That is, the number of calculations per predetermined time for calculating the traffic amount reference value is smaller than the number of calculations per predetermined time for creating the learning model. Therefore, the calculation load of the traffic amount reference value calculation unit 20 can be reduced. That is, the calculation load of the determination system 100 can be reduced.

FIG. 11 is a diagram illustrating abnormality determination target values. In the example illustrated in FIG. 11A, the learning model creation unit 210 creates a “learning model” by machine learning, for example, 14 to 30 data related to the traffic volume of past communication data. The determination target traffic amount acquisition unit 13 acquires “determination target traffic amount” measured at, for example, a port. The abnormality determination target value calculation unit 230 calculates an “abnormality determination target value” that is an outlier of the “determination target traffic amount” with respect to the “learning model”.
In the example illustrated in FIG. 11B, the learning model creation unit 210 performs machine learning on the traffic amount data of communication data at, for example, 14 to 30 past times 12:00, so that “12:00” Create a learning model. The learning model creation unit 210 similarly creates a “13:00 learning model”.
In the example illustrated in FIG. 11C, the abnormality determination target value calculation unit 230 performs “12:00 learning model” and “12:00 determination target traffic amount” acquired by the determination target traffic amount acquisition unit 13. Based on the above, “12:00 abnormality determination target value” is calculated. Similarly, the abnormality determination target value calculation unit 230 calculates the “13:00 abnormality determination target value”. The abnormality determination target value calculation unit 230 calculates the abnormality determination target value, but does not determine whether or not the abnormality determination target value is abnormal.

FIG. 12 is a diagram illustrating a method for determining whether or not the “12:00 abnormality determination target value” is abnormal. In the example shown in FIG. 12, the traffic volume reference value calculation unit 20 calculates the “traffic volume reference value” for the time period including time 12:00 based on the traffic volume data of less than 14 past communication data, for example. calculate. The “traffic amount reference value at 12:00” is equal to the “traffic amount reference value” in the time zone including the time 12:00.
The abnormality determination threshold value calculation unit 220 calculates the “12:00 abnormality determination threshold value” based on the “12:00 traffic amount reference value” and the “12:00 learning model”. The determination unit 30 compares the “12:00 abnormality determination target value” with the “12:00 abnormality determination threshold value”.
In the example illustrated in FIG. 12, since “12:00 abnormality determination target value” is equal to or less than “12:00 abnormality determination threshold value”, the determination unit 30 determines that there is no abnormal state at 12:00.

FIG. 13 is a diagram comparing the “12:00 abnormality determination threshold” and the “13:00 abnormality determination threshold”. In the example shown in FIG. 13B, time 12:00 and time 13:00 are included in a time zone in which the “traffic amount reference value” is a constant value. Therefore, “12:00 traffic volume reference value” and “13:00 traffic volume reference value” are equal.
In the example shown in FIG. 13A, the value (traffic amount) of “13:00 learning model” is larger than the value of “12:00 learning model”. Therefore, in the example shown in FIG. 13C, the “13:00 abnormality determination threshold” is larger than the “12:00 abnormality determination threshold”. As a result, in the example illustrated in FIG. 13, it is more difficult to determine that the state is abnormal at time 13:00 than at time 12:00.

FIG. 14 is a diagram illustrating an example of a traffic amount reference value calculation method. The horizontal axis of FIG. 14B shows the distribution. Similarly, the horizontal axis in FIG. 15B also shows the distribution.
In the example shown in FIG. 14, the traffic amount reference value is calculated so that the traffic amount reference value in the time period shown in FIG. Specifically, in the example illustrated in FIG. 14, the traffic volume reference value calculation unit 20 performs the process illustrated in FIG. 14B based on the “traffic volume of past communication data” in the time period illustrated in FIG. Calculate the distribution shown. Further, the traffic volume reference value calculation unit 20 calculates the traffic volume of the A percentile (A is a predetermined value) of the distribution shown in FIG. The traffic amount reference value calculation unit 20 calculates a “traffic amount reference value” by multiplying the traffic amount by a coefficient B (B is a value smaller than 1).

FIG. 15 is a diagram illustrating another example of a method for calculating a traffic amount reference value. In the example shown in FIG. 15, the traffic amount reference value is calculated so that the traffic amount reference value in the time period shown in FIGS. 15A and 15C becomes a constant value. Specifically, in the example illustrated in FIG. 15, the traffic volume of past communication data as illustrated in FIG. 15A is measured at a measurement point of the determination target traffic volume such as a port. The traffic amount reference value calculation unit 20 calculates the distribution shown in FIG. 15B based on the “traffic amount of past communication data (number of transmitted packets)” in the time zone shown in FIG. The traffic amount reference value calculation unit 20 sets a point at which the appearance rate is low (for example, 99.5% or less) as the first provisional value α. Specifically, the traffic amount reference value calculation unit 20 performs the following calculations (1) to (6).
(1) The traffic amount reference value calculation unit 20 calculates the standard deviation σ of the number of transmitted packets.
(2) The traffic amount reference value calculation unit 20 calculates a histogram shown in FIG. 15B, for example, with 0.001σ as one class.
(3) The traffic amount reference value calculation unit 20 calculates the appearance probability for each class.
(4) The traffic amount reference value calculation unit 20 adds the appearance probabilities sequentially from the largest number of transmission packets, and calculates, for example, the maximum value (first provisional value) α of the class exceeding 99.5%.
(5) In order to remove an abnormal value from the graph, the traffic amount reference value calculation unit 20 excludes transmission packets that are less than 0.4α, and performs the calculations (1) to (4) again.
(6) The traffic amount reference value calculation unit 20 sets the recalculated value as the second provisional value β shown in FIG. 15C, and the second provisional value β is x% (x% is smaller than 1). ) To calculate the “traffic amount reference value” shown in FIG.

FIG. 16 is a diagram illustrating the relationship between the abnormality determination target value and the abnormality determination threshold value. The vertical axis in FIG. 16 indicates the abnormality determination target value and the abnormality determination threshold value. In the example illustrated in FIG. 16, the determination unit 30 determines whether or not the abnormality determination target value is greater than the abnormality determination threshold at predetermined time intervals.
In the example illustrated in FIG. 16, at time t1, the determination unit 30 first determines that the abnormality determination target value is greater than the abnormality determination threshold. An abnormality notifying unit (not shown) of the network system 200 to which the determination system 100 of the first embodiment is applied executes a warning.
At time t3 when the determination after Xa times (for example, twice) is performed by the determination unit 30, the determination unit 30 outputs an alert to the log. Similarly, the determination unit 30 outputs an alert to the log every determination of Xb times (for example, 6 times), that is, at times t9 and t15.
At time t21 when the determination unit 30 determines that the abnormality determination target value is equal to or less than the abnormality determination threshold value Xc times (for example, 5 times), the abnormality notification unit determines that the network system 200 has been restored.

FIG. 17 is a diagram illustrating another example of the network system 200 to which the determination system 100 according to the first embodiment can be applied. In the example illustrated in FIG. 17, the network system 200 includes connection devices 260-1 and 260-2. The connection device 260-1 connects the VLANs 250-1, 250-2, and 250-3, and the connection device 260-2 connects the VLANs 250-4, 250-5, and 250-6. The connection device 260-1 has ports 270-1, 270-2, 270-3, 270-4, and the connection device 260-2 has ports 270-5, 270-6, 270-7, 270-8. .
In the example illustrated in FIG. 17, the determination unit 30 determines that the determination target traffic amount of the port 270-1 is in an abnormal state and the determination target traffic amounts of the ports 270-2, 270-3, and 270-4 are not in an abnormal state. judge. The network analysis unit 40 analyzes that the abnormal state of the determination target traffic amount of the port 270-1 is caused by the user U1. Further, the network analysis unit 40 analyzes that the connection device 260-1 is in a normal state.
In the example illustrated in FIG. 17, the determination unit 30 determines that the determination target traffic volume of the ports 270-5, 270-6, 270-7, and 270-8 is in an abnormal state. The network analysis unit 40 analyzes that the connection device 260-2 is in an abnormal state.
Specifically, when a silent failure occurs in the connection devices 260-1 and 260-2, the ports 270-1 to 270-4 in the connection device 260-1 and the port 270- in the connection device 260-2. The determination unit 30 determines whether each of 5 to 270-8 is in an abnormal state. Thereby, the network analysis unit 40 can analyze which ports 270-1 to 270-8 are in an abnormal state. Further, the alarm analysis unit 50 can analyze whether it is determined as an abnormal state due to a silent failure or whether it is determined as an abnormal state due to a normal failure. .

That is, in the example illustrated in FIG. 17, the determination target traffic volume is a determination target in the network system 200 including a plurality of VLANs 250-1 to 250-6 and connection devices 260-1 and 260-2 connecting them. This is the traffic volume of communication data. Further, the network analysis unit 40 analyzes whether the cause of the abnormal state is in each of the VLANs 250-1 to 250-6 or in the connection devices 260-1 and 260-2.
In the example illustrated in FIG. 17, the determination target traffic amount is the traffic amount of the determination target communication data in each of the plurality of ports 270-1 to 270-8 included in the network system 200. Further, the determination unit 30 determines whether or not the determination target traffic amount in each of the plurality of ports 270-1 to 270-8 is in an abnormal state.

FIG. 18 is a flowchart showing an overall processing flow by the determination system 100.
(Step S101) The learning model acquisition unit 10 acquires a learning model created by machine learning of the traffic amount of past communication data.
(Step S102) The traffic amount reference value acquisition unit 11 acquires a traffic amount reference value that is a reference value of the traffic amount of communication data set based on the traffic amount of past communication data.
(Step S103) The abnormality determination threshold value acquisition unit 12 acquires an abnormality determination threshold value that is an outlier of the traffic amount reference value for the learning model.
(Step S104) The determination target traffic amount acquisition unit 13 acquires a determination target traffic amount that is a traffic amount of communication data to be determined.
(Step S105) The abnormality determination target value acquisition unit 14 acquires an abnormality determination target value that is an outlier of the determination target traffic amount with respect to the learning model.
(Steps S106, S107, S108) The determination unit 30 determines whether or not there is an abnormal state based on the abnormality determination threshold value and the abnormality determination target value.
Specifically, the determination unit 30 determines whether or not the abnormality determination target value is larger than the abnormality determination threshold value (step S106). When the abnormality determination target value is larger than the abnormality determination threshold value, the determination unit 30 determines that the state is abnormal (step S107). When the abnormality determination target value is equal to or less than the abnormality determination threshold value, the determination unit 30 determines that there is no abnormal state (step S108).

[Summary of First Embodiment]
As described above, the determination system 100 is set based on the learning model acquisition unit 10 that acquires a learning model created by machine learning of the traffic amount of past communication data, and the traffic amount of past communication data. A traffic volume reference value acquisition unit 11 that acquires a traffic volume reference value that is a traffic volume reference value of the communication data that has been received, and an abnormality determination threshold that acquires an abnormality determination threshold that is an outlier of the traffic volume reference value for the learning model Unit 12, a determination target traffic amount acquisition unit 13 that acquires a determination target traffic amount that is the traffic amount of communication data to be determined, and an abnormality that acquires an abnormality determination target value that is an outlier of the determination target traffic amount for the learning model Whether or not the determination target value acquisition unit 14 is in an abnormal state based on the abnormality determination threshold value and the abnormality determination target value Determining comprises a determination unit 30, a.
In the determination system 100, the determination unit 30 determines that the abnormality determination target value is abnormal when the abnormality determination target value is larger than the abnormality determination threshold.
The determination system 100 further includes a traffic amount reference value calculation unit 20 that calculates a traffic amount reference value based on a statistical value of the traffic amount of past communication data, and the traffic amount reference value acquisition unit 11 includes a traffic amount reference value. The traffic amount reference value calculated by the value calculation unit 20 is acquired.
Thereby, the determination system 100 can suppress a possibility of being erroneously determined as being in an abnormal state.

Further, in the determination system 100, the traffic volume reference value calculation unit 20 calculates the traffic volume reference value based on the statistical value of the traffic volume of the past communication data over the first period tA to tB. A traffic amount of past communication data is generated by machine learning of a traffic amount of past communication data over a second period tC to tD shorter than one period tA to tB, and is used to calculate a traffic amount reference value. The first calculation load, which is a value obtained by dividing the first period tA to tB by the sampling interval, sets the second period tC to tD according to the sampling interval of the traffic amount of the past communication data for creating the learning model. The divided value is smaller than the second calculation load.
In the determination system 100, the traffic volume reference value calculation unit 20 calculates the traffic volume reference value in a time unit longer than the time unit in which the learning model is created, and per predetermined time for calculating the traffic volume reference value. Is less than the number of calculations per predetermined time for creating a learning model.
Thereby, the determination system 100 can reduce the calculation load of the traffic amount reference value calculation unit 20 as compared with the case where the first calculation load is larger than the second calculation load.

Further, in the determination system 100, the determination target traffic volume is the traffic volume of communication data to be determined in a network system including a plurality of VLANs and connection devices that connect the plurality of VLANs. A network analysis unit 40 is further provided for analyzing whether the cause of the state is in each VLAN or in the connection device.
Accordingly, the determination system 100 can accurately analyze whether the cause of the abnormal state is in each VLAN or in the connection device.

In the determination system 100, the determination target traffic amount is the traffic amount of communication data to be determined at each of the plurality of ports included in the network system, and the determination unit 30 determines the determination target traffic amount at each of the plurality of ports. It is determined whether or not is in an abnormal state.
Thereby, the determination system 100 can accurately determine whether or not the determination target traffic volume of the port in the network system is in an abnormal state.

In the determination system 100, the determination target traffic amount acquisition unit acquires the determination target traffic amount from the aggregation switch included in the network system, and the learning model uses the traffic amount of the past communication data acquired from the aggregation switch as a machine. Created by learning.
Thereby, the determination system 100 can accurately determine whether or not the determination target traffic amount acquired from the aggregation switch in the network system is in an abnormal state.
The above is the description of the first embodiment.

[Second Embodiment]
FIG. 19 is a diagram illustrating an example of a network system 200 to which the determination system 100 according to the second embodiment is applied. The determination system 100 according to the second embodiment can achieve the same effects as those of the determination system 100 according to the first embodiment described above, except for points described below.
In the example shown in FIG. 1 to which the determination system 100 of the first embodiment is applied, the traffic amount reference value calculation unit 20 is provided inside the determination system 100. On the other hand, in the example shown in FIG. 19 to which the determination system 100 of the second embodiment is applied, the traffic amount reference value calculation unit 240 having the same function as the traffic amount reference value calculation unit 20 shown in FIG. 100 is provided outside.
In the determination system 100 of the second embodiment, the calculation load of the determination system 100 can be reduced as compared with the determination system 100 of the first embodiment.

[Modification]
The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to the above-described embodiment, and includes a design and the like within a scope not departing from the gist of the present invention. For example, the configurations described in the first and second embodiments described above can be arbitrarily combined. In addition, for example, each configuration described in the first and second embodiments described above can be omitted if it is not necessary to exhibit a specific function. Further, for example, each configuration described in the first and second embodiments described above can be arbitrarily separated and provided in a separate device.

  In order to realize the functions of the learning model acquisition unit 10, the traffic amount reference value acquisition unit 11, the abnormality determination threshold acquisition unit 12, the determination target traffic amount acquisition unit 13, the abnormality determination target value acquisition unit 14, and the determination unit 30. Are recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed to thereby execute a learning model acquisition unit 10, a traffic amount reference value acquisition unit 11, an abnormality determination threshold value You may perform the process as the acquisition part 12, the determination target traffic amount acquisition part 13, the abnormality determination target value acquisition part 14, and the determination part 30. FIG. Here, “loading and executing a program recorded on a recording medium into a computer system” includes installing the program in the computer system. The “computer system” here includes an OS and hardware such as peripheral devices. Further, the “computer system” may include a plurality of computer devices connected via a network including a communication line such as the Internet, WAN, LAN, and dedicated line. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. As described above, the recording medium storing the program may be a non-transitory recording medium such as a CD-ROM. The recording medium also includes a recording medium provided inside or outside that is accessible from the distribution server in order to distribute the program. The code of the program stored in the recording medium of the distribution server may be different from the code of the program that can be executed by the terminal device. That is, the format stored in the distribution server is not limited as long as it can be downloaded from the distribution server and installed in a form that can be executed by the terminal device. Note that the program may be divided into a plurality of parts, downloaded at different timings, and combined in the terminal device, or the distribution server that distributes each of the divided programs may be different. Furthermore, a “computer-readable recording medium” holds a program for a certain period of time, such as a volatile memory (RAM) inside a computer system that becomes a server or client when the program is transmitted via a network. Including things. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 10 ... Learning model acquisition part, 11 ... Traffic amount reference value acquisition part, 12 ... Abnormality determination threshold value acquisition part, 13 ... Determination object traffic amount acquisition part, 14 ... Abnormality determination object value acquisition part, 20 ... Traffic amount reference value calculation part , 30 ... determination unit, 40 ... network analysis unit, 50 ... alarm analysis unit, 100 ... determination system, 200 ... network system, 200-1, 200-2, 200-3 ... network, 210 ... learning model creation unit, 220 ... abnormality determination threshold value calculation unit, 230 ... abnormality determination target value calculation unit, 240 ... traffic amount reference value calculation unit, 250-1, 250-2, 250-3, 250-4, 250-5, 250-6 ... VLAN , 260, 260-1, 260-2, 260-3 ... connection device, 270-1, 270-2, 270-3, 270-4, 270-5, 270-6, 70-7,270-8 ... port, 280 ... aggregation switch, 280a ... mirror port, 290 ... traffic collection server

Claims (10)

A learning model acquisition unit for acquiring a learning model created by machine learning of traffic volume of past communication data;
A traffic volume reference value acquisition unit that acquires a traffic volume reference value that is a reference value of the traffic volume of communication data set based on the traffic volume of the past communication data;
An abnormality determination threshold value acquisition unit that acquires an abnormality determination threshold value that is an outlier of the traffic amount reference value for the learning model;
A determination target traffic amount acquisition unit that acquires a determination target traffic amount that is a traffic amount of communication data to be determined;
An abnormality determination target value acquisition unit that acquires an abnormality determination target value that is an outlier of the determination target traffic amount with respect to the learning model;
A determination unit that determines whether or not an abnormal state is present based on the abnormality determination threshold and the abnormality determination target value;
A determination system comprising: The determination unit determines that the abnormality determination target value is in the abnormal state when the abnormality determination target value is greater than the abnormality determination threshold.
The determination system according to claim 1. A traffic amount reference value calculating unit that calculates the traffic amount reference value based on a statistical value of the traffic amount of the past communication data;
The traffic amount reference value acquisition unit acquires the traffic amount reference value calculated by the traffic amount reference value calculation unit.
The determination system according to claim 1 or 2. The traffic amount reference value calculation unit calculates the traffic amount reference value based on a statistical value of the traffic amount of the past communication data over a first period,
The learning model is created by machine learning the traffic amount of the past communication data over a second period shorter than the first period,
The first calculation load, which is a value obtained by dividing the first period by the sampling interval of the traffic volume of the past communication data for calculating the traffic volume reference value,
Less than the second calculation load, which is a value obtained by dividing the second period by the sampling interval of the traffic amount of the past communication data for creating the learning model,
The determination system according to claim 3. The traffic amount reference value calculation unit calculates the traffic amount reference value in a unit of time longer than the unit of time in which the learning model is created,
The number of calculations per predetermined time for calculating the traffic amount reference value is less than the number of calculations per predetermined time for creating the learning model,
The determination system according to claim 3 or claim 4. The determination target traffic amount is a traffic amount of the determination target communication data in a network system including a plurality of VLANs (Virtual Local Area Networks) and a connection device connecting the plurality of VLANs.
The determination system according to any one of claims 1 to 5, further comprising a network analysis unit that analyzes whether the cause of the abnormal state is in each VLAN or in the connection device. The determination target traffic amount is a traffic amount of the determination target communication data in each of a plurality of ports included in the network system,
The determination unit determines whether the determination target traffic amount in each of the plurality of ports is in the abnormal state;
The determination system according to claim 6. The determination target traffic amount acquisition unit acquires the determination target traffic amount from an aggregation switch included in the network system,
The learning model is created by machine learning the traffic amount of the past communication data acquired from the aggregation switch.
The determination system according to claim 6 or 7. Judgment system
A first step of acquiring a learning model created by machine learning of traffic volume of past communication data;
A second step of obtaining a traffic volume reference value that is a reference value of the traffic volume of communication data set based on the traffic volume of the past communication data;
A third step of obtaining an abnormality determination threshold value that is an outlier of the traffic amount reference value for the learning model;
A fourth step of acquiring a determination target traffic amount that is a traffic amount of communication data to be determined;
A fifth step of acquiring an abnormality determination target value that is an outlier of the determination target traffic amount with respect to the learning model;
A sixth step of determining whether or not an abnormal state is present based on the abnormality determination threshold and the abnormality determination target value;
A determination method including On the computer,
A first step of acquiring a learning model created by machine learning of traffic volume of past communication data;
A second step of obtaining a traffic volume reference value that is a reference value of the traffic volume of communication data set based on the traffic volume of the past communication data;
A third step of obtaining an abnormality determination threshold value that is an outlier of the traffic amount reference value for the learning model;
A fourth step of acquiring a determination target traffic amount that is a traffic amount of communication data to be determined;
A fifth step of acquiring an abnormality determination target value that is an outlier of the determination target traffic amount with respect to the learning model;
A sixth step of determining whether or not an abnormal state is present based on the abnormality determination threshold and the abnormality determination target value;
A program for running

Images