JP2009200993A - Failure detecting apparatus, failure detection method, and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To shorten the processing time required for the detection of failure. <P>SOLUTION: A failure detector includes a receiving part for receiving data flowing through a communication path that is a monitored object; an extraction part for extracting the values of the monitored items from data received by the receiving part; a decision part for deciding frequently appearing values, from the values of the monitored items; and a detecting part for performing failure detection, decided based on data from which values of monitored items which are decided as being extracted frequently. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an abnormality detection device, an abnormality detection method, and a computer program.

  Conventionally, an anomaly detection device that monitors data flowing through a data communication path and detects an anomaly has been proposed. As an abnormality detection device, a device that detects an abnormality for each packet (see Patent Document 1) and a device that detects an abnormality for each TCP (Transmission Control Protocol) flow (see Patent Document 2) have been proposed. Such an anomaly detection device detects an anomaly in the data communication path by using a predetermined rule (signature) and statistical values (average, standard deviation, etc.) relating to communication.

  The time required for the processing of such an abnormality detection device depends on the traffic on the data communication path to be monitored. For this reason, there has been a problem that the time required for the detection process increases with a significant increase in communication volume in recent years. For example, when the communication amount becomes 10 times, the amount of data to be processed becomes 10 times. In extreme cases, processing may not be completed in real time. To solve such a problem, a technique has been proposed in which a plurality of abnormality detection devices are used to perform distributed data abnormality detection processing to improve the performance of the entire system (see Patent Document 3).

JP 2002-073433 A Japanese Patent Publication No. 2005-050935 JP 2007-323384 A

However, since the system described above uses a plurality of abnormality detection devices, the cost increases.
In view of the above circumstances, an object of the present invention is to provide an abnormality detection device, an abnormality detection method, and a computer program that can shorten the processing time required for abnormality detection.

1st aspect of this invention is an abnormality detection apparatus, Comprising: The receiving part which receives the data which flow through the communication channel used as monitoring object, and extraction which extracts the value of a monitoring item from the data received by the recording part , A determination unit that determines a frequently occurring value from among the values of the monitoring item, and a detection unit that performs abnormality detection determination based on data obtained by extracting the value of the monitoring item determined to be frequent And comprising.
In the first aspect of the present invention configured as described above, the detection unit performs the abnormality detection determination based on the data related to the value determined by the determination unit as frequently appearing about the value of the monitoring item. For this reason, the amount of data to be processed by the detection unit for abnormality detection determination is reduced compared to the total amount of data passing through the communication path. Therefore, the abnormality detection device enables the detection unit to shorten the processing time required for the abnormality detection determination process.

  In the first aspect of the present invention, the extraction unit further extracts detection information used by the detection unit for abnormality detection determination from the data received by the reception unit, and determines that the detection unit is frequently appearing. An abnormality detection determination may be made based on detection information extracted from data obtained by extracting the values of the monitored items.

  According to a first aspect of the present invention, for each value of a monitoring item, a statistical value calculation unit that calculates a statistical value related to detection information extracted from the data from which the value of the monitoring item is extracted; And a storage unit that stores the statistical values in association with each other, and the detection unit is abnormal based on the statistical values stored in the storage unit in association with the values of the monitoring items determined to be frequently appearing. It may be configured to perform detection determination.

  The determination unit according to the first aspect of the present invention has a queue for performing first-in first-out of data, sequentially inputs values of monitoring items extracted by the extraction unit to the queue, and stores the monitoring items stored in the queue at a predetermined timing The value of the item may be determined as a frequently appearing value.

The determination unit according to the first aspect of the present invention has a queue for performing first-in first-out of data, sequentially inputs values of monitoring items extracted by the extraction unit to the queue, and stores each value stored in the queue at a predetermined timing. It may be configured to acquire the number of values of the monitoring item and determine a frequently occurring value based on this number.
The present invention may be specified as a computer program for operating a computer as the above-described abnormality detection device. The present invention may also be specified as an abnormality detection method performed by a computer that functions as the above-described abnormality detection device.

  According to the present invention, it is possible to shorten the processing time required for abnormality detection.

  FIG. 1 is a schematic block diagram illustrating a functional configuration of the abnormality detection device 1. The abnormality detection device 1 includes a reception unit 11, an extraction unit 12, a determination unit 13, a storage unit 14, a detection unit 15, and an output unit 16. The abnormality detection device 1 can be configured using an information processing device that is connected to a computing device or a storage device via a bus and operates based on a program stored in the storage device. Each functional unit of the abnormality detection device 1 or a part thereof may be configured using dedicated hardware. Hereinafter, each function part with which the abnormality detection apparatus 1 is provided is demonstrated using FIG.

  The receiving unit 11 receives data (for example, IP packets) flowing through this communication path from the data communication path to be monitored. The receiving unit 11 has a configuration according to the protocol of the communication path of data to be monitored. For example, if the data communication path to be monitored is a LAN (Local Area Network), the receiving unit 11 can be connected to the LAN (for example, a network interface for connecting a LAN cable and converting a signal into data). Etc.). If the data communication path to be monitored is a WAN (Wide Area Network), the receiving unit 11 can be connected to the WAN (for example, a network interface for connecting a fiber optic cable and converting a signal into data). Etc.). As described above, the configuration of the reception unit 11 is appropriately changed according to the communication path of the data to be monitored by the abnormality detection device 1. The receiving unit 11 transfers the received data to the extracting unit 12.

  The extraction unit 12 extracts the value of the monitoring item and the detection information from the received data. Further, the extraction unit 12 transfers the extracted monitoring item value and detection information value to the determination unit 13.

A monitoring item is an item included in data received from a communication path to be monitored. The monitoring items are set as appropriate according to the communication path to be monitored and the type of abnormality to be detected.
The monitoring items include, for example, a transmission destination IP address of the IP packet (transmission destination host identification information), a transmission source IP address of the IP packet (transmission source host identification information), a used protocol, a TCP / IP port number, and the like.

The detection information is information relating to data received from the communication path to be monitored. The detection information is appropriately set according to the communication path to be monitored, the type of abnormality to be detected, and the abnormality detection technology installed in the detection unit 15.
For example, when a communication path to be monitored is a communication path connected to a DNS server and an abnormality occurring in the DNS server is detected, the detection information is a value indicating whether it is a query or a response, a query Type (eg, A, PTR, MX, NS, etc.), response type (eg, A, PTR, MX, NS, etc.), response code (0, 1, 2, 3, 4, 5, etc.).

  In addition, for example, when a communication path to be monitored is a communication path connected to a WEB server and an abnormality occurring in the WEB server is detected, the detection information is a value indicating whether it is a request or a response. , Method (GET, HEAD, POST, etc.), request line length, status code, header field-name, header field-name length, header field-value length, header field-content Such as length.

The determination unit 13 determines a frequently appearing value from the extracted monitoring item values. The determination unit 13 determines a frequently occurring value by using a FIFO (First In First Out) queue. In this case, the determination unit 13 has a queue with a predetermined length (for example, a queue with a length of 10,000, 100,000, etc.), and inputs the value transferred from the extraction unit 12 to this queue as needed. Then, the determination unit 13 determines the value existing in the queue at that time as a frequently appearing value at a predetermined timing after repeating this input. Then, the determination unit 13 notifies the detection unit 15 of the value (monitor item value) determined to be frequently appearing.
The predetermined timing may be a time point when a predetermined time has elapsed, or may be a time point when the above-described input is repeatedly executed a predetermined number of times. This predetermined timing is appropriately set by the designer.

  2A to 2E are diagrams illustrating specific examples of processing of the determination unit 13. Hereinafter, the processing of the determination unit 13 will be described in detail with reference to FIG. The determination unit 13 has a queue and a counter. The queue is n in length. That is, the queue has n boxes (box 1 to box n), and the values input to the queue are stored in order from box 1. When a new value is input to the queue, the newly input value is stored in box 1 and the value stored in each box is stored in a box having a box number one larger. At this time, the queue discards the value stored in the box n.

  The counter counts the number of each value present in the queue. When the queue enters a new value in box 1, the counter increments the counter value of this value by one. When the counter discards the value stored in the box n, the counter decreases the counter value by one. As the counter operates in this manner, each counter value included in the counter indicates the number of values existing in the queue.

  In FIG. 2 (a), the queue inputs the first value A 1 and places it in box 1. At this time, the counter counts “1” as the counter value of A1. Next, in FIG. 2 (b), the queue inputs the next value A 2 and puts this value in box 1. At this time, the queue stores the value A1 stored in the box 1 in the box 2. At this time, the counter does not change the counter value of A1, and counts “1” as the counter value of A2. Next, in FIG. 2 (c), the queue inputs the next value A 1 and puts this value in box 1. At this time, the queue stores the values A2 and A1 stored in box 1 and box 2 in box 2 and box 3, respectively. At this time, the counter increments the counter value of A1 by one to “2” and does not change the counter value of A2.

FIG. 2D shows a state in which values are entered up to box n as a result of the queue repeating the input process thereafter. At this time, the queue stores the value A1 in the box n. The counter has “x” as the counter value of A1 and “y” as the counter value of A2.
Next, in FIG. 2E, the queue receives a new value A2 from the state of FIG. That is, the queue inputs the next value A2 and puts this value in box 1. Therefore, the counter increments the counter value of A2 by 1 to “y + 1”. The queue discards the value A1 stored in the box n and stores the value A2 stored in the box n-1 until then in the box n. Then, the counter decrements the counter value of the discarded value A1 by one to “x−1”. By such counter operation, the determination unit 13 acquires the number of values stored in the queue. Then, the determination unit 13 determines a value that is 1 or more in the queue as a frequently appearing value.

Further, the determination unit 13 calculates a statistical value of the detection information for each monitoring item value. Then, the determination unit 13 writes the calculated statistical value in the storage unit 14 in association with each monitoring item value. The statistical value of the detection information is a value used for the abnormality detection process of the detection unit 15. The statistical value calculated by the determination unit 13 is appropriately set according to the abnormality detection technology implemented in the detection unit 15.
This statistical value is, for example, the total number of the same information, the average of the total number of the same information obtained every predetermined time, the standard deviation of the total number of the same information obtained every predetermined time, or the predetermined number of received data The average of the total number of identical information obtained every time, the standard deviation of the total number of identical information obtained every predetermined number of data receptions, and the like. When the detection information has a data length, the statistical value may be an average or standard deviation of the detection information. The “predetermined time” and the “predetermined number of received data” when calculating the statistical value are shorter than the “predetermined timing” in the determination unit 13.

  The storage unit 14 is configured using a volatile storage device or a nonvolatile storage device. FIG. 3 is a diagram showing the contents stored in the storage unit 14. The storage unit 14 stores a statistical value of each detection information value for each monitoring item value in accordance with an instruction from the determination unit 13.

The detection unit 15 receives a notification from the determination unit 13 regarding the value of the monitoring item that is determined to appear frequently. And the detection part 15 performs abnormality detection judgment based on the data from which the value of this monitoring item was extracted. Specifically, the detection unit 15 performs abnormality detection determination based on the detection information extracted in the data from which the value of the monitoring item is extracted.
The detection unit 15 associates the detection information extracted in the data from which the value of the monitoring item determined to be frequently appearing with the value of the monitoring item, and stores the statistical value stored in the storage unit 14. Get by reading. And the detection part 15 performs abnormality detection judgment based on this statistical value. The detection unit 15 performs an abnormality detection process based on a known network monitoring technique or a known server monitoring technique. For example, the detection unit 15 has a threshold for each statistical value, and compares each statistical value stored in the storage unit 14 with the threshold in association with the value of the monitoring item. And the detection part 15 judges that abnormality has generate | occur | produced when there exists a statistical value exceeding a threshold value. If the detection unit 15 determines that an abnormality has occurred, the detection unit 15 transfers information such as a statistical value used in the determination process and a value of a monitoring item corresponding to the statistical value to the output unit 16.

  The output unit 16 outputs the detection result of the detection unit 15. Specifically, the output unit 16 outputs, as the detection result, the value of the monitoring item, the detection information, the statistical value of the detection information, and the like determined to be abnormal by the detection unit 15. For example, the output unit 16 outputs the detection result by displaying the detection result on the image display device. Further, the output unit 16 may perform the output by printing the detection result on a sheet, for example. The output unit 16 may output the detection result by writing the detection result to the storage device, for example.

  Next, the operation of the determination unit 13 will be described. FIG. 4 is a flowchart showing the operation of the determination unit 13. First, the determination unit 13 resets the queue and the counter (S01) and starts measuring time (S02). Next, the determination unit 13 determines whether or not a predetermined time has elapsed from the start of timing (S03). If the predetermined time has not yet elapsed (S03-NO), the determination unit 13 determines whether or not the queue box n is empty (S04). When the box n is not empty (S04-NO), the determination unit 13 subtracts 1 from the counter value related to the value of the box n (S09). Next, the determination unit 13 discards the value stored in the box n (S10). Next, the determination unit 13 inputs the value of the monitoring item newly extracted by the extraction unit 12 into the queue (S05). When the box n is empty (S04-YES), the determination unit 13 performs the processes after S05 without performing the processes of S09 and S10.

  Next, the determination unit 13 adds “1” to the counter value of the value of the newly input monitoring item (that is, the value stored in the box 1) (S06). Then, the determination unit 13 updates the statistical value of the detection information value corresponding to the newly input monitoring item value (S07). The determination unit 13 determines whether there is an instruction to end the process (S08). If there is no instruction to end the process (S08-NO), the determination unit 13 returns to the process of S03 and performs the processes after S03. On the other hand, when there is an instruction to end the process (S08-YES), the determination unit 13 ends the process.

  When the predetermined time has passed in the branch of S03 (S03-YES), the determination unit 13 determines that the value of the monitoring item whose counter value is 1 or more is a frequently appearing value (S11). . Next, the determination unit 13 resets the time measurement (S12). And the judgment part 13 performs the process after S01.

  In the abnormality detection device 1, the detection unit 15 makes an abnormality detection determination only for the data related to the value determined by the determination unit 13 as being frequently generated for the value of the monitoring item. For this reason, the amount of data to be processed by the detection unit 15 for abnormality detection determination is reduced compared to the total amount of data passing through the communication path. Therefore, the abnormality detection device 1 enables the detection unit 15 to shorten the processing time required for the abnormality detection determination process.

  Also, the conventional abnormality detection device has a problem of occurrence of erroneous detection that erroneously detects a normal state as an abnormality. In order to deal with such a problem, in the abnormality detection device 1, as described above, the detection unit 15 makes an abnormality detection determination only for the data related to the value that the determination unit 13 determines to be frequent for the value of the monitoring item. . In general, data relating to frequently occurring values often has a significant influence. Conversely, data relating to values that do not occur frequently (in other words, values that occur rarely) often do not cause anomalies. Therefore, the abnormality detection device 1 can suppress the occurrence of erroneous detection by not performing abnormality detection determination on data related to values that do not appear frequently.

  In addition, the amount of data output from the abnormality detection apparatus increases in accordance with the amount of processing for abnormality detection determination. For this reason, conventionally, output data has been increased by including unnecessary detection results (in other words, detection results for minor abnormalities that do not have a significant influence). Therefore, the administrator of the communication path and device has to monitor a huge amount of output data, and there has been a problem that it cannot cope with it. In response to such a problem, the abnormality detection device 1 does not output an unnecessary detection result by not performing abnormality detection determination on data related to a value that does not occur frequently as described above, and an output that should be monitored by the administrator. Data can be reduced.

<Modification>
The extraction unit 12 may operate by using a unit for extracting the value of the monitoring item as a unit different from that for each packet, such as for each TCP flow instead of for each packet.
The determination unit 13 may determine a frequently appearing value by using an algorithm different from the FIFO. For example, the determination unit 13 counts the number of each value transferred from the extraction unit 12 a predetermined number of times. Then, the determination unit 13 may determine that a predetermined number of values are frequently output in order from the largest count value. Further, for example, the determination unit 13 may determine that the count value exceeds a threshold value as a frequently appearing value. Further, for example, the determination unit 13 may determine a frequently occurring value by using LRU (Least Recently Used). Specifically, the determination unit 13 prepares a plurality of boxes, stores the value transferred from the extraction unit 12 (a value to be newly input) in an empty box, and refers to this value at this point. Record. When the value to be newly input already contains the same value in any of the boxes, the determination unit 13 records that the value is referred to at that time, and puts the value in the new box. do not do. Further, when a new value that is not contained in any box is input in a state where all boxes contain values, the determination unit 13 discards a value that has not been referenced for the longest time. Then, the determination unit 13 stores a new value in the box vacated by the discarding and records that it has been referred to at this time. After repeating such input a predetermined number of times, the determination unit 13 determines each value stored in the box at that time as a frequently appearing value.

Further, the determination unit 13 may update the statistical value of the detection information every time a newly extracted monitoring item value is input to the queue, but at a predetermined timing at which the statistical value is obtained. good.
Further, the determination unit 13 may determine a frequently occurring value based on the count value of each value existing in the queue at a predetermined timing after repeating the input to the queue. For example, when the count value is greater than or equal to a threshold value, the determination unit 13 may determine that value as a frequently occurring value. In addition, for example, the determination unit 13 may determine a predetermined number of values from the top in order of increasing count value as frequently appearing values.

The determination unit 13 may not acquire the statistical value of the detection information, and the detection unit 15 may acquire the statistical value of the detection information as necessary. In this case, the storage unit 14 may store only the detection information without storing the statistical value of the detection information.
Further, the detection information may overlap with the monitoring item. That is, the detection information may include a packet transmission destination IP address, a transmission destination host name, a transmission source IP address, a transmission source host name, a protocol used, a port number, and the like.

  Further, the monitoring item may be a combination of a plurality of items. For example, the monitoring item may be a combination of a packet transmission destination IP address and a protocol used. In this case, the extraction unit 12 extracts this combination. Moreover, the determination part 13 determines the combination which appears frequently about this combination.

  The determination unit 13 may calculate these new values using Equations 1 to 3 when updating the average value or the standard deviation, which are statistical values of the detection information. In Equations 1 to 3, A (n) represents a new average value, Q (n) represents a new mean square value, and S (n) represents a new standard deviation. A (n-1) is the latest average value before update, Q (n-1) is the latest square average value before update, and S (n-1) is the latest standard deviation before update. . X (n) indicates new detection information. N represents the number of pieces of detection information used for calculating the statistical value. By using Expression 1, the determination unit 13 can reduce the amount of calculation and execute processing at higher speed.

In addition, you may make it implement | achieve the function of a part of abnormality detection apparatus 1 in embodiment mentioned above, for example, the judgment part 13 and the detection part 15, with a computer. In that case, a program for realizing the information management function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed. The “computer system” here includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in the computer system. Further, the “computer-readable recording medium” dynamically holds a program for a short time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It is also possible to include those that hold a program for a certain time, such as a volatile memory inside a computer system serving as a server or client in that case. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.
The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

It is a schematic block diagram which shows the function structure of an abnormality detection apparatus. It is a figure which shows the specific example of the process of a judgment part. It is a figure which shows the memory content of a memory | storage part. It is a flowchart which shows operation | movement of a judgment part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Abnormality detection apparatus, 11 ... Reception part, 12 ... Extraction part, 13 ... Judgment part, 14 ... Memory | storage part, 15 ... Detection part, 16 ... Output part

Claims (7)

A receiver that receives data flowing through the communication path to be monitored;
An extraction unit for extracting the value of the monitoring item from the data received by the reception unit;
A determination unit for determining a frequently appearing value from the values of the monitoring items;
A detection unit that performs abnormality detection determination based on data extracted from the value of the monitoring item determined to be frequent;
An abnormality detection device comprising: The extraction unit further extracts detection information used by the detection unit for abnormality detection determination from the data received by the reception unit,
The detection unit performs an abnormality detection determination based on the detection information extracted from data obtained by extracting the value of the monitoring item determined to be frequent.
The abnormality detection device according to claim 1. For each monitoring item value, a statistical value calculation unit that calculates a statistical value related to the detection information extracted from the data from which the monitoring item value is extracted;
A storage unit that associates and stores the statistical value for each value of the monitoring item;
The detection unit performs an abnormality detection determination based on the statistical value stored in the storage unit in association with the value of the monitoring item determined to be frequent.
The abnormality detection device according to claim 2.   The determination unit has a queue that performs first-in first-out of data, sequentially inputs the values of monitoring items extracted by the extraction unit to the queue, and sets the values of the monitoring items stored in the queue at a predetermined timing. The abnormality detection device according to claim 1, wherein the abnormality detection device determines that the value is frequent.   The determination unit has a queue that performs first-in first-out of data, sequentially inputs the values of the monitoring items extracted by the extraction unit into the queue, and values of the monitoring items stored in the queue at a predetermined timing The abnormality detection device according to any one of claims 1 to 3, wherein a value that is frequently generated is determined based on the number. A computer receiving data flowing through a communication path to be monitored;
The computer extracting the value of the monitoring item from the received data;
The computer determining a frequently occurring value from the values of the monitoring items;
The computer performs an abnormality detection determination based on data from which the value of the monitoring item determined to be frequently generated is extracted;
An abnormality detection method comprising: Against the computer
Receiving data flowing through the communication path to be monitored;
Extracting the value of the monitoring item from the received data;
Determining a frequently occurring value from the values of the monitoring items;
Performing an abnormality detection determination based on data extracted from the value of the monitoring item that is determined to be frequent;
A computer program for running.

Images