JP2007243338A - Log analyzer, log analysis program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently generate statistical data obtained by summing up events recorded in a log for each prescribed unit time. <P>SOLUTION: A log preservation part 102 generates first statistical data showing a frequency per first unit time of an event relating to a specific parameter recorded in the log outputted from network equipment so as to preserve them in a storage 103. A log analyzer 104 reads out the first statistical data from the storage 103 so as to generate second statistical data showing a frequency per second unit time of an event relating to the specific parameter, while summing up a plurality of the first statistical data relating to events detected within a second unit time longer than the first unit time. The log analyzer 104 converts a distribution on each time axis composed of each of a plurality of the first/second statistical data to a distribution on a frequency axis, so as to calculate a value regarding an abnormality degree of a network by using the distribution on each frequency axis. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a log analyzer for analyzing logs output from network devices such as IDS (Intrusion Detection System), Router, Firewall, and traffic monitor. The present invention also relates to a log analysis program for causing a computer to execute processing relating to log analysis, and a computer-readable recording medium recording the log analysis program.

  In recent years, threats to cyber terrorist attacks are increasing, and various methods for detecting abnormalities occurring in a network by analyzing logs output from various network devices have been proposed. The increase / decrease in traffic on the network depends on the time zone and day of the week when the user uses a PC (Personal Computer), and the tendency of the attack is almost similar to the usage frequency of the PC. However, many of the recent attacks are due to viruses and worms having a latent period, onset / propagation period, and some appear with periodicity other than the frequency of PC use. In consideration of the periodicity of these attacks, a technique for detecting an attack tendency to an analysis target and a sign of a new attack has been proposed (see, for example, Patent Document 1 and Non-Patent Document 1).

Patent Document 1 describes a frequency analysis technique for detecting an abnormality by paying attention to the periodicity of statistics relating to the frequency of events recorded in a log. In the analysis of periodicity, a frequency distribution (frequency spectrum) generated by converting a statistical quantity on the time axis into a component quantity (power) on the frequency axis using Fourier transform is used. For example, the following method is used to detect an abnormality.
-The amount of components in the frequency distribution obtained from data for a specific period (or data related to the network of interest) and the component in the frequency distribution obtained from data for the past predetermined period (or data related to other networks) A technique for evaluating the degree of abnormality using the ratio to the quantity.
A method for evaluating the degree of abnormality using information entropy indicating the ambiguity of the log.

Non-Patent Document 1 also describes a technique similar to the technique described in Patent Document 1.
JP 2005-151289 A Hiroshi Nakakoji, Masatoshi Terada, “Examination of Incident Trend Detection Method Based on Frequency Analysis”, Information Processing Society of Japan, Research Report, 2005-CSEC-30, July 2005, pp.83-88

  In the conventional log frequency analysis, statistical data (data indicating the frequency of events within a unit time) that summarizes the events recorded in the log every predetermined unit time (for example, 1 minute, 30 minutes, 0.5 days, etc.) Fourier transform was performed using. In the case of generating a plurality of statistical data, it is necessary to perform a process of collecting events recorded in a log in a designated unit time a plurality of times. As the number and types of statistical data to be generated increase, the processing load related to the generation of statistical data increases. For example, when generating statistical data for 1 minute, 30 minutes, and 0.5 days, the events recorded in the log are summarized in 1 minute units, the same events are summarized in 30 minute units, and the same It was necessary to sequentially perform the process of collecting events in units of 0.5 days, and it was necessary to repeat inefficient processing. In view of this point, an object of the present invention is to efficiently generate statistical data in which events recorded in a log are collected every predetermined unit time.

  Further, in the conventional log frequency analysis focusing on the component amount in the frequency distribution, there is a problem that false detection of an attack tends to occur due to a change in the overall attack frequency accompanying the network expansion / contraction work. Furthermore, the conventional frequency analysis technique focusing on information entropy has a problem that although the occurrence of an abnormality can be detected, the frequency band where the abnormality has occurred cannot be specified. In view of this point, an object of the present invention is to provide a technique other than the technique used in the conventional frequency analysis.

  In addition, since there are many noise components in the event and it is unclear to which parameter in the log the trace of an unknown attack is recorded, the conventional log frequency analysis processes all parameter events. I had to. For example, when performing frequency analysis using the Destination Port that indicates the IP address of the device that is the destination of the packet as the parameter to be analyzed, more than 120,000 types including TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) Destination Port Had to perform analysis of events. Therefore, in order to reduce the processing load, it is desirable to perform frequency analysis by narrowing down events to be processed. In view of this point, the present invention also aims to provide a method for selecting a notable event.

  The present invention has been made to solve the above-described problem, and is a first statistical data indicating the frequency per unit time of an event related to a specific parameter recorded in a log output from a network device. A plurality of first statistical data relating to an event detected within a second unit time longer than the first unit time, and the specific parameter A second statistical data generating unit that generates second statistical data indicating the frequency of the event according to the second unit time, and a distribution on a time axis constituted by a plurality of the first statistical data. The distribution on the frequency axis is converted to the distribution on the frequency axis by the first frequency distribution generating means for converting to the distribution on the frequency axis and generating the first frequency distribution, and the plurality of second statistical data. Convert and second A second frequency distribution generating means for generating a frequency distribution of the network, and a calculating means for calculating a value relating to the degree of abnormality of the network based on the first frequency distribution and the second frequency distribution. Is a log analysis device.

  In the log analysis apparatus of the present invention, the second statistical data generation unit may include a plurality of the first statistical data when a value obtained by dividing the second unit time by the first unit time is an integer. The statistical data is collected to generate the second statistical data.

  In the log analysis apparatus of the present invention, the calculation means may calculate a ratio of a spectrum value in a specific frequency band in the first frequency distribution to a total sum of spectrum values in all frequency bands. The frequency distribution is calculated for each of the frequency distributions, and a value related to the degree of abnormality of the network is calculated using a probability distribution obtained from the calculated values of the plurality of ratios. A value related to the degree of abnormality is calculated.

  Further, in the log analysis device of the present invention, the calculation means includes the first value related to a plurality of networks in which spectrum values in a specific frequency band in the first frequency distribution related to a specific network include the specific network. Calculating a ratio of a total of spectrum values in the specific frequency band in one frequency distribution for each of the plurality of first frequency distributions, and using a probability distribution obtained from the calculated values of the plurality of ratios, A value relating to the degree of abnormality of the network is calculated, and a value relating to the degree of abnormality of the network is similarly calculated for the second frequency distribution.

  The log analysis apparatus according to the present invention further includes event extraction means for extracting an event relating to a specific area from events relating to a specific parameter recorded in the log, wherein the first statistical data generation means Is characterized in that a plurality of the first statistical data for the event relating to the specific area extracted by the event extraction means are generated.

  In the log analysis apparatus of the present invention, when the value related to the network abnormality degree calculated by the calculating means indicates a network abnormality, the first statistical data generating means further determines the event frequency. A plurality of the first statistical data for the event relating to the parameter selected as a reference is generated, and the second statistical data generating means further combines the plurality of newly generated first statistical data. A plurality of the second statistical data, and the first frequency distribution generating means further generates the first frequency distribution from the plurality of newly generated first statistical data, The frequency distribution generation means further generates the second frequency distribution from a plurality of the second statistical data newly generated, and the calculation means generates the newly generated frequency distribution Serial based on the first frequency distribution and the second frequency distribution, and calculates a value for the degree of abnormality of the network.

  In addition, the present invention provides a first step of generating first statistical data indicating a frequency per first unit time of an event relating to a specific parameter recorded in a log output from a network device; A plurality of the first statistical data relating to events detected within a second unit time longer than one unit time are collected, and the frequency per second unit time of the event relating to the specific parameter is determined. A second step of generating the second statistical data shown, and a distribution on the time axis composed of the plurality of first statistical data is converted into a distribution on the frequency axis to generate a first frequency distribution A third step, a fourth step of generating a second frequency distribution by converting a distribution on the time axis composed of the plurality of second statistical data into a distribution on the frequency axis, and the first step Frequency distribution and The second based on the frequency distribution, a log analysis program characterized by executing the fifth and steps on a computer to calculate a value for abnormality of the network.

  In the log analysis program of the present invention, in the second step, when the value obtained by dividing the second unit time by the first unit time is an integer, a plurality of the first statistical data is obtained. Collectively, the second statistical data is generated.

  In the log analysis program of the present invention, in the fifth step, a ratio of a spectrum value in a specific frequency band in the first frequency distribution to a total sum of spectrum values in all frequency bands is a plurality of the first frequency distributions. Calculate for each of the frequency distributions of 1 and use the probability distribution obtained from the calculated values of the plurality of ratios to calculate a value relating to the degree of abnormality of the network, and similarly for the second frequency distribution, A value related to the degree of abnormality of the network is calculated.

  In the log analysis program of the present invention, in the fifth step, spectrum values in a specific frequency band in the first frequency distribution related to a specific network are related to a plurality of networks including the specific network. The ratio of the first frequency distribution in the total sum of the spectrum values in the specific frequency band is calculated for each of the plurality of first frequency distributions, and a probability distribution obtained from the calculated plurality of ratio values is used. Then, a value related to the degree of abnormality of the network is calculated, and a value related to the degree of abnormality of the network is calculated similarly for the second frequency distribution.

  The log analysis program of the present invention further includes a sixth step of extracting an event related to a specific area from events related to a specific parameter recorded in the log. In the first step, A plurality of the first statistical data for the event related to the specific area extracted in the sixth step are generated.

  In the log analysis program of the present invention, when the value related to the network abnormality degree calculated in the fifth step indicates a network abnormality, the event relating to the parameter selected on the basis of the event frequency A seventh step of generating a plurality of the first statistical data for the first, and an eighth step of generating a plurality of the second statistical data by combining the plurality of newly generated first statistical data And a ninth step of generating the first frequency distribution from the plurality of newly generated first statistical data, and the second frequency from the plurality of newly generated second statistical data. A tenth step of generating a distribution, and an abnormality degree of the network based on the newly generated first frequency distribution and the second frequency distribution; Characterized by further comprising a eleventh step of calculating the value.

  The present invention is a computer-readable recording medium on which the log analysis program is recorded.

  According to the present invention, since the first statistical data is collected to generate the second statistical data, the processing load related to the generation of the statistical data can be reduced, and the statistical data can be generated efficiently. can get.

  The present invention also provides a technique for analyzing logs based on the relative proportion of spectral values within a specific frequency band. This can reduce false detection of attacks due to changes in the overall attack frequency of the network. Furthermore, by analyzing the log based on the ratio of the spectrum values in a specific frequency band in the frequency distribution to the total sum of the spectrum values in all frequency bands, the frequency band where the abnormality has occurred can be specified. Further, the log value in the specific frequency band in the frequency distribution related to the specific network is logged based on the proportion of the total spectrum value in the specific frequency band in the frequency distribution related to the plurality of networks including the specific network. By performing this analysis, it is possible to identify the network in which an abnormality has occurred.

  The present invention also provides a method for selecting a notable event. As a result, the processing load related to frequency analysis can be reduced.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows the configuration of a system including a log analysis device according to an embodiment of the present invention. In the figure, the log analysis device 10 collects logs output from the networks 20 to 22 to be analyzed and analyzes them. The networks 20 to 22 include network devices such as Router, Firewall, and IDS, and output logs generated by the network devices to the log analysis device 10 using a communication method such as syslogd. The security operation center 30 is provided with an operation terminal 301 connected to the log analysis device 10 via a communication line.

  The log analysis apparatus 10 according to the present embodiment collects and analyzes logs output from the analysis target networks 20 to 22 and outputs the analysis results to the operation terminal 301. The operator can determine the degree of abnormality of the networks 20 to 22 based on the analysis result displayed on the display unit of the operation terminal 301.

  The embodiment of the present invention is not limited to the form shown in FIG. 1, and the present invention may be applied to other forms. For example, the log analysis device 10 may output the analysis result to another device having a display unit. When the log analysis device 10 includes a display unit, the log analysis device 10 outputs the analysis result. It may be displayed. Further, the log analysis device 10 may be a device installed at a different location from the security operation center 30, and a device in the security operation center 30 including the operation terminal 301 has the function of the log analysis device 10. It may be.

  In the log analysis device 10, the log collection unit 101 periodically collects logs output from the network devices of the networks 20 to 22 and outputs the logs to the log storage unit 102. The log storage unit 102 generates statistical data (details will be described later) indicating the frequency per unit time of an event related to each parameter recorded in the log, and stores the statistical data in the storage unit 103. The log storage unit 102 may store the log in the storage unit 103 as it is. The storage unit 103 includes a recording medium such as an SRAM (Static Random Access Memory), an SDRAM (Synchronous Dynamic Random Access Memory), a flash memory, an HDD (Hard Disk Drive), a log, statistical data, an analysis result of the log, and the like. Remember. The log analysis unit 104 reads the statistical data from the storage unit 103 and analyzes it. The interface unit 105 controls communication performed between the log analysis device 10 and the operation terminal 301.

Next, main parameters to be analyzed in this embodiment will be described. The following parameters are parameters to be analyzed.
(A) Number of packets (b) Traffic volume (c) Protocol volume (d) Port volume (e) Attack Signature
(F) Source / Destination Port
(G) Source / Destination IP
(H) Packet size (i) Flag (j) Source / Destination Country

  By analyzing the number of packets or the amount of traffic, it is possible to detect whether an attack on the network is occurring. The protocol amount indicates the number of packets (or traffic amount) related to a protocol such as TCP / UDP / ICMP (Internet Control Message Protocol). By analyzing the protocol amount, various attacks can be detected. For example, since packets such as Ping and Traceroute related to ICMP are a preliminary check before an attack, it is possible to pay attention in advance to a full-scale attack by detecting these packets. The Port amount indicates the number of packets (or traffic amount) for each port number of the transmission source / destination device of a packet related to a protocol such as TCP / UDP. By analyzing the amount of ports, access to or intrusion to a specific port can be detected.

  Attack Signature shows the attack pattern. The signature matching type IDS simply compares a packet flowing on a network with an attack signature prepared in advance, and records an event in a log when the two match. Attack Signature includes events such as HTTP port probe and DNS port probe. By analyzing this attack signature, it is possible to identify the type of attack being performed on the analysis target network.

  Source / Destination Port indicates the port number of the source / destination device, and Source / Destination IP indicates the IP address of the source / destination device. By analyzing the Source / Destination Port, the port number of the attack source / target can be specified. Further, by analyzing Source / Destination IP, it is possible to identify the IP address of the attack source / target.

  The packet size is recorded in the log for each size divided by, for example, 64, 128, 256, 512, 768, and 1024 (Byte). By analyzing the packet size, an attack characteristic of the packet size can be detected. The flags recorded in the packet include TCP-SYN, ICMP-Unreachable, and the like. By analyzing such a flag, an attack caused by a virus or the like can be detected. Source / Destination Country indicates the country to which the source / destination device belongs. By analyzing Source / Destination Country, it is possible to specify the country of attack / destination.

  Next, the operation of the log analysis apparatus 10 according to the present embodiment will be described. FIG. 2 shows an operation procedure of the log analysis apparatus 10. The log collection unit 101 collects logs output from the devices in the networks 20 to 22 and outputs the collected logs to the log storage unit 102 (step S100). The log storage unit 102 generates statistical data based on the contents of the log output from the log collection unit 101, and stores the statistical data in the storage unit 103 as log information (step S110). When the operator operates the operation terminal 301 and inputs a log analysis instruction, instruction information is output from the operation terminal 301. This instruction information is input to the log analysis unit 104 via the interface unit 105. Based on this instruction information, the log analysis unit 104 reads statistical data in which log information is collected from the storage unit 103 (step S120).

  The log analysis unit 104 performs analysis processing to be described later using the read statistical data, and stores the analysis result in the storage unit 103 (step S130). Subsequently, the log analysis unit 104 outputs the analysis result to the operation terminal 301 via the interface unit 105 (step S140). The analysis result is displayed on the display unit of the operation terminal 301. Based on the analysis result, the operator can determine whether or not an attack is being performed on the networks 20 to 22 (whether an abnormality has occurred).

  Next, detailed contents of the analysis process performed by the log analysis unit 104 in step S130 will be described. As shown in FIG. 3, a log in the form of a header log in which the IP and Port of the source / destination device are associated is output from various network devices. Data constituting the log (data in units of lines in FIG. 3) is an event. In the case of IDS monitoring the backbone line, millions to tens of millions of logs may be output per day. In order to solve such a quantitative problem of large-scale logs, event information included in each log is compressed and stored as statistical data.

  FIG. 4 shows the contents of the statistical data excluding the header portion. As shown in FIG. 4, the type of parameter is associated with the frequency of events (number of detections) related to the parameter. This statistical data is a summary of information on all events detected within a unit time (1 second in FIG. 4). For example, “Packet, 37260” in FIG. 4 indicates that 37260 packets are detected within a unit time. The log storage unit 102 generates statistical data from the log output from each network device and stores the statistical data in the storage unit 103. Some network devices output statistical data, and when the statistical data is output from the log collection unit 101, the log storage unit 102 stores the statistical data in the storage unit 103 as it is.

  The log analysis unit 104 performs analysis processing using the stored statistical data. First, the log analysis unit 104 reads out statistical data from the storage unit 103, extracts data related to a specific parameter from the statistical data, and sets it as new statistical data (corresponding to the first statistical data of the present invention). When analysis processing is performed on all parameters, new statistical data is generated for each parameter. Further, when the user designates a parameter to be analyzed, new statistical data is generated for the designated parameter. In addition, as will be described later, the log analysis unit 104 may select a parameter suitable for analysis from all the parameters, and generate new statistical data for the parameter.

  FIG. 5 shows the flow of subsequent processing by the log analysis unit 104. The log analysis unit 104 collects statistical data information corresponding to each of a plurality of continuous unit times and generates a distribution (time distribution) on the time axis (step S200 in FIG. 5). FIG. 6 is an example of the distribution on the time axis. In the figure, the horizontal axis indicates time (time), and the vertical axis indicates the frequency of events (event amount) related to the parameter to be analyzed at each time (time). The width in the horizontal axis direction of the distribution on the time axis is the unit time of statistical data × the number of statistical data, and this width is defined as the observation window width.

  Subsequently, the log analysis unit 104 converts the distribution on the time axis into a distribution on the frequency axis (frequency distribution, frequency spectrum) by discrete Fourier transform (step S210 in FIG. 5). If the event quantity at the nth observation point on the time axis is e (n) and the kth frequency component quantity on the frequency axis is E (k) (where n and k are natural numbers), N (N is When a discrete time signal {e (n)}, n = 0, 1,..., N−1 is given as a natural number, the discrete Fourier transform is expressed as [Equation 1] below. In addition, you may perform FFT (Fast Fourier Transform) which performs a discrete Fourier transform at high speed. FIG. 7 shows an example in which the distribution on the time axis shown in FIG. 6 is converted to the distribution on the frequency axis. The horizontal axis indicates the time corresponding to each frequency, and the vertical axis indicates the power (spectrum amount).

  When the resolution of the frequency to be extracted is f, the relationship between the sampling frequency fs and f when generating the statistical data is expressed as fs = 2f. A sampling period that is the reciprocal of the sampling frequency fs corresponds to a unit time of statistical data. When a long-period (low frequency) abnormality occurs, a spectrum appears in the left region of the graph in FIG. 7, but detailed information on the abnormality cannot be obtained because the frequency resolution in this region is low. Therefore, a plurality of unit times of statistical data are prepared as shown in FIG. 8 so that long-period abnormalities can be detected. By providing multiple unit times in this way and performing frequency analysis using statistical data for each unit time, attacks with long attack intervals, so-called low-frequency attacks, which are difficult for the operator to notice subjectively, are more accurate. Can be detected.

  In FIG. 8, “statistical unit time” indicates the time resolution of the statistical data, and coincides with the above-described statistical data unit time. For example, when the unit time is 1 minute, one statistical data is composed of events detected in 1 minute. Due to the mathematical characteristics of the discrete Fourier transform, the most efficient Fourier transform processing can be implemented by using statistical data of 2 multipliers. Therefore, in FIG. 8, the discrete Fourier transform is performed using 32 pieces of statistical data.

  In FIG. 8, “frequency” indicates the maximum frequency and the minimum frequency in the distribution on the frequency axis. For example, when the unit time is 1 minute, the maximum frequency of the spectrum in the distribution on the frequency axis is 0.00833 Hz, and the minimum frequency is 0 Hz. There will be 32 spectra between these frequencies. As shown in FIG. 8, by providing a plurality of unit times, it is possible to obtain a distribution on the frequency axis suitable for detecting an attack having a period corresponding to each unit time. For example, when the unit time is 1 minute, the generated distribution on the frequency axis is suitable for detecting an attack with a period of several minutes. The log analysis unit 104 generates a distribution on the frequency axis corresponding to each unit time from the statistical data corresponding to each unit time.

  When providing a plurality of unit times as described above and generating a distribution on the frequency axis corresponding to each unit time, each unit time for configuring the distribution on the time axis that is the origin of the distribution on the frequency axis Need to generate statistical data. In the conventional method, events recorded in the log are summarized in 1 minute units, events recorded in the same log in the same manner are summarized in 0.5 hour units, and events recorded in the same log in the same way are combined for 0.5 days. A process is performed in which the events recorded in the same log are grouped in units of 0.5 months (see FIG. 17). However, this method has a problem that the greater the number of unit times set in stages, the higher the processing load and the lower the efficiency.

  Therefore, in the present embodiment, statistical data corresponding to each unit time is generated as follows (see FIG. 9). Based on the contents of the log (log 901) output from the log collection unit 101, the log storage unit 102 summarizes events with 1 minute as a unit time for each parameter, and sets a plurality of statistical data (1) (statistical data 902). It is generated and stored in the storage unit 103 (step S300). The log analysis unit 104 reads the statistical data (1) about the parameter to be analyzed from the storage unit 103. When the statistical data (1) that summarizes the events detected in one minute is counted as one piece of statistical data, the log analysis unit 104 has 30 pieces of statistical data (1) (30 corresponding to continuous unit times). A plurality of statistical data (2) (statistical data 903) having a unit time of 0.5 hour are generated (step S310).

  Similarly, the log analysis unit 104 collects 24 statistical data (2) corresponding to continuous unit time (12 hours, in other words, equivalent to 0.5 day data), and statistical data having 0.5 day as a unit time. (3) A plurality of (statistical data 904) are generated (step S320). Further, the log analysis unit 104 similarly collects 30 statistical data (3) (corresponding to 15 days, in other words, 0.5 months of data) corresponding to continuous unit times, and sets 0.5 months as a unit time. A plurality of statistical data (4) (statistical data 905) are generated (step S330).

  The relationship between the statistical data (1) to (4) and the first and second statistical data in the present invention will be described. The relationship between the statistical data (1) and the statistical data (2) of the present embodiment, the statistical data. Each of the relationship between (2) and statistical data (3) and the relationship between statistical data (3) and statistical data (4) corresponds to the relationship between the first statistical data and the second statistical data of the present invention. . Further, for example, the statistical data (3) may be generated from the statistical data (1). In this case, the relationship between the statistical data (1) and the statistical data (3) is the first statistical data and the first statistical data of the present invention. This corresponds to the relationship between the two statistical data.

  In the statistical data generation method described above, the first statistical data is collected to generate the second statistical data. Therefore, compared to the method of generating the second statistical data from the log data, the statistical data is generated. Such processing load can be reduced, and statistical data can be generated efficiently. In particular, the greater the number of unit times set in stages, the greater the effect of reducing the processing load related to the generation of statistical data. As a modification of the above method, when a log is stored in the storage unit 103, the log analysis unit 104 may generate statistical data (1).

  When statistical data having a unit time different from the statistical data (1) (for example, statistical data having a unit time of 1 hour) is stored in the storage unit 103 (there is not a log but a network device that outputs the statistical data) The log analysis unit 104 collects statistical data as follows (see FIG. 10). When the statistical data having a unit time different from the statistical data (1) (referred to as the first statistical data) is statistical data satisfying a predetermined condition, the log analysis unit 104 determines that the statistical data (1) The statistical data of 1 is collected, and statistical data (2) (referred to as second statistical data) is generated.

  First, the log analysis unit 104 divides the unit time of the second statistical data by the unit time of the first statistical data, and obtains a quotient thereof (step S400). At this time, division is performed after the units (minute, hour, day, etc.) of two unit times are aligned. Subsequently, the log analysis unit 104 determines whether or not the quotient obtained in step S400 is an integer (an integer greater than 1) (step S410). If the quotient is an integer, the log analysis unit 104 determines to use the first statistical data when generating the second statistical data (step S420). On the other hand, if the quotient is not an integer, the log analysis unit 104 determines to use the first statistical data when generating the statistical data of another unit time (step S430). When the determination in step S430 is performed, the same processing as described above is performed by changing the unit time of the second statistical data.

  FIG. 11 shows the state of statistical data generated by the above-described processing. In FIG. 11, statistical data (statistical data 1101) whose unit time is 1 minute is output from the network device A, and statistical data (statistical data 1102) whose unit time is 1 hour is output from the network device B. It is assumed that statistical data (statistical data 1103) whose unit time is 0.5 days is output from the network device C.

  For the unit time of statistical data 1101 (unit time: 1 minute) and statistical data 1104 (unit time 0.5 hour), 30 minutes (= 0.5 hour) ÷ 1 minute = 30 holds. Statistical data 1104 can be generated. On the other hand, for the unit time of the statistical data 1102 (unit time: 1 hour) and the statistical data 1104 (unit time: 0.5 hour), 0.5 hours ÷ 1 hour = 0.5 holds. Therefore, according to the above, when generating the statistical data 1104 In addition, the statistical data 1102 cannot be used.

  However, for the unit time of statistical data 1105 (unit time: 0.5 day) and statistical data 1102 (unit time: 1 hour), 12 hours (= 0.5 days) ÷ 1 hour = 12 holds. 1102 can be collected to generate statistical data 1105. In addition, regarding the unit time of statistical data 1105 (unit time: 0.5 day) and statistical data 1104 (unit time: 0.5 hour), 12 hours (= 0.5 days) /0.5 hours = 24 holds. The statistical data 1105 can be generated by grouping 1104.

  Therefore, the statistical data 1105 can be generated by combining the statistical data 1102 and 1104 having different unit time lengths. Similarly, the statistical data 1103 (unit time: 1.0 day) cannot be used when generating the statistical data 1104 and 1105, but the statistical data 1106 (unit time: 0.5 day) is used together with the statistical data 1105 (unit time: 0.5 day). Can be used to generate 0.5 month). A distribution on the time axis is generated from each of the statistical data 1101, 1104, 1105, and 1106 generated as described above, and the distribution on each time axis is converted into a distribution on the frequency axis.

  According to the statistical data generation method described above, statistical data having different unit time lengths can be collected and statistical data having a unit time longer than the unit time can be generated. In particular, even when statistical data is output from a network device instead of a log, the statistical data can be combined with the statistical data generated from the log to generate new statistical data.

  With reference to FIG. 5 again, the detailed contents of the processing performed by the log analysis unit 104 after the distribution on the frequency axis is generated will be described. If the distribution on the frequency axis generated from statistical data having the same unit time length is defined as the same type of distribution, when four types of statistical data unit time are prepared as shown in FIG. Will be generated. The following processing is processing using the same type of distribution, and the same processing as processing for a certain type of distribution is also performed for another type of distribution.

  After generating the distribution on the frequency axis, the log analysis unit 104 determines the ratio of the power in a specific frequency band in the distribution on the frequency axis to the sum of the power in all frequency bands (a specific frequency for the power in all frequency bands). The power content of the frequency band is calculated (step S220 in FIG. 5). For each of the distributions on a plurality of frequency axes based on events detected within different times, the power content rate for the same frequency band is calculated. FIG. 12 shows an example of the temporal change in the power content of each frequency band. As shown in FIG. 12, the ratio of the power in the frequency band A occupying the entire frequency band rapidly increases at time t1. In this way, the log analysis unit 104 performs the following process so that a sudden change in the power content of a certain frequency band can be detected as a network abnormality.

The log analysis unit 104 generates a probability distribution from the power content of a specific frequency band calculated from each of the distributions on the plurality of frequency axes (step S230 in FIG. 5), and subsequently calculates a value related to the degree of network abnormality. calculate. FIG. 13 shows an example of a probability distribution. μ is an average of the power content of the specific frequency band calculated in step S230, and σ is a standard deviation. The horizontal axis represents the distance from the average value μ with the standard deviation σ as a unit, and the vertical axis represents the appearance probability (probability density) of the power content in a specific frequency band. When E ₁ , E ₂ ,... E _k are content ratios of the power in a specific frequency band obtained from the distribution on the k frequency axes, the standard deviation σ is calculated by the following [Equation 2]. The

The log analysis unit 104 calculates a determination value R for determining whether there is a network abnormality according to the following [Equation 3]. E _{k + 1} is the power content of a specific frequency band in a period to be determined as to whether there is a network abnormality. In [Expression 3], f (E) is a density function of a normal distribution, and is expressed by the following [Expression 4]. In the above example, the distribution of power content in a specific frequency band is assumed to be a normal distribution, but a function such as an exponential distribution or a gamma distribution may be appropriately selected as f (E).

  The determination value R corresponds to the area of the region 1301 in FIG. When the determination value R is less than a predetermined value (for example, 5%), the log analysis unit 104 determines that an attack having a specific periodicity has appeared on the network and an abnormality has occurred in the network.

  As described above, by performing the analysis process while paying attention to the power content rate, it is possible to reduce false detection of an attack due to a change in the overall attack frequency of the network. Further, by performing an analysis process based on the power content of a specific frequency band, it is possible to specify the frequency band where the abnormality has occurred.

  When the above analysis processing is performed using logs or statistical data output from a network device of a specific network only, it is possible to detect whether or not a periodic attack has occurred in that network. . However, when the above analysis process is performed using a mixture of logs or statistical data output from a plurality of network devices, it is possible to detect that an attack having a specific periodicity has occurred. However, it is not possible to detect which network has such an attack. Therefore, a network in which an attack having periodicity is generated may be specified as follows.

  If the distribution on the frequency axis generated from statistical data having the same unit time length is defined as the same type of distribution, when four types of statistical data unit time are prepared as shown in FIG. Will be generated. The following processing is processing using the same type of distribution, and the same processing as processing for a certain type of distribution is also performed for another type of distribution.

  The log analysis unit 104 generates a distribution on the frequency axis in the same manner as described above. However, each distribution is generated for each network. In FIG. 14, a distribution on the frequency axis related to network A is generated from logs or statistical data output only from network devices belonging to network A. Similarly, for networks B and C, a distribution on the frequency axis related to each network is generated from logs or statistical data output from only network devices belonging to each network.

The log analysis unit 104 is configured such that the power in a specific frequency band (referred to as frequency band F) in the distribution on the frequency axis related to a specific network A ratio (content ratio of the power of the specific network with respect to the power of the entire network regarding the specific frequency band) in the total power in the frequency band (the same frequency band F as described above) is calculated (step S500 in FIG. 14). For example, the power in a specific frequency band in the distribution on the frequency axis for network A is P _A , the power in the specific frequency band in the distribution on the frequency axis for network _B is P _B , and the distribution on the frequency axis for network C is the power in a particular frequency band when the P _C in the content E of the power of the network a is calculated as follows.
_{E = P A / (P A} + P B + P C)

Further, if the power in a specific frequency band in the distribution on the frequency axis regarding the entire networks A, B, and C is P _ABC , the power content E of the network A may be calculated as follows.
E = P _A / P _ABC

  In the same manner as described above, the power content rate of a specific network related to the same frequency band is calculated for each of distributions on a plurality of frequency axes based on events detected in different times. FIG. 15 shows an example of the change over time in the power content of each network. As shown in FIG. 15, the ratio of the power of the network A occupying all the networks rapidly increases at time t2. In this way, the log analysis unit 104 performs the following processing so that a sudden change in the power content of a certain network can be detected as a network abnormality.

  The log analysis unit 104 generates a probability distribution (similar to FIG. 13) from the calculated power content ratio of the specific network (step S510 in FIG. 14), and subsequently calculates a value related to the degree of network abnormality. At this time, the log analysis unit 104 calculates a determination value R for determining the presence / absence of an abnormality in the network according to the above [Equation 3]. When the determination value R is less than a predetermined value (for example, 5%), the log analysis unit 104 determines that an attack having a specific periodicity has appeared on a specific network and an abnormality has occurred in that network To do. As described above, the network in which an abnormality has occurred can be specified by performing the analysis process based on the power content of a specific network.

  Next, a method for selecting a notable event will be described. When frequency analysis is performed using logs or statistical data acquired from different countries around the world, noise components increase, and periodic events cannot be detected. Therefore, an event related to a specific region may be extracted from events related to the parameter to be processed, and analysis processing may be performed using statistical data related to the event.

  Hereinafter, a specific example of processing will be described. It is assumed that the log output from each network device is stored in the storage unit 103. The log analysis unit 104 reads the log from the storage unit 103 and extracts data regarding a specific parameter. At this time, the log analysis unit 104 extracts an event related to a specific region (unit of one country or units of several countries) from the events recorded in the log based on Source Country or Source IP, and the event New statistical data (corresponding to the first statistical data of the present invention) is generated. The log analysis unit 104 performs the same process as described above using the generated statistical data.

  In addition, since frequency analysis including a discrete Fourier transform with a large processing load is performed on all parameters, the number of events to be noted may be reduced by narrowing down the parameters to be processed. Hereinafter, a specific example of processing will be described with reference to FIG. First, the log analysis unit 104 performs frequency analysis in the same manner as the above-described process, using statistical data regarding a specific parameter (number of packets or the like) (step S600). At this time, frequency analysis may be performed for an event related to a specific region among events related to the parameter.

  Subsequently, the log analysis unit 104 determines whether or not a network abnormality is detected in a specific frequency band by the frequency analysis in step S600 (step S610). If no network abnormality is detected in any frequency band, the process ends. Further, when a network abnormality is detected in a specific frequency band, the log analysis unit 104 selects N types of parameters having a high frequency among the parameters recorded in the statistical data used in step S600. (Step S620). Subsequently, the log analysis unit 104 performs frequency analysis for each selected parameter in order by the same method as described above (step S630).

  As described above, by selecting a notable event, it is possible to reduce the processing load related to frequency analysis.

  The embodiment of the present invention has been described in detail above with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design changes and the like without departing from the gist of the present invention. For example, a program for realizing the operations and functions of the log analysis device according to the above-described embodiment may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read and executed by the computer. Good.

  Here, the “computer” includes a homepage providing environment (or display environment) if the WWW system is used. The “computer-readable recording medium” refers to a storage device such as a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a hard disk built in the computer. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program described above may be transmitted from a computer storing the program in a storage device or the like to another computer via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting a program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the above-described program may be for realizing a part of the above-described function. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer, what is called a difference file (difference program) may be sufficient.

It is a block diagram which shows the structure of the system provided with the log analyzer by one Embodiment of this invention. It is a flowchart which shows the procedure of the log analysis process in one Embodiment of this invention. It is a reference figure showing the contents of the log in one embodiment of the present invention. It is a reference figure which shows the content of the statistical data in one Embodiment of this invention. It is a reference figure for demonstrating the analysis process of the log in one Embodiment of this invention. It is a reference figure which shows distribution on the time axis in one Embodiment of this invention. It is a reference figure which shows distribution on the frequency axis in one Embodiment of this invention. It is a reference figure for demonstrating the production | generation method of the statistical data in one Embodiment of this invention. It is a reference figure for demonstrating the production | generation method of the statistical data in one Embodiment of this invention. It is a flowchart which shows the production | generation procedure of the statistical data in one Embodiment of this invention. It is a reference figure for demonstrating the production | generation method of the statistical data in one Embodiment of this invention. It is a reference figure for demonstrating the analysis process of the log in one Embodiment of this invention. It is a reference figure showing probability distribution in one embodiment of the present invention. It is a reference figure for demonstrating the analysis process of the log in one Embodiment of this invention. It is a reference figure for demonstrating the analysis process of the log in one Embodiment of this invention. It is a flowchart which shows the procedure of the log analysis process in one Embodiment of this invention. It is a reference figure for demonstrating the production | generation method of the conventional statistical data.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Log analyzer, 20, 21, 22 ... Network, 30 ... Security operation center, 101 ... Log collection part, 102 ... Log storage part, 103 ... Storage part, 104 ... Log analysis unit, 105 ... Interface unit, 301 ... Operation terminal

Claims (13)

First statistical data generating means for generating first statistical data indicating a frequency per first unit time of an event relating to a specific parameter recorded in a log output from a network device;
Collecting a plurality of the first statistical data related to events detected within a second unit time longer than the first unit time, the events per second unit time of the event related to the specific parameter Second statistical data generating means for generating second statistical data indicating the frequency;
First frequency distribution generation means for converting a distribution on the time axis composed of a plurality of the first statistical data into a distribution on the frequency axis, and generating a first frequency distribution;
A second frequency distribution generating means for converting a distribution on the time axis composed of a plurality of the second statistical data into a distribution on the frequency axis, and generating a second frequency distribution;
Calculation means for calculating a value relating to the degree of network abnormality based on the first frequency distribution and the second frequency distribution;
A log analyzer characterized by comprising:   The second statistical data generating means collects the plurality of first statistical data together when the value obtained by dividing the second unit time by the first unit time is an integer. The log analysis apparatus according to claim 1, wherein data is generated.   The calculation means calculates and calculates the ratio of the spectrum value in the specific frequency band in the first frequency distribution to the total sum of the spectrum values in all frequency bands for each of the plurality of the first frequency distributions. Calculating a value related to the degree of abnormality of the network using a probability distribution obtained from a plurality of ratio values, and calculating a value relating to the degree of abnormality of the network in the same manner for the second frequency distribution; The log analysis apparatus according to claim 1, wherein the log analysis apparatus is characterized.   The calculating means is configured such that a spectrum value in a specific frequency band in the first frequency distribution related to a specific network has the specific frequency band in the first frequency distribution related to a plurality of networks including the specific network. A ratio of the total spectrum value in the sum is calculated for each of the plurality of first frequency distributions, and a value related to the degree of abnormality of the network is calculated using a probability distribution obtained from the calculated plurality of ratio values. The log analysis apparatus according to claim 1, wherein a value related to the degree of abnormality of the network is calculated similarly for the second frequency distribution. Event extracting means for extracting an event related to a specific region from events related to a specific parameter recorded in the log,
The first statistical data generation means generates a plurality of the first statistical data targeted for the event related to the specific area extracted by the event extraction means. The log analysis device according to claim 4. When the value related to the degree of network abnormality calculated by the calculation means indicates a network abnormality,
The first statistical data generation means further generates a plurality of the first statistical data for an event related to a parameter selected on the basis of an event frequency,
The second statistical data generating means further generates a plurality of the second statistical data by combining the plurality of newly generated first statistical data,
The first frequency distribution generation means further generates the first frequency distribution from a plurality of newly generated first statistical data,
The second frequency distribution generation means further generates the second frequency distribution from a plurality of newly generated second statistical data,
The said calculation means calculates the value regarding the abnormality degree of the said network based on the said 1st frequency distribution and the said 2nd frequency distribution which were newly produced | generated. The log analysis device according to any one of the above. A first step of generating first statistical data indicating a frequency per first unit time of an event relating to a specific parameter recorded in a log output from a network device;
Collecting a plurality of the first statistical data related to events detected within a second unit time longer than the first unit time, the events per second unit time of the event related to the specific parameter A second step of generating second statistical data indicating the frequency;
A third step of converting a distribution on the time axis composed of a plurality of the first statistical data into a distribution on the frequency axis, and generating a first frequency distribution;
A fourth step of converting a distribution on the time axis composed of the plurality of second statistical data into a distribution on the frequency axis to generate a second frequency distribution;
A fifth step of calculating a value relating to the degree of abnormality of the network based on the first frequency distribution and the second frequency distribution;
Log analysis program characterized by causing a computer to execute.   In the second step, when the value obtained by dividing the second unit time by the first unit time is an integer, the second statistical data is generated by collecting a plurality of the first statistical data. The log analysis program according to claim 7, wherein:   In the fifth step, the ratio of the spectrum value in the specific frequency band in the first frequency distribution to the total sum of the spectrum values in all frequency bands is calculated for each of the plurality of first frequency distributions, A value related to the degree of abnormality of the network is calculated using a probability distribution obtained from a plurality of calculated ratio values, and a value related to the degree of abnormality of the network is calculated in the same manner for the second frequency distribution. The log analysis program according to claim 7 or 8, wherein   In the fifth step, a spectrum value in a specific frequency band in the first frequency distribution related to a specific network is the specific value in the first frequency distribution related to a plurality of networks including the specific network. A ratio of the total of spectrum values in the frequency band is calculated for each of the plurality of first frequency distributions, and a probability distribution obtained from the calculated values of the plurality of ratios is used to calculate a value relating to the degree of abnormality of the network. The log analysis program according to claim 7 or 8, wherein the log analysis program calculates a value related to the degree of abnormality of the network in the same manner for the second frequency distribution. A step of extracting an event related to a specific area from events related to a specific parameter recorded in the log;
The said 1st step produces | generates a plurality of said 1st statistical data for the said event which concerns on the said specific area extracted by the said 6th step. The log analysis program according to any one of 10. When the value relating to the degree of network abnormality calculated in the fifth step indicates a network abnormality,
A seventh step of generating a plurality of the first statistical data for an event relating to a parameter selected on the basis of an event frequency;
An eighth step of collectively generating a plurality of the second statistical data by combining the plurality of newly generated first statistical data;
A ninth step of generating the first frequency distribution from a plurality of newly generated first statistical data;
A tenth step of generating the second frequency distribution from a plurality of newly generated second statistical data;
An eleventh step of calculating a value relating to the degree of abnormality of the network based on the newly generated first frequency distribution and the second frequency distribution;
The log analysis program according to claim 7, further comprising: A computer-readable recording medium on which the log analysis program according to any one of claims 7 to 12 is recorded.

Images