JP4883409B2 - Data similarity inspection method and apparatus - Google Patents

Description

  The present invention relates to a method and apparatus for checking the similarity of data output by software, and in particular, the similarity of data or the similarity of software is determined based on a packet transmitted from a computer that performs unauthorized processing on a network. This is a technique related to a method and apparatus for inspection.

In the research field of incident countermeasures on the Internet, active research is being conducted to detect incidents by performing passive monitoring in a wide area network and analyzing observed traffic.
In the project nicter (see Non-Patent Document 1) for incident countermeasures promoted by the present inventors, a technique for detecting an incident in real time from traffic observed in a wide-area observation network is being studied. .
The technique for analyzing actual incidents in a wide area network is referred to as macro analysis here.

  On the other hand, research is also progressing on techniques for collecting and analyzing malware samples such as viruses, worms, and bots, and extracting the characteristics of individual malware. Analyzing malware specimens in a network space closed in this way is referred to as micro analysis in contrast to the above macro analysis.

In order to deal with incidents caused by malware promptly and accurately, it is important to identify and present the malware that caused the event (result) detected in the wide-area observation network.
In order to obtain a correlation between such an incident (result) and malware (cause), it is necessary to extract each feature effectively and perform a correlation analysis.

  Although several previous studies have been proposed as feature extraction methods for scanning attacks in micro analysis, network behavior of individual hosts is analyzed on the premise of performing correlation analysis between incidents and malware in a wide area network. There is still little research to do. That is, there is almost no technology that identifies the malware for a specific host obtained in the macro analysis by examining the correlation between the macro analysis result and the micro analysis result.

 As an automatic malware analysis method, a technique such as Patent Document 1 is known.

"Nicter: An Incident Analysis System using Correlation between Network Monitoring and Malware Analysis" Proceedings of The 1st Joint Workshop on Information Security, JWIS2006, Page363-377, 2006 September Special Table 2006-522395

In this way, in the conventional technology, analysis of micro incidents and analysis of macro malware are performed separately, and incidents occurring on the network are caused by the action of any malware that has been analyzed. However, identification had to rely on human hands.
Even when automatically identifying, if the behavior is very similar, it is easy to get a correct answer, but the actual malware is created to behave in various ways so that it is difficult to detect, so the receiving computer Depending on the situation, the mode changes variously.

  The present invention provides a technique for automatically calculating the correlation between software such as malware that performs unauthorized processing on other computers in this way and software to be inspected at high speed and accurately.

In order to solve the above-described problems, the present invention provides the following data similarity inspection method.
That is, according to the first aspect of the present invention, the first transmission data transmitted by the first software that performs unauthorized processing on another computer on the network and the second transmission transmitted by the second software to be inspected. It is intended to provide a data similarity inspection method for a computer which compares the transmitted data with each other and inspects the similarity.
In this method, with respect to transmission data, at least one of a plurality of parameter types to which attention is paid in advance for any information included in a packet to be transmitted is stored as profile information in a storage unit.

  Then, the first packet analysis means refers to the profile information, receives the first packet transmitted from the first software, and selects a part or all of the parameters corresponding to the parameter type stored therein. get. This is recorded in the storage means as first profile data. (First packet analysis step)

  Then, the second packet analysis means refers to the profile information, receives the second packet transmitted from the second software, acquires part or all of the parameters corresponding to the parameter type, and receives the second packet. 2 is recorded in the storage means as profile data. (Second packet analysis step)

  In the subsequent correlation degree calculation step, the correlation degree calculation means of the computer uses a correlation expression corresponding to the type of parameter stored in advance in the storage means from the first profile data and the second profile data. The correlation is calculated, and the correlation is output in the output step.

The present invention can also provide the following data similarity inspection apparatus.
That is, as described in claim 14, the first transmission data transmitted by the first software that performs illegal processing on another computer on the network and the second transmission transmitted by the second software to be inspected. A data similarity inspection device that compares the transmitted data with each other and inspects the similarity is provided.

The apparatus includes a storage unit that stores, as profile information, one or more parameter types to which attention is paid in advance for at least one of information included in a transmitted packet with respect to transmission data.
Further, the profile information is referred to, the first packet transmitted from the first software is received, a part or all of the parameters corresponding to the parameter type are acquired, and the first profile data is stored in the storage means. First packet analysis means for recording, referring to profile information, receiving a second packet transmitted from the second software, obtaining part or all of the parameters corresponding to the type of the parameter, Second packet analysis means for recording the profile data in the storage means.

  Correlation degree calculation means for a computer that uses each profile data recorded by each of the packet analysis means as described above and calculates a correlation degree using a correlation equation corresponding to the type of parameter stored in the storage means in advance. Is provided.

  In the present invention, in the profile information, part or all of the port number of the destination port of the packet is stored as a parameter type, and the first and second packet analyzing means respectively determine the destination port number from the packet. The structure to extract may be sufficient.

  In the profile information, the port number of the destination port of the packet and the number of transmissions for each port number are stored as parameter types, and the first and second packet analyzing means respectively extract the destination port number from the packet. In addition, the configuration may be such that the number of packet transmissions for each port number in a predetermined time or in a predetermined number of execution processes of software is counted.

  In the profile information, transition information between the port numbers of the transmission destination ports of the packet is stored as parameter types, and the first and second packet analysis means respectively extract the transition information of the transmission destination port number from the packet. It may be configured.

  In the profile information, when there are a plurality of destination addresses of a packet, a difference value between the destination addresses or a calculation for deriving a predetermined statistical value based on the difference value is stored as a parameter type. The packet analysis unit may extract each transmission destination address from the packet and calculate a difference value between the transmission destination addresses or a predetermined statistical value based on the difference value.

  In the profile information, the destination address of the packet and the number of destination addresses to be transmitted are stored as parameter types, and the first and second packet analysis means respectively execute the execution process at a predetermined time or a predetermined number of times of software. The number of transmission destination addresses that transmit packets may be counted.

  In the profile information, the number of port numbers of the transmission source port of the packet is stored as a parameter type, and the first and second packet analysis units respectively store the packet at a predetermined time or in a predetermined number of execution processes of software. It may be configured to count the number of transmission source port numbers to be transmitted.

  In the profile information, when there are a plurality of port numbers of the source port of the packet, a difference value between the source port numbers or a calculation for deriving a predetermined statistical value based on the difference value is stored as a parameter type. Each of the first and second packet analyzing means may extract each transmission source port number from the packet and calculate a difference value between the transmission source port numbers or a predetermined statistical value based on the difference value.

  In the profile information, at least one type or the number of a protocol that the packet conforms to or a flag in the TCP protocol is stored as a parameter type, and the first and second packet analysis means respectively store the protocol or The flag may be acquired from the packet, and the number may be counted when the number is a parameter type.

  In the profile information, a calculation for deriving a predetermined statistical value based on a digest value of the payload component included in the packet and / or its number is stored as a parameter type, and the first and second parameters are stored. Each of the two-packet analysis means extracts the digest value of each payload from the packet, and counts the number of digest values if the number is a parameter type, and a predetermined statistical value based on the digest value or the number is obtained. In the case of a parameter type, a configuration for calculating it may be used.

  In the profile information, a data amount of a payload component included in the packet or a calculation for deriving a predetermined statistical value based on the data amount is stored as a parameter type, and the first and second packet analyzing means respectively The payload component may be extracted from the packet, and the data amount may be measured, or a predetermined statistical value based on the measured data amount may be calculated.

In the data similarity check method according to the present invention, the transmission data is stored in the storage means as profile information including at least the number of transmitted packets reaching the destination port at a predetermined time with respect to the transmission data. You can also keep it.
In this configuration, the number at which the first packet analyzing means receives the first packet transmitted from the first software at a predetermined time at any time before the correlation degree calculating step with reference to the profile information. The first packet reception frequency counting step for counting the number of times recorded in the storage means as the first profile data, the second packet analysis means refers to the profile information, and determines the second packet transmitted from the second software as a predetermined value. Each step of the second packet reception count counting step of counting the number received in time and recording it in the storage means as the second profile data is performed.
It is also possible to provide a data similarity inspection device that implements the present technology.

The present invention may provide the following malware inspection method by using the data similarity inspection method.
That is, according to the invention described in claim 13, as the first software, a known malware that is executed for inspection in a closed network is actually executed as a second software in a wide area network. The first packet analysis step is executed for one or a plurality of pieces of first software using each of the inspection target software.

  Thereafter, an inspection start instruction signal sending means of the computer sends an inspection start instruction signal based on the transmission data content or the packet of transmission data from the second software, triggered by this inspection start instruction signal The second packet analyzing step to be executed and the correlation degree calculating means of the computer use the one or more first profile data and the second profile data to store the types of parameters stored in advance in the storage means A correlation degree calculating step for calculating a correlation degree using a correlation formula corresponding to the first step and a malware correlation degree outputting step for outputting at least a part of each correlation degree with the first software are sequentially executed.

Furthermore, it can also be provided as a malware inspection system for inspecting malware.
That is, the data similarity inspection apparatus is provided, the first software is a known malware executed for inspection in a closed network, and the second software is actually executed in a wide area network. The first packet analysis means records each first profile data for one or a plurality of first software, and transmits transmission data contents or transmission data from the second software. A computer test start instruction signal sending means for sending a test start instruction signal based on the packet is provided.

  Then, the second packet analyzing means operates when receiving the inspection start instruction signal, and the correlation degree calculating means stores in advance storage means from the one or plural first profile data and the second profile data. The degree of correlation is calculated using a correlation formula corresponding to the type of parameter stored in, and the output means outputs at least a part of each degree of correlation with the first software.

By providing the above configuration, the present invention has the following effects.
First, when checking the similarity of data transmitted by two software programs, it is possible to focus on the types of suitable parameters from the packets output from both software programs and store them as profile information.

  And while extracting a parameter based on profile information, both are compared based on the correlation formula set for every parameter. According to this method, it is easy to obtain a degree of correlation by fusing a plurality of parameters even for malware that has a complicated packet transmission mode and changes frequently, and as a result, calculates the degree of correlation with high accuracy. be able to.

  In addition, it is possible to respond quickly and easily to detection of new malware simply by rewriting the profile information, and it is an optimal inspection method for malware that has evolved rapidly and in many ways.

  In particular, in the present invention, a particularly suitable parameter type is provided, so that a correlation degree can be efficiently obtained for a packet of general fraudulent processing of malware.

Hereinafter, embodiments of the present invention will be described based on examples shown in the drawings. The embodiment is not limited to the following.
FIG. 1 is an overall configuration diagram of a data similarity inspection apparatus (hereinafter referred to as this apparatus) (1) according to the present invention. The apparatus (1) can be easily constituted by a known personal computer or network server.

In this device (1), a CPU (10) that controls arithmetic processing and the like is the center, a memory (11) that works with the CPU (1), a keyboard and mouse (12) for user input, etc. A hard disk (13) stored in the network, a network adapter (14) for network connection such as the Internet, and the like. It is also possible to display a screen by connecting a monitor (not shown) or to output a sound by connecting a speaker.
These configurations are all well-known matters, and the description of the structure and operation is omitted.

  In the present invention, the CPU (10) includes a first packet analysis unit (20), a second packet analysis unit (30), a correlation degree calculation unit (15), and an output unit (16). The first and second packet analysis units acquire data extraction units (21) and (31), timing units (22) and (32), and counting units (23) and (33) in order to acquire various parameters described below. And arithmetic units (24) and (34).

First, the most basic processing of the present invention will be described with reference to FIG. In the present embodiment, malware is used as the first software for performing unauthorized processing, and software to be inspected is used as the second software.
First, profile information (13a) is read. (Profile reading process: S10)
Here, the profile information (13a) defines in advance the types of parameters to be focused on and the extraction method for checking the similarity of data transmitted from the software. The types of parameters used in the present invention are, for example, the port number at the destination host included in the packet, the port number of the source port, etc., which are defined by TCP (Transmission Control Protocol) (RFC793). ) In the first 0 to 15 bits (transmission source port number) and 16 bits to 31 bits (transmission destination port number) in the header portion.
In the present invention, what kind of parameter is used is also important.

  Specifically, when the parameter type is stored in the hard disk, it is recorded as the name of a variable used in the computer. For example, the variable representing the type of port number at the destination host is “DstPort”, and the port number of the source host is “SrcPort”. The parameters are data such as “139” and “445”. Hereinafter, the parameter types and parameters are distinguished from each other in this way.

In the first profile information reading process (S10), first, in this way, the parameter type (data relating to the variable and its extraction method) is read and processed. A specific processing method is based on the cooperation of the CPU (10) and the hard disk (13) and is well known.
Then, the packet from the malware is analyzed based on the read profile information (13a). (First packet analysis processing: S11)

Since the analysis method here varies depending on the profile information, the analysis method will be described later together with specific parameter types.
Then, the obtained parameter data is recorded in the storage means such as the hard disk (13) or the memory (11) as the first profile data (13b). (Profile recording process: S12)
The processes (S10 to S12) are processes performed by the first packet analysis unit (20).

  In the present invention, the first profile data relating to malware is generated through the above-described processes. It is only necessary to create profile data for one malware, but it is preferable to generate profile data for a number of malwares, which is the most correlated from the number of candidates when compared to the next software to be examined. Can identify malware.

The inspection target software is processed in the same way as malware. That is, the profile information (13a) is read (profile information reading process: S20), and the packet transmitted by the software to be inspected is analyzed based on the profile information. (Second packet analysis processing: S21)
Then, the acquired parameter is recorded as second profile data (13c). (Profile recording process: S22)
Each of the above processes (S20 to S22) is a process by the second packet analysis unit (30).

In order to compare the second profile data (13c) with the first profile data (13b) recorded in advance, the correlation calculation unit (15) performs correlation based on a predefined correlation equation. Calculate the degree. (Correlation degree calculation process: S30)
The correlation equation is preferably defined for each parameter type of the profile information. In this case, the correlation formula is stored in the profile information corresponding to the parameter type. However, in the present invention, one type of correlation equation may be stored separately in the hard disk (13). The correlation formula will also be described later.

And the correlation degree for every malware is output from an output part (16). (Output processing: S31)
The output method may be any method such as displaying a screen from a monitor in a list format, printing output from a printer, and transmitting data to another computer via a network adapter (14). The output unit (16) of the CPU (10) works with known hardware to manage the same processing.

  The present invention can also be provided as a malware identification method or malware identification system. The basic technique is the same as that of the data similarity inspection apparatus, but it is proposed to identify malware in combination with the nicter proposed by the applicants (see Non-Patent Document 1).

  In FIG. 3, first, a packet or the like for the dark net is detected by a plurality of sensors (41) provided in the wide area network (40) and input to the macro analyzer (42). The result of the macro analysis is stored in the database (43).

  On the other hand, on the network (44), a large number of malware specimens are collected by the capture (45), and their static and dynamic properties are analyzed by the microanalyzer (46). The analysis result is also stored in the database (47).

  In this way, the malware that actually caused the incident is analyzed in a macro manner by the macro analyzer, and the sample is analyzed to perform a micro analysis of the malware. From each database, the correlation analyzer (48) is used. Correlation analysis is considered.

  The result of the correlation analysis is stored in the database (49), and is notified to the user (51) or output as a report (52) via the incident handling system (50) by various output methods.

The present invention is applied to this system, and the data similarity inspection device (1) of the present invention is mounted on the correlation analyzer (48) together with the macro analyzer (42) and the micro analyzer (46).
However, the present apparatus (1) is divided, the macro analyzer (42) is provided with the second packet analysis unit (30), and the micro analyzer (46) is provided with the first packet analysis unit (20). And the result may be subjected to correlation analysis by a correlation analyzer (48) provided with a correlation calculation unit (15).

  Conventionally, it has been technically difficult to combine the results of macro analysis and micro analysis, but this is realized by applying the method of the present invention, and the cause of an incident occurring in a wide area network can be determined quickly and accurately. Can be specified.

As an operation in the micro analyzer (46), analysis using a malware dynamic analysis environment as shown in FIG. 4 is suitable.
In other words, the malware dynamic analysis environment is a miniature garden environment in which a local address space and a global address space are virtually provided to measure how an infected host behaves in each space.

Then, how to scan the global address from the infected host (70) to the host (61) of each IP address in the virtual global space (60), or a dummy IRC server (62), HTTP server (63) ), How to access the FTP server (64) and the TFTP server (65).
By using such a virtual global space (60), the behavior to the global address space can be measured.

In the virtual local space (66), how to scan local addresses from the infected host (70) to the hosts (67) having a plurality of local IP addresses, or a dummy SMTP server (68), It measures how to access the DNS server (69).
By using such a virtual local space (66), the behavior to the local address space can be measured.

On the other hand, a packet transmitted to an IP address area called a dark net that is not actually used is detected by a sensor (44) on the wide area network (40).
Since packets directed to such an IP address are not directed to a host that complies with the rules, it is considered to be a misconfiguration such as a misconfiguration or a scan, search by worm, or backscatter mail.

Here, the sensor (44) is composed of a network terminal to which an IP address corresponding to a dark net is assigned.
Furthermore, in the present invention, a function of detecting a known illegal behavior is given to the sensor (44) or the macro analyzer (42). For example, although the algorithm of the method is described in Non-Patent Document 2, various techniques can be applied to the configuration of the macro analyzer as disclosed in Non-Patent Document 3 and the like. Hereinafter, the sensor (44) will be described as having the same function.

Suzuki, K., Baba, S., Takakura, H .: Analyzing traffic directed to unused IPaddress blocks, IEICE Technical Report, vol.105, no.530, IA2005-23, pp.25-30 January 2006 Takeuchi, J., Sato, Y., Rikitake K., Nakao K .: Development of Incident Detection System Based on Change Point Detection, SCIS2006, The 2006 Symposium on Cryptography and Information Security, Japan, January 2006

  When the apparatus (1) is mounted on the correlation analyzer (48), it is convenient to use the raw data of the received packet as the inspection start instruction signal. Based on the signal, the profile information reading process of FIG. You may perform from a correlation calculation process (S30) to an output process (S31) from (S20).

At this time, an apparatus configuration as shown in FIG. 5 may be taken.
That is, in the correlation analyzer (48) of FIG. 3, a profile data generation unit (80) and a correlation degree calculation unit (82) are provided, and the profile data generation unit (80) includes the first packet analysis unit (20) and the first packet analysis unit. The processing unit common to the two-packet analysis unit (30) is provided, and the correlation degree calculation unit (82) is the same unit as the correlation degree calculation unit (15) of the apparatus (1).

In this configuration, the profile data generation unit (80) creates a plurality of profile data (83) to (87) for each malware obtained from the micro analysis in the correlation analysis result database (49) in advance.
When raw data from a dark net is input as an inspection start instruction signal, the same profile data generation unit (80) creates profile data (81) related to the raw data, and a correlation degree calculation unit (82) Calculate the degree.

  As described above, the first packet analysis unit, the second packet analysis unit, and the like in the present invention may be divided into a plurality of devices without being mounted on one device, and each packet analysis unit may be provided. It may be divided and arranged in a plurality of devices according to the function. Further, since each packet analysis unit has the same processing contents, it may be configured by one processing unit as described above.

Next, detailed processing contents of the packet analysis unit according to the present invention will be described. Here, since it has an important relationship with the type of parameter in the profile information (13a), it will be described together with a suitable type of parameter.
First, the types of parameters used in this embodiment include the items listed in Table 1 below.

(DstPort_Count)
The information on which destination port number the packet of transmission data in the illegal processing is sent to is the most basic information that characterizes malware.
Therefore, DstPort_Count as the first parameter represents the port number of the packet destination port and the number of transmissions for each port number. And the data extraction part (21) in a 1st packet analysis part (20) extracts the part of a transmission destination port number from the header part of the transmitted packet.
In this embodiment, only the destination port number may be used, but how many times the packet is sent to the destination port number is also used as a parameter.

Here, the number of times the packet has been sent is counted per unit time. That is, the transmission port number is extracted by the data extraction unit (21), the unit time is counted by the timing unit (22), and the number of transmissions during that time is counted by the counting unit (23). The unit time is a time determined as appropriate, for example, 1 second.
Hereinafter, the number of transmissions is obtained in the same manner for other parameters.

(DstPort_Trans)
Next, DstPort_Trans is transition information between the port numbers of the packet transmission destination ports. The transition information is information regarding the order in which the destination port is scanned in, for example, unit time or a series of packet transmissions to the host.
For example, when scanning is performed in the order of port numbers 139, 445, 3127, and 6659, information that 445 is 445 is used as a parameter.
More preferably, the probability that the next of 139 is 445 may be used as a parameter using the transition probability. Specifically, the transition order is stored, and the ratio of packets sent after 139 for each port number is defined as the transition probability. For example, the top 10 transition probabilities may be stored without storing all the port numbers.

(SrcPort_Unique)
The source port number has a feature that malware can freely set as compared with the destination port number. Since some malware uses a fixed source port number, it is preferable to use which port is used as a parameter. Further, even when the source port number varies, how the port number is changed can be used as a parameter.
Therefore, the data extraction unit (22) extracts the destination port number and stores the number as a parameter in the profile data.

(SrcPortDif_Stats)
As for the variation of the transmission source port number, the transmission source port number is sequentially extracted by the data extraction unit (21), the difference between the previous and next port numbers is obtained by the calculation unit (24), and the average value is calculated. . In addition, you may obtain | require a statistical value using arbitrary statistical calculation formulas other than an average value.
This is a suitable index when the transmission source port number is changed for each value and transmitted.

(DstIPDif_Stats)
In the global address space and the local address space, it is possible to use what IP address the packet is sent to. How the IP address is scanned is important for the malware to search for the target, and there are various methods for the search order. This information is useful for identifying malware such as whether the subnet mask is 16 bits or 24 bits, whether to search from a close IP address or randomly.
This information can be acquired from the IP address assigned to the virtual host.

  The IP address itself can be used as a parameter, such as when the IP address to be transmitted has unique characteristics. The IP address is detected in order, converted into a numerical value by a prescribed method, and the difference is obtained by the calculation unit. An average value may be used. The average value may be another statistical value.

(DstIP_Unique)
The DstIP_Unique variable is a parameter related to how many destination IP addresses are transmitted. Therefore, the number of destination IP addresses per unit time is counted. Malware includes those that transmit a large number of packets per unit time and those that transmit a certain amount of time, and the interval can be used as an index. It may be the number of times in a unit time or other predetermined time, or may be the transmission frequency in the transmission of a series of packets.

(Protocol_Count)
The Protocol_Count variable uses as a parameter the number of packets sent by the malware and the number of packets for each protocol. Since the malware uses various protocols, the data extraction unit (21) specifies whether TCP or UDP is used and which protocol is being scanned.

(Flag_Count)
Similarly, when TCP is used as a protocol, information on what kind of packet comes to which flag of TCP can also be used.
Examples of the flag include a URG (urgent) flag, an ACK (acknowledge) flag, a PSH (push) flag, an RST (reset) flag, a SYN (synchronize) flag, and a FIN (finis) flag.
Various scanning methods are known, and the type of scan such as TCP SYN scan, TCP FIN scan, TCPNull scan, TCP Xmas scan, TCP Maimon scan, TCP ACK scan, and UDP scan may be used as parameters.

(NumPacketRate)
In the present invention, information on how many packets are transmitted to the destination host per unit time can also be used. There are cases where it is sent continuously by malware, or packets are sent over time.
In the present invention, the time counting unit (22) and the counting unit (23) are used to count the parameters.

(PayloadSig_Count)
There are cases where the payload field of a packet contains an attack code, such as malware that searches by UDP. Therefore, it is also effective to compare payload digest values. First, actual data (digest value) is extracted, and the value itself or the number of digest values (number of types) can be used as a parameter.

(Payload_Stats)
Payload_Stats is a statistical value of the size of the payload. Any statistical value such as an average value can be used.

(TTL_Stats, Id_Stats, SeqNum_Stats)
In addition, in this embodiment, values such as a TTL field, an ID field, a sequence number field, and the like in TCP can be used.

The present invention proposes to use the types of parameters as described above, and the first packet analysis unit (20) uses this as profile data, the data extraction unit (21), the time measuring unit (22), the counting unit. (23) The analysis is performed by the calculation unit (24).
In the same manner, the second packet analyzing unit (30) analyzes the second software by the data extracting unit (31), the time measuring unit (32), the counting unit (33), and the calculating unit (34). .

Further, the correlation degree calculation unit calculates the correlation degree from the first profile data (13b) and the second profile data (13c) by an arbitrary correlation calculation formula.
Next, the correlation formula used in the present embodiment will be described.

The profile data used in the present invention is often multidimensional data, such as when there is a value for each port number, and it is necessary to calculate the degree of correlation between these multidimensional data.
For example, pεP _A and qεP _B are set for the port number group P _A used by the first software and the port number group P _B used by the second software. At this time, the number of transmissions C _A (p) of the first software and the number of transmissions C _B (q) of the second software at each port number can be expressed.

Consider C _A (p) and C _B (p) as two random variables. At this time, p∈P _A ∪P _B = Ω (sample space). The correlation between C _A (p) and C _B (p) is expressed by the following equation (Equation 1). Note that m _A and m _B are average values of C _A (p) and C _B (p), respectively.

  In the correlation degree calculation unit (15), the degree of correlation of the multidimensional data is obtained by calculating the degree of correlation according to the above formula. The calculation method can be based on a known program method. Similarly, multi-dimensional data includes DstPort_Count, Protocol_Count, Flag_Count, PayloadSig_Count, and the like.

On the other hand, the one-dimensional data includes SrcPort_Unique, DstIP_Unique, NumPacketRate, and the like, and the similarity can be obtained simply by the following equation (Equation 2).
(Equation 2)
(A / b) ^k

Note that a <b and k can be arbitrarily defined, and 1 is used here.

  The correlation expression used in the correlation calculation unit (15) is arbitrary depending on the profile data, but when the parameter is a data series, the following correlation expression can be used.

(1) Pearson's product moment correlation coefficient
For the two data strings x = {x _i }, y = {y _i }, i = 1, 2,..., N, the Pearson product moment correlation coefficient is obtained by the following equation (Equation 3).

ここで，

はそれぞれデータ系列x,yの平均値をあらわす．

here,

Represents the mean of the data series x and y, respectively.

(2) Spearman's rank correlation coefficient
For two data strings x = {x _i }, y = {y _i }, i = 1, 2,..., N, Spearman's rank correlation coefficient is obtained by the following equation (Equation 4).

ここで、D_iはx_iの系列x内での順位とy_iの系列y内での順位の差を表す。

Here, D _i represents the difference between the rank of x _{i in} the sequence x and the rank of y _{i in} the sequence y.

(3) Kendall's rank correlation coefficient
For two data strings x = {x _i }, y = {y _i }, i = 1, 2,..., N, Kendall's rank correlation coefficient is obtained by the following equation (Equation 5).

ここで、Kは、i=1, 2, …, nについてx_iの系列x内での順位がy_iの系列y内での順位より大きい（または順位が等しい）場合の数、Lは、y_iの系列y内での順位がx_iの系列x内での順位より大きい(または順位が等しい)場合の数とする。

Here, K is the number of cases where the rank in the sequence x of x _i is greater than (or equal to) the rank in the sequence y of y _i for i = 1, 2,. It is the number when the rank of y _{i in} the sequence y is greater than (or equal to) the rank in the sequence x of x _i .

(4) Ratio that satisfies specific conditions
For two data strings x = {x _i }, y = {y _i }, i = 1,2, ..., n, the number of cases where x _i and y _i both satisfy a specific condition is g _condition times . At this time, the similarity between x and y is defined as follows. (Equation 6)

(Equation 6)
g _condition / n

For example, when x _i and y _i are real values, x _i > 0 and y _i > 0 are considered as conditions.

  In addition, distance (dissimilarity) can be considered as a concept opposite to similarity, but there is the following definition of distance between two data series. When the distance is used for correlation analysis, it is considered that the similarity is higher as the distance between the two profile data is smaller.

(5) Euclidean distance
For the two data strings x = {x _i }, y = {y _i }, i = 1, 2,..., N, the Euclidean distance is obtained by the following equation (Equation 7).

(6) Euclidean square distance
For the two data strings x = {x _i }, y = {y _i }, i = 1, 2,..., N, the Euclidean square distance is obtained by the following equation (Equation 8).

(7) Chebyshev distance
For the two data strings x = {x _i }, y = {y _i }, i = 1, 2,..., N, the Chebyshev distance is obtained by the following equation (Equation 9).
(Equation 9)
max (| x _i -y _i |)

Here, | x | represents the absolute value of x. Max X represents the maximum value of X for i = 1, 2, ..., n.

(8) Manhattan distance
For the two data strings x = {x _i }, y = {y _i }, i = 1, 2,..., N, the Manhattan distance is obtained by the following equation (Equation 10).

(9) Mahalanobis distance
For the two data strings x = {x _i }, y = {y _i }, i = 1, 2,..., N, the Mahalanobis distance is obtained by the following equation (Equation 11).

但し、s^ijは、分散共分散行列s^ijの逆行列のi,j要素とする。

Here, s ^ij is the i, j element of the inverse matrix of the variance-covariance matrix s ^ij .

(10) Minkowski distance
For the two data strings x = {x _i }, y = {y _i }, i = 1, 2,..., N, the Minkowski distance is obtained by the following equation (Equation 12).

ここでrとpは重みを調整するためのパラメータであり、r≠0の範囲で任意の値を設定可能である。pは個々のデータの差分に与える重みを調節するために用い、rは全体としての距離を調節するために用いる。

Here, r and p are parameters for adjusting the weight, and arbitrary values can be set in the range of r ≠ 0. p is used to adjust the weight given to the difference between individual data, and r is used to adjust the distance as a whole.

(11) Mismatch rate
For two data strings x = {x _i }, y = {y _i }, i = 1, 2,..., N, when data x _i , y _i do not match c times, the distance between x and y Can be defined as follows: (Equation 13)

(Equation 13)
c / n

Also, the similarity between x and y can be defined as follows. (Equation 14)

(Equation 14)
(nc) / n

  As described above, the present invention has the greatest feature in comparing the transmission data of two software using profile data. These comparisons include various correlation equations shown above, Any known correlation equation can be used.

It is a block diagram of the data similarity inspection apparatus which concerns on this invention. It is a process flowchart of the data similarity inspection method which concerns on this invention. It is a block diagram of the specific system of the malware which concerns on this invention. It is explanatory drawing of the malware analysis method using the malware dynamic analysis environment in this invention. It is a block diagram which shows another structure of the correlation analysis part in this invention.

Explanation of symbols

1 Data similarity inspection device 10 CPU
DESCRIPTION OF SYMBOLS 11 Memory 12 Keyboard / mouse 13 Hard disk 13a Profile information 14 Network adapter 15 Correlation degree calculation part 16 Output part 20 1st packet analysis part 21 Data extraction part 22 Counting part 23 Counting part 24 Calculation part 30 2nd packet analysis part 31 Data extraction Unit 32 Timekeeping unit 33 Counting unit 34 Calculation unit

Claims (26)

The first transmission data transmitted by the first software that performs unauthorized processing on another computer on the network is compared with the second transmission data transmitted by the second software to be inspected, and the similarity A computer data similarity inspection method for inspecting
With respect to the transmission data, at least one of the types of parameters to be noted in advance for any of the information included in the transmitted packet is stored in the storage means as profile information,
The first packet analysis means refers to the profile information, receives the first packet transmitted from the first software, acquires part or all of the parameters corresponding to the parameter type, A first packet analysis step of recording in the storage means as profile data;
The second packet analysis means refers to the profile information, receives the second packet transmitted from the second software, acquires a part or all of the parameters corresponding to the parameter type, and receives the second packet A second packet analysis step of recording in the storage means as profile data;
Correlation in which the correlation degree calculating means of the computer calculates the degree of correlation from the first profile data and the second profile data using a correlation equation corresponding to the type of parameter stored in advance in the storage means Degree calculation step,
A data similarity inspection method comprising: an output step of outputting the degree of correlation. In the profile information,
Store part or all of the port number of the packet destination port as a parameter type,
The data similarity checking method according to claim 1, wherein each of the first and second packet analyzing means extracts a destination port number from the packet. In the profile information,
Store the port number of the packet destination port and the number of transmissions for each port number as a parameter type,
Each of the first and second packet analyzing means extracts a destination port number from the packet, and counts the number of packet transmissions for each port number in a predetermined time or in a predetermined number of execution processes of software. The data similarity inspection method according to claim 1 or 2. In the profile information,
Store transition information between port numbers of packet destination ports as parameter types,
The data similarity checking method according to any one of claims 1 to 3, wherein the first and second packet analysis units each extract transition information of a transmission destination port number from a packet. In the profile information,
When there are a plurality of transmission destination addresses of a packet, a difference value between the transmission destination addresses or a calculation for deriving a predetermined statistical value based on the difference value is stored as a parameter type,
The first and second packet analysis units respectively extract each destination address from a packet and calculate a difference value between the destination addresses or a predetermined statistical value based on the difference value. 5. The data similarity inspection method according to any one of 1 to 4. In the profile information,
Store the packet destination address and the number of destination addresses to send as parameter types,
6. The method according to claim 1, wherein each of the first and second packet analyzing means counts the number of transmission destination addresses to which packets are transmitted in a predetermined time or in a predetermined number of execution processes of software. The data similarity test method described. In the profile information,
Store the number of port numbers of the packet source port as a parameter type,
The first and second packet analysis units respectively count the number of transmission source port numbers that transmit packets in a predetermined time or a predetermined number of execution processes of software. The data similarity test method described in 1. In the profile information,
When there are a plurality of port numbers of the source port of the packet, a difference value between the source port numbers or a calculation for deriving a predetermined statistical value based on the difference value is stored as a parameter type,
Each of the first and second packet analyzing means extracts each transmission source port number from the packet, and calculates a difference value between the transmission source port numbers or a predetermined statistical value based on the difference value. The data similarity inspection method according to claim 1. In the profile information,
Store at least one of the protocol conforming to the packet or the flag in the TCP protocol or the number thereof as the parameter type,
Each of the first and second packet analyzing means obtains the protocol or flag from the packet and counts the number if the number is a parameter type.
9. The data similarity inspection method according to claim 1, wherein In the profile information,
A calculation for deriving a predetermined statistic value based on the digest value of the payload component included in the packet or the number thereof or one of them is stored as a parameter type,
Each of the first and second packet analyzing means extracts the digest value of each payload from the packet, and counts the number of digest values when the number is a parameter type, and based on the digest value or the number thereof The data similarity inspection method according to any one of claims 1 to 9, wherein when the predetermined statistical value is a parameter type, it is calculated. In the profile information,
Store the amount of data of the payload (Payload) component included in the packet or a calculation that derives a predetermined statistical value based on the amount of data as a parameter type,
The first and second packet analyzing means each extract the payload component from a packet, measure the data amount, or calculate a predetermined statistical value based on the measured data amount. Item 11. The data similarity inspection method according to any one of Items 1 to 10. In the computer data similarity inspection method,
Regarding the transmission data, at least the number of packets transmitted to the destination port at a predetermined time is included in the parameter type and stored in the storage means as profile information,
At any point before the correlation degree calculating step, the first packet analysis means refers to the profile information and counts the number of reception of the first packet transmitted from the first software in a predetermined time. A first packet reception count counting step for recording in the storage means as first profile data;
The second packet analysis means refers to the profile information, counts the number of second packets transmitted from the second software in a predetermined time, and records the number as second profile data in the storage means. The data similarity checking method according to any one of claims 1 to 11, wherein each step of counting the number of received packets is performed. The first software is known malware that is executed for inspection in a closed network, and the second software is software to be inspected that is actually executed in a wide area network;
The first packet analysis step in the data similarity checking method according to any one of claims 1 to 12 is executed for one or more pieces of first software,
A test start instruction step in which a test start instruction signal sending means of the computer sends a test start instruction signal based on the transmission data content or the packet of the transmission data from the second software;
The second packet analysis step executed by the inspection start instruction signal as a trigger;
Correlation degree calculation means of the computer uses the correlation equation corresponding to the type of parameter stored in advance in the storage means from the one or more first profile data and the second profile data. Correlation degree calculation step for calculating
A malware inspection method comprising: a malware correlation output step for outputting at least a part of each correlation with the first software. The first transmission data transmitted by the first software that performs unauthorized processing on another computer on the network is compared with the second transmission data transmitted by the second software to be inspected, and the similarity A data similarity inspection device for inspecting
Storage means for storing, as profile information, the type of parameter or parameters of interest for at least any of the information included in the transmitted packet with respect to the transmission data;
Referring to the profile information, receiving the first packet transmitted from the first software, obtaining a part or all of the parameters corresponding to the parameter type, and recording them in the storage means as the first profile data First packet analysis means for
Referring to the profile information, receiving the second packet transmitted from the second software, obtaining a part or all of the parameters corresponding to the parameter type, and recording them in the storage means as second profile data Second packet analysis means for
A correlation degree calculating means of a computer for calculating a degree of correlation from the first profile data and the second profile data using a correlation equation corresponding to the type of parameter stored in the storage means in advance;
An output means for outputting the degree of correlation. In the profile information,
Store part or all of the port number of the packet destination port as a parameter type,
The data similarity checking device according to claim 14, wherein each of the first and second packet analyzing means extracts a destination port number from the packet. In the profile information,
Store the port number of the packet destination port and the number of transmissions for each port number as a parameter type,
Each of the first and second packet analyzing means extracts a destination port number from the packet, and counts the number of packet transmissions for each port number in a predetermined time or in a predetermined number of execution processes of software. The data similarity inspection device according to claim 14 or 15. In the profile information,
Store transition information between port numbers of packet destination ports as parameter types,
The data similarity checking device according to any one of claims 14 to 16, wherein each of the first and second packet analyzing units extracts transition information of a transmission destination port number from a packet. In the profile information,
When there are a plurality of transmission destination addresses of a packet, a difference value between the transmission destination addresses or a calculation for deriving a predetermined statistical value based on the difference value is stored as a parameter type,
The first and second packet analysis units respectively extract each destination address from a packet and calculate a difference value between the destination addresses or a predetermined statistical value based on the difference value. The data similarity inspection apparatus according to any one of 14 to 17. In the profile information,
Store the packet destination address and the number of destination addresses to send as parameter types,
The said 1st and 2nd packet analysis means each counts the number of the transmission destination addresses which transmit a packet in predetermined time or in the execution processing of the predetermined number of times of software. The data similarity test device described. In the profile information,
Store the number of port numbers of the packet source port as a parameter type,
The first and second packet analysis units respectively count the number of transmission source port numbers that transmit packets in a predetermined time or a predetermined number of execution processes of software. The data similarity inspection device described in 1. In the profile information,
When there are a plurality of port numbers of the source port of the packet, a difference value between the source port numbers or a calculation for deriving a predetermined statistical value based on the difference value is stored as a parameter type,
Each of the first and second packet analyzing means extracts each transmission source port number from the packet, and calculates a difference value between the transmission source port numbers or a predetermined statistical value based on the difference value. The data similarity inspection apparatus according to any one of claims 14 to 20. In the profile information,
Store at least one of the protocol conforming to the packet or the flag in the TCP protocol or the number thereof as the parameter type,
Each of the first and second packet analyzing means obtains the protocol or flag from the packet and counts the number if the number is a parameter type.
The data similarity inspection device according to any one of claims 14 to 21, wherein In the profile information,
A calculation for deriving a predetermined statistic value based on the digest value of the payload component included in the packet or the number thereof or one of them is stored as a parameter type,
Each of the first and second packet analyzing means extracts the digest value of each payload from the packet, and counts the number of digest values when the number is a parameter type, and based on the digest value or the number thereof The data similarity inspection apparatus according to any one of claims 14 to 22, wherein when the predetermined statistical value is a parameter type, it is calculated. In the profile information,
Store the amount of data of the payload (Payload) component included in the packet or a calculation that derives a predetermined statistical value based on the amount of data as a parameter type,
The first and second packet analyzing means each extract the payload component from a packet, measure the data amount, or calculate a predetermined statistical value based on the measured data amount. Item 24. The data similarity inspection device according to any one of Items 14 to 23. In the computer data similarity inspection apparatus,
The storage means includes, as profile information, including at least the number of packets to be transmitted to the transmission destination port at a predetermined time with respect to the transmission data as a parameter type,
The first packet analyzing means refers to the profile information, counts the number of first packets transmitted from the first software in a predetermined time, records the first packet data in the storage means,
The second packet analyzing means refers to the profile information, counts the number of second packets transmitted from the second software in a predetermined time, and records the number as the second profile data in the storage means. 25. The data similarity inspection apparatus according to claim 14, wherein the data similarity inspection apparatus is a data similarity inspection apparatus. A malware inspection system for inspecting malware,
A data similarity inspection device according to any one of claims 14 to 25,
In a configuration in which the first software is known malware executed for inspection in a closed network, and the second software is software to be inspected that is actually executed in a wide area network.
The first packet analysis means records each first profile data for one or a plurality of first software,
A computer test start instruction signal sending means for sending a test start instruction signal based on the transmission data content or the packet of transmission data from the second software;
The second packet analyzing means operates when receiving the inspection start instruction signal,
The correlation degree calculating means calculates a correlation degree from the single or plural first profile data and the second profile data by using a correlation expression corresponding to the type of parameter stored in the storage means in advance. Calculate
The malware inspection system, wherein the output means includes outputting at least a part of each correlation degree with the first software.

Images