JP6880891B2 - Malware judgment method, malware judgment device and malware judgment program - Google Patents

Description

The present invention relates to a malware determination method, a malware determination device, and a malware determination program that perform malware determination using machine learning.

As a method of judging whether or not an executable file contains malware in a Windows (registered trademark) or UNIX (registered trademark) environment, mainly dynamic judgment that judges by executing an executable file and dynamic judgment that judges by executing an executable file and without executing the executable file are performed. There is a static judgment to judge. When the determination speed is important, static determination is used.

Patent Document 1 cites hash value matching determination and pattern matching determination (signature scan) as typical methods for static determination. In the hash value match determination, if the hash values of MD5, SHA1, SHA256, etc. of known malware are stored in advance and the hash value of the inspected file matches the stored hash value, the inspected file contains malware. It is said that. In pattern matching judgment, a specific character string or bytecode contained in known malware is stored in advance, and if the stored character string or bytecode is included in the file to be inspected, the file to be inspected is malware. It is said to be included.

Further, Patent Document 1 also has the following description. That is, since it is difficult to detect variants and new types of malware that are modified existing malware by hash value matching judgment and pattern matching judgment, heuristic judgment has been proposed as a method for judging variants and new types of malware. There is. In the heuristic judgment, malware-likeness is defined based on past experience, and whether or not it is malware is judged according to the definition.

Further, Patent Document 1 also has the following description. That is, in order to reduce false positives (false positives), there is known a method of referring to a white list (a list in which highly reliable programs and codes are listed). Specifically, a white list is created in advance, programs and codes listed in the white list are judged to be goodware (files that are not malware), and programs and codes not listed in the white list are classified as malware, etc. It is considered and execution is prohibited. However, it is said that efficiency is reduced because a storage area for storing a white list and a processing time for collation are required. In addition, a false judgment means that goodware is mistakenly judged as malware, and is sometimes called a false positive.

Patent Document 1 describes false detection goodware data from existing malware data and teacher data including existing goodware data in order to efficiently execute malware judgment with few false judgments in malware judgment using machine learning. A method for deleting existing malware data whose feature vector and feature vector are similar is described. Further, the scrutiny teacher data in which the false positive goodware data is inserted into the teacher data as the existing goodware data is created. Then, the classifier learns the feature vector of the scrutiny teacher data and calculates a score indicating the malware-likeness of the test target file from the feature vector of the test target file.

Japanese Unexamined Patent Publication No. 2016-206950

According to the technique described in Patent Document 1, a malware determination method that reduces the possibility that goodware is erroneously determined as malware is realized without using a white list.

However, the technique described in Patent Document 1 does not consider False Negative (missing detection), which does not determine (detect) malware as malware.

Attacks such as targeted attacks and ransomware are becoming more diverse and serious. Under such circumstances, in addition to improving operability by reducing False Positives, it is also important to improve safety by reducing False Negatives.

An object of the present invention is to perform malware determination by machine learning with less detection omission (False Negative) while preventing a decrease in efficiency and an increase in false detection (False Positive).

The malware determination method according to the present invention is a malware determination method that uses machine learning to determine malware, and is considered to be goodware from unlearned data, but has a score of a predetermined first threshold value or higher. And the extracted data and the trained data in order to extract the data that is considered to be malware but the score is less than the predetermined second threshold value and update the machine learning model. It is characterized in that it learns about the target data, calculates a score indicating the malware-likeness of the judgment target data, and determines whether or not the judgment target data is malware based on the calculated score.

The malware determination device according to the present invention is a malware determination device that determines malware by using machine learning, and is considered to be goodware from unlearned data, but has a score of a predetermined first threshold value or higher. An extraction means for extracting data that is considered to be malware and data whose score is less than a predetermined second threshold value, and a classification means for calculating a score indicating the malware-likeness of the judgment target data. , An update means for updating the classification means by learning the data extracted by the extraction means and the learned data, and a judgment means for determining whether the judgment target data is malware based on the score calculated by the classification means. It is characterized by having.

The malware determination program according to the present invention is a malware determination program for performing malware determination using machine learning, and is regarded as goodware from unlearned data on a computer, but a score is predetermined. To update the machine learning model and the process of extracting the data that is equal to or higher than the first threshold value and the data that is considered to be malware but whose score is less than the predetermined second threshold value. In addition, the process of learning the extracted data and the learned data, the process of calculating the score indicating the malware-likeness of the judgment target data, and the judgment of whether the judgment target data is malware based on the calculated score. It is characterized in that the processing to be performed is executed.

According to the present invention, it is possible to perform malware determination by machine learning with few omissions of detection while preventing a decrease in efficiency and an increase in false detection.

It is a block diagram which shows one Embodiment of a malware determination apparatus. It is a flowchart which shows an example of the processing of the learning phase of a malware determination apparatus. It is explanatory drawing which shows the data stored in the unlearned database and the trained database when a learning phase is started. It is a flowchart which shows an example of the process of the determination phase of the malware determination apparatus. It is a flowchart which shows the classification information input process. It is a block diagram which shows the main part of a malware determination device. It is a block diagram which shows the main part of the malware determination apparatus of another aspect.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of a malware determination device according to the present invention. Note that FIG. 1 also shows databases (learned database and unlearned database) and the like. Further, in FIG. 1, for ease of explanation, the target file 110, the determination result 120, and the classification information 130 are also shown, but these are the data input to the malware determination device 1 or the malware determination device 1. It is the data output from.

The malware determination device 1 shown in FIG. 1 includes a learning data extraction unit 10, a classifier update unit 20, a database update unit 30, a classifier 40, a determination unit 50, and a classification information input unit 60.

The learning data extraction unit 10 uses the unlearned data stored in the unlearned database 100 to output data for which it is difficult to determine whether or not the malware is malware by machine learning. The unlearned data stored in the database 100 is, for example, the unlearned data body (for example, an executable file such as a code or a program) and information indicating whether the executable file is malware or goodware (hereinafter, referred to as “goodware”). It is composed of a pair of classification information) and a score indicating the malware-likeness of the executable file.

Specifically, the learning data extraction unit 10 is among the unlearned data stored in the unlearned database 100, which is considered to be goodware but whose score is equal to or higher than a predetermined first threshold value. (Data A) and data (data B) that is considered to be malware but whose score is less than the predetermined second threshold value is determined as to whether or not it is malware by machine learning. Is extracted as difficult data.

The first threshold value (threshold value for extracting data A) and the second threshold value (threshold value for extracting data B) may be different or the same. If the first threshold value is raised or the second threshold value is lowered, the number of data to be extracted decreases. When the first threshold value is lowered or the second threshold value is raised, the number of data to be extracted increases. It can be said that the former is an adjustment that emphasizes efficiency and the latter is an adjustment that emphasizes accuracy.

It should be noted that other criteria may be used as criteria for data for which it is difficult to determine whether or not the malware is malware by machine learning.

The classifier update unit 20 includes all the trained data stored in the trained database 101, and data (data A and data B) for which it is difficult to determine whether or not the data is machine learning malware extracted by the training data extraction unit 10. And update the classifier 40.

Specifically, the classifier update unit 20 includes all the trained data stored in the trained database 101, the unlearned data main body in the data A and the data B extracted by the training data extraction unit 10, and the untrained data body in the data B. The classification information of all the learned data stored in the learned database 101 and the classification information of the unlearned data main body in the data A and the data B extracted by the learning data extraction unit 10 are received, and machine learning is performed on them.

The database update unit 30 adds the data extracted by the learning data extraction unit 20 to the learned database 101 after the classifier update unit 20 updates the classifier 40. After that, the unlearned database 100 is emptied. Further, after the determination unit 50 outputs the determination result 120, the database update unit 30 adds the target file 110 and the score included in the determination result 120 to the unlearned database 100 as a pair. At this time, the classification information is added to the unlearned database 100 in an empty state. The target file 110 is an executable file whose classification as malware or goodware is unknown.

The classifier 40 calculates a score indicating the malware-likeness of the target file 110.

The determination unit 50 determines whether or not the target file 110 is malware based on the score, and outputs the determination result 120. As an example, the determination unit 50 uses a threshold value determination in which if the score is equal to or higher than a predetermined threshold value, it is determined to be malware, and if it is less than the threshold value, it is regarded as goodware. The determination result 120 includes the classification and score of malware or goodware.

When threshold value judgment is used as the judgment method, increasing the threshold value lowers the judgment rate (the rate at which malware is judged to be the case), but also lowers the erroneous judgment rate. Decreasing the threshold increases both the judgment rate and the erroneous judgment rate. However, the threshold value is not proportional to the judgment rate and the false judgment rate, and follows the ROC curve (Receiver Operating Characteristic curve). That is, it is known that in a certain threshold range, the erroneous determination rate does not increase so much even if the threshold value is increased, and the determination rate increases significantly. Therefore, it is preferable to appropriately adjust the threshold value in advance so that a relatively high determination rate can be obtained while keeping the erroneous determination rate within the permissible range.

As an example, a method is adopted in which a test target file prepared in advance is held out and verified, and a threshold value is determined from the obtained score so that the false positive rate falls within the permissible range.

The classification information input unit 60 determines the data to be updated of the classification information among the unlearned data stored in the unlearned database 100 based on the classification information 130, and updates the classification information of the data. The classification information 130 includes a set of an ID for determining which of the unlearned data is to be updated and classification information indicating whether the data is malware or goodware.

The learning data extraction unit 10, the classifier update unit 20, the database update unit 30, the classifier 40, the determination unit 50, and the classification information input unit 60 (excluding the hardware part for realizing the communication function) are programs. This can be achieved by the CPU (Central Processing Unit) executing processing based on the program stored in the storage unit. However, they may be implemented in hardware (individual circuits or LSI (Large Scale Integration)).

Next, the operation of the malware determination device 1 will be described.

FIG. 2 is a flowchart showing an example of processing in the learning phase of the malware determination device 1. FIG. 3 is an explanatory diagram showing data stored in the unlearned database 100 and the learned database 101 when the learning phase is started.

As shown in FIG. 3, in the unlearned database 100, unlearned data (a set of unlearned data body such as code and program, classification information, and a score indicating malware-likeness of an executable file) is stored together with an ID. There is.

As shown in FIG. 2, the learning data extraction unit 10 classifies the unlearned data stored in the unlearned database 100 into goodware having a score indicating malware-likeness equal to or higher than a predetermined threshold value. Data (data A) and data (data B) that are classified as malware whose score indicating malware-likeness is less than a predetermined threshold are extracted as data for which it is difficult to determine whether or not the data is malware by machine learning. (Step S101).

The classifier update unit 20 learns the data A and B extracted in the process of step S101 and all the learned data stored in the learned database 101, and updates the classifier 40 (step S102). ).

The database update unit 30 adds the data A and B extracted in the process of step S101 to the learned database 101 and emptys the unlearned database 100 (step S103).

FIG. 4 is a flowchart showing an example of processing in the determination phase of the malware determination device 1.

As shown in FIG. 4, the classifier 40 calculates a score indicating the malware-likeness of the target file 110 (step S111). The determination unit 50 determines whether or not the target file 110 is malware based on the calculated score (step S112). As described above, the determination unit 50 determines, for example, malware if the calculated score is equal to or higher than a predetermined threshold value, and determines that the score is goodware if it is less than the threshold value. Then, the determination unit 50 outputs the determination result 120. The determination result 120 includes the classification information of malware or goodware and the score.

The database update unit 30 adds the target file 110 and the score included in the determination result 120 to the unlearned database 100 as a set (step S113). However, the classification information is added to the unlearned database 100 in an empty state.

FIG. 5 is a flowchart showing the classification information input process. As shown in FIG. 5, the classification information input unit 60 refers to the ID included in the classification information 130 and determines whether the data corresponding to the ID in the unlearned database 100 is malware or goodware. The classification information is updated with the classification information of whether it is malware or goodware included in the classification information 130 (step S121).

The classification information 130 is, for example, information input from the outside of the malware determination device 1 together with the target file 110.

As described above, in the present embodiment, the malware determination device 1 is an execution file (unlearned data main body) that has not yet been trained to the classifier 40, and whether the execution file is malware or not by goodware. The learning data extraction unit 10 and the learning data extraction unit 10 extract data for which it is difficult to determine whether or not the data is malware by machine learning from unlearned data including classification information indicating the existence and a score indicating the malware-likeness of the execution file. A classifier update unit 20 that learns data that is difficult to determine whether or not it is malware by extracted machine learning and learned data to update the classifier 40, and an unlearned database 100 that stores unlearned data. And the database update unit 30 that updates the contents of the learned database 101 in which the learned data is stored, and the classifier 40 that calculates the score indicating the malware-likeness of the target file 110 to be determined whether or not it is malware. , A determination unit 50 that determines whether or not the target file 110 is malware based on the calculated score, and a classification information input unit 60 that updates the classification information of unlearned data whose IDs in the unlearned database 100 match. And have.

Further, in the present embodiment, the malware determination method is an execution file (unlearned data main body) that is executed by the malware determination device 1 and has not yet been learned by the classifier 40, and its execution. Learning data extraction process that extracts data that is difficult to determine whether it is malware by machine learning from unlearned data including classification information indicating whether the file is malware or goodware and a score indicating the malware-likeness of the execution file. The classifier update process for updating the classifier 40 by learning the learned data and the data for which it is difficult to determine whether or not the data is malware by machine learning extracted in the training data extraction process, and the unlearned data The database update process for updating the contents of the stored unlearned database 100 and the learned data for storing the learned data 101, and the score indicating the malware-likeness of the target file from the target file to be determined whether or not it is malware. The classification process to be calculated, the judgment process to determine whether the target file is malware based on the calculated score, and the classification information to update the classification information of the unlearned data whose IDs in the unlearned database 100 match. It has an input process.

Based on such a configuration, learning is performed not on all the unlearned data, but only on the data that the classifier 40 could not clearly classify. That is, the data that the classifier 40 has already properly learned is excluded from the learning target. Therefore, the time required for learning is shortened. In addition, it is possible to perform malware judgment by machine learning with less detection omission (False Negative) while preventing the increase of false detection (False Positive).

In the prior art, a classifier that learns a large amount of data has been created in order to improve the accuracy of malware determination. In order to maintain the judgment accuracy for variants of malware and new types of malware, it is necessary to continue learning to keep the classifier up-to-date, and the time required for data learning increases the number of data. Since it increases accordingly, there is a problem that it may hinder long-term operation. Further, in machine learning, it is difficult for humans to understand what scale the classifier uses to calculate the score, so it is difficult to reduce the learning data while maintaining the judgment accuracy.

The elements, the unlearned database 100, and the learned database 101 in the malware determination device 1 shown in FIG. 1 are assigned roles by paying attention to their functions. , Not limited to the example shown in FIG.

FIG. 6 is a block diagram showing a main part of the malware determination device according to the present invention. The malware determination device 80 shown in FIG. 6 includes an extraction means 81 (in the embodiment, realized by the learning data extraction unit 10) for extracting data from unlearned data for which it is difficult to determine whether or not the malware is malware by machine learning. , Learning is performed on the classification means 82 (in the embodiment, realized by the classifier 40) for calculating the score indicating the malware-likeness of the judgment target data, and the data extracted by the extraction means 81 and the learned data. The update means 83 that updates the classification means 82 (in the embodiment, it is realized by the classifier update unit 20) and the determination means 84 that determines whether the determination target data is malware based on the score calculated by the classification means. (In the embodiment, it is realized by the determination unit 50.).

The malware determination method executed by the malware determination device 80 extracts data that is difficult to determine whether or not it is malware by machine learning from unlearned data, and updates the machine learning model (for example, classifier 40). The extracted data and the learned data are learned as a target, a score indicating the malware-likeness of the judgment target data is calculated, and it is determined whether or not the judgment target data is malware based on the calculated score.

FIG. 7 is a block diagram showing a main part of the malware determination device of another aspect according to the present invention. The malware determination device 80 shown in FIG. 7 further includes a classification information input means 85 for inputting information indicating whether the unlearned data is considered as malware or goodware.

1 Malware judgment device 10 Learning data extraction unit 20 Classification device update unit 30 Database update unit 40 Classification device 50 Judgment unit 60 Classification information input unit 80 Malfunction judgment device 81 Extraction means 82 Classification means 83 Update means 84 Judgment means 85 Classification information input means 100 Unlearned database 101 Learned database 110 Target file 120 Judgment result 130 Classification information

Claims (5)

It is a malware judgment method that uses machine learning to judge malware.
From unlearned data, data that is considered to be goodware but has a score above the predetermined first threshold, and data that is considered to be malware but has a score below the preset second threshold. Extract some data and
In order to update machine learning, the extracted data and the trained data are trained and trained.
Calculate the score indicating the malware-likeness of the judgment target data,
A malware judgment method characterized in that it judges whether or not the judgment target data is malware based on the calculated score. Malware determination method according to claim 1, wherein the unlearned data to input information indicating whether being as either good ware is malware. It is a malware judgment device that uses machine learning to judge malware.
From unlearned data, data that is considered to be goodware but has a score above the predetermined first threshold, and data that is considered to be malware but has a score below the preset second threshold. An extraction method that extracts certain data,
A classification method that calculates a score that indicates the malware-likeness of the data to be judged, and
An update means for updating the classification means by learning the data extracted by the extraction means and the learned data, and an update means.
A malware determination device including a determination unit for determining whether or not the determination target data is malware based on the score calculated by the classification means. The malware determination device according to claim 3 , further comprising a classification information input means for inputting information indicating whether unlearned data is regarded as malware or goodware. A malware judgment program for making malware judgments that uses machine learning to make malware judgments.
On the computer
From unlearned data, data that is considered to be goodware but has a score above the predetermined first threshold, and data that is considered to be malware but has a score below the preset second threshold. The process of extracting certain data and
In order to update machine learning, a process of learning the extracted data and the learned data as a target, and
The process of calculating the score indicating the malware-likeness of the judgment target data, and
A malware judgment program that executes a process to judge whether the judgment target data is malware based on the calculated score.

Images