JP2018160172A - Malware determining method, malware determining apparatus, malware determining program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To carry out malware determination with machine learning having less detection omission, with preventing efficiency from being lowered and preventing error detection from being increased.SOLUTION: A malware determining apparatus 80 has extracting means 81 for extracting, from unlearned data, data which is difficult to be determined by machine learning whether or not it is malware, classifying means 82 for calculating a score indicating a degree of malware as to the data to be determined, updating means 83 for subjecting the data extracted by the extracting means 81 and the learned data to learning to update the classifying means 82, and determining means 84 for determining, on the basis of the score calculated by the classifying means, whether or not the data to be determined is malware.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a malware determination method, a malware determination device, and a malware determination program for performing malware determination using machine learning.

  As a method to determine whether or not an executable file in a Windows (registered trademark) or UNIX (registered trademark) environment contains malware, dynamic determination mainly by executing the executable file and without executing the executable file There is static judgment to judge. Static determination is used when the determination speed is important.

  Japanese Patent Application Laid-Open No. 2004-151561 lists hash value match determination and pattern match determination (signature scan) as typical static determination methods. In the hash value match determination, hash values of known malware such as MD5, SHA1, and SHA256 are stored in advance, and if the hash value of the inspection target file matches the stored hash value, the inspection target file includes malware. It is said. In pattern match judgment, when a specific character string or byte code included in known malware is stored in advance and the character string or byte code stored in the inspection target file is included, the inspection target file It is supposed to include.

  Patent Document 1 also includes the following description. In other words, because it is difficult to detect variant malware and new malware modified from existing malware in hash value match determination and pattern match determination, heuristic determination has been proposed as a method for determining variant malware and new malware. Yes. In the heuristic determination, the likelihood of malware is defined based on past experience, and it is determined whether or not it is malware according to the definition.

  Further, Patent Document 1 also has the following description. That is, a method of referring to a white list (a list in which programs and codes considered to have high reliability) are known in order to reduce erroneous determination (false detection). Specifically, a whitelist is created in advance, the program or code on the whitelist is determined to be goodware (non-malware file), and the program or code that is not on the whitelist It is considered and execution is prohibited. However, since the storage area for storing the white list and the processing time for collation are required, it is said that the efficiency is lowered. In addition, misjudgment is misjudging goodware as malware, and it may be called False Positive.

  In Patent Document 1, in order to efficiently execute malware determination with few erroneous determinations in malware determination using machine learning, erroneous detection goodware data is extracted from teacher data including existing malware data and existing goodware data. A method for deleting existing malware data whose feature vector is similar to the feature vector is described. Further, scrutinizing teacher data in which erroneously detected goodware data is inserted into the teacher data as existing goodware data is created. Then, the classifier learns the feature vector of the scrutinizing teacher data, and calculates a score indicating the malware likeness of the inspection target file from the feature vector of the inspection target file.

JP, 2006-206950, A

  According to the technique described in Patent Document 1, a malware determination method is realized in which the possibility that goodware is erroneously determined as malware is reduced without using a white list.

  However, the technique described in Patent Document 1 does not consider False Negative (detection omission) that does not determine (detect) malware as malware.

  Attacks such as targeted attacks and ransomware are diversifying and becoming more serious. Under such circumstances, in addition to improving operability by reducing false positives, it is also important to improve safety by reducing false negatives.

  It is an object of the present invention to perform malware determination by machine learning with less detection failure (False Negative) while preventing increase in efficiency and false detection (False Positive).

  The malware determination method according to the present invention is a malware determination method that performs machine determination using machine learning, and extracts data that is difficult to determine whether or not it is malware by machine learning from unlearned data. In order to update, the extracted data and learned data are learned as targets, a score indicating the malware likeness of the determination target data is calculated, and it is determined whether the determination target data is malware based on the calculated score It is characterized by that.

  The malware determination apparatus according to the present invention is a malware determination apparatus that performs machine determination using machine learning, and extracts extraction means for extracting data that is difficult to determine whether it is malware by machine learning from unlearned data, Based on the classification means for calculating the score indicating the malware likeness of the target data, the update means for updating the classification means by learning the data extracted by the extraction means and the learned data, and the score calculated by the classification means And determining means for determining whether or not the determination target data is malware.

  The malware determination program according to the present invention is a malware determination program for performing malware determination using machine learning to determine malware, and it is difficult to determine whether or not the computer is malware by machine learning from unlearned data. A process for extracting data, a process for learning the extracted data and the learned data in order to update the machine learning model, and a process for calculating a score indicating the malware likeness of the determination target data. And determining whether the determination target data is malware based on the score.

  ADVANTAGE OF THE INVENTION According to this invention, the malware determination by machine learning with few detection omissions can be performed, without making decline in efficiency and false detection increase.

It is a block diagram which shows one Embodiment of a malware determination apparatus. It is a flowchart which shows an example of the process of the learning phase of a malware determination apparatus. It is explanatory drawing which shows the data stored in the unlearned database and the learned database when the learning phase is started. It is a flowchart which shows an example of the process of the determination phase of a malware determination apparatus. It is a flowchart which shows classification information input processing. It is a block diagram which shows the principal part of a malware determination apparatus. It is a block diagram which shows the principal part of the malware determination apparatus of another aspect.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing an embodiment of a malware determination device according to the present invention. FIG. 1 also shows a database (learned database and unlearned database) and the like. FIG. 1 also shows the target file 110, the determination result 120, and the classification information 130 for ease of explanation. These are the data input to the malware determination device 1 or the malware determination device 1. Is the data output from.

  The malware determination apparatus 1 illustrated in FIG. 1 includes a learning data extraction unit 10, a classifier update unit 20, a database update unit 30, a classifier 40, a determination unit 50, and a classification information input unit 60.

  The learning data extraction unit 10 uses the unlearned data stored in the unlearned database 100 and outputs data that is difficult to determine whether it is malware by machine learning. The unlearned data stored in the database 100 includes, for example, an unlearned data main body (for example, an executable file such as a code or a program) and information indicating whether the executable file is malware or goodware (hereinafter referred to as “goodware”). It is composed of a set of classification information) and a score indicating the malware likeness of the executable file.

  Specifically, the learning data extraction unit 10 is data that is goodware among the unlearned data stored in the unlearned database 100 but has a score equal to or higher than a predetermined first threshold value. (It is assumed that the data is A.) and whether the data is assumed to be malware but the score is less than a predetermined second threshold value (referred to as data B) is determined by machine learning as to whether it is malware. Is extracted as difficult data.

  The first threshold value (threshold value for extracting data A) and the second threshold value (threshold value for extracting data B) may be different or the same. If the first threshold value is increased or the second threshold value is decreased, the number of extracted data decreases. If the first threshold value is lowered or the second threshold value is raised, the number of extracted data increases. The former can be said to be an efficiency-oriented adjustment, and the latter is an accuracy-oriented adjustment.

  In addition, you may use another reference | standard as a reference | standard of the data with which it is difficult to determine whether it is malware by machine learning.

  The classifier update unit 20 has all the learned data stored in the learned database 101 and data that is difficult to determine whether it is malware by machine learning extracted by the learning data extraction unit 10 (data A and data B). Is learned and the classifier 40 is updated.

  Specifically, the classifier update unit 20 includes all the learned data stored in the learned database 101, the unlearned data main body in the data A and data B extracted by the learning data extraction unit 10, and The classification information of all the learned data stored in the learned database 101 and the classification information of the unlearned data main body in the data A and the data B extracted by the learning data extraction unit 10 are received, and machine learning is performed on them.

  The database update unit 30 adds the data extracted by the learning data extraction unit 20 to the learned database 101 after the classifier update unit 20 updates the classifier 40. Thereafter, the unlearned database 100 is emptied. Further, after the determination unit 50 outputs the determination result 120, the database update unit 30 adds the target file 110 and the score included in the determination result 120 to the unlearned database 100 as a set. At this time, the classification information is added to the unlearned database 100 in an empty state. Note that the target file 110 is an execution file whose classification as malware or goodware is unknown.

  The classifier 40 calculates a score indicating the malware likeness of the target file 110.

  The determination unit 50 determines whether the target file 110 is malware based on the score, and outputs a determination result 120. For example, the determination unit 50 determines that the malware is malware if the score is equal to or higher than a predetermined threshold value, and uses threshold determination that determines goodware if the score is less than the threshold value. The determination result 120 includes malware or goodware classification and score.

  When threshold determination is used as the determination method, if the threshold is increased, the determination rate (the rate at which malware is determined to be) decreases, but the false determination rate also decreases. Decreasing the threshold increases both the determination rate and the erroneous determination rate. However, the threshold is not proportional to the decision rate and false decision rate, and follows the ROC curve (Receiver Operating Characteristic curve). In other words, it is known that the misjudgment rate does not increase so much even if the threshold value is increased within a certain threshold range, and the judgment rate is very high. Therefore, it is preferable to appropriately adjust the threshold in advance so that a relatively high determination rate can be obtained while keeping the erroneous determination rate within the allowable range.

  As an example, a method is adopted in which a test target file prepared in advance is subjected to holdout verification, and a threshold value is determined from an obtained score so that an erroneous determination rate is within an allowable range.

  Based on the classification information 130, the classification information input unit 60 determines data to be updated for classification information among the unlearned data stored in the unlearned database 100, and updates the classification information of the data. The classification information 130 is composed of a set of an ID for determining which data to be updated among unlearned data and classification information indicating whether it is malware or goodware.

  Note that the learning data extraction unit 10, the classifier update unit 20, the database update unit 30, the classifier 40, the determination unit 50, and the classification information input unit 60 (excluding the hardware part for realizing the communication function) are programs. This can be realized by a CPU (Central Processing Unit) executing processing based on a program stored in the storage unit. However, they may be realized by hardware (individual circuit or LSI (Large Scale Integration)).

  Next, the operation of the malware determination device 1 will be described.

  FIG. 2 is a flowchart illustrating an example of the learning phase process of the malware determination apparatus 1. FIG. 3 is an explanatory diagram showing data stored in the unlearned database 100 and the learned database 101 when the learning phase is started.

  As shown in FIG. 3, the unlearned database 100 stores unlearned data (a set of unlearned data main body such as codes and programs, classification information, and a score indicating the malware likeness of an executable file) together with an ID. Yes.

  As shown in FIG. 2, the learning data extraction unit 10 is classified into goodware having a score indicating the likelihood of malware out of the unlearned data stored in the unlearned database 100 above a predetermined threshold. Data (data A) and data (data B) classified as malware with a malware score that is less than a predetermined threshold are extracted as data that is difficult to determine whether it is malware by machine learning (Step S101).

  The classifier update unit 20 learns the data A and B extracted in the process of step S101 and all the learned data stored in the learned database 101, and updates the classifier 40 (step S102). ).

  The database update unit 30 adds the data A and B extracted in the process of step S101 to the learned database 101, and empties the unlearned database 100 (step S103).

  FIG. 4 is a flowchart illustrating an example of the determination phase process of the malware determination apparatus 1.

  As shown in FIG. 4, the classifier 40 calculates a score indicating the malware likeness of the target file 110 (step S111). The determination unit 50 determines whether the target file 110 is malware based on the calculated score (step S112). As described above, for example, the determination unit 50 determines malware if the calculated score is equal to or greater than a predetermined threshold, and determines goodware if the calculated score is less than the threshold. Then, the determination unit 50 outputs a determination result 120. The determination result 120 includes malware or goodware classification information and a score.

  The database update unit 30 adds the target file 110 and the score included in the determination result 120 to the unlearned database 100 as a set (step S113). However, the classification information is added to the unlearned database 100 in an empty state.

  FIG. 5 is a flowchart showing classification information input processing. As shown in FIG. 5, the classification information input unit 60 refers to the ID included in the classification information 130 to determine whether the data corresponding to the ID in the unlearned database 100 is malware or goodware. The classification information is updated with the classification information indicating whether it is malware or goodware included in the classification information 130 (step S121).

  The classification information 130 is information input together with the target file 110 from the outside of the malware determination device 1, for example.

  As described above, in this embodiment, the malware determination device 1 is an executable file (unlearned data body) that has not yet been learned by the classifier 40, and the executable file is malware or goodware. A learning data extraction unit 10 that extracts data that is difficult to determine whether it is malware by machine learning from unlearned data that includes classification information that indicates whether or not there is a score that indicates the malware likeness of the executable file, and a learning data extraction unit 10 The classifier update unit 20 that updates the classifier 40 by learning the extracted data and the data that is difficult to determine whether it is malware by machine learning, and the unlearned database 100 in which the unlearned data is stored And a database update unit 30 for updating the content of the learned database 101 in which the learned data is stored. A classifier 40 that calculates a score indicating the malware likeness of the target file 110 that is a determination target of whether or not it is malware, and a determination unit that determines whether or not the target file 110 is malware based on the calculated score 50, and a classification information input unit 60 for updating the classification information of the unlearned data whose ID in the unlearned database 100 matches.

  Moreover, in this embodiment, the malware determination method is a malware determination method executed by the malware determination apparatus 1, and is an executable file (unlearned data body) that has not yet been learned to the classifier 40, and its execution A learning data extraction process for extracting data that is difficult to determine whether it is malware by machine learning from unlearned data including classification information indicating whether the file is malware or goodware and a score indicating the malware likeness of the executable file A classifier update step of updating the classifier 40 by performing learning on data that is difficult to determine whether it is malware or not by machine learning extracted in the learning data extraction step, and unlearned data The stored unlearned database 100 and the learned database 101 in which learned data is stored A database update step for updating the content, a classification step for calculating a score indicating the malware likeness of the target file from the target file to be determined whether it is malware, and whether the target file is malware based on the calculated score A determination step of determining whether or not, and a classification information input step of updating the classification information of the unlearned data whose ID in the unlearned database 100 matches.

  Based on such a configuration, learning is not performed on all unlearned data, but learning is performed only on data that the classifier 40 cannot clearly classify. That is, data that has already been properly learned by the classifier 40 is excluded from the learning target. Therefore, the time required for learning is shortened. In addition, it is possible to perform malware determination by machine learning with less false negatives while preventing false detections (False Positive) from increasing.

  In the prior art, in order to improve the accuracy of malware determination, a classifier that learns a lot of data has been created. In order to maintain the accuracy of detection of variants and new types of malware, it is necessary to continue learning to keep the classifier up-to-date, and the time required for data learning increases. Since it increases according to the problem, there is a problem that it may hinder long-term operation. Also, in machine learning, it is difficult for humans to understand what scale the classifier uses to calculate the score, so it is difficult to reduce learning data while maintaining determination accuracy.

  Note that each element, the unlearned database 100, and the learned database 101 in the malware determination apparatus 1 shown in FIG. 1 are assigned roles with a focus on functions. The embodiment is not limited to the example shown in FIG.

  FIG. 6 is a block diagram showing a main part of the malware determination device according to the present invention. The malware determination device 80 illustrated in FIG. 6 includes extraction means 81 (implemented by the learning data extraction unit 10 in the embodiment) that extracts data that is difficult to determine whether it is malware by machine learning from unlearned data. The classifying means 82 (which is realized by the classifier 40 in the embodiment) that calculates the malware-like score of the determination target data, and the data extracted by the extracting means 81 and the learned data are subjected to learning. Update means 83 for updating the classification means 82 (in the embodiment, realized by the classifier update unit 20), and determination means 84 for determining whether the determination target data is malware based on the score calculated by the classification means. (In the embodiment, it is realized by the determination unit 50).

  The malware determination method executed by the malware determination device 80 extracts data that is difficult to determine whether it is malware by machine learning from unlearned data, and updates a machine learning model (for example, the classifier 40). Learning is performed on the extracted data and learned data, a score indicating the malware likeness of the determination target data is calculated, and it is determined whether the determination target data is malware based on the calculated score.

  FIG. 7 is a block diagram showing a main part of a malware determination device according to another aspect of the present invention. The malware determination device 80 shown in FIG. 7 further includes classification information input means 85 that inputs information indicating whether the unlearned data is malware or goodware.

DESCRIPTION OF SYMBOLS 1 Malware determination apparatus 10 Learning data extraction part 20 Classifier update part 30 Database update part 40 Classifier 50 Determination part 60 Classification information input part 80 Malware determination apparatus 81 Extraction means 82 Classification means 83 Update means 84 Determination means 85 Classification information input means 100 unlearned database 101 learned database 110 target file 120 determination result 130 classification information

Claims (8)

A malware determination method for performing malware determination using machine learning,
From unlearned data, extract data that is difficult to determine whether it is malware by machine learning,
In order to update machine learning, learning is performed on the extracted data and learned data,
Calculate a score indicating the malware likeness of the data to be judged,
A malware determination method characterized by determining whether or not the determination target data is malware based on the calculated score. Data that is considered goodware but has a score equal to or higher than a predetermined first threshold value, and data that is considered malware but has a score lower than a predetermined second threshold value, The malware determination method according to claim 1, wherein the malware is extracted as data that is difficult to determine whether it is malware by machine learning. The malware determination method according to claim 2, wherein information indicating whether the unlearned data is malware or goodware is input. A malware determination device that performs machine determination using machine learning,
Extraction means for extracting data that is difficult to determine whether it is malware by machine learning from unlearned data;
Classification means for calculating a score indicating the malware likeness of the determination target data;
Updating means for performing learning on the data extracted by the extraction means and learned data and updating the classification means;
A malware determination apparatus comprising: determination means for determining whether or not the determination target data is malware based on the score calculated by the classification means. The extraction means is goodware, but the score is equal to or higher than a predetermined first threshold value, and the malware is considered to be malware but the score is lower than a predetermined second threshold value. The malware determination apparatus according to claim 4, wherein the data is extracted as data that is difficult to determine whether it is malware by machine learning.   6. The malware determination apparatus according to claim 5, further comprising classification information input means for inputting information indicating whether the unlearned data is malware or goodware. A malware determination program for performing malware determination using machine learning to determine malware,
On the computer,
A process of extracting data that is difficult to determine whether it is malware by machine learning from unlearned data,
A process of learning the extracted data and learned data to update machine learning;
A process of calculating a score indicating the malware likeness of the determination target data;
A malware determination program for executing a process of determining whether or not the determination target data is malware based on the calculated score. On the computer,
Data that is considered goodware but has a score equal to or higher than a predetermined first threshold value, and data that is considered malware but has a score lower than a predetermined second threshold value, The malware determination program according to claim 7 for extracting data as difficult to determine whether it is malware by machine learning.

Images