JP6039768B1 - ADJUSTMENT DEVICE, ADJUSTMENT METHOD, AND ADJUSTMENT PROGRAM - Google Patents

Abstract

[PROBLEMS] To efficiently adjust adjustment items in each accuracy improvement method when combining accuracy improvement methods for performing malware determination. In contrast to a malware determination device that performs preprocessing in the order of whitelist application, feature extraction, and dimension reduction, and determines whether a file is malware or not using a threshold value, the adjustment device has a score In the order of calculation processing, feature extraction and dimension reduction, white list application and determination processing, settings are applied from among the setting candidates to each processing, and malware determination accuracy is analyzed. At this time, for the process for which the optimal setting has already been determined, the determined optimal process is applied and analyzed thereafter. [Selection] Figure 1

Description

  The present invention relates to an adjustment device, an adjustment method, and an adjustment program for adjusting settings of a malware determination device.

  Anti-virus systems that determine whether executable files used on OS such as MS Windows (registered trademark), Apple OSX (registered trademark), Linux (registered trademark) and other Unix (registered trademark) are malware are known It has been. In the anti-virus system, two methods are used: dynamic determination in which an execution file is executed and determination is performed, and static determination in which determination is performed without executing the execution file. Judgment is used.

  Typical static determination methods include hash value match determination and pattern match determination (signature scan). The hash value match determination has a hash value of MD5, SHA1, SHA256, etc. of known malware in advance as a database, and is determined as malware if the hash value of the inspection target file matches the database. In addition, the pattern matching judgment has a specific character string or byte code included in known malware in advance as a database, and if the file to be inspected contains either a character string or byte code registered in the database, Judgment.

  However, even if the malware is modified a little, the hash value becomes a different value, and there is a possibility that a specific character string or the like used for pattern match determination is also changed. For this reason, it has been difficult for these methods to detect variants of new malware and variants of existing malware. Therefore, heuristic determination has been proposed as a method for determining sub-species and new types of malware. In this method, malware-likeness is defined based on previous experience, and a determination is made according to the definition.

  Several methods using machine learning techniques for heuristic determination have been proposed. For example, a method has been proposed in which a readable character string included in an executable file is learned in advance, and the likelihood of the malware is determined based on how many words frequently used in malware are included in the inspection file (for example, a patent) Reference 1).

  In machine learning, first, learning target data (teacher data) is converted into a set of several parameters, and then learning is performed using a machine learning algorithm. Each parameter is called a feature, and a set of parameters is called a feature vector. For example, in the example of Patent Document 1, a word name and the number of appearances are features, and the set is a feature vector. In addition, the number of features included in the set is referred to as a feature vector dimension. In the above example, the number of types of words is a feature vector dimension.

  In addition, heuristic determination using machine learning technology can also detect subtypes and new types of malware, while comparing the false positive rate (the rate at which non-malware files (goodware) are mistakenly determined as malware). There is a big tendency. Therefore, various accuracy improvement methods have been proposed in machine learning technology. For example, a malware determination method by machine learning using PE header information of an executable file has been proposed, and in this method, detection accuracy is improved by using an appropriate dimension compression and a machine learning algorithm (for example, non-deletion). (See Patent Document 1).

  The accuracy can also be improved by case selection, which is a technique for examining teacher data, such as removing data that seems to deteriorate the classification accuracy from the teacher data (see Non-Patent Document 2, for example). In Non-Patent Document 2, in image classification using a support vector machine (SVM), there is a case selection method that uses internal parameters αi of SVM to extract ambiguous image data that is difficult to classify and removes the extracted data from teacher data. It is used.

  In addition, a method for improving accuracy by adjusting parameters of the used machine learning algorithm is generally implemented (see, for example, Non-Patent Document 3). Further, as a general technique for correcting erroneous detection at the time of determination, a false detection correction technique using a white list is often used.

JP 2012-27710 A

Shafiq, et al., “PE-Miner: Mining Structural Information to Detect Malicious Executables in Realtime”, RAID '09, 2009. Takatori et al., “Proposal of case selection method based on internal parameters of support vector machine and application to video boundary detection problem”, Japanese Society for Artificial Intelligence (22nd), 2008. Hsu, et al., “A Practical Guide to Support Vector Classification”, http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf

  In malware determination using machine learning, if various accuracy improvement methods are combined, the determination accuracy can be greatly improved as compared to using only individual methods. Here, each accuracy improvement method has various adjustment items. However, in the prior art, there is a problem that adjustment items of the accuracy improvement method cannot be adjusted efficiently.

  In other words, in order to obtain the optimum setting of adjustment items for each method by combining various accuracy improvement methods, it is necessary to try each combination of accuracy improvement method and each adjustment item comprehensively and comprehensively. There is no efficient adjustment.

  As an example of adjustment that is exhaustively tried, consider adjusting parameters of a machine learning algorithm. As an example of the parameter adjustment method of the machine learning algorithm, there is a grid search method described in Non-Patent Document 2. In the grid search method, several candidate values of algorithm parameters are listed, each of the candidate values is applied one by one, actually subjected to learning / determination, and the candidate value showing the best accuracy is adopted. Is the method. When there are a plurality of parameters, candidate values are listed for each parameter, and the candidate values for each parameter are combined and tried. For example, when there are two parameters and ten candidate values are prepared for each, the number of trials is 10 × 10 = 100.

  It is also possible to adjust the technique of feature selection that reduces the dimension by removing some features and improves the accuracy of malware determination. For example, as a feature selection adjustment method, there is a wrapper method (see, for example, Japanese Patent Application No. 2014-120428). The wrapper method is a method that employs the feature selection setting that shows the best accuracy by actually learning and determining the feature selection setting candidates.

  Here, there is a variable increasing method as a method for creating feature selection setting candidates. In the variable increase method, if the number of features (or attributes) is p, the trial is 2p-1 times at the minimum and p × (p-1) / 2 times at the maximum. When finding the best accuracy from algorithm parameters and feature selection, it is necessary to exhaustively try the brute force combination of both candidate values.

  If the number of algorithm parameters to be adjusted is two, the number of candidate values is 10, and the number of features to be selected is 10, 10 × 10 × 19 to 10 × 10 × 45, that is, 1900 to 4500. Times are required. As described above, every time the method / parameter to be adjusted increases, the number of trials increases exponentially and the adjustment cannot be performed efficiently.

  Another method for improving accuracy is to use known false positive goodware to remove similar malware data from the teacher data, and make the false positive goodware data into teacher data so that it can be more easily learned. A case selection method that realizes a particularly low false detection rate by appropriately inserting can be considered (for example, see Japanese Patent Application No. 2015-087924). However, as with other methods, since adjustment of adjustment items in the case selection method has been exhaustively tried, there are cases where adjustment cannot be performed efficiently.

  The adjustment device of the present invention executes one or more pre-processes in a predetermined order, and then executes a score calculation process for calculating a score of a file by a classifier, and the file is malware based on the calculated score An adjustment unit that determines an optimal setting for executing each process in the malware determination apparatus that performs the determination process whether or not, and an instruction unit that selects each process of the malware determination apparatus in a predetermined order; Each time a process is selected by the instruction unit, the setting candidates for executing the selected process are sequentially applied, and an optimum setting is determined among processes other than the process selected by the instruction unit. A setting application unit that applies the optimal setting to the process that has been completed, and the setting applied by the setting application unit every time the setting application unit applies the setting. When each process of the malware determination device is executed, a verification unit that acquires a result corresponding to each of the setting candidates, and all of the setting candidates of the process selected by the verification unit Each time a result is acquired, a setting candidate that is determined to have the highest malware determination accuracy based on a result corresponding to each of the setting candidates is selected as the optimum setting for the selected process. And an analysis unit determined as:

  According to the adjustment method of the present invention, after executing one or more pre-processes in a predetermined order, a score calculation process for calculating a score of a file is executed by a classifier, and the file is malware based on the calculated score An adjustment method for determining an optimal setting for executing each process in a malware determination apparatus that performs a determination process on whether or not, and an instruction step of selecting each process of the malware determination apparatus in a predetermined order; Each time a process is selected by the instruction process, the setting candidates for executing the selected process are sequentially applied, and an optimal setting is determined among processes other than the process selected by the instruction process. Applied to the process that has already been applied, each time the setting is applied by the setting application step, and the setting application step applies the optimum setting. A verification step of obtaining results corresponding to each of the setting candidates when each processing of the malware determination device is executed according to the setting, and all of the setting candidates of the processing selected by the verification step Each time a corresponding result is obtained, the setting candidate determined to have the highest malware determination accuracy from the result corresponding to each of the setting candidates is selected as the optimum of the selected process. And an analysis step for determining as a proper setting.

  ADVANTAGE OF THE INVENTION According to this invention, when combining the accuracy improvement method for performing malware determination, the adjustment matter in each accuracy improvement method can be adjusted efficiently.

FIG. 1 is a diagram illustrating an example of a configuration of a malware determination system including an adjustment device and a malware determination device according to the first embodiment. FIG. 2 is a diagram illustrating an example of data in the setting storage unit of the adjustment device according to the first embodiment. Drawing 3 is a figure for explaining each adjustment process of the adjustment device concerning a 1st embodiment. FIG. 4 is a diagram for explaining the relationship between the score threshold, the detection rate, and the false detection rate. FIG. 5 is a diagram for explaining the ROC curve. FIG. 6 is a flowchart illustrating the process of the adjustment device according to the first embodiment. FIG. 7 is a flowchart showing processing of the malware determination device according to the first embodiment. FIG. 8 is a diagram illustrating an example of a configuration of a malware determination system including the adjustment device and the malware determination device according to the second embodiment. FIG. 9 is a diagram illustrating an example of a computer in which the adjustment apparatus is realized by executing a program.

  Hereinafter, embodiments of an adjustment device, an adjustment method, and an adjustment program according to the present application will be described in detail with reference to the drawings. Note that the adjustment device, the adjustment method, and the adjustment program according to the present application are not limited by this embodiment.

[First Embodiment]
In the following embodiments, the configuration, processing, and effects of a malware determination system including the adjustment device and the malware determination device according to the first embodiment will be described. After executing one or more pre-processes in a predetermined order, the adjustment apparatus 10 executes a score calculation process for calculating a score of the file by the classifier, and whether or not the file is malware based on the calculated score The optimal setting for executing each process in the malware determination apparatus 20 that performs the determination process is determined.

[Configuration of First Embodiment]
The configuration of the malware determination system including the adjustment device and the malware determination device according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of a configuration of a malware determination system including an adjustment device and a malware determination device according to the first embodiment. First, the data flow in the malware determination system 1 will be described. As shown in FIG. 1, first, an instruction from a user, original teacher data for machine learning, adjustment data for adjustment and verification, and original verification data are input to the adjustment device 10.

  The adjustment device 10 creates adjusted teacher data and adjusted settings based on these data, and outputs them to the malware determination device 20. And the malware determination apparatus 20 outputs the determination result of the determination object file input separately using each data acquired from the adjustment apparatus 10. FIG. Here, a part of the configuration and processing of the adjustment device 10 is determined by the configuration of the malware determination device 20 and the like. Therefore, first, the configuration of the malware determination device 20 will be described.

[Malware determination device]
As illustrated in FIG. 1, the malware determination device 20 includes a white list application unit 201, a feature extraction / dimension reduction unit 202, a classifier 203, a determination unit 204, and a setting storage unit 210. The malware determination device 20 first reads and initializes settings stored in the setting storage unit 210, and then learns teacher data composed of malware and an executable file (goodware) that is not malware. The malware determination device 20 determines whether the determination target file is malware or goodware after learning the teacher data. Note that the white list application unit 201, the feature extraction / dimension reduction unit 202, the classifier 203, and the determination unit 204 are used as appropriate for learning the teacher data and determining the determination target file.

  As described above, the malware determination device 20 performs the determination process after performing the learning process. First, the function of each part in the learning process will be described. The malware determination device 20 executes learning processing in the order of feature extraction and dimension reduction by the feature extraction / dimension reduction unit 202 and score calculation by the classifier 203. The teacher data to be learned is composed of existing malware and goodware executable files, and information indicating the classification of whether the executable file is malware or goodware.

  The feature extraction / dimension reduction unit 202 performs feature extraction from the teacher data. Furthermore, the feature extraction / dimension reduction unit 202 performs feature reduction or feature reduction on the extracted features as necessary to generate feature vectors. An example of a technique for weighting feature vectors by the feature extraction / dimension reduction unit 202 is, for example, tf-idf (Term Frequency-Inverse Document Frequency).

  Here, the feature selection is a method of selecting each feature so that the target accuracy is improved. As a typical technique for dimensional compression, for example, principal component analysis (PCA) is known in which correlated features are automatically combined into one feature. In the dimension reduction, only one of feature selection and dimension compression may be performed, or both may be performed.

  The classifier 203 performs machine learning using the feature vector and information indicating the classification of whether the execution file corresponding to the feature vector is malware or goodware. The classifier 203 can perform machine learning using algorithms such as logistic regression, SVM, perceptron, passive-aggressive, adaptive regularization of weight vectors (AROW), and naive Bayes.

  Next, the function of each unit in the determination process will be described. The malware determination device 20 applies the white list by the white list application unit 201, the feature extraction / dimension reduction by the feature extraction / dimension reduction unit 202, the score calculation by the classifier 203, and the determination unit 204 determines whether the malware is malware. Judgment processing is executed in this order. The determination target file that is the determination target is an executable file that is unknown whether it is malware or goodware.

  The white list application unit 201 determines whether or not the determination target file corresponds to the white list, and determines that the determination target file determined to be applicable is goodware. The white list may be a list of hashes using a hash algorithm such as MD5, SHA1, or SHA256, or may be a set of values (for example, feature vectors) necessary for calculating some defined similarity.

  In addition, the feature extraction / dimension reduction unit 202 generates a feature vector of a determination target file that does not correspond to the white list. The feature vector generation method is the same as in the learning process. Then, the classifier 203 outputs the malware-likeness of the determination target file from the feature vector as a numerical value called a score.

  The determination unit 204 determines whether the determination target file is malware based on the score, and outputs a determination result. The determination result may be only the classification of malware / goodware, or may be a classification added with a score. For example, the determination unit 204 may employ threshold determination to determine malware if the score exceeds a certain threshold, and goodware if the score does not exceed the threshold.

[Adjustments]
The learning process and the determination process are executed after reading and initializing settings stored in the setting storage unit 210. The settings stored in the setting storage unit 210 affect the processing of each unit that performs the learning process and the determination process of the malware determination apparatus 20 described so far.

  The adjustment device 10 creates an adjusted setting in which the adjustment items stored in the setting storage unit 210 have been adjusted, and outputs the adjusted setting to the malware determination device 20. The adjustment device 10 may also adjust the teacher data and output the adjusted teacher data to the malware determination device 20. In this case, the teacher data used in the learning process is adjusted teacher data.

  Here, the adjustment items will be described with specific examples. First, as examples of adjustment items related to the feature extraction / dimension reduction unit 202, weight setting at the time of feature extraction (for example, whether tf-idf is weighted), setting of a feature to be selected (feature selection setting), and dimension The algorithm used for compression and the parameters of the algorithm (for example, the number of compression dimensions) are mentioned.

  Examples of adjustment items related to the classifier 203 include algorithms to be used and algorithm adjustment parameters. Also, depending on the algorithm, the accuracy may be improved by learning the same teacher data repeatedly. When such an algorithm is used, the number of repeated learnings may be an adjustment item.

  Note that the white list itself used by the white list application unit 201 may be an adjustment item. Further, as an example of adjustment items related to the determination unit 204, a score threshold in the case where score threshold determination is performed can be given. In addition, the adjustment device 10 adjusts the teacher data by performing selection selection of the teacher data, change of the data arrangement order, and the like by case selection or the like, and outputs the adjusted teacher data.

[Adjustment device]
As illustrated in FIG. 1, the adjustment apparatus 10 includes an instruction unit 101, a teacher / verification data creation unit 102, a verification unit 103, an analysis unit 104, and a setting storage unit 110. The adjustment device 10 sequentially executes adjustment steps for adjustment items such as settings of the malware determination device 20 and teacher data to be given. Then, adjustment items and setting values are determined so that the index obtained in the adjustment process is further improved, and adjusted teacher data and adjusted settings are output to the malware determination device 20.

  The instruction unit 101 manages the adjustment process and gives an instruction to execute measurement / verification in accordance with the process. In addition, the instruction unit 101 determines the execution order of the adjustment process corresponding to the processing of each unit of the malware determination device 20 by selecting each processing of the malware determination device 20 in a predetermined order. For example, the instruction unit 101 may set the adjustment of the classifier 203 first, and the processing performed before the classifier 203 in the malware determination device 20 may be in the reverse order of the processing order.

  The teacher / verification data creation unit 102 creates teacher data and verification data to be given to the verification unit 103 using the original teacher data, the original verification data, and the adjustment data based on an instruction from the instruction unit 101.

  The verification unit 103 performs the same processing as the malware determination device 20 and performs verification. Thereby, the verification part 103 outputs a score as a determination result, for example. Then, the analysis unit 104 performs an analysis process corresponding to the adjustment process. Specifically, the analysis unit 104 calculates an index based on the learning or determination result of the verification unit 103, determines the optimum adjustment item and setting value based on the index, and sets and stores the determined adjustment item and setting value. Stored in the unit 110.

  The original teacher data and the original verification data are information bodies having the same configuration as the teacher data used in the malware determination device 20. Further, the adjustment data is goodware data that is easily detected erroneously, and is used, for example, in a case selection method (see, for example, Japanese Patent Application No. 2015-087924).

  Here, examples of adjustment items and setting values will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of data in the setting storage unit of the adjustment device according to the first embodiment. Number 1 in FIG. 2 is an adjustment item related to the classifier 203 of the malware determination device 20. 2 are adjustment items related to the feature extraction / dimension reduction unit 202 of the malware determination apparatus 20. 2 is an adjustment item related to the teacher data of the malware determination device 20. 2 is an adjustment item related to the determination unit 204 of the malware determination apparatus 20. 2 is an adjustment item related to the white list application unit 201 of the malware determination apparatus 20.

  Here, each process including the analysis process corresponding to each adjustment process and each adjustment process performed in the adjustment apparatus 10 will be described. Before executing each adjustment step, the instruction unit 101 instructs in advance the setting candidate range generated by the instruction unit 101 in each step, the verification method, the index calculated by the analysis unit 104, the target index value, and the like. As given. The adjustment process and the analysis process will be described with reference to FIG. Drawing 3 is a figure for explaining each adjustment process of the adjustment device concerning a 1st embodiment.

  In each adjustment step, the verification unit 103 sequentially applies setting candidates for executing the selected process, and the optimal setting has been determined among the processes other than the process selected by the instruction unit 101. Apply optimal settings for processing. Then, each time the setting is applied, the verification unit 103 acquires a result corresponding to each of the setting candidates when each process of the malware determination apparatus 20 is executed according to the applied setting.

  First, as a process in the previous stage of the adjustment process, the adjustment device 10 performs initialization of the setting storage unit 110 and data cleansing. Here, data cleansing will be described. In the malware static determination by machine learning, the tendency of the characteristics depends on the file type, that is, whether the execution file is a 32-bit execution application, a 64-bit execution application, or a Visual Basic application. Different. The accuracy is improved by learning and determining only a specific file type.

  Therefore, by performing data cleansing, only the designated file type is selected as data from the original data (original teacher data, original verification data). Also, there may be goodware in the original data that is incorrectly labeled as malware. Therefore, in data cleansing, malware very similar to goodware is removed from the original data. In the subsequent steps, unless otherwise specified, the original data indicates the cleansed data.

  Next, as shown in FIG. 3, the adjustment device 10 adjusts the classifier as the first adjustment step. The instructing unit 101 generates several setting group candidates such as the classifier algorithm, the algorithm parameters, the number of iteration learning, and the like, and temporarily sets one of them in the setting storage unit 110. Then, the teacher / verification data creation unit 102 generates teacher data and verification data using the original data.

  The verification unit 103 performs learning / determination using the teacher data and the verification data based on the settings temporarily set in the setting storage unit 110 and performs verification. When this verification is performed for all the set candidates, the analysis unit 104 calculates an index for each verification result, selects the setting set that was the best index, and sets the determined adjustment items as the adjustment items. Is stored in the setting storage unit 110.

  As an example of the verification method, there is holdout verification in which learning and determination are performed using the original teacher data as teacher data and the original verification data as verification data. An example of another verification method is K-division cross verification. In K-division cross-validation, first, original teacher data and original verification data are mixed to generate K pieces of data. Next, one piece of data is selected as verification data, and the remaining data is learned and determined as teacher data. If there is data that is not yet verified data, it is selected as data for verification, learning and determination are performed, and learning and determination are repeated until all data is selected as verification data once.

  Examples of the index include AUC and F value that are often used in machine learning verification. In addition, the detection rate when adjusted to be equal to or less than the specified false detection rate, or the false detection rate when adjusted to be equal to or higher than the specified detection rate (true negative rate) is used as an index. It may be used.

  In the malware determination, when the determination based on the score threshold is used, increasing the threshold decreases the detection rate but also decreases the false detection rate. Conversely, when the threshold value is decreased, the detection rate increases and the false detection rate also increases. The relationship between the threshold value, the detection rate, and the false detection rate is not a proportional relationship. As shown in FIG. 4, even if the threshold value is increased within a certain threshold range, the false detection rate does not increase so much, and the detection rate can be very high. Are known. FIG. 4 is a diagram for explaining the relationship between the score threshold, the detection rate, and the false detection rate. At this time, the relationship between the false detection rate and the detection rate follows an ROC curve as shown in FIG. FIG. 5 is a diagram for explaining the ROC curve.

  Therefore, by appropriately adjusting the threshold value in advance, it is possible to obtain a relatively high detection rate while keeping the false detection rate within a specified allowable range. Similarly, by adjusting the score threshold, it is possible to achieve a specified detection rate or higher. These score thresholds are determined by analyzing the score of the determination result in the analysis unit 104.

  Next, as illustrated in FIG. 3, the adjustment device 10 performs dimension reduction adjustment as the second adjustment step. At this time, data used for verification and classifier settings are determined in the first adjustment step and stored in the setting storage unit 110. Examples of setting items in feature selection, which is one method of dimension reduction, include File name, File size, and the like (see, for example, Japanese Patent Application No. 2014-120428). Another method is dimensional compression. A typical example of dimensional compression is principal component analysis.

  The setting item in the principal component analysis is, for example, the number of compression dimensions. The instructing unit 101 creates several candidates for feature extraction weights, feature selection settings, and number of compression dimensions, and temporarily sets one of them in the setting storage unit 110. Then, the teacher / verification data creation unit 102 generates teacher data and verification data using the original data.

  The verification unit 103 performs learning / determination using the teacher data and the verification data based on the settings temporarily set in the setting storage unit 110 and performs verification. When this verification is performed for all the set candidates, the analysis unit 104 calculates an index for each verification result, selects the setting set that was the best index, and sets the determined adjustment items as the adjustment items. Is stored in the setting storage unit 110.

  For the generation of feature selection setting candidates, techniques such as brute force, variable increase method, variable decrease method, stepwise method, and the like can be used (for example, see Japanese Patent Application No. 2014-120428). Further, the same verification method and index as those used in the other steps may be used, or different methods may be used depending on circumstances. For example, split intersection verification may be used in the previous process, and holdout verification may be used in this process in order to shorten the processing time.

  Next, as shown in FIG. 3, the adjustment device 10 adjusts teacher data as a third adjustment step. At this time, the data used for verification, the classifier, and the feature extraction / dimension reduction settings are determined in the first and second adjustment steps and stored in the setting storage unit 110. In this case, the teacher / verification data creation unit 102 sequentially applies the data candidates for executing the selected process as the setting candidates, as the verification unit 103 performs in other steps, and The optimum setting is applied to the process for which the optimum setting has been determined among the processes other than the process selected by the instruction unit 101.

  Depending on the classifier, the order of the teacher data may greatly affect the determination accuracy. Examples of classifiers whose data order affects accuracy include perceptron, passive-aggressive, AROW, and neural network. In addition, as teacher data, there is data in which the determination accuracy deteriorates when machine learning is performed. As an example, data that is too old may be adversely affected because its tendency is too different from the most recent data.

  The instruction unit 101 and the teacher / verification data creation unit 102 create several teacher data candidates having different periods and arrangement order of the data contained from the original teacher data. The verification unit 103 performs holdout verification by learning and determining each of the created teacher data candidates and verification data based on the settings determined in the previous steps. Then, the analysis unit 104 calculates an index for each candidate.

  When this verification is performed for all the set candidates, the analysis unit 104 calculates an index for each verification result, selects a teacher data candidate that is the best index, and determines it as adjusted teacher data. .

  Further, as a technique for further reducing false detection, a case selection technique using erroneous detection goodware data as adjustment data can be considered. By using the case selection method, the adjusted teacher data can be further adjusted to further improve the determination accuracy (see, for example, Japanese Patent Application No. 2015-087924).

  Further, the same verification method and index as those used in the other steps may be used, or different methods may be used depending on circumstances. For example, when the case selection method is used, the false detection rate can be further reduced (for example, see Japanese Patent Application No. 2015-087924). Therefore, for example, in this step, an index having a lower designated false detection rate than the previous step may be used.

  Next, as illustrated in FIG. 3, the adjustment device 10 adjusts the determination setting and generates a white list as the fourth adjustment step. At this time, the data used for verification, the classifier, and the feature extraction / dimension reduction setting are determined in the first to third adjustment steps and stored in the setting storage unit 110.

  This adjustment process is performed when the malware determination apparatus 20 performs determination based on the score threshold. First, the verification unit 103 performs holdout verification using the settings adjusted in the adjustment process so far, adjusted teacher data, and verification data. Next, the analysis unit 104 calculates an index and determines whether or not the index has reached a target index value given by an instruction or the like. As the target index value, for example, a detection rate adjusted to be equal to or lower than a specified erroneous detection rate may be used.

  When the objective index value is achieved, the score threshold value calculated by the analysis unit 104 in the verification is stored in the setting storage unit 110 as it is. On the other hand, when the target index value is not achieved, the analysis unit 104 performs the following processing to determine a white list and a score threshold. First, the analysis unit 104 calculates a score threshold value that exceeds the minimum detection rate. Next, goodware is extracted in order from the score that is higher by the number that is equal to or lower than the specified false detection rate, and is used as a white list. The analysis unit 104 stores the score threshold value and white list obtained in this way in the setting storage unit 110.

  When the adjustment process is completed, the adjustment device 10 outputs the determined adjustment items, that is, various settings and adjusted teacher data stored in the setting storage unit 110. The malware determination device 20 reads these outputs and reflects the adjustment items in each unit. In addition, the adjustment apparatus 10 may output as an electronic file, and may output as data on communication.

  Further, in each step, the process in which the analysis unit 104 selects and determines the setting from each candidate may include human judgment. For example, depending on the index, the best index value having the same value for a plurality of candidates may be measured. In such a case, it is conceivable that a candidate can determine one of these candidates.

[Process of First Embodiment]
The process of the first embodiment will be described. First, the process of the adjustment apparatus 10 is demonstrated using FIG. FIG. 6 is a flowchart illustrating the process of the adjustment device according to the first embodiment.

  As illustrated in FIG. 6, the instruction unit 101 reads an instruction or the like and performs initial setting of the setting storage unit 110 (step S <b> 101). Next, the teacher / verification data creation unit 102 performs data cleansing that removes errors, non-target data, and the like from the original teacher data, adjustment data, and original verification data (step S102). And the instruction | indication part 101 designates the adjustment process to perform (step S103).

  Then, the instruction unit 101 instructs each unit to execute verification in the adjustment process (step S104). The teacher / verification data creation unit 102 creates teacher data and verification data from the original teacher data, adjustment data, and original verification data, and stores them in the setting storage unit 110 (step S105). Then, the verification unit 103 learns teacher data based on the setting of the setting storage unit 110 and determines verification data (step S106).

  Here, the instruction unit 101 determines whether or not all verification of the current process is completed (step S107). Note that whether or not all the verifications have been completed is determined, for example, by whether or not verification data has been determined for all of the setting candidates. If the instruction unit 101 determines that all the verifications have not been completed (No in step S107), the instruction unit 101 returns to step S104 and further performs verification.

  When the instruction unit 101 determines that all the verifications have been completed (step S107, Yes), the analysis unit 104 performs analysis using an analysis method corresponding to each adjustment process, and stores the determined adjustment items in the setting storage unit 110. (Step S108).

  Here, when all the adjustment processes are not completed (No at Step S109), the instruction unit 101 returns to Step S103 and further specifies the adjustment process. When all the adjustment processes are completed (step S109, Yes), the adjustment device 10 outputs the adjusted settings and the like (step S110) and ends the process. In addition, the verification method and analysis method of each adjustment process are as showing in FIG.

  Next, processing of the malware determination device 20 will be described with reference to FIG. FIG. 7 is a flowchart showing processing of the malware determination device according to the first embodiment. First, the learning process will be described. In the learning process, teacher data that is known as malware or goodware is input. As shown in FIG. 7, when learning is performed, the feature extraction / dimension reduction unit 202 generates a feature vector from teacher data (step S201). Then, the classifier 203 performs learning using the feature vector (step S202).

  Next, the determination process will be described. In the determination process, determination target data that is unknown whether it is malware or goodware is input. As illustrated in FIG. 7, when the target file corresponds to the white list (Yes in step S <b> 211), the white list application unit 201 determines that the target file is goodware and ends the process.

  When the target file does not correspond to the white list (step S211, No), the feature extraction / dimension reduction unit 202 performs feature extraction of the target file and dimension reduction as necessary to generate a feature vector (step S212). Then, the classifier 203 calculates a malware-like score from the feature vector (step S213). And the determination part 204 performs malware determination from a score (step S214).

[Effect of the first embodiment]
After executing one or more pre-processes in a predetermined order, the adjustment apparatus 10 executes a score calculation process for calculating a score of the file by the classifier, and whether or not the file is malware based on the calculated score The optimal setting for executing each process in the malware determination apparatus 20 that performs the determination process is determined.

  The instruction unit 101 selects each process of the malware determination device 20 in a predetermined order. Each time the instruction unit 101 selects a process, the teacher / verification data creation unit 102 and the verification unit 103 sequentially apply the setting candidates for executing the selected process, and the instruction unit 101 selects the process. The optimum setting is applied to the process for which the optimum setting has been determined among the processes other than the processed process. Then, each time the setting is applied, the verification unit 103 acquires a result corresponding to each of the setting candidates when each process of the malware determination apparatus 20 is executed according to the applied setting.

  Each time the analysis unit 104 obtains corresponding results for all of the process setting candidates selected by the teacher / verification data creation unit 102 and the verification unit 103, the analysis unit 104 sets each of the setting candidates. The candidate of the setting determined that the determination accuracy of malware is the highest from the result corresponding to is determined as the optimal setting of the selected process.

  As a result, even when a plurality of processes of the malware determination apparatus 20 are adjusted, it is not necessary to perform verification and analysis for all combinations of each process and setting candidates, and adjustment items of the accuracy improvement method can be adjusted. It can be done efficiently.

  The instruction unit 101 selects each process in the reverse order to the order executed in the malware determination device 20, for example. When the malware determination apparatus 20 performs preprocessing in the order of whitelist application, feature extraction, and dimension reduction, and determines whether a file is malware or not using a threshold value, the instruction unit 101 performs score calculation processing. Each process may be specified in the order of feature extraction and dimension reduction, white list application, and determination process. Further, the adjustment device 10 may adjust the teacher data using each unit. In this way, by making adjustments in order starting from adjustment items that are closely related to the classifier, it is possible to efficiently make adjustments for more effective accuracy improvement.

[Second Embodiment]
In the first embodiment, the case where the malware determination device performs both the learning process and the determination process has been described. On the other hand, in 2nd Embodiment, the example in case the learning function is not provided in the malware determination apparatus is demonstrated.

[Configuration of Second Embodiment]
The configuration of the malware determination system including the adjustment device and the malware determination device according to the second embodiment will be described with reference to FIG. FIG. 8 is a diagram illustrating an example of a configuration of a malware determination system including the adjustment device and the malware determination device according to the second embodiment.

  As shown in FIG. 8, the classifier 203a of the malware determination device 20 does not have a learning function unlike the first embodiment. Therefore, the malware determination device 20 cannot create a learned identification model for malware determination using teacher data. Therefore, the verification unit 103 of the adjustment device 10 acquires an identification model for malware determination by machine learning, and outputs the acquired identification model to the malware determination device 20 together with the adjusted setting.

[System configuration, etc.]
Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. Further, all or any part of each processing function performed in each device is realized by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or hardware by wired logic. Can be realized as

  In addition, among the processes described in the present embodiment, all or part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

[program]
FIG. 9 is a diagram illustrating an example of a computer in which the adjustment apparatus is realized by executing a program. The computer 1000 includes a memory 1010 and a CPU 1020, for example. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to the display 1130, for example.

  The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the adjustment apparatus 10 is implemented as a program module 1093 in which a code executable by a computer is described. The program module 1093 is stored in the hard disk drive 1090, for example. For example, a program module 1093 for executing processing similar to the functional configuration in the adjustment apparatus 10 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

  The setting data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1090 to the RAM 1012 and executes them as necessary.

  The program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read out by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and the program data 1094 may be read by the CPU 1020 from another computer via the network interface 1070.

DESCRIPTION OF SYMBOLS 1 Malware determination system 10 Adjustment apparatus 20 Malware determination apparatus 101 Instruction part 102 Teacher and verification data creation part 103 Verification part 104 Analysis part 110, 210 Setting storage part 201 White list application part 202 Feature extraction / dimension reduction part 203, 203a Classification 204 Determination part

Claims (6)

After executing one or more pre-processes in a predetermined order, a score calculation process for calculating a score of a file by a classifier is performed, and a determination process for determining whether the file is malware based on the calculated score An adjustment device for determining an optimum setting for executing each process in the malware determination device to perform,
An instruction unit for selecting each process of the malware determination device in an order reverse to the order executed in the malware determination device ;
Each time a process is selected by the instruction unit, the setting candidates for executing the selected process are sequentially applied, and an optimum setting is determined among processes other than the process selected by the instruction unit. A setting application unit that applies the optimum setting to the already processed process;
Each time a setting is applied by the setting application unit, a result corresponding to each of the setting candidates is obtained when each process of the malware determination apparatus is executed according to the setting applied by the setting application unit. A verification unit to
Each time the result corresponding to all of the setting candidates of the selected process is acquired by the verification unit, the determination accuracy of malware is determined from the result corresponding to each of the setting candidates among the setting candidates. An analysis unit that determines a setting candidate determined to be the highest as an optimum setting of the selected process;
The adjustment apparatus characterized by having. 2. The process according to claim 1, wherein the instructions are selected by the instruction unit in an order reverse to the order executed in the malware determination device, and are whitelist application, feature extraction, and dimension reduction . Adjustment device. Adjustment device according to claim 1 or 2, characterized by further comprising a training data adjusting unit for adjusting the training data. Learning teacher data, the adjusting device according to any one of claims 1 to 3, further comprising a learning unit for creating an identification model for the score calculation process in the classifier. After executing one or more pre-processes in a predetermined order, a score calculation process for calculating a score of a file by a classifier is performed, and a determination process for determining whether the file is malware based on the calculated score An adjustment method that is executed by an adjustment device that determines an optimal setting for executing each process in the malware determination device to perform,
An instruction step of selecting each process of the malware determination device in an order reverse to the order executed in the malware determination device ;
Each time a process is selected by the instruction process, the setting candidates for executing the selected process are sequentially applied, and an optimal setting is determined among processes other than the process selected by the instruction process. A setting application step for applying the optimum setting to the process that has been completed;
Each time a setting is applied by the setting application step, a result corresponding to each of the setting candidates is obtained when each process of the malware determination device is executed according to the setting applied by the setting application step. A verification process to
Whenever the result corresponding to all the setting candidates of the selected process is acquired by the verification step, the determination accuracy of malware is determined from the result corresponding to each of the setting candidates among the setting candidates. An analysis step of determining a candidate for the setting determined to be the highest as the optimum setting for the selected process;
The adjustment method characterized by including. An adjustment program for causing a computer to function as the adjustment device according to any one of claims 1 to 4 .

Images