JP2016206950A - Perusal training data output device for malware determination, malware determination system, malware determination method, and perusal training data output program for malware determination - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make malware determination efficiently and free of false detection using machine learning.SOLUTION: The present invention extracts a feature vector of training data 100 that includes existing malware data 101 and existing good-ware data 102 relative to misdetected good-ware data 110, removes some of the existing malware data 101 the feature vector of which is similar to the misdetected good-ware data 110, and inserts the misdetected good-ware data 110 into the training data and makes it perusal training data 120. At this time, a sorter that has learned the perusal training data 120 inserts the misdetected good-ware data 110 at a position at which a misdetection rate to the misdetected good-ware data 110 is reduced.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a scrutinizing teacher data output device for malware determination, a malware determination system, a malware determination method, and a scrutinizing teacher data output program for malware determination.

  In an anti-virus system that determines whether or not an executable file of Windows (registered trademark) or UNIX (registered trademark) is malware, dynamic determination that determines whether or not the executable file is executed and static determination that determines without executing the executable file The method is used. Furthermore, static determination is used when speed is particularly required for determination.

  Representative examples of static determination methods include hash value match determination and pattern match determination (signature scan). First, the hash value match determination has a hash value such as MD5, SHA1, SHA256, etc. of known malware in advance as a database, and if the hash value of the inspection file matches that database, the hash value is determined as malware. Next, pattern match determination has a specific character string or byte code included in known malware in advance as a database, and the file to be inspected contains any character string or byte code registered in the database. If it is, it will be malware.

  However, in the hash value match determination and the pattern match determination, it is difficult to detect a variant malware or a new malware that is a modification of the existing malware. For this reason, heuristic determination has been proposed as a method for determining sub-species / new-type malware. This defines malware likeness based on past experience, and determines whether or not it is malware according to the definition.

  Several methods using machine learning techniques have been proposed as heuristic determination methods. For example, a method has been proposed in which a human-readable character string included in an executable file is learned in advance, and the malware-likeness of the inspection file is determined based on how many words frequently used in malware are included in the inspection file. (For example, refer to Patent Document 1). In the proposed method, for example, teacher data that is data to be learned is converted into a set of several parameters, and learning is performed by a machine learning algorithm. Here, each parameter is called a feature, and a set of parameters is called a feature vector. For example, a combination of a word name and the number of appearances is a feature vector. Various methods for improving classification accuracy (F value, etc.) and learning processing speed in a method using heuristic determination using machine learning technology have been proposed.

  For example, in one method, in the malware determination by machine learning using the PE header information of the executable file, by using an appropriate dimension compression and a machine learning algorithm, a detection rate of 99% and a false detection of 0.5% or less The rate is achieved (see, for example, Non-Patent Document 1).

Also, in machine learning, a method related to case selection that improves accuracy by examining the teacher data, such as removing data that seems to deteriorate the classification accuracy from the teacher data, has been proposed. For example, in image classification using SVM (Support Vector Machine), a case selection method that uses internal parameters α _{i of} SVM to extract ambiguous image data that is difficult to classify and removes these data from teacher data Has been proposed (see, for example, Non-Patent Document 2). Also, in image classification using the sparse representation classification method (SRC method), the contribution rate is calculated using a score representing the certainty of the classification during learning, and only a typical case with a high contribution rate, that is, a more probable probability. A technique has been proposed in which learning data is shortened while maintaining high classification accuracy by using as teacher data (see, for example, Patent Document 2). Furthermore, a method for comparing a detection result with a white list is generally known.

JP 2012-27710 A JP 2012-173895 A

Shafiq, et al., "PE-Miner: Mining Structural Information to Detect Malicious Executables in Realtime", RAID '09, 2009. Takatori et al., “Proposal of case selection method based on internal parameters of support vector machine and application to video boundary detection problem”, Japanese Society for Artificial Intelligence (22nd), 2008.

  However, the above-mentioned conventional technology cannot sufficiently reduce the false detection in malware judgment, and the method using the white list requires an extra processing time for white list matching, a white list storage area, and the like. There is a problem that is necessary.

  Generally, in malware determination, since the number of goodware to be determined is large, a particularly low false detection rate is required. For example, there are 2000 or more executable files in the system directory of Windows 7 which is a Microsoft OS. Therefore, even if the false detection rate is 0.5%, 10 or more pieces of goodware are mistakenly detected as malware. Therefore, it cannot be said that the false detection rate is sufficiently low.

  In addition, there is a known method of correcting a false detection by creating a pre-recorded white list and referring to this white list at the time of determination and determining that the match is goodware. Since a storage area for storing and a processing time for collation are required, efficiency is lowered.

  Therefore, an object of the present invention is to perform malware determination using machine learning efficiently and with few false detections.

  The scrutinizing teacher data output device for malware determination of the present invention may be existing malware data that is a file known to be malignant, existing goodware data that is a file known to be benign, and benign. From the feature extraction unit that extracts each feature of erroneously detected goodware data that is a known but determined to be malignant as a feature vector, and the teacher data including the existing malware data and the existing goodware data, the error A similar malware removal unit that deletes existing malware data whose feature vector is similar to the feature vector of the detected goodware data, and a false detection that inserts the erroneously detected goodware data into the teacher data as the existing goodware data And a good wear insertion portion.

  In addition, the malware determination system of the present invention includes existing malware data that is a file known to be malignant, existing goodware data that is a file known to be benign, and malignant that is known to be benign. From the first feature extraction unit that extracts each feature of the erroneously detected goodware data that is a file determined as a feature vector, and the teacher data including the existing malware data and the existing goodware data, the erroneously detected good A similar malware removal unit that deletes existing malware data whose feature vector is similar to the feature vector of the wear data, and the false detection goodware that inserts the false detection goodware data into the teacher data as the existing goodware data A scrutinizing teacher data output device having an insertion portion, and a front A second feature extraction unit that extracts each feature of a target file that is a target for determining whether or not it is teacher data and malware with the erroneous detection goodware data inserted by the erroneous detection goodware insertion unit. A first classifier that learns the feature vector of the teacher data and calculates a score indicating the malware likeness of the target file from the feature vector of the target file; and the target file is malware based on the score A learning determination device having a determination unit for determining whether or not.

  In addition, the malware determination method of the present invention is a malware determination method executed by the malware determination system. The existing malware data is a file that is known to be malignant, and the existing good that is a file that is known to be benign. A first feature extraction step of extracting each feature of the wear data and the misdetected goodware data which is a file known to be benign but determined to be malignant as a feature vector, the existing malware data and the A similar malware removal step of deleting existing malware data having a feature vector similar to the feature vector of the erroneously detected goodware data from the teacher data including the existing goodware data, and the erroneously detected goodware data to the existing goodware data. Wear data into the teacher data Each feature of the target file that is a target for determining whether or not it is a teacher data and malware in which the erroneously detected goodware data is inserted by the known goodware inserting step and the erroneously detected goodware inserting step is used as a feature vector. A second feature extraction step to extract, a classification step of learning a feature vector of the teacher data and calculating a score indicating the malware likeness of the target file from the feature vector of the target file; and the target based on the score And a determination step of determining whether or not the file is malware.

  According to the present invention, it is possible to perform malware determination using machine learning efficiently and with few false detections.

FIG. 1 is a diagram illustrating an example of the configuration of the malware determination system according to the first embodiment. FIG. 2 is a flowchart illustrating an example of a case selection process in the malware determination system according to the first embodiment. FIG. 3 is a flowchart illustrating an example of learning processing in the malware determination system according to the first embodiment. FIG. 4 is a flowchart illustrating an example of determination processing in the malware determination system according to the first embodiment. FIG. 5 is a diagram illustrating an example of the configuration of the malware determination system according to the second embodiment. FIG. 6 is a diagram illustrating an example of a configuration of a false detection goodware data output apparatus according to the second embodiment. FIG. 7 is a flowchart illustrating an example of processing in the malware determination system according to the second embodiment. FIG. 8 is a diagram illustrating an example of a configuration of a malware determination device according to another embodiment. FIG. 9 is a diagram illustrating an example of a computer that realizes a malware determination device and a close examination teacher data output device by executing a program.

  Hereinafter, a scrutinizing teacher data output device for malware determination, a malware determination system, a malware determination method, and a scrutinizing teacher data output program for malware determination according to the present application will be described in detail with reference to the drawings. The embodiment does not limit the scrutinizing teacher data output device for malware determination, the malware determination system, the malware determination method, and the scrutinizing teacher data output program for malware determination according to this embodiment.

[Configuration of First Embodiment]
First, the configuration of the first embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of the configuration of the malware determination system according to the first embodiment. As shown in FIG. 1, the malware determination system 1 includes a scrutinizing teacher data output device 10 and a learning / determination device 20. The scrutinizing teacher data output device 10 performs case selection using the teacher data 100 composed of the existing malware data 101 and the existing goodware data 102 and the erroneous detection goodware data 110, and outputs the scrutinizing teacher data 120. . Further, the learning / determination device 20 learns the scrutinizing teacher data 120, determines the target file 130, and outputs a determination result 140.

  The scrutinizing teacher data output device 10 includes a feature extraction unit 11, a dimension reduction unit 12, a similar malware removal unit 13, and a false detection goodware insertion unit 14. Note that the existing malware data 101 and the existing goodware data 102 of the teacher data 100 include information indicating the existing malware and the executable file of the goodware, and the classification indicating whether the executable file is malware or goodware. Yes. The erroneous detection goodware data 110 includes a plurality of erroneously detected goodware execution files. The erroneously detected goodware data 110 may be an executable file that is not included in the teacher data 100. For example, it is possible to include goodware or the like for which a false detection is reported by the user. Note that the processing performed by the scrutinizing teacher data output device 10 can also be called case selection, and the scrutinizing teacher data output device can also be called a case selection device or the like.

  The feature extraction unit 11 determines that the existing malware data 101 is a file that is known to be malignant, the existing goodware data 102 that is a file that is known to be benign, and a benign that is known to be benign, but is determined to be malignant. Each feature of the erroneously detected goodware data 110 that is a file is extracted as a feature vector. At this time, the feature extraction unit 11 may perform selection of each feature for the purpose of improving accuracy before the conversion to the feature vector. The dimension reduction unit 12 performs dimension compression of the feature vector using principal component analysis or the like. Here, the dimension reduction unit 12 can automatically combine the correlated features into one feature by using the principal component analysis.

  The similar malware removing unit 13 deletes the existing malware data 101 whose feature vector is similar to the feature vector of the erroneously detected goodware data 110 from the teacher data 100 including the existing malware data 101 and the existing goodware data 102. Specifically, first, the similar malware removing unit 13 calculates the similarity using the feature vector of the executable file of the existing malware data 101 and the feature vector of the executable file of each false detection goodware data 110. Then, the similar malware removing unit 13 determines whether or not the similarity is similar by performing a condition determination on the calculated similarity, and determines the execution file in the teacher data 100 determined to be similar to the existing malware data 101, that is, the teacher data 100. Remove from inside.

  Here, the similar malware removal unit 13 can use any one of Euclidean distance, cosine similarity, Jaccard coefficient, and the like as the similarity. For example, the Euclidean distance is defined as the square root of the inner product of the difference between the two feature vectors. Therefore, for example, the similar malware removing unit 13 may determine that they are similar if the Euclidean distance is equal to or less than a specified threshold.

  The cosine similarity is defined as the cosine of the angle formed by the two feature vectors. Therefore, for example, the similar malware removing unit 13 may determine that they are similar to each other if the cosine similarity is equal to or greater than a specified threshold value. Further, the Jaccard coefficient is defined as the number of elements that match each other divided by the number of all elements for each element whose two feature vector values are not zero. Therefore, for example, the similar malware removing unit 13 may determine that they are similar if the Jaccard coefficient is equal to or greater than a specified threshold value.

  The erroneous detection goodware insertion unit 14 inserts the erroneous detection goodware data 110 into the teacher data 100 and outputs it as the scrutinizing teacher data 120. Here, in some classifiers using machine learning algorithms, the arrangement order of teacher data greatly affects the classification accuracy. Therefore, erroneous detection can be reduced by inserting the erroneous detection goodware data 110 into the teacher data 100 in an appropriate arrangement order according to the type of the classifier 23 of the learning / determination device 20 described later.

  For example, when an online machine learning algorithm AROW (Adaptive Regularization of Weight Vectors) is used as the classifier 23, the erroneous detection goodware insertion unit 14 inserts the erroneous detection goodware data 110 at the end of the teacher data 100. It was observed that false detection can be further reduced. On the other hand, when a support vector machine is used as the classifier 23, it is observed that the false detection goodware insertion unit 14 inserts the false detection goodware data 110 at the head of the teacher data 100, thereby further reducing the false detection. It was. Further, when a perceptron or the like is used as the classifier 23, the same number as the erroneously detected goodware data 110 is inserted at different positions in the teacher data 100 to increase the number of times the erroneously detected goodware data 110 is learned. It was effective.

  Finally, the scrutinizing teacher data output device 10 outputs the teacher data 100 corrected by the similar malware removing unit 13 and the erroneous detection goodware inserting unit 14 as the scrutinizing teacher data 120. Similar to the teacher data 100, the scrutinizing teacher data 120 includes existing malware and goodware executable files, and information indicating the classification of whether the executable file is malware or goodware. However, instead of the execution file, fragment information obtained by extracting only a part required by the learning / determination device 20 may be used. For example, when the learning / determination device 20 performs the malware determination using the PE header, the fragment information obtained by extracting only the region portion of the PE header information may be used as the close examination teacher data 120. The scrutinizing teacher data 120 is information on a medium such as a fixed storage medium, a memory, or communication.

  The learning / determination device 20 includes a feature extraction unit 21, a dimension reduction unit 22, a classifier 23, and a determination unit 24. The feature extraction unit 21 extracts features of the scrutinizing teacher data 120 and the target file 130 as feature vectors. Here, the target file 130 is an executable file whose classification as malware or goodware is unknown. The dimension reduction unit 22 performs dimension compression in the same manner as the dimension reduction unit 12. Note that the feature extraction unit 21 and the dimension reduction unit 22 may have the same functions as the feature extraction unit 11 and the dimension reduction unit 12.

  The classifier 23 learns the feature vector of the scrutinizing teacher data 120 and calculates a score indicating the malware likeness of the target file 130 from the feature vector of the target file 130. Specifically, the classifier 23 receives the feature vector of the scrutinizing teacher data 120 and information on the classification of whether the execution file of the scrutinizing teacher data 120 is malware or goodware, and performs machine learning. For example, as the classifier 23, a classifier such as logistic regression, support vector machine, perceptron, AROW, naive Bayes, or the like can be used.

  The classifier 23 that has performed machine learning outputs the malware likeness of the target file 130 from the feature vector of the target file 130 as a numerical value called a score. And the determination part 24 determines whether it is malware based on a score, and outputs a determination result. The determination result may be only the classification of malware or goodware, or may include a score. The determination unit 24 may use threshold determination, for example, when the score is equal to or higher than a certain threshold, and malware when the score is equal to or lower than the threshold.

  When threshold determination is used as the determination method, if the threshold value is increased, the detection rate decreases, but the false detection rate also decreases. Conversely, if the threshold value is decreased, the detection rate / false detection rate increases. However, the relationship between the threshold, the detection rate, and the false detection rate is not a proportional relationship. According to the ROC curve, increasing the threshold within a certain threshold range does not increase the false detection rate so much, and the detection rate is known to increase greatly. ing. Therefore, by appropriately adjusting the threshold value in advance, it is possible to obtain a relatively high detection rate while keeping the false detection rate within the allowable range. For example, there is an adjustment method in which a target file for testing is prepared and determined (holdout verification), and a threshold value is determined so that the false detection rate is within an allowable range from the obtained score.

[Process of First Embodiment]
The process of the first embodiment will be described with reference to FIGS. 2-4 is a flowchart which shows an example of each process in the malware determination system which concerns on 1st Embodiment. The case selection process will be described with reference to FIG. Here, the case selection process indicates a series of processes performed by the scrutinizing teacher data output device 10 as described above.

  As shown in FIG. 2, the feature extraction unit 11 first extracts a feature vector from the teacher data 100 and the erroneous detection goodware data 110 (step S101). The teacher data 100 includes existing malware data 101 and existing goodware data 102. Next, the dimension reduction unit 12 compresses the dimension of the feature vector extracted by the feature extraction unit 11 (step S102).

  And the similar malware removal part 13 extracts the existing malware data 101 similar to the misdetection goodware data 110 using a feature vector, and deletes it from the teacher data 100 (step S103). Thereafter, the erroneous detection goodware insertion unit 14 inserts the erroneous detection goodware data 110 into an appropriate position of the teacher data 100 (step S104), and outputs the examination teacher data 120. Note that the appropriate position where the misdetected goodware data 110 is inserted includes the beginning, the end, and the like of the teacher data 100, the position that is most easily learned in the scrutinizing teacher data 120, and the classification in which the scrutinizing teacher data 120 is learned. It is a position where the false detection rate of the vessel becomes low.

  Next, the learning process performed by the learning / determination device 20 will be described with reference to FIG. As shown in FIG. 3, the feature extraction unit 21 first extracts a feature vector from the scrutinizing teacher data 120 (step S111). Next, the dimension reduction unit 22 compresses the dimension of the feature vector extracted by the feature extraction unit 21 (step S112). Then, the classifier 23 learns the scrutinizing teacher data 120 (step S113).

  Finally, the determination process performed by the learning / determination device 20 will be described with reference to FIG. As shown in FIG. 4, first, the feature extraction unit 21 extracts a feature vector from the target file 130 (step S121). Next, the dimension reduction unit 22 compresses the dimension of the feature vector extracted by the feature extraction unit 21 (step S122). Then, the classifier 23 calculates a malware-like score of the target file 130 from the feature vector (step S123), and the determination unit 24 determines whether the target file 130 is malware based on the score (step S123). S124).

[Effect of the first embodiment]
The scrutinizing teacher data output device 10 extracts the features of the existing malware data 101, the existing goodware data 102, and the false detection goodware data 110 as feature vectors, respectively, and includes the existing malware data 101 and the existing goodware data 102. The existing malware data 101 whose feature vector is similar to the feature vector of the erroneous detection goodware data 110 is deleted from the data 100, the erroneous detection goodware data 110 is inserted into the teacher data 100 as the existing goodware data 102, Output as scrutinizing teacher data 120.

  Previously, misclassifications that determined goodware as malware occurred as a result of the classifier learning malware with similar characteristics to the relevant goodware, and the relevant goodware itself was difficult to learn well Two of the conditions are listed. In the first embodiment of the present invention, it is possible to prevent the classifier from learning malware similar to goodware by removing erroneously detected goodware data from teacher data. In addition, by including erroneously detected goodware data in the teacher data, it is easy to learn erroneously detected goodware data. Also, since no white list is used, no extra processing time or storage area is required. As described above, according to Embodiment 1 of the present invention, it is possible to perform malware determination using machine learning efficiently and with few false detections.

  Further, the scrutinizing teacher data output device 10 determines in which position of the teacher data 100 the erroneous detection goodware data 110 is inserted according to the type of classifier that learns the feature vector of the scrutinizing teacher data 120. Then, erroneous detection goodware data 110 is inserted at the determined position. Depending on the type of classifier, there are cases where it is most easily learned by inserting it at the beginning, and cases where it is most easily learned by inserting it at the end. By determining the insertion position in consideration of the characteristics of the classifier and the like, learning can be performed so that the false detection rate is further reduced. Moreover, the effect that it becomes easier to learn can also be obtained by inserting the same plurality of data as the erroneously detected goodware data into the teacher data.

[Configuration of Second Embodiment]
A device for outputting the erroneous detection goodware data 110 may be added to the malware determination system 1 to have a series of functions from the output of the erroneous detection goodware data 110 to the malware determination. Therefore, in the second embodiment, as shown in FIG. 5, the erroneous detection goodware data output device 30 is included in the malware determination system 1. FIG. 5 is a diagram illustrating an example of the configuration of the malware determination system according to the second embodiment.

  As illustrated in FIG. 5, the erroneous detection goodware data output device 30 includes erroneous detection goodware data 110 from the erroneous detection goodware candidate data 105, the existing malware data 101 and the existing goodware data 102 that are the teacher data 100. Is extracted and output. Then, the output erroneous detection goodware data 110 is used in the scrutinizing teacher data output device 10 and the learning / determination device 20 as in the first embodiment. The erroneous detection goodware candidate data 105 includes goodware that may be erroneously detected by the malware determination system 1. For example, the erroneous detection goodware candidate data 105 may include a part or all of the existing goodware data 102 included in the teacher data 100, or teacher data such as goodware for which a false detection is reported by the user. Goodware not included in 100 may be included.

  The erroneous detection goodware data output device 30 will be described in detail with reference to FIG. FIG. 6 is a diagram illustrating an example of a configuration of a false detection goodware data output apparatus according to the second embodiment. As shown in FIG. 6, the erroneous detection goodware data output device 30 includes a learning / determination unit 30 a and an erroneous detection goodware selection unit 40. The learning / determination unit 30 a includes a feature extraction unit 31, a dimension reduction unit 32, a classifier 33, and a determination unit 34.

  The function of each part of the learning / determination unit 30a is the same as the function of each part of the learning / determination device 20 in the first embodiment. That is, the feature extraction unit 31 extracts features of the teacher data 100 and the erroneous detection goodware candidate data 105 as feature vectors. In addition, the dimension reduction unit 32 performs dimension compression in the same manner as the dimension reduction unit 22.

  The classifier 33 learns the feature vector of the teacher data 100 and calculates a score indicating the malware likeness of the erroneous detection goodware candidate data 105 from the feature vector of the erroneous detection goodware candidate data 105. Specifically, the classifier 33 performs machine learning based on the feature vector of the teacher data 100 and information on the classification of whether the execution file of the teacher data 100 is malware or goodware. The classifier 33 that has performed machine learning outputs the malware-likeness of the false positive goodware candidate data 105 as a numerical value called a score from the feature vector of the false positive goodware candidate data 105. And the determination part 34 outputs the determination result 106 with a score. Here, the scored determination result 106 includes a name for identifying goodware (for example, a file name, a file hash value, or an erroneously detected goodware candidate data sequence number) and its score.

  Then, the erroneous detection goodware selection unit 40 selects the erroneous detection goodware from the erroneous detection goodware candidate data 105 according to the specified condition based on the scored determination result 106, and outputs the erroneous detection goodware data 110. Examples of the designated condition include selecting goodware having a score equal to or higher than the specified threshold, or selecting the specified number of goodware in descending order of score.

[Process of Second Embodiment]
The processing of the second embodiment will be described with reference to FIG. FIG. 7 is a flowchart illustrating an example of processing in the malware determination system according to the second embodiment. As shown in FIG. 7, the feature extraction unit 31 first extracts a feature vector from the teacher data 100 (step S201). The teacher data 100 includes existing malware data 101 and existing goodware data 102. Next, the dimension reduction unit 32 compresses the dimension of the feature vector extracted by the feature extraction unit 31 (step S202). Then, the classifier 33 learns the teacher data 100 (step S203).

  Next, the feature extraction unit 31 extracts a feature vector from the erroneous detection goodware candidate data 105 (step S204). Then, the dimension reduction unit 32 compresses the dimension of the feature vector extracted by the feature extraction unit 31 (step S205). Then, the classifier 33 calculates a score of the likelihood of malware of the erroneous detection goodware candidate data 105 from the feature vector (step S206), and the determination unit 34 outputs the scored determination result 106 (step S207). Finally, the erroneous detection goodware selection unit 40 extracts the high-score goodware from the erroneous detection goodware candidate data 105 in the order of high-score goodware data 110 and outputs it as the erroneous detection goodware data 110 (step S208).

[Effects of Second Embodiment]
In the second embodiment, a false detection goodware data output device 30 is added to the configuration of the malware determination system 1 of the first embodiment. The false detection goodware data output device 30 extracts features of the teacher data 100 and the false detection goodware candidate data 105 that is a candidate of the false detection goodware data 110 as feature vectors, and the feature vector of the teacher data 100 is extracted. Learning, calculating a score indicating the likelihood of malware of the false positive goodware candidate data 105 from the feature vector of the false positive goodware candidate data 105, and calculating the score of the false positive goodware candidate data 105 that has a score greater than a predetermined threshold Extracted and output as false detection goodware data. Thereby, the erroneous detection goodware data 110 can be selected and generated appropriately and efficiently.

[Other Embodiments]
The present invention is not limited to the above-described embodiment. For example, the feature extraction unit 11 and the dimension reduction unit 12 of the scrutinizing teacher data output device 10 are the feature extraction unit 21 and the dimension reduction unit of the learning / determination device 20. 22 may be the same or different. For example, when a Jaccard coefficient is used as the similarity in the similar malware removal unit 13, since the similarity cannot be calculated if dimension compression is performed, the dimension reduction unit 12 does not perform dimension compression, and only the dimension reduction unit 22 performs the calculation. Different processing such as dimensional compression may be performed.

  Moreover, in the case selection, the example in which the error detection goodware insertion unit 14 performs processing after the similar malware removal unit 13 has been described. However, the order is reversed, and the similar malware removal unit next to the error detection goodware insertion unit 14. Processing may be performed in step 13. In addition, both processes are not necessarily performed, but only the process of the similar malware removing unit 13 or the process of the false detection goodware insertion unit 14 may be performed.

  In the first embodiment, the malware determination system 1 includes the scrutinizing teacher data output device 10 and the learning / determination device 20. However, the malware determination device has the scrutinizing teacher data output device 10 and the learning / determination device. It is good also as a structure which has the function of the apparatus 20. FIG. FIG. 8 is a diagram illustrating an example of a configuration of a malware determination device according to another embodiment. As shown in FIG. 8, the malware determination device 1 a includes a scrutinizing teacher data output unit 10 a having the same function as the scrutinizing teacher data output device 10 and a learning / determination unit 20 a having the same function as the learning / determination device 20. Is done.

[System configuration, etc.]
Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. Further, all or any part of each processing function performed in each device is realized by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or hardware by wired logic. Can be realized as

  Also, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

[program]
FIG. 9 is a diagram illustrating an example of a computer that realizes the malware determination device 1a and the scrutinizing teacher data output device 10 by executing a program. The computer 1000 includes a memory 1010 and a CPU 1020, for example. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to the display 1130, for example.

  The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the malware determination device 1a and the scrutinizing teacher data output device 10 is implemented as a program module 1093 in which a code executable by a computer is described. The program module 1093 is stored in the hard disk drive 1090, for example. For example, a program module 1093 for executing processing similar to the functional configuration in the malware determination device 1a and the scrutinizing teacher data output device 10 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

  The setting data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1090 to the RAM 1012 and executes them as necessary.

  The program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read out by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and the program data 1094 may be read by the CPU 1020 from another computer via the network interface 1070.

DESCRIPTION OF SYMBOLS 1 Malware determination system 1a Malware determination apparatus 10 Scrutinizing teacher data output apparatus 10a Scrutinizing teacher data output part 11, 21, 31 Feature extraction part 12, 22, 32 Dimension reduction part 13 Similar malware removal part 14 False detection goodware insertion part 20 Learning -Judgment device 20a, 30a Learning / determination unit 23, 33 Classifier 24, 34 Judgment unit 30 False detection goodware data output device 40 False detection goodware selection unit 100 Teacher data 101 Existing malware data 102 Existing goodware data 105 False detection Goodware candidate data 106 Judgment result with score 110 False detection goodware data 120 Examination teacher data 130 Target file 140 Judgment result

Claims (8)

Existing malware data that is known to be malignant, existing Goodware data that is known to be benign, and false positive that is known to be benign but determined to be malignant A feature extraction unit for extracting each feature of the wear data as a feature vector,
A similar malware removing unit that deletes existing malware data whose feature vector is similar to the feature vector of the false positive goodware data from the teacher data including the existing malware data and the existing goodware data;
The erroneous detection goodware data, the erroneous detection goodware insertion unit for inserting into the teacher data as the existing goodware data,
A scrutinizing teacher data output device for malware determination, characterized by comprising: The erroneous detection goodware insertion part is
According to the type of classifier that learns the feature vector of the teacher data, it is determined in which position of the teacher data to insert the erroneous detection goodware data, and the erroneous detection goodware is determined at the determined position. The scrutinizing teacher data output device for malware determination according to claim 1, wherein data is inserted. The erroneous detection goodware insertion part is
The scrutinizing teacher data output device for malware determination according to claim 1, wherein the erroneously detected goodware data is inserted at a head or a tail of the teacher data. The erroneous detection goodware insertion part is
The scrutinizing teacher data output device for malware determination according to any one of claims 1 to 3, wherein a plurality of data identical to the erroneously detected goodware data is inserted into the teacher data. Existing malware data that is known to be malignant, existing Goodware data that is known to be benign, and false positive that is known to be benign but determined to be malignant The feature vector of the false detection goodware data is similar to the feature vector from the first feature extraction unit that extracts each feature of the wear data as a feature vector, and the teacher data including the existing malware data and the existing goodware data. A similar malware removal unit that deletes existing malware data, and a false detection goodware insertion unit that inserts the erroneous detection goodware data into the teacher data as the existing goodware data When,
Second feature extraction for extracting each feature of a target file as a target for determining whether or not it is malware and teacher data into which the erroneously detected goodware data is inserted by the erroneously detected goodware insertion unit A first classifier that learns a feature vector of the teacher data and calculates a score indicating the malware likeness of the target file from the feature vector of the target file, and the target file is malware based on the score A determination unit for determining whether or not there is a learning determination device,
A malware determination system characterized by comprising:   A third feature extraction unit that extracts each feature of the candidate data of false detection goodware, which is a candidate of the teacher data and the false detection goodware data, as a feature vector; learning the feature vector of the teacher data; and A second classifier that calculates a malware-like score of the erroneously detected goodware candidate data from the feature vector of the detected goodware candidate data; and the score of the erroneously detected goodware candidate data is greater than a predetermined threshold The malware determination system according to claim 5, further comprising a false detection goodware data output device having a false detection goodware extraction unit that extracts a thing and outputs it as the false detection goodware data. A malware determination method executed by a malware determination system,
Existing malware data that is known to be malignant, existing Goodware data that is known to be benign, and false positive that is known to be benign but determined to be malignant A first feature extraction step of extracting each feature of the wear data as a feature vector,
A similar malware removal step of deleting existing malware data whose feature vector is similar to the feature vector of the false positive goodware data from the teacher data including the existing malware data and the existing goodware data;
The erroneous detection goodware insertion step of inserting the erroneous detection goodware data into the teacher data as the existing goodware data;
Second feature extraction for extracting each feature of a target file as a target for determining whether or not it is malware and teacher data into which the erroneously detected goodware data has been inserted in the erroneously detected goodware insertion step Process,
A classification step of learning a feature vector of the teacher data and calculating a score indicating the malware likeness of the target file from the feature vector of the target file;
A determination step of determining whether the target file is malware based on the score;
Malware determination method characterized by including.   A scrutinizing teacher data output program for malware determination for causing a computer to function as the scrutinizing teacher data output device for malware determination according to any one of claims 1 to 4.

Images