KR20240039407A - Robustness measurement system and application of ai model for malware variant analysis - Google Patents

Abstract

The present invention, in a system for measuring the robustness of an AI model to respond to adversarial attacks, uses the distance parameter between the original data and the attack sample generated due to the success of the adversarial attack to prevent misclassification while remaining close to the original. a preprocessing module that generates challenging adversarial training samples; a robustness measurement module that calculates the average value of the distance between the adversarial training sample and the original data as a robustness index value for evaluating the robustness level of the AI model; And an output module that outputs the robustness level measurement status for each malicious code group of a hostile attack using the value of the robustness index calculated by the robustness measurement module; including, outputting an evaluation index of the robustness of the AI model against hostile attacks. It's characteristic is to do something.

Description

Robustness measurement system and application of AI model for analyzing malware variants {ROBUSTNESS MEASUREMENT SYSTEM AND APPLICATION OF AI MODEL FOR MALWARE VARIANT ANALYSIS}

The present invention seeks to provide an AI model robustness measurement system and application that can measure the robustness of an AI model in order to respond to adversarial attacks that induce learning classification errors and a decrease in reliability of the AI model.

As the digital industry becomes more sophisticated, the methods and types of cyber attacks are becoming more diverse. Evaluation to quickly and accurately detect suspicious data or malicious access is becoming especially important in high-volume networks with high traffic.

Existing AI models learn from a given data set and determine whether it is malicious or normal. However, there is a lack of reliability that AI models that have learned from existing attacks can effectively respond to new attacks. Newly emerging attacks will be attacks that have been modified to evade technological advancements and existing detection systems.

As an example, there is an adversarial attack to cause misclassification and lower reliability of the AI model. Adversarial attack is an attack technique that deceives the AI model by giving a very small perturbation to the original that is difficult to distinguish. There is increasing interest in adversarial attacks to evade detection by AI models. Zeroth Order Optimization based Black-Box Attacks (ZOO Attack) is a technology that creates hostile attacks. It prevents attacks due to exposure of gradient values and hides gradients through Gradient Masking.

Although cases of hostile attacks and attack-related papers have been increasing continuously since 2014, there are no hostile attack-related items in the National Vulnerability Database (NVD), and there are not many actual response activities to hostile attacks. Accordingly, a method is needed to create a robust model that can respond to such attacks.

As a related prior art, Korean Patent Publication No. 10-2022-0049967 discloses the invention of a defense method and device against an adversarial attack. The prior patent includes the steps of acquiring a target image that is the object of image recognition; Classifying the acquired target image by color to generate a plurality of color maps; Extracting features of objects included in each color map based on a Convolutional Neural Network (CNN) model, and classifying objects included in the color map based on the extracted features; and assigning priorities to objects included in the color map and recognizing the target image based on classification information and priority information about the objects. The above prior patent protects against hostile attacks by assigning different priorities to objects.

In order to respond to adversarial attacks targeting machine learning, it will be necessary to check whether the AI model is robust enough to respond to malware variants and the level of robustness. AI models are not simply measured using conventional evaluation indicators such as Accuracy, Precision, Recall, F1 score, etc., but there is a need to introduce a separate evaluation indicator at the level of robustness to confirm its reliability.

Nevertheless, there has been no agreed upon formal, standardized method for evaluating the robustness of AI models. In addition, the indicators used in research to evaluate the robustness of existing AI models are unsuitable for use in cases where the range of feature values is not the same, such as image data. In the conventional case, this is based on the premise that the feature scales of the data are all the same, and a unified robustness index that can be applied equally even when the meaning or range of each feature is different has not been presented.

In addition, existing research on robustness evaluation focuses on measuring the size of perturbation through the distance from the original, and has the limitation of not considering the amount of change in probability due to modulation. Since the number of features used in AI models is different for each model, the comparison of robustness measurements cannot be unified when comparing with different AI models, so standardization of robustness measurements is still difficult.

Korean Patent Publication No. 10-2022-0049967

The present invention can check whether the robust AI model is capable of responding to malware variants and the level of robustness. In particular, the robustness level is the same even when the number of features used for each AI model is different or the feature scale is different. We aim to provide a robustness measurement system capable of measuring . Accordingly, we would like to provide a system that calculates a robustness index that allows comparison of the degree of robustness even for different AI models.

In order to achieve the above object, the present invention provides a robustness measurement system for an AI model to respond to an adversarial attack, using the distance parameter between the original data and the attack sample generated due to the success of the adversarial attack to determine the original data. A preprocessing module that generates adversarial training samples that are close to but cause misclassification; a robustness measurement module that calculates the average value of the distance between the adversarial training sample and the original data as a robustness index value for evaluating the robustness level of the AI model; And an output module that outputs the robustness level measurement status for each malicious code group of a hostile attack using the value of the robustness index calculated by the robustness measurement module; including, outputting an evaluation index of the robustness of the AI model against hostile attacks. It's characteristic is to do something.

Preferably, the robustness measurement module is such that, as the index of the robustness index becomes more difficult to generate the adversarial training sample, noise increases, and the value of the warning index increases accordingly, and as the value of the warning index increases, the noise increases. It has the characteristic of making it difficult to generate adversarial training samples, and the robustness index, which is an evaluation standard for the robustness of the AI model, may be set based on the difficulty of generating the adversarial training samples.

Preferably, the robustness measurement module can calculate the robustness index based on [Relational Equation 1].

[Relational Expression 1]

Here, ARS refers to the Adversarial Robustness Score and refers to the robustness index, S refers to the set of data that successfully generated the adversarial training sample for each group, i refers to the feature index, and n refers to the number of features. , r means the range of the feature, x is the original data, x' is the adversarial training sample, and P means the predicted value of the AI model.

Preferably, when calculating the robustness index, the robustness measurement module excludes data that failed to generate the adversarial training sample, and then calculates an average value for the N numbers that succeeded in generating the adversarial training sample for each group of data. .

In addition, the present invention is to execute an AI model that has an input means for inputting data, a processing means for processing the input data, and an output means and learns general data on a smartphone, tablet, laptop, or computer stored in a medium. In an application for measuring the robustness of an AI model stored in a medium, a preprocessing step of generating an adversarial training sample that is close to the original but causes misclassification using a distance parameter between the original data and an attack sample generated due to a successful adversarial attack; A robustness measurement step of calculating the average value of the distance between the adversarial training sample and the original data as a value of the robustness index to evaluate the robustness level of the AI model; and an output step of outputting the robustness level measurement status for each malicious code group of the hostile attack using the value of the robustness index calculated in the robustness measurement step.

According to the present invention, the preprocessing module finds the optimal attack sample by minimizing the sum of the loss function responsible for minimal noise and the loss function that increases the attack success rate, and finds the distance between the original data and the attack sample generated by the success of the attack. Based on parameters, an adversarial training sample that is closest to the original and causes misclassification is constructed as a learning data set.

The present invention can evaluate the robustness level of an AI model with a robustness index value based on a learning data set of adversarial training samples. In particular, the robustness index according to the present invention considers the diversity of feature scales in calculating the magnitude of modulation and determines the number of features. Consider the range of change in modulation according to .

Accordingly, the present invention has the advantage of being able to measure the level of robustness that can be objectively applied to different AI models.

1 is a schematic diagram of the structure of a robustness measurement system according to an embodiment of the present invention.
Figure 2 is a schematic diagram of the processing process of the robustness measurement system according to an embodiment of the present invention.
Figure 3 shows the process of calculating the robustness index for evaluating the robustness level of an AI model according to an embodiment of the present invention.
Figure 4 shows the results of an experiment comparing the robustness index before adversarial learning and the robustness index after adversarial learning according to an embodiment of the present invention.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited or limited by the exemplary embodiments. The same reference numerals in each drawing indicate members that perform substantially the same function.

The purpose and effect of the present invention can be naturally understood or become clearer through the following description, and the purpose and effect of the present invention are not limited to the following description. Additionally, in describing the present invention, if it is determined that a detailed description of known techniques related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

1 is a schematic diagram of the structure of a robustness measurement system 1 according to an embodiment of the present invention. Figure 2 is a schematic diagram of the processing process of the robustness measurement system 1 according to an embodiment of the present invention.

Referring to FIGS. 1 and 2, the robustness measurement system 1 according to this embodiment may include a preprocessing module 10, a robustness measurement module 30, and an output module 50. The robustness measurement system 1 according to this embodiment can evaluate the degree of robustness by generating samples based on adversarial attacks in order to respond to adversarial attacks of an AI model that learns general data. This robustness assessment can be applied to enhance the robustness of AI models through retraining on adversarial samples.

An adversarial attack is an attack technique that causes a misclassification of the model and a decrease in reliability by giving a very small transformation to the original that is difficult to distinguish, leading the original to an area where an adversarial example can be created. am. Even if minimal modification (pertubation) is applied, adversarial samples are generated whose labels are changed during learning, so although the modification cannot be easily identified, classification errors are generated with a high probability, causing problems in service applications. .

In order to respond to hostile attacks, a fundamental solution to classification errors and reduced reliability is needed. Methods to respond to hostile attacks include gradient masking, which is a method of hiding gradients, and input pre-processing, which refines data to prevent input containing noise that could be an attack. Additionally, there is adversarial training, which generates samples of data that can be attacked based on an adversarial attack, adds them to model training, and then retrains them. Adversarial learning can be expected to improve the model's robustness level by retraining the model to respond to similar attacks using adversarial attack samples. In the case of gradient masking, it is impossible to respond to an attack if it is not a technique that attacks using gradients, and in the case of input pre-processing, it is difficult to be sure whether it is the best method because optimization cannot be confirmed. Therefore, when retraining a model through adversarial learning, it would be most desirable to use a method that can increase accuracy against known attacks and respond to new attacks.

In order to retrain a model through adversarial learning, it is necessary to create adversarial training samples. Hostile training samples must be created by performing an adversarial attack that inserts modified noise into the training data to cause misclassification and lower reliability. Additionally, an evaluation index of robustness is required as a feature element during re-learning. At this time, the robustness index, which indicates the level of robustness, must be set so that it can be applied to different AI models by considering the number of features in the training sample, the scale of the features, and the amount of change in prediction.

In the following, a new robustness measurement ARS index of the robustness measurement system 1 according to this embodiment will be presented to explain the detailed configuration of measuring the robustness of the AI model.

The preprocessing module 10 can use the distance parameter between the original data and the attack sample generated due to the success of the adversarial attack to generate an adversarial training sample that is close to the original but causes misclassification. The generation of adversarial training samples in the preprocessing module 10 is an approach that uses an attack technique to find the optimal adversarial training sample by minimizing the sum of the loss function responsible for the minimum noise and the loss function that increases the attack success rate.

In an embodiment, the preprocessing module 10 may generate the adversarial training sample based on [Equation 1].

[Equation 1]

Here, D means Distance metric for calculating the distance between the original and the attack sample, and f is the Objective function. It is defined as, Z(x) _i means the probability that x is calculated as label i.

The function according to [Equation 1] can be used to determine whether the adversarial training sample has normally produced the intended label. In particular, Z(x) _i refers to the probability that x is calculated with label i. If the adversarial training sample is calculated with label t(target) as intended, the output value of the corresponding Objective function becomes negative. Through this processing, the output label is changed and an adversarial training sample that is closest to the original can be created.

The preprocessing module 10 uses the distance parameter as a Linf variable, which is the value of the most changed feature due to an adversarial attack, an L0 variable indicating the number of changed features, or points to each of the original data and the attack sample. When expressed as , it can include an L2 variable that calculates the distance between points using the Euclidean algorithm. The preprocessing module 10 according to this embodiment has excellent performance for L0 and L2 variables. In particular, the preprocessing module 10 uses the distance metric of the L2 variable the most among the two distance-related parameters.

The preprocessing module 10 generates the adversarial training sample using parameters based on the L2 variable related to D (Distance metric) in [Equation 1], and the L2 variable can be calculated from [Equation 2] .

[Equation 2]

Here, x refers to the original data, x’ refers to the generated attack sample, and n is defined as the number of features the data has.

The robustness measurement module 30 may calculate the average value of the distance between the adversarial training sample and the original data as the value of the robustness index to evaluate the robustness level of the AI model.

The robustness measurement module 30 determines that as the robustness index index becomes more difficult to generate the adversarial training sample, the noise increases, and thus the value of the warning index increases. As the value of the warning index increases, the hostile training sample becomes larger. It has the characteristic of making it difficult to generate training samples, and the robustness index, which is an evaluation standard for the robustness of the AI model, can be set based on the difficulty of generating the adversarial training samples.

When calculating the robustness index, the robustness measurement module 30 may exclude data that failed to generate the adversarial training samples and then calculate an average value for the N numbers that succeeded in generating the adversarial training samples for each data group.

The robustness measurement module 30 can calculate the robustness index based on [Equation 3].

[Equation 3]

Here, ARS is the Adversarial Robustness Score, which refers to the robustness index, S refers to the set of data that successfully generated the adversarial training sample for each group, and s refers to the adversarial training sample belonging to each group. , d _s means the distance value between the original data and s.

Referring to [Equation 3], calculating the approximate average value for each group of the measured distance values can be seen as the basic concept of ARS, and to prevent ARS from being infinitely calculated due to failure in generating adversarial training samples, the overall Among N samples, the approximate average value of N/2 distance values with close distance values was used. The robustness measurement module 30 may determine that the larger the ARS, the more robust the model.

However, since the robustness index based on [Equation 3] simply measures the average distance from the original, it has the limitation of being unsuitable for use when the range of feature values is not the same as image data. In addition, there is a limitation in that it only measures the size of the modulation based on the distance from the original and does not take into account the change in probability accordingly.

Accordingly, in a more preferred embodiment, the robustness measurement module 30 may calculate the robustness index based on [Equation 4] below.

[Equation 4]

Here, MARS refers to the Model Adversarial Robustness Score and refers to the robustness index, S refers to the set of data that successfully generated the adversarial training sample for each group, i refers to the feature index, and n refers to the number of features. This means, r means the range of the feature, x is the original data, x' is the adversarial training sample, and P means the predicted value of the AI model.

The robustness measurement module 30 according to this embodiment can perform general-purpose robustness level measurement by calculating a robustness index based on [Equation 4]. The robustness index according to this embodiment is calculated by dividing the distance difference between the original adversarial attack sample by the scale size of each feature, and thus can reflect the diversity of feature scales. In addition, the robustness index is calculated by dividing the modulation size by the difference between the probability of the original data determined by the AI model and the probability of the sample by the adversarial train, so the change in probability can be taken into consideration. In addition, the robustness index is calculated by dividing the number of features used by the modulation size to consider the variation of modulation, so it is possible to secure versatility in various AI models by considering the variation of modulation according to the number of features.

Figure 3 shows the process of calculating the robustness index for evaluating the robustness level of an AI model according to an embodiment of the present invention. Referring to Figure 3, the ARS score, which is a robustness index, calculates and compares the ARS in an AI model based on a previously collected learning data set and an adversarial learning system built by retraining the AI model according to this embodiment. It represents the process.

In this embodiment, the AI model that learns general data may be a model that uses an artificial neural network model consisting of multiple hidden layers as a learning network, and preferably may be a DNN model. When an AI model is built based on a previously collected learning dataset, an adversarial training sample is generated by performing the preprocessing module 10 on the learning dataset. In Figure 3, it can be an Adversrial Sample. The generated Adversrial Sample is added to the existing learning dataset to retrain the AI model, and the Adversrial Sample is created once again for each group's data to compare the robustness level with before retraining. The learning logic of this adversarial learning system can be to improve the ARS value by comparing the previous ARS value with the ARS value after adversarial learning.

The output module 50 may output the robustness level measurement status for each malicious code group of a hostile attack using the value of the robustness index calculated by the robustness measurement module.

The robustness measurement system according to this embodiment can be built and utilized as a system that performs retraining of the AI model to improve the robustness index by adding the adversarial training sample to the AI model to the learning dataset.

Adversarial Learning Experiment: Enhancing and Evaluating Robustness

Most adversarial attack techniques are developed considering AI models whose input data is images. However, it may be inappropriate to apply to data where the range of each feature is not constant, such as images. Accordingly, in this experimental example, the feature values of the data were converted between 0 and 1 using the min-max normalization technique for malicious code featuring.

The method of measuring ARS, a robustness index, is as follows. The distance between the original data and the adversarial training sample is measured using the Euclidean distance measurement method based on L2 variables. Based on the measured distance values, the average of the distance values for each group is calculated. At this time, data that failed to generate adversarial training samples is excluded, and then the average is calculated for the N successful adversarial training samples for each group.

The 2019 KISA Datachallenge malware dataset was used as the dataset for the experiment. The composition of the dataset is as shown in [Table 1].

[Table 1]

The training dataset used a total of 29,130 data, including 17,562 malicious and 11,568 normal, and the test dataset used a total of 9,031 data, including 4,513 malicious and 4,518 normal. This experiment was conducted using only the top 5 most detected AVClasses among approximately 800 AVClasses identified in the data set.

The composition of AVClass is as shown in [Table 2].

[Table 2]

A total of 1,653 learning data were used: 517 autoit, 369 ramnit, 281 scar, 288 winactivator, and 198 zegost. Test data used was 171 autoit, 57 ramnit, 17 scar, 56 winactivator, and 37 zegost. A total of 388 pieces were used. The feature composition of the dataset was extracted by analyzing the PE (Portable Executable) structure of the malicious code. The PE header and PE section of the PE file contain information necessary for executing the file. Among these, 37 features were extracted from the information in the PE header and converted to 128 features using the Entropy of the PE section. A total of 165 features were extracted from the PE file through the above method, and these were used as features of the learning dataset.

The AI model that learns general data used a DNN model consisting of multiple hidden layers. As for the parameters used, Dropout 0.25 was applied to the first three layers to prevent overfitting, and batch_size = 100 and epochs = 93 were specified as learning parameters. Relu was used as the activation function, and the model was designed to calculate probability values for each label using softmax as the two outputs in the last layer. The learning results of the AI model are shown in [Table 3].

[Table 3]

Afterwards, an Adversarial Sample was created using the preprocessing module (10). The parameters of the hostile attack at the time of creation are as shown in [Table 4].

[Table 4]

In the case of adversarial samples created before adversarial training, a total of 200 adversarial samples were created, 40 for each of the three families. As a result, Adversarial samples were successfully created for 16 autoit, 23 ramnit, 15 scar, 39 winactivator, and 31 zegost data. As a result of measuring the ARS level, the values were autoit 0.3650, ramnit 0.7513, scar 4.4400, winactivator 0.6370, and zegost 1.0574. In the case of adversarial samples created after adversarial training, a total of 120 adversarial samples were created, 40 for each of the three families, as before adversarial training. As a result, adversarial samples were successfully created for 16 autoit, 22 ramnit, 32 scar, 39 winactivator, and 31 zegost data. As a result of measuring the ARS level, the values were autoit 0.5765, ramnit 2.3748, scar 0.9097, winactivator 0.6809, and zegost 0.4536.

Accuracy was calculated for five families of the model before adversarial training and the model after adversarial training. Accuracy calculation results are shown in [Table 5].

[Table 5]

In the model before adversarial training, autoit 1.0, ramnit 0.98245, scar 0.88235, zegost 1.0, and winativator 0.98214 were calculated. In the model prior to adversarial training, the average Accuracy for 5 AVClasses was 0.99817.

The model created after adversarial training yielded autoit 0.99415, ramnit 1.0, scar 0.88235, zegost 0.97297, and winativator 0.98214. After adversarial training, the average Accuracy for the five AVClasses in the model was 0.98520. Through the calculation results, it was confirmed that among the five AVClasses after adversarial training, the accuracy increased in the case of ramnit, the accuracy was maintained in scar and winativator, and the accuracy tended to decrease in autoit and zegost.

Figure 4 shows the results of an experiment comparing the robustness index before adversarial learning and the robustness index after adversarial learning according to an embodiment of the present invention.

Referring to Figure 4, ARS_1 refers to the ARS value before adversarial training, and ARS_2 refers to the ARS value after adversarial training. In the case of autoit, ramnit, and winactivator among the families, it was confirmed that the ARS level increased normally, but in the case of scar and zegost, it was confirmed that the ARS level actually decreased.

Although the present invention has been described in detail through representative embodiments above, those skilled in the art will understand that various modifications can be made to the above-described embodiments without departing from the scope of the present invention. will be. Therefore, the scope of rights of the present invention should not be limited to the described embodiments, but should be determined not only by the claims described later, but also by all changes or modified forms derived from the claims and the concept of equivalents.

1: Robustness measurement system
10: Preprocessing module
30: Robustness measurement module
50: output module

Claims (5)

In the robustness measurement system of AI models to respond to adversarial attacks,
A preprocessing module that generates an adversarial training sample that is close to the original but causes misclassification using the distance parameter between the original data and the attack sample generated due to the success of the adversarial attack;
a robustness measurement module that calculates the average value of the distance between the adversarial training sample and the original data as a robustness index value for evaluating the robustness level of the AI model; and
Including an output module that outputs the robustness level measurement status for each malicious code group of a hostile attack using the value of the robustness index calculated by the robustness measurement module.
An AI model robustness measurement system characterized by outputting an evaluation index of the robustness of the AI model against adversarial attacks.
According to claim 1,
The robustness measurement module is,
As the robustness index index makes it more difficult to generate the adversarial training samples, the noise increases, and thus the value of the warning index increases. As the value of the robustness index increases, the generation of the adversarial training samples becomes more difficult. have characteristics,
A robustness measurement system for an AI model, characterized in that the robustness index, which is an evaluation standard for the robustness of the AI model, is set based on the difficulty of generating the adversarial training sample.
According to claim 2,
The robustness measurement module is,
A robustness measurement system for an AI model, characterized in that the robustness index is calculated based on [Relational Equation 1].
[Relational Expression 1]

Here, MARS refers to the Model Adversarial Robustness Score and refers to the robustness index, S refers to the set of data that successfully generated the adversarial training sample for each group, i refers to the feature index, and n refers to the number of features. This means, r means the range of the feature, x is the original data, x' is the adversarial training sample, and P means the predicted value of the AI model.
According to claim 1,
The robustness measurement module is,
A robustness measurement system for an AI model, characterized in that when calculating the robustness index, data that fails to generate the adversarial training samples is excluded, and then the average value is calculated for the N numbers that succeeded in generating the adversarial training samples for each group of data. .
A smartphone, tablet, laptop, or computer that has an input means for inputting data, a processing means for processing the input data, and an output means and has an AI model for learning general data stored in the medium,
A preprocessing step of generating an adversarial training sample that is close to the original but causes misclassification using the distance parameter between the original data and the attack sample generated due to the success of the adversarial attack;
A robustness measurement step of calculating the average value of the distance between the adversarial training sample and the original data as a value of the robustness index to evaluate the robustness level of the AI model; and
An output step of outputting the robustness level measurement status for each malicious code group of a hostile attack using the value of the robustness index calculated in the robustness measurement step; A robustness measurement application of an AI model stored in a medium to execute.