JP7249306B2 - Evaluation device, evaluation method and evaluation program - Google Patents

Description

The present invention relates to an apparatus, method, and program for evaluating the security of program obfuscation techniques.

In general, software may contain information that should be kept secret to users, such as valuable algorithms and content encryption keys. On the other hand, the current situation is that many reverse engineering techniques for analyzing software have been developed. Therefore, if software is analyzed by these techniques, there is a threat that confidential information may be obtained by an unauthorized person.
To deal with such threats, there is a technology called obfuscation that makes it difficult to analyze software while maintaining software specifications (see, for example, Patent Document 1).

JP 2017-58994 A

However, we were unable to quantitatively assess the threat of confidential information being analyzed from obfuscated programs.

An object of the present invention is to provide an evaluation device, an evaluation method, and an evaluation program capable of quantitatively evaluating the security of an obfuscation technique.

The evaluation device according to the present invention selects an evaluation program from a plurality of programs before obfuscation, assigns a predetermined label, and uses the evaluation program and other programs with labels different from the predetermined label. As training data, a learning unit that generates a trained model, and one of the evaluation program and the other program that has undergone evaluation target obfuscation processing is used as the trained model. and an evaluation unit that calculates an evaluation value of the security of the obfuscation process based on the percentage of correct answers of the plurality of determination results.

The evaluation unit may calculate the security evaluation value of the obfuscation process based on the percentage of correct answers in the determination result for each of the evaluation program and the other program.

The evaluation unit may adjust the percentage of correct answers by weighting the determination result according to the probability of correct answers when the label answer is randomly selected.

The learning unit may repeatedly select the evaluation program from the plurality of programs, and the determination unit may acquire the determination result from each of the plurality of trained models.

The evaluation unit may statistically process the percentage of correct answers in the determination result by changing the evaluation program and acquiring the percentage a plurality of times.

The determination unit may acquire the determination results when the obfuscation processing is performed on any of the programs a plurality of times with different parameters.

The evaluation unit may statistically process the percentage of correct answers of the determination result by changing parameters of the obfuscation process and acquiring the correct answer rate a plurality of times.

The input to the training data and the trained model may be a program converted to a two-dimensional image.

The training data and the input to the trained model may be a program converted into vectors.

An evaluation method according to the present invention selects an evaluation program from a plurality of pre-obfuscated programs, assigns a predetermined label to the evaluation program, and assigns another program with a label different from the predetermined label. As training data, a learning step of generating a trained model, and one of the evaluation program and the other program that has undergone evaluation target obfuscation processing is used as the trained model. , the computer executes a determination step of obtaining determination results, and an evaluation step of calculating an evaluation value of the security of the obfuscation processing based on the percentage of correct answers of the plurality of determination results.

An evaluation program according to the present invention is for causing a computer to function as the evaluation device.

According to the present invention, the security of an obfuscation technique can be quantitatively evaluated.

It is a figure which shows the functional structure of the evaluation apparatus in embodiment. It is a figure which illustrates the processing procedure of the evaluation method of safety in embodiment.

An example of an embodiment of the present invention will be described below.
FIG. 1 is a diagram showing the functional configuration of an evaluation device 1 according to this embodiment.
The evaluation device 1 is an information processing device (computer) such as a server or a personal computer, and includes a control unit 10 and a storage unit 20, input/output devices for various data, communication devices, and the like.

The control unit 10 is a part that controls the entire evaluation apparatus 1, and implements each function in this embodiment by appropriately reading and executing various programs stored in the storage unit 20. FIG. The control unit 10 may be a CPU.

The storage unit 20 is a storage area for various programs, various data, and the like for causing the hardware group to function as the evaluation device 1, and may be a ROM, RAM, flash memory, hard disk (HDD), or the like. Specifically, the storage unit 20 stores a program (evaluation program) for causing the control unit 10 to execute each function of the present embodiment, a learning model, and the like.

The control unit 10 includes a learning unit 11 , a determination unit 12 and an evaluation unit 13 .
The control unit 10 evaluates the security of the obfuscation method by using the determination result using the learning model by these functional units.

The learning unit 11 selects one evaluation program from a plurality of pre-obfuscation programs, assigns a predetermined label to it, and stores this evaluation program and other programs assigned labels different from the predetermined label as training data. is learned by a DNN (Deep Neural Network) or the like to generate a trained model.
It should be noted that the assigned labels may be of two types, a label t1 indicating the evaluation program and a label t2 indicating others, or different labels (t2, t3, . . . ) may be assigned to the other programs. may That is, the outputs of the trained model are two or more types including t1.

Here, the input to the training data and the trained model may be, for example, the bit string of the program converted into two-dimensional image data. A finished model is generated.
Alternatively, the training data and input to the trained model may be, for example, a program converted into a vector. In this case, the trained model is generated by learning a clustering method such as DBScan. .

In order to collect various evaluation results for safety evaluation, the learning unit 11 repeatedly selects a new evaluation program from a plurality of programs, and determines a trained model based on each of the selected evaluation programs. can be provided to

The determination unit 12 inputs, into the generated trained model, one or more of the evaluation programs and the other programs after performing the evaluation target obfuscation process, and Get label judgment result.

At this time, the determination unit 12 may acquire determination results from a plurality of trained models generated for each evaluation program.
Further, the determination unit 12 may acquire determination results when the obfuscation processing is performed on each program a plurality of times with different parameters.

The evaluation unit 13 calculates the strength of the obfuscation process, that is, the safety evaluation value, based on the correct answer rate of the plurality of determination results.
Here, the determination result is a correct answer when the obfuscated evaluation program is determined as t1, and when other obfuscated programs are determined as other than t1 (for example, t2, t3, . . . ). It can be. Alternatively, when different labels such as t2, t3, .

Also, the percentage of correct answers is calculated, for example, as follows.
(1) The evaluation unit 13 aggregates all the evaluation results for each of the evaluation program and the other programs by the determination unit 12 and calculates the percentage of correct answers.
At this time, the evaluation unit 13 may adjust the percentage of correct answers by weighting the determination results according to the probability of correct answers when the label answers are randomly selected. For example, when a plurality of other programs (label t2) are prepared, the judgment result of classifying the evaluation program (t1) as t1 is weighted more than the judgment result of classifying the other program (t2) as t2. you can

(2) The evaluation unit 13 obtains the correct answer rate of the determination results for each selected evaluation program, and acquires the correct answer rate multiple times by changing the evaluation program, thereby obtaining statistical values such as average values or maximum values. calculate.

(3) The evaluation unit 13 obtains the correct answer rate from the determination results obtained multiple times by changing the evaluation program. Calculate statistical values such as

FIG. 2 is a diagram illustrating the processing procedure of the safety evaluation method according to this embodiment.
In this example, there are two types of labels given to the training data, ie, among the multiple programs, the label for the evaluation program is t1, and the common label for the other programs is t2. Also, as a learning model, a DNN that receives image data is adopted.

The learning unit 11 converts these programs into image data and inputs them to the DNN for learning. The trained model thus generated classifies the image data of the input program into t1 or t2 and outputs it as a determination result.

The determination unit 12 converts the evaluation program obfuscated using the obfuscation method, which is the object of evaluation, into image data and inputs it to the trained model, and obtains t1 or t2 as a determination result. Then, the case where the determination result is t1 is counted as a correct answer.
At this time, the determination unit 12 may similarly convert other programs into image data and input them to the trained model, and may count the case where the determination result is t2 as a correct answer.

Here, when the obfuscation method to be evaluated generates different obfuscated programs each time from the same program by changing parameters (for example, random numbers), obfuscation of the evaluation program and other programs is repeatedly performed. The determination unit 12 similarly obtains a determination result when each obfuscated program is imaged and input to the trained model.

Then, the evaluation unit 13 calculates an evaluation value such as the reciprocal of the correct answer rate, based on the assumption that the lower the correct answer rate of the collected determination results is, the safer the obfuscation method is.

At this time, the evaluation unit 13 may calculate an evaluation value based on a statistical value such as an average value or a maximum value after replacing the evaluation program and calculating the percentage of correct answers in the same manner as described above.
Alternatively, after calculating the percentage of correct answers for each parameter of the obfuscation method, the evaluation unit 13 may calculate an evaluation value based on a statistical value such as an average value or maximum value.

According to the present embodiment, the evaluation device 1 generates a trained model using an evaluation program to which a predetermined label is assigned and another program to which a different label is assigned as training data. Input the obfuscated program to the trained model and obtain the judgment result. Then, the evaluation device 1 calculates an evaluation value of the safety of the obfuscation processing to be evaluated based on the correct answer rate of the plurality of determination results.
Therefore, the evaluation apparatus 1 determines that an obfuscated program is dangerous when AI (Artificial Intelligence) classifies it as the same program as before obfuscation. The evaluation can be automated and the security of the obfuscation method can be quantitatively evaluated.

The evaluation device 1 easily collects the judgment results and evaluates the safety by calculating the evaluation value of the safety of the obfuscation processing based on the correct answer rate of the judgment results for each of the evaluation program and other programs. Appropriate evaluation value can be calculated.
At this time, the evaluation device 1 adjusts the percentage of correct answers by weighting the judgment results according to the probability of correct answers when randomly selecting the label answer, thereby reflecting the importance of each judgment result, Safety can be evaluated appropriately.

The evaluation device 1 may repeatedly select an evaluation program from a plurality of programs, and acquire determination results from a plurality of trained models generated each time a selection is made.
As a result, the evaluation device 1 can appropriately evaluate the security of the obfuscation technique from multiple perspectives using a plurality of evaluation programs.
In addition, the evaluation device 1 can calculate a more appropriate evaluation value by statistically processing the percentage of correct answers for each selection of the evaluation program.

The evaluation device 1 may acquire the determination results when the obfuscation processing is performed multiple times with different parameters.
As a result, the evaluation device 1 can appropriately evaluate the security of the obfuscation technique from multiple perspectives using a plurality of obfuscation results.
In addition, the evaluation device 1 can calculate a more appropriate evaluation value by statistically processing the percentage of correct answers for each parameter.

The evaluation device 1 uses a program bit string converted to two-dimensional image data as training data and an input to a trained model, so that it can be adapted to a classification method such as a DNN and can appropriately implement an evaluation method.
In addition, the evaluation apparatus 1 uses a program converted into a vector as training data and an input to a trained model, thereby making it suitable for a clustering method such as DBScan and realizing an appropriate evaluation method.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments. Moreover, the effects described in the above-described embodiments are merely enumerations of the most suitable effects produced by the present invention, and the effects of the present invention are not limited to those described in the embodiments.

The evaluation method by the evaluation device 1 is realized by software. When it is implemented by software, a program constituting this software is installed in an information processing device (computer). Further, these programs may be recorded on removable media such as CD-ROMs and distributed to users, or may be distributed by being downloaded to users' computers via a network. Furthermore, these programs may be provided to the user's computer as a web service through the network without being downloaded.

1 evaluation device 10 control unit 11 learning unit 12 determination unit 13 evaluation unit 20 storage unit

Claims (11)

An evaluation program is selected from a plurality of programs before obfuscation and given a predetermined label, and the evaluation program and another program given a label different from the predetermined label are used as training data, and a trained model is generated. a learning unit to generate;
a determination unit for inputting a program after obfuscation processing of an evaluation target to one of the evaluation program and the other program into the trained model and acquiring a determination result;
and an evaluation unit that calculates an evaluation value of security of the obfuscation process based on correct answer rates of the plurality of determination results. 2. The evaluation device according to claim 1, wherein the evaluation unit calculates the safety evaluation value of the obfuscation processing based on the percentage of correct answers in the determination result for each of the evaluation program and the other program. 3. The evaluation apparatus according to claim 2, wherein the evaluation unit adjusts the percentage of correct answers by weighting the determination results according to the probability of correct answers when the label answers are randomly selected. The learning unit repeatedly selects the evaluation program from the plurality of programs,
The evaluation apparatus according to any one of claims 1 to 3, wherein the determination unit acquires the determination result from each of the plurality of trained models. 5. The evaluation apparatus according to claim 4, wherein the evaluation unit performs statistical processing on the percentage of correct answers in the determination result by changing the evaluation program and acquiring the percentage a plurality of times. 6. The determination unit according to any one of claims 1 to 5, wherein the determination unit acquires the determination results when the obfuscation processing is performed on the program a plurality of times with different parameters. Evaluation device. 7. The evaluation apparatus according to claim 6, wherein the evaluation unit statistically processes the percentage of correct answers in the determination result by changing parameters of the obfuscation processing and obtaining the accuracy a plurality of times. 8. The evaluation apparatus according to any one of claims 1 to 7, wherein the input to the training data and the learned model is obtained by converting a program into a two-dimensional image. 8. The evaluation apparatus according to any one of claims 1 to 7, wherein the training data and the input to the learned model are obtained by converting a program into vectors. An evaluation program is selected from a plurality of programs before obfuscation and given a predetermined label, and the evaluation program and another program given a label different from the predetermined label are used as training data, and a trained model is generated. a learning step to generate;
a determination step of inputting into the trained model a program after obfuscation processing to be evaluated for one of the evaluation program and the other program, and acquiring a determination result;
an evaluation step of calculating an evaluation value of the security of the obfuscation process based on the percentage of correct answers of the plurality of determination results; and an evaluation method executed by a computer. An evaluation program for causing a computer to function as the evaluation device according to any one of claims 1 to 9.

Images