JP2018085040A - Program test device, program test program and program test method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a program test device, program test program and program test method that improves the efficiency in a test using as testing data a file to be read implicitly by a program.SOLUTION: A program test device comprises a learning object monitoring part 102 that monitors a plurality of files to be read with a test object of program and a file priority determining part 105 that gives priority to the plurality of files in accordance with the reading order and size into a program. A file fuzzing execution part 111 gives files to a file fuzzing processing part 120 in the higher order of priority and controls the execution of file fuzzing.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a program test apparatus, a program test program, and a program test method.

  Fuzzing is known as a test method for detecting problems such as program vulnerabilities, by giving unpredictable input data to a program to intentionally generate an exception and checking the behavior of the exception (Patent Literature) 1, see Non-patent Document 1). Fuzzing is often used as a black box security test method that tries a large amount of test data (for example, a long character string or a large integer) effective for vulnerability detection. Long strings are prone to buffer overflow exceptions when copying strings beyond the internal buffer. A large integer (for example, the maximum value “0xFFFFFFFF” of a 32-bit unsigned integer) is likely to cause an exception of an integer overflow exceeding the maximum value of the integer in the arithmetic processing.

  File fuzzing, in which a test file is passed to a program that reads a file and a parser is tested, can be applied to any program that reads a file. Therefore, it is often used for security evaluation of software. As the test file, an abnormal file obtained by partially mutating a normal file may be used.

JP 2014-26579 A

"Fuzzing", Wikipedia, https://en.wikipedia.org/wiki/%E3%83%95%E3%82%A1%E3%82%B8%E3%83%B3%E3%82% B0, last updated: April 2, 2016

  Conventionally, in a test by file fuzzing, a file that is explicitly read into a test target program is used as test data, and a file that is implicitly read is not used. An explicitly read file is a file to be opened that is designated by a user or the like. An implicitly read file is a setting file or a data file that is read independently after the program is started.

  The reason why implicitly read files were not used is that there may be many files that are read implicitly, and it takes a long time to load all the files into the program under test. It is not. For example, a test using one file often takes 1 to 2 days in a general computer environment, and if there are 30 target files, it takes 1 to 2 months, which is not realistic. .

  Therefore, in one aspect, an object is to improve the efficiency in a test using a file that is implicitly read by a program.

  In one form, a monitoring unit that monitors a plurality of files read by a test target program, and a grant unit that gives priority to the plurality of files according to the order and size read by the program are provided.

  It is possible to improve the efficiency in the test using the file read implicitly by the program as test data.

It is a figure which shows the flow of the program test concerning one Embodiment. It is a figure which shows the function structural example of the program test apparatus concerning one Embodiment. It is a figure which shows the example of test setting information. It is a figure which shows the hardware structural example of a program test apparatus. It is a flowchart which shows the process example of embodiment. It is a figure which shows the example of the monitoring result of Process Monitor. It is a figure which shows the example of a time series file list. It is a figure which shows the example of a priority order file list. It is a figure which shows the example of file fuzzing. It is a figure which shows the example of a test result list | wrist.

  Hereinafter, preferred embodiments of the present invention will be described.

<Configuration>
FIG. 1 is a diagram showing a flow of a program test according to an embodiment. In FIG. 1, the program test apparatus 1 includes a configuration for a learning phase and a configuration for an execution phase. The learning phase is a phase in which a file to be read implicitly by a program to be tested is prioritized. The execution phase is a phase in which file fuzzing is executed in descending order of priority for a predetermined number of files prioritized in the learning phase. By limiting file fuzzing targets to a predetermined number of high-priority files and setting a limit on the overall test time, tests can be executed efficiently in real time without degrading the quality of the test. It becomes possible.

  In FIG. 1, the program test apparatus 1 includes a learning target monitoring unit 102 and a file priority determination unit 105 in addition to the target program P as a main configuration of the learning phase. The learning target monitoring unit 102 monitors a file to be read after the target program P is activated, and records the path, timing (time from activation to reading), and file size in the time-series file list. It has a function to do. The file priority determination unit 105 has a function of assigning a priority from the viewpoint of “ease of attack” and recording it in a priority order file list for a plurality of files in the time-series file list, focusing on timing and size. Have.

  In addition, the program test apparatus 1 includes a file fuzzing execution unit 111 and a file fuzzing processing unit 120 as main components of the execution phase. The file fuzzing execution unit 111 refers to the priority order file list obtained in the learning phase, and gives the files (shown as file #x) in descending order of priority to the file fuzzing processing unit 120 to control the execution of file fuzzing. It has a function. The file fuzzing processing unit 120 has a function of generating a test file based on a normal file, executing file fuzzing, and outputting a test result. When the file fuzzing is completed for one file, the file fuzzing execution unit 111 proceeds to processing of the next file. Further, when the time limit is reached or the processing is completed for a predetermined number of files, the file fuzzing execution unit 111 outputs a test result list and ends the processing.

  FIG. 2 shows an example of the functional configuration of the embodiment showing the program test apparatus 1 more specifically.

  In FIG. 2, the program test apparatus 1 stores test setting information ST set by the operator before the start of the test, and the test setting information ST is referred to by each unit described later. FIG. 3 is a diagram showing an example of the test setting information ST, which includes a test target program start command, a learning phase monitoring time, an execution phase maximum monitoring time, and an execution phase limit time. The test target program start command is a command character string for starting the test target program P. In the illustrated example, “/ c” at the end is a parameter for designating an operation mode or the like. The learning phase monitoring time is the time (milliseconds) for monitoring the target program P in the learning phase. The execution phase maximum monitoring time is the maximum time (milliseconds) for monitoring the target program P in one file fuzzing process in the execution phase. The execution phase limit time is the maximum time (milliseconds) that may be taken for the entire processing of the execution phase.

  Returning to FIG. 2, the program test apparatus 1 includes a learning target activation unit 101, a learning target monitoring unit 102, a time-series file list storage unit 103, a learning target end unit 104, and a file priority determination unit 105 as a learning phase configuration. A priority order file list storage unit 106.

  The learning target starting unit 101 has a function of starting the target program P in accordance with the test target program start command of the test setting information ST. The learning target monitoring unit 102 has a function of monitoring a file reading event generated by the target program execution body PP, which is a process of the target program P, and recording the read file information in the time-series file list storage unit 103 in time series. Have.

  The learning target end unit 104 has a function of ending the execution of the target program P when the learning phase monitoring time of the test setting information ST has elapsed since the start of the target program P. The file priority determination unit 105 gives priority to a plurality of read files recorded in the time-series file list storage unit 103 from the viewpoint of “ease of attack”, focusing on timing and size. The file list storage unit 106 has a function of recording.

  On the other hand, the program test apparatus 1 includes a file fuzzing execution unit 111, a test file generation unit 112, a test target activation unit 113, a test target monitoring unit 114, a test target end unit 115, and a test result list storage unit 116 as the configuration of the execution phase. And. The test file generation unit 112, the test target activation unit 113, the test target monitoring unit 114, the test target end unit 115, and the test result list storage unit 116 correspond to the file fuzzing processing unit 120 in FIG.

  In FIG. 2, the file fuzzing execution unit 111 has a function of starting file fuzzing and controlling it to the end after the processing of the file priority determination unit 105 ends and the learning phase ends. That is, the file fuzzing execution unit 111 refers to the priority order file list of the priority order file list storage unit 106 and executes file fuzzing in order from the file with the highest priority. Then, the test result R in the middle is recorded in the test result list storage unit 116, and finally the test result list RL is output. The test file generation unit 112 has a function of generating a test file TF based on a normal file F. For example, the test file TF is generated by partially mutating the file F.

  The test target starting unit 113 has a function of starting the target program P by a test target program start command of the test setting information ST. The test target monitoring unit 114 has a function of monitoring the behavior (error occurrence, etc.) of the target program execution body PP ′, which is the process of the target program P. The test target termination unit 115 has a function of terminating the target program P and outputting the test result R when the execution phase maximum monitoring time of the test setting information ST elapses after the target program P is started or when execution is stopped due to an error. doing.

  FIG. 4 is a diagram illustrating a hardware configuration example of the program test apparatus 1. In FIG. 4, a program test apparatus 1 includes a CPU (Central Processing Unit) 1002, a ROM (Read Only Memory) 1003, a RAM (Random Access Memory) 1004, and an NVRAM (Non-Volatile Random Access Memory) connected to a system bus 1001. It has 1005. The program test apparatus 1 includes an I / F (Interface) 1006, an I / O (Input / Output Device) 1007, an HDD (Hard Disk Drive) / SSD (Solid State Drive) 1008 connected to the I / F 1006. , NIC (Network Interface Card) 1009. Further, the program test apparatus 1 includes a monitor 1010, a keyboard 1011, a mouse 1012 and the like connected to the I / O 1007. A CD / DVD (Compact Disk / Digital Versatile Disk) drive or the like can be connected to the I / O 1007.

  The functions of the program test apparatus 1 described with reference to FIGS. 1 and 2 are realized by a predetermined program being executed by the CPU 1002. The program may be acquired via a recording medium, may be acquired via a network, or may be embedded in a ROM.

<Operation>
FIG. 5 is a flowchart showing a processing example of the embodiment shown in FIG. In FIG. 5, when the program test apparatus 1 starts processing in response to an operator processing start instruction or the like, first, it starts a learning phase. The learning target activation unit 101 of the program test apparatus 1 activates the learning target program P according to a normal procedure in accordance with the test target program activation command of the test setting information ST (step S101).

  The learning target monitoring unit 102 monitors the behavior of the target program P (more precisely, the target program execution body PP which is the corresponding process) to grasp the file access, and specifies the file read by the target program P ( Step S102). The file read by a specific program (process) can be identified by existing technology. For example, Sysinternals Process Monitor (https://technet.microsoft.com/en-us/sysinternals/processmonitor.aspx published by Microsoft) ) Function can be used. FIG. 6 is a diagram showing an example of the monitoring result of Process Monitor. It is possible to monitor file access of all programs currently being executed, event occurrence time, reading source program, access type (file reading, file writing). Etc.), and the path of the read file can be obtained. From these events, the list of all files read by the target program P can be acquired by identifying the files corresponding to the target program and having the access type read.

  Returning to FIG. 5, the learning target monitoring unit 102 stores the acquired information about the read file in the time-series file list storage unit 103. An example of the time-series file list to be saved is shown in FIG. In FIG. 7, a file path is a path of a read file. The timing is the elapsed time (milliseconds) from the start of the target program. The size is the size (in bytes) of the read file. The size of the read file can be obtained by actually accessing the file properties when the path is specified.

  Returning to FIG. 5, after the learning phase monitoring time specified by the test setting information ST has elapsed, the learning target end unit 104 stops monitoring the target program P by the learning target monitoring unit 102 and ends the target program P. (Step S103).

  Next, the file priority determination unit 105 reads the time-series file list stored in the time-series file list storage unit 103 and assigns priorities (rearranged in order of priority) according to “ease of attack” (step S104). .

  Here, “ease of attack” is determined by the read timing and the file size. As for the read timing, priority is given to a shorter elapsed time from the start of the program to the start of file reading. The reason is that files that are read at an early stage during the execution of the program are considered to be important files for the program and are more likely to be damaged in the presence of vulnerabilities, making them vulnerable to attack. is there. A larger file size is given priority. The reason is that if the file size is large, the complexity of the file structure also increases, and there is a high possibility that there is a vulnerability in the file read processing of the program, so it is easy to be attacked. Reading timing and file size have priority over reading timing. Therefore, first, sorting is performed with priority on the shorter reading timing, and when the reading timing is the same, sorting is performed with priority on the larger file size.

  The file priority determination unit 105 stores the sorted priority file list in the priority file list storage unit 106. An example of the priority order file list is shown in FIG. 8, and the file paths of the read files are associated with the priorities arranged in descending order. The learning phase is completed by the processing so far, and the process proceeds to the execution phase.

  Returning to FIG. 5, the execution phase is entered, and the file fuzzing execution unit 111 starts executing file fuzzing (step S105). The file fuzzing execution unit 111 passes the execution phase time limit of the test setting information ST (Yes in step S106), and completes processing of a predetermined number of files registered in the priority order file list storage unit 106 in advance (Yes in step S107). Is the end condition.

  If the end condition is not met (No in step S107), the file fuzzing execution unit 111 reads the priority order file list from the priority order file list storage unit 106, and loads the file F from the file path according to the priority order described therein. get.

  The test file generation unit 112 generates a test file TF based on the file F (step S108). The generated test file TF is replaced with the original file F. However, the file F is temporarily stored in an appropriate location in order to restore the original file after the test.

  FIG. 9 shows an example of generating a test file from a normal file. For example, the test file # 1 is generated by mutating 4 bytes from the head of the header to “0xFFFFFFFF”, and the position to be further mutated is 1 byte. The test file # 2 is generated by shifting and is continued. In the case of an image file, when it is read by a program such as an image viewer, the image is displayed normally in a normal file, but normal display is not performed in a test file that is partially mutated, and an error due to an offset error Occurrence may occur.

  Returning to FIG. 5, the test target activation unit 113 activates the target program P in accordance with the test target program activation command of the test setting information ST, and the test target monitoring unit 114 monitors the behavior of the target program P (step S109). In this case, the behavior of the target program P is monitored by acquiring information from a log output from the OS (Operating System) of the program test apparatus 1 or by obtaining a system call to the OS that acquires the state of the target program P. Can be done.

  Thereafter, the test target monitoring unit 114 ends the target program P after the execution phase maximum monitoring time of the test setting information ST has elapsed (step S110). When the test for one target program P is completed, the replaced test file TF is replaced with a normal file F.

  The file fuzzing execution unit 111 records the test result R obtained by monitoring by the test target monitoring unit 114 in the test result list of the test result list storage unit 116 (step S111).

  Thereafter, when the test for one target program P is not completed, the file fuzzing execution unit 111 repeats the test by replacing the test file TF. When the test for one target program P is completed, the same processing is repeated for the next rank file in the priority order file list from the priority order file list storage unit 106.

  The file fuzzing execution unit 111 completes the process at any time point whether the execution phase limit time has elapsed (Yes in step S106) or all the files have been tested (Yes in step S107). Then, the test result list collected so far is output as a result (step S112). FIG. 10 shows an example of the test result list, and a link to the test result is associated with the file path of the read file. The test result after the link stores the file path of the read file (can be omitted), the test date and time, the number of tests, the number of exception occurrences, and the like.

<Summary>
As described above, according to the present embodiment, a file that is easily attacked is preferentially used for testing, so even if all files are not read into the test target program, more security can be achieved within a realistic time. Quality can be improved.

  In the above, it demonstrated by preferred embodiment. While specific embodiments have been illustrated and described herein, it will be apparent that various modifications and changes may be made thereto without departing from the broad spirit and scope as defined in the claims. . That is, it should not be construed as being limited by the details of the specific examples and the accompanying drawings.

Regarding the above description, the following items are further disclosed.
(Appendix 1)
A monitoring unit that monitors multiple files read by the program under test;
An assigning unit that gives priority to the plurality of files according to the order and size read by the program;
A program test apparatus comprising:
(Appendix 2)
The assigning unit determines the priority by prioritizing the order of reading into the program rather than the size of the file;
The program test apparatus according to appendix 1, wherein:
(Appendix 3)
Read into the program under test in order from the highest priority file,
The test ends when a preset time has elapsed or when a predetermined number of files have been processed,
The program test apparatus according to appendix 1 or 2, characterized by the above.
(Appendix 4)
In the test, a plurality of abnormal files generated by partially mutating normal files for each file are read into the program.
The program test apparatus according to Supplementary Note 3, wherein
(Appendix 5)
Monitor multiple files read by the program under test,
Giving priority to the plurality of files according to the order and size read by the program;
A program test program for causing a computer to execute processing.
(Appendix 6)
Determining the priority by prioritizing the order of loading into the program over size;
The program test program according to appendix 5, characterized by:
(Appendix 7)
Read into the program under test in order from the highest priority file,
The test ends when a preset time has elapsed or when a predetermined number of files have been processed,
The program test program according to appendix 5 or 6, characterized by the above.
(Appendix 8)
In the test, a plurality of abnormal files generated by partially mutating normal files for each file are read into the program.
The program test program according to appendix 7, characterized by:
(Appendix 9)
Run the program under test,
Monitor multiple files read by the program,
Giving priority to the plurality of files according to the order and size read by the program;
A program test method, wherein a computer executes a process.
(Appendix 10)
Determining the priority by prioritizing the order of loading into the program over size;
The program test method according to appendix 9, wherein:
(Appendix 11)
Read the program under test in order from the highest priority file,
End the test when a preset time has elapsed or when processing of a predetermined number of files is completed,
The program test method according to appendix 9 or 10, characterized by the above.
(Appendix 12)
Causing the program to read a plurality of abnormal files generated by partially mutating normal files for each file,
The program test method as set forth in appendix 11, wherein:

  The learning target monitoring unit 102 is an example of a “monitoring unit”. The file priority determination unit 105 is an example of a “grant unit”.

DESCRIPTION OF SYMBOLS 1 Program test apparatus 101 Learning object starting part 102 Learning object monitoring part 103 Time series file list preservation | save part 104 Learning object end part 105 File priority determination part 106 Priority order file list preservation | save part 111 File fuzzing execution part 112 Test file generation part 113 Test target activation unit 114 Test target monitoring unit 115 Test target end unit 116 Test result list storage unit 120 File fuzzing processing unit ST Test setting information P Target program PP, PP 'Target program execution body F file TF test file R Test result RL test Results list

Claims (6)

A monitoring unit that monitors multiple files read by the program under test;
An assigning unit that gives priority to the plurality of files according to the order and size read by the program;
A program test apparatus comprising: The assigning unit determines the priority by prioritizing the order of reading into the program rather than the size of the file;
The program test apparatus according to claim 1. Read into the program under test in order from the highest priority file,
The test ends when a preset time has elapsed or when a predetermined number of files have been processed,
The program test apparatus according to claim 1, wherein the program test apparatus is a program test apparatus. In the test, a plurality of abnormal files generated by partially mutating normal files for each file are read into the program.
The program test apparatus according to claim 3. Monitor multiple files read by the program under test,
Giving priority to the plurality of files according to the order and size read by the program;
A program test program for causing a computer to execute processing. Run the program under test,
Monitor multiple files read by the program,
Giving priority to the plurality of files according to the order and size read by the program;
A program test method, wherein a computer executes a process.