CN116541280B

CN116541280B - Fuzzy test case generation method based on neural network

Info

Publication number: CN116541280B
Application number: CN202310502110.7A
Authority: CN
Inventors: 李斌斌; 马锐; 董建; 徐洋
Original assignee: China Electronics Standardization Institute
Current assignee: China Electronics Standardization Institute
Priority date: 2023-05-06
Filing date: 2023-05-06
Publication date: 2023-12-26
Anticipated expiration: 2043-05-06
Also published as: CN116541280A

Abstract

The invention relates to the technical field of software testing, in particular to a fuzzy test case generation method based on a neural network, wherein an overall framework comprises data preprocessing, path construction, path compression, model training and case generation, a data processing stage comprises program basic information construction, data processing and generation of a non-execution path, model construction and use of a model structure in the field of machine translation, the non-execution path is constructed by a method of splicing executed paths, and compressed path information is carried out by subtracting fixed values and converting character string representation modes, so that model training tasks are reduced.

Description

Fuzzy test case generation method based on neural network

Technical Field

The invention relates to the technical field of software testing, in particular to a fuzzy test case generation method based on a neural network.

Background

The fuzzy test is a method for finding out software vulnerabilities by providing unexpected inputs to a target system or software and monitoring abnormal results, and is widely used in the field of vulnerability discovery at home and abroad because the fuzzy test technology can convert a large number of manual tests into highly automated tests, which are between black box tests and white box tests.

The fuzzy test includes test case, basic block, path information and program branch.

The test case is input content of a fuzzy test object, such as a file of characters, pictures, audio, video and the like. The generation strategy of the use cases determines the testing efficiency of the fuzzy test, and better use cases can cover more paths of the detected program, so that more program vulnerabilities are detected.

The fuzzy test case generation method can be divided into two types, one is a case generation method based on generation and the other is a case generation method based on variation. The generation-based method is to construct use cases according to specifications by modeling the specific program to be tested or behaviors. These model-based or grammar-based fuzzy test case generation methods may generate inputs that easily pass an integrity check (e.g., checksum) or grammar check. The mutation-based approach is to generate new use cases by modifying well-formatted use cases. These mutation-based methods can effectively test programs with compact and unstructured data formats (e.g., images and video).

The basic block is one or a combination of a plurality of instructions in a program, when the program is executed to a certain basic block, the program starts to execute from the first instruction of the basic block, and then sequentially executes other instructions in the block until the last instruction is executed. The path in the program is composed of one or more basic blocks.

The path information is a piece of path information including one or more basic blocks and the execution order of the basic blocks.

Program branches are edges between two basic blocks in a set symbol branch representing a program control flow diagram, the edges are called a program branch in a fuzzy test, the program branches generally comprise conditional jump branches, unconditional direct jump branches, indirect branches, near return branches, long jump branches and the like, and the branches are mainly used for judging coverage conditions of program execution in the existing fuzzy test based on variation.

While ambiguity testing can effectively discover vulnerabilities in many scenarios, it still has the following three limitations:

although the existing mutation-based fuzzy test can test unstructured use cases efficiently through mutation, the existing mutation-based fuzzy test cannot support file format grammar well, so that the existing mutation-based fuzzy test has a certain defect in testing some target binary programs for processing high-structured use cases.

Although fuzzy tests performed based on symbols can generate program branches which can trigger constraint conditions in a program in a constraint solving mode, partial constraint conditions are too complex to solve.

Most of the existing fuzzy test technologies do not perform relevance mining on the one-to-one correspondence between program paths and use cases, so that the fuzzy test based on variation is limited to the test of program part paths, the path coverage rate of the test is lower, and the mining of potential vulnerabilities is further affected.

Therefore, in order to solve the problems, the application provides a fuzzy test case generation method based on a neural network, which utilizes a classical neural network model in the field of machine translation to realize the generation of the case and can efficiently assist in carrying out fuzzy test on a binary program.

Disclosure of Invention

The invention aims to overcome the defects of the prior art, provides a fuzzy test case generation method based on a neural network, realizes the generation of the case by using a classical neural network model in the field of machine translation, and can efficiently assist in carrying out fuzzy test on a binary program.

In order to achieve the above purpose, the present invention provides a fuzzy test case generating method based on a neural network, comprising the following steps:

s1, an overall framework comprises data preprocessing, path construction, path compression, model training and use case generation;

the data preprocessing is an effective case queue and an executed path set which are obtained after the fuzzy test is executed, and the output is a transcoded case and a program executed branch;

the path construction is an executed branch obtained in the target program and data preprocessing stage, and the output is a spliced and constructed unexecuted path;

the path compression is a use case generated by the fuzzy test, a program path executed by the fuzzy test and an unexecuted path constructed by the path, and the output is a training data pair constructed by the executed path after the compression processing and the use case and the unexecuted path after the compression processing;

model training is to use cases and paths, wherein the paths are used as input of a model, the cases are used as output of the model, and the output is a model obtained through training;

the use case generation is a compressed unexecuted path, and the output is a to-be-verified use case queue which can be used for assisting the fuzzy test;

s2, a data processing stage comprises program basic information construction, data processing and generation of a non-execution path;

the program basic information construction specifically comprises the following steps:

s2-1, acquiring control flow information of a program by using an existing binary program dynamic analysis tool, wherein the control flow information comprises an initialization part before entering a program main body, the initialization part is necessary to be executed for the program capable of being normally executed, and in order to reduce space overhead, basic blocks before main functions are filtered;

s2-2, setting a dictionary for storing executed branches of a program, wherein keys of the dictionary are current basic block ids, and values of the dictionary are a subsequent basic block set of the current basic block; the other setting is a set for storing the basic blocks that the program has executed;

the data processing is specifically as follows:

sequentially traversing the executed path of the program and the effective example generated by the fuzzy test to process data;

s2-3, sequentially constructing executed branches and basic blocks of a program, constructing a follow-up unexecuted path, and simultaneously storing a predecessor sequence and a successor sequence of a current basic block in the program path;

s2-4, sequentially converting the use cases and the program paths into data which is convenient to process by the neural network model, and selecting to convert the use case byte character strings into integer character strings and convert the program addresses from hexadecimal character strings into decimal character strings;

the generation of the unexecuted path is specifically:

traversing the basic blocks executed by the program in sequence, and searching unexecuted branches in all branches of the program; s2-5, acquiring a subsequent basic block set of the current basic block;

s2-6, judging whether the subsequent basic block is in the executed basic block or not;

s2-7, if S2-6 is not in the executed basic block, ignoring the subsequent basic block;

s2-8, if S2-6 is in the executed basic block, judging whether the branch is executed or not;

s2-9, if S2-8 is not executed, indicating that a new branch is found, splicing a predecessor sequence and a successor sequence of the new branch in a new branch set stored by the branch, and generating a new unexecuted path;

s3, setting up and using a model, and along with a seq2seq model structure in the field of machine translation, the seq2seq model structure comprises an embedded layer, two hidden layers and a projection layer, and an attention mechanism is added in order to prevent the model from dispersing attention on all source input sequences when decoding a long path;

s3-1, an input sequence is a compressed program path, and a single word represents the relative address of a basic block; the output sequence is a use case sequence transcoded according to single byte, and single word represents that the byte transcoded is an integer of 0-255;

s3-2, obtaining a use case generation model through training by using the path use case data set constructed in the data processing stage;

s3-3, after the neural network model is built and trained, inputting a non-execution path of the built program into the model for use case generation;

s3-31, in order to enable the fuzzy test to call the model generation use case, a storage space for storing the model generation use case can be created;

s3-32, traversing program unexecuted paths in sequence, and inputting the currently selected unexecuted paths into the trained model to generate new use cases;

s3-33, judging whether the generated use cases are generated or not because the use cases with identical sequences exist in the use cases generated by the model;

if the use case generated in S3-33 is not generated, the use case is added to the to-be-verified use case queue, otherwise, the use case is directly discarded, and the same use case is prevented from being stored.

Compared with the prior art, the invention has the following beneficial effects:

the generation of the high-structured use case is improved, and the fuzzy test efficiency of the binary program for processing the high-structured use case is improved; a new unexecuted path is constructed by utilizing a method for splicing executed branches, so that the problem that partial paths cannot be obtained through constraint solving is solved; the method creatively proposes the program path and the use cases as two special languages for representing the program behaviors, utilizes a neural network commonly used in the translation field to fit the one-to-one correspondence between the program path and the use cases, then utilizes the unexecuted path to generate new use cases, constructs a dictionary or compresses path information in a mode of subtracting a fixed value from the dictionary, lightens model training tasks, and assists in fuzzy test to improve test effects.

Drawings

FIG. 1 is a schematic diagram of a fuzzy test case generating system framework based on a neural network.

FIG. 2 is a schematic diagram of the generation and construction of an unexecuted path according to the present invention.

FIG. 3 is a schematic diagram of a neural network model according to the present invention.

Detailed Description

The invention will now be further described with reference to the accompanying drawings.

Referring to fig. 1 to 3, the invention provides a fuzzy test case generation method based on a neural network, aiming at paths and cases obtained by a fuzzy test execution target program, aiming at utilizing a classical neural network model (such as seq2 seq) in the field of machine translation to excavate the relationship between the program paths and the cases, and finally realizing the generation of new cases.

The system mainly comprises five stages (shown in figure 1) which are respectively data preprocessing, path construction, path compression, model training and use case generation, wherein the input of the data preprocessing stage is an effective use case queue and an executed path set which are obtained after the execution of the fuzzy test, and the output is a use case and a program executed branch after the transcoding.

The input of the path construction stage is the executed branch obtained by the target program and the data preprocessing stage, and the output is the spliced and constructed unexecuted path.

The input of the path compression stage is a use case generated by the fuzzy test, a program path executed by the fuzzy test and an unexecuted path constructed by the path, and the output is a pair of the executed path after compression processing and training data constructed by the use case and the unexecuted path after compression processing.

The input of the model training stage is a path and use case pair (the path is taken as the input of the model, the use case is taken as the output of the model), and the output is the model obtained through training.

The input of the use case generation stage is a compressed unexecuted path, and the output is a to-be-verified use case queue which can be used for assisting in fuzzy test.

Specifically, the main tasks of the five phases include:

stage 1: after the fuzzy test is executed for a period of time, the use case generation part based on the neural network starts to execute, firstly loads the program path and the use case of the execution from the fuzzy test, secondly converts the acquired use case into an integer sequence of 0-255 in bytes, and then deletes all the public predecessor sequences of the executed paths, because the basic blocks are positioned in front of main functions, and finally converts the executed paths into a program branch form to be provided for the phase 2 for path construction.

Stage 2: path construction, after the executed branches of the program are loaded, the path construction module firstly loads the target binary program by using a dynamic analysis tool to generate control flow information of the program, secondly generates all branches of the program by using the control flow information, then constructs unexecuted new branches from the executed basic blocks based on all branches of the program, and finally constructs new paths in a mode of splicing the executed paths based on the new branches.

Stage 3: the same prefixes exist in the basic block addresses tracked by the program and generated by dynamic analysis in the path compression and fuzzy test process, such as 0x40 in the figure, and the same prefixes do not cause different use cases, but rather aggravate the training task of the neural network model, so that the compression can be performed by constructing a dictionary or subtracting a fixed value, such as main function addresses, head addresses of program code segments and the like.

Stage 4: during model training, firstly, word dictionaries of input and output sequences are constructed, although program paths and use cases are expressed in integer forms, the value ranges are not necessarily the same, so that two word dictionaries are required to be constructed, secondly, the input sequences and the output sequences are simultaneously input into a coding layer, iterative updating of intermediate state vectors is started, and finally, models obtained through training in each round and the best models in the process are stored in a hard disk for use case generation.

Stage 5: and (3) generating use cases, namely sequentially inputting the unexecuted paths constructed in the stage 2 into a trained model to generate corresponding use cases, and if the newly generated use cases are different from the use cases generated in the past, saving the newly generated use cases into a to-be-verified use case queue, otherwise, directly discarding the newly generated use cases.

Of the 5 stages described above, the most critical is the phase 2 path construction. If all paths of the program are constructed by directly using the control flow information generated by the dynamic analysis tool, the generated paths contain a large number of basic blocks which are not in training data, so that a 'no midwifery phenomenon' occurs when the model is used; if all branches of a program and all paths of an executed basic block building program are directly relied upon, a large number of unreachable paths are generated.

2) Data processing

In the fuzzy test case generation scheme based on the neural network, the key content is the construction of a program unexecuted path, and in the construction process, the generation of an unreachable path is avoided, and the neural network model cannot be 'without midwifery', namely, a basic block address which is not stored in a training dictionary cannot be utilized to generate a new case, so that the data processing comprises three parts, namely program basic information construction, data preprocessing and unexecuted path generation.

(1) Program basic information construction

First, the existing binary program dynamic analysis tool is used to obtain the control flow information of the program, and since the control flow information includes the initialization part before entering the main body of the program, but these parts must be executed for the program that can be executed normally, the basic blocks before main function need to be filtered in order to reduce the space overhead. Secondly, setting a dictionary for storing executed branches of a program, wherein keys of the dictionary are current basic block ids, and dictionary values are a subsequent basic block set of the current basic block; the further setting is a set for storing the basic blocks that the program has executed.

(2) Data processing

In the process, the data is taken out by traversing the executed path of the program and the effective example generated by the fuzzy test in sequence. Firstly, sequentially constructing executed branches and basic blocks of a program for constructing a subsequent unexecuted path, and simultaneously saving a predecessor sequence and a successor sequence of a current basic block in the program path, wherein in the path p1, the predecessor sequence of the basic block b3 is (b 1) and the successor sequence is (b 5, b8, b 9), and the predecessor and successor sequences are used for splicing the subsequent unexecuted path as shown in fig. 2 (a); then, the use cases and the program paths are sequentially converted into data which is convenient to process by the neural network model, the use case byte character strings can be selectively converted into integer character strings, and the program addresses are converted into decimal character strings from hexadecimal character strings.

(3) Generation of unexecuted paths

In this process, the basic blocks executed by the program are traversed in turn, and unexecuted branches are found among all branches of the program. Specifically, firstly, acquiring a subsequent basic block set of the current basic block b1, then judging whether the subsequent basic block b2 is in the executed basic block, if not, ignoring the subsequent basic block, so that the neural network model can be prevented from being made to be 'without midwifery'; if in the executed basic block, judging whether the branch (b 1, b 2) is executed or not, if not, indicating that a new branch is found, then, in the new branch set saved in the step (b 1, b 2), splicing the predecessor sequence of the new branch b1 and the successor sequence of the new branch b2, and generating a new unexecuted path.

Specifically, assuming that the program control flow diagram shown in fig. 2 (b) exists, all basic blocks in the diagram have been executed, but branches (b 3, b 4) and paths (b 1, b3, b4, b7, b 9) shown by dotted lines are not executed, according to the foregoing method, a predecessor sequence of b3 and a successor sequence of b4 are spliced to form a new unexecuted path, and a specific splicing manner is shown in fig. 2 (c). In particular, since the manner of directly splicing all executed branches and new branches generates a large number of unreachable paths, the new paths are constructed in a manner of splicing predecessor and successor sequences, so that the generation of program unreachable paths can be reduced.

3) Model construction and use

For the design of neural network models classical seq2seq model structure in the field of machine translation can be followed, as shown in fig. 3, the seq2seq model can be designed as 1 embedding layer, 2 hiding layers and 1 projection layer, while in order to avoid that the model will be distracting on all source input sequences when decoding longer paths, an attention mechanism needs to be added. In particular, the input sequence in the figure is a compressed program path, and a single word indicates the relative address of the basic block; the output sequence is a use case sequence transcoded in single byte, and single word represents that the byte transcoded is an integer from 0 to 255. The < path, use case > dataset constructed by the data processing stage is then used to derive the use case generation model by training.

The Seq2Seq used above is an option in a specific code implementation, and other neural network models that meet the requirements can be used.

After the neural network model is built and trained, a built program non-execution path is required to be input into the model for use case generation. First, in order to enable fuzzy testing to invoke model-generated use cases, a memory space may be created for storing model-generated use cases. Secondly, traversing program unexecuted paths in sequence, and inputting the currently selected unexecuted path p into a trained model to generate a new use case; in addition, because the use cases generated by the model may have the same sequence, whether the generated use cases are generated or not needs to be judged, if not, the generated use cases are added to the to-be-verified use case queue, otherwise, the generated use cases are directly discarded, and the same use cases are prevented from being stored.

The above is only a preferred embodiment of the present invention, only for helping to understand the method and the core idea of the present application, and the protection scope of the present invention is not limited to the above examples, and all technical solutions belonging to the concept of the present invention belong to the protection scope of the present invention. It should be noted that modifications and adaptations to the present invention may occur to one skilled in the art without departing from the principles of the present invention and are intended to be within the scope of the present invention.

The method solves the problems that file format grammar is not supported, constraint conditions are too complex and relevance mining is not carried out on the program path and the use case in the prior art, builds a new unexecuted path by improving high structuring of the use case, fits relevance between the program path and the use case by using a neural network, and generates a new use case by learning a corresponding relation between the program path and the use case through a model, and builds the new path as model input in a mode of splicing real paths when in use, thereby greatly improving the authenticity of a sample and integrally and greatly improving the discovery rate and detection efficiency of software loopholes.

Claims

1. The fuzzy test case generation method based on the neural network is characterized by comprising the following steps of:

the data preprocessing takes an effective case queue and an executed path set obtained after the fuzzy test is executed as input, and the output is a transcoded case and a program executed branch;

the path construction takes an executed branch obtained in a target program and data preprocessing stage as input, and the output is a spliced and constructed unexecuted path;

the path compression is to take a use case generated by the fuzzy test, a program path executed by the fuzzy test and an unexecuted path constructed by the path as inputs, and output the unexecuted path as a data set of the executed path and the path use case constructed by the use case after compression and an unexecuted path set after compression;

the model training takes a path use case pair as input, and the output is a model obtained by training;

the use case generation takes the compressed unexecuted path as input, and the output is a to-be-verified use case queue for auxiliary fuzzy test;

s2-1, acquiring control flow information of a program by using an existing binary program dynamic analysis tool, wherein the control flow information comprises an initialization part before entering a program main body, the initialization part is necessary to execute a program capable of being normally executed, and filtering basic blocks before a main function in order to reduce space overhead;

the data processing specifically comprises the following steps:

sequentially traversing the executed path of the program and the effective example generated by the fuzzy test to process data; s2-3, sequentially constructing executed branches and basic blocks of a program, constructing a follow-up unexecuted path, and simultaneously storing a predecessor sequence and a successor sequence of a current basic block in the program path;

the generation of the unexecuted path specifically includes:

s2-7, if the S2-6 is not in the executed basic block, ignoring the subsequent basic block;

s2-8, if the S2-6 is in the executed basic block, judging whether the branch is executed or not;

s2-9, if the S2-8 is not executed, indicating that a new branch is found, storing the branch into a new branch set, and splicing a predecessor sequence and a successor sequence of the new branch to generate a new unexecuted path;

s3, setting up and using a model, and along with a seq2seq model structure in the field of machine translation, the seq2seq model structure comprises an embedded layer, two hidden layers and a projection layer, and an attention mechanism is added in order to avoid that the model distributes attention to all source input sequences when decoding a long path;

s3-2, using the path use case data set constructed in the path compression stage, and obtaining a use case generation model through training;

s3-31, in order to enable the fuzzy test to call the use case generated by the model, creating a storage space for storing the use case generated by the model;

if the use cases generated in the S3-33 are not generated, the use cases are added to a to-be-verified use case queue, otherwise, the use cases are directly discarded, and the same use cases are prevented from being stored.