DE102020208543A1 - Method and device for performing a dynamic security check of software programs - Google Patents

Abstract

The invention relates to a computer-implemented method for testing a program code (p) using fuzzing, with stateful fuzzing processes for the program code (p) to be tested each having a varying input sequence (i1...in) made up of one or more input elements ( i) are carried out, with each stateful fuzzing process carrying out a program execution with the successively supplied input elements (i) of the input sequence (i1...in) and, after execution with the input element (i) supplied in each case, storing a program flow observation (obs) which indicates whether a program execution error has occurred or which program paths of the program code (p) have been run through, the supply of the input elements (i) of the input sequence (i1...in) to the executed program code without resetting a program state of the program code (p). a predetermined program state occurs.

Description

technical field

The invention relates to methods for testing a program code by so-called fuzzing testing. The present invention relates in particular to measures for increasing the efficiency of fuzzing test methods.

Technical background

A conventional method for detecting errors in a program code executed on a computer system, which can be implemented in software or hardware, consists in examining this for program execution errors or system crashes using a fuzzing test method. The so-called fuzzing input elements for the computer system are generated, a program code to be tested is executed with the input elements and the function of the algorithm of the program code is monitored. Monitoring the execution of the program code includes determining whether the execution of the algorithm leads to a program execution error, i.e. e.g. a system crash or an unexpected execution stop.

While the program is being executed, the internal behavior of the program flow is monitored, in particular with regard to the program flow paths executed by the program code. This process is repeated with different input elements in order to obtain information about the behavior of the program code for a wide range of input elements. The aim of program code monitoring is to generate the input elements in such a way that the greatest possible overlap of the program flow paths is achieved, i.e. the largest possible number of program flow paths is run through during the repeated variation of the input elements.

If an error or unexpected behavior occurs during the execution of a program code, this is recognized by the fuzzing tool and appropriate information is signaled which specifies which fuzzing input elements led to the error. Examples of such fuzzing tools are American Fuzzing Lop, libFuzz or honggfuzz.

Disclosure of Invention

According to the invention, a method for carrying out a fuzzing test method for a software algorithm according to claim 1 and by the device and the fuzzing test system according to the independent claims are provided.

Further developments are specified in the dependent claims.

According to a first aspect, a computer-implemented method for testing a program code using fuzzing, wherein stateful fuzzing processes for the program code to be tested are each performed with a varying input sequence of one or more input elements, each stateful fuzzing process having a program execution the successively supplied input elements of the input sequence and, after execution, stores a program flow observation with the respectively supplied input element, which indicates whether a program execution error has occurred or which program paths of the program code have been run through, the supply of the input elements of the input sequence to the executed program code without resetting a Program status of the program code takes place on a predetermined program status.

While conventional fuzzing tools are limited to specifying a single input element for execution starting from a defined initial state, protocol fuzzing makes it possible to test a software algorithm of a program code by specifying input sequences. Starting from a defined initial state, a series of input elements for the execution of the software program are successively generated and a next input element is transferred to the computer system executing the software when a corresponding state has been reached by the previous input element. However, the input sequence has hitherto been provided independently of a monitoring of the program flow, since it is not possible to take into account the internal behavior of the program flow in the case of input sequences. In contrast, conventional fuzzing tools can use the Pro test program code only through a single input element, but the input element allows the program flow to be taken into account.

A program code is understood herein in a broad sense and includes conventional software executable on a computer system as well as algorithms implemented in hardware in computer systems or other data processing devices.

The above method makes it possible to provide an input sequence for fuzzing that takes into account an internal behavior of the software algorithm to be checked. The input sequences are composed of input elements, each of which can consist of a character or a character string made up of several characters, a code sequence or the like. In particular, input sequences leading to predetermined internal states of the program code are systematically searched for and then further analysis is resumed starting from that particular state. Here, the term state designates a series of executed program paths, also referred to below as program flow observations. As a result, the coverage of the state space of a program code to be tested, i. H. the program flow paths run through during testing are improved.

In contrast to conventional protocol fuzzing, the internal state of a program code is observed over several input elements of an input sequence according to the above method and the analysis is then carried out according to the internal behavior of the program code, i. H. the coverage of the program flow path, optimized.

It can be provided that the input sequence is generated before each execution of a stateful fuzzing process, wherein the generation of the input sequence includes replacing or inserting an input element generated in particular by a fuzzing tool according to a heuristic depending on the results of previous program executions, which depends on one or more recent executions of the program code.

An auxiliary input sequence can thus be determined as a function of a determined input sequence set, with the individual input being inserted at a predetermined position in the auxiliary input sequence in order to obtain a next input sequence to be tested.

Furthermore, one or more input sequences for the stateful fuzzing processes can be generated by dividing input elements of the input sequence set.

Provision can be made for the input sequence to be generated before each execution of a fuzzing process, with the generation of the input sequence comprising replacing or inserting an input element which is generated in particular by a fuzzing tool according to a heuristic depending on the results of previous program executions one or more last executions of the program code.

With conventional fuzzing tools, the fuzzing tool is either given a set of input elements at the beginning or it randomly generates this set of input elements. While executing the program to be tested with one of these input elements, the fuzzing tool observes which program paths are executed and notes these outputs. Then the fuzzing tool modifies this input element by inserting, omitting or changing parts of the input element. With repeated execution with a new input element, the fuzzing tool recognizes which parts of the input element lead to which changes in the executed program path. With this knowledge, after executing many different input elements and observing the changes in program execution, the fuzzing tool can specifically search for input elements that lead to execution paths that have not been observed before.

Alternatively, the input sequence can be generated before each execution of one of the fuzzing processes as a function of an auxiliary input sequence, with the auxiliary input sequence being generated as a function of a difference between program flow observations with two input sequences that are generated from an input sequence set, with one of the Input sequences has a number of input elements that is part of the other input sequences.

According to a further alternative, the input sequence can be generated before each execution of one of the fuzzing processes as a function of an auxiliary input sequence, the auxiliary input sequence is generated depending on a program state that the program code has after execution with the input sequence.

Furthermore, the input sequence can be generated before each execution of one of the fuzzing processes as a function of an auxiliary input sequence, the auxiliary input sequence being determined as a function of the result of a static program analysis as a function of a specified program state, with different program states being specified for the fuzzing processes will.

According to a further aspect, a device for testing a program code using fuzzing is provided, the device being designed to carry out stateful fuzzing processes for the program code to be tested, each with a varying input sequence of one or more input elements, each stateful fuzzing process carries out a program execution with the successively supplied input elements of the input sequence and, after execution with the respectively supplied input element, stores a program flow observation which indicates whether a program execution error has occurred or which program paths of the program code have been run through, the supply of the input elements of the input sequence to the executed program code takes place without resetting a program state of the program code to a predetermined program state.

character list

• 1 ein Flussdiagramm und ein Pseudocode zur Veranschaulichung eines Verfahrens zum Durchführen eines Fuzzing-Testverfahrens,
• 2 ein Flussdiagramm zur Veranschaulichung eines Unterprogramms für den Ablaufplan der 1;
• 3 ein Pseudocode zum Erzeugen einer zu testenden Eingabesequenz;
• 4 ein Pseudocode zum Erzeugen einer zu testenden Eingabesequenz;
• 5a ein Pseudocode zum Erzeugen einer Hilfs-Eingabesequenz für den Pseudocode der 4;
• 5b ein Pseudocode zum Erzeugen einer Hilfs-Eingabesequenz für den Pseudocode der 4;
• 5c ein Pseudocode zum Erzeugen einer Hilfs-Eingabesequenz für den Pseudocode der 4;
• 5d ein Pseudocode zum Erzeugen einer Hilfs-Eingabesequenz für den Pseudocode der 4;

Embodiments are explained in more detail below with reference to the accompanying drawings. Show it:

• 1 a flowchart and pseudocode to illustrate a method for performing a fuzzing test method,
• 2 a flowchart illustrating a subprogram for the flowchart 1 ;
• 3 pseudocode for generating an input sequence to be tested;
• 4 pseudocode for generating an input sequence to be tested;
• 5a a pseudocode for generating an auxiliary input sequence for the pseudocode of 4 ;
• 5b a pseudocode for generating an auxiliary input sequence for the pseudocode of 4 ;
• 5c a pseudocode for generating an auxiliary input sequence for the pseudocode of 4 ;
• 5d a pseudocode for generating an auxiliary input sequence for the pseudocode of 4 ;

Description of Embodiments

In the following, procedures are presented using flowcharts and pseudo-codes.

1 shows a flow chart and a corresponding pseudo-code for performing a flow of a fuzzing test method with a stateful program code (stateful program) p for a predetermined set of input sequences (set of interesting) and a length n (int: integer) of the input sequence. The input sequence set contains an input sequence made up of a number of input elements and associated program flow observations. The input sequence can be a code, a character, or a string.

The program code p contains a number of program paths that are run through depending on the given input element when the program code p is executed.

In step S1, a set of errors (set of crashes) is first initialized. This indicates which input sequence(s) led to a program execution error. Initially, the error set is empty (line 1).

Then, in step S2, an input sequence i1,...,in is generated or selected from the predefined set of input sequences (set of interesting) using the create function. The selection is based on the input sequences of the set of input sequences together with the associated program flow observations. The program flow observations can include the program flow paths run through by the respective input sequence.

The generated input sequence i1,...,in is used in step S3 to check/test the specified program code. This is done by calling a StatefulExecution subroutine (line 4) described below in conjunction with the flow chart and pseudocode of the 2 is described. The result r is either an error statement with the input sequence i1,...,ij, which led to a program execution error, or an observation set obs, in which the input sequence i1,...in and the program paths run through are stored, if none Program execution error occurred.

In step S4 it is checked (line 5) whether the execution with the selected input sequence has led to a program execution error. If this is the case (alternative: yes), in step S5 the part of the input sequence up to the occurrence of the program execution error is added to the error set (line 6). Otherwise (alternative: no), in step S6, if the observation set obs proves to be interesting, the corresponding input sequence including the program paths of the input sequence set (set of interesting) is added.

Then, in step S7, it is checked whether a termination criterion for the fuzzing process has been reached. If this is the case (alternative: yes), the method is ended and the error quantity set of crashes is output, otherwise (alternative: no) the system jumps back to step S2.

The termination criterion may include a corresponding user input, a coverage criterion corresponding to the number of program flow paths traversed, a progress criterion dependent on the amount of errors and/or the amount of input sequences, or other criteria such as expiry of a predetermined test time and the like.

The above test method enables the testing of input sequences without resetting the state of the program code to an initial state before each use of an input element.

In 2 A flow chart and pseudo code for the sub-program of step S3 is shown. In step S11, an observation set obs is first initialized as an empty set (line 1).

In step S12, the program code p is executed with an input element i from the input sequence, beginning with the first input element i1 of the input sequence, and the program flow o is observed (lines 3-4). The program flow contains the observations, in particular the program paths reached or traversed by the relevant input element.

Then, in step S13, after the execution of the program code p with the respective input element, the result of the program flow o is examined to determine whether it indicates a program execution error (line 6). If the program flow o indicates a program execution error crash (alternative: yes), then in step S14 this program execution error is _{returned to the main program with the sequence of input elements i 1 ,} . . . , i _j executed up to that point (line 7).

Otherwise (alternative: no), in step S15 the program flow o is added to the observation set obs (line 9).

In step S16, after processing all the input elements of the input sequence i _1, ...., i, the execution of the program code _n p terminated (line 12) and the program state to an initial state.

In step S17, the observation set obs with the observation set including the added program flows o is returned to the main program (line 13).

The program runs o of the observation set obs can contain internal information about the program run during testing, such as the program paths that have been executed for each of the input elements, with the supplied input elements associated features or other features recorded during the program run. In the event of a program execution error crash, ie if program execution hangs or program execution is terminated unexpectedly, the input sequence that led to the program execution error is added to the error set adds. As long as the program execution does not show any program execution errors, the input sequences and the corresponding program flows o are added to the observation set obs.

The input sequence set set of interesting is then supplemented by the observation set obs in step S6.

The generation of the input sequences from the input sequence set set of interesting of step S2 can be carried out according to one of the methods described below.

As in the create subroutine of the pseudocode of the 3 shown, according to a first alternative, an input element input is generated by a conventional fuzzing tool, which generates exactly one input element, in a fuzzing tool-internal process getlnputFromFuzzer. Traditional fuzzing tools may include LibFuzz, AFL-Fuzz, or Honggfuzz, for example. A conventional fuzzing tool generates, in a manner known per se, one or more input elements input depending on previously used input elements and program executions within the framework of the execution of the fuzzing subroutine StatefulExecution as well as a predetermined set of input sequences which has been considered interesting. Conventional software fuzzing tools usually use the program flow observations associated with the respective input sequences, in particular the executed program flow paths that have been run through during the previous executions of the fuzzing subroutine StatefulExecution, as additional criteria. The series of input elements input is then divided into an input sequence i _1, ..., in with a given length n by the function split.

                  l=length(input)/n

                  for (k=1; k<=n; k++) do 

                  ik=substring(input, (k-1)*|,k*|-1)

wobei substring(a,b,c) für eine Zeichenkette a die Teilzeichenkette zurückgibt, bei beim Zeichen an Stelle b beginnt und bei dem Zeichen an Stelle c endet.For example, the split(input) function can split a string input, such as an input element, into n slices according to a specification, like this:

 l=length(input)/n

                  for (k=1; k<=n; k++) do 

                  ik=substring(input, (k-1)*|,k*|-1)

where substring(a,b,c) for a string a returns the substring starting at the character at position b and ending at the character at position c.

Traditional software fuzzing tools typically use observations from previous program executions, as well as from previous input elements and an input sequence set. _{An auxiliary input sequences l 1} ... l _n determined in a higher-level process are modified with a The input sequences generated for this purpose are subdivided (line 2) so that they are divided into input sequences i _1, ..., i _{n of the} same or different length be divided depending on, for example, the value of the first byte of the input element or the like.

As in the create sub-routine of the pseudocode of the 4 as an alternative to the subroutine of 3 shown, according to a second alternative, an auxiliary input sequence I ₁ ,...,I _f-1 ,I _f + ₁ ...,I _{n is} predefined and fixed from n-1 input elements, and a further input element whose position in of the input sequence is specified by a predetermined position index f is generated by a conventional fuzzer in a fuzzer-internal process getlnputFromFuzzer. For this purpose, the conventional fuzzer generates, in a manner known per se, one or more input elements depending on previously used input elements and program executions within the framework of the execution of the fuzzing subroutine StatefulExecution as well as a predetermined set of input sequences that has been considered interesting. A combination of the predefined input sequence and the generated input element is then used as the input sequence to be tested.

The pseudocode of the 4 provides that first an input element is generated by the fuzzing tool depending on the observations of the previous program executions (line 1). Then all input elements not indexed by the given position index f are set to a specified auxiliary input element I1...In an auxiliary input sequence (line 4). The input element i _{f of} the input sequence of input elements is set to the input element generated by the fuzzing tool (line 6). Finally, the input elements i _1, ... , generated in this way are used in the input sequence as the result of the pseudocode of the 4 for the input sequence of step S3 (line 8).

In several variants, the pseudocodes in the 5a-5d are shown, the execution of the stateful fuzzing is embedded in an overall process of a fuzzing test method. This allows the auxiliary input sequences I ₁ ,...,I _n to be used in the pseudocode of the 4 to be generated.

The variant of 5a includes a metric-based stateful fuzzing, ie a fuzzing for different system states.

First, a stateful fuzzing process sf (sf: process handler) with n=1 (line 1) is started. The executions of the fuzzing process sf are added to an execution set executions (line 2). This is used to record the executed fuzzing processes StatefulFuzz in order to then be able to read out information from these fuzzing processes. Specifically, the following information is required for each execution: input sequence set of the respective fuzzing process, number n of input elements in this fuzzing process.

The method then waits a specific time (line 3) until the fuzzing has executed one of the executions of the program code (line 3). The processes sf can be parallel processes. It is awaited until the fuzzing process has had enough time to carry out a first execution of the program code p to be tested and to fill the input sequence set. As long as the maximum number of stateful fuzzing processes StatefulFuzz (maxexecs) executed in parallel has not been reached (line 4), new stateful fuzzing processes StatefulFuzz can be started in parallel to those already executed (lines 5 to 18).

To start new fuzzing executions, an execution sf1 is first selected from the execution set executions (line 5). The executions can be selected iteratively one after the other using a heuristic, e.g. depending on the progress of the fuzzing process StatefulFuzz, randomly or according to other methods.

From the selected executions executions an execution sf1 is selected (line 5) and in line 6 an auxiliary input sequence set Is is determined from the selected execution sf1. The selected execution sf1 is an execution of the stateful fuzzing process, which in turn performs many executions of the program code p to be tested, thus building up an input sequence set. Again, the selection from the input sequence set of sf1 can be done in different ways: randomly; according to the newest entry in the input sequence set, according to heuristics e.g. regarding the observation set obs of entries in the input sequence set and the like (line 6). Then the number n of input elements that have been executed for the selected execution sf1 (line 7) is determined.

Two input sequences are then selected from the input sequence set is and _{stored in i 1,} ...,in and i _n+1 ,...,i _2n (line 8).

Then the program p is executed (exec) with the consecutive input elements i _1, ..., i _2n as an input sequence, ie the connection of the two input sequences i _1, ..., in and i _n+1 , ..., i _2n , and the corresponding program flow observation of the execution is stored in a first program flow observation set obs1 (line 9). Thereafter, the program execution observation of the execution of the program with i _1, ..., is stored in a second program flow observation set obs2 (line 10) and the program execution observation of the execution of the program code p with i _n-1 , ..., i _2n to the second program flow observation set obs2 (line 11) appended.

If the program flow observation sets obs1 and obs2 differ from each other, ie the program flow observation set obs1 of the auxiliary input sequences i _1, ..., i _2n from the concatenation of the program flow observation sets of the separately determined two input sequences i _1, ..., in and i _n+1, ..., i _2n differ (line 12), the input sequence i _1, ..., in is provided as the auxiliary input sequence I ₁ ,..., I _n (line 13).

The deviation between the program flow observation sets obs1, obs2 is determined using the difference function, which determines a difference between the program flow observation sets. The difference of program flow observation sets can be evaluated by semantic difference functions, so-called string distance metrics, such as Levenshtein distance, Hamming distance and others.

In line 14, an additional stateful fuzzing process StatefulFuzz is started, which uses the new auxiliary input sequence 11, ..., I2n as fixed input elements for this new fuzzing process via the subroutine of 4 taken into account. This additional fuzzing process StatefulFuzz has an input sequence length of 2n+1. Emptyset corresponds to an empty set from which an input sequence is determined in the create subroutine. "2n+1" corresponds to the number of input elements for the fuzzing process. In line 7, the number of input elements of the process in sf1 is equal to "n". The newly started process uses twice the number of input elements plus 1 (2*n+1) instead of "n". For this purpose, it is assumed that the observed program flow set obs1 and the observed program flow set obs2 do not differ from one another. The start of the additional stateful fuzzing process (line 14) can take place parallel to the fuzzing processes that are still active, so that the time-consuming fuzzing processes can be evaluated in parallel, while the algorithm can evaluate them and possibly start new fuzzing processes.

The parallel executions of the fuzzing processes StatefulFuzz are stored in the execution set executions (line 15) and the number of stateful fuzzing processes numExecs of lines 5 to 16 executed in parallel is incremented (line 16). The steps of lines 5 to 16 are repeated according to the predetermined maximum number maxexecs of the stateful fuzzing processes executed in parallel.

in the in 5a In the algorithm shown, the unspecified input element corresponds to the last input element for the input sequence (f=2n+1), but it is also possible that the input element found by the fuzzing tool can be something other than the last input element of the input sequence, like this from the pseudo code of 4 is evident.

The execution of the stateful fuzzing processes in the program executions can also be stopped when a termination criterion is reached. This makes it possible to test further combinations of input elements in a fuzzing procedure and to delete ineffective fuzzing processes.

Instead of comparing the program flow observation set of executions of the concatenation of the input sequences with the concatenation of the program flow observation set, the program flow observation set of previously calculated executions of the concatenation of the input sequences can be compared with the concatenation of the program flow observation set.

To increase the performance of the stateful fuzzing process, the program state reached by the input sequence i _{1 ,} ..., i _f-1 , can be saved once and restored each time the program under test is executed. In particular, if f is large, this can be significantly faster than the execution of the test program code by specifying the input sequence i ₁ ... i _f-1 to restore the program state.

The pseudocode of the 5b shows a second variant of the method for generating an auxiliary input sequence 11, ..., In for use in the methods of FIG 4 . Various states are identified therein, which are determined by comparing the observations, ie the executed program flow paths, the abstract states reached and the like. It is assumed that different observations indicate that different states have been reached. The second variant uses a more direct approach, in which properties of interesting different states are potentially known and can be queried _{by conditions cond 1} , ..., cond _{m .} In particular, the second variant assumes that there are conditions that can be evaluated on the program state that indicate different states of the program.

When a stateful fuzzing process with an auxiliary input sequence I ₁ to I _{n reaches} a program state that deviates from the previously seen states according to the conditions, a new stateful fuzzing process StatefulFuzz is started with I ₁ to I _n as specified input elements. In this variant, a series of conditions corresponding to Boolean expressions about the program state of p, cond ₁ , ..., cond _m about the program state of p are specified. Here, program state means the allocation of program variables or the contents of memory. Different assessments of the conditions characterize states that can differ from each other. These conditions can e.g. be whether users are logged in, what mode p has been taken, such as e.g. B. Parking, highway driving, reversing for automobile and more. The evaluation of the conditions cond ₁ , . . . , cond _m in a program state corresponds to an array, in particular a matrix, with Boolean values of length m-1.

In 5b the pseudo code for the second variant is given. First, the conditions cond ₁ , . . . , cond _{m of} a predetermined set of conditions are evaluated (evaluate) for the initial program state of the program to be tested without an input element being supplied (line 1). For example, such conditions may include querying the following program states: length(loggedUsers) > 2, drivingMode == Highway, and the like. The result is a list of boolean values, for example [true, false, true, false].

A fuzzing process sf with n=1 (line 2) is then started. This can be executed in parallel with other fuzzing processes. The executions of the processes sf are added to an execution set executions (line 3).

Then the evaluation of the achieved program state with the conditions cond ₁ , ..., cond _m (line 4) is added to an evaluation set condition set, which _{stores the combinations of evaluations according to the conditions cond 1} , ..., cond _m and the one Characterized program state for which a fuzzing process is already running.

As with the first alternative, additional fuzzing processes are started, but depending on whether some of the input elements have reached a program state for which the evaluation of the conditions is new.

The method then waits a specific time (line 5) until the fuzzing has executed at least one of the executions of the program code (line 5). The processes sf are parallel processes that are started on line 2. It is awaited until the stateful fuzzing process StatefulFuzz has had enough time to carry out the first execution of the program to be tested and to fill the input sequence set. As long as a predetermined maximum number maxexecs of the (parallel) executed stateful fuzzing processes is not reached (line 6), new stateful fuzzing processes StatefulFuzz can be started (lines 7 to 17).

To start a new fuzzing process, an execution sf1 is first selected from the execution set executions (line 7). The executions can be selected iteratively one after the other using a heuristic, e.g. depending on the progress of the fuzzing process StatefulFuzz, randomly or according to other methods. An input sequence set is of the selected execution sf1 is determined from the selected executions sf1. (line 8). Then one or more start parameters of the selected input sequence sf1 (line 9) are determined. The start parameters can be part of a data structure that is provided for the process handler and that is responsible for starting the new process on the one hand, but also stores other information, such as the start parameters n, but also information such as runtime information can be queried, such as the input sequence set set of interesting, number of executed executions, etc. The communication can be implemented in various ways, eg inter-process communication or communication via the file system, as well as the most diverse levels of abstraction, which provide eg cloud middleware systems or other server-client architectures .

An input sequence is then selected from this input sequence set is and _{stored in i 1,} ..., in (line 10).

Then the conditions cond ₁ ,..., cond _{m are} evaluated in a program state reached after the execution of the input sequence i1,..., in (line 11).

If no fuzzing process has yet been performed for this particular combination of evaluated conditions, ie the combination of evaluations is not included in the evaluation set condition set (line 12), the auxiliary input sequence corresponds to I ₁ ,..., In the input sequence i1, ..., in (line 13).

A new fuzzing process with the auxiliary input sequence I ₁ ,...,In as first input is started (line 14) and the combination of evaluated conditions is added to the evaluation set (line 16).
The parallel executions of the fuzzing process are stored in the execution set executions (line 15). An evaluation of the program state reached according to the conditions cond ₁ , . . . , cond _m is carried out and the result is stored in the evaluation set. Then the number numExecs of the (parallel) executed stateful fuzzing processes StatefulFuzz of lines 6 to 15 is incremented. The steps in lines 6 to 15 are repeated according to the specified maximum number maxExecs of the fuzzing processes StatefulFuzz executed in parallel.

In 5c a pseudo code for a further variant of the generation of the auxiliary input sequence 11, ..., In is shown. Therein, instead of executing the stateful program code dynamically with the evaluation of the conditions cond ₁ , ..., cond _{m ,} static program analyzes are used to generate input elements leading to a program state with an evaluation of the conditions cond ₁ , ..., cond _m lead, for which no fuzzing process has been executed so far. In these static program analyses, programs are examined to determine what input elements must look like in order to reach a certain program state after program execution. The program does not have to be executed for this, only analyzed. Examples of techniques known in computer science include: symbolic execution, taint analysis, concolic execution, program dependency graph analysis, and the like.

A rating c1,...,cm is selected from the various combinations of possible ratings c1,...,cm of the conditions cond ₁ to cond _{m (line 7).} A static analysis technique is then used to generate the auxiliary input sequence I ₁ ,...,I _k leading to states for which application of the conditions leads to selected results (lines 8-10). The evaluation can be selected a) randomly from the set of evaluations that have not yet been selected or b) by ordering according to a priority criterion and selecting the first evaluation that has not yet been selected.

Potentially applicable static analysis techniques may include symbolic execution, abstract interpretation, theorem proofing, model checking, and others.

In line 11, an additional stateful fuzzing process is started, with new input sequences of length k+1. k is the length of the input sequence as determined by the static analysis of line 8.

The executions of the fuzzing process are stored in the execution set executions (line 12). Furthermore, the selected condition c1,...,cm is added to the evaluation set condition set. Then the number numExecs of the (parallel) executed fuzzing processes StatefulFuzz of lines 6 to 14 is incremented. The steps of lines 6 to 14 are repeated according to the specified maximum number of runs maxExecs.

In a fourth variant, in the pseudocode in the 5d , executes a combination of the evaluation results c1,...,cm of the conditions cond ₁ ,...,cond _{m of} the evaluation set condition set and uses them in a static analysis method with a generated formula frm. The formula frm is created according to the evaluation results c1,...,cm of the retrieved conditions cond1,..., condm. From this, the static analysis method generates, in a manner known per se, an auxiliary input sequence I ₁ ,...,I _k , so that a program state is reached in which the generated formula is not fulfilled.

The above variant thus provides for reaching a program state in which the conditions cond ₁ to cond _{m result in} a combination of the evaluations that has not yet been reached in any fuzzing process before. Instead of a previously unused combination of evaluations of the conditions cond ₁ through cond _m as in the pseudocode alternative of the 5c a formula is generated that encodes all previously used combinations of scores (rows 8 through 14). Then a static analysis is used to generate the input elements leading to a program state where this formula is not satisfied (line 15), ie the combinations of scores resulting from the application of the conditions cond ₁ to cond _m . An evaluation c1,...,cm of the conditions cond ₁ ,...,cond _m on the program state is determined by the input sequence found in this way.

Potential static analysis techniques that can be used to accomplish this task may include Symbolic Execution, Abstract Interpretation, Theorem Proofing, Model Checking, and others. These methods are known from the prior art, such as from Baldoni et al, A Survey of Symbolic Execution Techniques, ACM Computing Surveys, May 2018, D'Silva et al, A survey of automated techniques for formal software verification, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , July 2008 and Thum et al. "A classification and survey of analyzes strategies for software product lines", ACM Computing Surveys, June 2014 .

As in the variant of 5c the found input sequence is provided as an auxiliary input sequence (line 17). In addition, the selected evaluations c1, ..., cm of the conditions cond ₁ to cond _m , which have been reached by the input sequence determined by the static analysis, are determined (line 16) and stored in the evaluation set condition set (line 20).

Another fuzzing process with increased numbers k+1 of input elements in the input sequence is started and a corresponding process handler sf is added to the execution set executions (line 19), and a fuzzing process is started (line 18).

QUOTES INCLUDED IN DESCRIPTION

This list of documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Non-patent Literature Cited

• Baldoni et al, A Survey of Symbolic Execution Techniques, ACM Computing Surveys, May 2018, D'Silva et al, A survey of automated techniques for formal software verification, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , July 2008 [0081]
• Thum et al. "A classification and survey of analyzes strategies for software product lines", ACM Computing Surveys, June 2014 [0081]

Claims (10)

Computer-implemented method for testing a program code (p) using fuzzing, wherein stateful fuzzing processes for the program code (p) to be tested are each performed with a varying input sequence (i ₁ ... in) from one or more input elements (i). each stateful fuzzing process carries out a program execution with the successively supplied input elements (i) of the input sequence (i ₁ ... i _n ) and after execution with the respectively supplied input element (i) stores a program flow observation (obs) that indicates whether a program execution error has occurred or which program paths of the program code (p) have been run through, the supply of the input elements (i) of the input sequence (i ₁ ... i _n ) to the executed program code without resetting a program state of the program code (p) takes place on a given program state. procedure after claim 1 , Wherein the input sequence (i ₁ ... in) is generated before each execution of a stateful fuzzing process, the generation of the input sequence (i ₁ ... in) replacing or inserting a particular of a fuzzing tool according to a Heuristic dependent on the results of previous program executions generated input element (i) includes, which depends on one or more recent executions of the program code (p). procedure after claim 1 or 2 , wherein the input sequence (is) is generated before each execution of the stateful fuzzing process as a function of an auxiliary input sequence (I ₁ ... I _n ), the auxiliary input sequence (i ₁ ... i _n ) as a function of a Difference between program flow observations (obs1, obs2) with two input sequences (i ₁ ... i _n ), which are generated from an input sequence set (is9), in particular one of the input sequences (i ₁ ... i _n ) is a series of Input elements (i), which is part of the other of the input sequences (i ₁ ... in). procedure after claim 1 or 2 , where the input sequence (i ₁ ... in) is generated before each execution of one of the fuzzing processes depending on an auxiliary input sequence (I ₁ ...I _n ), where the auxiliary input sequence (I ₁ ... I _n ) is generated depending on a program state that the program code (p) has after execution with the input sequence (i ₁ . . . in). procedure after claim 1 or 2 , the input sequence (i ₁ ... in) being generated before each execution of one of the stateful fuzzing processes depending on an auxiliary input sequence (I ₁ ...I _n ), the auxiliary input sequence (I ₁ ... I _n ) is determined as a function of the result of a static program analysis as a function of a specified program state, different program states being specified for the state-related fuzzing processes. Procedure according to one of Claims 1 until 5 , wherein one, several or all of the input elements (i) comprise a code, a character or a character string. Procedure according to one of Claims 1 until 6 , wherein one or more input sequences (i ₁ ... in) for the stateful fuzzing processes are generated by splitting input elements of the input sequence set (is). Device for testing a program code (p) using fuzzing, wherein the device is designed to stateful fuzzing processes for the program code to be tested (p) each with a varying input sequence (i ₁ ... i _n ) from one or more input elements to be carried out, with each stateful fuzzing process carrying out a program _{execution with the successively supplied input elements of the input sequence (i 1} ... i _n ) and after execution with the respectively supplied input element (i) storing a program flow observation (obs) which indicates whether a A program execution error has occurred or which program paths of the program code have been run through, the input elements of the input sequence (i ₁ ... i _n ) being fed to the executed program code (p) without resetting a program state of the program code (p) to a predetermined program state. Computer program with program code means that is set up to implement a method according to one of Claims 1 until 7 to be carried out when the computer program is executed on a data processing device. Machine-readable storage medium with a computer program stored on it claim 9 .

Images