CN116893956A - Fuzzy test with software overlay feedback by dynamic instrumentation based on spacing of statement blocks in a control flow graph - Google Patents

Abstract

Fuzzy testing with software overlay feedback is performed by dynamic instrumentation based on the spacing of statement blocks in the control flow graph. A first aspect of the present disclosure relates to a computer-implemented method for obtaining software coverage feedback when fuzzing software on a hardware target, wherein the hardware target has at least one breakpoint register and is designed for stopping execution of the software before executing a statement of the software if a statement of the software is reached while executing the software and a memory address of the statement is set in the at least one breakpoint register, the method comprising selecting a first statement block of the software; setting a first breakpoint in at least one breakpoint register prior to a statement of the first statement block; performing first execution or first continuation on the fuzzy test iteration of the software; first inspection: whether a first breakpoint is reached upon a first execution or first continuation of the fuzzy test; the first log information is stored.

Description

Fuzzy test with software overlay feedback by dynamic instrumentation based on spacing of statement blocks in a control flow graph

Technical Field

The invention relates to fuzzy testing with software overlay feedback by dynamic instrumentation based on the spacing of statement blocks in a control flow graph.

Background

Fuzzing (English: fuzzing or fuzzing) is an automated technique for testing software. In this case, the software is executed in a number of fuzzy test iterations with invalid, unexpected and/or random input data, and is monitored here for exceptions (English: exceptions), built-in code assertions (English: events) such as crashes, failures, potential memory leaks, etc. Software for which the input data must exist in a predetermined data structure may use obfuscators (english: fuzzers) designed for such a predetermined data structure. The predetermined data structure is specified, for example, in a file format and/or a log. The (efficient) fuzzifier is designed to generate invalid, unexpected and/or random input data in a predetermined data structure such that execution of a respective fuzzification test iteration of the software can be initiated based on the input data without parsing errors. By means of fuzzy testing, it is possible in particular to determine unexpected behavior, such as unexpected (program) paths and/or programming errors, and edge situations in complex software, such as software for controlling, regulating and/or monitoring technical systems. With a better software understanding thus achieved, the software and in particular its security can be improved.

The fuzzing targets may be software (e.g., programs) and/or portions thereof (e.g., functions) that should be tested by fuzzing. The fuzzy test object may be configured such that it accepts potentially untrusted input data that may be generated by the fuzzifier for a number of fuzzy test iterations at the time of the fuzzy test. In this regard, the fuzzy test may be regarded as an automated process for sending arbitrary and especially invalid, unexpected and/or random input data to a fuzzy test object and then observing its reaction during execution of the fuzzy test iteration. The fuzzifier or fuzzifier test machine is a computer program designed to automatically generate input data for each fuzzification test iteration for a fuzzification test target. The fuzzifier is not part of the fuzzified test objects but is independent of the fuzzified test objects. Typically, the fuzzifier is not instrumented (instrumentiert). Known blumers are for example afl or libfuzzers. The combination of the Fuzzing target and the fuzzers associated therewith may be referred to as a Fuzzing Test (Fuzzing-Test). The fuzziness test trial is executable. The fuzzifier can generate different (fuzzification test) input data for a large number of fuzzification test iterations, for example hundreds or thousands of fuzzification test iterations per second, and can initiate, monitor and, if necessary, also stop the fuzzification test using the input data associated therewith, respectively. The fuzziness test iteration includes executing a fuzziness test objective/software based on the (fuzziness test) input data generated for the fuzziness test iteration. By storing the corresponding input data, the fuzzification test iteration can be reproduced at a later point in time, in particular when an unexpected behavior of the software (e.g. a path and/or programming error that is not yet known) is identified during the fuzzification test iteration, wherein then the fuzzification test object can be executed without a fuzzifier but from the stored (fuzzification test) input data.

During execution of the fuzzing test trial, information from the software object may be output. Such software coverage feedback (english: coverage-guided fuzzing) may advantageously be used at fuzzing to identify yet unknown paths/blocks and/or to locate programming errors in the software. For example, software coverage feedback at the time of the fuzzy test may be implemented by static instrumentation of the fuzzy test object, e.g., as in the alf case. In the case of static instrumentation, the fuzzy test objective, i.e. the software, is changed (e.g. at compile time) such that information about, for example, the last executed statement and/or (program) path in the software can be invoked during execution of the software and in particular during the fuzzy test iteration. Alternatively or additionally, software coverage feedback may be obtained from dynamic instrumentation. In this case, the execution of the software is controlled at run-time by the system functionality and/or the emulator in order to obtain information about the flow in the software. Especially when the software is present in compiled form (english: closed-source), software coverage feedback by dynamic instrumentation is advantageous.

JinSeok Oh, sungyu Kim, eunji Jeong and Soo-Mook Moon in 2015 IEEE Symposium in LowPower and High-Speed Chips (COOL CHIPS XVIII), "Os-less dynamic binary instrumentation for embedded firmware" (pages 1-3, IEEE, 2015) discloses software interrupt-based dynamic instrumentation through debuggers and breakpoints. Here, the binary code (english) of the software is actively changed, and the instruction is replaced by a software interrupt instruction.

Disclosure of Invention

A first general aspect of the present disclosure relates to a computer-implemented method for obtaining software coverage feedback when fuzzing software on a hardware target, wherein the hardware target has at least one breakpoint register and is designed to stop execution of the software before executing the statement if a statement of the software is reached while executing the software, and a memory address of the statement is set in the at least one breakpoint register. The method includes selecting a first statement block of the software. The method further comprises setting a first breakpoint in the at least one breakpoint register prior to the statement of the first statement block. The method further includes performing a first execution or first continuation of the fuzziness test iteration of the software. The method further comprises performing a first test: whether the first breakpoint is reached at a first execution or first continuation of the fuzzy test. The method further includes storing first log information including that the first statement block has been reached in the fuzzy test iteration if the first check yields a positive result. The method may include deleting the first breakpoint if the first check yields a positive result. The software coverage feedback at the time of the fuzzy test of the software may include the first log information. The selection of the first statement block of the software may be based on a distance from the statement block to at least one statement block reached at the time of the fuzzy test (e.g., at an earlier fuzzy test iteration or at a fuzzy test iteration of the software) in a control flow graph of the software. Alternatively or additionally, the selection of the first statement block of the software may be based on a distance weight from the statement block to at least one statement block that arrives at the fuzzing test (e.g., at an earlier fuzzing test iteration or at a fuzzing test iteration of the software) in a control flow graph of the software.

A second general aspect of the present disclosure relates to a computer system designed to perform a computer-implemented method for obtaining software coverage feedback when fuzzing software on a hardware target according to the first general aspect (or embodiments thereof).

A third general aspect of the present disclosure relates to a computer program designed to perform a computer-implemented method for obtaining software coverage feedback when fuzzing software on a hardware target according to the first general aspect (or embodiments thereof).

A fourth general aspect of the present disclosure relates to a computer-readable medium or signal storing and/or containing a computer program according to the third general aspect (or embodiments thereof).

The method proposed in the present disclosure according to the first aspect (or embodiments thereof) is directed towards obtaining software coverage feedback when fuzzing software on a hardware target, in particular when the software itself or on a hardware target cannot be statically instrumented.

For computer systems such as laptop systems (PCs, etc.) and especially for software in the form of programming code (e.g., open source), static instrumentation is a known method for obtaining software coverage feedback when executing the software and especially during fuzzy test iterations of the software.

However, static instrumentation for fuzzy testing may be difficult for software that is executed on embedded systems (english) incorporated into the technical context for the following reasons: the fuzzers required for fuzzing typically (e.g., due to insufficient computing and/or storage capacity in an embedded system) must be executed on another computer. It follows that software coverage feedback from the embedded system must first be passed to the fuzzifier (or another computer). Furthermore, it is typically necessary to test an overall system that may include a large number of components entirely. The software for the overall system may have third party libraries and software components for other vendors and/or customers. Such software components are often supplied as binary files (i.e. in compiled form, also: binary code), which cannot be modified or are only modified at cost. Because such software components are no longer compiled, they can only be statically instrumented at a cost. While software (or software components thereof) that exists as programming code can be easily statically instrumented at compile time. However, due to static instrumentation, the size of the software increases anyway, such that due to typically limited resources, static instrumented software is often no longer suitable for the memory of embedded systems. The same applies to other functionalities to which the software for fuzzing is extended.

In the alternative, the software of the embedded system may be executed in an emulator such as, for example, a QEMU. In this case, the transparency and configurability of the emulator can be used to provide software coverage feedback during the fuzzing test. Unfortunately, setting up such emulators for a particular hardware target is associated with significant effort costs, as the functionality of the software of the embedded system is typically based on the availability of external hardware components (e.g., sensors and/or actuators). However, if such hardware components are absent from the emulator, the software cannot be tested under conditions for which it was actually conceived. In fact, in this case, the software will behave differently than in the context of the technical system for which it is conceived. For example, this may be manifested in that the software may traverse other paths that are only conditionally relevant to the actual purpose of use of the software. Therefore, it is not appropriate to use an emulator for the purpose of software coverage feedback at the time of the fuzzy test.

If the debugger is connected to an embedded system, breakpoint statements (via at least one hardware breakpoint register) may be used to stop execution of software in the embedded system at a targeted location in the code. However, almost complete software coverage feedback will not be obtained with any break points, as the number of break points that are activated at the same time is typically strongly limited. For example, an ARM Cortex-M0 microcontroller is designed to maximize four simultaneously active hardware breakpoints.

The method proposed in the present disclosure in accordance with the first aspect (or embodiments thereof) is particularly suitable for fuzzy testing on hardware target/embedded systems and in (typical) cases where the number of breakpoints is strongly limited. As already stated, by performing the fuzzing test on a (real) hardware target, the software can be tested more practically and thus better than an emulator. It is sufficient for the method if the hardware target grants at least one breakpoint. Starting from, for example, a mostly specified start-up statement, it is possible to determine and/or test a control flow graph of the software at least for the part of the software covered by the fuzzification iteration, thanks to the method proposed in the present disclosure and in particular by systematically stopping the statement by strategically setting at least one breakpoint. If unexpected behavior occurs during the fuzzing test iterations, software coverage feedback can be used to improve the software and in particular its security through software changes. In addition, the functionality and in particular the security of the embedded system controlled, regulated and/or monitored by the software can be improved thereby.

The method known in the prior art (Oh et al, see above) for dynamic instrumentation through debugger and breakpoints based on software interrupts is equally likely to be used in accordance with the present disclosure to obtain software coverage feedback when fuzzing software on hardware targets. For example, the replacement instruction with a software interrupt instruction that has been described may be repeated multiple times and different breakpoints may be set accordingly. However, the memory must be rewritten here at each repetition. However, in the case of the EEPROM/flash memory used in the microcontroller, this results in a large overhead. Thus obtaining software coverage feedback efficiency through software interrupts becomes inefficient or even impractical when fuzzing software on a hardware target.

Instead, in the method proposed by the present disclosure, a hardware breakpoint is set (by means of at least one hardware breakpoint register). It is advantageously shown here that hardware breakpoints can be set without significant outlay. Thus, when the software is subjected to fuzzy test on a hardware target, the software coverage feedback can be obtained efficiently and quickly.

The method presented in the present disclosure is furthermore also applicable to use cases (e.g. products for testing/testing presence) for executing software from a read-only memory (ROM) of a hardware target. In this case, it is possible to replace statements in the binary code of the software (if any) with software interrupt instructions only with great effort.

In the method proposed in the present disclosure according to the first aspect (or embodiments thereof), at least one breakpoint or, for example, a strategic setting (or selection) of one breakpoint for each existing hardware breakpoint register, respectively, may be set on the hardware target, for example, one, two, three, four or more than four, eight or more than eight hardware breakpoint registers, including based on the distance, in particular the distance weight, from a statement block to at least one statement block reached at the time of the fuzzy test in the control flow graph. In one embodiment of the method, the strategic setting of the at least one breakpoint may comprise, for example, setting the breakpoint consecutively before the statement of the statement block, which statement block is associated in the control flow graph to a high extent with the statement block that has arrived during the fuzzy test and/or is associated with each other and is close together, i.e. has a small distance, in particular a small distance weight, from the at least one statement block that arrived at the time of the fuzzy test. The probability of finding an execution block of interest and/or relevance to a software developer/user of the software is significantly increased by a strategic setting based on the (small) distance, in particular (high) distance weight, from a statement block to at least one statement block reached at the time of the fuzzy test in the control flow graph, for example, compared to setting the breakpoint consecutively in front of adjacent statements in an abstract syntax tree of the software and/or even for any statement of the software and in particular in the case of sufficiently complex software, which in practice almost always satisfies and/or can be very easily satisfied. Thus, the blur test becomes significantly more efficient, better and/or faster. The software and in particular the system controlled, regulated and/or monitored by the software can thereby be improved.

In an embodiment of the method set forth in the present disclosure according to the first aspect (or embodiments thereof), the strategic setting (or selection) of at least one breakpoint or, for example, one breakpoint for each existing hardware breakpoint register, respectively, may be guided at least in a tendency by (small) distances, in particular (high) distance weights, of the statement blocks on the one hand, and may be based on probabilistic selection of the statement blocks on the other hand. For this purpose, for example, a probability measure can be calculated, which assigns (normalized) distance weights to statement blocks or parts thereof in the control flow graph. The selection of the statement block before or in which at least one breakpoint should then be set may include extracting the statement block one or more times based on the probability metric. In other words: statement blocks may be thrown into sub-instances based on a probability measure (e.g., a la Monte Carlo). An advantage is that while statement blocks with small distance/high distance weights are preferred and at least pseudo-randomly selected, it is still possible and sometimes also happens that more distant but still networked statement blocks are selected. In particular, randomness may cause the control flow graph to be well covered enough at the time of fuzzy testing, i.e., the choice of statement blocks is not limited to only a small portion of the control flow graph. Thereby also making the blur test more efficient, better and/or faster. In this way, the software and in particular the system controlled, regulated and/or monitored by the software can be improved.

Drawings

FIG. 1a schematically illustrates a computer-implemented method for obtaining software coverage feedback when fuzzing software on a hardware target.

FIG. 1b schematically illustrates a continuation of a computer-implemented method for obtaining software coverage feedback when fuzzing software on a hardware target.

FIG. 2 schematically illustrates an exemplary embodiment of a computer-implemented method for obtaining software coverage feedback when fuzzing software on a hardware target.

FIG. 3 shows an exemplary and schematic control flow diagram of software with statement blocks and directed edges.

Detailed Description

The method 100 presented in this disclosure enables software coverage feedback to be obtained when software is fuzzed on a hardware target. The hardware target may be, for example, an electronic control unit, wherein the software may be designed to control, regulate and/or monitor the electronic control unit.

The method 100 presented in this disclosure may be particularly applicable where software is not being used for fuzzy testing by static instrumentation. Furthermore, the software may be (fully or partially) closed-source. Alternatively, software coverage feedback may be obtained by dynamic instrumentation when fuzzing the software.

When there is a debug connection to a hardware target, software coverage feedback may be obtained at the time of the fuzzy test and in particular during the fuzzy test iteration, for example as follows and as exemplarily illustrated in fig. 2:

first, for example, before performing the fuzzing test iteration, the zeroth breakpoint may be set before the start statement (English: function to instrument). The start statement may be characterized in that it is executed independently of the input data when executing the software and thus at each fuzzing iteration. The start statement may be identified, for example, from a specification of the software (e.g., symbol file) and/or by a test engineer. Alternatively or additionally, the first breakpoint may be set in or before the first statement block prior to performing the fuzzard iteration.

The fuzzing iterations of the software may then be performed based on the fuzzing input data for the fuzzing iterations. If a zeroth or first breakpoint is reached when performing a fuzziness test iteration, the zeroth or first breakpoint may be marked as reached. Alternatively and in particular if the maximum number of (hardware) breakpoint registers is strongly limited, the zeroth and/or first breakpoint may be deleted.

At least one second breakpoint may then be set in or before the second statement block. If a second breakpoint is again reached when performing the fuzzing test iteration, the second breakpoint may be marked as reached. Alternatively and in particular if the maximum number of breakpoint registers is strongly limited, the second breakpoint may be deleted. The fuzzy test input data that has resulted in reaching the second breakpoint may be stored and stored with the statement block associated therewith in the control flow graph 10 of the software.

By continuously implementing (or deleting and resetting) at least one breakpoint, software coverage feedback can be obtained during execution of the fuzzy test iteration. For example, the software coverage feedback may include a path in control flow graph 10, where the path may include a sequence of statement blocks of control flow graph 10.

However, the control flow graph 10 of the (compiled, closed-source) software is generally not known a priori. However, the control flow graph may be constructed during fuzzy testing with the method 100 set forth in this disclosure at a reasonable cost, such as by continuously recorded set breakpoints.

It is entirely possible that the set breakpoint is not reached when performing the fuzzing test iteration. It may even happen that the set breakpoint is not reached when a large number of fuzzy test iterations are performed. The breakpoint may be marked as skipped according to predetermined criteria (e.g., if the breakpoint is not reached after a predetermined number of fuzzy test iterations and/or when a time is exceeded). The breakpoint or new breakpoint may then be set, for example, in or before a statement block adjacent to the statement block not reached in control flow graph 10.

In method 100, the selection of a statement block or breakpoint is based on the distance, optionally distance weight, of the statement block in control flow graph 10 from at least one statement block that has been reached at the time of the fuzzy test. However, various other policies for setting breakpoints available on hardware targets may additionally be applied in method 100. Furthermore, different strategies may be combined and/or alternated in the method 100. Alternatively or additionally, the policy may include probabilistic searching. Alternatively or additionally, the policy may include an entropy search. Alternatively or additionally, the policy may include guiding the search. Alternatively or additionally, the policy may include other search policies. In the case of entropy searching, statement blocks (and/or directed edges) in control flow graph 10 that arrive at (or pass at) the time of the fuzzy test may be provided with corresponding values for information gain (also: entropy). The strategy in the case of entropy searching may then comprise generating, by the fuzzifier, such fuzzy input data that maximizes the overall information gain. Statement blocks (and/or directed edges) with high information gain may thus be preferred rather. Thereby finding fewer blocks of statements that have arrived (and/or directed edges that have passed). Thus, new statement blocks (and/or directed edges) may be discovered more efficiently. In the case of a guided search, the setting of the respective at least one breakpoint can furthermore be based on user input. If, for example, a user, programmer and/or auditor of the software knows the critical point in the control flow graph 10 of the software or software, such knowledge can be used by the user interface for selecting the breakpoint and/or fuzzy test input data such that the critical point is reached and thus tested in depth at least one fuzzy test iteration.

A computer-implemented method 100 for obtaining software coverage feedback when fuzzing software on a hardware target having at least one breakpoint register and designed to stop execution of the software before executing the statement of the software if the statement is reached when executing the software and a memory address of the statement is set in the at least one breakpoint register is disclosed. The at least one breakpoint register may be a hardware breakpoint register. A hardware breakpoint is a breakpoint set through a hardware breakpoint register.

The method 100 schematically shown in fig. 1a comprises selecting 119 a first statement block of software.

The method 100 comprises setting 120 a first breakpoint in at least one breakpoint register prior to a statement of the first statement block. Setting 120 the first breakpoint before the statement of the software may include setting a memory address of the statement into at least one breakpoint register.

The method 100 may include performing a first execution 130 of a fuzziness test iteration of the software. Alternatively, the method 100 may comprise performing a first continuation 131 of the fuzziness test iteration of the software (which has been partially performed, but stopped).

The method 100 includes a first test 140: whether the first breakpoint is reached at the first execution 130 or the first continuation 131 of the fuzzing test iteration. If the first statement is to be executed without the first breakpoint based on the fuzzy test input data of the fuzzy test iteration when the software is executed, the first breakpoint is reached.

The method 100 includes storing 150 first log information, wherein the first log information includes a first statement block (or a corresponding statement preceding the first statement block or in the first statement block) that has been reached in a fuzzy test iteration if the first check 140 has a positive result.

The method 100 may comprise, as shown for example as an optional step in fig. 1 a: if the first check 140 has a positive result, the first breakpoint is deleted 151.

The software coverage feedback when fuzzing the software includes first log information. For example, the first log information may further include fuzzy test input data of a fuzzy test iteration. Storing the fuzzy test input data may be used to (continuously) create a mapping from training blocks arriving at the fuzzy test to the fuzzy test input data (or vice versa). Such a mapping is schematically shown in fig. 2.

The selection 119 of the first statement block of the software may be based on the distance (e.g., d (B, b_i)) in the control flow graph 10 of the software from the statement block 11 to at least one statement block 13 (e.g., b_i in b_erreicht (b_arrival) b_i) that arrived at the time of the fuzzy test, i.e., e.g., at a previous fuzzy test iteration or at the time of the fuzzy test iteration of the software. Here, B is a statement block from the set of statement blocks B of the control flow graph or a part thereof, for example, and b_i is a statement block in the set of reached statement blocks b_ereicht, where b_ereicht may be a subset of the set B. In other words: the selection 119 of the first statement block of the software may be based on the distance from at least one statement block 13 reached at the time of the fuzzy test, respectively, to the statement block.

Alternatively or additionally, the selection 119 of the first statement block of the software may be based on a distance (e.g., (d (b, b_i)) -1) from the statement block 11 to at least one statement block 13 reached at the time of the fuzzing, i.e., for example, at a previous fuzzing iteration or at the time of the fuzzing iteration of the software, in the control flow graph 10 of the software. In other words: the selection 119 of the first statement block of the software may be based on the distance weights from at least one statement block 13, respectively, reached at the time of the fuzzy test to the statement block.

The control flow graph 10, which is shown exemplary and schematically in fig. 3, may be a directed graph for describing the control flow of software (or a portion thereof). The control flow graph may have a set of nodes 11 (e.g. b1, b2, b3, b4, b5, b6, b 7) representing statement blocks of the described program, including possible transitions between statement blocks, i.e. a set of directed edges 12 representing the program flow. For example, statement block B5 may be at least one statement block (from B_erreicht) that arrives at the time of the fuzzy test. Control flow graph 10 of sufficiently complex software may include > =1e2, > =1e3, > =1e4, > =1e5, > =1e6, a > =1e7 and > =1e2 > =1e3 > =1e4 > =1e5 > =1e6 > =1e7 > =1e8 > =1e9 directed edges.

For example, the statement block with the smallest distance, optionally the highest distance weight, may be extracted with or without return and thus selected 119.

The distance (e.g., d (b, b_i)) of the (arbitrary) statement block 11 (e.g., statement block b) from at least one statement block (e.g., statement block b_i) reached at the time of the fuzzy test may be, for example:

minimum path length (e.g. min { PL (b, b_i| via all paths from b to b_i }) from statement block 11 (e.g. statement block b) to at least one statement block arriving at the fuzzy test (e.g. statement block b_i), and/or

Minimum path length from at least one statement block arriving at the fuzzy test (for example statement block b_i) to statement block 11 (for example statement block b) (for example min { PL (b_i, b| via all paths from b_1 to b })

Is the minimum value of (a).

For example, d (b, b_i) =min { PL (b, b_i| via all paths from b to b_i } may apply.

The distance may be a number, for example, in the (real) interval [0, infinity ] (inclusive). For example, the distance may be a natural number (including zero). In particular, the distance may be positive or zero. Alternatively, the distance may be infinity (large). Alternatively, the distance may be a real number (or natural number) in, for example, the interval (0, infinity) (including the right endpoint). Alternatively, the distance may be a real number (or natural number) in, for example, the interval (0, infinity) (excluding the endpoints). The minimum of a and infinity (or vice versa) may be defined as a. Similarly, infinity and the minimum value of infinity may be defined as infinity.

The path length from one (arbitrary) statement block to another (arbitrary) statement block may be at least an approximate measure for

Whether there is a path in the control flow graph 10 or a part thereof, said path comprising at least one directed edge 12 starting from said one (arbitrary) statement block and in the flow direction to another (arbitrary) statement block; and/or;

how long such a path is, if such a path is present, in particular how much of the path comprises at least one directed edge 12.

The path length may be defined for each path (as long as there is at least one path). If no path exists, the path length may be infinite (large).

In fig. 3, the following minimum path lengths and distances are exemplarily derived:

the path from at least one statement block (e.g., statement block b_i) that arrives at the time of the fuzzy test to at least one statement block (e.g., statement block b_i) that arrives at the time of the fuzzy test may have a path length of 0 and may be ignored.

The distance weight of (e.g., any) statement block 11 (e.g., b) to at least one statement block 13 (e.g., b_i) that arrives at the fuzzing test may be based on the distance of (any) statement block 11 (e.g., b) from at least one statement block 13 (e.g., b_i) that arrives at the fuzzing test. For example, when the distance is small, the distance weight may be large, and when the distance is large, the distance weight may be small. For this purpose arbitrary (continuous) relationships are conceivable. Further, for example, the distance weight may be inversely proportional to the distance. Further, for example, the distance weight may be the inverse of the distance, i.e., distance weight= (d (b, b_i)) ∈ (-1).

The distance weight may be a real number in the interval 0, infinity (including the left endpoint). The distance weight may be a natural number (including zero). The distance weight may be positive or zero. Furthermore, (infinity)/(1) can be defined as zero.

In fig. 3, the following distance weights are derived, for example, as the inverse of the distance:

statement block bx	bx distance to b	Distance weight of bx to b
			b1	2	0.5
b2	1	1.0
			b3	2	0.5
b4	2	0.5
			b5	0	Infinity (neglect)
b6	3	0.333333333
			b7	1	1.0

In this case, for example, the sentence blocks b2 or b7 having high distance weights 119, 159 can be selected.

As shown for example in fig. 1a, as an optional step, the method 100 may include calculating 118a at least one distance from the statement block 11 to at least one statement block 13 reached at the time of the fuzziness test (e.g., at an earlier fuzziness test iteration or at a fuzziness test iteration of the software). Alternatively or additionally, the method may include calculating 118a at least one distance weight from the statement block 11 to at least one statement block that arrived at the time of the fuzzing (e.g., at an earlier fuzzing iteration or at a fuzzing iteration of the software).

For example, as shown in FIG. 1a, as an optional step, the method 100 may include calculating 118b a first probability measure for at least one statement block that arrived at the time of the fuzzy test. The first probability measure may, for example, assign normalized weights G to at least two sentence blocks of the control flow graph 10 or a portion thereof, respectively, wherein the weights G may be normalized such that a sum of all weights of the at least two sentence blocks may result in 1.

The selection 119 of the first sentence block of the software, in particular the selection 119 of the first sentence block of the software based on the distance, in particular the distance weight, of the sentence block from at least one sentence block arriving at the fuzzy test in the control flow graph 10, may comprise a random or at least pseudo-random first extraction of sentence blocks as first sentence blocks from the sentence blocks distributed according to the first probability measure.

Such extraction may also be referred to herein or in general as craps (a la Monte Carlo). The (first, subsequent second) extraction may for example comprise extracting at least one number over a uniformly distributed (quasi) real interval [0,1], and evaluating an inverse distribution function of the (first, subsequent second) probability measure of the at least one extracted number. Other implementations of extraction are known in the art.

The weight G of the (arbitrary) sentence block 11 (e.g., b) may be, for example, a sum of distance weights of the (arbitrary) sentence block 11 from each of a large number of sentence blocks or a part thereof arrived at the time of the fuzzy test

G Sigma_ { b_ijin B_erreicht } distance weight (B, b_i).

The sum may depend on the distance, for example, in the following manner:

G～Sigma_{b_i in B_erreicht}(d(b，b_i))^(-1)

the sum of the distance weights may be generalized to a (continuous) function, in particular to a function whose image is non-negative (i.e. positive or zero). Alternatively or additionally (the sum then having only one addend for at least one statement block 13 (e.g. b_i) arriving at the time of the fuzzy test), the weight of the (arbitrary) statement block 11 may be the distance weight of the (arbitrary) statement block 11 from each of a number of statement blocks arriving at the time of the fuzzy test or a part thereof. In this case, applicable for the weight G is:

G-distance weight (b, b_i)

Or for example:

G～(d(b，b_i))^(-1)

the first extraction of the statement blocks may be performed such that the first plurality of statement blocks (i.e., the first plurality of statement blocks predetermined at the point in time of the first extraction) are not extracted. Alternatively or additionally, the first extraction of statement blocks may be performed such that a large number of statement blocks, or parts thereof, of the statement blocks 13 reached (to date) at the time of the fuzzy test are not extracted.

The exclusion of a particular statement block may be achieved, for example, by extracting the statement block until a statement block that should not be excluded has been extracted. Alternatively, the (first, later second) probability measure may be modified, for example, before extraction, so that only non-excluded sentence blocks can be extracted. For example, such a possibility consists in re-scaling the (distance) weights of the excluded statement blocks by zero and re-normalizing the probability measures thus modified.

By excluding specific statement blocks, the efficiency of the fuzzy test can be further improved.

As schematically shown as an optional step, for example in fig. 1b (continuation of fig. 1 a), the method 100 may comprise selecting 159 a second statement block of the software if the first check 140 has a positive result (e.g. at the latest before step 160, e.g. already in step 119). Furthermore, as shown as an optional step, such as in fig. 1b, the method 100 may comprise setting 160 a second breakpoint in at least one breakpoint register or in other breakpoint registers before a statement of a second statement block of the software if the first check 140 has a positive result (e.g. at the latest before step 170, e.g. also already in step 120). Setting 160 the second breakpoint before the statement of the second statement block of software may include setting a memory address of the statement of the second statement block into at least one breakpoint register or into other breakpoint registers.

As shown for example as an optional step in fig. 1b, the method 100 may comprise a second execution 170 of a fuzzing test iteration of the software (based on the fuzzing test input data related to the fuzzing test iteration). Alternatively or additionally, as shown for example as an optional step in fig. 1b, the method 100 may comprise a second continuation 171 of the (already partially executed but stopped) fuzziness test iteration of the software.

As shown for example as an optional step in fig. 1b, the method 100 may comprise a second check 180: whether a second breakpoint is reached at the second execution 170 or the second continuation 171 of the fuzzing test iteration. If the statement of the second execution block has been executed based on the fuzzy test input data of the fuzzy test iteration when the software is executed without the second breakpoint, the second breakpoint is reached.

As shown for example as an optional step in fig. 1b, the method 100 may comprise storing 190 second log information, if the second check 180 has a positive result, a second statement block has been reached in the fuzzy test iteration.

As shown for example as an optional step in fig. 1b, the method 100 may comprise deleting 191 the second breakpoint if the second check 180 has a positive result.

The software coverage feedback when fuzzing the software may include second log information. The second log information may include, for example, previous or other fuzzy test input data for the fuzzy test iteration.

The selection 159 of the second statement block of the software may be based on the distance (e.g., d (b, b_i)) in the control flow graph 10 of the software from the statement block 11 to at least one other statement block that was reached at the time of the fuzzing test (e.g., at the previous fuzzing test iteration or at the time of the fuzzing test iteration of the software). Alternatively or additionally, the selection 119 of the second statement block of the software may be based on a distance weight (e.g., (d (b, b_i)) -1) in the control flow graph 10 of the software from the statement block 11 to at least one other statement block that arrived at the time of the fuzzy test (e.g., at a previous fuzzy test iteration or at the time of the fuzzy test iteration of the software).

At least one other statement block that arrived at the time of the fuzzing test may, but need not, have arrived at the time of the first execution 130 or the first continuation 131 of the fuzzing test iteration of the software.

The at least one other statement block that arrives at the fuzzing test may be at least one statement block that arrives at the fuzzing test.

As shown for example as an optional step in fig. 1b, the method 100 may comprise calculating 158a at least one distance from the statement block 11 to at least one other statement block that arrives at the time of the fuzziness test (e.g. at an earlier fuzziness test iteration or at a fuzziness test iteration of the software). Alternatively or additionally, the method 100 may include calculating 118a at least one distance weight from the statement block 11 to at least one other statement block that arrived at the time of the fuzzing (e.g., at an earlier fuzzing iteration or at a fuzzing iteration of the software).

The distance may be, but need not be, recalculated. Furthermore, the distance weights may be, but need not be, recalculated. For example, when recalculating distances and/or distance weights for another portion of control flow graph 10, i.e., when the portion changes at an intermediate time, the recalculation may be meaningful.

For example, as shown as an optional step in fig. 1b, the method may include calculating 158b a second probability measure for at least one other statement block that arrived at the fuzzy test, wherein the second probability measure assigns normalized weights to at least two statement blocks of the control flow graph 10 or a portion thereof, respectively, wherein the weights are normalized such that a sum of all weights of the at least two statement blocks may result in 1.

The selection 159 of the second sentence block of the software, in particular the selection 159 of the second sentence block of the software based on the distance, in particular the distance weight, of the sentence blocks from the sentence blocks in the control flow graph 10 may comprise a random or at least pseudo-random second extraction of the sentence blocks as second sentence blocks from the sentence blocks according to the second probability metric distribution.

As for example in the case of the first probability measure, the weight G of the (arbitrary) statement block 11 may be the sum of the distance weights of the (arbitrary) statement block 11 from each of the other number of statement blocks or a part thereof reached during the fuzzy test. Alternatively or additionally, the weight G may be a distance weight of the (arbitrary) statement block 11 from each of another number of statement blocks arriving during the fuzzy test or a part thereof.

Another large number of statement blocks that arrive at the time of the fuzzy test may be, but need not be, a large number of statement blocks that arrive at the time of the fuzzy test. For example, the further number of statement blocks arriving at the time of the fuzzy test may differ from the number of statement blocks arriving at the time of the fuzzy test, since the further number of statement blocks arriving at the time of the fuzzy test now additionally comprises the first statement block, which has already been reached in steps 130, 131.

In principle, the second probability measure may be the first probability measure. In this case, recalculation may not be necessary. However, the second probability measure may (in general) be different from the first probability measure, especially in case another number of blocks of sentences arriving at the fuzzy test is different from a number of blocks of sentences arriving at the fuzzy test. Recalculation of probability measures may also be significant, for example, when portions of control flow graph 10 are changed at the same time.

The second extraction of the sentence blocks may be performed such that the second plurality of sentence blocks (i.e., the second plurality of sentence blocks predetermined at the point of time of the second extraction) are not extracted. The second plurality of statement blocks may, but need not, be the first plurality of statement blocks. Alternatively or additionally, the second extraction of statement blocks may be performed such that other large number of statement blocks in the statement blocks or parts thereof reached at the time of the fuzzy test are not extracted. Alternatively or additionally, the second extraction of the statement blocks may be performed such that the first statement block is not extracted.

As already stated, the efficiency of the fuzzy test may be further improved by excluding specific blocks of statements.

As shown as an optional step in fig. 1b, for example, the method 100 may include optionally generating 169 further fuzzy test input data for a fuzzy test iteration based on the first log information. The second execution 170 of the fuzzy test iteration may be based on other fuzzy test input data.

The setting 120, 160 of the breakpoint may be performed through a debug connection to the hardware target. Further, execution 130, 170 and/or continuation 131, 171 of the fuzzy test iteration may occur via a debug connection to a hardware target. As shown as an optional step in fig. 1a, for example, method 100 may include initializing 110 a debug connection to a hardware target.

The first statement block of software may include a predetermined start function (english: function to instrument) of the software. Such a selection may be suitable for starting a fuzzing test or at least one fuzzing test iteration of a fuzzing test. Alternatively, the first statement block may be any statement block of software (e.g., upon repetition 199). Alternatively or additionally, the method 100 may be started with the (zeroth) breakpoint set before any statement of the software, in particular before the statement of any (zeroth) statement block in the control flow graph 10.

As shown for example in fig. 1a as an optional step, method 100 may include generating 105 a control flow graph 10 of software based programming code of the software. Alternatively or additionally, the method 100 may include generating 105 the control flow graph 10 of the software via reverse engineering of the software. Alternatively or additionally, the method 100 may continuously generate 105 the control flow graph 10 of the software by obtaining software coverage feedback when fuzzing the software on the hardware at present.

As shown as an optional step in fig. 1a, for example, the method 100 may include storing 152 first log information, wherein the first log information includes that the first statement block has not been reached in the fuzzy test iteration if the first check 140 is negative. The method 100 may include deleting 151 the first breakpoint if the first check 140 is negative. For example, the first test 140 may be negative if a predetermined criterion is met. For example, the predetermined criterion may be met if the first breakpoint is not reached after a predetermined duration or before the end of the fuzziness test iteration.

As shown for example as an optional step in fig. 1b, the method 100 may comprise storing 192 second log information, wherein the second log information comprises a second statement block that has not been reached in the fuzzy test iteration if the second check 180 is negative. The method 100 may include deleting 191 the second breakpoint if the second check 180 is negative. For example, the second test 180 may be negative if a predetermined criterion/the predetermined criterion is met. For example, the predetermined criterion may be met if the second breakpoint is not reached after a predetermined duration or before the end of the fuzziness test iteration.

The software coverage feedback when fuzzing the software may include the first and/or second log information. For example, the first and/or second log information may further comprise (other) fuzzy test input data.

As shown for example in fig. 1a or 1b, respectively, as an optional step, the method 100 may comprise repeating 199 the method 100. For example, the method 100 may be repeated 199 until execution of the fuzziness test iteration and/or the end of the fuzziness test. Successive breakpoints can be set by repeating 199, and thus software coverage feedback can be obtained at the time of the fuzzy test. For example, repetition 199 may begin with a renewed step 110, 118a, 118b, 119, or 120. Alternatively or additionally, the repetition 199 may begin with the renewed steps 158a, 158b, 159, 160 with each successive number of breakpoints, statement blocks, etc. incremented by 1. For example, the contents of the re-step 160 may thus be as follows: if the second check has a positive result, a third breakpoint or the like is set before the statement of the third statement block.

Further disclosed is a computer system designed to perform a computer-implemented method 100 for obtaining software coverage feedback when fuzzing software on a hardware target. The computer system may include a processor and/or a working memory. The computer system may be designed to communicate with the hardware target through a debug connection. The computer system may include a fuzzifier designed to generate and provide fuzziness test input data for at least one fuzziness test iteration of software on a hardware target.

Furthermore, a computer program is disclosed, which is designed for executing the computer-implemented method 100 for obtaining software coverage feedback when fuzzing software on a hardware target. For example, a computer program may exist in an interpretable form or in a compiled form. The computer program may be loaded (also partly) into the RAM of a computer for execution, for example, as a sequence of bits or bytes.

Furthermore, a computer readable medium or signal storing and/or containing a computer program is disclosed. For example, the medium may include one of RAM, ROM, EPROM, HDD, sdd.

Claims (20)

1. A computer-implemented method (100) for obtaining software coverage feedback when fuzzing software on a hardware target, wherein the hardware target has at least one breakpoint register and is designed to stop execution of software before executing a statement of software if the statement is reached while executing the software and a memory address of the statement is set in the at least one breakpoint register, the computer-implemented method comprising:

-selecting (119) a first statement block of the software;

-setting (120) a first breakpoint in the at least one breakpoint register prior to a statement of the first statement block;

-performing a first execution (130) or a first continuation (131) of a fuzziness test iteration of the software;

-a first test (140): whether the first breakpoint is reached at a first execution (130) or a first continuation (131) of the fuzzy test iteration;

-storing (150) first log information, said first log information comprising that if said first check (140) yields a positive result, said first statement block has been reached in said fuzzy test iteration-and optionally deleting (151) said first breakpoint;

wherein the software coverage feedback at the fuzzy test of the software includes the first log information;

wherein the selection (119) of the first statement block of the software is based on a distance, optionally a distance weight, from the statement block (11) to at least one statement block (13) reached at the time of the fuzzy test in a control flow graph (10) of the software.

2. The method (100) according to claim 1, wherein the distance of the statement block (11) from at least one statement block reached at the time of the fuzzy test is:

-a minimum path length from the statement block (11) to the at least one statement block reached at the time of fuzzy test; and/or

-minimum path length from said at least one statement block arriving at the fuzzy test to said statement block (11)

Is the minimum value of (a).

3. The method (100) of claim 2, wherein the path length from one statement block to another statement block is at least an approximate measure for

-whether a path exists in the control flow graph (10) or a part thereof, the path comprising at least one directed edge (12) starting from the one statement block and in the flow direction to the other statement block; and/or

-how long such a path is, if such a path is present, in particular how much of the path comprises at least one directed edge (12).

4. The method (100) according to any one of the preceding claims, wherein a distance weight of a sentence block (11) to the at least one sentence block reached at the time of the fuzzy test is based on a distance of the sentence block (11) from the at least one sentence block reached at the time of the fuzzy test;

-optionally wherein the distance weight is large if the distance is small, and wherein the distance weight is small if the distance is large;

-optionally, wherein the distance weight is inversely proportional to the distance;

-optionally, wherein the distance weight is the inverse of the distance.

5. The method (100) according to any one of the preceding claims, the method comprising:

-calculating (118 a) at least one distance, optionally at least one distance weight, from the sentence block (11) to the at least one sentence block reached at the time of the fuzzy test.

6. The method (100) according to any one of the preceding claims, the method comprising:

-calculating (118 b) a first probability measure for the at least one statement block arriving at the fuzzy test, wherein the first probability measure assigns normalized weights to at least two statement blocks of the control flow graph (10) or a part thereof, respectively, wherein the weights are normalized such that the sum of all weights of the at least two statement blocks yields 1;

wherein the selection (119) of the first sentence block of the software, in particular the selection (119) of the first sentence block of the software based on the distance, in particular the distance weight, of the sentence block from at least one sentence block reached at the time of the fuzzy test in the control flow graph (10), comprises a random first extraction of sentence blocks as first sentence blocks from the sentence blocks distributed according to the first probability measure;

Wherein the weight of the sentence block (11) is:

-a sum of distance weights of the sentence blocks (11) from each of a number of sentence blocks or a part thereof reached at the time of the fuzzy test; and/or

-a distance weight of the statement block (11) from each of a number of statement blocks or a part thereof reached at the time of the fuzzy test.

7. The method (100) of claim 6, wherein the first extraction of the statement block is performed such that:

-a first plurality of statement blocks; and/or

-a large number of blocks of statements or parts thereof arriving at the time of the fuzzy test;

is not extracted.

8. The method (100) according to any one of the preceding claims, the method comprising:

-if the first check (140) gives a positive result, selecting (159) a second statement block of the software;

-if the first check (140) gives a positive result, setting (160) a second breakpoint in at least one breakpoint register or in another breakpoint register before a statement of a second statement block of the software;

-performing a second execution (170) of the fuzzy test iteration or a second continuation (171) of the fuzzy test iteration of the software;

-a second test (180): whether the second breakpoint is reached at a second execution (170) or a second continuation (171) of the fuzzy test iteration;

-storing (190) second log information, said second log information comprising second statement block one-and optionally deleting (191) said second breakpoint in a fuzzy test iteration if said second check (180) yields a positive result;

wherein the software coverage feedback includes the second log information at a fuzzing test of the software,

wherein the selection (159) of the second statement block of the software is based on the distance, optionally distance weight, of the statement block (11) from at least one other statement block reached at the time of the fuzzy test in the control flow graph (10) of the software;

optionally, wherein the at least one other statement block reached at the time of the fuzzing has been reached at the time of the first execution (130) or the first continuation (131) of the fuzzing iteration of the software.

9. The method (100) according to claim 8, the method comprising:

-calculating (158 a) at least one distance, optionally at least one distance weight, from the sentence block (11) to at least one other sentence block reached at the time of the fuzzy test.

10. The method (100) according to claim 8 or 9, the method comprising:

-calculating (158 b) a second probability measure for at least one other sentence block arriving at the fuzzy test, wherein the second probability measure assigns a normalized weight to at least two sentence blocks of the control flow graph (10) or a part thereof, respectively, wherein the weights are normalized such that the sum of all weights of the at least two sentence blocks yields 1;

Wherein the selection (159) of the second sentence block of the software, in particular the selection (159) of the second sentence block of the software based on the distance, in particular the distance weight, of the sentence blocks in the control flow graph (10), comprises a random second extraction of sentence blocks as second sentence blocks from the sentence blocks according to the second probability metric distribution;

wherein the weight of the sentence block (11) is:

-a sum of distance weights of the sentence block (11) from each of the other number of sentence blocks or a part thereof reached at the time of the fuzzy test; and/or

-a distance weight of the statement block (11) from each of the other number of statement blocks or a part thereof reached at the time of the fuzzy test.

11. The method (100) of claim 10, wherein the second extraction of the statement block is performed such that:

-a second plurality of statement blocks;

-other large blocks of statements or parts thereof arriving at the fuzzy test; and/or

-said first statement block

Is not extracted.

12. The method (100) according to any one of claims 8 to 11, the method comprising:

generating (169) further fuzzy test input data for the fuzzy test iteration optionally based on the first log information,

Wherein a second execution (170) of the fuzzy test iteration is based on other fuzzy test input data.

13. The method (100) according to any of the preceding claims, wherein the software is not used for fuzzy testing and/or is closed-source by static instrumentation.

14. The method (100) according to any one of the preceding claims, the method comprising:

-generating (105) a control flow graph (10) of the software by reverse engineering of the software and/or continuously by obtaining software coverage feedback when fuzzing the software on a hardware target, based on programming code of the software.

15. The method (100) according to any one of the preceding claims, wherein the hardware target is an electronic control unit and the software is designed for controlling, regulating and/or monitoring the electronic control unit.

16. The method (100) according to any one of the preceding claims, the method comprising:

-storing (152) first log information, the first log information comprising if the first check (140) is negative, the first statement block has not been reached in the fuzziness test iteration-and optionally deleting (151) the first breakpoint, wherein the first check (140) is negative if the first breakpoint has not been reached after a predetermined duration or before the end of a fuzziness test iteration; and/or

-storing (192) second log information, the second log information comprising if the second check (180) is negative, the second statement block has not been reached in the fuzziness test iteration-and optionally deleting (191) the second breakpoint, wherein the second check (180) is negative if the second breakpoint has not been reached after a predetermined duration or before the end of a fuzziness test iteration;

wherein the software coverage feedback at the time of the fuzzing of the software comprises the first and/or second log information.

17. The method (100) according to any one of the preceding claims, the method comprising:

-repeating (199) the method (100) according to any of the preceding claims.

18. A computer system designed to perform the computer implemented method (100) for obtaining software coverage feedback when fuzzing software on a hardware target according to any of the preceding claims.

19. A computer program designed for executing the computer implemented method (100) for obtaining software coverage feedback when fuzzing software on a hardware target according to any of the preceding claims 1 to 17.

20. A computer readable medium or signal storing and/or containing the computer program of claim 19.