EP1706833B1 - System and method for modeling, abstraction, and analysis of software - Google Patents
System and method for modeling, abstraction, and analysis of software Download PDFInfo
- Publication number
- EP1706833B1 EP1706833B1 EP05705999A EP05705999A EP1706833B1 EP 1706833 B1 EP1706833 B1 EP 1706833B1 EP 05705999 A EP05705999 A EP 05705999A EP 05705999 A EP05705999 A EP 05705999A EP 1706833 B1 EP1706833 B1 EP 1706833B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- software program
- program
- variables
- model checker
- basic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Not-in-force
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3604—Software analysis for verifying properties of programs
- G06F11/3608—Software analysis for verifying properties of programs using formal methods, e.g. model checking, abstract interpretation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/30—Circuit design
- G06F30/32—Circuit design at the digital level
- G06F30/33—Design verification, e.g. functional simulation or model checking
- G06F30/3323—Design verification, e.g. functional simulation or model checking using formal methods, e.g. equivalence checking or property checking
Definitions
- the present invention is related to techniques for formal analysis and verification of software.
- Model checking is an automatic technique for the verification of concurrent systems. It has several advantages over simulation, testing, and deductive reasoning, and has been used successfully in practice to verify complex sequential circuit designs and communication protocols. See E.M. Clarke, O. Grumberg, and D.A. Peled, "Model Checking," MIT Press, 2000 . In particular, model checking is automatic, and, if the design contains an error, model checking produces a counter-example (i.e., a witness of the offending behavior of the system) that can be used for effective debugging of the system. While symbolic model checking using binary decision diagrams (BDDs) offer the potential of exhaustive coverage of large state-spaces, it often does not scale well enough in practice.
- BDDs binary decision diagrams
- BMC bounded model checking
- the satisfiability check in the BMC approach is typically performed by what is known as a back-end SAT-solver. See, e.g., MK. Ganai, L Zhang, P. Ashar, and A Gupta, "Combining strength of circuit-based and CNF-based algorithms for a high performance SAT solver," in Design Automation Conference, 2002 ; E. Goldberg and Y. Novikov, "Berkmin: A fast and robust SAT solver," in Design Automation and Test in Europe, pages 132-39,2002 ; J.P.
- Prior art document US 2003/0204834 A1 provides a method that enables the automatic generation of a boolean program that is a predicate abstraction of a program written using a general programming language.
- the method is capable of abstracting code statements within the program that include procedure calls, assignments, goto statements, conditionals, and pointers.
- predicates of interest are identified for each code statement in the program.
- the process For each particular code statement, the process generates predicate statements that describe an effect that the statement has on the predicates of interest. If the effect of a particular code statement is indeterminable, nondeterministic predicate statements are included in the boolean program to model the indeterminable nature of the code statement.
- the arguments and return value of the procedure call are translated to associated predicates in the calling context.
- a verification system and method for software which advantageously translates a software program into a Boolean representation based on one or more basic blocks.
- each basic block can represent a sequence of instructions in the software program as a set of parallel assignments and a set of transitions to other basic blocks.
- a model checker such as a back-end SAT solver that uses bounded model checking, can be applied to the Boolean representation.
- the model checker proceeds by iteratively unrolling the basic blocks, each unrolling understood to be one step in a block-wise execution of the software program.
- the state of the system can be defined to contain a location indicating the current active basic block and the evaluation of all variables in scope.
- a label can be generated for each basic block and a program counter variable introduced to track progress of allowed executions of the software program during the model checking.
- Each unrolling of a basic block by the model checker advantageously has a limited number of successors. Although each unrolling introduces the whole software program into the satisfiability problem of the model checker, many basic blocks in the new unrolling can be declared unreachable by purely considering the control flow of the software program. Thus, knowledge of the model generation process can be incorporated into the decision heuristics used by the model checker to further improve the efficiency of the verification.
- the software program can be abstracted in a manner that advantageously contains both abstracted and concrete data variables. This allows a seamless tradeoff between accuracy and easy of analysis, potentially enabling accurate analysis with a fewer number of abstraction-refinement iterations.
- the use of predicate abstraction for the analysis of the software can be improved by computing the transition relations amongst basic blocks using symbolic predicate abstraction and the use of approximate transition relations. The combination of the two approaches, as well as the inclusion of heuristics to compute the many transition relations found in a whole software program more efficiently by proper scheduling of computations and memorizing of partial results, results in a more efficient approach to computing the transition relations.
- Code simplification can be applied automatically in a manner that does not affect the semantics of the program.
- Range analysis can be performed prior to modeling in order to limit the size of the variables in the Boolean representation of the software program.
- Program slicing can be performed to decompose the software program into semantically meaningful decompositions where the elements are not necessarily textual continuous.
- Refinement analysis based on spurious transitions, fragments and full paths can be used in an efficient manner by keeping the learnt clauses persistent during the abstraction-refinement iterative loop, promising a more efficient implementation of a counter-example-guided abstraction refinement framework.
- FIG. 1 shows an abstract diagram illustrating the processing performed on a software program in accordance with an embodiment of the present invention.
- FIG. 2 is pseudo-code for constructing basic blocks from a software program.
- FIG. 3 is pseudo-code for constructing basic blocks from a software program which includes non-recursive function calls.
- FIG. 4 is example C code that shows the control flow information generated for the code.
- FIG. 5 is another example of C code with a control flow graph generated for the code.
- FIG. 6 is a truth table for the control logic.
- FIG. 7 is pseudo-code for constructing combinational circuits to obtain the variable assignment logic.
- FIG. 1 is an abstract diagram illustrating the operation of a verification system, in accordance with an embodiment of the present invention.
- the input to the system is a software program 101 and a representation 105 of the property to be checked.
- the output of the system is a result that either proves that the software program 101 satisfies the property at 191 or provides a counter-example that disproves the property at 192 (or the analysis can terminate without a conclusive result due to resource constraints such as a time-out or memory exhaustion).
- the software program 101 to be analyzed can be input as a source code file or a set of source code files written in a programming language, such as the C programming language.
- a programming language such as the C programming language.
- the present invention shall be described herein with particular reference to a subset of the C programming language allowing bounded recursion, although the present invention is not limited to C and may readily be extended by one of ordinary skill in the art to other programming languages.
- the property representation 105 to be checked can come from a variety of sources and be specified in a number of different ways. For example, and without limitation, an advantageous property to check is reachability, i.e. whether a particular part of the source code is reachable.
- An advantageous mechanism for specifying the property is by modifying the source code to include a property monitor 105, namely a small portion of code that monitors the selected property for the system.
- the property monitor 105 can be provided by a label such as "ERROR_LABEL:" at the line of source code that is to be analyzed for reachability.
- the process of constructing the property monitor 105 can be readily automated by using, for example, a script.
- FIG. 1 assumes, without limitation, that the property to be checked is a reachability property.
- the system performs an automated analysis of the software program, in the form of a static analysis 110, a modeling step 120, and an abstraction step 130, which are further described below.
- the result is a translation of the program into a Boolean representation that can be analyzed by a back-end model checker at 150.
- the software program 101 is translated by the system into one or more "basic blocks.”
- each basic block of the software can contain a series of sequential instructions from the software program.
- a basic block can represent a (possibly empty) set of assignments followed by a (possibly empty) set of guarded (conditional) transitions leading to other basic blocks.
- the state of the system can be advantageously defined to contain a location indicating the current active basic block and the evaluation of all variables in scope.
- a bounded-length stack (per process) can be added to the set of global variables to model a function call stack, which allows modeling of bounded recursion.
- the model checking of the Boolean representation of the software program proceeds by unrolling the basic blocks, rather than the individual statements of the software program.
- a label can be generated for each basic block, and a program counter variable can be introduced to monitor the progress in the state transition graph composed of the basic blocks.
- the program counter variable can be used to track progress of the allowed executions of the software program during the model checking.
- An unrolling during the model checking is understood to be one step in a block-wise execution of the software program. In the context of bounded model checking, this is an efficient way of processing the software program, since, for each basic block, there are only a limited number of possible successors.
- model checking at 150 in FIG. 1 can proceed in a manner that incorporates knowledge of the model generation process into the decision heuristics.
- the program counter variable can be used by the model checker to eliminate many unreachable basic blocks in new unrollings, thereby increasing the likelihood that the model checker can decide based on the program counter variable.
- FIG. 1 assumes that the property checked by the model checker 150 is a reachability property. If the model checker 150 proves that the specified section of the software program 101 is not reachable in the abstracted Boolean representation, it is guaranteed that the section is not reachable in the original source code for the software program 101-thus, ending the verification procedure at 191 in FIG. 1 . If the model checker 150 discovers a counter-example, a check is made at 160 to see if the counter-example is spurious. If the analysis of the counter-example at 160 determines that the path is actually feasible in the original software program, a concrete counter-example has been discovered-thereby, ending the verification procedure at 192 in FIG. 1 . Otherwise, the Boolean representation is refined at 170 in order to obtain a new representation for the model checker 150.
- Boolean verification framework that utilizes advanced SAT-based and BDD-based methods for performing both bounded and unbounded verification.
- DiVER system which is described in United States Utility Patent Application, Serial No. 10/157,486 , entitled “EFFICIENT APPROACHES FOR BOUNDED MODEL CHECKING,” filed on May 30, 2002, and Aarti Gupta, Malay K. Ganai, Zijiang Yang, and Pranav Ashar, "Iterative Abstraction using SAT-based BMC with Proof Analysis," In Proceedings of International Conference on Computer-Aided Design (ICCAD), pp.416-423 (Nov. 2003 ), which are incorporated by reference herein.
- ICCAD International Conference on Computer-Aided Design
- STATIC ANALYSIS It is preferable to perform some code simplification before further processing of the software 101. For example, it can be advantageous to remove nested or embedded function calls inside other function calls by adding temporary variables. It can also be advantageous to rewrite statements with side-effects into equivalent code that does not contain statements with side-effects. See G.C. Necula, S. McPeak, S.P. Rahul, and W. Weimer, "CIL: Intermediate Language and Tools for Analysis and Transformation of C Programs," Proceedings of the 11th International Conference on Computer Construction, LNCS, pp. 213-28, Springer-Verlag (2002 ).
- a static analysis 110 is performed on the software.
- a control flow graph of the software is computed which is used in the subsequent processing for the modeling of the software.
- variables be renamed to have a unique name for the entire software program, allowing all variables to be treated as globally defined. It is also preferable to flatten out recursive data structures, for example by specifying a bound on the length of such data structures, in order to guarantee a finite model that can be used for analysis purposes.
- a "slice" of a program with respect to a set of program elements is a projection of the program that includes only program elements that might somehow affect the values of variables used at members of the considered set of program elements. See F. Tip, "A Survey of Program Slicing Techniques," Journal of Programming Languages, Vol.3, No.3, pp.121-189 (Sept. 1995 ). Slicing, thus, allows the system to find a semantically meaningful decomposition of the program where the decompositions consist of elements that are not necessarily textually continuous.
- Range analysis techniques can also be used to limit the number of bits needed to represent the various statements in the software. See R Rugina and M. C. Rinard, "Symbolic Bounds Analysis of Pointers, Array Indices, and Accessed Memory Regions," SIGPLAN Conference on Programming Language Design and Implementation,” pp. 182-95 (2000 ).
- control logic 121 describes the flow of control that can be represented by a control flow graph of the basic blocks.
- data logic 122 describes the assignments of variables given finite ranges. Assuming the software program consists of N basic blocks, each basic block can be represented by a label of [log N ] bits. A program counter variable of [log N ] bits can be introduced to monitor progress in the graph of the basic blocks.
- the control logic defines the transition relation for this program counter variable given the basic block graph and the conditions guarding some of the transitions between the basic blocks. The transitions, thus, model transfer of control within the basic block graph representing the software program.
- the program counter variable can be used to track progress of allowed executions of the software program during the model checking phase.
- Each basic block of the source code can be replaced by a tight bit-blasted version of the data variables as well as the control logic given the aforementioned program counter variable and the basic block labels.
- each basic block contains a (possibly empty) set of assignments with a (possibly empty) set of guarded (conditional) transitions leading to other basic blocks.
- the basic block can be structured more formally as follows.
- the set of all variables in the program is denoted by X .
- a type-consistent evaluation of all variables in X is denoted by x
- the set of all type-consistent evaluations is denoted by ⁇
- the set of allowed C-expressions is represented by ⁇ .
- the set V can be referred to as the assignment set of the block.
- the set of locally active variables of a basic block l can be denoted as X l and the set of type-consistent evaluations with ⁇ l .
- a function Vars : ⁇ ⁇ 2 X denotes the set of variables that occur in a C-expression ⁇ ⁇ ⁇ using Vars( ⁇ ).
- the set Vars( ⁇ ) ⁇ X includes variables that correspond to pointers, pointers of pointers, etc., of variables mentioned in ⁇ as well as address dereferences.
- a state of the program can be defined to comprise a location l E L describing the current basic block and a type-consistent evaluation of data variables x ⁇ ⁇ l where out-of-scope variables at location l are assigned the undefined value ⁇ .
- a bounded-length stack (per process) can be added to the set of global variables to model a function call stack, thereby allowing modeling of bounded recursion.
- FIG. 2 sets forth pseudo-code for the construction of basic blocks for some programming statements such as if , goto , etc.
- the main function CREATE_BASIC_BLKS is recursive.
- the first parameter entry_blk is the entry block to the current statement.
- the second parameter statement is the current statement. Depending on its type, statement may or may not become part of entry_blk.
- exit a set consists of pairs of basic blocks and conditions and is used to correctly connect outgoing edges to the following basic block which has not been created yet. Assume
- exit > 1
- next_blk will be the first basic block of the next statement.
- FIG. 4 shows sample C code that illustrates the process of constructing the basic blocks.
- the code consists of three functions, and the control flow information is shown in the comments.
- the comments are in the form C ⁇ ( N i ) which show the indices of the current and next basic blocks, where C is the index of the basic block to which the current statement belongs.
- Different statements may belong to the same basic block, e.g., the first two assignments in function bar ().
- each function call corresponds to two basic blocks although it is a single statement: one basic block holds parameter passing and the other basic block holds the returning position.
- N i is a list of destinations of the current basic block. The if statement has two destination basic blocks. Which one is chosen depends on the condition logic.
- the return statement in each function may have multiple destinations which correspond to multiple calling positions to this function.
- FIG. 5 shows another example of sample C code along with the computed control flow graph on the right.
- the example again shows how the basic blocks are computed for various types.
- Each basic block is identified in FIG. 5 by a unique number shown inside the hexagons adjacent to the basic blocks.
- the source node of the control flow graph is basic block 0, while the sink node is the highlighted basic block 8.
- the example in FIG. 5 pictorially shows how non-recursive function calls can be included in the control flow of the calling function.
- a preprocessing analysis determines the function "foo" is not called in any recursive manner.
- the two return points are recorded by an encoding that passes a unique return location as a special parameter using the variable "rtr.”
- the control logic can be expressed using 2 ⁇ log N ⁇ bits, where N is the number of basic blocks in the program.
- FIG. 6 depicts a truth table for the control logic.
- c 1 , c 2 , ... c n denote the current state variable
- the j th table line represents a control flow graph edge ⁇ 1 j ⁇ ⁇ 2 j ⁇ ⁇ n j ⁇ ⁇ 1 j ⁇ ⁇ ⁇ 2 j ⁇ ⁇ ⁇ n j ⁇ with condition k j where ⁇ i j ⁇ 0 1 is an assignment to c i and ⁇ i j ⁇ ⁇ 0 1 is an assignment to c' i .
- the data logic can be constructed as follows. All of the variables after simplification have finite domains. Assume that t bits can be used to represent a variable var with var j (1 ⁇ j ⁇ t ) being the current state bits and var' j (1 ⁇ j ⁇ t ) being the next state bits. Let var be assigned in blocks ⁇ b 1 , b 2 , ... b k ⁇ and not assigned in the remaining blocks ⁇ b k +1 , ... b N ⁇ . The logic assigned to var j is V ji at block b i (1 ⁇ i ⁇ k ). Also, let I i be the index of the basic block b i .
- FIG. 7 shows pseudo-code for constructing combinational circuits in order to obtain the variable assignment logic.
- the function "AND_gate(bit1, bit2)" creates an AND gate with two inputs bit1 and bit2.
- "OR_gate()” and "XOR_gate()” create two-input OR and XOR gate, respectively.
- Lines 2-6 in FIG. 7 handle the expression of type expr 1 &expr 2. It first recursively builds a circuit for the sub-expressions expr 1 and expr 2.
- vectors vec 1 and vec 2 be the outputs of the circuits for expr 1 and expr 2.
- the result vector has the same bitwidth as vec 1 and vec 2 .
- Each result bit is the output of an AND gate with two inputs being the corresponding bits in vec 1 and vec 2.
- the logic for carry_in is ( vec 1[ i ] ⁇ vec 2[ i ]) ⁇ ( vec 1[ i ] ⁇ carry_in ) v ( vec 2[ i ] ⁇ carry_in ) with initial value being zero.
- the result vector has only one bit. If the statement is a constant, the result vector is a binary representation of the constant. Finally, if the statement is a variable, new signals are generated.
- the modeling embodiment can be extended, with some special handling, to pointer variables.
- additional variable can be introduced.
- the number of additional variables can be the same as the number of pointers.
- the declaration ⁇ int ***p, ***q ⁇ will create four variables v p "', v p ", v p ', and v p for pointer p x each of which has its own data logic.
- the same number of variables can be created for q. All the newly generated variables are regular finite domain variables.
- v p denotes the referenced value of v p ' .
- the present invention is not limited to the specific modeling techniques described above.
- one can use an alternative translation such as the more conventional one of modeling the heap as a set of locations.
- the abstraction step 130 takes advantage of the technique of predicate abstraction 132. This approach depends on the ability to identify interesting predicates to be used for abstraction and on the ability to compute abstract transitions correctly. In contrast to early work on predicate abstraction for software which uses extensive and inherently expensive calls to various theorem provers to compute the transition relations in the abstract system, it is advantageous to follow a SAT-based approach to compute the transition relation, as disclosed herein. Additionally, and as further discussed below, it is advantageous to allow the computation of approximate abstract models and to constrain the implementation of the enumeration of the various transition relations to reuse certain common computations, thereby allowing faster computation.
- R p x ⁇ X
- R p P R p ⁇ ⁇ ⁇ R q
- the set of predicates required as P R Preds ( R ( P,P ))
- a new Boolean variable b can be added to the abstraction of the program.
- P ⁇ p 1 , ... , p n ⁇
- B ⁇ b 1 ,..., b n ⁇
- the following discussion discloses two ways of implementing the computation of a transition relation describing the current-state to next-state relationship for these Boolean variables b i .
- the current-state of a Boolean variable denotes the evaluation of the Boolean variable when entering a new basic block
- the next-state denotes the evaluation of the same Boolean variable after executing all statements in the basic block, that is, just before exiting the basic block.
- the initial state of the abstraction is completely random, that is any truth combination to B is valid.
- T ⁇ 0,1, ⁇ ,? ⁇ as a superset of B for abstraction purposes.
- the symbol ⁇ corresponds to the same symbol in the concrete state-space, namely a predicate evaluates to ⁇ when it is out-of-scope in a given basic block.
- the symbol "?” is used to denote the fact that although the predicate is defined in a given basic block, we are not interested in its evaluation to true(1) or false(0). This notation can be used for the description of approximate abstractions.
- Q P L x T n .
- T l P a location-specific set which only contains consistent vectors b with respect to the scope of predicates for a given location l .
- a bit-blasted expression can be generated that can be used to enumerate all such assignments using the algorithm described in K.L. McMillan, "Applying SAT Methods in Unbounded Symbolic Model Checking," in 14th Int'l Conference on Computer Aided Verification, Vol.
- ⁇ ⁇ ⁇ is an expression
- Y ⁇ X is a set of k variables
- ⁇ 1 ,..., ⁇ k are k expressions
- ⁇ [ Y ⁇ 1 ,..., ⁇ k ] denotes the expression obtained by point-wise substitution of the expressions ⁇ i for the variables Y in ⁇ .
- the set B R was computed on the basis of all predicates, and can be relaxed if a single predicate at a time is being considered. That is, if a single predicate p is considered, the set P R is Preds( R ( p ))).
- T( b,b' , X ) For each basic block, one transition relation T( b,b' , X ) is thus generated. Every satisfying assignment A of the transition relation represents a transition in the concrete system - the considered C-block - and its abstraction given the current set of Boolean predicates.
- the enumeration of abstract transitions proceeds by iterative additions of so-called blocking clauses that disallow a previous solution to reappear. After adding the blocking clause ⁇ A B to the Boolean expression , T( b,b',X ), the SAT-solver can then be asked to find another satisfying assignment of the enriched problem formulation, if such an assignment still exists.
- the enumeration of the transition relation using a SAT-solver is a more efficient way of computing the transition relation compared to using expensive and exponentially many calls to a theorem prover.
- the implementation of the enumeration of the various transition relations can be further improved by constraining the implementation to reuse certain common computations, thereby allowing faster overall computation. Since many transition relations need to be computed, where the set of considered Boolean variables often remains only partially the same, an intelligent scheduling of the enumeration of these transition relations and a proper memory management of derived implications, can potentially reduce the computation time of the overall abstraction procedure as viewed for the entire program significantly.
- the 35 predicates thus can at most represent 2.9 2 ⁇ 10 2 ⁇ 2 14 many abstract consistent states. This reduces the analysis at this point of the computation by a factor of more than 2 21 > 2 ⁇ 10 6 .
- the total saved computation time is a multiple of this since this saving can be re-used in multiple transition relation computations.
- the overall run-time can be further reduced by allowing the , computation of approximate abstract models to reduce the abstraction time. For example, while it may not be feasible to enumerate the set of possible solutions for the full transition relation between current-state and next-state Boolean predicate variables, it may be possible to reduce the run-time by enumerating multiple transition relations where only a subset of or even individual next-state Boolean predicate variables are considered at a time. For example, if not all these transition relations can be enumerated, it may be possible to leave some transition relations unspecified thus allowing a nondeterministic choice to the following model checking step.
- an approximation can also be computed based on the overall transition relation by excluding some abstracted satisfying assignments from further consideration. Such decisions are usually made based on a predetermined maximal length of clauses to be considered. To compute an approximate abstraction, it has been noted that it is only necessary to check the following expressions for unsatisfiability.
- the transition in the abstract state-space between the abstract states ( l 1 , b 1 ) and (l 2 , b 2 ) is included in the approximate transition relation, that is ( l 1 , b 1 ) ⁇ (l 2 , b 2 ), if and only if the expression ⁇ x 1 ⁇ ⁇ b 1 , x 2 ⁇ X l 2 ⁇ ⁇ b 2 ⁇ l 1 x 1 ⁇ l 2 x 2 is unsatisfiable.
- an over-approximation of the transition relation can be simply constructed.
- An abstract counter-example of length k then is a path that ends in an unsafe location, that is l k -1 ⁇ Bad.
- approximate transition relations are allowed, define approximate abstract paths and approximate abstract counter-examples analogously using the relation ⁇ P instead of ⁇ P .
- an expression that corresponds to a fragment of the counter-example of length k .
- T i a timed version of the transition relation ⁇ in the concrete state-space or a time-step or unrolling i with 0 ⁇ i ⁇ k -1 over the timed or unrolled variables as T i .
- the SAT solver can generate a subformula that is unsatisfiable, otherwise known as an unsatisfiability core.
- the refinement of spurious counterexamples can be further focused by using the unsatisfiability cores computed by the SAT solver.
- This heuristic can be implemented by increasing the score for the bits of the pc variables, which in turn makes the back-end SAT solver choose these variables as decision variables first.
- This heuristic forces the model checker to focus first on a static reachability computation of the control flow graph and is, thus, able to eliminate many traces quickly.
- scoring pc variables higher than other variables in the system it can also be advantageous to control how to vary the scoring of variables over various time frames. For example, increasing the relevance of pc variables more in later time-frames than in previous ones takes advantage of the SAT-solver decisions inherent in the above-mentioned DIVER tool.
- One-Hot Encoding Another heuristic that is useful is a one-hot encoding of the pc variables which allows the SAT solver to make decisions on the full pc.
- a new selection bit can be added for each basic block. The selection bit can be set iff the basic block is active, i.e., when a certain combination of pc variables bits is valid. This provides a mechanism for word-level decisions since a certain basic block selection bit automatically invalidates all other basic block selection bits through the pc variables.
- By increasing the score to set a basic block selection bit compared to other variables in the system one is able to influence the SAT solver to make quick decisions on the location first.
- An obvious disadvantage to this heuristic is that one needs to include one selection bit to the Boolean model per basic block in the software program.
- Each basic block in the control flow graph can be given a unique number which would be used to describe the one-hot encoding.
- This heuristic can be customized by choosing a subset of basic blocks for which to apply the heuristic. This customization allows the user to fine-tune the amount of additional constraints generated, since too many additional constraints may burden the SAT-solver rather than improve efficiency. For example, and without limitation, a user can be allowed to specify the following choices:
Abstract
Description
- The present invention is related to techniques for formal analysis and verification of software.
- Model checking is an automatic technique for the verification of concurrent systems. It has several advantages over simulation, testing, and deductive reasoning, and has been used successfully in practice to verify complex sequential circuit designs and communication protocols. See E.M. Clarke, O. Grumberg, and D.A. Peled, "Model Checking," MIT Press, 2000. In particular, model checking is automatic, and, if the design contains an error, model checking produces a counter-example (i.e., a witness of the offending behavior of the system) that can be used for effective debugging of the system. While symbolic model checking using binary decision diagrams (BDDs) offer the potential of exhaustive coverage of large state-spaces, it often does not scale well enough in practice. An alternative approach is bounded model checking (BMC) focusing on the search for counter-examples of bounded length only. See A. Biere, A. Cimatti, E.M Clarke, M. Fujita, and Y. Zhu, "Symbolic model checking using SAT procedures instead of BDDs," Proc. of the 36th ACM/IEEE Design Automation Conference, pp. 317-20 (1999). Effectively, the problem is translated to a Boolean formula, such that the formula is satisfiable if and only if there exists a counter-example of length k. In practice, k can be increased incrementally starting from one to find a shortest counter-example if one exists. However, additional reasoning is needed to ensure completeness of the verification when no couuter-example exists. The satisfiability check in the BMC approach is typically performed by what is known as a back-end SAT-solver. See, e.g., MK. Ganai, L Zhang, P. Ashar, and A Gupta, "Combining strength of circuit-based and CNF-based algorithms for a high performance SAT solver," in Design Automation Conference, 2002; E. Goldberg and Y. Novikov, "Berkmin: A fast and robust SAT solver," in Design Automation and Test in Europe, pages 132-39,2002; J.P. Marques-Silva and K.A Sakallah, "GRASP: A search algorithm for prepositional satisfiability," IEEE Transactions on Computers, 48:506-21, 1999; and M. Moskewicz, C. Madigan, Y. Zhao, L. Zhang, and S. Malik, "Chaff: Enginnering an efficient SAT solver," in Design Automation Conference, 2001.
- Recently, it has been proposed to apply bounded model checking techniques to the formal verification of software such as C programs. See E. Clarke, D. Kroening, "Hardware Verification using ANSI-C Programs as a Reference," Proceedings of ASP-DAC 2003, pp. 308-11 (January 2003). In this approach, a C program is translated into a monolithic SAT formula, namely a bit vector equation, which is then used with SAT-based bounded model checking to check consistency properties, including checking the equivalence of the C program to a register-transfer level (RTL) hardware design. Each individual statement in the C program is considered to be an atomic component of the program. Unfortunately, this statement-based approach has limitations in terms of concisely handling loops and functions and does not take full advantage of recent advances in model checking.
- Prior art document
US 2003/0204834 A1 provides a method that enables the automatic generation of a boolean program that is a predicate abstraction of a program written using a general programming language. The method is capable of abstracting code statements within the program that include procedure calls, assignments, goto statements, conditionals, and pointers. In accordance with the invention, predicates of interest are identified for each code statement in the program. For each particular code statement, the process generates predicate statements that describe an effect that the statement has on the predicates of interest. If the effect of a particular code statement is indeterminable, nondeterministic predicate statements are included in the boolean program to model the indeterminable nature of the code statement. In addition, if a particular code statement includes a procedure call, the arguments and return value of the procedure call are translated to associated predicates in the calling context. - A verification system and method for software is disclosed which advantageously translates a software program into a Boolean representation based on one or more basic blocks. Rather than dealing with individual statements as the atomic components of the software program, each basic block can represent a sequence of instructions in the software program as a set of parallel assignments and a set of transitions to other basic blocks. Then, a model checker, such as a back-end SAT solver that uses bounded model checking, can be applied to the Boolean representation. The model checker proceeds by iteratively unrolling the basic blocks, each unrolling understood to be one step in a block-wise execution of the software program. The state of the system can be defined to contain a location indicating the current active basic block and the evaluation of all variables in scope. Thus, a label can be generated for each basic block and a program counter variable introduced to track progress of allowed executions of the software program during the model checking. Each unrolling of a basic block by the model checker advantageously has a limited number of successors. Although each unrolling introduces the whole software program into the satisfiability problem of the model checker, many basic blocks in the new unrolling can be declared unreachable by purely considering the control flow of the software program. Thus, knowledge of the model generation process can be incorporated into the decision heuristics used by the model checker to further improve the efficiency of the verification.
- The software program can be abstracted in a manner that advantageously contains both abstracted and concrete data variables. This allows a seamless tradeoff between accuracy and easy of analysis, potentially enabling accurate analysis with a fewer number of abstraction-refinement iterations. The use of predicate abstraction for the analysis of the software can be improved by computing the transition relations amongst basic blocks using symbolic predicate abstraction and the use of approximate transition relations. The combination of the two approaches, as well as the inclusion of heuristics to compute the many transition relations found in a whole software program more efficiently by proper scheduling of computations and memorizing of partial results, results in a more efficient approach to computing the transition relations.
- Code simplification can be applied automatically in a manner that does not affect the semantics of the program. Range analysis can be performed prior to modeling in order to limit the size of the variables in the Boolean representation of the software program. Program slicing can be performed to decompose the software program into semantically meaningful decompositions where the elements are not necessarily textual continuous.
- Refinement analysis based on spurious transitions, fragments and full paths can be used in an efficient manner by keeping the learnt clauses persistent during the abstraction-refinement iterative loop, promising a more efficient implementation of a counter-example-guided abstraction refinement framework.
- The verification system and method disclosed herein provides for faster and more efficient operation and is applicable to a wider range of software applications than in the prior art. These and other advantages of the invention will be apparent to those of ordinary skill in the art by reference to the following detailed description and the accompanying drawings.
-
FIG. 1 shows an abstract diagram illustrating the processing performed on a software program in accordance with an embodiment of the present invention. -
FIG. 2 is pseudo-code for constructing basic blocks from a software program. -
FIG. 3 is pseudo-code for constructing basic blocks from a software program which includes non-recursive function calls. -
FIG. 4 is example C code that shows the control flow information generated for the code. -
FIG. 5 is another example of C code with a control flow graph generated for the code. -
FIG. 6 is a truth table for the control logic. -
FIG. 7 is pseudo-code for constructing combinational circuits to obtain the variable assignment logic. -
FIG. 1 is an abstract diagram illustrating the operation of a verification system, in accordance with an embodiment of the present invention. The input to the system is asoftware program 101 and arepresentation 105 of the property to be checked. The output of the system is a result that either proves that thesoftware program 101 satisfies the property at 191 or provides a counter-example that disproves the property at 192 (or the analysis can terminate without a conclusive result due to resource constraints such as a time-out or memory exhaustion). - The
software program 101 to be analyzed can be input as a source code file or a set of source code files written in a programming language, such as the C programming language. The present invention shall be described herein with particular reference to a subset of the C programming language allowing bounded recursion, although the present invention is not limited to C and may readily be extended by one of ordinary skill in the art to other programming languages. Theproperty representation 105 to be checked can come from a variety of sources and be specified in a number of different ways. For example, and without limitation, an advantageous property to check is reachability, i.e. whether a particular part of the source code is reachable. An advantageous mechanism for specifying the property is by modifying the source code to include aproperty monitor 105, namely a small portion of code that monitors the selected property for the system. Where reachability properties are involved, for example, theproperty monitor 105 can be provided by a label such as "ERROR_LABEL:" at the line of source code that is to be analyzed for reachability. The process of constructing theproperty monitor 105 can be readily automated by using, for example, a script.FIG. 1 assumes, without limitation, that the property to be checked is a reachability property. - The system performs an automated analysis of the software program, in the form of a
static analysis 110, amodeling step 120, and anabstraction step 130, which are further described below. The result is a translation of the program into a Boolean representation that can be analyzed by a back-end model checker at 150. - In accordance with an embodiment of an aspect of the invention, the
software program 101 is translated by the system into one or more "basic blocks." Rather than dealing with individual statements as the atomic components of the software program, as done in the prior art, each basic block of the software can contain a series of sequential instructions from the software program. A basic block can represent a (possibly empty) set of assignments followed by a (possibly empty) set of guarded (conditional) transitions leading to other basic blocks. By grouping the sequential assignments into a single basic block, these instructions can be rewritten into a set of parallel assignments. This can be accomplished, for example, by replacing certain variables appearing on the left-hand side of an assignment of an instruction inside a basic block with the expression that it was assigned previously in the same basic block if such an assignment occurs as a previous instruction in the basic block. The state of the system can be advantageously defined to contain a location indicating the current active basic block and the evaluation of all variables in scope. A bounded-length stack (per process) can be added to the set of global variables to model a function call stack, which allows modeling of bounded recursion. - The model checking of the Boolean representation of the software program, at 150 in
FIG. 1 , proceeds by unrolling the basic blocks, rather than the individual statements of the software program. A label can be generated for each basic block, and a program counter variable can be introduced to monitor the progress in the state transition graph composed of the basic blocks. The program counter variable can be used to track progress of the allowed executions of the software program during the model checking. An unrolling during the model checking is understood to be one step in a block-wise execution of the software program. In the context of bounded model checking, this is an efficient way of processing the software program, since, for each basic block, there are only a limited number of possible successors. Given a single initial block label, there is only a limited number of possible next blocks reachable in new unrollings. Although each unrolling introduces the whole software program into the satisfiability problem, many blocks in the new unrolling can be declared unreachable by merely considering the control flow of the software program. Thus, the search space can be pruned considerably. - Moreover, the model checking at 150 in
FIG. 1 can proceed in a manner that incorporates knowledge of the model generation process into the decision heuristics. The program counter variable can be used by the model checker to eliminate many unreachable basic blocks in new unrollings, thereby increasing the likelihood that the model checker can decide based on the program counter variable. -
FIG. 1 , as stated above, assumes that the property checked by themodel checker 150 is a reachability property. If themodel checker 150 proves that the specified section of thesoftware program 101 is not reachable in the abstracted Boolean representation, it is guaranteed that the section is not reachable in the original source code for the software program 101-thus, ending the verification procedure at 191 inFIG. 1 . If themodel checker 150 discovers a counter-example, a check is made at 160 to see if the counter-example is spurious. If the analysis of the counter-example at 160 determines that the path is actually feasible in the original software program, a concrete counter-example has been discovered-thereby, ending the verification procedure at 192 inFIG. 1 . Otherwise, the Boolean representation is refined at 170 in order to obtain a new representation for themodel checker 150. - Although the present invention is not limited to any specific model checking architecture, it is advantageous to use a Boolean verification framework that utilizes advanced SAT-based and BDD-based methods for performing both bounded and unbounded verification. An example of such a system is the DiVER system, which is described in United States Utility Patent Application, Serial No.
10/157,486 - With reference to
FIG. 1 , the processing involved in thestatic analysis 110, themodeling 120, and theabstraction 130 are further described in detail below. Also, the process of analyzing for spurious counter-examples at 170 and refinement at 180 are further described in detail below. - STATIC ANALYSIS. It is preferable to perform some code simplification before further processing of the
software 101. For example, it can be advantageous to remove nested or embedded function calls inside other function calls by adding temporary variables. It can also be advantageous to rewrite statements with side-effects into equivalent code that does not contain statements with side-effects. See G.C. Necula, S. McPeak, S.P. Rahul, and W. Weimer, "CIL: Intermediate Language and Tools for Analysis and Transformation of C Programs," Proceedings of the 11th International Conference on Computer Construction, LNCS, pp. 213-28, Springer-Verlag (2002). - As depicted in
FIG. 1 , astatic analysis 110 is performed on the software. At 111, a control flow graph of the software is computed which is used in the subsequent processing for the modeling of the software. During construction of the control flow graph, it is preferable that variables be renamed to have a unique name for the entire software program, allowing all variables to be treated as globally defined. It is also preferable to flatten out recursive data structures, for example by specifying a bound on the length of such data structures, in order to guarantee a finite model that can be used for analysis purposes. - In accordance with another aspect of the invention, it is also advantageous to perform pre-processing such as program slicing 112 and
range analysis 113 as part of the analysis framework. A "slice" of a program with respect to a set of program elements is a projection of the program that includes only program elements that might somehow affect the values of variables used at members of the considered set of program elements. See F. Tip, "A Survey of Program Slicing Techniques," Journal of Programming Languages, Vol.3, No.3, pp.121-189 (Sept. 1995). Slicing, thus, allows the system to find a semantically meaningful decomposition of the program where the decompositions consist of elements that are not necessarily textually continuous. Range analysis techniques can also be used to limit the number of bits needed to represent the various statements in the software. See R Rugina and M. C. Rinard, "Symbolic Bounds Analysis of Pointers, Array Indices, and Accessed Memory Regions," SIGPLAN Conference on Programming Language Design and Implementation," pp. 182-95 (2000). - MODELING. As depicted in
FIG. 1 , themodeling 120 of the software considers two kinds of logic:control logic 121 anddata logic 122. Thecontrol logic 121 describes the flow of control that can be represented by a control flow graph of the basic blocks. Thedata logic 122 describes the assignments of variables given finite ranges. Assuming the software program consists of N basic blocks, each basic block can be represented by a label of [log N] bits. A program counter variable of [log N] bits can be introduced to monitor progress in the graph of the basic blocks. The control logic defines the transition relation for this program counter variable given the basic block graph and the conditions guarding some of the transitions between the basic blocks. The transitions, thus, model transfer of control within the basic block graph representing the software program. The program counter variable can be used to track progress of allowed executions of the software program during the model checking phase. Each basic block of the source code can be replaced by a tight bit-blasted version of the data variables as well as the control logic given the aforementioned program counter variable and the basic block labels. - Similar modeling approaches have been explored in prior art software model checking tools (such as Verisoft, Java Pathfinder, and Bogor), but limited to a non-bounded model checking setting. The software modeling is herein described in a manner that takes advantage of recent progress in SAT-based bounded model checking, while also improving the efficiency of software verification by customizing the back-end model checker.
- As discussed above, each basic block contains a (possibly empty) set of assignments with a (possibly empty) set of guarded (conditional) transitions leading to other basic blocks. The basic block can be structured more formally as follows. The set of all variables in the program is denoted by X. A type-consistent evaluation of all variables in X is denoted by x, and the set of all type-consistent evaluations is denoted by χ, while the set of allowed C-expressions is represented by Σ. Then, the parallel assignments of a basic block can be written as v 1, ... , v n←e 1, ... , en , where V = {v 1, ... , v n} ⊆ X and E = {e 1, ... , en } ⊆ Σ. The set V can be referred to as the assignment set of the block. The set of locally active variables of a basic block l can be denoted as Xl and the set of type-consistent evaluations with χ l. A function Vars : Σ → 2 X denotes the set of variables that occur in a C-expression σ ∈ Σ using Vars(σ). The set Vars(σ) ⊆ X includes variables that correspond to pointers, pointers of pointers, etc., of variables mentioned in σ as well as address dereferences. This function is generalized naturally to Vars : 2 Σ → 2 X as Vars(E) = Uε∈E Vars(e). For a particular C-block with assignment set V ⊆ X the set of required variables R can then be defined as R = Vars(E) and the set of unused variables U as U = X \ (Vars(V) ∪ R).
- In this framework, a state of the program can be defined to comprise a location l E L describing the current basic block and a type-consistent evaluation of data variables x ∈ χ l where out-of-scope variables at location l are assigned the undefined value ⊥. The set of initial states is Q 0 = {(l 0, x)|x ∈ χ l
0 } where the initial state of the program is considered to be completely random in a single location l 0 where each variable in X can take a value that is type-consistent with its specification. A bounded-length stack (per process) can be added to the set of global variables to model a function call stack, thereby allowing modeling of bounded recursion. - Control Logic. The control flow graph for the software program is a finite graph G = (L, E) where L consists of basic blocks/locations and E denotes the edges between basic blocks representing transfer of control. Except the first and last instruction, the instructions in each basic block have a unique predecessor and successor.
FIG. 2 sets forth pseudo-code for the construction of basic blocks for some programming statements such as if, goto, etc. The main function CREATE_BASIC_BLKS is recursive. The first parameter entry_blk is the entry block to the current statement. The second parameter statement is the current statement. Depending on its type, statement may or may not become part of entry_blk. The function returns a set called exit which consists of pairs of basic blocks and conditions and is used to correctly connect outgoing edges to the following basic block which has not been created yet. Assume |exit| > 1 and next_blk will be the first basic block of the next statement. Then, for any (blk, cond) ∈ exit, there is a guarded transition from blk to next_blk with condition cond. The following describes how the syntax is processed inFIG. 2 : - if (cond) {statement1} else {statement2} The condition cond joins the entry block because there is no branch between the previous statement and the condition. Since the control branches over cond, it is advantageous to initialize new basic blocks entry1 and entry2 to be the entry blocks for statement1 and statement2, respectively. The basic blocks of statement1 and statement2 are then created recursively. There are edges (entry_blk → entry1) with the condition cond to be true, and (entry_blk → entry2) with the condition cond to be false. Finally, the exit blocks of the if statement are the union of the exit locks of statement1 and statement2.
- while (cond) {body} Unlike the condition in an if statement that becomes part of entry_blk, the condition cond in a while statement starts a new block cond_blk because it is the destination of more than one basic blocks. Since the control may or may not transfer to body after cond, a new entry block body_blk is initialized and the basic blocks for body are created recursively. Let body_exit_blki (1 ≤ i ≤ n) be the n exit blocks of body. There are unconditional edges (entry_blk → cond_blk) and (body_exit_blki → cond_blk). A new basic block exit_blk is initialized to be the block that contains the statement immediately after the while statement. There are two conditional edges (cond_blk → body_blk) with the condition cond to be true, and (cond_blk → exit_blk) with the condition cond to be false.
- goto label If the statement labeled by a label has already been parsed, an unconditional edge (entry_blk → label_blk) is created, where label_blk is the basic block that contains the labeled statement. Otherwise, the basic block that contains the goto statement, i.e. entry_blk, is stored in a hash table G. The keys of the entries in the hash table are the label strings of the goto statements, and the contents are a list of basic blocks that contains the goto statements with the same destination. The hash table is used to create edges when a label statement is parsed.
- label statement A new basic block label_blk is initialized for the labeled statement. It is preferable to create a new block so that the labeled statement may be the destination of goto statements, which makes the labeled statement destination of more than one basic block. Note that no basic blocks contain more than one labeled statement. If there are basic blocks in the hash table G that contain goto statement with the destination to this statement, an unconditional edge is created for each of the block to label_blk.
- function calls After simplification of the program code, there are only two types of function calls: function_call(...) and var = function_call(...).
FIG. 3 sets forth pseudo-code for generating basic blocks for non-recursive function calls. If there is parameter passing, statements are added that assign actual parameters to corresponding formal parameters to the basic block entry_blk. Note that only call-by-value is considered inFIG. 3 . If the basic blocks of the called function have not been called, basic blocks can be created for it atline 3. A non-conditional edge from entry_blk to the first basic block in called_func is created. If the function call is of type var = function_call(...), a statement can be added by assigning the return expression in called_func to var. The statement is added to the block post_func_blk that is the exit block of the function call statement. - default By default a statement, such as an assignment, will not create any new basic blocks. In such a case, the statement is simply appended to the entry block.
-
FIG. 4 shows sample C code that illustrates the process of constructing the basic blocks. The code consists of three functions, and the control flow information is shown in the comments. The comments are in the form C → (Ni ) which show the indices of the current and next basic blocks, where C is the index of the basic block to which the current statement belongs. Different statements may belong to the same basic block, e.g., the first two assignments in function bar(). Note that each function call corresponds to two basic blocks although it is a single statement: one basic block holds parameter passing and the other basic block holds the returning position. Ni is a list of destinations of the current basic block. The if statement has two destination basic blocks. Which one is chosen depends on the condition logic. The return statement in each function may have multiple destinations which correspond to multiple calling positions to this function. -
FIG. 5 shows another example of sample C code along with the computed control flow graph on the right. The example again shows how the basic blocks are computed for various types. Each basic block is identified inFIG. 5 by a unique number shown inside the hexagons adjacent to the basic blocks. The source node of the control flow graph isbasic block 0, while the sink node is the highlightedbasic block 8. The example inFIG. 5 pictorially shows how non-recursive function calls can be included in the control flow of the calling function. A preprocessing analysis determines the function "foo" is not called in any recursive manner. The two return points are recorded by an encoding that passes a unique return location as a special parameter using the variable "rtr." - The control logic can be expressed using 2 ┌logN┐ bits, where N is the number of basic blocks in the program.
FIG. 6 depicts a truth table for the control logic. InFIG. 6 , c 1, c 2, ... cn denote the current state variable, and c'1, c'2, ... c'n denote the next state variable where n = ┌logN┐. The j th table line represents a control flow graph edge - Data Logic. The data logic can be constructed as follows. All of the variables after simplification have finite domains. Assume that t bits can be used to represent a variable var with varj (1 ≤ j ≤ t) being the current state bits and var'j (1 ≤ j ≤ t) being the next state bits. Let var be assigned in blocks {b 1, b 2, ... bk } and not assigned in the remaining blocks {b k+1, ... bN }. The logic assigned to varj is Vji at block bi (1 ≤ i ≤ k). Also, let Ii be the index of the basic block bi . The data logic for varj is:
-
FIG. 7 shows pseudo-code for constructing combinational circuits in order to obtain the variable assignment logic.FIG. 7 shows how to construct the circuit based on some common expressions, bitwise AND '&', sum '+', equality '==', constant and variable. The function "AND_gate(bit1, bit2)" creates an AND gate with two inputs bit1 and bit2. Similarly, "OR_gate()" and "XOR_gate()" create two-input OR and XOR gate, respectively. Lines 2-6 inFIG. 7 handle the expression of type expr1&expr2. It first recursively builds a circuit for the sub-expressions expr1 and expr2. Let vectors vec1 and vec2 be the outputs of the circuits for expr1 and expr2. The result vector has the same bitwidth as vec1 and vec2. Each result bit is the output of an AND gate with two inputs being the corresponding bits in vec1 and vec2. Lines 8-17 inFIG. 7 handle the expression of type expr1 + expr2. It creates an adder for each bit: var [ i]=vec[i]⊕vec[i]⊕carry_in. The logic for carry_in is (vec1[i] ∧ vec2[i]) ∨ (vec1[i] ∧ carry_in) v (vec2[i] ∧ carry_in) with initial value being zero. As to the equality in lines 19-24 inFIG. 7 , the result vector has only one bit. If the statement is a constant, the result vector is a binary representation of the constant. Finally, if the statement is a variable, new signals are generated. - The modeling embodiment can be extended, with some special handling, to pointer variables. When a pointer variable is declared, additional variable can be introduced. The number of additional variables can be the same as the number of pointers. For example, the declaration {int ***p, ***q} will create four variables vp "', vp ", vp ', and vp for pointer p x each of which has its own data logic. The same number of variables can be created for q. All the newly generated variables are regular finite domain variables. However, implicitly vp denotes the referenced value of vp'. The same relationship holds for (vp ', vp ") and (vp ", vp "'). Note that an assignment to a pointer will change the face value as well as the referenced values, which result in additional assignments. If there is an assignment {*p = *q;} in the program, the following three assignments are generated: vp " = vq ", vp ' = vq ', vp = vq '. Similarly, {p = q;} results in four assignments that include vp "' = vq "'. A dereference in the C code also leads to additional variables and assignments. Let r be an integer variable. An assignment {*p = &r;} will first generate a new variable vr' that denotes the dereferenced value of vr , and then creates two assignments vp' = vr', vp = vr .
- As discussed above, the code simplification process causes sequential assignments in each basic block to become parallel. Consider the following example assignments:
- 0. int * * p, *a, *b, x;
- 1. p = &a;
- 2. a = &x;
- 3. b = *p;
- 4. *a = 5;
- 1.1. vp = 'va ;
- 1.2. v'p = va ;
- 1.3. v"p = v'a ;
- 2.1. va = 'vx ;
- 2.2. v'a = v x;
- 2.3. v'p = (vp == 'va )?'vx : v'p ;
- 2.4. v"p = (vp == 'va )?vx : v"p ;
- 3.1. vb = v'p ;
- 3.2. v'b = v"p ;
- 4.1. v'a = 5;
- 4.2. v"p = (v'p == va )?5 : v"p ;
- 4.3. v'b = (vb == va )?5: v'b ;
- 4.4. vx = ('vx == va )?5: 'vx ;
- Some of the conditions in the conditional assignments can be removed based on the previous assignments in the same basic block. For example, the condition (vp = = 'va ) at 2.3 and 2.4 can be removed because the assignment at 1.1 forces the condition to be true. Therefore, the assignments at 2.3 and 2.4 can be evaluated to vp' = 'vx and vp" = vx, respectively. The assignments at 2.1 and 2.3 together imply the condition v'p = = va at 4.2 to be true. Similarly, the conditions at 4.3 and 4.4 are true. Therefore, the conditional assignments can be simplified to the following form:
- 2.3. v'p = 'vx ;
- 2.4.
- 4.2. v"p = 5;
- 4.3. v'b = 5;
- 4.4. vx = 5;
- Then, in order to convert the sequential assignments to parallel assignments, all possible read-after-write hazards are removed. By doing so, the read variables on the right hand side are replaced with the assigned value if the same variables were assigned previously in the same basic block. As a result, the assignments at 3.1 and 3.2 would be changed to vb = va and vb' = va', respectively. In addition, some assignments are redundant when considering a basic block as one atomic step. In particular, the assignments at later steps may overwrite previous assignments. The final assignments after removing prior assignments is shown below:
- 1.1. vp = 'va ;
- 2.1. va = 'vx ;
- 2.3. v'p = 'vx ;
- 3.1. vb = va ;
- 4.1. v'a = 5;
- 4.2.
- 4.3.
- 4.4. vx = 5;
- It should be noted that the present invention is not limited to the specific modeling techniques described above. For example, and without limitation, one can use an alternative translation such as the more conventional one of modeling the heap as a set of locations.
- ABSTRACTION. Abstraction is probably the most important technique for reducing the state explosion problem in model checking. Predicate abstraction has emerged to be a powerful and popular technique for extracting finite-state model from complex, potentially infinite state systems. As depicted in
FIG. 1 , theabstraction step 130 takes advantage of the technique ofpredicate abstraction 132. This approach depends on the ability to identify interesting predicates to be used for abstraction and on the ability to compute abstract transitions correctly. In contrast to early work on predicate abstraction for software which uses extensive and inherently expensive calls to various theorem provers to compute the transition relations in the abstract system, it is advantageous to follow a SAT-based approach to compute the transition relation, as disclosed herein. Additionally, and as further discussed below, it is advantageous to allow the computation of approximate abstract models and to constrain the implementation of the enumeration of the various transition relations to reuse certain common computations, thereby allowing faster computation. - Consider a set of n Boolean predicates P = {p 1, ... , pn } ⊂Σ. The truth values to these Boolean predicates are to be updated after the election of each block given the values to the Boolean predicates when entering the considered block. Define a function Preds : 2 x → 2 P that maps a set M of variables to the predicates that range over these variables and their references and dereferences Preds(M)={p ∈ P|Vars(p)∩M ≠ ∅}. The set of required variables for a predicate p given a parallel assignment v1 , ... , vn ← e 1, ... en is
Although the variables in R(p) influence p directly, it is necessary to include more predicates and variables into the analysis to reach the full precision provided by the predicate set P as
which can also be computed iteratively. This can be generalized for a set of predicates P ⊆ P to R(P,P) = U p∈ P R(p,P) ⊆ X. The sets of unused variables for a predicate p and a set of predicates P can then be defined as
U(p,P)=X\(Vars(p)∪R(p,P)) and U(P,P) = X\(Vars(P)∪R(P,P)). For a parallel assignment, the set of predicates to be updated can be defined as Pv = Preds(V), the set of predicates required as PR = Preds (R(P,P)) and the set of unused predicates as P ∪=P\(Pv ∪PR ). - For each predicate p ∈ P a new Boolean variable b can be added to the abstraction of the program. For the set P = {p 1, ... , pn }, consider an abstraction based on the set B ={b 1 ,...,bn }. For notational convenience, consider the evaluation of the Boolean representation often to be a vector b = (b 1,...bn ) T ∈ Bn rather than a set. The following discussion discloses two ways of implementing the computation of a transition relation describing the current-state to next-state relationship for these Boolean variables bi . In this context, the current-state of a Boolean variable denotes the evaluation of the Boolean variable when entering a new basic block, while the next-state denotes the evaluation of the same Boolean variable after executing all statements in the basic block, that is, just before exiting the basic block. It should be noted that the initial state of the abstraction is completely random, that is any truth combination to B is valid.
- More formally, define the set T = {0,1,⊥,?} as a superset of B for abstraction purposes. The symbol ⊥ corresponds to the same symbol in the concrete state-space, namely a predicate evaluates to ⊥ when it is out-of-scope in a given basic block. The symbol "?" is used to denote the fact that although the predicate is defined in a given basic block, we are not interested in its evaluation to true(1) or false(0). This notation can be used for the description of approximate abstractions.
- The set of abstract states QP for a vector of predicates P ∈ Σ n of length n is thus QP = L x T n . Similar to the concrete state-space, define a location-specific set Tl P which only contains consistent vectors b with respect to the scope of predicates for a given location l. The set of initial abstract states then is
- The concretization γ(b) ofa Boolean predicate vector b=(b 1,..,bn ) T ∈ Bn is the set of true assignments to the expression
Assuming one knows the range of all variables in Pv , a bit-blasted expression can be generated that can be used to enumerate all such assignments using the algorithm described in K.L. McMillan, "Applying SAT Methods in Unbounded Symbolic Model Checking," in 14th Int'l Conference on Computer Aided Verification, Vol. 2404 of Lecture Notes in Computer Science, pp. 250-64 (2002). The range analysis framework can be used to find an appropriate bitwidth for each variable and statement in the program. Note also that it is easy to extend these definitions to the set T instead of B, as well as to extend it to the abstract states QP . If one considers the concrete transition relation →⊆ Q x Q, one can then define the abstract transition relation → P ⊆ QP x QP given a vector of predicates P as: -
- For a block with assignment set V, a set of updated predicates PV , and the set of required predicates PR , the set B can be partitioned accordingly into sets BV and BR , corresponding to PV and PR respectively. A Boolean expression can be obtained representing a transition relation T(b,b'j,X) for the next state of a Boolean variable
3 , as
(It should be noted that the set BR was computed on the basis of all predicates, and can be relaxed if a single predicate at a time is being considered. That is, if a single predicate p is considered, the set PR is Preds(R(p))). Similarly, a Boolean expression can be obtained which represents a transition relation T(b, b',X) for the next state of all Boolean variables in BV depending on the current state Boolean variables in BR as -
- For each basic block, one transition relation T(b,b', X) is thus generated. Every satisfying assignment A of the transition relation represents a transition in the concrete system - the considered C-block - and its abstraction given the current set of Boolean predicates. In order to compute T(b,b'), one can project such a satisfying assignment onto the Boolean variables b and b', and thus obtain one possible transition in the abstracted system by defining an abstract satisfying assignment AB as
- The implementation of the enumeration of the various transition relations can be further improved by constraining the implementation to reuse certain common computations, thereby allowing faster overall computation. Since many transition relations need to be computed, where the set of considered Boolean variables often remains only partially the same, an intelligent scheduling of the enumeration of these transition relations and a proper memory management of derived implications, can potentially reduce the computation time of the overall abstraction procedure as viewed for the entire program significantly.
- It should be noted that in many cases the set of n predicates does not imply that there are 2 n many consistent abstract interpretations of these predicates. In fact, as it turns out, often there are many redundant or parallel predicates where many combinations of these evaluations do not correspond to any concrete state. A simple preprocessing step analyzing whether predicates are parallel to each other can save considerable time at run-time. Consider for example a case that involved 35 relevant predicates which could be divided into four sets of parallel predicates of size eight, eight, nine and nine and a single non-parallel predicate. Parallel predicates in the present context includes predicates such as
- In addition to the above-described analysis of parallel predicates, there are often other non-consistent evaluations of the predicates in the concrete state-space. For example, for a set of separation predicates which are predicates of the form xi - xj ∼ c with ~∈ {>,≥} and c a constant, one can decide the valid combination of the set of these predicates. Such reductions of the set of feasible valid combinations can be helpful for later processing steps if the set of feasible combinations can be expressed concisely. A full enumeration of the feasible combinations may also be possible; however, the resulting set of feasible combinations may not be concise enough. It is also helpful to discover certain relationships and implications during an enumeration and remember these for later computation steps.
- For the enumeration of the transition relation from current-state to next-state Boolean variables, the effect of reducing the size of the feasible Boolean predicate truth combinations shows up both in the current-state representation as well as the next-state representation of these variables. In addition, an analysis of various basic block computations and an intelligent scheduling of the enumeration of various transition relations can significantly save time during the computation of the abstraction. Consider, for example, the following code fragment which is often found in some similar form in many software applications:
if ( condition ) x++; /* then-part */ else x += 2 ; /* else-part */For illustration purposes, consider here two predicates b 1 representing x > 2 and b 2 representing x > 3. Then, the expression x>1 represents the pre-condition for
- Include additional constraints for all basic blocks in the control flow graph.
- Include additional constraints only for those basic blocks that have multiple incoming transitions. This option requires the SAT-solver to pick one incoming edge when there are several choices. This allows the SAT-solver to quickly focus on a single feasible path at a time.
- Include additional constraints only for those basic blocks that have a single incoming transition. This option helps the SAT-solver to quickly propagate a path backwards when there are no multiple incoming edges. This process alleviates unnecessary decisions on pc variables immediately in such basic blocks.
Claims (23)
- A method of verifying a software program (101) characterized by comprising:modeling the software program, which can have bounded recursion, as a Boolean representation comprising a plurality of basic blocks, each basic block representing a sequence of one or more instructions in the software program as a set of parallel assignments and a set of transitions to other basic blocks; andapplying a SAT-based model checker (150) to the Boolean representation of the software program.
- The method of claim 1 wherein the model checker is a back-end SAT solver that uses bounded model checking.
- The method of claim 1 wherein the model checker proceeds by iterative unrolling of the basic blocks, each unrolling being a block-wise execution of the software program.
- The method of claim 3 wherein each basic block has a label and wherein a program counter variable is used to track progress of allowed executions of the software program during processing by the model checker.
- The method of claim 4 wherein the program counter variable is given an increased score when the model checker is applied, so as to cause the model checker to focus first on control flow in the software program.
- The method of claim 4 wherein each basic block is assigned a selection bit which is set to a value based on the program counter variables, so as to allow the model checker to focus on parts of the control flow in the software program identified by the selection bits.
- The method of claim 4 wherein constraints are added when the model checker is applied, in which the constraints prune the model checker's search space of impossible predecessor-successor basic block combinations.
- The method of claim 7 wherein the constraints are only applied to a subset of the basic blocks in the Boolean representation.
- The method of claim 1 wherein the software program is further modeled using a bounded stack to handle bounded recursion in the software program.
- The method of claim 1 wherein the software program is partially abstracted during the modeling such that the Boolean representation contains both abstracted variables and variables left in their concrete representations.
- The method of claim 1 wherein transition relations in the Boolean representation of the software program are enumerated using SAT-based enumeration.
- The method of claim 1 wherein common computations in transition relations in the Boolean representation of the software program are reused to speed up computation.
- The method of claim 1 wherein range analysis (113) is performed prior to modeling the software program in order to limit the size of variables in the Boolean representation of the software program.
- The method of claim 1 wherein program slicing (112) is performed to decompose the software program into related elements prior to modeling the software program.
- The method of claim 1 wherein approximate abstraction modeling is utilized and wherein, if the model checker generates a counter-example, the counter-example is checked for whether it is spurious.
- The method of claim 17 wherein the counter-example checking further comprises testing feasibility of sub-paths.
- The method of claim 17 wherein, if the counter-example is spurious, refinement is conducted using an unsatisfiability core computed by the model checker.
- A software verification system comprising:an input module which receives a software program to be analyzed, where the software program can have bounded recursion;whereby the software verification system is characterized by further comprising:a modeling module which translates the software program into a Boolean representation, the Boolean representation comprising a plurality of basic blocks, each basic block representing a sequence of instructions in the software program as a set of parallel assignments and a set of transitions to other basic blocks; anda SAT-based model checker (150) which receives and processes the Boolean representation of the software program.
- The system of claim 20 wherein the model checker is a back-end SAT solver that uses bounded model checking.
- The system of claim 20 wherein the model checker processes the Boolean representation of the software program by iterative unrolling of the basic blocks, each unrolling being a block-wise execution of the software program.
- The system of claim 22 wherein each basic block has a label and wherein a program counter variable is used to track progress of allowed executions of the software program during processing by the model checker.
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US53852404P | 2004-01-22 | 2004-01-22 | |
US11/040,409 US7346486B2 (en) | 2004-01-22 | 2005-01-21 | System and method for modeling, abstraction, and analysis of software |
PCT/US2005/001963 WO2005072257A2 (en) | 2004-01-22 | 2005-01-21 | System and method for modeling, abstraction, and analysis of software |
Publications (3)
Publication Number | Publication Date |
---|---|
EP1706833A2 EP1706833A2 (en) | 2006-10-04 |
EP1706833A4 EP1706833A4 (en) | 2010-03-17 |
EP1706833B1 true EP1706833B1 (en) | 2011-09-28 |
Family
ID=34798169
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP05705999A Not-in-force EP1706833B1 (en) | 2004-01-22 | 2005-01-21 | System and method for modeling, abstraction, and analysis of software |
Country Status (5)
Country | Link |
---|---|
US (1) | US7346486B2 (en) |
EP (1) | EP1706833B1 (en) |
JP (1) | JP2007528059A (en) |
AT (1) | ATE526628T1 (en) |
WO (1) | WO2005072257A2 (en) |
Families Citing this family (204)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7058925B2 (en) * | 2002-04-30 | 2006-06-06 | Microsoft Corporation | System and method for generating a predicate abstraction of a program |
US9106694B2 (en) | 2004-04-01 | 2015-08-11 | Fireeye, Inc. | Electronic message analysis for malware detection |
US8549638B2 (en) | 2004-06-14 | 2013-10-01 | Fireeye, Inc. | System and method of containing computer worms |
US8171553B2 (en) | 2004-04-01 | 2012-05-01 | Fireeye, Inc. | Heuristic based capture with replay to virtual machine |
US8566946B1 (en) | 2006-04-20 | 2013-10-22 | Fireeye, Inc. | Malware containment on connection |
US8898788B1 (en) | 2004-04-01 | 2014-11-25 | Fireeye, Inc. | Systems and methods for malware attack prevention |
US8881282B1 (en) | 2004-04-01 | 2014-11-04 | Fireeye, Inc. | Systems and methods for malware attack detection and identification |
US8793787B2 (en) | 2004-04-01 | 2014-07-29 | Fireeye, Inc. | Detecting malicious network content using virtual environment components |
US7587537B1 (en) | 2007-11-30 | 2009-09-08 | Altera Corporation | Serializer-deserializer circuits formed from input-output circuit registers |
US8528086B1 (en) | 2004-04-01 | 2013-09-03 | Fireeye, Inc. | System and method of detecting computer worms |
US8584239B2 (en) | 2004-04-01 | 2013-11-12 | Fireeye, Inc. | Virtual machine with dynamic data flow analysis |
US7743350B2 (en) * | 2004-05-21 | 2010-06-22 | Fujitsu Limited | Verifying one or more properties of a design using SAT-based BMC |
DE102004033339A1 (en) * | 2004-07-09 | 2006-02-02 | Infineon Technologies Ag | Method and device for detecting circuit deviations |
US7322016B2 (en) * | 2005-01-11 | 2008-01-22 | International Business Machines Corporation | Impact checking technique |
US20060247907A1 (en) * | 2005-04-29 | 2006-11-02 | Microsoft Corporation | Deciding assertions in programs with references |
US7930659B2 (en) * | 2005-06-03 | 2011-04-19 | Nec Laboratories America, Inc. | Software verification |
US7640536B1 (en) * | 2005-09-27 | 2009-12-29 | Rockwell Collins, Inc. | Variable graph minimization for improved model-checking performance |
US7835898B2 (en) * | 2005-11-22 | 2010-11-16 | International Business Machines Corporation | Satisfiability (SAT) based bounded model checkers |
US7810085B2 (en) * | 2005-12-07 | 2010-10-05 | Microsoft Corporation | Removal of unnecessary read-to-update upgrades in software transactional memory |
US8799882B2 (en) * | 2005-12-07 | 2014-08-05 | Microsoft Corporation | Compiler support for optimizing decomposed software transactional memory operations |
US7926025B2 (en) * | 2005-12-30 | 2011-04-12 | Microsoft Corporation | Symbolic program model compositions |
US8683441B2 (en) * | 2006-01-11 | 2014-03-25 | International Business Machines Corporation | Software equivalence checking |
US7536602B2 (en) * | 2006-02-17 | 2009-05-19 | Alcatel-Lucent Usa Inc. | Method and apparatus for evaluating paths in a state machine |
US7926039B2 (en) * | 2006-03-28 | 2011-04-12 | Nec Laboratories America, Inc. | Reachability analysis for program verification |
US7509534B2 (en) * | 2006-06-27 | 2009-03-24 | Microsoft Corporation | Counterexample driven refinement for abstract interpretation |
US8171438B2 (en) * | 2006-08-25 | 2012-05-01 | International Business Machines Corporation | Verification of a program partitioned according to the control flow information of the program |
JP2010504572A (en) * | 2006-09-20 | 2010-02-12 | ナショナル アイシーティー オーストラリア リミテッド | Generation of transition systems used in model checking |
US7934201B2 (en) | 2006-10-17 | 2011-04-26 | Artoftest, Inc. | System, method, and computer readable medium for universal software testing |
US7921411B2 (en) * | 2006-10-20 | 2011-04-05 | International Business Machines Corporation | Model checking of non-terminating software programs |
JP4884177B2 (en) * | 2006-11-17 | 2012-02-29 | ヤマハ発動機株式会社 | Ship steering device and ship |
FR2911971B1 (en) * | 2007-01-26 | 2009-04-24 | Commissariat Energie Atomique | METHOD AND SYSTEM FOR VERIFYING PROPERTIES OF A COMPUTER PROGRAM |
US7685471B2 (en) * | 2007-02-01 | 2010-03-23 | Fujitsu Limited | System and method for detecting software defects |
US9405819B2 (en) * | 2007-02-07 | 2016-08-02 | Fujitsu Limited | Efficient indexing using compact decision diagrams |
US20090007038A1 (en) * | 2007-04-05 | 2009-01-01 | Nec Laboratories America, Inc. | Hybrid counterexample guided abstraction refinement |
JP4924188B2 (en) * | 2007-04-27 | 2012-04-25 | トヨタ自動車株式会社 | Cross verification device |
CL2008001961A1 (en) * | 2007-07-03 | 2009-03-27 | Georgia Pacific Resins | Formulation of chemically modified maleated unsaturated fatty acid and its salts, where the chemical modification is selected from esterification, amidation combinations thereof, and methods comprising the addition of said formulation. |
US8069434B2 (en) * | 2007-10-29 | 2011-11-29 | Sap Ag | Integrated model checking and issue resolution framework |
US7949511B2 (en) * | 2007-11-12 | 2011-05-24 | Nec Laboratories America, Inc. | System and method for tunneling and slicing based BMC decomposition |
US20090193416A1 (en) * | 2008-01-24 | 2009-07-30 | Nec Laboratories America, Inc. | Decidability of reachability for threads communicating via locks |
US7441216B1 (en) * | 2008-03-31 | 2008-10-21 | International Business Machines Corporation | Applying CNF simplification techniques for SAT-based abstraction refinement |
US8549486B2 (en) * | 2008-04-21 | 2013-10-01 | Microsoft Corporation | Active property checking |
US20090282289A1 (en) * | 2008-05-06 | 2009-11-12 | Microsoft Corporation | Generation and evaluation of test cases for software validation and proofs |
US8402439B2 (en) * | 2008-06-27 | 2013-03-19 | Microsoft Corporation | Program analysis as constraint solving |
JP5093508B2 (en) * | 2008-10-15 | 2012-12-12 | 日本電気株式会社 | Loop optimization system, loop optimization method, and loop optimization program |
US8997219B2 (en) | 2008-11-03 | 2015-03-31 | Fireeye, Inc. | Systems and methods for detecting malicious PDF network content |
US8850571B2 (en) | 2008-11-03 | 2014-09-30 | Fireeye, Inc. | Systems and methods for detecting malicious network content |
US20100169618A1 (en) * | 2008-12-30 | 2010-07-01 | Microsoft Corporation | Identifying concurrency control from a sequential proof |
US8875111B2 (en) * | 2009-04-23 | 2014-10-28 | Microsoft Corporation | Intermediate language representation and modification |
US8539451B2 (en) * | 2009-05-12 | 2013-09-17 | Nec Laboratories America, Inc. | Systems and methods for model checking the precision of programs employing floating-point operations |
US8832829B2 (en) | 2009-09-30 | 2014-09-09 | Fireeye, Inc. | Network-based binary file extraction and analysis for malware detection |
US8402444B2 (en) * | 2009-10-09 | 2013-03-19 | Microsoft Corporation | Program analysis through predicate abstraction and refinement |
US8280899B2 (en) * | 2009-10-14 | 2012-10-02 | Microsoft Corporation | Abstracting events for data mining |
US8156459B1 (en) * | 2009-11-10 | 2012-04-10 | Xilinx, Inc. | Detecting differences between high level block diagram models |
US8775150B1 (en) * | 2009-12-17 | 2014-07-08 | Cadence Design Systems, Inc. | Method and system for providing an implicit unknown value to user enum data constructs in an HDL system to model power shutoff in simulation |
JP5287693B2 (en) * | 2009-12-18 | 2013-09-11 | トヨタ自動車株式会社 | Program inspection device |
US8595707B2 (en) * | 2009-12-30 | 2013-11-26 | Microsoft Corporation | Processing predicates including pointer information |
US8495588B2 (en) * | 2010-04-16 | 2013-07-23 | International Business Machines Corporation | Abstraction-guided synthesis |
US8490057B2 (en) | 2010-09-30 | 2013-07-16 | International Business Machines Corporation | Confidence-based static analysis |
US9020796B2 (en) | 2010-11-22 | 2015-04-28 | Certon Software Inc. | Model based verification using intelligent connectors |
US8832659B2 (en) * | 2010-12-06 | 2014-09-09 | University Of Washington Through Its Center For Commercialization | Systems and methods for finding concurrency errors |
US20120198399A1 (en) * | 2011-01-31 | 2012-08-02 | Sean Arash Safarpour | System, method and computer program for determining fixed value, fixed time, and stimulus hardware diagnosis |
US8650546B2 (en) * | 2011-06-30 | 2014-02-11 | International Business Machines Corporation | Static analysis based on observed string values during execution of a computer-based software application |
US8893102B2 (en) * | 2011-07-27 | 2014-11-18 | Oracle International Corporation | Method and system for performing backward-driven path-sensitive dataflow analysis |
US20130247000A1 (en) * | 2012-03-13 | 2013-09-19 | International Business Machine Corporation | Dynamic synthesis of program synchronization |
US9292693B2 (en) * | 2012-10-09 | 2016-03-22 | International Business Machines Corporation | Remediation of security vulnerabilities in computer software |
US8973131B2 (en) * | 2012-11-02 | 2015-03-03 | International Business Machines Corporation | Refinement-based security analysis |
US10572665B2 (en) | 2012-12-28 | 2020-02-25 | Fireeye, Inc. | System and method to create a number of breakpoints in a virtual machine via virtual machine trapping events |
US20140195208A1 (en) * | 2013-01-09 | 2014-07-10 | GM Global Technology Operations LLC | Efficient partition refinement based reachability checking for simulinks/stateflow models |
US9367681B1 (en) * | 2013-02-23 | 2016-06-14 | Fireeye, Inc. | Framework for efficient security coverage of mobile software applications using symbolic execution to reach regions of interest within an application |
US9176843B1 (en) | 2013-02-23 | 2015-11-03 | Fireeye, Inc. | Framework for efficient security coverage of mobile software applications |
US9159035B1 (en) | 2013-02-23 | 2015-10-13 | Fireeye, Inc. | Framework for computer application analysis of sensitive information tracking |
US9195829B1 (en) | 2013-02-23 | 2015-11-24 | Fireeye, Inc. | User interface with real-time visual playback along with synchronous textual analysis log display and event/time index for anomalous behavior detection in applications |
US8990944B1 (en) | 2013-02-23 | 2015-03-24 | Fireeye, Inc. | Systems and methods for automatically detecting backdoors |
US9009823B1 (en) | 2013-02-23 | 2015-04-14 | Fireeye, Inc. | Framework for efficient security coverage of mobile software applications installed on mobile devices |
US9009822B1 (en) | 2013-02-23 | 2015-04-14 | Fireeye, Inc. | Framework for multi-phase analysis of mobile applications |
US9104867B1 (en) | 2013-03-13 | 2015-08-11 | Fireeye, Inc. | Malicious content analysis using simulated user interaction without user involvement |
US9355247B1 (en) | 2013-03-13 | 2016-05-31 | Fireeye, Inc. | File extraction from memory dump for malicious content analysis |
US9626509B1 (en) | 2013-03-13 | 2017-04-18 | Fireeye, Inc. | Malicious content analysis with multi-version application support within single operating environment |
US9430646B1 (en) | 2013-03-14 | 2016-08-30 | Fireeye, Inc. | Distributed systems and methods for automatically detecting unknown bots and botnets |
US9311479B1 (en) | 2013-03-14 | 2016-04-12 | Fireeye, Inc. | Correlation and consolidation of analytic data for holistic view of a malware attack |
WO2014145805A1 (en) | 2013-03-15 | 2014-09-18 | Mandiant, Llc | System and method employing structured intelligence to verify and contain threats at endpoints |
US10713358B2 (en) | 2013-03-15 | 2020-07-14 | Fireeye, Inc. | System and method to extract and utilize disassembly features to classify software intent |
US9495180B2 (en) | 2013-05-10 | 2016-11-15 | Fireeye, Inc. | Optimized resource allocation for virtual machines within a malware content detection system |
US9635039B1 (en) | 2013-05-13 | 2017-04-25 | Fireeye, Inc. | Classifying sets of malicious indicators for detecting command and control communications associated with malware |
US9189318B2 (en) * | 2013-05-15 | 2015-11-17 | Oracle International Corporation | Path-sensitive analysis framework for bug checking |
US10133863B2 (en) | 2013-06-24 | 2018-11-20 | Fireeye, Inc. | Zero-day discovery system |
US9300686B2 (en) | 2013-06-28 | 2016-03-29 | Fireeye, Inc. | System and method for detecting malicious links in electronic messages |
EP2838013A1 (en) * | 2013-08-14 | 2015-02-18 | Bitreactive AS | Improved state space analysis using partial descriptions |
US9294501B2 (en) | 2013-09-30 | 2016-03-22 | Fireeye, Inc. | Fuzzy hash of behavioral results |
US9628507B2 (en) | 2013-09-30 | 2017-04-18 | Fireeye, Inc. | Advanced persistent threat (APT) detection center |
US9171160B2 (en) | 2013-09-30 | 2015-10-27 | Fireeye, Inc. | Dynamically adaptive framework and method for classifying malware using intelligent static, emulation, and dynamic analyses |
US9690936B1 (en) | 2013-09-30 | 2017-06-27 | Fireeye, Inc. | Multistage system and method for analyzing obfuscated content for malware |
US10515214B1 (en) | 2013-09-30 | 2019-12-24 | Fireeye, Inc. | System and method for classifying malware within content created during analysis of a specimen |
US9736179B2 (en) | 2013-09-30 | 2017-08-15 | Fireeye, Inc. | System, apparatus and method for using malware analysis results to drive adaptive instrumentation of virtual machines to improve exploit detection |
US9921978B1 (en) | 2013-11-08 | 2018-03-20 | Fireeye, Inc. | System and method for enhanced security of storage devices |
US9747446B1 (en) | 2013-12-26 | 2017-08-29 | Fireeye, Inc. | System and method for run-time object classification |
US9756074B2 (en) | 2013-12-26 | 2017-09-05 | Fireeye, Inc. | System and method for IPS and VM-based detection of suspicious objects |
US9292686B2 (en) | 2014-01-16 | 2016-03-22 | Fireeye, Inc. | Micro-virtualization architecture for threat-aware microvisor deployment in a node of a network environment |
US9262635B2 (en) | 2014-02-05 | 2016-02-16 | Fireeye, Inc. | Detection efficacy of virtual machine-based analysis with application specific events |
US9241010B1 (en) | 2014-03-20 | 2016-01-19 | Fireeye, Inc. | System and method for network behavior detection |
US10242185B1 (en) | 2014-03-21 | 2019-03-26 | Fireeye, Inc. | Dynamic guest image creation and rollback |
US9591015B1 (en) | 2014-03-28 | 2017-03-07 | Fireeye, Inc. | System and method for offloading packet processing and static analysis operations |
US9432389B1 (en) | 2014-03-31 | 2016-08-30 | Fireeye, Inc. | System, apparatus and method for detecting a malicious attack based on static analysis of a multi-flow object |
US9223972B1 (en) | 2014-03-31 | 2015-12-29 | Fireeye, Inc. | Dynamically remote tuning of a malware content detection system |
US9438623B1 (en) | 2014-06-06 | 2016-09-06 | Fireeye, Inc. | Computer exploit detection using heap spray pattern matching |
US9973531B1 (en) | 2014-06-06 | 2018-05-15 | Fireeye, Inc. | Shellcode detection |
US9594912B1 (en) | 2014-06-06 | 2017-03-14 | Fireeye, Inc. | Return-oriented programming detection |
US10084813B2 (en) | 2014-06-24 | 2018-09-25 | Fireeye, Inc. | Intrusion prevention and remedy system |
US9398028B1 (en) | 2014-06-26 | 2016-07-19 | Fireeye, Inc. | System, device and method for detecting a malicious attack based on communcations between remotely hosted virtual machines and malicious web servers |
US10805340B1 (en) | 2014-06-26 | 2020-10-13 | Fireeye, Inc. | Infection vector and malware tracking with an interactive user display |
US10002252B2 (en) | 2014-07-01 | 2018-06-19 | Fireeye, Inc. | Verification of trusted threat-aware microvisor |
US9363280B1 (en) | 2014-08-22 | 2016-06-07 | Fireeye, Inc. | System and method of detecting delivery of malware using cross-customer data |
US10671726B1 (en) | 2014-09-22 | 2020-06-02 | Fireeye Inc. | System and method for malware analysis using thread-level event monitoring |
US9773112B1 (en) | 2014-09-29 | 2017-09-26 | Fireeye, Inc. | Exploit detection of malware and malware families |
US10027689B1 (en) | 2014-09-29 | 2018-07-17 | Fireeye, Inc. | Interactive infection visualization for improved exploit detection and signature generation for malware and malware families |
US9690933B1 (en) | 2014-12-22 | 2017-06-27 | Fireeye, Inc. | Framework for classifying an object as malicious with machine learning for deploying updated predictive models |
US10075455B2 (en) | 2014-12-26 | 2018-09-11 | Fireeye, Inc. | Zero-day rotating guest image profile |
US9934376B1 (en) | 2014-12-29 | 2018-04-03 | Fireeye, Inc. | Malware detection appliance architecture |
US9838417B1 (en) | 2014-12-30 | 2017-12-05 | Fireeye, Inc. | Intelligent context aware user interaction for malware detection |
US10148693B2 (en) | 2015-03-25 | 2018-12-04 | Fireeye, Inc. | Exploit detection system |
US9690606B1 (en) | 2015-03-25 | 2017-06-27 | Fireeye, Inc. | Selective system call monitoring |
US9438613B1 (en) | 2015-03-30 | 2016-09-06 | Fireeye, Inc. | Dynamic content activation for automated analysis of embedded objects |
US10417031B2 (en) | 2015-03-31 | 2019-09-17 | Fireeye, Inc. | Selective virtualization for security threat detection |
US10474813B1 (en) | 2015-03-31 | 2019-11-12 | Fireeye, Inc. | Code injection technique for remediation at an endpoint of a network |
US9483644B1 (en) | 2015-03-31 | 2016-11-01 | Fireeye, Inc. | Methods for detecting file altering malware in VM based analysis |
US9654485B1 (en) | 2015-04-13 | 2017-05-16 | Fireeye, Inc. | Analytics-based security monitoring system and method |
US9514025B2 (en) * | 2015-04-15 | 2016-12-06 | International Business Machines Corporation | Modeling memory use of applications |
US9594904B1 (en) | 2015-04-23 | 2017-03-14 | Fireeye, Inc. | Detecting malware based on reflection |
US10642753B1 (en) | 2015-06-30 | 2020-05-05 | Fireeye, Inc. | System and method for protecting a software component running in virtual machine using a virtualization layer |
US10726127B1 (en) | 2015-06-30 | 2020-07-28 | Fireeye, Inc. | System and method for protecting a software component running in a virtual machine through virtual interrupts by the virtualization layer |
US11113086B1 (en) | 2015-06-30 | 2021-09-07 | Fireeye, Inc. | Virtual system and method for securing external network connectivity |
US10454950B1 (en) | 2015-06-30 | 2019-10-22 | Fireeye, Inc. | Centralized aggregation technique for detecting lateral movement of stealthy cyber-attacks |
US10715542B1 (en) | 2015-08-14 | 2020-07-14 | Fireeye, Inc. | Mobile application risk analysis |
US10176321B2 (en) | 2015-09-22 | 2019-01-08 | Fireeye, Inc. | Leveraging behavior-based rules for malware family classification |
US10033747B1 (en) | 2015-09-29 | 2018-07-24 | Fireeye, Inc. | System and method for detecting interpreter-based exploit attacks |
US10210329B1 (en) | 2015-09-30 | 2019-02-19 | Fireeye, Inc. | Method to detect application execution hijacking using memory protection |
US10601865B1 (en) | 2015-09-30 | 2020-03-24 | Fireeye, Inc. | Detection of credential spearphishing attacks using email analysis |
US9825989B1 (en) | 2015-09-30 | 2017-11-21 | Fireeye, Inc. | Cyber attack early warning system |
US10706149B1 (en) | 2015-09-30 | 2020-07-07 | Fireeye, Inc. | Detecting delayed activation malware using a primary controller and plural time controllers |
US10817606B1 (en) | 2015-09-30 | 2020-10-27 | Fireeye, Inc. | Detecting delayed activation malware using a run-time monitoring agent and time-dilation logic |
US9825976B1 (en) | 2015-09-30 | 2017-11-21 | Fireeye, Inc. | Detection and classification of exploit kits |
US10284575B2 (en) | 2015-11-10 | 2019-05-07 | Fireeye, Inc. | Launcher for setting analysis environment variations for malware detection |
US10447728B1 (en) | 2015-12-10 | 2019-10-15 | Fireeye, Inc. | Technique for protecting guest processes using a layered virtualization architecture |
US10846117B1 (en) | 2015-12-10 | 2020-11-24 | Fireeye, Inc. | Technique for establishing secure communication between host and guest processes of a virtualization architecture |
US10108446B1 (en) | 2015-12-11 | 2018-10-23 | Fireeye, Inc. | Late load technique for deploying a virtualization layer underneath a running operating system |
US10621338B1 (en) | 2015-12-30 | 2020-04-14 | Fireeye, Inc. | Method to detect forgery and exploits using last branch recording registers |
US10565378B1 (en) | 2015-12-30 | 2020-02-18 | Fireeye, Inc. | Exploit of privilege detection framework |
US10050998B1 (en) | 2015-12-30 | 2018-08-14 | Fireeye, Inc. | Malicious message analysis system |
US10133866B1 (en) | 2015-12-30 | 2018-11-20 | Fireeye, Inc. | System and method for triggering analysis of an object for malware in response to modification of that object |
US11552986B1 (en) | 2015-12-31 | 2023-01-10 | Fireeye Security Holdings Us Llc | Cyber-security framework for application of virtual features |
US10581874B1 (en) | 2015-12-31 | 2020-03-03 | Fireeye, Inc. | Malware detection system with contextual analysis |
US9824216B1 (en) | 2015-12-31 | 2017-11-21 | Fireeye, Inc. | Susceptible environment detection system |
EP3217284B1 (en) | 2016-03-09 | 2022-06-08 | Tata Consultancy Services Limited | Data structure abstraction for model checking |
US10601863B1 (en) | 2016-03-25 | 2020-03-24 | Fireeye, Inc. | System and method for managing sensor enrollment |
US10785255B1 (en) | 2016-03-25 | 2020-09-22 | Fireeye, Inc. | Cluster configuration within a scalable malware detection system |
US10671721B1 (en) | 2016-03-25 | 2020-06-02 | Fireeye, Inc. | Timeout management services |
US10616266B1 (en) | 2016-03-25 | 2020-04-07 | Fireeye, Inc. | Distributed malware detection system and submission workflow thereof |
US10893059B1 (en) | 2016-03-31 | 2021-01-12 | Fireeye, Inc. | Verification and enhancement using detection systems located at the network periphery and endpoint devices |
US10169585B1 (en) | 2016-06-22 | 2019-01-01 | Fireeye, Inc. | System and methods for advanced malware detection through placement of transition events |
US10462173B1 (en) | 2016-06-30 | 2019-10-29 | Fireeye, Inc. | Malware detection verification and enhancement by coordinating endpoint and malware detection systems |
US10592678B1 (en) | 2016-09-09 | 2020-03-17 | Fireeye, Inc. | Secure communications between peers using a verified virtual trusted platform module |
US10491627B1 (en) | 2016-09-29 | 2019-11-26 | Fireeye, Inc. | Advanced malware detection using similarity analysis |
CN106528407B (en) * | 2016-10-19 | 2019-01-25 | 中国航空综合技术研究所 | A kind of embedded software safety automatic Verification system and its verification method |
US10795991B1 (en) | 2016-11-08 | 2020-10-06 | Fireeye, Inc. | Enterprise search |
US10587647B1 (en) | 2016-11-22 | 2020-03-10 | Fireeye, Inc. | Technique for malware detection capability comparison of network security devices |
US10581879B1 (en) | 2016-12-22 | 2020-03-03 | Fireeye, Inc. | Enhanced malware detection for generated objects |
US10552610B1 (en) | 2016-12-22 | 2020-02-04 | Fireeye, Inc. | Adaptive virtual machine snapshot update framework for malware behavioral analysis |
US10523609B1 (en) | 2016-12-27 | 2019-12-31 | Fireeye, Inc. | Multi-vector malware detection and analysis |
US10904286B1 (en) | 2017-03-24 | 2021-01-26 | Fireeye, Inc. | Detection of phishing attacks using similarity analysis |
US10554507B1 (en) | 2017-03-30 | 2020-02-04 | Fireeye, Inc. | Multi-level control for enhanced resource and object evaluation management of malware detection system |
US10902119B1 (en) | 2017-03-30 | 2021-01-26 | Fireeye, Inc. | Data extraction system for malware analysis |
US10791138B1 (en) | 2017-03-30 | 2020-09-29 | Fireeye, Inc. | Subscription-based malware detection |
US10798112B2 (en) | 2017-03-30 | 2020-10-06 | Fireeye, Inc. | Attribute-controlled malware detection |
US10601848B1 (en) | 2017-06-29 | 2020-03-24 | Fireeye, Inc. | Cyber-security system and method for weak indicator detection and correlation to generate strong indicators |
US10503904B1 (en) | 2017-06-29 | 2019-12-10 | Fireeye, Inc. | Ransomware detection and mitigation |
US10855700B1 (en) | 2017-06-29 | 2020-12-01 | Fireeye, Inc. | Post-intrusion detection of cyber-attacks during lateral movement within networks |
US10893068B1 (en) | 2017-06-30 | 2021-01-12 | Fireeye, Inc. | Ransomware file modification prevention technique |
US10970449B2 (en) * | 2017-09-20 | 2021-04-06 | International Business Machines Corporation | Learning framework for software-hardware model generation and verification |
US10747872B1 (en) | 2017-09-27 | 2020-08-18 | Fireeye, Inc. | System and method for preventing malware evasion |
US10805346B2 (en) | 2017-10-01 | 2020-10-13 | Fireeye, Inc. | Phishing attack detection |
US11108809B2 (en) | 2017-10-27 | 2021-08-31 | Fireeye, Inc. | System and method for analyzing binary code for malware classification using artificial neural network techniques |
US11005860B1 (en) | 2017-12-28 | 2021-05-11 | Fireeye, Inc. | Method and system for efficient cybersecurity analysis of endpoint events |
US11271955B2 (en) | 2017-12-28 | 2022-03-08 | Fireeye Security Holdings Us Llc | Platform and method for retroactive reclassification employing a cybersecurity-based global data store |
US11240275B1 (en) | 2017-12-28 | 2022-02-01 | Fireeye Security Holdings Us Llc | Platform and method for performing cybersecurity analyses employing an intelligence hub with a modular architecture |
US10826931B1 (en) | 2018-03-29 | 2020-11-03 | Fireeye, Inc. | System and method for predicting and mitigating cybersecurity system misconfigurations |
US11558401B1 (en) | 2018-03-30 | 2023-01-17 | Fireeye Security Holdings Us Llc | Multi-vector malware detection data sharing system for improved detection |
US11003773B1 (en) | 2018-03-30 | 2021-05-11 | Fireeye, Inc. | System and method for automatically generating malware detection rule recommendations |
US10956477B1 (en) | 2018-03-30 | 2021-03-23 | Fireeye, Inc. | System and method for detecting malicious scripts through natural language processing modeling |
US11075930B1 (en) | 2018-06-27 | 2021-07-27 | Fireeye, Inc. | System and method for detecting repetitive cybersecurity attacks constituting an email campaign |
US11314859B1 (en) | 2018-06-27 | 2022-04-26 | FireEye Security Holdings, Inc. | Cyber-security system and method for detecting escalation of privileges within an access token |
US11228491B1 (en) | 2018-06-28 | 2022-01-18 | Fireeye Security Holdings Us Llc | System and method for distributed cluster configuration monitoring and management |
US11316900B1 (en) | 2018-06-29 | 2022-04-26 | FireEye Security Holdings Inc. | System and method for automatically prioritizing rules for cyber-threat detection and mitigation |
US11182473B1 (en) | 2018-09-13 | 2021-11-23 | Fireeye Security Holdings Us Llc | System and method for mitigating cyberattacks against processor operability by a guest process |
US11763004B1 (en) | 2018-09-27 | 2023-09-19 | Fireeye Security Holdings Us Llc | System and method for bootkit detection |
US11368475B1 (en) | 2018-12-21 | 2022-06-21 | Fireeye Security Holdings Us Llc | System and method for scanning remote services to locate stored objects with malware |
EP3715975B1 (en) * | 2019-03-28 | 2023-03-01 | Mitsubishi Electric R&D Centre Europe B.V. | Method and apparatus for analysing a ladder program |
US11258806B1 (en) | 2019-06-24 | 2022-02-22 | Mandiant, Inc. | System and method for automatically associating cybersecurity intelligence to cyberthreat actors |
US11556640B1 (en) | 2019-06-27 | 2023-01-17 | Mandiant, Inc. | Systems and methods for automated cybersecurity analysis of extracted binary string sets |
US11392700B1 (en) | 2019-06-28 | 2022-07-19 | Fireeye Security Holdings Us Llc | System and method for supporting cross-platform data verification |
US11886585B1 (en) | 2019-09-27 | 2024-01-30 | Musarubra Us Llc | System and method for identifying and mitigating cyberattacks through malicious position-independent code execution |
US11637862B1 (en) | 2019-09-30 | 2023-04-25 | Mandiant, Inc. | System and method for surfacing cyber-security threats with a self-learning recommendation engine |
CN111625223B (en) * | 2020-05-26 | 2023-04-28 | 中国人民解放军国防科技大学 | Software design reconstruction method based on static analysis and abstraction |
US11379468B1 (en) * | 2021-05-12 | 2022-07-05 | International Business Machines Corporation | Control flow graph refining via execution data |
US20230315412A1 (en) * | 2022-03-30 | 2023-10-05 | Microsoft Technology Licensing, Llc | Scalable behavioral interface specification checking |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4418591B2 (en) * | 1998-11-03 | 2010-02-17 | ワンスピン ソリューションズ ゲゼルシャフト ミット ベシュレンクテル ハフツング | Method and apparatus for comparing a preset characteristic of a technical system with a first characteristic |
US6292916B1 (en) * | 1998-12-10 | 2001-09-18 | Lucent Technologies Inc. | Parallel backtracing for satisfiability on reconfigurable hardware |
US6539345B1 (en) * | 1999-04-09 | 2003-03-25 | Intel Corporation | Symbolic simulation using input space decomposition via Boolean functional representation in parametric form |
US20030005407A1 (en) * | 2000-06-23 | 2003-01-02 | Hines Kenneth J. | System and method for coordination-centric design of software systems |
US7031896B1 (en) * | 2000-06-30 | 2006-04-18 | Intel Corporation | Methods for performing generalized trajectory evaluation |
US7047139B2 (en) * | 2000-12-22 | 2006-05-16 | International Business Machines Corporation | Sharing information between instances of a propositional satisfiability (SAT) problem |
US6665848B2 (en) * | 2001-01-12 | 2003-12-16 | International Business Machines Corporation | Time-memory tradeoff control in counterexample production |
US6944848B2 (en) * | 2001-05-03 | 2005-09-13 | International Business Machines Corporation | Technique using persistent foci for finite state machine based software test generation |
US7058925B2 (en) * | 2002-04-30 | 2006-06-06 | Microsoft Corporation | System and method for generating a predicate abstraction of a program |
US7089534B2 (en) * | 2002-05-01 | 2006-08-08 | International Business Machines Corporation | Model based test generation for validation of parallel and concurrent software |
US7711525B2 (en) * | 2002-05-30 | 2010-05-04 | Nec Corporation | Efficient approaches for bounded model checking |
US7653520B2 (en) * | 2002-07-19 | 2010-01-26 | Sri International | Method for combining decision procedures with satisfiability solvers |
US6957404B2 (en) * | 2002-12-20 | 2005-10-18 | International Business Machines Corporation | Model checking with layered localization reduction |
US7584455B2 (en) * | 2003-10-23 | 2009-09-01 | Microsoft Corporation | Predicate-based test coverage and generation |
US7249333B2 (en) * | 2005-01-18 | 2007-07-24 | Microsoft Corporation | Quantified boolean formula (QBF) solver |
US7930659B2 (en) * | 2005-06-03 | 2011-04-19 | Nec Laboratories America, Inc. | Software verification |
US7784035B2 (en) * | 2005-07-05 | 2010-08-24 | Nec Laboratories America, Inc. | Method for the static analysis of concurrent multi-threaded software |
-
2005
- 2005-01-21 US US11/040,409 patent/US7346486B2/en active Active
- 2005-01-21 JP JP2006551309A patent/JP2007528059A/en active Pending
- 2005-01-21 WO PCT/US2005/001963 patent/WO2005072257A2/en not_active Application Discontinuation
- 2005-01-21 AT AT05705999T patent/ATE526628T1/en not_active IP Right Cessation
- 2005-01-21 EP EP05705999A patent/EP1706833B1/en not_active Not-in-force
Also Published As
Publication number | Publication date |
---|---|
JP2007528059A (en) | 2007-10-04 |
WO2005072257A3 (en) | 2007-04-19 |
ATE526628T1 (en) | 2011-10-15 |
US20050166167A1 (en) | 2005-07-28 |
WO2005072257A2 (en) | 2005-08-11 |
EP1706833A2 (en) | 2006-10-04 |
EP1706833A4 (en) | 2010-03-17 |
US7346486B2 (en) | 2008-03-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP1706833B1 (en) | System and method for modeling, abstraction, and analysis of software | |
US7930659B2 (en) | Software verification | |
Kapur et al. | Interpolation for data structures | |
Ivančić et al. | Efficient SAT-based bounded model checking for software verification | |
Møller et al. | The pointer assertion logic engine | |
Ivancic et al. | Model checking C programs using F-Soft | |
Tristan et al. | Evaluating value-graph translation validation for LLVM | |
Navas et al. | User-definable resource bounds analysis for logic programs | |
US8131532B2 (en) | Software verification using range analysis | |
Jacobs et al. | Featherweight verifast | |
Bozga et al. | Automatic verification of integer array programs | |
Bueno et al. | euforia: Complete software model checking with uninterpreted functions | |
Shaikh et al. | Evaluation of tools and slicing techniques for efficient verification of UML/OCL class diagrams | |
Ganesh | Decision procedures for bit-vectors, arrays and integers | |
Li et al. | An explicit transition system construction approach to LTL satisfiability checking | |
Banda et al. | Analysis of linear hybrid systems in CLP | |
Fedyukovich et al. | Bridging arrays and ADTs in recursive proofs | |
Ruf et al. | Using MTBDDs for composition and model checking of real-time systems | |
Kong et al. | On accelerating smt-based bounded model checking of hstm designs | |
Legay et al. | Automatic Verification of LLVM Code | |
Cyphert et al. | Solvable Polynomial Ideals: The Ideal Reflection for Program Analysis | |
Agerholm et al. | On the verification of VDM specification and refinement with PVS | |
Balan et al. | A security analyser for finding vulnerabilities in c programs | |
Wintersteiger | Termination Analysis for Bit-Vector Programs | |
Ansotegui et al. | A Benchmark Generator for Combinatorial Testing |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20060809 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL BA HR LV MK YU |
|
DAX | Request for extension of the european patent (deleted) | ||
PUAK | Availability of information related to the publication of the international search report |
Free format text: ORIGINAL CODE: 0009015 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G06F 17/10 20060101ALI20070604BHEP Ipc: G06F 7/60 20060101ALI20070604BHEP Ipc: G06F 9/45 20060101AFI20070604BHEP Ipc: G06F 17/50 20060101ALI20070604BHEP |
|
A4 | Supplementary search report drawn up and despatched |
Effective date: 20100212 |
|
17Q | First examination report despatched |
Effective date: 20100902 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R079 Ref document number: 602005030230 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: G06F0017500000 Ipc: G06F0009450000 |
|
GRAJ | Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted |
Free format text: ORIGINAL CODE: EPIDOSDIGR1 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G06F 7/60 20060101ALI20110221BHEP Ipc: G06F 17/50 20060101ALI20110221BHEP Ipc: G06F 11/36 20060101ALN20110221BHEP Ipc: G06F 9/45 20060101AFI20110221BHEP Ipc: G06F 17/10 20060101ALI20110221BHEP |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602005030230 Country of ref document: DE Effective date: 20111222 |
|
RAP2 | Party data changed (patent owner data changed or rights of a patent transferred) |
Owner name: NEC CORPORATION |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: VDEP Effective date: 20110928 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110928 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110928 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110928 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R082 Ref document number: 602005030230 Country of ref document: DE Representative=s name: BETTEN & RESCH, DE |
|
LTIE | Lt: invalidation of european patent or patent extension |
Effective date: 20110928 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110928 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110928 Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110928 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20111229 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 526628 Country of ref document: AT Kind code of ref document: T Effective date: 20110928 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R082 Ref document number: 602005030230 Country of ref document: DE Representative=s name: PATENTANWAELTE BETTEN & RESCH, DE Effective date: 20120207 Ref country code: DE Ref legal event code: R081 Ref document number: 602005030230 Country of ref document: DE Owner name: NEC CORPORATION, JP Free format text: FORMER OWNER: NEC LABORATORIES AMERICA, INC., PRINCETON, US Effective date: 20120207 Ref country code: DE Ref legal event code: R081 Ref document number: 602005030230 Country of ref document: DE Owner name: NEC CORPORATION, JP Free format text: FORMER OWNER: NEC LABORATORIES AMERICA, INC., PRINCETON, US Effective date: 20111028 Ref country code: DE Ref legal event code: R081 Ref document number: 602005030230 Country of ref document: DE Owner name: NEC CORPORATION, JP Free format text: FORMER OWNER: NEC LABORATORIES AMERICA, INC., PRINCETON, N.J., US Effective date: 20120207 Ref country code: DE Ref legal event code: R081 Ref document number: 602005030230 Country of ref document: DE Owner name: NEC CORPORATION, JP Free format text: FORMER OWNER: NEC LABORATORIES AMERICA, INC., PRINCETON, N.J., US Effective date: 20111028 Ref country code: DE Ref legal event code: R082 Ref document number: 602005030230 Country of ref document: DE Representative=s name: BETTEN & RESCH PATENT- UND RECHTSANWAELTE PART, DE Effective date: 20120207 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110928 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20120128 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110928 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110928 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110928 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110928 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20120130 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110928 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110928 |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: 732E Free format text: REGISTERED BETWEEN 20120628 AND 20120704 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110928 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110928 Ref country code: MC Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20120131 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
26N | No opposition filed |
Effective date: 20120629 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602005030230 Country of ref document: DE Effective date: 20120629 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20120131 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20120131 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20120121 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20120108 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20111228 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110928 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20120121 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20050121 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 12 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20160112 Year of fee payment: 12 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20160120 Year of fee payment: 12 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 13 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20161215 Year of fee payment: 13 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R119 Ref document number: 602005030230 Country of ref document: DE |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20170121 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170121 Ref country code: DE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170801 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180131 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: ST Effective date: 20180928 |