US20140089872A1 - Method of proving formal test bench fault detection coverage - Google Patents
Method of proving formal test bench fault detection coverage Download PDFInfo
- Publication number
- US20140089872A1 US20140089872A1 US13/626,249 US201213626249A US2014089872A1 US 20140089872 A1 US20140089872 A1 US 20140089872A1 US 201213626249 A US201213626249 A US 201213626249A US 2014089872 A1 US2014089872 A1 US 2014089872A1
- Authority
- US
- United States
- Prior art keywords
- design
- component
- representation
- rtl
- system architecture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/30—Circuit design
- G06F30/32—Circuit design at the digital level
- G06F30/33—Design verification, e.g. functional simulation or model checking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/30—Circuit design
- G06F30/32—Circuit design at the digital level
- G06F30/33—Design verification, e.g. functional simulation or model checking
- G06F30/3323—Design verification, e.g. functional simulation or model checking using formal methods, e.g. equivalence checking or property checking
Definitions
- FIG. 1 illustrates a design to verification flow for an application-specific integrated circuit (ASIC).
- ASIC application-specific integrated circuit
- FIG. 2 illustrates some embodiments of a design to verification flow for a mutatable ASIC.
- FIG. 3 illustrates some embodiments of a design to verification flow for a mutatable ASIC for determining structural coverage.
- FIG. 4 illustrates some embodiments of a control vector configured to modify one or more components of an HDL system architecture.
- FIG. 5 illustrates some embodiments of a design to verification flow for a mutatable ASIC which is modified by a control vector.
- FIG. 6 illustrates some embodiments of a processing unit configured to determine test coverage of a design.
- FIG. 7 illustrates some embodiments of a method to determine test coverage of a design.
- FIGS. 8A-8C illustrate some embodiments of simulation versus formal verification.
- FIG. 1 illustrates a design to verification flow 100 for an application-specific integrated circuit (ASIC).
- ASIC application-specific integrated circuit
- a high-level system specification 102 is established defining the major components and interactions of the ASIC.
- a hardware description language (HDL) e.g., VHDL or Verilog
- HDL hardware description language
- RTL register transfer level
- RTL design 104 RTL is a design abstraction which models logic components of the ASIC in terms of hardware registers which synchronize logic component operation.
- a test bench suite 106 is created comprising a comprehensive set of test stimuli, or test cases, to verify operation of the RTL design 104 per the high-level system specification 102 .
- the RTL design 104 is simulated 108 to validate the functionality of the test bench suite 106 against the high-level system specification 102 .
- the RTL design 104 is synthesized to produce a physical system architecture comprising a functional representation of the design for formal verification 110 .
- the functional representation of the design may be at the transistor level or gate level of the design, and describes the function and interaction of the various components of the ASIC (e.g., an extracted netlist).
- Formal verification 110 relies on mathematically proving that a certain behavior which can be described in Boolean logic holds on a Boolean logic representation of the design.
- Formal verification 110 also relies on a formal verification tool to synthesize the RTL design 104 into a Boolean representation, and to use mathematical techniques to reduce the size of the design using logical equivalence techniques and other mathematical methods.
- the formal tool then takes the behavior described (i.e., a property), and mathematically proves that the property either holds or fails on the design representation, implicitly trying all possible input combinations to the design.
- the mathematical reduction of the Boolean representation of the design creates a representation of the RTL design, rather than the RTL design itself, wherein some structural information about the design may be lost. As a consequence, it is not possible to get statement coverage of functional aspects of the RTL design 104 directly from a proof, given that all possible input stimuli have been tried, which is the equivalent to simulating all possible vector combinations on the reduced design.
- some aspects of the present disclosure provide for a system and method to discover which parts of a design a formal test suite can detect faults in, and thus how much of a design structure is covered by a property set.
- a mutatable RTL design is defined which allows for modification of a part of an RTL design from its intended behavior to a non-intended behavior, thus introducing unwanted effects.
- the mutatable RTL design can then be synthesized to produce a functional representation of the design.
- the property set can be re-run on the synthesized design to see whether the functional representation of the design is sensitive to the unwanted effect and thus whether formal verification can detect the modification.
- FIG. 2 illustrates some embodiments of a design to verification flow 200 for a mutatable ASIC.
- a high-level system specification 102 is established and defined as an RTL abstraction (or RTL design) 104 in HDL.
- a mutatable RTL design 202 is defined by augmenting the RTL design 104 , allowing for modification of one or more components of the RTL design 104 .
- a respective component of the mutatable RTL design 202 is modified by selecting a fault 204 to be introduced into the mutatable RTL design 202 .
- a signal X may be assigned with signal Y, modifying the code so that it is now assigned 0 at all times, or changing an AND operation to an OR operation.
- a test bench suite 106 is created comprising a comprehensive set of test cases to verify operation of the ASIC.
- the mutatable RTL design 202 is simulated 108 to validate the functionality of the test cases against the high-level system specification 102 , specifically to determine if selection of the fault 204 is detected.
- the mutatable RTL design 202 is then synthesized to produce a Boolean logic representation of the mutatable RTL design 202 during formal verification 110 , to determine if selection of the fault 204 is detected. If selection of the fault 204 is detected in formal verification 110 , then the structure associated with the fault is covered in the test bench suite.
- FIG. 3 illustrates some embodiments of a design to verification flow 300 for a mutatable ASIC for determining structural coverage.
- a high-level system specification 102 and associated abstract representation as an RTL design 104 is defined in HDL code.
- a mutatable RTL design 202 is defined by augmenting the RTL design 104 with a control vector configured to modify one or more components of the RTL design 104 (e.g., modifying various segments of HDL code).
- the control vector is enabled to modify a respective component of the mutatable RTL design 202 , selecting a fault 204 to introduce into the mutatable RTL design 202 .
- the mutatable RTL design 202 is simulated 108 to validate the functionality of test cases created in the test bench suite 106 to determine if selection of the fault 204 is detected.
- the mutatable RTL design 202 is then synthesized 302 during formal verification 110 , producing a Boolean logic representation of the mutatable RTL design 202 .
- Formal verification further 110 comprises a mathematical reduction 304 of the Boolean logic representation of the mutatable RTL design 202 through logical equivalence techniques and other mathematical methods.
- a proof of properties 306 is then performed, mathematically proving that a property holds for the Boolean logic representation of the design synthesized from the mutatable RTL design 202 , to determine if selection of the fault 204 is detected.
- This process may be repeated for various components (e.g., modifying various segments of a line of HDL code) by iteratively modifying each component of the mutatable RTL design 202 one at a time, leaving remaining components unmodified, synthesizing the mutatable RTL design 202 into a plurality of Boolean logic representations of the design, one for each modified component, and mathematically proving that a set of properties hold for each respective Boolean logic representation of the design by a proof of properties 306 .
- various components e.g., modifying various segments of a line of HDL code
- a formal comparison 308 may then be made between results of simulation 108 and results of the proof of properties 306 for each respective Boolean logic representation of the design to determine which component modifications are not detected by the proof of properties 306 , to further determine which components of the RTL design 104 are not covered in formal verification 110 .
- the test bench suite 106 may then be refined to provide additional coverage.
- FIG. 4 illustrates some embodiments of a control vector 410 configured to modify one or more components of a component of an HDL system architecture 400 .
- An RTL design 404 is represented in HDL code 402 .
- a first feature configured to modify a respective component of the RTL design 404 is defined.
- the first feature comprises a fault identification instrumentation 406 of the RTL design 404 , which utilizes the control vector 410 configured to modify various segments of the HDL code 402 , a respective segment of HDL code 402 comprising a component of the design.
- a second feature comprising an entity identification instrumentation 408 of the RTL design 404 is also defined, which utilizes a location vector 412 configured to identify a sub-module of the design comprising one or more components, and to identify all respective components within the sub-module.
- the RTL design 404 is then augmented with the fault identification instrumentation 406 and the entity identification instrumentation 408 .
- the control vector 410 comprises a binary vector comprising a number of n bits that is equal to a number of components of the RTL design 404 , wherein a respective component comprises one or more devices, or a portion of a device (e.g., a transistor, collection of transistors, or sub-device such as gate, source, drain, or contact).
- the n bits of the control vector are configured to be modified such that a bit value of zero leaves a corresponding component unaffected, and a bit value of non-zero modifies the corresponding component.
- a component of the design is modified by introducing a fault into the component, wherein the fault models an effect of a manufacturing defect to the component, and the effect of a manufacturing defect on the component is to produce an output that is not anticipated from an input.
- the location vector 412 comprises a binary vector comprising m bits that is equal to a number of sub-modules of the RTL design 404 . In a preferred embodiment the location vector 412 is contained within the control vector 410 .
- the synthesized design works as specified. If a mutation is enabled by driving one of the n bits of the control vector 410 to 1, then faults can be injected into the RTL design 404 to mimic stuck at faults, incorrect gate logic (e.g., an AND rather than an OR) and other such faults.
- the property set is then re-run on the synthesized design with the mutation enabled. If the property set fails, then the mutation has been detected and the property set can find the fault.
- the same mutations can be injected into the RTL design 404 which is simulated, and the simulation test bench is then run to see whether the simulation based tests detect the fault.
- the results from the formal run and the coverage obtained can then be compared to the same coverage obtained during simulation, and thus a combined coverage figure for the whole verification approach obtained.
- the advantage of this approach is that it is automated, and provides an objective measure of coverage which can be compared and merged between formal and simulation based verification.
- FIG. 5 illustrates some embodiments of a design to verification flow 500 for a mutatable ASIC which is modified by a control vector 410 .
- a mutatable RTL design 202 is defined by augmenting the RTL design 104 with a control vector 410 configured to modify one or more components of the RTL design 104 .
- the control vector 410 comprises a binary vector comprising n bits that is equal to a number of components of the mutatable RTL design 202 , which are configured to modify a respective component of the mutatable RTL design 202 when an associated respective bit is set to non-zero, introducing a fault into the respective component 504 .
- control vector 410 is configured to identify a sub-module of the mutatable RTL design 202 comprising one or more components 502 , prior to identifying a respective component 504 within the sub-module.
- the mutatable RTL design 202 is then simulated 108 , and synthesized for formal verification 110 to determine if the fault to the respective component 504 is detected.
- FIG. 6 illustrates some embodiments of a processing unit configured to determine test coverage of a design comprising an HLD interface 602 , a test bench 604 , an instrumentation tool 606 , a simulator 608 , a formal verification tool 610 , and a comparator 612 .
- the HLD interface 602 is configured to define an ASIC system architecture at the RTL level abstraction (i.e., RTL design).
- the test bench 604 is configured to define a test suite comprising a plurality of test cases configured to test functionality of the RTL design against a system specification.
- the simulator 608 is configured to simulate the plurality of test cases against the RTL design to verify functionality against the system specification.
- the instrumentation tool 606 is configured to modify an RTL design coded within the HLD interface 602 , augmenting the RTL design by instrumenting lines of the HDL code such that a control vector 410 can be injected into the RTL design to alter a segment of the HDL code.
- the control vector 410 is configured to modify components of the RTL to mimic manufacturing faults.
- the formal verification tool 610 is configured to synthesize the RTL design into a physical representation of the design comprising a Boolean logic representation of the design, reduce the Boolean logic representation of the design through mathematical equivalencies, and mathematically prove that a property holds for the Boolean logic representation of the design.
- the comparator 612 is configured to compare results of the simulation of the plurality of test cases against the RTL design to the mathematical proof of one or more properties of the Boolean logic representation of the design to determine if a modification to the RTL design is detected by the mathematical proof of the formal verification tool 610 .
- FIG. 7 illustrates some embodiments of a method 700 to determine test coverage of a design. It will be appreciated that while the method 700 is illustrated and described as a series of acts or events, that the illustrated ordering of such acts or events are not to be interpreted in a limiting sense. For example, some acts may occur in different orders and/or concurrently with other acts or events apart from those illustrated and/or described herein. In addition, not all illustrated acts may be required to implement one or more aspects or embodiments of the disclosure herein. Also, one or more of the acts depicted herein may be carried out in one or more separate acts and/or phases. Furthermore, the disclosed methods may be implemented as an apparatus, or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to implement the disclosed subject matter.
- an HDL representation (e.g., VHDL or Verilog) of a design is constructed based upon a high-level system specification.
- the HDL representation is used to define aspects of a functioning ASIC system at the register transfer level (RTL) level of abstraction, or RTL design.
- RTL register transfer level
- the RTL design is augmented by modifying lines of the HDL code such that a control vector can be injected into the RTL design to modify a segment of the HDL code.
- the control vector is configured to modify components of the RTL design to mimic manufacturing defects.
- the augmentation of the RTL design in 704 results in a mutatable RTL design, which may be modified by the control vector.
- the control vector is further configured to identify a sub-module of the mutatable RTL design comprising one or more components, and to identify all respective components within the sub-module.
- a sub-module is identified within the mutatable RTL design containing a component (e.g., a device or sub-device) wherein a manufacturing defect is to be modeled.
- a component e.g., a device or sub-device
- control vector is configured to adjust a respective bit value from zero to non-zero, wherein the bit value corresponds to the respective component within the sub-module, introducing a fault into the respective component.
- simulation test bench code comprising test cases are simulated against the mutatable RTL design to validate the functionality of the test cases against the high-level system specification.
- the results of the simulation of the test bench code are evaluated to determine if the fault is detected by the simulation.
- the mutatable RTL design is synthesized in a formal verification tool to produce a Boolean logic representation of all or a part of the RTL design.
- Boolean logic representation is reduced through logical equivalence techniques and other mathematical methods.
- a proof of properties is performed on the reduced Boolean logic representation to mathematically prove that a respective property holds.
- the proof of properties is examined to determine if the fault is detected by the proof of properties.
- test bench code may then be refined to provide additional coverage, if desired.
- FIG. 8A illustrates some embodiments physical system architecture 800 A of a 4-to-1 XOR tree coupled to an inverter.
- the physical system architecture 800 A comprises a first XOR gate 802 A, a second XOR gate 804 A, a third XOR gate 806 A, and an inverter 808 A in the arrangement shown.
- an RTL level of abstraction (i.e., RTL design) of physical system architecture 800 A comprises HLD code further comprising three XOR functions configured function on a rising clock edge, wherein a first respective output of a first XOR function corresponding to the first XOR gate 802 A, and a second respective output of a second XOR function corresponding to the second XOR gate 804 A, form first and second respective inputs to a third XOR function corresponding to the third XOR gate 806 A.
- a third output of the third XOR function forms a third input to an inverter function corresponding to the inverter 808 A.
- a test suite comprising a plurality of test cases or test vectors configured to test functionality of the RTL design further comprises vector1-vector16 illustrated in the table of FIG. 8C , wherein a respective test case comprises a 4 bit binary vector, a respective bit configured to serve as an input to the HLD code (i.e., in1-in4).
- a simulation of the RTL design comprises stimulating the HLD code with the test cases bits in columns in1-in4 as inputs to the code, and cataloging the outputs (i.e., out1).
- the out1 column of the table of FIG. 8C comprises an HLD simulation of a defect-free RTL design of the physical system architecture 800 A.
- FIG. 8A the out1 column of the table of FIG. 8C comprises an HLD simulation of a defect-free RTL design of the physical system architecture 800 A.
- the out2 column of the table of FIG. 8C comprises an HLD simulation of the RTL design wherein a fault 810 A has been introduced along an intermediate channel 812 A between the second output of the second XOR gate 804 A and the second input of the third XOR gate 806 A.
- the fault 810 A comprises a “stuck-at-1” fault, wherein the value of the second input of the third XOR gate 806 A is “stuck” at 1 regardless of a value of the second output of the second XOR gate 804 A.
- the fault 810 A comprises a physical defect in the physical system architecture 800 A, and may be captured in the RTL design by updating the HLD code.
- the test suite comprising vector1-vector16 is exhaustive, and comprises a set of properties that can be simulated against the RTL design, or tested against a synthesized design.
- simulating vector1-vector16 against the RTL design comprises a complete set of properties because outputs of the RTL have been tested over all possible input conditions.
- the synthesized design may also be completely verified against vector1-vector16, either by directly testing vector1-vector16 or through mathematical proof in this simplified example.
- testing the physical system architecture against all possible test vector combinations would prove computationally impractical if not impossible for most ASIC applications comprising greater than approximately 10 6 gates.
- FIG. 8B illustrates some embodiments of a synthesized Boolean logic representation 800 B of the RTL design (i.e., the physical system architecture 800 A).
- a simple example of a mathematical technique to reduce the size of the RTL design using logical equivalence techniques comprises reducing the physical system architecture 800 A of the 4-to-1 XOR tree coupled to an inverter to a 4-input XNOR.
- This simplified example is utilized in the embodiments of the Boolean logic representation 800 B to illustrate that structural information about the intermediate channel 812 A, inverter 808 a , and other intermediate channels have been lost in this representation.
- a mathematical proof of the properties on this (i.e., reduced) Boolean logic representation 800 B of the RTL design will not comprise any structural information about the intermediate channel 812 A where a manufacturing defect modeled by the fault 810 A occurs.
- the out1 column of the table of FIG. 8C comprises the outputs of the mathematical proof of properties of the (defect-free) Boolean logic representation 800 B of the physical system architecture 800 A.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Geometry (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Tests Of Electronic Circuits (AREA)
Abstract
Description
- State of the art formal verification methods do not yield coverage metrics which provide a quantitative measurement of how much of a register transfer level (RTL) design has been exercised during verification. Properties can be gauged for completeness to determine if all outputs of a design have been tested over all possible input conditions. However, there is no analogous metric to provide a statement of structural coverage to determine if all of the lines of a hardware description language (HDL) representation of an RTL design have been executed during operation. Such a metric would be directly comparable to simulation metrics to provide a statement of structural coverage.
-
FIG. 1 illustrates a design to verification flow for an application-specific integrated circuit (ASIC). -
FIG. 2 illustrates some embodiments of a design to verification flow for a mutatable ASIC. -
FIG. 3 illustrates some embodiments of a design to verification flow for a mutatable ASIC for determining structural coverage. -
FIG. 4 illustrates some embodiments of a control vector configured to modify one or more components of an HDL system architecture. -
FIG. 5 illustrates some embodiments of a design to verification flow for a mutatable ASIC which is modified by a control vector. -
FIG. 6 illustrates some embodiments of a processing unit configured to determine test coverage of a design. -
FIG. 7 illustrates some embodiments of a method to determine test coverage of a design. -
FIGS. 8A-8C illustrate some embodiments of simulation versus formal verification. - The description herein is made with reference to the drawings, wherein like reference numerals are generally utilized to refer to like elements throughout, and wherein the various structures are not necessarily drawn to scale. In the following description, for purposes of explanation, numerous specific details are set forth in order to facilitate understanding. It may be evident, however, to one skilled in the art, that one or more aspects described herein may be practiced with a lesser degree of these specific details. In other instances, known structures and devices are shown in block diagram form to facilitate understanding.
-
FIG. 1 illustrates a design toverification flow 100 for an application-specific integrated circuit (ASIC). First, a high-level system specification 102 is established defining the major components and interactions of the ASIC. Second, a hardware description language (HDL) (e.g., VHDL or Verilog) is used to define aspects of a processor system architecture of the ASIC system at the register transfer level (RTL) level of abstraction, or RTLdesign 104. RTL is a design abstraction which models logic components of the ASIC in terms of hardware registers which synchronize logic component operation. This is a higher level abstraction for ASIC definition than the transistor level, wherein every device is modeled, or at the gate level, wherein sub-devices (e.g., source, drain, gate, contacts, etc.) are modeled. The transistor level of abstraction and gate level of abstraction provide more modeling accuracy, but are computationally impractical for most ASIC applications. Simultaneous to the RTLdesign 104, atest bench suite 106 is created comprising a comprehensive set of test stimuli, or test cases, to verify operation of the RTLdesign 104 per the high-level system specification 102. Next, the RTLdesign 104 is simulated 108 to validate the functionality of thetest bench suite 106 against the high-level system specification 102. Finally, the RTLdesign 104 is synthesized to produce a physical system architecture comprising a functional representation of the design forformal verification 110. The functional representation of the design may be at the transistor level or gate level of the design, and describes the function and interaction of the various components of the ASIC (e.g., an extracted netlist). -
Formal verification 110 relies on mathematically proving that a certain behavior which can be described in Boolean logic holds on a Boolean logic representation of the design.Formal verification 110 also relies on a formal verification tool to synthesize the RTLdesign 104 into a Boolean representation, and to use mathematical techniques to reduce the size of the design using logical equivalence techniques and other mathematical methods. The formal tool then takes the behavior described (i.e., a property), and mathematically proves that the property either holds or fails on the design representation, implicitly trying all possible input combinations to the design. The mathematical reduction of the Boolean representation of the design creates a representation of the RTL design, rather than the RTL design itself, wherein some structural information about the design may be lost. As a consequence, it is not possible to get statement coverage of functional aspects of the RTLdesign 104 directly from a proof, given that all possible input stimuli have been tried, which is the equivalent to simulating all possible vector combinations on the reduced design. - Accordingly, some aspects of the present disclosure provide for a system and method to discover which parts of a design a formal test suite can detect faults in, and thus how much of a design structure is covered by a property set. A mutatable RTL design is defined which allows for modification of a part of an RTL design from its intended behavior to a non-intended behavior, thus introducing unwanted effects. The mutatable RTL design can then be synthesized to produce a functional representation of the design. The property set can be re-run on the synthesized design to see whether the functional representation of the design is sensitive to the unwanted effect and thus whether formal verification can detect the modification.
-
FIG. 2 illustrates some embodiments of a design toverification flow 200 for a mutatable ASIC. A high-level system specification 102 is established and defined as an RTL abstraction (or RTL design) 104 in HDL. A mutatable RTLdesign 202 is defined by augmenting the RTLdesign 104, allowing for modification of one or more components of the RTLdesign 104. A respective component of the mutatable RTLdesign 202 is modified by selecting afault 204 to be introduced into the mutatable RTLdesign 202. For example, a signal X may be assigned with signal Y, modifying the code so that it is now assigned 0 at all times, or changing an AND operation to an OR operation. Simultaneous to defining the mutatable RTLdesign 202, atest bench suite 106 is created comprising a comprehensive set of test cases to verify operation of the ASIC. - The mutatable RTL
design 202 is simulated 108 to validate the functionality of the test cases against the high-level system specification 102, specifically to determine if selection of thefault 204 is detected. The mutatable RTLdesign 202 is then synthesized to produce a Boolean logic representation of the mutatable RTLdesign 202 duringformal verification 110, to determine if selection of thefault 204 is detected. If selection of thefault 204 is detected informal verification 110, then the structure associated with the fault is covered in the test bench suite. -
FIG. 3 illustrates some embodiments of a design toverification flow 300 for a mutatable ASIC for determining structural coverage. A high-level system specification 102 and associated abstract representation as an RTLdesign 104 is defined in HDL code. A mutatable RTLdesign 202 is defined by augmenting the RTLdesign 104 with a control vector configured to modify one or more components of the RTL design 104 (e.g., modifying various segments of HDL code). The control vector is enabled to modify a respective component of the mutatable RTLdesign 202, selecting afault 204 to introduce into the mutatable RTLdesign 202. The mutatable RTLdesign 202 is simulated 108 to validate the functionality of test cases created in thetest bench suite 106 to determine if selection of thefault 204 is detected. The mutatable RTLdesign 202 is then synthesized 302 duringformal verification 110, producing a Boolean logic representation of the mutatable RTLdesign 202. - Formal verification further 110 comprises a mathematical reduction 304 of the Boolean logic representation of the mutatable RTL
design 202 through logical equivalence techniques and other mathematical methods. A proof ofproperties 306 is then performed, mathematically proving that a property holds for the Boolean logic representation of the design synthesized from the mutatable RTLdesign 202, to determine if selection of thefault 204 is detected. This process may be repeated for various components (e.g., modifying various segments of a line of HDL code) by iteratively modifying each component of the mutatable RTLdesign 202 one at a time, leaving remaining components unmodified, synthesizing the mutatable RTLdesign 202 into a plurality of Boolean logic representations of the design, one for each modified component, and mathematically proving that a set of properties hold for each respective Boolean logic representation of the design by a proof ofproperties 306. Aformal comparison 308 may then be made between results ofsimulation 108 and results of the proof ofproperties 306 for each respective Boolean logic representation of the design to determine which component modifications are not detected by the proof ofproperties 306, to further determine which components of the RTLdesign 104 are not covered informal verification 110. Thetest bench suite 106 may then be refined to provide additional coverage. -
FIG. 4 illustrates some embodiments of acontrol vector 410 configured to modify one or more components of a component of anHDL system architecture 400. An RTLdesign 404 is represented inHDL code 402. A first feature configured to modify a respective component of the RTLdesign 404 is defined. The first feature comprises afault identification instrumentation 406 of the RTLdesign 404, which utilizes thecontrol vector 410 configured to modify various segments of theHDL code 402, a respective segment ofHDL code 402 comprising a component of the design. A second feature comprising anentity identification instrumentation 408 of the RTLdesign 404 is also defined, which utilizes alocation vector 412 configured to identify a sub-module of the design comprising one or more components, and to identify all respective components within the sub-module. The RTLdesign 404 is then augmented with thefault identification instrumentation 406 and theentity identification instrumentation 408. - For the embodiments of the
HDL system architecture 400, thecontrol vector 410 comprises a binary vector comprising a number of n bits that is equal to a number of components of theRTL design 404, wherein a respective component comprises one or more devices, or a portion of a device (e.g., a transistor, collection of transistors, or sub-device such as gate, source, drain, or contact). The n bits of the control vector are configured to be modified such that a bit value of zero leaves a corresponding component unaffected, and a bit value of non-zero modifies the corresponding component. A component of the design is modified by introducing a fault into the component, wherein the fault models an effect of a manufacturing defect to the component, and the effect of a manufacturing defect on the component is to produce an output that is not anticipated from an input. Similarly, thelocation vector 412 comprises a binary vector comprising m bits that is equal to a number of sub-modules of theRTL design 404. In a preferred embodiment thelocation vector 412 is contained within thecontrol vector 410. - If design mutations are disabled (i.e., the
control vector 410 is driven to disable all of the faults by setting all n bits equal to zero), then the synthesized design works as specified. If a mutation is enabled by driving one of the n bits of thecontrol vector 410 to 1, then faults can be injected into theRTL design 404 to mimic stuck at faults, incorrect gate logic (e.g., an AND rather than an OR) and other such faults. The property set is then re-run on the synthesized design with the mutation enabled. If the property set fails, then the mutation has been detected and the property set can find the fault. - The same mutations can be injected into the
RTL design 404 which is simulated, and the simulation test bench is then run to see whether the simulation based tests detect the fault. Thus the results from the formal run and the coverage obtained can then be compared to the same coverage obtained during simulation, and thus a combined coverage figure for the whole verification approach obtained. The advantage of this approach is that it is automated, and provides an objective measure of coverage which can be compared and merged between formal and simulation based verification. -
FIG. 5 illustrates some embodiments of a design toverification flow 500 for a mutatable ASIC which is modified by acontrol vector 410. Amutatable RTL design 202 is defined by augmenting theRTL design 104 with acontrol vector 410 configured to modify one or more components of theRTL design 104. Thecontrol vector 410 comprises a binary vector comprising n bits that is equal to a number of components of themutatable RTL design 202, which are configured to modify a respective component of themutatable RTL design 202 when an associated respective bit is set to non-zero, introducing a fault into therespective component 504. In some embodiments thecontrol vector 410 is configured to identify a sub-module of themutatable RTL design 202 comprising one ormore components 502, prior to identifying arespective component 504 within the sub-module. Themutatable RTL design 202 is then simulated 108, and synthesized forformal verification 110 to determine if the fault to therespective component 504 is detected. -
FIG. 6 illustrates some embodiments of a processing unit configured to determine test coverage of a design comprising anHLD interface 602, atest bench 604, aninstrumentation tool 606, asimulator 608, aformal verification tool 610, and acomparator 612. TheHLD interface 602 is configured to define an ASIC system architecture at the RTL level abstraction (i.e., RTL design). Thetest bench 604 is configured to define a test suite comprising a plurality of test cases configured to test functionality of the RTL design against a system specification. Thesimulator 608 is configured to simulate the plurality of test cases against the RTL design to verify functionality against the system specification. - The
instrumentation tool 606 is configured to modify an RTL design coded within theHLD interface 602, augmenting the RTL design by instrumenting lines of the HDL code such that acontrol vector 410 can be injected into the RTL design to alter a segment of the HDL code. Thecontrol vector 410 is configured to modify components of the RTL to mimic manufacturing faults. Theformal verification tool 610 is configured to synthesize the RTL design into a physical representation of the design comprising a Boolean logic representation of the design, reduce the Boolean logic representation of the design through mathematical equivalencies, and mathematically prove that a property holds for the Boolean logic representation of the design. Thecomparator 612 is configured to compare results of the simulation of the plurality of test cases against the RTL design to the mathematical proof of one or more properties of the Boolean logic representation of the design to determine if a modification to the RTL design is detected by the mathematical proof of theformal verification tool 610. -
FIG. 7 illustrates some embodiments of amethod 700 to determine test coverage of a design. It will be appreciated that while themethod 700 is illustrated and described as a series of acts or events, that the illustrated ordering of such acts or events are not to be interpreted in a limiting sense. For example, some acts may occur in different orders and/or concurrently with other acts or events apart from those illustrated and/or described herein. In addition, not all illustrated acts may be required to implement one or more aspects or embodiments of the disclosure herein. Also, one or more of the acts depicted herein may be carried out in one or more separate acts and/or phases. Furthermore, the disclosed methods may be implemented as an apparatus, or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to implement the disclosed subject matter. - At 702 an HDL representation (e.g., VHDL or Verilog) of a design is constructed based upon a high-level system specification. The HDL representation is used to define aspects of a functioning ASIC system at the register transfer level (RTL) level of abstraction, or RTL design.
- At 704 the RTL design is augmented by modifying lines of the HDL code such that a control vector can be injected into the RTL design to modify a segment of the HDL code. The control vector is configured to modify components of the RTL design to mimic manufacturing defects.
- At 706 the augmentation of the RTL design in 704 results in a mutatable RTL design, which may be modified by the control vector. The control vector is further configured to identify a sub-module of the mutatable RTL design comprising one or more components, and to identify all respective components within the sub-module.
- At 708 a sub-module is identified within the mutatable RTL design containing a component (e.g., a device or sub-device) wherein a manufacturing defect is to be modeled.
- At 710 the control vector is configured to adjust a respective bit value from zero to non-zero, wherein the bit value corresponds to the respective component within the sub-module, introducing a fault into the respective component.
- At 712 simulation test bench code comprising test cases are simulated against the mutatable RTL design to validate the functionality of the test cases against the high-level system specification.
- At 714 the results of the simulation of the test bench code are evaluated to determine if the fault is detected by the simulation.
- At 716 the mutatable RTL design is synthesized in a formal verification tool to produce a Boolean logic representation of all or a part of the RTL design.
- At 718 the Boolean logic representation is reduced through logical equivalence techniques and other mathematical methods.
- At 720 a proof of properties is performed on the reduced Boolean logic representation to mathematically prove that a respective property holds.
- At 722 the proof of properties is examined to determine if the fault is detected by the proof of properties.
- At 724 a formal comparison is made between results of the simulation in 712 and results of the proof of properties in 720 to determine which component modifications are not detected by the proof of properties. The test bench code may then be refined to provide additional coverage, if desired.
- For the purpose of illustrating some embodiments of simulation versus formal verification, the embodiments of
FIGS. 8A-8C are examined.FIG. 8A illustrates some embodimentsphysical system architecture 800A of a 4-to-1 XOR tree coupled to an inverter. Thephysical system architecture 800A comprises afirst XOR gate 802A, asecond XOR gate 804A, athird XOR gate 806A, and aninverter 808A in the arrangement shown. Some embodiments of an RTL level of abstraction (i.e., RTL design) ofphysical system architecture 800A comprises HLD code further comprising three XOR functions configured function on a rising clock edge, wherein a first respective output of a first XOR function corresponding to thefirst XOR gate 802A, and a second respective output of a second XOR function corresponding to thesecond XOR gate 804A, form first and second respective inputs to a third XOR function corresponding to thethird XOR gate 806A. A third output of the third XOR function forms a third input to an inverter function corresponding to theinverter 808A. - A test suite comprising a plurality of test cases or test vectors configured to test functionality of the RTL design further comprises vector1-vector16 illustrated in the table of
FIG. 8C , wherein a respective test case comprises a 4 bit binary vector, a respective bit configured to serve as an input to the HLD code (i.e., in1-in4). A simulation of the RTL design comprises stimulating the HLD code with the test cases bits in columns in1-in4 as inputs to the code, and cataloging the outputs (i.e., out1). For the embodiments ofFIG. 8A , the out1 column of the table ofFIG. 8C comprises an HLD simulation of a defect-free RTL design of thephysical system architecture 800A. Furthermore, for the embodiments ofFIG. 8A the out2 column of the table ofFIG. 8C comprises an HLD simulation of the RTL design wherein afault 810A has been introduced along anintermediate channel 812A between the second output of thesecond XOR gate 804A and the second input of thethird XOR gate 806A. Thefault 810A comprises a “stuck-at-1” fault, wherein the value of the second input of thethird XOR gate 806A is “stuck” at 1 regardless of a value of the second output of thesecond XOR gate 804A. Thefault 810A comprises a physical defect in thephysical system architecture 800A, and may be captured in the RTL design by updating the HLD code. Note that this is the effect of the instrumentation of the HLD code, and the injection of the control vector into the RTL design to alter a segment of the HDL code. The RTL design, once altered, is then simulated to provide the outputs of column out2. In this manner faults may be introduced in the HLD code comprising the RTL design of thephysical system architecture 800A. - The test suite comprising vector1-vector16 is exhaustive, and comprises a set of properties that can be simulated against the RTL design, or tested against a synthesized design. Thus, simulating vector1-vector16 against the RTL design comprises a complete set of properties because outputs of the RTL have been tested over all possible input conditions. Moreover, the synthesized design may also be completely verified against vector1-vector16, either by directly testing vector1-vector16 or through mathematical proof in this simplified example. However, testing the physical system architecture against all possible test vector combinations would prove computationally impractical if not impossible for most ASIC applications comprising greater than approximately 106 gates. Moreover, a mathematic proof of the properties comprising exercising vector1-vector16 on a Boolean representation of the RTL design would occur on a representation of the RTL design, rather than the RTL design itself, wherein some structural information about the design is be lost during mathematical reduction. So while all possible inputs of the properties are proved, not all possible structural features are included in the representation of the RTL design.
-
FIG. 8B illustrates some embodiments of a synthesizedBoolean logic representation 800B of the RTL design (i.e., thephysical system architecture 800A). A simple example of a mathematical technique to reduce the size of the RTL design using logical equivalence techniques comprises reducing thephysical system architecture 800A of the 4-to-1 XOR tree coupled to an inverter to a 4-input XNOR. This simplified example is utilized in the embodiments of theBoolean logic representation 800B to illustrate that structural information about theintermediate channel 812A, inverter 808 a, and other intermediate channels have been lost in this representation. Therefore, a mathematical proof of the properties on this (i.e., reduced)Boolean logic representation 800B of the RTL design will not comprise any structural information about theintermediate channel 812A where a manufacturing defect modeled by thefault 810A occurs. For the embodiments of theFIG. 8B , the out1 column of the table ofFIG. 8C comprises the outputs of the mathematical proof of properties of the (defect-free)Boolean logic representation 800B of thephysical system architecture 800A. - For some physical system architecture embodiments it may be possible to simulate every possible input sequence. However, there may be logic in the design which cannot be stimulated, either because it is redundant, and therefore does not contribute to the overall function and could be removed, or because it was misconnected in some way, and although you have stimulated the logic around it, the input sequence of vectors was not good enough to propagate the failure caused to somewhere where it could be observed. Therefore, when looking at an RTL representation of a physical system architecture, this method allows one to know which lines of the RTL design have been executed, in order to search for redundant code, or code which should have been stimulated and was not executed.
- It will be appreciated that equivalent alterations and/or modifications may occur to those skilled in the art based upon a reading and/or understanding of the specification and annexed drawings. The disclosure herein includes all such modifications and alterations and is generally not intended to be limited thereby. For example, although the figures provided herein, are illustrated and described to have a particular doping type, it will be appreciated that alternative doping types may be utilized as will be appreciated by one of ordinary skill in the art.
- In addition, while a particular feature or aspect may have been disclosed with respect to only one of several implementations, such feature or aspect may be combined with one or more other features and/or aspects of other implementations as may be desired. Furthermore, to the extent that the terms “includes”, “having”, “has”, “with”, and/or variants thereof are used herein, such terms are intended to be inclusive in meaning—like “comprising.” Also, “exemplary” is merely meant to mean an example, rather than the best. It is also to be appreciated that features, layers and/or elements depicted herein are illustrated with particular dimensions and/or orientations relative to one another for purposes of simplicity and ease of understanding, and that the actual dimensions and/or orientations may differ substantially from that illustrated herein.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/626,249 US8689155B1 (en) | 2012-09-25 | 2012-09-25 | Method of proving formal test bench fault detection coverage |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/626,249 US8689155B1 (en) | 2012-09-25 | 2012-09-25 | Method of proving formal test bench fault detection coverage |
Publications (2)
Publication Number | Publication Date |
---|---|
US20140089872A1 true US20140089872A1 (en) | 2014-03-27 |
US8689155B1 US8689155B1 (en) | 2014-04-01 |
Family
ID=50340223
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/626,249 Active US8689155B1 (en) | 2012-09-25 | 2012-09-25 | Method of proving formal test bench fault detection coverage |
Country Status (1)
Country | Link |
---|---|
US (1) | US8689155B1 (en) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150121323A1 (en) * | 2013-10-24 | 2015-04-30 | International Business Machines Corporation | Determining a quality parameter for a verification environment |
US20160124824A1 (en) * | 2014-10-31 | 2016-05-05 | Yogitech S.P.A. | Method for measuring the effect of microscopic hardware faults in high-complexity applications implemented in a hardware electronic system, and corresponding system and computer program product |
US20190147121A1 (en) * | 2017-11-14 | 2019-05-16 | Synopsys, Inc. | Efficient Mechanism for Interactive Fault Analysis in Formal Verification Environment |
US10885251B2 (en) * | 2019-02-22 | 2021-01-05 | Bae Systems Information And Electronic Systems Integration Inc. | Software integration into hardware verification |
CN112699023A (en) * | 2020-12-25 | 2021-04-23 | 深圳市彬讯科技有限公司 | Project interface testing method and device, computer equipment and storage medium |
US20220114312A1 (en) * | 2020-10-09 | 2022-04-14 | Xepic Corporation Limited | Method, emulator, and storage media for debugging logic system design |
US11386251B2 (en) * | 2020-09-16 | 2022-07-12 | Kioxia Corporation | Logic simulation verification system, logic simulation verification method, and program |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9760663B2 (en) | 2014-10-30 | 2017-09-12 | Synopsys, Inc. | Automatic generation of properties to assist hardware emulation |
US10789403B1 (en) | 2019-05-14 | 2020-09-29 | International Business Machines Corporation | Grouping and partitioning of properties for logic verification |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH04211871A (en) * | 1990-05-02 | 1992-08-03 | Toshiba Corp | Inspection supporting system for logic design |
US5557774A (en) * | 1993-03-22 | 1996-09-17 | Hitachi, Ltd. | Method for making test environmental programs |
US6477683B1 (en) * | 1999-02-05 | 2002-11-05 | Tensilica, Inc. | Automated processor generation system for designing a configurable processor and method for the same |
US7398263B2 (en) * | 2002-02-26 | 2008-07-08 | International Business Machines Corporation | Sequenced modification of multiple entities based on an abstract data representation |
US8244702B2 (en) * | 2002-02-26 | 2012-08-14 | International Business Machines Corporation | Modification of a data repository based on an abstract data representation |
US7155689B2 (en) * | 2003-10-07 | 2006-12-26 | Magma Design Automation, Inc. | Design-manufacturing interface via a unified model |
US7889019B2 (en) * | 2006-10-13 | 2011-02-15 | Andrew Roman Gizara | Pulse width modulation sequence generating a near critical damped step response |
-
2012
- 2012-09-25 US US13/626,249 patent/US8689155B1/en active Active
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150121323A1 (en) * | 2013-10-24 | 2015-04-30 | International Business Machines Corporation | Determining a quality parameter for a verification environment |
US9443044B2 (en) * | 2013-10-24 | 2016-09-13 | International Business Machines Corporation | Determining a quality parameter for a verification environment |
US20160124824A1 (en) * | 2014-10-31 | 2016-05-05 | Yogitech S.P.A. | Method for measuring the effect of microscopic hardware faults in high-complexity applications implemented in a hardware electronic system, and corresponding system and computer program product |
US9898379B2 (en) * | 2014-10-31 | 2018-02-20 | Intel Corporation | Method for measuring the effect of microscopic hardware faults in high-complexity applications implemented in a hardware electronic system, and corresponding system and computer program product |
US20190147121A1 (en) * | 2017-11-14 | 2019-05-16 | Synopsys, Inc. | Efficient Mechanism for Interactive Fault Analysis in Formal Verification Environment |
US11010522B2 (en) * | 2017-11-14 | 2021-05-18 | Synopsys, Inc. | Efficient mechanism for interactive fault analysis in formal verification environment |
US10885251B2 (en) * | 2019-02-22 | 2021-01-05 | Bae Systems Information And Electronic Systems Integration Inc. | Software integration into hardware verification |
US11386251B2 (en) * | 2020-09-16 | 2022-07-12 | Kioxia Corporation | Logic simulation verification system, logic simulation verification method, and program |
US20220114312A1 (en) * | 2020-10-09 | 2022-04-14 | Xepic Corporation Limited | Method, emulator, and storage media for debugging logic system design |
US11625521B2 (en) * | 2020-10-09 | 2023-04-11 | Xepic Corporation Limited | Method, emulator, and storage media for debugging logic system design |
CN112699023A (en) * | 2020-12-25 | 2021-04-23 | 深圳市彬讯科技有限公司 | Project interface testing method and device, computer equipment and storage medium |
Also Published As
Publication number | Publication date |
---|---|
US8689155B1 (en) | 2014-04-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8689155B1 (en) | Method of proving formal test bench fault detection coverage | |
EP1980964A1 (en) | Method and computer program product for performing failure mode and effects analysis of an integrated circuit | |
US20170316118A1 (en) | Graphical View And Debug For Coverage-Point Negative Hint | |
US6453437B1 (en) | Method and system for performing transition fault simulation along long circuit paths for high-quality automatic test pattern generation | |
Fujita et al. | Partial synthesis through sampling with and without specification | |
US9626468B2 (en) | Assertion extraction from design and its signal traces | |
US11789077B2 (en) | Single-pass diagnosis for multiple chain defects | |
Gielen et al. | Review of methodologies for pre-and post-silicon analog verification in mixed-signal SOCs | |
US6691078B1 (en) | Target design model behavior explorer | |
Ahmad et al. | Adding pseudo-random test sequence generator in the test simulator for DFT approach | |
US9404972B2 (en) | Diagnosis and debug with truncated simulation | |
Ye et al. | Diagnose failures caused by multiple locations at a time | |
JP2001052043A (en) | Error diagnosis method and error site proving method for combinational verification | |
Guerreiro et al. | Analogue and mixed-signal production test speed-up by means of fault list compression | |
Dehbashi et al. | Debug automation for logic circuits under timing variations | |
JP2022166154A (en) | System and method for analyzing formal fault propagation | |
Chou et al. | Finding reset nondeterminism in RTL designs-scalable X-analysis methodology and case study | |
Tam et al. | SLIDER: A fast and accurate defect simulation framework | |
US10769332B2 (en) | Automatic simulation failures analysis flow for functional verification | |
Choudhary et al. | Trace signal selection methods for post silicon debugging | |
Raik et al. | Automated design error debug using high-level decision diagrams and mutation operators | |
US20180189435A1 (en) | System and method for capturing transaction specific stage-wise log data | |
Xu et al. | Mutant generation for analog circuit designs | |
Bhandari et al. | LLM-Aided Testbench Generation and Bug Detection for Finite-State Machines | |
Khatri et al. | Development and Verification of Serial Fault Simulation for FPGA Designs using the Proposed RASP-FIT Tool |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: INFINEON TECHNOLOGIES AG, GERMANY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:GALPIN, DARREN;REEL/FRAME:029020/0899 Effective date: 20120925 |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551) Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |