GB2572632A

GB2572632A - Verification of hardware design for data transformation pipeline

Info

Publication number: GB2572632A
Application number: GB1805722.4A
Authority: GB
Inventors: Elliott Sam
Original assignee: Imagination Technologies Ltd
Current assignee: Imagination Technologies Ltd
Priority date: 2018-04-05
Filing date: 2018-04-05
Publication date: 2019-10-09
Anticipated expiration: 2038-04-05
Also published as: GB201805722D0; GB2572632B

Abstract

A method is provided for verifying a hardware design for a data transformation pipeline, which comprises elements that perform data transformation on one or more inputs. Firstly, a hardware design for a base transaction module is generated, configured to generate base transaction data comprising records of inputs and outputs to the pipeline and to a set of the data transformation elements, for a base transaction. Secondly, a first version of the hardware design for the data transformation pipeline is generated, in which at least one data transformation element of the set of data transformation elements has been replaced with a deterministic black box element configured to generate outputs in accordance with the inputs and outputs recorded in the base transaction data for the corresponding data transformation element. Thirdly, the first version of the hardware design for the data transformation pipeline is linked to the hardware design for the base transaction module. Finally, it is formally verified that if an instantiation of the first version of the hardware design for the data transformation pipeline executes the base transaction, the outputs match the pipeline outputs recorded in the base transaction data.

Description

VERIFICATION OF HARDWARE DESIGN FOR DATA TRANSFORMATION PIPELINE

BACKGROUND [0001] Many electronic devices, such as systems-on-chips (SoCs), include hardware that implements a data transformation pipeline. As is known to those of skill in the art, a pipeline comprises a set of processing elements wherein the output of a processing element in the set is the input to a processing element in the set. A data transformation pipeline is a type of pipeline that is configured to perform a data transformation on a set of inputs. The term “data transformation” is used herein to mean any operation (e.g. mathematical operations such as, but not limited to, arithmetic operations including addition, subtraction, multiplication, division etc.) that can be performed on, or applied to, a set of data to produce new data. Accordingly, a data transformation pipeline receives a set of inputs and outputs the result of a data transformation performed on the set of inputs. A data transformation pipeline comprises at least one data transformation element configured to perform a data transformation on a set of inputs, but it may also include other types of processing elements. When a pipeline generates an output for a set of inputs the pipeline is said to execute a ‘transaction’. Accordingly, a pipeline is said to execute the same transaction if it generates an output for the same set of inputs.

[0002] A data transformation pipeline may be a linear or non-linear pipeline and it may have a single processing element or multiple processing elements. FIGS. 1-3 illustrate example data transformation pipelines configured to perform an arithmetic calculation on a set of inputs. FIG. 1 illustrates a first example data transformation pipeline 100 configured to calculate the sum of eight inputs (a₀ + a_± + a₂ + a₃ + a₄ + a₅ + a₆ + a₇). The data transformation pipeline 100 of FIG. 1 comprises a single data transformation element 102, a register 104 and control logic 106. The data transformation element 102 receives an input at in each of eight cycles (numbered 0 to 7) and adds the input to the sum of the previous inputs bt (i.e. the value of the register 104 in the previous cycle) and stores the new sum in the register 104. The control logic 106 is configured to provide the value of the register 104 to the data transformation element 102 in each cycle, and output the value of the register 104 after the eighth cycle (i.e. after the sum 00 + ^ + 02 + ^3 + ^4 + ^5 + ^0 + ^ai has been generated). The control logic 106 is also configured to set the value of the register 104 to 0 on reset and after the value of the register 104 is output. Where the data transformation element 102 is configured to generate the sum of the two inputs («j and b^ in a single cycle the output of the pipeline may be expected to be output after eight cycles. However, there may be random external events, such as stalls or interrupts, which affect the movement of data through the pipeline by one or more clock cycles.

[0003] FIG. 2 illustrates a second example data transformation pipeline 200 configured to calculate (a + h) * c. The data transformation pipeline 200 comprises two data transformation elements 202, 204, a register 206, and control logic 208. The first data transformation element 202 is configured to calculate the sum of the inputs a and b and store the sum in the register 206. The second data transformation element 204 is configured to calculate the product of the register 206 value (i.e. the output of the first data transformation element 202 y) and the input c. The control logic 208 controls the operation of the register 206 to ensure that the value of the register 206 is provided to the second data transformation element 204 at the correct time.

[0004] FIG. 3 illustrates a third example data transformation pipeline 300 configured to calculate (a + ύ). The data transformation pipeline 300 comprises two processing elements 302, 304, a register 306 and control logic 308. The first processing element 302 is a data transformation element configured to calculate the sum of the inputs a and b. The second processing element 304 is not a data transformation element as it does not transform the data input thereto, it simply stores the value of the register 306 (i.e. the output of the data transformation element 304) y until the result is requested by a downstream component. The control logic 308 controls the operation of the register 306 to ensure that the value of the register 306 is provided to the second processing element 304 at the correct time.

[0005] Generating hardware that implements a data transformation pipeline typically includes developing a hardware design that describes the structure and/or function of an integrated circuit that implements the data transformation pipeline; verifying or testing the hardware design to ensure that an integrated circuit manufactured according to the design will behave as expected; and once verified, manufacturing an integrated circuit, at an integrated circuit manufacturing system, in accordance with the hardware design. Verifying the hardware design for a data transformation pipeline comprises verifying that an instantiation of the hardware design produces the correct result to the data transformation regardless of when the transformation is performed. Specifically, since a data transformation pipeline may perform a transformation at any time after reset (e.g. immediately after reset, or, for example, 10,000 cycles after reset) and with a variable amount of latency (e.g. due to stalls or interrupts) it is important to verify that regardless of when the transformation is performed that an instantiation of the hardware design will produce the correct result to the transformation.

[0006] A hardware design may be verified, for example, by formal verification or simulationbased verification. Formal verification is a systematic process that uses a mathematical model of the hardware design and mathematical reasoning to verify the hardware design. In contrast, simulation-based verification is a process in which a hardware design is tested by applying stimuli to an instantiation of the hardware design and monitoring the output of the instantiation of the hardware design in response to the stimuli.

[0007] In formal verification, the hardware design is transformed into a mathematical model (e.g. a state-transition system, or a flow graph) to thereby provide an instantiation of the hardware design which can be tested to verify the hardware design, and formal properties to be verified are expressed using mathematical logic using a precise syntax or a language with a precise mathematical syntax and semantics.

[0008] Formal verification is performed using a formal verification tool (i.e. a software tool that is capable of performing formal verification of a hardware design). Some formal verification tools, such as formal equivalence checkers (e.g. Synopsys® HECTOR, , and other logical equivalence checkers (LECs) and sequential logical equivalence checkers (SLECs)) are proficient at verifying properties related to data transformations (such as arithmetic operations), but can typically only be used to prove a property over a finite period of time. Other verification tools, such as formal property checkers, which may also be referred to as formal model checkers, (e.g. OneSpin 360 DV™, Mentor Graphics Questa® Formal Verification, Synopsys® VC Formal, Cadence® Incisive® Enterprise Verifier, and JasperGold®) are proficient at verifying a property over an infinite time period but are poor at verifying properties related to data transformations (such as arithmetic operations).

[0009] Since existing formal verification tools are proficient at verifying properties related to data transformations, or proficient at verifying a property over infinite time, not both, it is difficult, if not impossible, using an existing formal verification tool to verify that an instantiation of a hardware design for a data transformation pipeline will always transform the data correctly regardless of when the transformation is performed.

[0010] The embodiments described below are provided by way of example only and are not limiting of implementations which solve any or all of the disadvantages of known methods and systems for verifying a hardware design for a data transformation pipeline.

SUMMARY [0011] This summary is provided to introduce a selection of concepts that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0012] Described herein are methods and systems for verifying that an instantiation of a hardware design for a data transformation pipeline comprising one or more data transformation elements will produce the same result to the same transaction, regardless of when the transaction is executed wherein the data transformation elements of the hardware design are deterministically black boxed. Deterministically black boxing a data transformation element means that the logic driving the output of the data transformation element (i.e. the logic performing the data transformation) is replaced with logic that does not perform the data transformation, but when presented with the same inputs produces the same output.

[0013] A first aspect provides a method of verifying a hardware design for a data transformation pipeline, the data transformation pipeline comprising one or more data transformation elements that perform a data transformation on one or more inputs, the method comprising: generating a hardware design for a base transaction module that is configured to generate base transaction data, the base transaction data comprising a record of the pipeline inputs and outputs for a base transaction and a record of a set of one or more inputs to and a set of one or more outputs from a set of the data transformation elements for the base transaction; generating a first version of the hardware design for the data transformation pipeline in which at least one data transformation element of the set of data transformation elements has been replaced with a deterministic black box element configured to generate outputs in accordance with the inputs and outputs recorded in the base transaction data for the corresponding data transformation element; linking the first version of the hardware design for the data transformation pipeline to the hardware design for the base transaction module; and formally verifying that if an instantiation of the first version of the hardware design for the data transformation pipeline executes the base transaction outputs of the instantiation of the first version of the hardware design match the pipeline outputs recorded in the base transaction data.

[0014] A second aspect provides a system for verifying a hardware design for a data transformation pipeline, the data transformation pipeline comprising one or more data transformation elements that perform a data transformation on one or more inputs, the system comprising: memory configured to store: a hardware design for a base transaction module that is configured to generate base transaction data, the base transaction data comprising a record of the pipeline inputs and outputs for a base transaction and a record of a set of one or more inputs to and a set of one or more outputs from data transformation elements in a set of the data transformation elements for the base transaction; a first version of the hardware design for the data transformation pipeline linked to the hardware design for the base transaction module, wherein in the first version of the hardware design for the data transformation pipeline at least one data transformation element in the set of data transformation elements has been replaced with a deterministic black box element which is configured to generate outputs in accordance with the inputs and outputs recorded in the base transaction data for the corresponding data transformation element; and a formal verification tool; and one or more processors configured to: formally verify, using the formal verification tool, that if an instantiation of the first version of the hardware design for the data transformation pipeline executes the base transaction outputs of the instantiation of the first version of the hardware design for the data transformation pipeline match the pipeline outputs recorded in the base transaction data.

[0015] A hardware design for a data transformation pipeline, when processed in an integrated circuit manufacturing system, configures the system to manufacture the data transformation pipeline. There may be provided a non-transitory computer readable storage medium having stored thereon a hardware design for a data transformation pipeline that, when processed in an integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture an integrated circuit embodying the data transformation pipeline.

[0016] There may be provided an integrated circuit manufacturing system comprising: a nontransitory computer readable storage medium having stored thereon a hardware design for a data transformation pipeline; a layout processing system configured to process the computer readable description so as to generate a circuit layout description of an integrated circuit embodying the data transformation pipeline; and an integrated circuit generation system configured to manufacture an integrated circuit embodying the data transformation pipeline according to the circuit layout description.

[0017] There may be provided computer program code for performing a method as described herein. There may be provided non-transitory computer readable storage medium having stored thereon computer readable instructions that, when executed at a computer system, cause the computer system to perform the methods as described herein.

[0018] The above features may be combined as appropriate, as would be apparent to a skilled person, and may be combined with any of the aspects of the examples described herein.

BRIEF DESCRIPTION OF THE DRAWINGS [0019] Examples will now be described in detail with reference to the accompanying drawings in which:

[0020] FIG. 1 is a block diagram of a first example data transformation pipeline;

[0021] FIG. 2 is a block diagram of a second example data transformation pipeline;

[0022] FIG. 3 is a block diagram of a third example data transformation pipeline;

[0023] FIG. 4 is a flow diagram of a first example method for verifying a hardware design for a data transformation pipeline;

[0024] FIG. 5 is a flow diagram of an example method for generating a hardware design for a base transaction module;

[0025] FIG. 6 is a block diagram of the example data transformation pipeline of FIG. 1 where the data transformation element has been replaced with a deterministic black box element;

[0026] FIG. 7 is a block diagram of the example data transformation pipeline of FIG. 2 where the data transformation elements have been replaced with deterministic black box elements;

[0027] FIG. 8 is a block diagram of the example data transformation pipeline of FIG. 1 where the data transformation element has been replaced with a black box element;

[0028] FIG. 9 is a block diagram of the example data transformation pipeline of FIG. 2 where the data transformation elements have been replaced with black box elements;

[0029] FIG. 10 is a block diagram of an example system for verifying a hardware design for a data transformation pipeline in accordance with the method of FIG. 4;

[0030] FIG. 11 is a flow diagram of a second example method for verifying a hardware design for a data transformation pipeline;

[0031] FIG. 12 is a block diagram of an example system for verifying a hardware design for a data transformation pipeline in accordance with the method of FIG. 11;

[0032] FIG. 13 is a block diagram of an example computing-based device;

[0033] FIG. 14 is a block diagram of an example computer system in which a data transformation pipeline may be implemented; and [0034] FIG. 15 is a block diagram of an example integrated circuit manufacturing system for generating an integrated circuit embodying a data transformation pipeline.

[0035] The accompanying drawings illustrate various examples. The skilled person will appreciate that the illustrated element boundaries (e.g., boxes, groups of boxes, or other shapes) in the drawings represent one example of the boundaries. It may be that in some examples, one element may be designed as multiple elements or that multiple elements may be designed as one element. Common reference numerals are used throughout the figures, where appropriate, to indicate similar features.

DETAILED DESCRIPTION [0036] The following description is presented by way of example to enable a person skilled in the art to make and use the invention. The present invention is not limited to the embodiments described herein and various modifications to the disclosed embodiments will be apparent to those skilled in the art. Embodiments are described by way of example only.

[0037] A “hardware design” is a description of the structure and/or function of an integrated circuit which when processed at an integrated circuit manufacturing system causes the integrated circuit manufacturing system to generate an integrated circuit described by the hardware design. For example, as described in more detail below with respect to FIG. 15, when a hardware design is processed at an integrated circuit manufacturing system the integrated circuit manufacturing system may generate the integrated circuit by synthesizing the hardware design into silicon, or, by loading configuration data into a field-programmable gate array (FPGA).

[0038] A hardware design may be implemented in a high-level hardware description language (HDL), such as, but not limited to, a register transfer level (RTL) language. Examples of register transfer level languages include, but are not limited to, VHDL (VHSIC Hardware Description Language) and Verilog®. It will be evident to a person of skill in the art that other high-level hardware description languages may be used such as proprietary highlevel hardware description languages.

[0039] An instantiation of a hardware design is a representation of the hardware and/or functionality of the hardware defined by the hardware design. An instantiation of a hardware design includes, but is not limited to, an emulation model of the hardware design that simulates the behaviour of the hardware defined by the hardware design, a synthesized version (e.g. netlist) of the hardware design, a hardware implementation (e.g. integrated circuit or a field-programmable gate array (FPGA)) of the hardware design, and a mathematical model of the hardware design generated by a formal verification tool. An instantiation of the hardware design embodies the hardware design in a form which can be tested to verify the hardware design.

[0040] A hardware design for a data transformation pipeline is thus a description of the structure and/or function of an integrated circuit to implement a data transformation pipeline which, when processed at an integrated circuit manufacturing system causes the integrated circuit manufacturing system to generate an integrated circuit that implements a data transformation pipeline. As described above, a data transformation pipeline comprises at least one data transformation element which is configured to perform a data transformation on a set of inputs. Accordingly, a hardware design for a data transformation pipeline includes a description of the structure and/or function of each data transformation element of the pipeline.

[0041] As described above, because existing formal verification tools are proficient at verifying properties related to data transformations (such as arithmetic operations), or proficient at verifying a property over infinite time, but not both, it is difficult, if not impossible, to verify, using an existing formal verification tool, that an instantiation of a hardware design for a data transformation pipeline will always transform a set of inputs correctly (i.e. execute a transaction correctly), regardless of when the transformation is performed after reset. In other words, it is difficult to verify that an instantiation of a hardware design for a data transformation pipeline will produce the correct result to a transaction executed from any state of the data transformation pipeline (e.g. a transaction executed from the state that arises after reset, or a transaction executed from the state that arises after reset, a first transaction, a second transaction and a stall etc.).

[0042] Specifically, a formal verification tool that is proficient at verifying data transformations, such as a formal equivalence checker, may be able to formally verify that an instantiation of a data transformation pipeline will produce the correct result to a data transformation performed on a set of inputs (i.e. a transaction executed) from a predetermined state of the pipeline (e.g. the state that arises after reset) under one or more predetermined conditions (e.g. no stalls or interrupts). This may be accomplished for example, by formally verifying, via such a formal verification tool, that the result of a transformation performed by an instantiation of the hardware design on a set of inputs from the predetermined state (e.g. the state that occurs after reset) under the one or more conditions matches the result of a test bench component that is configured to simulate the behaviour of the data transformation pipeline. Since the formal verification tool verifies that the outputs match for all possible sets of inputs, this verifies that the instantiation of the hardware design will produce the correct result for any set of inputs. However, this only verifies that an instantiation of the hardware design will produce the correct result when the transformation is performed from the predetermined state (e.g. the state arising from reset).

It doesn’t verify that an instantiation of the hardware design will produce the correct result if a transformation is performed on a set of inputs from another predetermined state of the pipeline (e.g. the state arising after a first transaction and a stall or the state arising after two transactions and two stalls).

[0043] To verify that an instantiation of the hardware design will produce the correct result for a transformation performed on a set of inputs (i.e. a transaction executed) from another predetermined state (e.g. a state that occurs a period of time after reset), the formal verification tool may be configured to verify the result of a transformation performed by an instantiation of the hardware design on a set of inputs from another predetermined state (e.g. the state that arises after reset, a first calculation and a stall). While verifying the result of a transformation performed from multiple states of the pipeline increases the chances of catching obscure bugs, this still only verifies that that an instantiation of the hardware design will produce the correct result to transformations performed (i.e. transactions executed) from the predetermined states (e.g. the state arising from reset, the state arising after a first transformation is performed on a set of inputs and a stall) explicitly tested, not from any other state of the pipeline.

[0044] The inventors have identified, however, that if it has been verified, via for example, a formal equivalence checker, that an instantiation of the hardware design will produce the correct result to a transformation performed on any set of inputs from a predetermined state (e.g. the state arising from reset) under a predetermined set of conditions (e.g. no stalls or interrupts once the calculation begins) that verifying that an instantiation of the hardware design will always produce the correct result to a transformation performed on any set of inputs, regardless of when the transformation is performed, can be done by verifying that the result of a transformation performed by the instantiation of the hardware design for a set of inputs from the predetermined state (e.g. the state arising from reset) matches the result of the same transformation (i.e. same inputs) performed by the instantiation of the hardware design from any other state.

[0045] However, this cannot simply be done by verifying, using a formal verification tool, such as a formal property checker, that is proficient at proving a property over infinite time that the result of a transformation performed by an instantiation of the hardware design for a set of inputs from a predetermined state matches the result of a transformation performed by an instantiation of the hardware design for the same set of inputs from any other state of the pipeline. This is because formal verification tools that are proficient at verifying a property over infinite time are not able to efficiently verify the result of data transformations and thus are not able to verify the output of the data transformation elements.

[0046] Since it can be verified, using a formal equivalence checker, that the pipeline will produce the correct result to a transformation performed on any set of inputs from a predetermined state, what is left to verify is regardless of when a transformation is performed on a set of inputs the pipeline will produce the same results. To verify this, it is not imperative to verify that the result of a transformation performed by the instantiation of the hardware design from any other state of the pipeline is absolutely correct, only that the pipeline will produce the same output given the same inputs. In other words, what is left to verify is that when the pipeline is presented with the same inputs, regardless of the state of the pipeline, the data will move through the pipeline in the same manner - i.e. that the control logic of the pipeline works as expected. Accordingly, what is left to verify is that when a pipeline executes the same transaction (i.e. processes the same inputs) it will produce the same output.

[0047] As a result, to verify that regardless of when the pipeline performs a transformation on the same set of inputs the pipeline will produce the same output, the data transformation elements can be replaced with simpler logic. For example, the data transformation elements may be black boxed (i.e. replaced with a black box element). As described in more detail below, when a component in a hardware design is black boxed it cuts out the logic driving the output(s) and makes the output(s) ‘free’, meaning the output(s) can take any value regardless of the input(s). Accordingly, even if the black box element receives the same input value(s) at two different times it may generate different output value(s).

[0048] Since black boxing the data transformation elements of a data transformation pipeline removes the transformation logic from the pipeline, which formal property checkers are typically unable to handle, a formal property checker may be able to formally verify that regardless of when such a black boxed pipeline executes a transaction (i.e. processing a set of inputs) it will produce the same output. For example, a formal property checker may be able to verify this by comparing the output of (i) a first instance of the black boxed pipeline which is constrained to execute transactions from a predetermined state; and (ii) the output of a second instance of the black boxed pipeline which can execute transactions from any state, when (i) and (ii) execute the same transaction (i.e. the pipeline processes the same inputs) to see if they match. However, since the output of any black box element is ‘free’ it may not produce the same output for the same input(s). Accordingly, the formal property checker will find that when the same transaction is executed (i.e. the same inputs are processed) by the first and second instances of the black boxed pipeline the outputs are different. What is needed is a mechanism to black box data transformation elements, and other complex elements or components, in a deterministic manner.

[0049] Accordingly, described herein are methods and systems for verifying that an instantiation of a hardware design for a data transformation pipeline will produce the same result for data transformations performed on the same inputs, regardless of when the transformation is performed in which one or more of the data transformation elements of the data transformation pipeline are deterministically black boxed. Deterministically black boxing a data transformation element means that the logic driving the output of the data transformation element (i.e. the logic performing the data transformation) is replaced with logic that does not perform the data transformation, but when presented with the same inputs produces the same output. It is noted that this does not verify that an instantiation of the hardware design will always produce the correct result, only that an instantiation of the hardware design will always produce the same result for data transformations performed on the same inputs. However, this verification in combination with a verification that the hardware design produces the correct result for a data transformation performed on any set of inputs from a particular state of the pipeline (which, as described above, can be performed using some formal verification tools, such as formal equivalence checkers) exhaustively verifies that an instantiation of the hardware design will always produce the correct result to a transformation.

[0050] A first embodiment for verifying that an instantiation of a hardware design for a data transformation pipeline will produce the same output for data transformations performed on the same set of inputs, regardless of when the data transformation is performed is described with respect to FIGS. 4 to 9. In this embodiment the method comprises generating a hardware design for a base transaction module that dynamically generates base transaction data. A first version of the hardware design for the data transformation pipeline is also generated in which each of a selected set of data transformation elements has been replaced by a deterministic black box element that operates in accordance with the base transaction data. Specifically, when a deterministic black box element receives a set of inputs that match a set of inputs recorded in the base transaction data for the corresponding data transformation element, the deterministic black box element outputs the set of outputs recorded in base transaction data for the corresponding data transformation element for that set of inputs. The first version of the hardware design for the data transformation pipeline is linked to the hardware design for the base transaction module. It is then formally verified that if an instantiation of the first version of the hardware design executes the base transaction that an output of the instantiation of the first version of the hardware design matches the pipeline output recorded in the transaction data for the base transaction.

[0051] A second embodiment for verifying that an instantiation of a hardware design for a data transformation pipeline will produce the same result to data transformations performed on the same set of inputs, regardless of when the transformation is performed is described below with respect to FIGS. 10 to 11. In this embodiment, the method comprises: generating a version of the hardware design for the data transformation pipeline in which one or more of the data transformation elements are replaced with a function element which is treated as an unevaluated function of its combinational inputs by a formal verification tool such that during formal verification the function element will produce the same output for the same inputs. It is then formally verified that if an instantiation of the version of the hardware design for the data transformation pipeline produces a result to a transaction executed from a predetermined state of the pipeline that the instantiation of the version of the hardware design for the data transformation pipeline produces the same result if the same transaction is executed from any other state of the pipeline.

[0052] Reference is now made to FIG. 4 which illustrates a first example method 400 for verifying that an instantiation of a hardware design for a data transformation pipeline will produce the same result for data transformations performed on the same inputs, regardless of when the data transformation is performed. The method 400 may be implemented by a computing-based device such as, but not limited to, the computing-based device 1300 described below with respect to FIG. 13. For example, there may be a computer readable storage medium having stored thereon computer readable instructions that, when executed at a computing-based device, cause the computing-based device to perform the method 400 of FIG. 4.

[0053] In the method 400 of FIG. 4, base transaction data is dynamically generated which includes a record of the pipeline inputs and outputs for a base transaction, and a record of the inputs to, and outputs from, one or more data transformation elements for the base transaction. A first version of the hardware design is also generated for the data transformation pipeline. In the first version of the hardware design for the data transformation pipeline at least one of the data transformation elements recorded in the base transaction data is deterministically black boxed in accordance with the base transaction data. Specifically, one or more of the selected data transformation elements are replaced with a deterministic black box element which is configured to, if it receives a set of inputs recorded in the base transaction data for the corresponding data transformation element, produce the output recorded in the base transaction data for the corresponding data transformation element for that set of inputs. It is then formally verified, using a formal verification tool, that if an instantiation of the first version of the hardware design for the data transformation pipeline executes the base transaction (i.e. processes the same pipeline inputs recorded in the base transaction data) that the output of the instantiation of the first version of the hardware design for the data transformation pipeline matches the pipeline output recorded in the base transaction data. The formal verification tool will verify that the output of the instantiation of the first version of the hardware design for the data transformation pipeline matches the pipeline output recorded in the transaction data for each possible base transaction (i.e. for each possible set of inputs). This verifies that, for any base transaction (i.e. that for any set of pipeline inputs), the instantiation of the hardware design will produce the same output regardless of when the base transaction is executed.

[0054] The method 400 begins at block 402 where a hardware design for a base transaction module is generated. The base transaction module is configured to dynamically generate base transaction data for the data transformation pipeline. The base transaction data comprises a record of pipeline inputs and outputs for a base transaction, and a record of the inputs to and outputs from one or more data transformation elements for the base transaction. Where the output of a data transformation element represents the pipeline output then the pipeline output may not be recorded separately from the input and outputs of the data transformation element but may be deduced from the outputs of the relevant data transformation element. Similarly, where the inputs to one or more of the data transformation elements represent the pipeline inputs then the pipeline inputs may not be recorded separately from the inputs and outputs of the data transformation elements.

[0055] For example, the example data transformation pipeline 100 of FIG. 1 is configured to calculate the sum of eight inputs (a₀ + a_± + a₂ + a₃ + a₄ + a₅ + a₆ + a₇). The data transformation pipeline 100 of FIG. 1 comprises a single data transformation element 102 which is configured to output the sum of two inputs. To execute a transaction (calculate the sum of eight inputs) the data transformation element 102 is used in eight cycles - in each of these eight cycles the data transformation element 102 receives two inputs - a new input value and an input representing the sum of the previous inputs - and outputs the sum of the two inputs. Accordingly, the base transaction data for the data transformation pipeline 100 of FIG. 1 may comprise a record of the inputs (a_t and bt) to, and output (yi) from, the data transformation element 102 for eight consecutive cycles.

[0056] The outputs (yi) recorded in the base transaction data for the data transformation element do not necessarily have to be the actual outputs that would be generated by the data transformation element, but may be any possible valid value for the output of the data transformation element. For example, if the recorded inputs for the data transformation element 102 for a cycle are 5 and 0 then the recorded output for the data transformation element 102 does not have to be 5 (i.e. the sum of the inputs), but could be 7 or 20 for example. In this case, since the inputs to the pipeline (a₀, a₄, a₂, a₃, a₄, a₅, a₆,a₇) are a subset of the inputs to the data transformation element 102 and the pipeline output (y₇) is equal to one of the outputs of the data transformation element 102 the pipeline inputs and outputs can be deduced from the inputs and outputs recorded for the data transformation element and do not need to be stored or recorded separately.

[0057] In some examples, the hardware design for the base transaction module may be generated according to the method 500 described below with respect to FIG. 5. In this method 500, a second version of the hardware design for the data transformation pipeline is generated in which the one or more of the data transformation elements are black-boxed (i.e. replaced with black box elements) Black-boxing will be described in more detail below. A hardware design is then generated for a capture module which is configured to record the inputs and outputs to each black box element of the second version of the hardware design for the data transformation pipeline for a transaction executed from a predetermined state of the pipeline; and the second version of the hardware design for the data transformation pipeline is linked to the hardware design for the capture module to form the hardware design for the base transaction module. It will be evident to a person of skill in the art that this is an example only and that other methods may be used to generate the hardware design for the base transaction module.

[0058] Once the hardware design for the base transaction module has been generated the method 400 proceeds to block 404.

[0059] At block 404, a first version of the hardware design for the data transformation pipeline is generated. The first version of the hardware design is generated by replacing at least one of the data transformation elements recorded in the base transaction data with a deterministic black box element which responds to input data in the manner set out in the base transaction data for the corresponding data transformation element. Specifically, a deterministic black box element is configured to, when it receives a set of inputs recorded in the base transaction data for the corresponding data transformation element, produce the outputs recorded in the base transaction data for the corresponding data transformation element for that set of inputs. For example, if the base transaction data has recorded inputs 4 and 5 and output 34 for a data transformation element then the deterministic black box element for that data transformation element will be configured to output 34 when it receives inputs 4 and 5. A deterministic black box element may be generated by replacing the data transformation element with a black box element and adding a constraint to the behaviour of the black box element.

[0060] The process of replacing a component in a hardware design with a black box element is referred to as black boxing the component. Accordingly, the first version of the hardware design for the data transformation pipeline may be generated by first black boxing one or more of the data transformation elements. As is known to those of skill in the art, when a component in a hardware design is black boxed it cuts out the logic driving the output(s) and makes the output(s) ‘free’, meaning the output(s) can take any value regardless of the input(s). Accordingly, without a constraint on the behaviour of the black box element even if the black box element receives the same input value(s) at two different times it may generate different output value(s). Black boxing may alternatively be described as replacing the logic driving the output of a data transformation element with a cutpoint or a stopat.

[0061] In some cases, the constraint may be implemented by an assumption statement. As is known to those of skill in the art, an assumption statement in a hardware design is interpreted by a formal verification tool as a constraint on the behaviour of an instantiation of the hardware design during formal verification. Specifically, when a property is assumed, formal verification tools constrain the inputs so that the property holds. Accordingly, assumptions define an environment in which a property of a hardware design is verified. Therefore, adding such an assumption to the hardware design for a black box element effectively causes the black box element to act in a deterministic manner during formal verification. Specifically, an assumption may be added that states that if the inputs to the black box element match a set of inputs recorded in the base transaction data for the corresponding data transformation element, then the output is the output recorded in the base transaction data for the corresponding data transformation element for that set of inputs. This effectively causes the black box element to respond to the input data recorded in the base transaction data in the manner set out in the base transaction data.

[0062] For example, as shown in FIG. 6, the first version of the hardware design 600 for the data transformation pipeline 100 of FIG. 1 may be generated by replacing the data transformation element 102 with a black box element 602 and adding an assumption 604 that states that if the inputs {a and b) to the black box element 602 match a set of inputs recorded in the base transaction data for the data transformation element 102 then the output (y) is equal to the output recorded in the base transaction data for the data transformation element 102 for that set of inputs. This effectively causes the black box element 502 to act in accordance with the base transaction data. For example, if the base transaction data records inputs a = 2 and b = 7, and output y = 26 for the data transformation element 102, then when the black box element 602 of an instantiation of the first version of the hardware design for the data transformation pipeline receives inputs a = 2 and b = 7 it will output y = 26.

[0063] An example assumption 604 written in SV for the black box element 602 is shown below. In this example assumption, stored_add_a [i] is the first input a recorded in the base transaction data for the data transformation element in the i^th cycle, stored_add_b [i] is the second input/? recorded in the base transaction data for the data transformation element in the i^th cycle, stored_add_y [i] is the output y recorded in the base transaction data for the data transformation element in the i^th cycle, dut_add_a is the first input a of the black box element of an instantiation of the first version of the hardware design for the data transformation pipeline, dut_add_b is the second input b of the black box element of an instantiation of the first version of the hardware design for the data transformation pipeline, dut_add_y is the output y of the black box element of an instantiation of the first version of the hardware design for the data transformation pipeline and max_cycles is 7 in this example.

generate for (i=0; i<=MAX CYCLES; i=i+l) begin: add assumptions assume add: assume property ((stored add a[i] == dut add a) && (stored add b[i] == dut add b) |-> (stored add y[i] == dut add y));

end end [0064] As will be understood to a person of skill in the art this example assumption assumes that if the first and second inputs of the black box element 602 of the first version of the hardware design for the data transformation pipeline match a set of first and second inputs recorded in the base transaction data for the data transformation element for any cycle (i) (stored add a[i] == dut add a) && (stored add b[i] == dut add b) then the output of the black box element 602 of an instantiation of the first version of the hardware design for the data transformation element is equal to the output recorded in the base transaction data for the data transformation element in that cycle (i) (stored_add_y [i] == dut add y)).

[0065] Similarly, as shown in FIG. 7, the first version of the hardware design 700 for the data transformation pipeline 200 of FIG. 2 may be generated by replacing the first and second data transformation elements 202, 204 with black box elements 702, 704, and adding an assumption 706, 708 for each of the first and second black box elements 702 and 704. The assumption 706 for the first black box element 702 may state that if the inputs {a and b) to the black box element 702 match a set of inputs recorded in the base transaction data for the first data transformation element 202, then the output (y) is equal to the output recorded in the base transaction data for the first data transformation element 202 for those inputs. The assumption 708 for the second black box element 704 may state that if the inputs (y and c) match a set of inputs recorded in the base transaction data for the second data transformation element 204 then the output (z) is equal to the output recorded in the base transaction data for the second data transformation element 204 for those inputs. This causes each black box element 702, 704 to produce a recorded output if it receives a recorded set of inputs, otherwise the output is ‘free’.

[0066] In some cases, further restrictions or constraints may be placed on the output of one or more black box elements to ensure that the black box element will not generate unexpected outputs (or outputs with unexpected properties). Such restrictions may be imposed on the black box element by, for example, adding an additional assumption to the first version of the hardware design for the data transformation pipeline. For example, if a data transformation element is configured to generate the square of an input, the downstream components may expect for the data transformation element to output a number greater than or equal to zero, an assumption may be added for the black box element that replaces this data transformation element that states that the output of the black box element is greater than or equal to zero. Similarly, if a data transformation element is configured to output the sine of the input then an assumption may be added that the output of the corresponding black box element is between -1 and 1 inclusive. It will be evident to a person of skill in the art that these are examples only and that other restrictions or constraints may be imposed on the output of a black box element.

[0067] Returning to FIG. 4, once the first version of the hardware design for the data transformation pipeline has been generated the method 400 proceeds to block 406.

[0068] At block 406, the hardware design for the base transaction module is linked to the first version of the hardware design for the data transformation pipeline so that the operation of the deterministic black box elements is controlled by the base transaction data. In some cases, the hardware design for the base transaction may be linked to the first version of the hardware design for the data transformation pipeline by incorporating the hardware design for the base transaction module into the first version of the hardware design for the data transformation pipeline. In other cases, the hardware design for the base transaction module may be linked to the first version of the hardware design for the data transformation pipeline by binding the base transaction module to the first version of the hardware design for the data transformation pipeline. For example, where the first version of the hardware design for the data transformation is implemented in VHDL and the hardware design for the base transaction module is implemented in SV, the VHDL instance may be bound to the base transaction data of the SV instance.

[0069] Once the hardware design for the base transaction module has been linked to the first version of the hardware design for the data transformation pipeline, the method 400 proceeds to block 408.

[0070] At block 408, it is formally verified, using a formal verification tool, that if an instantiation of the first version of the hardware design executes the base transaction from any state of the pipeline that it produces the pipelined output recorded in the base transaction data. As described above, a transaction is defined by the inputs to the pipeline. Accordingly, an instantiation of the first version of the hardware design is said to execute the base transaction if it executes a transaction on the same pipeline inputs as recorded in the base transaction data. For example, the pipeline inputs for a transaction executed by the example data transformation pipeline 100 of FIG. 1 are a₀, a_±, a₂, a₃, a₄, a₅, a₆ and a₇. If the base transaction data records pipeline inputs 3, 4, 2, 6, 8, 3, 9, 10 for d₀ j , CZ-2 j ϊ ϊ j and a₇ respectively then the formal verification proves that if an instantiation of the first version of the hardware design for the data transformation pipeline receives 3, 4, 2, 6, 8, 3, 9, 10 respectively for a₀, a_±, a₂, a₃, α₄, α₅, a₆ and a₇ from any state of the pipeline that the output of the instantiation of the second version of the hardware design for the data transformation pipeline matches the pipeline output recorded in the base transaction data. It is noted that this is an example only and that during the formal verification the formal verification tool will verify the property for all possible base transactions (i.e. for all possible sets of pipeline inputs).

[0071] Formally verifying that if an instantiation of the second version of the hardware design for the data transformation pipeline executes the base transaction, it produces the pipeline output recorded in the base transaction data may comprise generating one or more assertions to be verified, linking the one or more assertions to the first version of the hardware design for the data transformation pipeline, and formally verifying, using a formal verification tool, that the one or more assertions are true (or hold) for the second version of the hardware design for the data transformation pipeline, and outputting one or more signals indicating whether the one or more assertions were successfully verified.

[0072] An assertion is a statement that a particular property is expected to hold for a hardware design (i.e. is always true). An assertion of the form “assert property [evaluable expression]” is said to “assert” the property specified by the “evaluable expression”. If an asserted property (e.g. the evaluable expression) is evaluated to be false for the hardware design for any input the hardware design is not behaving as expected and there is an error.

For example, in the example assertion “assert property a = b”\ if a is not equal to b at any point then the hardware design is not behaving as expected and there is an error.

[0073] Assertions are used to capture required temporal and combinational behaviour of the hardware design in a formal and unambiguous way. The hardware design can then be verified to determine that it conforms to the requirement as captured by the assertion(s). Since assertions capture the hardware design behaviour on a cycle-by-cycle basis they can be used to verify intermediate behaviours.

[0074] Assertions are typically expressed in an assertion language. An assertion language, which may also be referred to as a property language, captures the hardware design behaviour spread across multiple hardware design cycles (e.g. clock cycles) in a concise, unambiguous manner. While traditional hardware description languages (HDL), such as an RTL language, have the ability to capture individual cycle behaviour, they are too detailed to describe properties at a higher level. In particular, assertion languages provide means to express temporal relationships and complex hardware design behaviours in a concise manner. Assertion languages include, but are not limited to, System Verilog Assertions (SVA), Property Specification Language (PSL), Incisive Assertion Library (IAL), Synopsys OVA (OpenVera Assertions), Symbolic Trajectory Evaluation (STE), SystemC Verification (SCV), 0-ln, Specman, and OpenVera Library (OVL).

[0075] To verify that if an instantiation of the first version of the hardware design for the data transformation pipeline executes the base transaction from any state of the pipeline that it produces the pipeline output recorded in the base transaction data, an assertion may be generated that establishes that if an instantiation of the first hardware design receives the same pipeline inputs in the same order as the pipeline inputs recorded in the base transaction data that it produces the pipeline output recorded in the base transaction data. The following is an example SV assertion which may be used wherein expect_watched_output_on_dut is high when an instantiation of the first version of the hardware design for the data transformation pipeline executes the base transaction (i.e. processes the same set of pipeline inputs in the same order recorded in the base transaction data) and is now expected to have completed the transaction and be ready to output the result(s), dut_output is the output of the instantiation of the first version of the hardware design for the data transformation pipeline for the base transaction, and stored_output is the pipeline output recorded in the base transaction data:

assert output: assert property (expect watched output on dut |> (stored output == dut output) [0076] It will be evident to a person of skill in the art that this assertion states that if an instantiation of the first version of the hardware design for the data transformation pipeline executes the base transaction that the output of the instantiation of the first version of the hardware design for the data transformation pipeline is equal to the pipeline output recorded in the base transaction data.

[0077] Once the one or more assertions have been generated the one or more assertions are linked to the first version of the hardware design for the data transformation pipeline. The one or more assertions may be linked to the first version of the hardware design for the data transformation pipeline by incorporating the assertions into the first version of the hardware design for the data transformation pipeline or binding the one or more assertions to the first version of the hardware design for the data transformation pipeline. Specifically, the one or more assertions may be bound to the relevant signals of the first version of the hardware design for the data transformation pipeline to determine when an instantiation of the first version of the hardware design for the data transformation pipeline has received the same inputs in the same order as recorded in the base transaction data, and to determine whether the output matches the pipeline output recorded in the base transaction data.

[0078] Once the one or more assertions have been linked to the first version of the hardware design for the data transformation pipeline the first version of the hardware design for the data transformation pipeline, the one or more assertions and bindings (if any) are loaded into a formal verification tool and the formal verification tool is configured to verify that the one or more assertions are true for the first version of the hardware design for the data transformation pipeline.

[0079] A formal verification tool is a software tool that is capable of performing formal verification of a hardware design. Formal verification tools include, but are not limited to, formal model checkers (which also may be referred to as formal property checkers) which are configured to formally verify a property of a hardware design; and formal equivalence checkers which are configured to formally verify the equivalence of two designs (e.g. two hardware designs, or a hardware design and a netlist etc.). Examples of formal model checkers include, but are not limited to, OneSpin 360® DV, Mentor Graphics Questa® Formal Verification, Synopsys® VC Formal, Cadence® Incisive® Enterprise Verifier, and JasperGold®. Examples of formal equivalence checkers include, but are not limited to

Synopsys® HECTOR, and other logical equivalence checkers (LECs) and sequential logical equivalence checkers (SLECs).

[0080] Formal verification is a systematic process that uses mathematical reasoning to verify a property of a hardware design. Formal verification can be contrasted to simulation-based verification in which a hardware design is tested by applying stimuli to an instantiation of the hardware design and monitoring the output of the instantiation of the hardware design in response to the stimuli.

[0081] In formal verification the hardware design (e.g. a hardware design for an data transformation pipeline) is transformed into a mathematical model (e.g. a state-transition system, or a flow graph) to thereby provide an instantiation of the hardware design which can be tested to verify the hardware design, and the formal properties to be verified (e.g. the asserted properties) are expressed using mathematical logic using a precise syntax or a language with a precise mathematical syntax and semantics.

[0082] An assertion is verified by searching the entire reachable state space of the instantiation of the hardware design (e.g. state-transition system, or flow graph) without explicitly traversing all the states. The search is done by, for example, encoding the states using efficient Boolean encodings using Binary decision diagrams (BDDS), or using advanced SAT (satisfiability-based bounded model checking) based techniques. In some cases, tools can be used to implement techniques, such as, but not limited to, abstraction, symmetry, symbolic indexing, and invariants to improve performance and achieve scalability. Since formal verification of a property algorithmically and exhaustively explores all valid input values over time, verifying a property in this manner allows a property to be exhaustively proved or disproved for all valid states.

[0083] When a formal verification tool is used to verify an assertion, the formal verification tool may output an indication of whether or not the assertion is valid (i.e. the asserted property is true for all valid states or sequences of states), which may also be referred to herein as the assertion being successfully verified. The output may be yes, the assertion is valid or has been successfully verified; no, the assertion is not valid (i.e. the asserted property it is not true or has failed for at least one valid state or sequence of states) or has not been successfully verified; or the formal verification was inconclusive. The formal verification may be inconclusive, for example, because the computing-based device running the formal verification tool has run out of memory or because the formal verification tool has determined that a certain amount of progress has not been made after a predefined period of time.

[0084] Where an assertion is not valid or has not been successfully verified, the formal verification tool may also output information indicating a state or sequence of states of the hardware design which causes the asserted property to fail. For example, the formal verification tool may output a trace of the verification indicating at what point, state or sequence of states the failure occurred.

[0085] Replacement of the data transformation elements of the data transformation pipeline with black box elements and an assumption in the first version of the hardware design for the data transformation pipeline significantly simplifies the logic of the first version of the hardware design for the data transformation pipeline (relative to the original hardware design for the data transformation pipeline) making it much more likely that the formal verification will converge (i.e. produce a conclusive answer-yes, the assertion(s) hold; or, no, at least one of the assertion(s) does not hold). Specifically, instead of the formal verification tool having to determine the result of the transformation for each data transformation element the formal verification tool merely has to compare the inputs to each black box element to the inputs recorded in the base transaction data for the corresponding data transformation element, and if they match, output the output recorded in the base transaction data for the corresponding data transformation element, otherwise output any value.

[0086] Once the formal verification of the first version of the hardware design for the data transformation pipeline is complete the method 400 may end or the method may proceed to block 410.

[0087] At block 410, a determination may be made as to whether the verification of the first version of the hardware design for the data transformation pipeline was successful. As described above, when a formal proof is used to verify an assertion for a hardware design, the formal verification tool may output a signal or a set of signals that indicates whether or not the assertion is valid (i.e. the asserted property is true for all valid states or sequence of states). In these cases, the determination as to whether the verification was successful may be based on the output signal(s).

[0088] If it is determined that the verification of the first version of the hardware design for the data transformation pipeline was successful (indicating that an instantiation of the hardware design will produce the same result for transformations performed on the same set of inputs, regardless of when the transformation is performed) the method 400 may proceed to block 414 where an integrated circuit embodying the data transformation pipeline described by the hardware design is manufactured. If, however, it is determined that the verification of the first version of the hardware design for the data transformation pipeline was not successful (indicating that there is an error in the hardware design) then the method 400 may proceed to block 412 where the hardware design is modified to correct the error in the hardware design that caused the unexpected behaviour. Once the hardware design has been modified the modified hardware design may be re-verified (e.g. blocks 402-410 may be repeated for the modified hardware design).

[0089] Although in the example method 400 of FIG. 4, blocks 402 to 408 are executed in a particular order, in other example methods the blocks 402 to 408 may be executed in another order. For example, in other cases block 404 may be executed before block 402 or they may be executed in parallel.

[0090] Reference is now made to FIG. 5 which describes an example method 500 for generating a hardware design for a base transaction module which may be used to implement block 402 of method 400.

[0091] The method 500 begins at block 502 where a second version of the hardware design for the data transformation pipeline is generated. The second version of the hardware for the data transformation pipeline design may be generated from the initial hardware design for the data transformation pipeline by replacing the description of the structure and/or function of one or more data transformation elements with a description of the structure and/or function of a black box element. In other words, the second version of the hardware design for the data transformation pipeline describes the data transformation pipeline of the initial hardware design except one or more of the data transformation elements has been replaced with a black box element. In some cases, all of the data transformation elements of the data transformation pipeline will be replaced with a black box element, however, in other cases only a subset of the data transformation elements of the data transformation pipeline will be replaced with a black box element.

[0092] As described above, the process of replacing a component in a hardware design with a black box element is referred to as black boxing the component. Accordingly, the second version of the hardware design for the data transformation pipeline is generated from the hardware design for the data transformation pipeline by black boxing one or more of the data transformation elements. As is known to those of skill in the art, when a component in a hardware design is black boxed it cuts out the logic driving the output(s) and makes the output(s) ‘free’, meaning the output(s) can take any value regardless of the input(s). Accordingly, even if the black box element receives the same input value(s) at two different times it may generate different output value(s). Black boxing may alternatively be described as replacing the logic driving the output of a data transformation element with a cutpoint or a stopat.

[0093] For example, where the original hardware design is for a data transformation pipeline, such as that shown in FIG. 1, that comprises a single data transformation element 102 that receives two inputs and outputs the sum of the two inputs, the second version 800 of the hardware design for the data transformation pipeline 100 may be generated by replacing the data transformation element 102 with a black box element 802 that receives the two inputs and has a ‘free’ output (i.e. the output can take on any value regardless of the input) as shown in FIG. 8. By black boxing the data transformation element 102 the logic that performs the addition {a + b) is removed from the hardware design.

[0094] Similarly, where the original hardware design is for a data transformation pipeline, such as the data transformation pipeline 200 of FIG. 2, that comprises a first data transformation element 202 that receives two inputs and outputs the sum of the two inputs, and a second data transformation element 204 that receives two inputs and outputs the product of the two inputs, the second version 900 of the hardware design for this data transformation pipeline 200 may be generated by replacing each data transformation element 202, 204 with corresponding black box element 902, 904 as shown in FIG. 9. Specifically, the first data transformation element 202 is replaced with a black box element 902 that receives two inputs and has a free output and the second data transformation element 204 is replaced with a block box element 904 that receives two inputs and has a ‘free’ output.

[0095] Once the second version of the hardware design for the data transformation pipeline has been generated the method 500 proceeds to block 504.

[0096] At block 504, a hardware design for a capture module configured to capture the inputs and outputs of each black box element of an instantiation of the second version of the hardware design for the data transformation pipeline is generated.

[0097] The hardware design for the capture module defines the function and structure of logic configured to monitor the input and output signals of each black box element of an instantiation of the second version of the hardware design for the data transformation pipeline and store, in one or more memory units (e.g. one or more registers), the value of each input and output signal for a transaction executed from a predetermined state of the pipeline. In other words, the logic is configured to capture the inputs and outputs of each black box element for a transaction executed by an instantiation of the second version of the hardware design for the data transformation pipeline from a predetermined state of the pipeline.

[0098] This will be explained by reference to the example data transformation pipelines 100 and 200 of FIGS. 1 and 2. As described above, the example data transformation pipeline 100 of FIG. 1 is configured to calculate the sum of eight inputs (a₀ + ^ + (22 + ^3 + ^4 + ^5 + a₆ + a₇). The data transformation pipeline 100 of FIG. 1 comprises a single data transformation element 102 which is configured to output the sum of two inputs. To execute a transaction (calculate the sum of eight inputs) the data transformation element 102 is used in eight cycles - in each of these eight cycles the data transformation element 102 receives two inputs - a new input value and an input representing the sum of the previous inputs and outputs the sum of the two inputs. As described above, in the second version of the hardware design for the data transformation pipeline 100 of FIG. 1 the data transformation element 102 is replaced with a black box element 802. Thus, to execute a transaction the black box element 802 is used eight times. Accordingly, capturing the input and output signals of the black box element 802 of an instantiation of the second version of the hardware design for the data transformation pipeline for a transaction executed from a predetermined state comprises capturing the input and output signals of the black box element 802 for eight consecutive cycles from the predetermined state (assuming there are no stalls or other events which extend the calculation).

[0099] Similarly, the example data transformation pipeline 200 of FIG. 2 is configured to calculate (a + b) * c. The example data transformation pipeline 200 of FIG. 2 comprises two data transformation elements - a first data transformation element 202 configured to calculate the sum of the inputs a and b, and a second data transformation element 204 configured to calculate the product of the output of the first data transformation element y and the input c. Thus, to execute a transaction the data transformation element 202 is used in a first cycle and the second data transformation element 204 is used in a second cycle. As described above, in the second version of hardware design for the data transformation pipeline 200 of FIG. 2 the data transformation elements 202, 204 are replaced with black box elements 902, 904. Accordingly, capturing the input and output signals of the black box elements 902, 904 of an instantiation of the second version of the hardware design for the data transformation pipeline for a transaction executed from a predetermined state comprises capturing the input and output signals of the first black box element 902 in the first cycle from the predetermined state, and capturing the input and output signals of the second black box element 904 in a second cycle from the predetermined state of the pipeline.

[00100] Where it is described herein that a transaction is executed from a predetermined state of the pipeline the phrase “state” of the pipeline is used herein to describe the variable elements of a design, and their current values. In some cases, the predetermined state may be the state occurring immediately after reset. However, it will be evident to a person of skill in the art that this is an example only, and that other states of the pipeline may be used as the predetermined state. For example, in other cases the predetermined state may be the state that arises after reset, a first random transaction and a stall.

[00101] The hardware design for the capture module may be implemented in, for example, a hardware description language (HDL), such as VHDL, or it may be implemented in an assertion-based language, such as, but not limited to SVA. The following is example SVA code representing a hardware design for a capture module for the second version of the hardware design for the example data transformation pipeline 100 of FIG. 1. In this example add. a is the input a to the black box element 802, add. b is the input b to the black box element 802, add. y is the output y of the black box element 802, stored_add_a_reg is a register used to store the input a values to the black box element 802, stored_add_b_reg is a register used to store the input b values to the black box element 802, stored_add_y_reg is a register used to store the output y values of the black box element 802, cydes_since_reset is the number of cycles since reset, and max_cycles are the maximum number of cycles for which the inputs and outputs of the black box element 802 are recorded (which may be 7 for the example data transformation pipeline 100 of FIG. 1 when the count starts at 0).

always @ (posedge elk) begin if (cycles since reset <= MAX CYCLES) begin stored add a reg [cycles since reset] <= add. a;

stored add b reg [cycles since reset] <= add.b;

stored add y reg [cycles since reset] <= add.y;

end end [00102] It will be evident to a person of skill in the art, that this example capture module causes the inputs (a and b) and output (y) of the black box element 802 to be added to stored add a reg, stored add b reg and stored add y reg respectively each cycle after reset up to the MAX_CYLES (which may be 7 for the example data transformation pipeline 100 of FIG. 1 if the first cycle is cycle 0). Tables 1-3 illustrates an example of what may be stored in the registers (stored add a reg, stored add b reg and stored_add_y_reg) where a_t, b_it y_t represent the inputs and outputs in cycle i after reset. Together the data stored in the registers forms the base transaction data.

Table 1

stored_a<
[0]	[1]	[2]	[3]
do		n₂	«3

dd_a_reg
[4]	[5]	[6]	[7]
n₄	a₅	n₆	a₇

Table 2

stored_add_b_reg
[0]	[1]	[2]	[3]	[4]	[5]	[6]	[7]
b₀	bi		^3	h₄	b₅	b₆	b₇

Table 3 stored_add_y_reg

[0]	[1]	[2]	[3]	[4]	[5]	[6]	[7]
yo	yi	y-i	Ύ3	Ύ4	Ύ5	Ύ6	y?

[00103] Once the hardware design for the capture module is generated the method 500 proceeds to block 506.

[00104] At block 506, the hardware design for the capture module is linked to the second version of the hardware design for the data transformation pipeline so as to form the hardware design for the base transaction module.

[00105] Reference is now made to FIG. 10 which illustrates an example system 1000 in accordance with the first embodiment for verifying an instantiation of a hardware design for a data transformation pipeline will produce the same result for transformations performed on the same set of inputs, regardless of when the transformation is performed. The system 1000 may be implemented by one or more computing-based devices, such as the computingbased device 1300 described below with respect to FIG. 13. For example, one or more of the components of the system 1000 of FIG. 10 may be implemented as computer readable instructions, which when executed by a computing-based device, cause the computingbased device to perform the functions of the component described below.

[00106] The system 1000 comprises a first version of the hardware design for the data transformation pipeline 1002, a hardware design for a base transaction module 1004, one or more formal assertions 1006 for the first version of the hardware design for the data transformation pipeline and a formal verification tool 1008. As described above, a data transformation pipeline comprises one or more data transformation elements. The base transaction module is configured to dynamically generate base transaction data that includes, the pipeline inputs and outputs for a base transaction, and the inputs to, and outputs from, one or more data transformation element for the base transaction. In the first version of the hardware design one or more of the data transformation elements have been replaced with a deterministic black box element which will act in accordance with the base transaction data. The one or more assertions 1006 may state that if an instantiation of the first version of the hardware design executes the base transaction (at any time) it produces the pipeline output recorded in the base transaction data. In general, the formal verification tool 1008 formally verifies the one or more assertions 1006 are true for the first version of the hardware design for the data transformation pipeline 1002.

[00107] The first version of the hardware design for the data transformation pipeline 1002 is a version of the hardware design for the data transformation pipeline in which the description of the structure and/or function of a selected set of data transformation elements of the data transformation pipeline has been replaced with a description of the structure and/or function of deterministic black box element. As described above a deterministic black box element is configured to, when it receives a set of inputs recorded in the base transaction data for the corresponding data transformation element, produce the output recorded in the base transaction data for the corresponding data transformation element for that set of inputs. For example, if the base transaction data recorded a set of inputs 4 and 6, and an output 8 for a particular data transformation element then when the corresponding deterministic black box element in an instantiation of the first version of the hardware design receives inputs 4 and 6 it will output 8. In other words, the first version of the hardware design describes the data transformation pipeline of the initial hardware design for the data transformation pipeline (e.g. the hardware design under test) except one or more of the data transformation elements have been replaced with a deterministic black box element. As described above, in some cases the first version of the hardware design for the data transformation pipeline may be generated from the initial hardware design for the data transformation pipeline by replacing the selected set of data transformation elements with a black box element and adding an assumption for each black box element that states that if the set of inputs to that black box element matches a set of inputs recorded in the base transaction data for the corresponding data transformation element then the output of the black box element is the output recorded in the base transaction data for the corresponding data transformation element for that set of inputs.

[00108] The hardware design for the base transaction module 1004 is a hardware design for logic configured to dynamically generate base transaction data. The base transaction data comprises pipeline inputs and outputs for a base transaction, and inputs to, and outputs from, one or more data transformation element for the base transaction. In some cases, the hardware design for the base transaction module may comprise a second version of the hardware design for the data transformation pipeline 1010 and a hardware design for a capture module 1012.

[00109] The second version of the hardware design for the data transformation pipeline 1010 is a version of initial hardware design for the data transformation pipeline in which the description of the structure and/or function of the selected set of data transformation elements of the data transformation pipeline is replaced with a description of the structure and/or function of black box element which has a ‘free’ output. In other words, the second version of the hardware design describes the data transformation pipeline of the initial hardware design except the selected set of data transformation elements have been replaced with a black box element.

[00110] The hardware design for the capture module 1012 is a description of the structure and/or function of logic configured to monitor and record the inputs and outputs of each black box element of an instantiation of the second version of the hardware design for the data transformation pipeline 1010 for a transaction executed from a predetermined state (e.g. the state arising from reset) of the pipeline. In some cases, the capture module may be configured to record the inputs and outputs of each black box element in one or more storage elements 1014 (e.g. one or more registers).

[00111] The hardware design for the capture module is linked to the second version of the hardware design for the data transformation pipeline so that (i) the capture module is connected to the appropriate signals of the pipeline so as to be able to monitor and capture the inputs and outputs of the black box elements; and (ii) the hardware design for the capture module and the second version of the hardware design for the data transformation pipeline are treated as one hardware design for purposes of formal verification. Accordingly, once the hardware design for the capture module is linked to the second version of the hardware design for the data transformation pipeline when the formal verification tool creates an instantiation of the second version of the hardware design for the data transformation pipeline it includes an instantiation of the capture module. As described above, the hardware design for the capture module may be linked to the second version of the hardware design for the data transformation pipeline by incorporating the hardware design for the capture module into the second version of the hardware design for the data transformation pipeline or binding the hardware design for the capture module to the second version of the hardware design for the data transformation pipeline.

[00112] The one or more assertions 1006 for the first version of the hardware design comprise one or more formal assertions which if verified to be true for the first version of the hardware design for the data transformation pipeline establishes that if an instantiation of the initial hardware design for the data transformation pipeline (i.e. an instantiation of the data transformation pipeline defined by the initial hardware design) performs a transformation on any inputs from the predetermined state that the result will be the same if the instantiation of the initial hardware design for the data transformation pipeline performs the transformation on the same inputs from any other state of the pipeline. As described above, an assertion is a statement that a particular property is expected to hold for the hardware design (i.e. is always true). As described above, the one or more assertions 1006 may assert that if an instantiation of the second version of the hardware design for the data transformation pipeline performs the base transaction that it produces the pipeline output recorded in the base transaction data.

[00113] The one or more assertions 1006 are linked to the first version of the hardware design for the data transformation pipeline so that the assertion is connected to the relevant signals of the first version of the hardware design for the data transformation pipeline so as to be able to evaluate the asserted property. As described above, the one or more assertions may be linked to the first version of the hardware design for the data transformation pipeline by incorporating the one or more assertions 1006 in the first version of the hardware design for the data transformation pipeline, or, binding the one or more assertions 1006 to first version of the hardware design for the data transformation pipeline.

[00114] As described above, the formal verification tool 1008 is a software tool that is capable of performing formal verification of a hardware design. Formal verification tools include, but are not limited to formal model checkers (which also may be referred to as formal property checkers) which are configured to formally verify a property of a hardware design, and formal equivalence checkers which are configured to formally verify the equivalence of two designs (e.g. two hardware designs, a hardware design and a netlist, or a hardware design and a software model of the hardware design etc). Since the formal verification ideally verifies the assertion over infinite time it is advantageous to use a formal verification tool, such as a formal property checker, that are efficient at proving properties over time.

[00115] The first version of the hardware design for the data transformation pipeline 1002, the hardware design for the base transaction module 1004, the one or more assertions 1006, and the bindings (if any) are loaded in the formal verification tool 1008. The formal verification tool is then configured to formally verify the one or more assertions are true for the first version of the hardware design for the data transformation pipeline.

[00116] As described above, in formal verification the hardware design is transformed into a mathematical model (e.g. a state transition system, or a flow graph) to thereby provide an instantiation of the hardware design which can be tested to verify the hardware design, and the formal properties to be verified (e.g. the asserted properties) are expressed using mathematical logic using a precise syntax or language with a precise mathematical syntax and semantics.

[00117] A formal assertion is verified by searching the entire reachable state space of the instantiation of the hardware design (e.g. state-transition system, or flow graph) without explicitly traversing all the states. The search is done by, for example, encoding the states using efficient Boolean encodings using Binary decision diagrams (BDDS), or using advanced SAT (satisfiability-based bounded model checking) based techniques. In some cases, tools can be used to implement techniques, such as, but not limited to, abstraction, symmetry, symbolic indexing, and invariants to improve performance and achieve scalability. Accordingly, the formal verification tool will verify that the assertion is true for all possible base transactions thus proving that the regardless of when a transaction is executed the pipeline will produce the same output to the same transaction.

[00118] When the formal verification tool 1008 is used to verify an assertion, the formal verification tool 1008 may output an indication of whether or not the assertion is valid (i.e. the asserted property is true for all valid states or sequence of states), which may also be referred to herein as the assertion being successfully verified. The output may be yes, the assertion is valid or has been successfully verified; no, the assertion is not valid (i.e. it is not true or has failed for at least one valid state or sequence of states) or has not been successfully verified; or the formal verification was inconclusive. The formal verification may be inconclusive, for example, because the computing-based device running the formal verification tool 1008 has run out of memory or because the formal verification tool 1008 has determined that a certain amount of progress has not been made after a predefined period of time.

[00119] Where an assertion is not valid or has not been successfully verified, the formal verification tool 1008 may also output information indicating a state or sequence of states of the hardware design which causes the assertion to fail. For example, the formal verification tool 1008 may output a trace of the verification indicating at what point, state or sequence of states the failure occurred.

[00120] Reference is now made to FIGS. 11-12 which describe a second embodiment for verifying that an instantiation of a hardware design for a data transformation pipeline produces the same result for data transformations performed on the same inputs, regardless of when the transformation is performed. In this embodiment, a version of the hardware design for the data transformation pipeline is generated in which a selected set of data transformation elements are replaced with a function element which is treated as unevaluated function of its inputs by a formal verification tool such that during formal verification the element will produce the same output for the same inputs; and formally verifying that if an instantiation of the version of the hardware design for the data transformation pipeline produces a result to a transaction executed from a predetermined state of the pipeline that the instantiation of the version of the hardware design for the data transformation pipeline produces the same result if the same transaction is executed from any other state of the pipeline.

[00121] Reference is now made to FIG. 11 which illustrates a second example method 1100 for verifying an instantiation of a hardware design for a data transformation pipeline will produce the same result for transformations performed on the same inputs, regardless of when the transformations are performed. The method 1100 begins at block 1102 where a version of the hardware design for the data transformation pipeline is generated in which a selected set of the data transformation elements (which may be all or a subset of the data transformation elements) of the pipeline are replaced with a function element which is treated as unevaluated function of the inputs to the data transformation element. The unevaluated function F is a symbolic object such that F(x) = F(x) Vx. This means that a formal verification tool will be able to verify that F(x) = F(x) but a formal verification tool cannot verify F(x) = anything else, this will fail.

[00122] For example, the version of the hardware design for the example data transformation pipeline 100 of FIG. 1 may be generated by replacing the data transformation element 102 with a function element that is treated as an unevaluated function F(a, b) =y such that whenever the same set of a and b are received the output y will be the same, but no other properties of F are assumed. Similarly, the version of the hardware design for the data transformation pipeline 200 of FIG. 2 may be generated by replacing the first data transformation element 202 with a function element that is treated as an unevaluated function F(a, b) = y such that whenever the same set of a and b are received the output y will be the same; and replacing the second data transformation element 204 with a function element that is treated as an unevaluated function G(y, c) = z such that whenever the same set of y and c are received the output z will be the same.

[00123] Replacing a data transformation element with such a function element means that if the same set of inputs pass through the same function elements at any time the data transformation pipeline should produce the same result (assuming the control logic is working as expected).

[00124] Once the version of the hardware design for the data transformation pipeline has been generated the method 1100 proceeds to block 1104.

[00125] At block 1104 it is formally verified, that an instantiation of the version of the hardware design for the data transformation pipeline will produce the same result (or output) for the same transaction regardless of when the transaction is executed (i.e. regardless of the state of the pipeline when the transaction is executed). As described above, a transaction is defined by the inputs to the transaction. Accordingly, an instantiation of the version of the hardware design for the data transformation pipeline is said to perform the same transaction if it processes the same set of inputs. For example, the inputs for a transaction for the example data transformation pipeline 100 of FIG. 1 are d₀ j 1 ^-2’ ^3 > >

a₅, a₆ and a₇. In this example an instantiation of the version of the hardware design for the data transformation pipeline is said to execute the same transaction from two different states if it executes a transaction for a₀, a_±, a₂, a₃, α₄, α₅, a₆, a₇ = 3, 4, 2, 6, 8, 3, 9, 10 from one state of the pipeline (e.g. the state arising from reset) and executes a transaction for a₀, a_lt a₂, a₃, α₄, α₅, a₆, a₇ = 3, 4, 2, 6, 8, 3, 9, 10 from another state of the pipeline.

[00126] Formally verifying that the version of the hardware design for the data transformation pipeline will produce the same result for the same transaction regardless of when the transaction is executed may comprise generating one or more assertions, linking the one or more assertions to the version of the hardware design for the data transformation pipeline, formally verifying, using a formal verification tool, that the one or more assertions are true (or hold) for the version of the hardware design for the data transformation pipeline, and outputting one or more signals indicating whether the one or more assertions were successfully verified.

[00127] Once the formal verification is complete the method 1100 may end or the method 1100 may proceed to block 1106.

[00128] At block 1106, a determination may be made as to whether the verification of the version of the hardware design for the data transformation pipeline was successful. As described above, when a formal verification is used to verify an assertion for a hardware design, the formal verification tool may output a signal or a set of signals that indicates whether or not the assertion is valid (i.e. the asserted property is true for all valid states or sequence of states). In these cases, the determination as to whether the verification was successful may be based on the output signal(s).

[00129] If it is determined that the verification of the version of the hardware design for the data transformation pipeline was successful indicating that an instantiation of the hardware design will produce the same result for the same transaction regardless of when the transaction is executed, the method 1100 may proceed to block 1110 where an integrated circuit embodying the data transformation pipeline described by the hardware design is manufactured. If, however, it is determined that the verification of the version of the hardware design for the data transformation pipeline was not successful indicating that there is an error in the hardware design then the method 1100 may proceed to block 1108 where the hardware design is modified to correct the error in the hardware design that caused the unexpected behaviour. Once the hardware design has been modified the modified hardware design may be re-verified (e.g. blocks 1102-1106 may be repeated for the modified hardware design).

[00130] The verification method 1100 of FIG. 11 is significantly simpler that the verification method 400 of FIG. 4 described above as it does not involve generating base transaction data, nor does it involve controlling the output of black box elements in accordance with the base transaction data. However, the verification method 1100 of FIG.

will typically only be viable with formal verification tools that interpret such function elements in the manner described above. Specifically, the verification method 1100 of FIG.

will typically only be viable with formal verification tools that interpret such function elements as an unevaluated function of its inputs.

[00131] In some cases, restrictions may be placed on the output of one or more of the function elements to ensure that the function element will not generate unexpected outputs (or outputs with unexpected properties). Such restrictions may be imposed on the function element by, for example, adding an assumption to the version of the hardware design for the data transformation pipeline. For example, if a data transformation element is configured to generate the square of an input, the downstream components may expect for the data transformation element to output a number greater than or equal to zero, an assumption may be added for the function element that replaces this data transformation element that states that the output of the function element is greater than or equal to zero. Similarly, if a data transformation element is configured to output the sine of the input then an assumption may be that the output of the function element is between -1 and 1 inclusive. It will be evident to a person of skill in the art that these are examples only and that other restrictions may be imposed on a function element.

[00132] Reference is now made to FIG. 12 which illustrates an example system 1200 in accordance with the second embodiment for verifying an instantiation of a hardware design for a data transformation pipeline will produce the same result for data transformations performed on the same inputs, regardless of when the transformation is performed.

[00133] The system 1200 comprises a version of the hardware design for the data transformation pipeline 1202, one or more formal assertions 1204 for the version of the hardware design for the data transformation pipeline and a formal verification tool 1206. In general, the formal verification tool 1206 is configured to formally verify the one or more assertions 1204 are true for the version of the hardware design for the data transformation pipeline 1202.

[00134] The version of the hardware design for the data transformation pipeline 1202 is a version of initial hardware design for the data transformation pipeline in which the description of the structure and/or function of a selected set of data transformation elements of the data transformation pipeline is replaced with a description of the structure and/or function of a function element which is treated as an unevaluated function of its inputs by a formal verification tool. In other words, the version of the hardware design for the data transformation pipeline describes the data transformation pipeline of the initial hardware design except the selected set of data transformation elements have been replaced with a function element.

[00135] The one or more assertions 1204 for the version of the hardware design for the data transformation pipeline comprise one or more formal assertions which if verified to be true for the version of the hardware design for the data transformation pipeline establishes that an instantiation of the hardware design for the data transformation pipeline (i.e. an instantiation of the data transformation pipeline defined by the initial hardware design) will produce the same outputs for transformations performed on the same inputs regardless of when the transformation is performed . As described above, an assertion is a statement that a particular property is expected to hold for the hardware design (i.e. is always true). As described above, the one or more assertions 1204 may assert that if an instantiation of the version of the hardware design for the data transformation pipeline executes a transaction it has seen before that the output is the same.

[00136] The one or more assertions 1204 are linked to the version of the hardware design for the data transformation pipeline so that the one or more assertions are connected to the relevant signals of the version of the hardware design for the data transformation pipeline to be able to evaluate the asserted property. As described above, the one or more assertions may be linked to the version of the hardware design by incorporating the one or more assertions into the version of the hardware design for the data transformation pipeline, or by binding the one or more assertions 1204 to the version of the hardware design for the data transformation pipeline.

[00137] The formal verification tool 1206 is a software tool that is capable of performing formal verification of a hardware design. Formal verification tools include, but are not limited to formal model checkers (which also may be referred to as formal property checkers) which are configured to formally verify a property of a hardware design, and formal equivalence checkers which are configured to formally verify the equivalent of two designs (e.g. two hardware designs, a hardware design and a netlist, or a hardware design and a model of the hardware design etc). Since the formal verification ideally verifies the assertion over infinite time it is advantageous to use a formal verification tool, such as a formal property checker, that are efficient at proving properties over time.

[00138] The version of the hardware design for the data transformation pipeline 1202, and the one or more assertions 1204, and the bindings (if any) are loaded in the formal verification tool 1206. The formal verification tool 1206 is then configured to formally verify the one or more assertions are true for the version of the hardware design for the data transformation pipeline.

[00139] As described above, in formal verification the hardware design is transformed into a mathematical model (e.g. a state transition system, or a flow graph) to thereby provide an instantiation of the hardware design which can be tested to verify the hardware design, and the formal properties to be verified (e.g. the asserted properties) are expressed using mathematical logic using a precise syntax or language with a precise mathematical syntax and semantics.

[00140] A formal assertion is verified by searching the entire reachable state space of the instantiation of the hardware design (e.g. state-transition system, or flow graph) without explicitly traversing all the states. The search is done by, for example, encoding the states using efficient Boolean encodings using Binary decision diagrams (BDDS), or using advanced SAT (satisfiability-based bounded model checking) based techniques. In some cases, tools can be used to implement techniques, such as, but not limited to, abstraction, symmetry, symbolic indexing, and invariants to improve performance and achieve scalability.

[00141] As described above, the formal verification tool 1206 may output information indicating whether the assertions were successfully verified and/or if one or more of the assertions were not successfully verified information indicating a state or sequence of states of the hardware design which causes the assertion to fail.

[00142] FIG. 13 illustrates various components of an exemplary computing-based device 1300 which may be implemented as any form of a computing and/or electronic device, and in which embodiments of the methods and systems described herein may be implemented.

[00143] Computing-based device 1300 comprises one or more processors 1302 which may be microprocessors, controllers or any other suitable type of processors for processing computer executable instructions to control the operation of the device in order to verify a hardware design for a data transformation pipeline. In some examples, for example where a system on a chip architecture is used, the processors 1302 may include one or more fixed function blocks (also referred to as accelerators) which implement a part of the method of verifying a hardware design for a data transformation pipeline, in hardware (rather than software or firmware). Platform software comprising an operating system 1304 or any other suitable platform software may be provided at the computing-based device to enable application software, such as a formal verification tool, to be executed on the device.

[00144] The computer executable instructions may be provided using any computerreadable media that is accessible by computing-based device 1300. Computer-readable media may include, for example, computer storage media such as memory 1306 and communications media. Computer storage media (i.e. non-transitory machine-readable media), such as memory 1306, includes volatile and non-volatile, removable and nonremovable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device. In contrast, communication media may embody computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transport mechanism. As defined herein, computer storage media does not include communication media. Although the computer storage media (i.e. non-transitory machine-readable media, e.g. memory 1306) is shown within the computing-based device 1300 it will be appreciated that the storage may be distributed or located remotely and accessed via a network or other communication link (e.g. using communication interface 1308).

[00145] The computing-based device 1300 also comprises an input/output controller 610 arranged to output display information to a display device 1312 which may be separate from or integral to the computing-based device 1300. The display information may provide a graphical user interface. The input/output controller 1310 is also arranged to receive and process input from one or more devices, such as a user input device 1314 (e.g. a mouse or a keyboard). This user input may be used to initiate verification. In an embodiment the display device 1312 may also act as the user input device 1314 if it is a touch sensitive display device. The input/output controller 1310 may also output data to devices other than the display device, e.g. a locally connected printing device (not shown in FIG. 13).

[00146] FIG. 14 shows a computer system in which any of the data transformation pipelines described herein may be implemented. The computer system comprises a CPU 1402, a GPU 1304, a memory 1406 and other devices 1414, such as a display 1416, speakers 1418 and a camera 1406. A data transformation pipeline 1410 (such as, but not limited to, any of the data transformation pipelines 100, 200 or 300 described above with respect to FIGs. 1-3) is implemented on the GPU 1404. In other examples, the data transformation pipeline 1410 may be implemented on the CPU 1402. The components of the computer system can communicate with each other via a communications bus 1422. A store 1412 is implemented as part of the memory 1406.

[00147] Generally, any of the functions, methods, techniques or components described above can be implemented in software, firmware, hardware (e.g., fixed logic circuitry), or any combination thereof. The terms “module,” “functionality,” “component”, “element”, “unit”, “block” and “logic” may be used herein to generally represent software, firmware, hardware, or any combination thereof. In the case of a software implementation, the module, functionality, component, element, unit, block or logic represents program code that performs the specified tasks when executed on a processor. The algorithms and methods described herein could be performed by one or more processors executing code that causes the processor(s) to perform the algorithms/methods. Examples of a computerreadable storage medium include a random-access memory (RAM), read-only memory (ROM), an optical disc, flash memory, hard disk memory, and other memory devices that may use magnetic, optical, and other techniques to store instructions or other data and that can be accessed by a machine.

[00148] The terms computer program code and computer readable instructions as used herein refer to any kind of executable code for processors, including code expressed in a machine language, an interpreted language or a scripting language. Executable code includes binary code, machine code, bytecode, code defining an integrated circuit (such as a hardware description language or netlist), and code expressed in a programming language code such as C, Java or OpenCL. Executable code may be, for example, any kind of software, firmware, script, module or library which, when suitably executed, processed, interpreted, compiled, executed at a virtual machine or other software environment, cause a processor of the computer system at which the executable code is supported to perform the tasks specified by the code.

[00149] A processor, computer, or computer system may be any kind of device, machine or dedicated circuit, or collection or portion thereof, with processing capability such that it can execute instructions. A processor may be any kind of general purpose or dedicated processor, such as a CPU, GPU, System-on-chip, state machine, media processor, an application-specific integrated circuit (ASIC), a programmable logic array, a field-programmable gate array (FPGA), or the like. A computer or computer system may comprise one or more processors.

[00150] It is also intended to encompass software which defines a configuration of hardware as described herein, such as HDL (hardware description language) software, as is used for designing integrated circuits, or for configuring programmable chips, to carry out desired functions. That is, there may be provided a computer readable storage medium having encoded thereon computer readable program code in the form of an integrated circuit definition dataset (e.g. hardware design) that when processed (i.e. run) in an integrated circuit manufacturing system configures the system to manufacture a device (e.g. processor or other computing-based device) comprising any apparatus (e.g. data transformation pipeline) described herein. An integrated circuit definition dataset may be, for example, an integrated circuit description.

[00151] Therefore, there may be provided a method of manufacturing, at an integrated circuit manufacturing system, a data transformation pipeline as described herein. Furthermore, there may be provided an integrated circuit definition dataset (e.g. hardware design) that, when processed in an integrated circuit manufacturing system, causes the method of manufacturing a data transformation pipeline to be performed.

[00152] An integrated circuit definition dataset (e.g. hardware design) may be in the form of computer code, for example as a netlist, code for configuring a programmable chip, as a hardware description language defining hardware suitable for manufacture in an integrated circuit at any level, including as register transfer level (RTL) code, as high-level circuit representations such as Verilog or VHDL, and as low-level circuit representations such as OASIS (RTM) and GDSII. Higher level representations which logically define hardware suitable for manufacture in an integrated circuit (such as RTL) may be processed at a computer system configured for generating a manufacturing definition of an integrated circuit in the context of a software environment comprising definitions of circuit elements and rules for combining those elements in order to generate the manufacturing definition of an integrated circuit so defined by the representation. As is typically the case with software executing at a computer system so as to define a machine, one or more intermediate user steps (e.g. providing commands, variables etc.) may be required in order for a computer system configured for generating a manufacturing definition of an integrated circuit to execute code defining an integrated circuit so as to generate the manufacturing definition of that integrated circuit.

[00153] An example of processing an integrated circuit definition dataset (e.g. hardware design) at an integrated circuit manufacturing system so as to configure the system to manufacture a data transformation pipeline will now be described with respect to FIG. 15.

[00154] FIG. 15 shows an example of an integrated circuit (IC) manufacturing system 1502 which is configured to manufacture a data transformation pipeline as described in any of the examples herein. In particular, the IC manufacturing system 1502 comprises a layout processing system 1504 and an integrated circuit generation system 1506. The IC manufacturing system 1502 is configured to receive an IC definition dataset, such as a hardware design, (e.g. defining an data transformation pipeline as described in any of the examples herein), process the IC definition dataset, and generate an IC according to the IC definition dataset (e.g. which embodies a data transformation pipeline as described in any of the examples herein). The processing of the IC definition dataset configures the IC manufacturing system 1502 to manufacture an integrated circuit embodying a data transformation pipeline as described in any of the examples herein.

[00155] The layout processing system 1504 is configured to receive and process the IC definition dataset (e.g. hardware design) to determine a circuit layout. Methods of determining a circuit layout from an IC definition dataset are known in the art, and for example may involve synthesising RTL code to determine a gate level representation of a circuit to be generated, e.g. in terms of logical components (e.g. NAND, NOR, AND, OR, MUX and FLIP-FLOP components). A circuit layout can be determined from the gate level representation of the circuit by determining positional information for the logical components. This may be done automatically or with user involvement in order to optimise the circuit layout. When the layout processing system 1504 has determined the circuit layout it may output a circuit layout definition to the IC generation system 1506. A circuit layout definition may be, for example, a circuit layout description.

[00156] The IC generation system 1506 generates an IC according to the circuit layout definition, as is known in the art. For example, the IC generation system 1506 may implement a semiconductor device fabrication process to generate the IC, which may involve a multiple-step sequence of photo lithographic and chemical processing steps during which electronic circuits are gradually created on a wafer made of semiconducting material. The circuit layout definition may be in the form of a mask which can be used in a lithographic process for generating an IC according to the circuit definition. Alternatively, the circuit layout definition provided to the IC generation system 1506 may be in the form of computerreadable code which the IC generation system 1506 can use to form a suitable mask for use in generating an IC.

[00157] The different processes performed by the IC manufacturing system 1502 may be implemented all in one location, e.g. by one party. Alternatively, the IC manufacturing system 1502 may be a distributed system such that some of the processes may be performed at different locations, and may be performed by different parties. For example, some of the stages of: (i) synthesising RTL code representing the IC definition dataset to form a gate level representation of a circuit to be generated, (ii) generating a circuit layout based on the gate level representation, (iii) forming a mask in accordance with the circuit layout, and (iv) fabricating an integrated circuit using the mask, may be performed in different locations and/or by different parties.

[00158] In other examples, processing of the integrated circuit definition dataset at an integrated circuit manufacturing system may configure the system to manufacture data transformation pipeline without the IC definition dataset being processed so as to determine a circuit layout. For instance, an integrated circuit definition dataset may define the configuration of a reconfigurable processor, such as an FPGA, and the processing of that dataset may configure an IC manufacturing system to generate a reconfigurable processor having that defined configuration (e.g. by loading configuration data to the FPGA).

[00159] In some embodiments, an integrated circuit manufacturing definition dataset, when processed in an integrated circuit manufacturing system, may cause an integrated circuit manufacturing system to generate a device as described herein. For example, the configuration of an integrated circuit manufacturing system in the manner described above with respect to FIG. 15 by an integrated circuit manufacturing definition dataset may cause a device as described herein to be manufactured.

[00160] In some examples, an integrated circuit definition dataset could include software which runs on hardware defined at the dataset or in combination with hardware defined at the dataset. In the example shown in FIG. 15, the IC generation system may further be configured by an integrated circuit definition dataset to, on manufacturing an integrated circuit, load firmware onto that integrated circuit in accordance with program code defined at the integrated circuit definition dataset or otherwise provide program code with the integrated circuit for use with the integrated circuit.

[00161] The implementation of concepts set forth in this application in devices, apparatus, modules, and/or systems (as well as in methods implemented herein) may give rise to performance improvements when compared with known implementations. The performance improvements may include one or more of increased computational performance, reduced latency, increased throughput, and/or reduced power consumption. During manufacture of such devices, apparatus, modules, and systems (e.g. in integrated circuits) performance improvements can be traded-off against the physical implementation, thereby improving the method of manufacture. For example, a performance improvement may be traded against layout area, thereby matching the performance of a known implementation but using less silicon. This may be done, for example, by reusing functional blocks in a serialised fashion or sharing functional blocks between elements of the devices, apparatus, modules and/or systems. Conversely, concepts set forth in this application that give rise to improvements in the physical implementation of the devices, apparatus, modules, and systems (such as reduced silicon area) may be traded for improved performance. This may be done, for example, by manufacturing multiple instances of a module within a predefined area budget.

[00162] The applicant hereby discloses in isolation each individual feature described herein and any combination of two or more such features, to the extent that such features or combinations are capable of being carried out based on the present specification as a whole in the light of the common general knowledge of a person skilled in the art, irrespective of whether such features or combinations of features solve any problems disclosed herein. In view of the foregoing description it will be evident to a person skilled in the art that various modifications may be made within the scope of the invention.

Claims

1. A method of verifying a hardware design for a data transformation pipeline, the data transformation pipeline comprising one or more data transformation elements that perform a data transformation on one or more inputs, the method comprising:

generating a hardware design for a base transaction module that is configured to generate base transaction data, the base transaction data comprising a record of the pipeline inputs and outputs for a base transaction and a record of a set of one or more inputs to and a set of one or more outputs from a set of the data transformation elements for the base transaction;

generating a first version of the hardware design for the data transformation pipeline in which at least one data transformation element of the set of data transformation elements has been replaced with a deterministic black box element configured to generate outputs in accordance with the inputs and outputs recorded in the base transaction data for the corresponding data transformation element;

linking the first version of the hardware design for the data transformation pipeline to the hardware design for the base transaction module; and formally verifying that if an instantiation of the first version of the hardware design for the data transformation pipeline executes the base transaction outputs of the instantiation of the first version of the hardware design match the pipeline outputs recorded in the base transaction data.

2. The method of claim 1, wherein generating the hardware design for the base transaction module comprises:

generating a second version of the hardware design for the data transformation pipeline in which each of the data transformation elements in the set of data transformation elements has been replaced by a black box element having a free output;

generating a hardware design for a capture module configured to record a set of one or more inputs to and a set of one or more outputs from each black box element of the second version of the hardware design for the data transformation pipeline when an instantiation of the second version of the hardware design for the data transformation pipeline executes a transaction from a predetermined state of the pipeline; and linking the second version of the hardware design for the data transformation pipeline and the hardware design for the capture module to form the hardware design for the base transaction module.

3. The method of claim 2, wherein the predetermined state of the pipeline is a state arising after a reset of the instantiation of the second version of the hardware design for the data transformation pipeline.

4. The method of claim 2 or claim 3, wherein the capture module is configured to record the set of one or more inputs to and the set of one or more outputs from each black box element of an instantiation of the second version of the hardware design for the data transformation pipeline for a predetermined number of cycles following a reset of the instantiation of the second version of the hardware design for the data transformation pipeline.

5. The method of claim 4, wherein the predetermined number of cycles is equal to a minimum number of cycles for the data transformation pipeline to execute a transaction.

6. The method of any preceding claim, wherein replacing a data transformation element with a deterministic black box element comprises replacing the data transformation element with a black box element that is configured to operate under a constraint that when the black box element receives a set of one or more inputs that matches a set of one or more inputs recorded in the base transaction data for the corresponding data transformation element the deterministic black box element outputs the set of one or more outputs recorded in the base transaction data for the corresponding data transformation element for that set of one or more inputs.

7. The method of claim 6, wherein the constraint is implemented in the first version of the hardware design for the data transformation pipeline as an assumption statement.

8. The method of any preceding claim, wherein the output of one or more of the deterministic black box elements is constrained to ensure that the deterministic black box element will not generate outputs with unexpected properties.

9. The method of claim 8, wherein the constraint is implemented in the first version of the hardware design for the data transformation pipeline as an assumption statement.

10. The method of any preceding claim, further comprising formally verifying that an instantiation of the hardware design for the data transformation pipeline generates an expected output to a transformation performed on any set of inputs from a predetermined state of the pipeline.

11. The method of any preceding claim, further comprising verifying that an instantiation of the hardware design for the data transformation pipeline generates the correct pipeline outputs for transformations executed from a state arising after reset of the instantiation of the hardware design for the data transformation pipeline.

12. The method of any preceding claim, further comprising outputting one or more control signals indicating whether the formal verification was successful.

13. The method of any preceding claim, further comprising, in response to determining that the formal verification was not successful, modifying the hardware design for the data transformation pipeline to generate an updated hardware design for the data transformation pipeline.

14. The method of claim 13, further comprising generating a first version of the updated hardware design for the data transformation pipeline, linking the hardware design for the base transaction module and the first version of the updated hardware design for the data transformation pipeline, and formally verifying the first version of the updated hardware design for the data transformation pipeline.

15. The method of any preceding claim, further comprising, in response to determining that the formal verification was successful, manufacturing, using an integrated circuit manufacturing system, an integrated circuit embodying the data transformation pipeline according to the hardware design.

16. The method of claim 15, wherein manufacturing, using an integrated circuit manufacturing system, an integrated circuit embodying the data transformation pipeline according to the hardware design comprises:

processing, using a layout processing system, the hardware design so as to generate a circuit layout description of an integrated circuit embodying the data transformation pipeline; and manufacturing, using an integrated circuit generation system, an integrated circuit embodying the data transformation pipeline according to the circuit layout description.

17. The method of any preceding claim, wherein, when processed in an integrated circuit manufacturing system, the hardware design for the data transformation pipeline configures the integrated circuit manufacturing system to manufacture an integrated circuit embodying the data transformation pipeline.

18. A system for verifying a hardware design for a data transformation pipeline, the data transformation pipeline comprising one or more data transformation elements that perform a data transformation on one or more inputs, the system comprising:

memory configured to store:

a hardware design for a base transaction module that is configured to generate base transaction data, the base transaction data comprising a record of the pipeline inputs and outputs for a base transaction and a record of a set of one or more inputs to and a set of one or more outputs from data transformation elements in a set of the data transformation elements for the base transaction;

a first version of the hardware design for the data transformation pipeline linked to the hardware design for the base transaction module, wherein in the first version of the hardware design for the data transformation pipeline at least one data transformation element in the set of data transformation elements has been replaced with a deterministic black box element which is configured to generate outputs in accordance with the inputs and outputs recorded in the base transaction data for the corresponding data transformation element; and a formal verification tool; and one or more processors configured to:

formally verify, using the formal verification tool, that if an instantiation of the first version of the hardware design for the data transformation pipeline executes the base transaction outputs of the instantiation of the first version of the hardware design for the data transformation pipeline match the pipeline outputs recorded in the base transaction data.

19. The system of claim 18, wherein the hardware design for the base transaction module comprises:

a second version of the hardware design for the data transformation pipeline in which each of the data transformation elements in the set of data transformation elements has been replaced by a black box element having a free output;

a hardware design for a capture module that is linked to the second version of the hardware design for the data transformation pipeline, the capture module configured to record a set of one or more inputs to and a set of one or more outputs from each black box element of the second version of the hardware design for the data transformation pipeline when an instantiation of the second version of the hardware design for the data transformation pipeline executes a transaction from a predetermined state of the pipeline.

20. The system of claim 19, wherein the predetermined state of the pipeline is a state arising after a reset of the instantiation of the second version of the hardware design for the data transformation pipeline.

21. The system of claim 19 or claim 20, wherein the capture module is configured to record the set of one or more inputs to and the set of one or more outputs from each black box element of an instantiation of the second version of the hardware design for the data transformation pipeline for a predetermined number of cycles following a reset of the instantiation of the second version of the hardware design for the data transformation pipeline.

22. The system of claim 21, wherein the predetermined number of cycles is equal to a minimum number of cycles for the data transformation pipeline to execute a transaction.

23. The system of any of claims 18 to 22, wherein each deterministic black box element comprises a black box element that is configured to operate under a constraint that when the black box element receives a set of one or more inputs that matches a set of one or more inputs recorded in the base transaction data for the corresponding data transformation element, the black box element outputs the set of one or more outputs recorded in the base transaction data for the corresponding data transformation element for that set of one or more inputs

24. The system of claim 23, wherein the constraint is implemented in the first version of the hardware design for the data transformation pipeline as an assumption statement.

25. The system of any of claims 18 to 24, wherein the output of one or more of the deterministic black box elements is constrained to ensure that the black box element will not generate outputs with unexpected properties.

26. The system of claim 25, wherein the constraint is implemented in the first version of the hardware design for the data transformation pipeline as an assumption statement.

27. The system of claim 26, wherein the one or more processors are further configured to formally verify that an instantiation of the hardware design for the data transformation pipeline generates an expected output to a transformation performed on any set of inputs from a predetermined state of the pipeline.

28. The system of any of claims 18 to 27 wherein the one or more processors are further configured to verify that an instantiation of the hardware design for the data transformation pipeline generates the correct pipeline outputs for a transaction executed from a state that arises after a reset of the instantiation of the hardware design for the data transformation pipeline.

29. The system of any of claims 18 to 28, wherein the one or more processors are further configured to output one or more signals indicating whether the verification was successful.

30. The system of any of claims 18 to 29, wherein the one or more processors are further configured to, when the verification is not successful, receive a first version of an updated hardware design for the data transformation pipeline.

31. The system of claim 30, wherein the one or more processors are further configured to repeat the verifying for the first version of the updated hardware design for the data transformation pipeline.

32. The system of any of claims 18 to 31, wherein the one or more processors are further configured to, when the verification is successful, provide the hardware design for the data transformation pipeline to an integrated circuit manufacturing system, which when processed by the integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture an integrated circuit embodying the data transformation pipeline.

33. The system of claim 32, further comprising the integrated circuit manufacturing system.

34. The system of claim 33, wherein the integrated circuit manufacturing system comprises:

a layout processing system configured to process the hardware design for the data transformation pipeline so as to generate a circuit layout description of an integrated circuit embodying the data transformation pipeline; and an integrated circuit generation system configured to manufacture an integrated circuit embodying the data transformation pipeline according to the circuit layout description.

35. Computer readable code configured to cause the method of any of claims 1 to 12 to be performed when the code is run.

36. A computer readable storage medium having encoded thereon the computer readable code of claim 35.