CN117313595B

CN117313595B - Random instruction generation method, equipment and system for function verification

Info

Publication number: CN117313595B
Application number: CN202311616957.4A
Authority: CN
Inventors: 王登运
Original assignee: Chengdu Denglin Technology Co ltd; Hangzhou Denglin Hanhai Technology Co ltd
Current assignee: Chengdu Denglin Technology Co ltd; Hangzhou Denglin Hanhai Technology Co ltd
Priority date: 2023-11-30
Filing date: 2023-11-30
Publication date: 2024-02-23
Anticipated expiration: 2043-11-30
Also published as: CN117313595A

Abstract

The application provides a random instruction generation method, equipment and system for function verification, which comprises the following steps: acquiring hardware configuration information and test requirement information related to a design to be tested; determining the hierarchical structure of instructions in the test program to be generated and the constraint conditions associated with each hierarchy according to the acquired information, and sequentially generating fields, constructing instructions and composing instruction blocks based on the constraints of each hierarchy to obtain the test program. In the test program, all instructions are randomly generated, each field of the instructions is randomly generated, the combination modes of the fields are more various, the test program has sufficient randomness and flexibility, and various possible test points can be covered to the greatest extent, so that the verification efficiency and coverage rate are improved.

Description

Random instruction generation method, equipment and system for function verification

Technical Field

The present disclosure relates to the field of chip verification, and in particular, to a method, an apparatus, and a system for generating a random instruction for functional verification.

Background

The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art to the present disclosure.

Functional verification of a chip generally refers to verifying whether written register transfer level (RTL, register transfer level) code meets the design specification of the chip, where the RTL code to be verified is generally referred to as a design under test (Design Under Test, DUT). The function verification process generally includes: writing test cases according to test points extracted from the chip design specification, compiling the test cases, running the compiled test cases on a verification platform (testbench) to generate test sequences and transmitting the test sequences to a DUT, responding the input test sequences by the DUT to generate output results, checking the difference between the output results and expected results, and collecting coverage rate. Such a functional verification process is an iterative process that often requires a verifier to write or modify test cases to cover possible test points. However, for the complex chip design at the processor level, the test cases written in this way are still constrained by the individual experience of the verifier, and it is difficult to cover all possible test points, so that the randomness is insufficient. In addition, the test case is frequently required to be modified and recompiled in the verification process, so that not only is the flexibility low, but also the verification period is prolonged, and the verification efficiency is required to be improved.

It should be noted that the foregoing is only used to help understand the technical solution of the present application, and is not used as a basis for evaluating the prior art of the present application.

Disclosure of Invention

The purpose of the application is to provide a random instruction generation method and equipment for functional verification, which can automatically generate random instructions with high coverage rate to perform rapid and efficient functional verification on complex chip designs such as processor level.

The above purpose is achieved by the following technical scheme:

according to a first aspect of embodiments of the present application, there is provided a random instruction generation method for functional verification, including: acquiring hardware configuration information and test requirement information related to a design to be tested; determining a hierarchy of instructions and constraint conditions associated with each hierarchy in a test program to be generated according to the acquired information, wherein the test program comprises a plurality of instruction blocks, each instruction block comprises a plurality of instructions, each instruction consists of one or more fields, and the hierarchy of the instructions at least comprises a field hierarchy, an instruction hierarchy and an instruction block hierarchy from bottom to top; and generating corresponding instructions according to the constraint conditions associated with each level in the hierarchy of the instructions. In this scheme, the support is based on the hierarchical structure of such instructions from bottom to top, each field of the instruction is first randomly generated, and then the instructions are spliced by the randomly generated fields. Because the combination mode of the fields generated at random is more various, a plurality of unexpected instructions or abnormal scenes can be generated, and therefore, the generated instructions have better randomness. The function verification is performed by the test program formed by a large number of instructions generated randomly, the dependence on individual experience of verification personnel is reduced, various real running conditions of the processor can be fully simulated, various possible test points are covered to the greatest extent, and therefore the verification efficiency and coverage rate are improved.

In some embodiments, the generating the corresponding instruction according to the constraint condition associated with each level in the hierarchy of instructions may include determining the number of types of instructions to be generated in each instruction block according to the constraint condition associated with the hierarchy of instruction blocks, and generating the corresponding number of instructions by: randomly generating each field value of the instruction according to the constraint condition of the field hierarchy in the hierarchy structure of the instruction; and concatenating the generated field values into instructions according to constraints associated with the instruction hierarchy. Thus, from the lowest field level of the instruction hierarchy, fields, construction instructions and composition instruction blocks are randomly generated in sequence according to the constraint conditions associated with each level, and then a test program is obtained.

In some embodiments, determining the hierarchy of instructions and the constraints associated with each hierarchy in the test program to be generated according to the acquired information may include:

and respectively determining the total number of instructions of each instruction block and the number of various types of instructions in the instruction block for the test program to be generated according to the instruction number and the instruction proportion specified in the test requirement information.

In some embodiments, the hardware configuration information related to the design under test refers to information related to the hardware architecture, hardware specifications, and instruction set employed by the design under test. The hardware configuration information related to the design under test may be obtained from a design specification file related to the design under test. The test requirement information related to the design under test may include at least information related to the test scope, the test content and the instruction generation manner.

In some embodiments, the method may further comprise: and receiving a design specification file related to the design to be tested, and acquiring hardware configuration information related to the design to be tested from the received file.

In some embodiments, the method may further comprise: user input information is received through a preset test configuration interface, and test requirement information related to the design to be tested is generated based on the received user input information, wherein the test requirement information at least comprises information related to a test range, test contents and an instruction generation mode.

In some embodiments, constraints associated with the instruction block hierarchy may include at least an instruction type, a total number of instructions, an association between instructions associated with each instruction block.

In some implementations, constraints associated with the instruction hierarchy may include at least the format of each type of instruction, the fields supported by each type of instruction and its type, the order of instruction in the same instruction block, instruction timing, register address ranges accessible to the instruction, memory address ranges accessible to the instruction.

In some implementations, constraints associated with a field hierarchy may include at least the value or range of values supported by each field.

In some embodiments, the method may further comprise:

in response to detecting that an exception check is contained in a constraint associated with the instruction block hierarchy, selecting values or values outside a range of values supported by each field when generating each field value of the instruction to simulate an exception condition;

in response to detecting that the association relation between the instructions is set to have resource dependence in the constraint condition associated with the instruction block hierarchy, instructions which need to use the same storage resources are randomly generated for the test program so as to simulate resource conflict conditions.

In some embodiments, the method generates the corresponding instruction according to the constraints associated with each level in the hierarchy of instructions by:

a1 For a plurality of fields belonging to the same type of instruction, randomly generating each field value according to constraint conditions of field layers;

a2 Splicing the generated field values into an instruction according to constraint conditions associated with the instruction hierarchy;

a3 Continuing the execution of the processes of A1) and A2) above until the number of instructions generated satisfies the number of instructions of the type set in the constraint associated with the instruction block hierarchy; and

repeating the processes of A1), A2) and A3) to generate other types of instructions in the same instruction block; and then after the generation of the instructions corresponding to the different instruction blocks is completed in sequence, combining the instructions of the instruction blocks to form a test program.

By the implementation mode, a large number of random instructions can be automatically and rapidly generated, time cost and labor cost are reduced, and verification efficiency is improved; and because each field of the instruction is randomly generated and the combination mode of the fields is more various, a plurality of unexpected instructions or abnormal scenes can be generated, and therefore, unexpected defects of some verifiers can be covered more likely, and the completeness of verification is facilitated.

According to a second aspect of embodiments of the present application, there is provided a random instruction generating device for functional verification, comprising a memory and a processor, the memory having stored thereon computer instructions which, when executed by the processor, implement a method according to the first aspect of embodiments of the present application.

According to a third aspect of embodiments of the present application, a system for functional verification is provided that includes a verification platform associated with a design under test and a random instruction generator. The random instruction generator generates a test program of the design to be tested on the basis of the design specification file and the test requirement information of the design to be tested according to the method of the first aspect of the embodiment of the application, and provides the test program to the verification platform. The verification platform is used for generating a binary test sequence based on the received test program, importing the binary test sequence into the design to be tested, collecting and analyzing the execution result of the design to be tested, and outputting the test result.

In the system, a large number of random instructions can be automatically and quickly generated through the random instruction generator, so that the time cost and the labor cost are reduced, and the verification efficiency is improved; and because each field of the instruction is randomly generated and the combination mode of the fields is more various, a plurality of unexpected instructions or abnormal scenes can be generated, so that unexpected defects of some verifiers can be covered more likely, various possible test points can be covered to the greatest extent, and the coverage rate is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application. It is apparent that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art. In the drawings:

fig. 1 shows a flow diagram of a random instruction generation method for functional verification according to one embodiment of the present invention.

Fig. 2 shows a schematic diagram of the structure of a test program according to an embodiment of the invention.

Fig. 3 shows a schematic structure of a test program according to still another embodiment of the present invention.

Fig. 4 shows a schematic structural diagram of a system for functional verification according to an embodiment of the present invention.

Detailed Description

For the purposes of this application, technical solutions and advantages will be more clearly understood, and the present application will be further described in detail by the following specific examples with reference to the accompanying drawings. It should be understood that the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure based on the embodiments herein.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the present application. One skilled in the relevant art will recognize, however, that the aspects of the application can be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the application.

The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, the functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

The flow diagrams depicted in the figures are exemplary only, and do not necessarily include all of the elements and operations/steps, nor must they be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.

The verification technology is very important in the process of chip design development, and can basically occupy about 70% of the development time of chip products and about 80% of the total code. Chip verification (Design Verification, DV) refers to the verification of chip design for each link prior to streaming, in order to ensure that the yield at the current design stage meets the functional and performance requirements defined by the chip design specifications. Functional verification is a very important step in the chip verification process. During the design process, the chip designer writes the RTL (register transfer level ) code according to the chip design specification, but whether the RTL code thus written is functionally compliant with the corresponding design specification is determined by functional verification of the RTL. Whether the chip design can be fully and effectively functionally verified before streaming can directly affect the time and cost of the development of the chip product.

Functional verification of a chip generally refers to verifying whether written RTL code meets design specifications. The RTL code to be verified may also be referred to as a design under test DUT. Functional verification typically includes the following stages: 1) Extracting test points according to the chip design specification; 2) Building a verification platform; 3) Covering the test points one by one, and writing test cases; 4) And checking the DUT by using the constructed verification platform and test cases. The main components of the verification platform include a generator (generator), a driver (driver), a monitor (monitor), a scoreboard (scoreboard), and the like. Wherein the generator generates various stimuli based on the test cases to simulate various real use cases of the DUT, the driver drives the stimuli generated by the generator to the DUT, the monitor gathers the output of the DUT and passes it to the scoreboard, which determines whether the behavior of the DUT matches the expectations based on the output of the DUT to find out functional defects present in the DUT. Functional verification can be classified into directed verification, which is mainly used to test known design functions inside the DUT, and random verification, which tends to cover unknown functional point boundary conditions, possibly covering possible functional points beyond the design expectations of the verifier, according to the stimulus generation means.

Currently, these incentives, whether in directed verification or random verification, are often binary files or binary test sequences that are generated by a compiler compiling test cases written by a verifier using a high-level language such as C/C++. The functional verification process is an iterative process that often requires a verifier to write or modify test cases to cover possible test points. However, for complex chip designs at the processor level, the binary file so generated is still constrained by the individual experience of the verifier, and it is difficult to cover all possible test points, with insufficient randomness. In addition, the test case is frequently required to be revised and compiled again in the verification process, so that not only is the flexibility low, but also the verification period is prolonged, and the verification efficiency is required to be improved.

A processor (e.g., CPU, GPU, etc.) is required to have a more efficient functional verification method as one of the chips with the highest design complexity. The functional verification work for the processor chip requires the generation of a large number of instructions to simulate the actual operation of the processor to discover more functional defects in the DUT as soon as possible. With the development of integrated circuit technology, the processor structure is more and more complex, and a variety of instruction sets are accompanied, how to effectively generate large-scale instructions with high coverage rate in a reasonable time is a very critical link in the function verification work of the processor, the coverage rate is used for measuring the coverage degree of all test points, and the common coverage rate mainly comprises code coverage rate and function coverage rate.

For convenience of description, in the following description, a General-purpose graphics processor (GPGPU, general-purpose computing on graphics processing units) supporting a single instruction stream multithreading model (SIMT, single Instruction Multiple Thread) is exemplified for the hardware architecture of the processor, but it should be understood that the method of the present embodiment is also applicable to processors of other hardware architectures. From a hardware perspective, one GPGPU includes a plurality of SMs (multithreaded streaming multiprocessors ). Each SM contains multiple processor groups, register resources, shared memory resources, schedulers, load/store units, instruction pipelines, caches, and the like. Each processor group contains a plurality of cores, each core executes a thread, and a plurality of cores in the same processor group share the same instruction unit to execute the same instruction in parallel, but the processed data are different. That is, the instruction unit fetches an instruction and issues it to all cores in a processor group at the same time. Each core is a basic execution unit and can be understood as a complete logical operation unit. For example, a GPGPU may include two SMs, each SM including 4 processor sets, each processor set including 32 cores, each SM including substantially 128 cores, and the GPGPU including 256 cores.

From a software perspective, the smallest logical unit is a thread that corresponds in hardware to the smallest hardware unit (i.e., a core); the smallest unit of execution is a thread group, which corresponds to a processor group in hardware. Multiple threads in the same thread group execute the same instruction on different data in parallel. An instruction is composed of a string of binary numbers, typically comprising two parts: an opcode and an operand. The opcode indicates the type or nature of the operation to be performed by the instruction, such as fetching, adding, or outputting data; the operand indicates the contents of the operation object of the instruction or the address of the memory location where it resides. The function of an instruction is determined by the instruction's opcode and can be generally divided into: operational instructions such as add, subtract, multiply, shift, logical operations, etc.; memory access instructions responsible for reading and writing to the memory; flow direction transfer instructions for controlling the program; special instructions for a particular application, etc. Execution of an instruction may generally be broken down into instruction fetch (IF, instruction fetch), instruction decode (ID, instruction decode), instruction execution (EX, execution), access count (MEM), and result Write Back (WB). In processor designs, pipelining is commonly used to increase the execution speed of instructions. The core of pipelining is that the execution of a subsequent instruction begins when the previous instruction has not yet ended, so that some conditions called "conflicts" often occur in the pipeline; for example, a structural conflict resulting from two or more instructions in the pipeline accessing the same physical resource (e.g., register, memory, etc.); data collision caused by data dependence among a plurality of simultaneously executed instructions; control conflicts caused by branch instructions, and the like.

The storage structure of the GPGPU includes registers, local memory, shared memory, global memory, and the like. Where registers are the fastest memory locations that are allocated to a given thread for use, the design typically specifies a maximum number of registers, e.g., 64, 256, that one thread is allowed to use. The local memory is designed for the condition that the register cannot meet the storage requirement, and is mainly used for storing a large array and a variable of a single thread. The registers and local memory are thread private and invisible between threads. Shared memory is a storage space that allows multiple threads in the same thread group to access. As each thread group begins execution, it is allocated a certain amount of shared memory whose address space is shared by all threads in the thread group. The shared memory can be divided into a plurality of banks (banks) according to 4 bytes or 8 bytes, and data of different banks can be read and written simultaneously in parallel, but when a plurality of threads access different addresses in the same Bank simultaneously, bank conflict can be caused. Global memory is the storage space accessible to all thread groups.

In verifying the design of a processor, a large number of instructions are required to simulate the actual operation of the processor. FIG. 1 presents a flow diagram of a method for generating random instructions in functional verification in accordance with an embodiment of the present invention. As shown in fig. 1, the method mainly comprises: step S1), obtaining hardware configuration information and test requirement information related to a design to be tested; step S2) determining the hierarchical structure of instructions in the test program to be generated and the constraint conditions associated with each level according to the acquired information; step S3) generating corresponding instructions according to the constraint conditions associated with each level in the hierarchical structure of the instructions so as to obtain the test program. The function verification is carried out through the test program formed by the randomly generated instructions, the method has sufficient randomness and flexibility, is not dependent on the constraint of individual experience of verification personnel, can fully simulate various real running conditions of the processor, and covers various possible test points to the greatest extent, thereby being beneficial to improving the verification efficiency and coverage rate.

More specifically, in step S1), hardware configuration information and test requirement information related to the design under test are acquired. The hardware configuration information related to the design to be tested refers to information for describing or describing a hardware architecture, a hardware specification and an instruction set adopted by the design to be tested. Test requirement information related to a design under test refers to information that describes or specifies specific test requirements. The information may be obtained from a design specification file of the chip, from a related configuration file, or from configuration options entered or selected by the user via an interface such as a graphical interface, or directly from a test document pre-configured by the user, or the like, or any combination thereof. The manner of obtaining the above information is not limited in any way.

In one embodiment, the hardware configuration information associated with the design under test may include the following types of information:

(1) Hardware architecture of design to be tested: such as the architecture type of the design under test, e.g., GPGPU, GPU, CPU or other chip type; each functional module constituting the design to be tested, etc.

(2) Hardware specification of design to be tested: different hardware architectures, even different designs to be tested having the same hardware architecture, typically have different hardware specifications. Assuming that the design under test is a GPGPU, the hardware specifications associated with the GPGPU may include: the number of SMs, the number of thread groups in each SM, the global memory size, the number of scalar registers, the number of vector registers, the number of threads in each thread group, the maximum number of registers allocated to each thread, the size of shared memory for each thread group, the number of banks in each shared memory, the number of pipeline stages, and so forth.

(3) Instruction configuration of the design to be tested: such as the specific format of the various types of instructions supported by the design under test, the type and range of values of the various fields each instruction has, and so forth.

In one embodiment, the test requirement information related to the design under test may include the following types of information:

(a) The test scope, for example, may specify whether to test the entire design under test or some of its functional modules.

(b) Test content-for example, it may be specified whether to perform a functional check, a boundary check, an anomaly check, or any combination of the above. Wherein the function check means to check whether the DUT can normally implement the functions described in the design file; boundary checking is to check if the DUT is missing some extra cases by giving various boundary conditions; the exception check is to check whether the DUT can handle an exception condition such as a data collision, a bank collision, etc., and to check whether a response conforming to expectations can be given for the exception condition.

(c) Instruction generation means-for example, the number of instructions to be generated and the instruction ratio may be specified, i.e., how many instructions need to be generated, and in what ratio to generate instructions for each test content.

In some embodiments, the method may further include receiving a design specification file related to the design under test, obtaining hardware configuration information related to the design under test from the received file, and/or receiving user input information through a preset test configuration interface, generating test requirement information related to the design under test based on the received user input information, where the test requirement information includes at least information related to a test range, test content, and an instruction generation manner.

With continued reference to fig. 1, in step S2), the hierarchical structure of the instructions in the test program to be generated and the constraint conditions associated with each hierarchy are determined according to the hardware configuration information and the test requirement information related to the design to be tested acquired in step S1). The test program to be generated here refers to a combination of instructions required for functional verification of the design under test.

FIG. 2 presents a hierarchical illustration of instructions in a test program according to one embodiment of the present invention. As shown in FIG. 2, the test program (which may be referred to as a kernel) includes a plurality of instruction blocks, each instruction block (which may be referred to as a group) including a set of instructions (instructions) having a correlation, and there is no correlation between instructions in the different instruction blocks. Dependencies among instructions mean that there are relationships of coordination, data dependence, execution order dependence, etc. among multiple instructions. For example, there is a dependency on a group of instructions that together cooperate to perform a particular function; a group of instructions that process the same data object have dependencies; a group of instructions accessing the same hardware resource have dependencies; there is a dependency on a set of instructions that have a dependency on the order of execution. Without limiting this in any way, each instruction block may be understood as a meaningful sequence of instructions or program segments that accomplish a task. For example, the instruction block may be a set of instructions for testing functions such as addition, subtraction, multiplication, and division of integers, a set of instructions for testing functions such as addition, subtraction, multiplication, and division of floating point numbers, or a set of instructions for testing memory access functions, control functions, and various abnormal conditions. Each instruction is made up of one or more fields (which may be referred to as field). The instruction-related fields may include opcodes, operands, and the like.

It should be understood that the hierarchy of instructions shown in FIG. 2 is by way of example only and not by way of any limitation. A schematic of the hierarchy of instructions in a test program according to yet another embodiment of the invention is given in fig. 3. Unlike FIG. 2, a subroutine (also referred to as an instruction stream) hierarchy is added above the instruction block hierarchy to reflect the coupling or association between instruction blocks, e.g., multiple instruction blocks that need to be associated with each other are included in the same subroutine.

The instruction blocks in the instruction hierarchy may be determined according to the hardware configuration information and the test requirement information related to the design under test acquired in step S1. For example, in the process of designing a processor, the processor is generally divided into different functional modules according to the functional module division of the chip, so that when performing functional verification, a corresponding instruction block can be generated for each functional module in the hardware architecture of the design to be tested to facilitate functional inspection of the functional module, for example, floating point operation, integer operation, complex number processing, memory access processing, micro-operation control, interrupt processing, and the like. In yet another example, corresponding instruction blocks may be provided for a portion of the functional modules specified in the test scope. In yet another example, instruction blocks related to performing functional checks, boundary checks, and anomaly checks may be set according to test content. For example, for exception checking, a plurality of instruction blocks for detecting data conflicts, a plurality of instruction blocks for detecting bank conflicts, and a plurality of instruction blocks for detecting control conflicts in a pipeline may be provided.

After determining the instruction hierarchy of the present test program in step S2), it is also necessary to determine the constraint conditions associated with each hierarchy. The following description will be given by taking, as an example, an instruction hierarchy including an instruction block hierarchy, an instruction hierarchy, and a field hierarchy, which are shown in fig. 2.

Wherein the constraints associated with the instruction block hierarchy may include: the instruction type associated with each instruction block, the total number of instructions, the association between instructions, etc. For example, the total number of instructions in each instruction block and the number of instructions of each type in the instruction block are respectively determined according to the number of instructions and the instruction proportion specified in the test requirement information. If the user does not intentionally specify the proportion, the random generation of instructions for the instruction blocks in the test program can be performed according to the instruction number or the proportion which is set for each level by default. The associations between instructions may include collaboration dependencies, data dependencies, resource dependencies, control dependencies, and the like. Taking an addition instruction as an example, the instructions with cooperative dependence include, for example, a memory access instruction for reading operands and writing back calculation results; instructions for which there is a data dependency are, for example, instructions for further calculation operations on the calculation result of the addition instruction; the instruction on which resource dependency exists may be an instruction that performs a write operation to a register in which an operand thereof exists, or the like.

Constraints associated with the instruction hierarchy may include: the format of each type of instruction, all possible fields involved in each type of instruction and their types (i.e., the number of fields and the type of fields in the instruction), the sequencing of instructions determined based on the association between instructions from the instruction block hierarchy set forth above, instruction timing, register address ranges accessible to the instructions, memory address ranges accessible to the instructions, and so forth. Taking the addition type instruction as an example, the instruction includes an ADD instruction without carry addition, an ADC instruction with carry addition, an ADD 1 instruction INC, and the like. The ADD instruction involves three fields: the instruction formats of the operation code ADD, the destination operand OPD, and the source operand OPS are, for example, "ADD OPD OPS". It will be appreciated that the number of each field in each actual instruction may be determined based on the design specifications of the processor, for example only. Wherein the destination operand OPD field type may be a general purpose register or a memory location; the source operand OPS field type may be an immediate, a general purpose register, or a memory location. Relevant information about the instructions may be obtained from a design specification document of the processor.

Constraints associated with a field hierarchy may include: each field may take a value or range of values. Taking an addition instruction as an example, the possible values of the operation code field are ADD, ADC, INC and the like; the operand field may be 8 bits, 32 bits, 64 bits, etc., depending on the hardware specifications of the design to be tested.

In step S3), as one embodiment, fields, construction instructions, and constituent instruction blocks are sequentially generated according to constraint conditions associated with each level in the hierarchy of the instructions, and then a test program is obtained. The method mainly comprises the steps of determining the number of various types of instructions to be generated in each instruction block according to constraint conditions associated with instruction block layers, and generating the corresponding number of instructions through the following random instruction generation steps:

a) Randomly generating each field value of the instruction according to the constraint condition of the field hierarchy in the hierarchy of the instruction;

b) And splicing the generated field values into the instruction according to the constraint conditions associated with the instruction hierarchy.

Still taking an add instruction as an example, for a plurality of fields belonging to the same instruction, each field value is randomly generated based on the possible values or value ranges of each field (i.e., constraints of the field hierarchy) starting from the lowest field hierarchy. The generated field values are then concatenated according to constraints associated with the instruction hierarchy (e.g., instruction format) to obtain a randomly generated add instruction. Further, with reference to the constraint condition associated with the instruction block hierarchy, it is determined whether the number of generated add-type instructions satisfies the condition, and if not, the above-described process is repeated to continue to randomly generate a corresponding plurality of add-type instructions until a sufficient number of add-type instructions are acquired. And continuing to generate other types of instructions (e.g., instructions that have dependency on the add instruction) for the same instruction block until the total number of instructions for the instruction block is reached. After the generation of the instructions corresponding to the different instruction blocks is completed in sequence, the instructions of the instruction blocks are combined together to form the test program. It will be appreciated that if the hierarchy of instructions also includes subroutines, different subroutines may also be generated for the test program based on instruction blocks containing a large number of random instructions.

In the above embodiment, since each field of the instruction is randomly generated and the combination manner of these fields is more various, many unexpected instructions or abnormal scenes can be generated, so that unexpected defects of some verifiers are more likely to be covered, which is helpful for completeness of verification. Random generation based on various fields of instructions provides finer granularity and more flexible randomness of instructions, all instructions being randomly generated in the acquired test program. Therefore, the method also reduces the constraint depending on individual experience of the verifier, but can fully simulate various real running conditions of the processor and cover various possible test points to the greatest extent, thereby being beneficial to improving verification efficiency and coverage rate.

In the application, the thought of sequentially generating each layer of content from bottom to top according to the hierarchical structure of the instruction is supported, and the thought of processing from top to bottom according to the hierarchical structure is also supported.

In still other embodiments, the random instruction generation process described above may be limited in terms of constraints according to instruction block hierarchy, taking into account dependencies between instructions. For example, for a randomly generated add instruction, the hardware resources (e.g., registers or memory) it involves are recorded. If the association relation between the instructions is set as resource dependence in the constraint condition associated with the instruction block hierarchy, the registers used in the previous instruction can be used in the subsequent instruction in the process of generating the subsequent instruction so as to simulate resource conflict situations, namely, the registers for storing the calculation results of the previous instruction are used by the subsequent instruction when the content is not written into the memory. For another example, if the pipeline stage number set in the test requirement information is 8, at least 8 randomly generated instructions are saved, and the related hardware resources are recorded, so that when a subsequent instruction is generated, the operand required by the subsequent instruction is just the operation result of the previous instruction, and the abnormal situation of data collision is simulated.

In still other embodiments, the method may further comprise: in response to detecting that an exception check is included in a constraint associated with the instruction block hierarchy, values or values outside a range of values supported by each field are selected to simulate an exception condition when generating values for each field of the instruction. For example, when an exception check is included in a constraint associated with the instruction block hierarchy, for an add instruction, operands exceeding its operational capabilities or operands of mismatched types may be generated to simulate an exception condition; for a division instruction, the divisor field may be zeroed to simulate a divide by zero exception; for jump-type instructions, operand fields beyond their address range may be generated to simulate an exception condition that jumps to an unknown address resulting in uncontrollable instruction execution sequences. The special conditions such as conflict or abnormality are simulated by generating the instruction meeting the requirements, so that the method is beneficial to detecting whether the actual response or state of the design to be tested can reach the expectations when the design to be tested faces the special conditions. In some application scenarios, different random modes can be adopted in the instruction generation process of different levels, for example, global random can be adopted in the field level, and limited random generation can be carried out according to the configuration requirements or proportions in a limited range under the condition that the user test requirements are met according to the hardware configuration information and the test requirement information in the generation process of the instruction block level. That is, in this embodiment, the manner of randomly generating the instruction may be flexibly adjusted by setting the constraint conditions associated with each level in the instruction hierarchy, so that various actual running conditions of the processor may be fully simulated, and various possible test points may be covered to the greatest extent.

In the above embodiment, the randomly generated instruction may be an assembler instruction, and the resulting test program may be directly an assembler file. The assembler file is translated by the assembler into a binary test sequence (i.e., a machine code file). The binary test sequence is imported into the design under test for execution. The assembler file can be directly provided for reference model execution in the verification platform, and possible defects in the design to be tested can be detected and found by comparing the execution result of the design to be tested with the execution result of the reference model in the verification platform, so as to obtain a test result. And the related verifier can also judge whether all test points are covered by the randomly generated instructions through the assembler file. In some embodiments, the test program may also be output in the form of a binary file or an array file. In some embodiments, multiple alternative forms of test programs may be provided simultaneously for selection by a user.

By the method, a large number of random instructions can be automatically and rapidly generated, time cost and labor cost are reduced, and verification efficiency is improved; and because each field of the instruction is randomly generated and the combination mode of the fields is more various, a plurality of unexpected instructions or abnormal scenes can be generated, and therefore, unexpected defects of some verifiers can be covered more likely, and the completeness of verification is facilitated.

In a further embodiment of the present invention, there is also provided a random instruction generating device for function verification, including a memory and a processor, on which computer instructions are stored, which when executed by the processor implement the random instruction generating method for function verification described above in connection with fig. 1 and 2.

Fig. 4 presents a schematic view of the architecture of a system 400 for functional verification according to an embodiment of the invention. As shown in fig. 4, the system may include a design under test 401, a verification platform 402 associated with the design under test, and a random instruction generator 403. Wherein the random instruction generator 403 generates a test program based on the design specification file of the design under test and the test requirement information from the user using the random instruction generation method for function verification described above in connection with fig. 1 and 2, and provides it to the verification platform. The verification platform 402 generates a binary test sequence based on the received test program and introduces the binary test sequence into the design under test, and collects the execution result of the design under test, detects and discovers possible defects in the design under test by comparing the execution result of the design under test with the execution result of the reference model in the verification platform, and outputs the test result, which may be presented in the form of a test report, which may generally be a data statistics report obtained by analyzing the execution result of the design under test DUT. It will be appreciated that the design under test itself may not be considered part of the system.

In the function verification system, a large number of random instructions can be automatically and quickly generated through the random instruction generator, so that the time cost and the labor cost are reduced, and the verification efficiency is improved; and because each field of the instruction is randomly generated and the combination mode of the fields is more various, a plurality of unexpected instructions or abnormal scenes can be generated, so that unexpected defects of some verifiers can be covered more likely, various possible test points can be covered to the greatest extent, and the coverage rate is improved. In addition, based on the principle of the application, the required test program can be generated in the function verification system rapidly and in real time, online function verification is supported, excitation is generated online, other offline or switching problems caused by repeated changing of test cases and compiling of the test cases due to constraint of individual experience of verification personnel can be solved, and the verification period is shortened.

Reference in the specification to "various embodiments," "some embodiments," "one embodiment," or "an embodiment" or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, appearances of the phrases "in various embodiments," "in some embodiments," "in one embodiment," or "in an embodiment" in various places throughout this specification are not necessarily referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Thus, a particular feature, structure, or characteristic described in connection with or illustrated in one embodiment may be combined, in whole or in part, with features, structures, or characteristics of one or more other embodiments without limitation, provided that the combination is not non-logical or inoperable.

The terms "comprises," "comprising," and "having" and the like, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Nor does "a" or "an" exclude a plurality. Additionally, the various elements in the drawings of the subject application are for illustration purposes only and are not drawn to scale.

Although the present application has been described by way of the above embodiments, the present application is not limited to the embodiments described herein, but includes various changes and modifications made without departing from the scope of the present application.

Claims

1. A random instruction generation method for functional verification, comprising:

acquiring hardware configuration information and test requirement information related to a design to be tested;

determining a hierarchy of instructions and constraint conditions associated with each hierarchy in a test program to be generated according to the acquired information, wherein the hierarchy of instructions at least comprises a field hierarchy, an instruction hierarchy and an instruction block hierarchy from bottom to top;

Generating a corresponding instruction according to constraint conditions associated with each level in the hierarchical structure of the instruction; the hardware configuration information related to the design to be tested is information related to a hardware architecture, a hardware specification and an instruction set adopted by the design to be tested; the test program comprises a plurality of instruction blocks, wherein the instruction blocks comprise a plurality of instructions with relevance, the instructions are composed of one or more fields, and the relevance among the instructions refers to the combination of one or more of coordination, data dependence and execution sequence dependence among the plurality of instructions.

2. The method according to claim 1, wherein determining the hierarchy of instructions and the constraint conditions associated with each hierarchy in the test program to be generated according to the acquired information comprises:

3. The method of claim 1, the method further comprising:

receiving a design specification file related to the design to be tested, and acquiring hardware configuration information related to the design to be tested from the received file;

And/or receiving user input information through a preset test configuration interface, and generating test requirement information related to the design to be tested based on the received user input information, wherein the test requirement information at least comprises information related to a test range, test contents and an instruction generation mode.

4. The method of claim 1, wherein the constraints associated with the instruction block hierarchy include: the instruction type, the total number of instructions, and the association relationship between instructions associated with each instruction block.

5. The method of claim 1, wherein the constraints associated with the instruction hierarchy include: the format of each type of instruction, the fields supported by each type of instruction and its type, the order of instructions in the same instruction block, the instruction timing, the register address range accessible to the instruction, the memory address range accessible to the instruction.

6. The method of claim 1, wherein the constraints associated with the field hierarchy include at least a value or range of values supported by each field.

7. The method of any one of claims 1-6, further comprising:

8. The method of claim 1, the generating a corresponding instruction according to the constraints associated with each level in the hierarchy of instructions, comprising:

determining the number of various instruction types to be generated in each instruction block according to constraint conditions associated with the instruction block hierarchy, and generating the corresponding number of instructions through the following steps:

randomly generating each field value of the instruction according to the constraint condition of the field hierarchy in the hierarchy structure of the instruction;

and splicing the generated field values into the instruction according to the constraint conditions associated with the instruction hierarchy.

9. The method of claim 8, the generating a corresponding instruction according to the constraints associated with each level in the hierarchy of instructions, comprising:

A3 Continuing to execute the above A1) and A2) until the number of generated instructions satisfies the number of instructions of the type set in the constraint associated with the instruction block hierarchy; and

10. A random instruction generating device for functional verification comprising a memory and a processor, the memory having stored thereon computer instructions which, when executed by the processor, implement the method of any of claims 1-9.

11. A system for functional verification comprising a verification platform associated with a design under test and a random instruction generator, wherein:

the random instruction generator is used for generating a test program of the design to be tested on the basis of a design specification file and test requirement information of the design to be tested according to the method of any one of claims 1-8, and providing the test program to the verification platform; and

the verification platform is used for generating a binary test sequence based on the received test program, importing the binary test sequence into the design to be tested, collecting and analyzing the execution result of the design to be tested, and outputting the test result.