CN114924925A - Instruction set simulator test method and device, electronic equipment and storage medium - Google Patents
Instruction set simulator test method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114924925A CN114924925A CN202210611601.0A CN202210611601A CN114924925A CN 114924925 A CN114924925 A CN 114924925A CN 202210611601 A CN202210611601 A CN 202210611601A CN 114924925 A CN114924925 A CN 114924925A
- Authority
- CN
- China
- Prior art keywords
- test
- instruction set
- file
- instruction
- set simulator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/2268—Logging of test results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/2273—Test methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/2289—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing by configuration test
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/40—Transformation of program code
- G06F8/41—Compilation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
- G06F8/71—Version control; Configuration management
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Software Systems (AREA)
- Computer Security & Cryptography (AREA)
- Debugging And Monitoring (AREA)
Abstract
The present disclosure provides a method, an apparatus, an electronic device and a storage medium for testing an instruction set simulator, wherein the method comprises: acquiring an executable file corresponding to the instruction set simulator and a test file matched with the test content of the test point; wherein the test point comprises at least one instruction to be tested; executing the executable file based on the test file to generate output data corresponding to the test content of the test point; and determining a test result corresponding to the instruction set simulator based on the output data and preset data corresponding to the test content of the test point.
Description
Technical Field
The present disclosure relates to the field of chip design technologies, and in particular, to a method and an apparatus for testing an instruction set simulator, an electronic device, and a storage medium.
Background
An Instruction Set Simulator (ISS) is a simulation model, typically encoded in a high-level programming language, that simulates the behavior of a mainframe or microprocessor by "reading" instructions and maintaining internal variables representing processor registers. In an Artificial Intelligence (AI) chip design stage, an AI instruction set simulator is used.
Generally, an AI instruction set simulator needs to meet the requirements provided in the design process of an AI chip, that is, the AI instruction set simulator is used to simulate various instruction information required by the operation of the AI chip. It can be known that the performance of the AI instruction set simulator plays an important role in the design process of the AI chip. Therefore, it is important to provide a method for efficiently and accurately verifying the AI instruction set simulator.
Disclosure of Invention
In view of the above, the present disclosure at least provides a method and an apparatus for testing an instruction set simulator, an electronic device and a storage medium.
In a first aspect, the present disclosure provides a method for testing an instruction set simulator, including:
acquiring an executable file corresponding to the instruction set simulator and a test file matched with the test content of the test point; wherein the test point comprises at least one instruction to be tested;
executing the executable file based on the test file to generate output data corresponding to the test content of the test point;
and determining a test result corresponding to the instruction set simulator based on the output data and preset data corresponding to the test content of the test point.
In the method, the executable file corresponding to the instruction set simulator is executed by utilizing the test file matched with the test content of the test point, and the output data corresponding to the test content of the test point is generated; because the preset data corresponding to the test content of the test point is the true value data corresponding to the test content, the output data is compared with the preset data corresponding to the test content of the test point, and the test result of the instruction set simulator can be determined more accurately. Meanwhile, in the test process, cross verification of the output data of the instruction set simulator and simulation results of other simulation models is not needed, the self-test regression process of the instruction set simulator is realized, and the test efficiency of the instruction set simulator is improved.
In a possible embodiment, before obtaining the executable file corresponding to the instruction set simulator and the test file matching the test content of the test point, the method further includes:
carrying out test point decomposition on the instruction set corresponding to the instruction set simulator to obtain at least one test point and test contents corresponding to each test point; wherein the at least one test point includes at least one instruction of the set of instructions to be tested.
In the above embodiment, the test point decomposition is performed on the instruction set corresponding to the instruction set simulator to obtain at least one test point and test contents corresponding to each test point, so that each test content of each test point can be tested, the test range is comprehensive and detailed, and the test accuracy of the instruction set simulator is improved.
In a possible implementation manner, the test file includes an instruction assembly file corresponding to the instruction to be tested and a configuration file required for executing the executable file according to the instruction to be tested;
wherein the configuration file comprises:
a first configuration file for indicating a storage address of input data corresponding to the instruction set simulator;
a second configuration file for indicating a storage address of output data corresponding to the instruction set simulator;
a third configuration file for indicating data content of input data corresponding to the instruction set simulator;
and the fourth configuration file is used for indicating the real-time operating system kernel configuration information required by the operation of the instruction set simulator.
In a possible embodiment, the executing the executable file based on the test file to generate output data corresponding to the test content of the test point includes:
compiling the instruction assembly file by using a preset assembler to generate an instruction coding file;
and executing the executable file based on the instruction coding file and the configuration file to generate output data corresponding to the test content of the test point.
In the above embodiment, the preset assembler is used to compile the instruction assembly file to generate an instruction code file, and since the instruction code file is compiled by machine code, the instruction code file can be directly read by the processor to complete the process of executing the executable file, generate output data corresponding to the test content of the test point, and provide data support for subsequently determining the test result of the instruction set simulator.
In a possible embodiment, in a case that the test point is multiple and/or each test point includes multiple test contents, the executing the executable file based on the test file to generate output data corresponding to the test contents of the test point includes:
taking each test content of each test point as a current test content;
executing the executable file based on the test file corresponding to the current test content to generate output data corresponding to the current test content;
the determining the test result corresponding to the instruction set simulator based on the output data and the preset data corresponding to the test content of the test point comprises:
and responding to the output data corresponding to each test content of each test point, and determining the test result corresponding to the instruction set simulator based on the output data corresponding to each test content and the preset data corresponding to the test content.
Here, when the test point is multiple or the test point includes multiple test contents, each test content may be tested separately to obtain output data corresponding to the test content; after the output data corresponding to each test content of each test point is detected, the test result corresponding to the instruction set simulator is more accurately determined based on the output data corresponding to each test content and the preset data corresponding to the test content, and the test accuracy of the instruction set simulator is improved.
In one possible embodiment, after determining the test result corresponding to the instruction set simulator, the method further includes:
and in response to the test result being that the test fails, determining an adjustment strategy corresponding to the instruction set simulator.
Here, in response to that the test result is that the test does not pass, an adjustment policy corresponding to the instruction set simulator may be determined, so as to adjust at least one of the instruction assembly file, the configuration file, and the instruction set simulator by using the adjustment policy, for example, the instruction assembly file and the configuration file may be adjusted, so as to perform a relatively accurate test on the instruction set simulator by using the adjusted instruction assembly file and configuration file, thereby improving the test accuracy of the instruction set simulator; alternatively, the instruction set simulator may be adjusted to improve the performance of the instruction set simulator.
The following descriptions of the effects of the apparatus, the electronic device, and the like refer to the description of the above method, and are not repeated here.
In a second aspect, the present disclosure provides a test apparatus for an instruction set simulator, including:
the acquisition module is used for acquiring an executable file corresponding to the instruction set simulator and a test file matched with the test content of the test point; wherein the test point comprises at least one instruction to be tested;
the generating module is used for executing the executable file based on the test file and generating output data corresponding to the test content of the test point;
and the determining module is used for determining a test result corresponding to the instruction set simulator based on the output data and preset data corresponding to the test content of the test point.
In a possible embodiment, before the obtaining an executable file corresponding to the instruction set simulator and a test file matching test contents of a test point, the method further includes:
the decomposition module is used for decomposing the test points of the instruction set corresponding to the instruction set simulator to obtain at least one test point and test contents corresponding to each test point; wherein the at least one test point includes at least one instruction of the set of instructions to be tested.
In a third aspect, the present disclosure provides an electronic device comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is running, the machine-readable instructions when executed by the processor performing the steps of the method of testing an instruction set simulator as described in the first aspect or any embodiment above.
In a fourth aspect, the present disclosure provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method for testing an instruction set simulator as described in the first aspect or any one of the embodiments above.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.
FIG. 1 is a schematic flow chart diagram illustrating a method for testing an instruction set simulator provided in an embodiment of the present disclosure;
fig. 2 is a schematic diagram illustrating a test process of each test content in a test method of an instruction set simulator provided by an embodiment of the present disclosure;
FIG. 3 is a flow chart illustrating another method for testing an instruction set simulator provided by an embodiment of the disclosure;
FIG. 4 is a schematic diagram illustrating an architecture of a test apparatus of an instruction set simulator according to an embodiment of the present disclosure;
fig. 5 shows a schematic structural diagram of an electronic device provided in an embodiment of the present disclosure.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions in the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the disclosure, provided in the accompanying drawings, is not intended to limit the scope of the disclosure, as claimed, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.
The AI instruction set simulator needs to meet the requirements provided in the design process of the AI chip, i.e., the AI instruction set simulator is used to simulate various instruction information required by the operation of the AI chip. It can be known that the performance of the AI instruction set simulator plays an important role in the design process of the AI chip.
Generally, the output result of the AI instruction set simulator and the simulation result of the Register Transfer Level (RTL) simulation model may be cross-verified to determine the test result of the AI instruction set simulator. For example, a test file of the to-be-tested instruction corresponding to the AI chip may be designed, and the test file may be transmitted to the AI instruction set simulator and the RTL simulation model, respectively, to obtain an output result of the AI instruction set simulator and a simulation result of the RTL simulation model. And if the output result is consistent with the simulation result, the test result of the AI instruction set simulator is that the test is passed. If the two simulation models are inconsistent, whether an AI instruction set simulator or an RTL simulation model has a fault needs to be determined. The test mode of the AI simulator cannot achieve self-test regression of the AI simulator, and the cross validation process enables the test process of the AI simulator to be manually completed, so that the test process is complicated and the test efficiency is low.
In order to alleviate the above problem, embodiments of the present disclosure provide a method and an apparatus for testing an instruction set simulator, an electronic device, and a storage medium.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
For the convenience of understanding the embodiments of the present disclosure, a detailed description will be first given of a method for testing an instruction set simulator disclosed in the embodiments of the present disclosure. The execution subject of the test method of the instruction set simulator provided by the embodiment of the present disclosure is generally a computer device with certain computing power, and the computer device includes, for example: a terminal device, which may be a User Equipment (UE), a mobile device, a User terminal, a Personal Digital Assistant (PDA), or the like, or a server or other processing device. In some possible implementations, the method for testing the instruction set simulator may be implemented by a processor calling computer readable instructions stored in a memory.
Referring to fig. 1, a schematic flow chart of a method for testing an instruction set simulator provided in the embodiment of the present disclosure is shown, the method includes S101-S103, where:
s101, obtaining an executable file corresponding to the instruction set simulator and a test file matched with test contents of a test point; wherein the test point comprises at least one instruction to be tested.
S102, based on the test file, executing the executable file, and generating output data corresponding to the test content of the test point.
S103, determining a test result corresponding to the instruction set simulator based on the output data and preset data corresponding to the test content of the test point.
In the method, the executable file corresponding to the instruction set simulator is executed by utilizing the test file matched with the test content of the test point, and the output data corresponding to the test content of the test point is generated; the preset data corresponding to the test content of the test point is the true value data corresponding to the test content, and the output data is compared with the preset data corresponding to the test content of the test point, so that the test result of the instruction set simulator can be determined more accurately. Meanwhile, in the test process, the output data of the instruction set simulator and the simulation results of other simulation models do not need to be subjected to cross verification, the self-test regression process of the instruction set simulator is realized, and the test efficiency of the instruction set simulator is improved.
S101 to S103 will be specifically described below.
For S101:
the executable file corresponding to the instruction set simulator may be a binary file obtained by compiling the instruction set simulator, and the binary file may be loaded and executed by an operating system. The test points are obtained by carrying out test point decomposition on an instruction set corresponding to the instruction set simulator, each test point comprises at least one instruction to be tested, and each test point corresponds to at least one test content.
Before obtaining the executable file corresponding to the instruction set simulator and the test file matched with the test content of the test point, the method further comprises the following steps: carrying out test point decomposition on an instruction set corresponding to the instruction set simulator to obtain at least one test point and test contents corresponding to each test point; wherein at least one test point comprises at least one instruction to be tested of an instruction set.
During implementation, test point decomposition can be carried out on the instruction set according to the type of the instruction to obtain at least one test point; for example, the at least one test point may include: arithmetic operation arithmectic instruction test point, control instruction test point, memory instruction test point and the like. Or, meaningful combination can be carried out according to the instruction of the AI chip to obtain at least one test point; for example, in the control instruction, various combinations of branch instructions and loop instructions, various combinations of branch instructions and unconditional jmp instructions, and the like are provided, and each combination scenario is a test point.
In this embodiment, at least one test point obtained by decomposition may satisfy the following requirements: the test points cover all instruction types in the instruction set; such as Arithmetic and Logic Unit (ALU) instructions, memory access related instructions, special instructions (e.g., shuffle instructions), control instructions, etc. The test point covers the operational level of the instruction, such as a thread beam warp instruction, a thread read instruction, and the like. The test point covers the configurable information of each instruction control Ctrl field, and can also cover the relevant information of real-time operating system kernel configuration, the data types and register types supported by the instructions, the combination scenes among the instructions and the like.
For example, the test points and the test contents of the test points obtained by decomposition may include the following:
the arithmectic instruction test point comprises the following steps: 1) numerical systems that may be supported (float32, signed Long32, signed Long32, float16, bridge float16, signed short16, signed short16, signed char8, signed char 8); 2) rounding modes (nearest rounding mode RN, positive infinite rounding mode RP, zero rounding mode RZ, infinite rounding mode RI); 3) whether to saturation clipping; 4) the operand is a register or an immediate.
Second, logic instruction and shift instruction test point includes: 1) possibly supported numerical systems (s32, u32, u16, s16, s8, u 8); 2) whether the result is stored in a predicate register (for the logic instruction); 3) the operand is an immediate.
Thirdly, the memory access instruction test point comprises: 1) the address source covers global memory, shared memory, local memory and the like; 2) an address bit width; 3) the operand of the address source covers a thread beampregfileregister, a thread threeadregfile register and a global constregfile register; 4) a continuous address access mode; 5) the atomic operation covers the scenario of whether there is a return value.
And fourthly, testing points of special instructions, including: 1) a supported number system; 2) four modes of the shuffle instruction.
Fifthly, controlling the test point of the control instruction, comprising: 1) except for basic functions, the method covers the scenarios of nesting of if else, nesting of if else and loop, and the like; 2) if else nests the precedence of the branch instruction rendezvous point in the middle.
Sixthly, the test points of the relevant information configured by the kernel comprise: 1) the number of threads in each block (maximum, boundary, integer multiple of 32, integer multiple of non-32, etc.); 2) when the number of threads in each block is not an integral multiple of 32, a control instruction is contained; 3) a combination of register resources; 4) the memory resources are shared.
For example, for an arithmectic instruction test point, when performing addition operation between floating point numbers, test contents may be as follows: under the condition that saturation amplitude limiting needs to be considered and an infinite rounding mode RI is adopted, 2 floating point numbers with the data type of float32 are subjected to addition operation to obtain an operation result, namely output data, whether the output data is consistent with preset data corresponding to the test content of the test point or not is judged, and if the output data is consistent with the preset data, the test result of the test content of the test point is determined to be passed. And if the test results are not consistent, determining that the test result of the test content of the test point is not passed. The test points and the test contents of the test points are only exemplary illustrations, and in implementation, the test contents included in the test points and the test contents included in the test points can be set according to project requirements. For example, the test point may further include a matrix multiply instruction test point, a matrix accumulate multiply instruction test point, and the like.
After the test point and the test content of the test point are obtained, a test file corresponding to the test content can be generated according to the test content of the test point. The test file comprises an instruction assembly file corresponding to an instruction to be tested of the test point and a configuration file required by executing the executable file according to the instruction to be tested. One test point may correspond to one instruction assembly file or a plurality of instruction assembly files. The instruction assembly file is a source program file written in an assembly language.
The configuration files may include the following files:
the first configuration file is used for indicating the storage address of input data corresponding to the instruction set simulator.
And the second configuration file is used for indicating the storage address of the output data corresponding to the instruction set simulator.
And thirdly, a third configuration file used for indicating the data content of the input data corresponding to the instruction set simulator.
And fourthly, a fourth configuration file used for indicating the real-time operating system kernel configuration information required by the operation of the instruction set simulator.
Referring to fig. 2, a first configuration file is used for indicating a storage address of input data corresponding to an instruction set simulator, a second configuration file is used for indicating a storage address of output data corresponding to the instruction set simulator, and a third configuration file is used for indicating data content of the input data corresponding to the instruction set simulator, such as hexadecimal numbers in fig. 2: 05e23536, etc.; the fourth configuration file is used for indicating kernel configuration information required by the instruction set simulator to operate.
In implementation, the test file can be adjusted in response to the triggered file adjustment operation to obtain an adjusted test file, and the adjusted test file is stored in the preset address, so that the subsequent execution main body can obtain the adjusted test file from the preset address, and the adjusted test file can be used for testing the instruction set simulator.
In the above embodiment, the test point decomposition is performed on the instruction set corresponding to the instruction set simulator to obtain at least one test point and test contents corresponding to each test point, so that each test content of each test point can be tested, the test range is comprehensive and detailed, and the test accuracy of the instruction set simulator is improved.
Meanwhile, after the adjusted instruction set simulator is obtained in response to the adjustment operation of the instruction set simulator, the test point decomposition result in the embodiment can be subjected to inheritance multiplexing in the test process of the adjusted instruction set simulator, and the speed of optimizing the instruction set simulator is improved.
For S102:
in implementation, the executable file may be executed based on the test file to generate output data corresponding to the test content of the test point. For example, if the test point is a memory storage instruction test point and the test content is data read from global, the output data may be storage data obtained from a specific address in global. If the test point is the arithmetric instruction test point and the test content is the addition of float32 numbers, the output data can be the sum of two float32 numbers.
In a possible embodiment, executing the executable file based on the test file to generate output data corresponding to the test content of the test point includes:
and S1021, compiling the instruction assembly file by using a preset assembler to generate an instruction coding file.
S1022, based on the instruction coding file and the configuration file, the executable file is executed, and output data corresponding to the test content of the test point is generated.
Here, the instruction assembly file written in the assembly language may be compiled into an instruction encoding file written in machine code by a preset assembler, and the executable file corresponding to the instruction set simulator is executed by using the instruction encoding file and the configuration files, such as the first configuration file, the second configuration file, the third configuration file and the fourth configuration file, to generate the output data corresponding to the test content of the test point.
In the above embodiment, the preset assembler is used to compile the instruction assembly file to generate the instruction code file, and since the instruction code file is compiled by machine code, the instruction code file can be called by a computer to complete the process of executing the executable file, generate the output data corresponding to the test content of the test point, and provide data support for subsequently determining the test result of the instruction set simulator.
Referring to fig. 2, the instruction code file may be input to an assembler (i.e., a preset assembler), and the preset assembler compiles the instruction assembly file to generate the instruction code file. And then inputting the instruction coding file and the configuration files (namely the first configuration file, the second configuration file, the third configuration file and the fourth configuration file) into the executable file to generate output data corresponding to the test content of the test point. Furthermore, the output data can be compared with preset data corresponding to the test content of the test point to obtain a test result.
For example, when the test point is an arithmetric instruction test point and the test content is an addition to the float32 number system, as shown in fig. 2, a first configuration file, a second configuration file, a third configuration file, and a fourth configuration file corresponding to the test content of the test point may be generated, and an instruction assembly file may be compiled by an assembler to generate an instruction encoding file related to the addition instruction information. Then, based on the instruction encoding file and the configuration file, the executable file is executed, that is, the executable file executes the addition operation of the data of the two float32 numbers, so as to generate the output data corresponding to the test content of the test point, and the output data can be stored in the storage address indicated by the second configuration file. So as to compare the output data stored in the storage address indicated by the second configuration file with the preset data to obtain a test result; the preset data may be true value data obtained by two float32 numerical system additions. And if the output data is not consistent with the preset data, determining that the test result of the instruction set simulator under the test content of the test point is a test failure.
For another example, when the test point is a memory access instruction test point and the test content is data read from a specific address of global, first, a first configuration file, a second configuration file, a third configuration file, and a fourth configuration file corresponding to the test content of the test point may be generated, and an instruction encoding file related to instruction fetching information may be generated by compiling the instruction assembly file through an assembler. Then, based on the instruction encoding file and the configuration file, the executable file is executed, that is, the executable file executes data reading operation, output data corresponding to the test content of the test point is generated, and the output data can be stored in the storage address indicated by the second configuration file. And comparing the output data stored in the storage address indicated by the second configuration file with preset data to obtain a test result, wherein the preset data is true value data stored in a global specific address.
For S103:
during implementation, the output data and the preset data corresponding to the test content of the test point can be compared to determine the test result corresponding to the instruction set simulator. When the test content of the test point is one, if the output data corresponding to the test content of the test point is consistent with the preset data corresponding to the test content of the test point, determining that the test result of the instruction set simulator passes the test. If not, determining that the test result of the instruction set simulator is a test failure.
When the number of the test points is multiple and/or the number of the test contents is multiple, determining whether the output data corresponding to the test contents is consistent with the preset data corresponding to the test contents for each test content of each test point to obtain an intermediate result of the test contents of the test points, wherein the intermediate result comprises: consistent and inconsistent. If the intermediate results corresponding to the test contents of the test points are consistent, determining that the test result corresponding to the instruction set simulator passes the test; if the intermediate result corresponding to the test content of any test point is inconsistent, the test result corresponding to the instruction set simulator is determined to be a test failure, and the test content of the test point with the inconsistent intermediate result can be output.
In a possible embodiment, in a case that a test point is multiple and/or each test point includes multiple test contents, executing an executable file based on a test file to generate output data corresponding to the test contents of the test point, the method includes:
and S1031, respectively taking each test content of each test point as a current test content.
S1032, based on the test file corresponding to the current test content, the executable file is executed, and output data corresponding to the current test content is generated.
Determining a test result corresponding to the instruction set simulator based on the output data and preset data corresponding to the test content of the test point, wherein the test result comprises the following steps: and responding to the detected output data corresponding to each test content of each test point, and determining the test result corresponding to the instruction set simulator based on the output data corresponding to each test content and the preset data corresponding to the test content.
Here, in the case that there is one test point and the test point includes only one test content, a test file corresponding to the test content of the test point may be generated. And compiling the instruction assembly file in the test file by using a preset assembler to generate an instruction coding file. Executing the executable file based on the instruction coding file and the configuration file to generate output data corresponding to the test content of the test point; and determining a test result corresponding to the instruction set simulator based on the output data and preset data corresponding to the test content of the test point. For example, when the output data is consistent with the preset data, the test result of the instruction set simulator is determined to be test pass. And when the output data is inconsistent with the preset data, determining that the test result of the instruction set simulator is that the test is not passed.
When the number of the test points is multiple and/or each test point comprises multiple test contents, each test content of each test point can be respectively used as the current test content, and then the executable file is executed based on the test file corresponding to the current test content to generate the output data corresponding to the current test content. And when detecting that the detection content of any detection point does not have corresponding output data, taking the test content of the detection point without the output data as current test content, returning to the test file corresponding to the current test content, executing the executable file, and generating the output data corresponding to the current test content until each test content of each detection point has corresponding output data.
When it is detected that corresponding output data exists in each test content of each test point, a test result corresponding to the instruction set simulator is determined based on the output data corresponding to each test content and preset data corresponding to the test content of the test point. For example, when the output data corresponding to each test content is consistent with the preset data corresponding to the test content, the test result of the instruction set simulator is determined to be passed. And when the output data corresponding to any test content is inconsistent with the preset data, determining that the test result of the instruction set simulator is that the test fails.
Here, when the test point is multiple or the test point includes multiple test contents, each test content may be tested separately to obtain output data corresponding to the test content; after the output data corresponding to each test content of each test point is detected, the test result corresponding to the instruction set simulator is more accurately determined based on the output data corresponding to each test content and the preset data corresponding to the test content, and the test accuracy of the instruction set simulator is improved.
The method for testing the instruction set simulator is described in conjunction with fig. 3. The method can comprise the following steps:
And step 304, copying the compiled executable file and the assembler corresponding to the source program of the instruction set simulator to the corresponding path of each use case.
And step 306, running an executable file corresponding to the source program of the instruction set simulator, and testing each test content of each test point. Here, each test content of each test point forms a task cluster, and each test content in the task cluster is tested to obtain a test result. Specifically, for each test content, based on the instruction encoding file, the first configuration file, the second configuration file, the third configuration file and the fourth configuration file corresponding to the test content, the executable file corresponding to the source program of the instruction set simulator is run, and the output data corresponding to the test content is generated.
And 308, comparing the results and outputting a comparison file. Comparing the output data corresponding to each test content of each test point generated in step 306 with the preset data corresponding to the test content, and outputting a comparison file. The comparison file may include an intermediate result corresponding to each test content of each test point. And further determining the test result of the instruction set simulator according to the intermediate result of each test content of each test point.
In one possible embodiment, after determining the test result corresponding to the instruction set simulator, the method further includes: and determining an adjustment strategy corresponding to the instruction set simulator in response to the test result that the test fails.
Here, when the output data corresponding to each test content of each test point is consistent with the preset data corresponding to the test content, the test result corresponding to the instruction set simulator is determined to be a test pass, and at this time, an adjustment strategy does not need to be determined. And if the output data corresponding to any one of the test contents is inconsistent with the preset data corresponding to the test contents, determining that the test result of the instruction set simulator is that the test fails. Meanwhile, the execution subject outputs inconsistent test contents. At least one of the instruction assembly file, the configuration file, and the instruction set simulator may then be adjusted. For example, if it is determined that the first configuration file has a problem according to the inconsistent test content, the first configuration file in the configuration files may be adjusted; if the instruction set simulator is determined to have a problem according to the inconsistent test content, the program of the instruction set simulator can be modified and the like.
Here, in response to that the test result is that the test fails, an adjustment policy corresponding to the instruction set simulator may be determined, so as to adjust at least one of the instruction assembly file, the configuration file, and the instruction set simulator by using the adjustment policy, for example, the instruction assembly file and the configuration file may be adjusted, so as to perform a more accurate test on the instruction set simulator by using the adjusted instruction assembly file and configuration file, thereby improving the test accuracy of the instruction set simulator; alternatively, the instruction set simulator may be adjusted to improve the performance of the instruction set simulator.
It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.
Based on the same concept, an embodiment of the present disclosure further provides a test apparatus for an instruction set simulator, as shown in fig. 4, which is an architecture schematic diagram of the test apparatus for an instruction set simulator provided in the embodiment of the present disclosure, and includes an obtaining module 401, a generating module 402, and a determining module 403, specifically:
an obtaining module 401, configured to obtain an executable file corresponding to the instruction set simulator and a test file matching test contents of the test point; wherein the test point comprises at least one instruction to be tested;
a generating module 402, configured to execute the executable file based on the test file, and generate output data corresponding to test content of the test point;
a determining module 403, configured to determine a test result corresponding to the instruction set simulator based on the output data and preset data corresponding to the test content of the test point.
In a possible embodiment, before the obtaining an executable file corresponding to the instruction set simulator and a test file matching test contents of a test point, the method further includes:
a decomposition module 404, configured to perform test point decomposition on the instruction set corresponding to the instruction set simulator to obtain at least one test point and test content corresponding to each test point; wherein the at least one test point comprises at least one instruction to be tested of the instruction set.
In a possible implementation manner, the test file includes an instruction assembly file corresponding to the instruction to be tested and a configuration file required for executing the executable file according to the instruction to be tested;
wherein the configuration file comprises:
a first configuration file for indicating a storage address of input data corresponding to the instruction set simulator;
a second configuration file for indicating a storage address of output data corresponding to the instruction set simulator;
a third configuration file for indicating data content of input data corresponding to the instruction set simulator;
and the fourth configuration file is used for indicating the real-time operating system kernel configuration information required by the instruction set simulator to run.
In a possible implementation manner, the generating module 402, when executing the executable file based on the test file to generate output data corresponding to the test content of the test point, is configured to:
compiling the instruction assembly file by using a preset assembler to generate an instruction coding file;
and executing the executable file based on the instruction coding file and the configuration file to generate output data corresponding to the test content of the test point.
In a possible implementation manner, the generating module 402, when the test point is multiple and/or each test point includes multiple test contents, the executable file is executed based on the test file to generate output data corresponding to the test contents of the test point, and is configured to:
taking each test content of each test point as the current test content;
executing the executable file based on the test file corresponding to the current test content to generate output data corresponding to the current test content;
the determining module 403, when determining the test result corresponding to the instruction set simulator based on the preset data corresponding to the test content of the test point and the output data, is configured to:
and responding to the output data corresponding to each test content of each test point, and determining the test result corresponding to the instruction set simulator based on the output data corresponding to each test content and the preset data corresponding to the test content.
In one possible implementation, after determining the test result corresponding to the instruction set simulator, the apparatus further includes:
and an adjusting module 405, configured to determine an adjusting policy corresponding to the instruction set simulator in response to that the test result is that the test fails.
In some embodiments, the functions of the apparatus provided in the embodiments of the present disclosure or the included templates may be used to execute the method described in the above method embodiments, and for specific implementation, reference may be made to the description of the above method embodiments, and for brevity, details are not described here again.
Based on the same technical concept, the embodiment of the present disclosure also provides an electronic device 500. Referring to fig. 5, a schematic structural diagram of an electronic device provided in the embodiment of the present disclosure includes a processor 501, a memory 502, and a bus 503. The memory 502 is used for storing execution instructions and includes a memory 5021 and an external memory 5022; the memory 5021 is also referred to as an internal memory, and is used for temporarily storing operation data in the processor 501 and data exchanged with an external storage 5022 such as a hard disk, the processor 501 exchanges data with the external storage 5022 through the memory 5021, and when the electronic device 500 operates, the processor 501 communicates with the storage 502 through the bus 503, so that the processor 501 executes the following instructions:
acquiring an executable file corresponding to the instruction set simulator and a test file matched with the test content of the test point; wherein the test point comprises at least one instruction to be tested;
executing the executable file based on the test file to generate output data corresponding to the test content of the test point;
and determining a test result corresponding to the instruction set simulator based on the output data and preset data corresponding to the test content of the test point.
The specific processing flow of the processor 501 may refer to the description of the above method embodiment, and is not described herein again.
In addition, the embodiment of the present disclosure further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program performs the steps of the method for testing the instruction set simulator described in the above method embodiment. The storage medium may be a volatile or non-volatile computer-readable storage medium.
The embodiments of the present disclosure also provide a computer program product, where the computer program product carries a program code, and instructions included in the program code may be used to execute steps of the method for testing an instruction set simulator in the above method embodiments, which may be referred to specifically for the above method embodiments, and are not described herein again.
The computer program product may be implemented by hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK) or the like.
It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working process of the system and the apparatus described above may refer to the corresponding process in the foregoing method embodiment, and details are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in software functional units and sold or used as a stand-alone product, may be stored in a non-transitory computer-readable storage medium executable by a processor. Based on such understanding, the technical solutions of the present disclosure, which are essential or part of the technical solutions contributing to the prior art, may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above are only specific embodiments of the present disclosure, but the scope of the present disclosure is not limited thereto, and any person skilled in the art can easily think of the changes or substitutions within the technical scope of the present disclosure, and shall cover the scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (10)
1. A method for testing an instruction set simulator, comprising:
acquiring an executable file corresponding to the instruction set simulator and a test file matched with the test content of the test point; wherein the test point comprises at least one instruction to be tested;
executing the executable file based on the test file to generate output data corresponding to the test content of the test point;
and determining a test result corresponding to the instruction set simulator based on the output data and preset data corresponding to the test content of the test point.
2. The method of claim 1, wherein before the obtaining an executable file corresponding to the instruction set simulator and a test file matching test contents of the test point, further comprising:
carrying out test point decomposition on the instruction set corresponding to the instruction set simulator to obtain at least one test point and test contents corresponding to each test point; wherein the at least one test point includes at least one instruction of the set of instructions to be tested.
3. The test method according to claim 1 or 2, wherein the test file comprises an instruction assembly file corresponding to the instruction to be tested and a configuration file required for executing the executable file according to the instruction to be tested;
wherein the configuration file comprises:
a first configuration file for indicating a storage address of input data corresponding to the instruction set simulator;
a second configuration file for indicating a storage address of output data corresponding to the instruction set simulator;
a third configuration file for indicating data content of input data corresponding to the instruction set simulator;
and the fourth configuration file is used for indicating the real-time operating system kernel configuration information required by the operation of the instruction set simulator.
4. The method according to any one of claims 1 to 3, wherein the executing the executable file based on the test file to generate output data corresponding to test contents of the test point comprises:
compiling the instruction assembly file by using a preset assembler to generate an instruction coding file;
and executing the executable file based on the instruction coding file and the configuration file to generate output data corresponding to the test content of the test point.
5. The test method according to any one of claims 1 to 4, wherein in a case where the test point is plural and/or each test point includes plural test contents, the executing the executable file based on the test file to generate output data corresponding to the test contents of the test point includes:
taking each test content of each test point as the current test content;
executing the executable file based on the test file corresponding to the current test content to generate output data corresponding to the current test content;
the determining the test result corresponding to the instruction set simulator based on the output data and the preset data corresponding to the test content of the test point comprises:
and responding to the output data corresponding to each test content of each test point, and determining the test result corresponding to the instruction set simulator based on the output data corresponding to each test content and the preset data corresponding to the test content.
6. The testing method of any of claims 1-5, wherein after determining the test result corresponding to the instruction set simulator, the method further comprises:
and in response to the test result being that the test fails, determining an adjustment strategy corresponding to the instruction set simulator.
7. An apparatus for testing an instruction set simulator, comprising:
the acquisition module is used for acquiring an executable file corresponding to the instruction set simulator and a test file matched with the test content of the test point; wherein the test point comprises at least one instruction to be tested;
the generating module is used for executing the executable file based on the test file and generating output data corresponding to the test content of the test point;
and the determining module is used for determining a test result corresponding to the instruction set simulator based on the output data and preset data corresponding to the test content of the test point.
8. The test apparatus according to claim 7, further comprising, before the obtaining of the executable file corresponding to the instruction set simulator and the test file matching the test content of the test point:
the decomposition module is used for decomposing the test points of the instruction set corresponding to the instruction set simulator to obtain at least one test point and test contents corresponding to each test point; wherein the at least one test point includes at least one instruction of the set of instructions to be tested.
9. An electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is running, the machine-readable instructions when executed by the processor performing the steps of the method of testing an instruction set simulator as claimed in any one of claims 1 to 6.
10. A computer-readable storage medium, having stored thereon a computer program for performing the steps of the method of testing a simulator of an instruction set according to any one of claims 1 to 6 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210611601.0A CN114924925A (en) | 2022-05-31 | 2022-05-31 | Instruction set simulator test method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210611601.0A CN114924925A (en) | 2022-05-31 | 2022-05-31 | Instruction set simulator test method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114924925A true CN114924925A (en) | 2022-08-19 |
Family
ID=82812301
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210611601.0A Pending CN114924925A (en) | 2022-05-31 | 2022-05-31 | Instruction set simulator test method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114924925A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115714725A (en) * | 2022-10-17 | 2023-02-24 | 江西科骏实业有限公司 | Network instruction virtual simulation implementation method and system for teaching application |
-
2022
- 2022-05-31 CN CN202210611601.0A patent/CN114924925A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115714725A (en) * | 2022-10-17 | 2023-02-24 | 江西科骏实业有限公司 | Network instruction virtual simulation implementation method and system for teaching application |
| CN115714725B (en) * | 2022-10-17 | 2023-09-12 | 江西科骏实业有限公司 | Teaching application-oriented network instruction virtual simulation implementation method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7270764B2 (en) | artificial intelligence chip verification | |
| Godefroid et al. | Automated synthesis of symbolic instruction encodings from I/O samples | |
| CN109189479B (en) | Parallel automatic verification method for processor instruction set | |
| US20100180263A1 (en) | Apparatus and method for detecting software error | |
| CN107480327B (en) | Simulation verification method and device and electronic equipment | |
| JP6342129B2 (en) | Source code error position detection apparatus and method for mixed mode program | |
| US10936474B2 (en) | Software test program generation | |
| KR101788279B1 (en) | Measuring system and method of worst-case execution time using test techniques | |
| CN115686631B (en) | Random instruction generation method and device based on knowledge base and storage medium | |
| Herdt et al. | Efficient cross-level testing for processor verification: A RISC-V case-study | |
| CN112905447B (en) | Test method and system for block chain virtual machine | |
| CN114924925A (en) | Instruction set simulator test method and device, electronic equipment and storage medium | |
| CN111813672A (en) | Non-invasive coverage rate statistical method for various processor architectures | |
| CN116594682A (en) | Automatic testing method and device based on SIMD library | |
| CN114428642B (en) | Random instruction generation environment based on novel processor architecture | |
| US10409624B1 (en) | Data array compaction in an emulation system | |
| US10776255B1 (en) | Automatic verification of optimization of high level constructs using test vectors | |
| CN117313595B (en) | Random instruction generation method, equipment and system for function verification | |
| CN114428630B (en) | Chip algorithm upgrading method and device and chip | |
| Singhal et al. | Profiling minisat based on user defined execution time--GPROF | |
| Truong et al. | RISC-V random test generator | |
| CN115408264A (en) | RISC-V processor based instruction set verification method and device | |
| US9830174B2 (en) | Dynamic host code generation from architecture description for fast simulation | |
| JP2004118518A (en) | Simulator, and recording medium recording the simulator readable in computer | |
| Domínguez-Sánchez et al. | MBPlib: Modular Branch Prediction Library |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |