WO2021238006A1 - 人工智能芯片验证 - Google Patents

人工智能芯片验证 Download PDF

Info

Publication number
WO2021238006A1
WO2021238006A1 PCT/CN2020/119406 CN2020119406W WO2021238006A1 WO 2021238006 A1 WO2021238006 A1 WO 2021238006A1 CN 2020119406 W CN2020119406 W CN 2020119406W WO 2021238006 A1 WO2021238006 A1 WO 2021238006A1
Authority
WO
WIPO (PCT)
Prior art keywords
use case
verification
template
verified
file
Prior art date
Application number
PCT/CN2020/119406
Other languages
English (en)
French (fr)
Inventor
侯化成
卜凡伟
蒋科
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Priority to JP2021557138A priority Critical patent/JP7270764B2/ja
Priority to KR1020217031299A priority patent/KR20210149045A/ko
Publication of WO2021238006A1 publication Critical patent/WO2021238006A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G06F11/3668—Software testing
    • G06F11/3672—Test management
    • G06F11/3684—Test management for test design, e.g. generating new test cases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/36—Preventing errors by testing or debugging software
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the present disclosure relates to the field of artificial intelligence (AI) chip verification, and specifically to an AI chip verification system, method, device, and storage medium.
  • AI artificial intelligence
  • Verification is a very important stage in the chip development process. Before the chip is delivered to the manufacturer, it needs to be verified to ensure the correctness of the chip's functions.
  • the chips are required to have stronger adaptability and robustness, so that the corresponding requirements for chip function verification can also be adapted to various types of neural networks.
  • the present disclosure provides an AI chip verification system, method, equipment, storage medium, and computer program product.
  • an AI chip verification system includes: a use case generator for generating a verification use case of the object to be verified based on the information of the object to be verified and a preset use case template , Wherein the verification use case satisfies the format defined by the preset use case template, the object to be verified includes the target operator in the neural network model or the neural network model; the chip component to be tested is connected to the use case generator, and The verification use case is executed through the design to be tested to obtain the test result; the comparison component is connected to the chip component to be tested for comparing the test result with the reference result corresponding to the verification use case to obtain the The verification result of the design to be tested on the object to be verified.
  • a verification use case is generated according to a preset use case template in a pre-defined format, so that the AI chip verification system provided in the embodiment of the present disclosure can be adapted to verify various types of neural network models.
  • the preset use case template includes at least one of the following: an instruction flow template file, which contains the instruction flow template of the verification use case; an initialization data template file, which contains initialization data of the verification use case Template; a register configuration template file, which contains the register configuration template of the verification use case.
  • system further includes: an environment driver, which connects the use case generator and the chip component under test, and is configured to drive the verification use case to the In the chip component under test, the verification use case is executed by the design under test.
  • environment driver which connects the use case generator and the chip component under test, and is configured to drive the verification use case to the In the chip component under test, the verification use case is executed by the design under test.
  • the preset use case template includes: a result template file, which contains the output result template of the verification use case; the system further includes: an environment sampler, which connects the chip component under test and the The comparison component is configured to sample the test result from the chip component under test based on the result template file, and transmit the sampled test result to the comparison component.
  • the use case generator includes: a directional generator for obtaining the parameter file of the object to be verified, and by parsing the object parameters contained in the parameter file of the object to be verified, The verification use case of the object to be verified is generated.
  • the hierarchy of the neural network model can be verified, and the entire neural network model can also be verified, which improves the verification depth and flexibility of the neural network for the design under test.
  • the parameter file includes: a fixed-point parameter file and a network structure file.
  • the preset use case template includes: a reference result file containing the reference result corresponding to the verification use case; the reference result used by the comparison component for comparison is obtained from the preset Suppose it is obtained from the use case template.
  • the reference result of the verification use case can be directly obtained from the verification use case, and the verification use case is not required to be executed by referring to the model component, which improves the execution efficiency of the comparison device, thereby improving the verification efficiency of the design under test.
  • the use case generator includes: a random generator for obtaining the configuration file of the object to be verified, randomly generating the object parameters of the object to be verified based on the configuration file, and based on the generated all The object parameter generates the verification use case of the object to be verified.
  • the random generator uses random technology to generate different verification use cases based on specific verification use cases according to the configuration file, so that the verification scenario coverage is more comprehensive, and the verification completeness is effectively improved.
  • system further includes: a reference model component, which connects the use case generator and the comparison component, and is configured to execute the verification use case generated by the use case generator to obtain the Verify the reference result corresponding to the use case.
  • a reference model component which connects the use case generator and the comparison component, and is configured to execute the verification use case generated by the use case generator to obtain the Verify the reference result corresponding to the use case.
  • the preset use case template includes: a dispatch core instruction flow template file, which contains the dispatch core instruction flow template of the verification use case; the system further includes: a dispatch instruction compiler, which is connected to the use case The generator, the chip component under test, and the comparison component are configured to generate the scheduling core instruction stream of the design under test based on the scheduling core instruction flow template file, wherein the scheduling core of the design under test The instruction flow is included in the verification use case; the design under test processes the instruction flow in the verification use case according to the scheduling core instruction flow in the verification use case.
  • other modules of the AI chip verification system for the level 1 DUT are multiplexed. It reduces the workload of chip verification, improves the verification efficiency of the design under test, and saves verification resources.
  • the system further includes a performance statistic, connected to the chip component under test, for obtaining performance parameters of the design under test to execute the verification use case, and obtaining the performance parameters based on the performance parameters Performance statistics.
  • the performance statistic can count the calculation time and bandwidth of the verification use case at the level of the neural network model for the design to be tested, and it can also calculate the calculation time and bandwidth of the verification use case at the level of the neural network model for the design to be tested. Bandwidth statistics enrich the performance statistics for design verification under test.
  • an AI chip verification method including: generating a verification use case of the subject to be verified based on the information of the subject to be verified and a preset use case template, wherein the verification use case satisfies all requirements.
  • the format defined by the preset use case template, the object to be verified includes the target operator in the neural network model or the neural network model; the verification use case is executed through the design to be tested to obtain the test result; the test result is compared with the The reference results corresponding to the verification use cases are compared, and the verification results of the design under test on the object to be verified are obtained.
  • the verification use case generated based on the information of the object to be verified is a verification use case generated based on a pre-defined format in a preset use case template.
  • the verification use cases generated by the objects to be verified corresponding to different types of neural networks have a unified format, so the AI chip verification system provided in the embodiments of the present disclosure can be adapted to verify multiple types of neural networks.
  • the preset use case template includes at least one of the following: an instruction flow template file, which contains the instruction flow template of the verification use case; an initialization data template file, which contains initialization data of the verification use case Template; a register configuration template file, which contains the register configuration template of the verification use case.
  • the method further includes: based on the preset use case template, adding the verification use case to Drive into the chip component under test to execute the verification use case by the design under test.
  • the preset use case template includes: a result template file, which contains the output result template of the verification use case; after the verification use case is executed through the design under test and the test result is obtained, it further includes : Based on the result template file, sample the test result from the chip component to be tested, and transmit the sampled test result to the comparison component.
  • the generating a verification use case of the subject to be verified based on the information of the subject to be verified and a preset use case template includes: obtaining a parameter file of the subject to be verified, and analyzing the subject to be verified. Verify the object parameters included in the parameter file of the object to generate the verification use case of the object to be verified.
  • the parameter file includes: a fixed-point parameter file and a network structure file.
  • the preset use case template includes: a reference result file containing the reference result of the verification use case.
  • the generating a verification use case of the subject to be verified based on the information of the subject to be verified and a preset use case template includes: obtaining a configuration file of the subject to be verified, and randomly based on the configuration file Generate object parameters of the object to be verified, and based on the generated object parameters, generate the verification use case of the object to be verified.
  • the method further includes: executing the verification use case through a reference model component to obtain a corresponding reference result.
  • the preset use case template includes: a dispatch core instruction flow template file, including a dispatch core instruction flow template of the verification use case; the method further includes: based on the dispatch core instruction flow template file , Generating the scheduling core instruction flow of the design under test; generating the verification use case of the object to be verified based on the scheduling core instruction flow; the design under test according to the scheduling core instruction flow in the verification use case, Process the instruction stream in the verification use case.
  • the method further includes: obtaining performance parameters of the design to be tested for executing the verification use case, and obtaining performance statistical results based on the performance parameters.
  • a computer device including a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the second aspect when the program is executed.
  • a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the processor is prompted to implement any of the processes described in the second aspect AI chip verification method.
  • a computer program product including a computer program, which, when executed by a processor, prompts the processor to implement the AI chip verification method described in the second aspect.
  • Fig. 1 is a schematic diagram showing an AI chip verification system according to an exemplary embodiment.
  • Fig. 2 is a schematic diagram showing a preset use case template according to an exemplary embodiment.
  • Fig. 3 is a schematic diagram showing an AI chip verification system according to another exemplary embodiment.
  • Fig. 4 is a schematic diagram showing a preset use case template according to another exemplary embodiment.
  • Fig. 5 is a schematic diagram showing an AI chip verification system according to another exemplary embodiment.
  • Fig. 6 is a schematic diagram showing an AI chip verification system according to still another exemplary embodiment.
  • Fig. 7 is a schematic diagram showing a configuration file according to an exemplary embodiment.
  • Fig. 8 is a schematic diagram showing a secondary design under test according to an exemplary embodiment.
  • Fig. 9 is a schematic diagram showing an AI chip verification system according to still another exemplary embodiment.
  • Fig. 10 is a flowchart showing an AI chip verification method according to an exemplary embodiment.
  • first, second, third, etc. may be used in this disclosure to describe various information, the information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other.
  • first information may also be referred to as second information, and similarly, the second information may also be referred to as first information.
  • word “if” as used herein can be interpreted as "when” or “when” or “in response to a certainty”.
  • FIG. 1 is a schematic diagram of an AI chip verification system shown in an embodiment of the present disclosure.
  • the AI chip verification system may include: a use case generator 101, a chip component under test 102, and a comparison component 103.
  • the use case generator 101 is used to generate a verification use case, which is input to
  • the chip component under test 102 is processed by the design under test to obtain a test result, and the test result and the verification case corresponding to the test result are input to the comparison component 103 for processing, and the verification result of the design under test is obtained.
  • the AI chip verification it can be verified whether the newly designed AI chip can implement certain operations (operators) in the neural network model, such as convolution and pooling in the neural network. Can it be used? The AI chip successfully performs these operations. In this case of verifying some operators in the neural network model, the operators in the neural network model are used as the object to be verified. In some examples, it can also verify whether the newly designed AI chip can successfully build a complete neural network model. In this case of verifying the complete neural network model, the entire neural network model is used as the object to be verified.
  • the neural network model may be a neural network used for image recognition, such as a convolutional neural network, it may be a neural network used for speech recognition or a neural network used for text recognition, which is not limited in the present disclosure.
  • the use case generator 101 can parse the information of the object to be verified to obtain parameters for generating the verification use case of the object to be verified.
  • the information of the object to be verified can be extracted from the design specification of the object to be verified by referring to the characteristics of the object to be verified, for example.
  • the information of the object to be verified can be the parameter information contained in the fixed-point parameter file and the network structure file, or it can be the parameter information in the configuration file corresponding to the object to be verified.
  • the fixed-point parameter file may be, for example, a parameter for fixed-point processing of floating-point parameters for the characteristics of the AI network.
  • the use case generator 101 can parse the parameter information in the fixed-point parameter file and the network structure file corresponding to the convolution operation to obtain the information used to generate the convolution operation Verify the parameters of the use case.
  • the use case generator 101 can parse the fixed-point parameter file and network structure file corresponding to the neural network model to obtain the parameters of the verification use case used to generate the neural network model.
  • the use case generator 101 After the use case generator 101 obtains the parameters for generating the verification use case of the object to be verified, it generates the verification use case of the object to be verified according to the parameters and based on the format defined in the preset use case template.
  • the preset use case template defines the format of the data in the verification use case to be generated.
  • the preset use case template can define the data format of a specific part of the verification use case to be generated.
  • the preset use case template can define the format of the instruction flow data and the initialization data in the verification use case to be generated, that is, the format of the specific part (the instruction flow data and the initialization data) is defined.
  • the preset use case template can be defined according to the specific situation.
  • the preset use case template may define the format of all data of the verification use case to be generated, which is not limited in the embodiment of the present disclosure.
  • the preset use case template includes at least one of the following: an instruction flow template file, which contains the instruction flow template of the verification use case; an initialization data template file, which contains initialization data of the verification use case Template; a register configuration template file, which contains the register configuration template of the verification use case.
  • the use case generator 101 may generate different data in the verification use case according to each template file in the preset use case template.
  • the use case generator 101 can generate the instruction stream in the format defined by the instruction stream template according to the instruction stream template in the instruction stream template file; generate the initialization data in the format defined by the initialization data template according to the initialization data template in the initialization data template file;
  • the register configuration template in the register configuration template file generates register configuration data in the format defined by the register configuration template.
  • the use case generator 101 generates a verification use case of the object to be verified according to the format defined by the preset use case template.
  • Figure 2 shows a preset use case template.
  • the use case generator 101 can generate the instruction flow in the verification use case according to the "instruction flow” template file; according to the "picture/weight data” template file and the initialization list (Initial list) template file, it can generate the initialization data in the verification use case: “picture/ Weight data” and "Initial list” data; according to the register configuration (Register config) template file, the register configuration data in the verification use case is generated.
  • the chip component under test 102 can obtain the verification use case and execute the verification use case by the DUT (Design Under Test) to obtain the test result.
  • multiple test cases can be used to test the chip component under test.
  • multiple test cases there can be multiple different instruction streams and/or different images/weight data.
  • the system further includes: an environment driver 301, configured to drive the verification use case to the chip component under test 102 based on the preset use case template, so that The design under test executes the verification use case.
  • an environment driver 301 configured to drive the verification use case to the chip component under test 102 based on the preset use case template, so that The design under test executes the verification use case.
  • the environment driver 301 can analyze the specific data in the verification use case, and the analyzed data can be stimulated and driven.
  • the environment driver 301 can send the instruction stream, initialization data, and register configuration data in the verification use case to and/or configure the chip component under test 102 according to the data, so that the design under test can execute the verification use case.
  • the environment driver 301 can analyze the initialization data, instruction flow, and register configuration data in the verification use case, and drive the four parts of the chip component 102 under test through the Driver component in UVM (Universal Verification Methodology).
  • UVM Universal Verification Methodology
  • the first part Use the register configuration data in the verification use case to configure the registers of the design under test.
  • the register configuration operation needs to strictly follow the interface protocols such as AHB (Advanced High performance Bus) and APB (Advanced Peripheral Bus).
  • the second part initialize the external storage model (such as vip model) connected to the design under test.
  • the initialization of the external storage model requires the use of vip backdoor technology to operate the memory area used by the external storage module.
  • the third part initialize the internal storage model (such as memory model) of the design under test.
  • the initialization of the internal storage model of the DUT needs to encapsulate the DUT memory function to perform precise address write operations on the internal shared storage space.
  • Part 4 Encourage and drive the instruction flow in the verification use case. Utilizing the timing characteristics of UVM Driver, the instruction stream in the instruction stream file of the verification use case is sent to the instruction interface of the DUT in the specified format.
  • the environment driver 301 drives the data of the verification use case to the chip component under test 102, and the verification use case is executed by the design under test.
  • the design to be tested included in the chip component under test 102 may be a complete chip of the new design to be verified, or a certain part of the circuit and/or code of the new design that constitutes the complete chip to be verified, such as completing a specific
  • the functional circuits and/or codes are not limited in the present disclosure.
  • the comparison component 103 can obtain the test result of the test design after executing the verification case from the chip component 102 under test.
  • the preset use case template includes: a result template file containing an output result template of the verification use case.
  • the system further includes: an environment sampler 302, configured to sample the test result from the chip component under test 102 based on the result template file, and transmit the sampled test result To the comparison component 103.
  • a dump list template file can be used as a result template file, which contains the output result template of the verification use case.
  • the environment sampler 302 may sample the test results obtained after the test design is executed and the verification use case is executed from the chip component under test 102 according to the output result template in the "Dump list" template file.
  • the environmental sampler 302 is extended by the Monitor component in UVM, the sampling task of the result data dump_data is customized, and the result data in the external storage model is sampled into a temporary array according to the timing requirements in the DUT input/output interface. Then, according to the format of the output result template in the result template file, for example, according to the format of the "Dump list" template file, the temporary array is stored in the file.
  • the sampled result data includes the network-level calculation result of the AI network, and the sampling condition is the signal at the end of the network layer, such as the msg signal.
  • the environment driver 301 and the environment sampler 302 in the AI chip verification system shown in FIG. 3 are independent of the chip component under test 102.
  • the chip component under test 102 can also integrate the environment driver 301 and The environmental sampler 302 is not specifically limited in the present disclosure.
  • the reference result used in the comparison component 103 to compare with the test result may be the result data obtained by the comparison component 103 directly from the verification use case, or it may be the result data obtained after the verification use case is executed by the reference model component.
  • the reference model component is a model for verifying each function of the AI chip or the design to be tested, and is responsible for simulating the logic behavior of the DUT of the design to be tested.
  • the System C coding model can be used to implement a reference model component.
  • the reference model component can be responsible for generating golden results for comparison. When generating the golden results, it is necessary to ensure the consistency of the data collected with the environmental sampler. In some achievable ways, the reference model component can be a single component.
  • the comparison component 103 is responsible for comparing the test results of the environmental sampler 302 with the reference results corresponding to the verification use case for correctness, and supports multiple forms of result data comparison. For example, it supports the comparison of the result data stored in the DUT; it supports the comparison of the network results in the external storage model. Data comparison can be performed at the end of each network layer of the network, and the data results of each layer can be compared; the data results of all network layers can be compared at the end of the verification use case execution. Therefore, R&D personnel can make targeted modifications to the currently verified design under test based on the verification result.
  • the verification use case is generated according to the preset use case template of the predefined format, so that the verification use cases generated by the objects to be verified corresponding to different types of neural networks have a unified format, so the AI chip verification provided in the embodiment of the present disclosure
  • the system can be adapted to verify various types of neural networks.
  • another embodiment of the present disclosure provides an AI chip verification system
  • the use case generator 101 in the system includes an orientation generator 401.
  • the orientation generator 401 can obtain the parameter file of the object to be verified, analyze the object parameters of the object to be verified contained in the parameter file of the object to be verified, and generate a verification use case of the object to be verified.
  • the object parameters in the parameter file are the parameters required by the directional generator 401 when generating the verification use case of the object to be verified.
  • the parameter file may be one or more of a fixed-point parameter file or a network structure file, or may be another parameter file containing object parameters, which is not limited in the embodiment of the present disclosure.
  • the orientation generator 401 can obtain the fixed-point parameter file and the network structure file of the object to be verified, and parse the object parameters contained in the two files, based on the object parameters Generate verification cases for the object to be verified.
  • the directional generator 401 can analyze the fixed-point parameter file and the network structure file of the neural network model, and call the Printf function to generate an instruction stream according to the object parameters obtained by the analysis. Based on the generated instruction flow, the directional generator 401 generates the instruction flow of the verification use case according to the format defined by the instruction flow template in the preset use case template; further, according to other template files in the preset use case template, generates the verification use case Other data to get a complete verification use case.
  • the data of the verification use case can be driven to the chip component under test 102 through the environment driver 301 (not shown in FIG. 5), and the verification use case is executed by the DUT of the design under test.
  • the environment sampler 302 (not shown in FIG. 5) samples to obtain the test result corresponding to the verification use case.
  • the related specific process is similar to the related description in the foregoing embodiment, and will not be repeated here.
  • the verification use case generated by the orientation generator 401 may include the reference result of the verification use case.
  • the preset use case template includes: a reference result file, which contains the reference result of the verification use case.
  • the "level data" template file can be used as a reference result file, and contains the reference result of the verification use case generated by the orientation generator 401.
  • the comparison component 103 can directly obtain the reference result from the verification use case, and compare the reference result with the test result obtained from the chip component 102 to be tested.
  • the orientation generator 401 when it generates a verification use case, regardless of whether it generates or does not generate the reference result of the verification use case, it can execute the verification use case through the reference model component to obtain the reference result.
  • the reference result is transmitted to the comparison component 103.
  • the reference model component is a model for verifying each function of the chip to be verified or the design to be tested, and is responsible for simulating the logic behavior of the DUT of the design to be tested.
  • the comparison component can directly obtain the reference result of the verification use case from the verification use case, and does not need to refer to the model component to execute the verification use case, which improves the execution efficiency of the comparison device, thereby improving the design under test Verification efficiency.
  • the orientation generator 401 may generate a certain layer or certain layers in the neural network model as a verification use case for the object to be verified.
  • a certain convolutional layer in the convolutional neural network model can be generated as a verification use case for the object to be verified.
  • a certain layer or several layers in the neural network model are used as the object to be verified, which can be called the level verification object.
  • the orientation generator 401 can parse the parameter file corresponding to the target network layer in the neural network model, and generate the target network layer according to the format defined by the preset use case template As the verification use case of the level verification object.
  • the target network layer can be one or more network layers to be verified in the neural network model, such as a certain convolutional layer, pooling layer, or multiple convolutional layers, multiple fully connected layers, etc. .
  • the orientation generator 401 generates the convolutional layer as a level verification object based on the format defined by the preset use case template by analyzing the fixed-point parameter file and network structure file corresponding to the convolutional layer in the neural network model Use cases for verification.
  • the orientation generator 401 may also generate a certain complete neural network model as a verification use case of the object to be verified.
  • a verification use case corresponding to the convolutional neural network model can be generated.
  • This kind of the whole neural network model as the object to be verified can be called the network-level verification object.
  • the orientation generator 401 generates the convolutional neural network model as a network-level verification object by analyzing the fixed-point parameter file and network structure file corresponding to the convolutional neural network model, and based on the format defined by the preset use case template Use cases for verification.
  • the AI chip verification method can verify the hierarchy of the neural network model, and can also verify the entire neural network model, which improves the verification depth and flexibility of the neural network for the design under test. Thereby, the defects (Bugs) generated in the chip research and development can be located and corrected more accurately, and the convergence of the chip verification can be accelerated.
  • another embodiment of the present disclosure provides an AI chip verification system.
  • the use case generator 101 in the system includes a random generator 501, a verification use case and a comparison device
  • a reference model part 502 is also included between 103.
  • the reference model component 502 is a model for verifying each function of the AI chip or the design under test, and is responsible for simulating the logic behavior of the DUT of the design under test. For example, it is implemented with the System C coding model, which is responsible for generating the Golden result for comparison. When generating the Golden result, it is necessary to ensure the consistency of the data collected with the environmental sampler.
  • the random generator 501 can obtain the configuration file of the object to be verified, use randomization technology to generate the object parameters of the object to be verified based on the parameters in the configuration file, and generate verification use cases using the randomly generated object parameters.
  • the configuration file is the file that the verifier configures for the design to be tested to be verified.
  • the verifier can use a table tool, such as Excel, to generate a table file of parameter configuration.
  • Figure 7 illustrates a form configuration file.
  • the random generator 501 can parse the configuration file, and use the parameters configured by the configuration file to randomly generate the object parameters of the object to be verified and/or the values of the object parameters. For example, using the random constraint technology of the SystemVerilog language, the source operand range, source operand address, target operand range, target operand address, and calculation parameters of the operator are randomly generated. Taking the configuration file illustrated in Figure 7 as an example, the random generator 501 can parse the n parameters in case-1 in the table configuration file, and use random constraint technology to randomly generate a part of the n parameters and/or the parameters of this part. Numerical value.
  • the “part of the parameter” may be one of the n parameters or several of the n parameters, which is not limited in the embodiment of the present disclosure. After randomization, multiple sets of parameters and/or parameter values that are different from case-1 are obtained, as the object parameters of the object to be verified, the random generator 501 can generate verification use cases based on these randomly generated object parameters and/or parameter values .
  • Figure 7 illustrates a form configuration file configured by the verifier for the neural network model to be verified and the design to be tested.
  • the configuration file includes parameters and/or parameter values for generating multiple specific verification use cases.
  • the row of serial number 1 including the n parameters of case-1: param-1-1 ⁇ param-1-n, corresponding case-2, case-3 ⁇ case-n, etc. It consists of n parameters.
  • the n configuration parameters and their parameter values in each case correspond to a specific verification use case of the case.
  • the random generator 501 can use randomization technology to generate verification use cases with a wider coverage on the basis of specific verification use cases.
  • the AI chip verification system shown in FIG. 6 includes a reference model component 502.
  • the reference model component 502 is used to execute the verification use case generated by the random generator 501 to obtain the reference result of the verification use case.
  • the reference model component 502 is a model for verifying various functions of the chip to be verified or the design to be tested, and is responsible for simulating the logic behavior of the DUT of the design to be tested. For example, it is implemented with the System C coding model, which is responsible for generating the Golden result for comparison. When generating the Golden result, it is necessary to ensure the consistency of the data collected by the environmental sampler. For different chip components 102 to be tested, there may be different reference model components 502.
  • the random generator 501 uses random technology to generate different verification use cases based on specific verification use cases according to the configuration file, so that the verification scenario coverage is more comprehensive, and the verification completeness is effectively improved.
  • the use case generator 101 only includes a random generator 501. In other achievable manners, the use case generator 101 may include the orientation generator 401 and the random generator 501 at the same time.
  • the AI chip verification system shown in FIG. 5 and FIG. 6 are shown separately for the sake of clarity in the description of the embodiment, without any form restriction.
  • the reference model component 502 can be used to execute the test case to obtain the reference result of the verification case; it is also possible to directly use the verification generated by the orientation generator 401 without using the reference model component 502 Reference results of the use case.
  • the design under test to be verified can be divided into a level 1 DUT and a level 2 DUT.
  • the level 2 DUT adds a scheduling core to the level 1 DUT.
  • the level 1 DUT can execute the instruction flow in the verification use case.
  • the scheduling core in the level 2 DUT can execute the scheduling core instruction flow to carry, schedule, control and distribute the instruction flow in the verification use case, and improve the flexibility of the design under test.
  • the level 1 DUT can directly execute the instruction flow generated according to the instruction flow template file in the verification use case; the scheduling core in the level 2 DUT can execute the scheduling core instruction flow generated according to the scheduling core instruction flow template file in the verification use case.
  • the use case generators including random generators and directional generators
  • the AI chip verification system also includes a scheduling instruction compiler 601.
  • the scheduling instruction compiler 601 can compile the instruction control flow to obtain the scheduling core instruction flow executable by the scheduling core in the level 2 DUT.
  • the instruction control flow is an instruction described in a certain programming language for controlling the instruction flow in a verification use case.
  • the scheduling instruction compiler 601 compiles the instruction control flow described in C language code, and outputs the binary assembly file executable by the scheduling core in the level 2 DUT, to obtain the scheduling core instruction stream.
  • the preset use case template may include: a dispatch core instruction flow template file, which contains a dispatch core instruction flow template of the verification use case.
  • the scheduling instruction compiler 601 compiles the instruction control flow, and generates the scheduling core instruction flow in the verification use case according to the format defined by the scheduling core instruction flow template. Take the preset use case template shown in Figure 4 as an example.
  • the use case generator 101 can also generate other data other than the "scheduling core instruction flow" in the level 2 verification use case based on other template files other than the "scheduling core instruction flow template file", thereby generating a complete verification use case corresponding to the level 2 DUT .
  • the AI chip verification system in the embodiment of the present disclosure only adds a scheduling instruction compiler 601 and multiplexes other modules of the AI chip verification system for the level 1 DUT. It reduces the workload of chip verification, improves the verification efficiency of the design under test, and saves verification resources.
  • the AI chip verification system further includes a performance statistic 602 connected to the chip component 102 under test. It is understandable that this is only an exemplary specific structure of the AI chip verification system, and it can also be in other structural forms, such as the verification level 1 DUT described in Figure 1, Figure 3, Figure 5 or Figure 6 above.
  • the performance counter 602 can also be added.
  • the performance statistic unit 602 can obtain the performance parameters of the test design execution verification use case from the chip component 102 under test, and perform performance statistics based on the obtained performance parameters.
  • the performance statistic 602 can count the time for the design under test to execute the verification use case.
  • the performance statistic unit 602 can use the reset signal as a starting point, for example, the reset signal changes from a low level to a high level. Statistics for the entire execution time of the model.
  • the performance statistic unit 602 can count the execution time of the network layer in the neural network model corresponding to the verification use case in the design under test, such as convolution The calculation time of the layer in the design to be tested is counted. For example, if the verification use case executed by the design under test is the verification use case corresponding to the entire neural network model, the performance statistic 602 can count the time for the design under test to calculate the entire neural network model. For example, the entire convolutional neural network model is calculated in the design under test The calculation in the time statistics.
  • the performance statistic unit 602 can also calculate the actual bandwidth of the design under test. Specifically, the performance statistic 602 can count the amount of read and write data of all interfaces in the design to be tested, and calculate the actual bandwidth of the design to be tested based on the calculation time of the calculated design to be tested corresponding to the amount of read and write data, such as the design to be tested. The bus bandwidth of AHB and APB. For example, if the design under test executes a level-level verification use case, the performance statistic 602 may calculate the actual bandwidth of the network layer corresponding to the neural network model of the design under test execution. For example, the actual bandwidth of the verification use case where the design under test executes the convolutional layer.
  • the performance statistic 602 can calculate the actual bandwidth of the design under test to execute the entire neural network model. For example, the actual bandwidth of the entire convolutional neural network for the design under test.
  • a performance statistic is added to the AI chip verification system, which can perform statistics on performance parameters such as the time and bandwidth for the design to be tested to execute the verification use case.
  • the performance statistic can count the calculation time and bandwidth of the verification use cases at the level of the neural network model for the design under test, and it can also calculate the calculation time and bandwidth of the verification use cases at the network level of the entire neural network model for the design under test. Enriched performance statistics for design verification under test.
  • the AI chip verification system may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
  • the codes corresponding to the functions of the above components can be implemented in SystemVerilog language, and the codes are stored in the memory of the server.
  • the code is run on the processor of the server to complete the chip components under test Verification.
  • the above components can be implemented in the form of circuits, such as by one or more application-specific integrated circuits (ASIC), digital signal processors (DSP), programmable logic devices (PLD), field programmable gate arrays (FPGA) ), controller, microcontroller, microprocessor or other electronic components.
  • ASIC application-specific integrated circuits
  • DSP digital signal processors
  • PLD programmable logic devices
  • FPGA field programmable gate arrays
  • controller microcontroller
  • microprocessor microprocessor or other electronic components.
  • an embodiment of the present disclosure shows an AI chip verification method.
  • the AI chip verification method or the AI chip verification system provided by the embodiments of the present disclosure can be applied to servers, terminal devices, or other types of electronic devices that perform AI chip verification.
  • the AI chip verification process shown in Figure 10 includes the following steps.
  • Step 101 Based on the information of the object to be verified and a preset use case template, a verification use case of the object to be verified is generated, wherein the verification use case satisfies the format defined by the preset use case template, and the object to be verified includes a neural network The target operator or neural network model in the model.
  • the information of the object to be verified can be parsed to obtain the parameters of the verification use case used to generate the verification object.
  • the object to be verified can be an operator in a neural network model, or can be an entire neural network model.
  • the information of the object to be verified may be the parameter information contained in the designated parameter file and the network structure file, or it may be the parameter information in the configuration file corresponding to the verification object.
  • the verification use case of the object to be verified is generated according to the generated parameters and based on the format defined in the preset use case template.
  • the preset use case template defines the format of the data in the verification use case to be generated.
  • the parameter file of the object to be verified can be obtained, the object parameters contained in the parameter file of the object to be verified can be parsed, and the verification use case of the object to be verified can be generated.
  • the relevant description can refer to the description of the relevant part of the verification system embodiment.
  • This implementation method can generate corresponding reference results in the verification use case, without the need for reference model components to execute the verification use case, which improves the efficiency of verification of the design under test.
  • it can verify the level of the neural network model, verify the entire neural network model, and improve the verification depth and flexibility of the neural network for the design under test.
  • the configuration file of the object to be verified can be obtained, and the randomization technology is used based on the parameters in the configuration file to randomly generate the object parameters of the object to be verified, and the randomly generated object parameters are used to generate verification use cases.
  • the randomization technology is used based on the parameters in the configuration file to randomly generate the object parameters of the object to be verified, and the randomly generated object parameters are used to generate verification use cases.
  • This implementation method uses random technology to generate different verification use cases based on specific verification use cases, making verification scenarios more comprehensive and effectively improving verification completeness.
  • Step 102 Execute the verification case through the design to be tested to obtain a test result.
  • the verification use case can be executed through the design to be tested, and the test results after execution can be obtained.
  • the design under test may be included in the chip component under test, and based on a preset use case template, the data of the verification use case is driven to the chip component under test through the environment driver to be executed by the design under test.
  • the environment sampler can sample the test results of the verification case from the chip component under test.
  • Step 103 Compare the test result with the reference result corresponding to the verification use case to obtain a verification result of the design under test on the object to be verified.
  • the reference result compared with the test result of the verification use case in this step may be the result data that already exists in the verification use case, or it may be the result data obtained after the verification use case is executed through the reference model component.
  • the reference model component is a model for verifying each function of the AI chip or the design to be tested, and is responsible for simulating the logic behavior of the DUT of the design to be tested.
  • a preset use case template is used to predefine the format of the verification use case, and the verification use cases generated by the objects to be verified corresponding to different types of neural networks have a unified format, so the AI chip verification method can be adapted to verify multiple categories Neural network.
  • the instruction control flow may be compiled to obtain the dispatch core instruction flow executable by the dispatch core in the level 2 DUT.
  • the corresponding preset use case templates include: dispatch core instruction flow template file, which contains the dispatch core instruction flow template of the verification use case.
  • the instruction control flow can be compiled, and the scheduling core instruction flow in the verification use case can be generated according to the format defined by the scheduling core instruction flow template. Then, according to other template files other than the "scheduling core instruction flow template file", other data other than the "scheduling core instruction flow" in the level 2 verification use case is generated, thereby generating a complete verification use case corresponding to the level 2 DUT.
  • the design under test processes the instruction flow in the verification use case according to the dispatch core instruction flow in the verification use case including the 2-level DUT. This method of generating level 2 DUT verification use cases can reuse the modules that generate level 1 DUT verification use cases, reducing the workload of chip verification and improving verification efficiency.
  • the performance parameters of the execution verification use case of the design to be tested can be obtained, and performance statistics can be performed based on the obtained performance parameters. For example, by counting the time for the design to be tested to execute the verification use case, the actual bandwidth of the design to be tested can be counted.
  • the performance statistics of this embodiment can count the calculation time and bandwidth of the verification use case at the level of the neural network model for the design to be tested, and it can also calculate the calculation time and bandwidth of the verification use case at the network level for the design to be tested that executes the entire neural network model. Performing statistics enriches the performance statistics for verification of the design under test.
  • the present disclosure also provides a computer device, including a memory, a processor, and a computer program stored on the memory and running on the processor.
  • the processor can implement the AI of any embodiment of the present disclosure when the processor executes the program. Chip verification method.
  • the present disclosure also provides a computer-readable storage medium on which a computer program is stored.
  • the program is executed by a processor, the processor is prompted to implement the AI chip verification method of any embodiment of the present disclosure.
  • non-transitory computer-readable storage medium may be ROM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., which is not limited in the present disclosure.
  • the present disclosure also provides a computer program product, including a computer program, which implements the AI chip verification method of any embodiment of the present disclosure when the program is executed by a processor.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Neurology (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Test And Diagnosis Of Digital Computers (AREA)
  • Debugging And Monitoring (AREA)
  • Testing Or Measuring Of Semiconductors Or The Like (AREA)
  • Storage Device Security (AREA)

Abstract

一种人工智能AI芯片验证系统、方法、设备及存储介质，其中，该系统包括：用例生成器（101），用于基于待验证对象的信息和预设用例模板，生成所述待验证对象的验证用例，其中，所述验证用例满足所述预设用例模板定义的格式，所述待验证对象包括神经网络模型中的目标算子或者神经网络模型；待测芯片部件（102），连接所述用例生成器（101），用于通过待测设计执行所述验证用例，得到测试结果；比对部件（103），连接所述待测芯片部件（102），用于将所述测试结果与所述验证用例对应的参考结果进行比对，获得所述待测设计对所述待验证对象的验证结果。

Description

人工智能芯片验证 技术领域
本公开涉及人工智能(Artificial Intelligence，AI)芯片验证领域，具体涉及一种AI芯片验证系统、方法、设备及存储介质。
背景技术
验证是芯片研发过程中很重要的一个阶段，在芯片交付给厂家生产之前需要通过验证来保证芯片功能的正确性。
在针对AI芯片的研发工程中，由于多种类别神经网络的引入，要求芯片具备更强的适配性和鲁棒性，从而对应要求芯片功能验证也能够适配多种类别的神经网络。
发明内容
本公开提供了一种AI芯片验证系统、方法、设备、存储介质及计算机程序产品。
根据本公开实施例的第一方面，提供一种AI芯片验证系统，所述系统包括：用例生成器，用于基于待验证对象的信息和预设用例模板，生成所述待验证对象的验证用例，其中，所述验证用例满足所述预设用例模板定义的格式，所述待验证对象包括神经网络模型中的目标算子或者神经网络模型；待测芯片部件，连接所述用例生成器，用于通过待测设计执行所述验证用例，得到测试结果；比对部件，连接所述待测芯片部件，用于将所述测试结果与所述验证用例对应的参考结果进行比对，获得所述待测设计对所述待验证对象的验证结果。
本公开实施例中，根据预先定义格式的预设用例模板生成验证用例，使得本公开实施例中提供的AI芯片验证系统能够适配验证多种类别的神经网络模型。
在一些可选实施例中，所述预设用例模板包括下列中的至少一项：指令流模板文件，包含所述验证用例的指令流模板；初始化数据模板文件，包含所述验证用例的初始化数据模板；寄存器配置模板文件，包含所述验证用例的寄存器配置模板。
在一些可选实施例中，所述系统还包括：环境驱动器，连接所述用例生成器和所述待测芯片部件，用于基于所述预设用例模板，将所述验证用例驱动到所述待测芯片部件中，以由所述待测设计执行所述验证用例。
在一些可选实施例中，所述预设用例模板包括：结果模板文件，包含所述验证用例的输出结果模板；所述系统还包括：环境采样器，连接所述待测芯片部件和所述比对部件，用于基于所述结果模板文件，从所述待测芯片部件采样所述测试结果，并将采样得到的所述测试结果传输至所述比对部件。
在一些可选实施例中，所述用例生成器包括：定向生成器，用于获取所述待验证对象的参数文件，并通过解析所述待验证对象的所述参数文件中包含的对象参数，生成所述待验证对象的所述验证用例。
本公开实施例中，能够对神经网络模型的层级进行验证，还能对整个神经网络模型进行验证，提高了待测设计对神经网络的验证深度和灵活性。
在一些可选实施例中，所述参数文件包括：定点化参数文件和网络结构文件。
在一些可选实施例中，所述预设用例模板包括：参考结果文件，包含所述验证用例对应的参考结果；所述比对部件进行比对所利用的所述参考结果是从所述预设用例模板中获取的。
本公开实施例中，可以直接从验证用例中获取该验证用例的参考结果，不需要参考模型部件执行验证用例，提高了比对装置的执行效率，从而提高了对待测设计的验证效率。
在一些可选实施例中，所述用例生成器包括：随机生成器，用于获取所述待验证对象的配置文件，基于所述配置文件随机生成待验证对象的对象参数，并基于生成的所述对象参数，生成所述待验证对象的所述验证用例。
本公开实施例中，随机生成器根据配置文件，基于特定验证用例，利用随机技术生成不同的验证用例，使得验证场景覆盖更全面，有效提升了验证完备性。
在一些可选实施例中，所述系统还包括：参考模型部件，连接所述用例生成器和所述比对部件，用于执行所述用例生成器生成的所述验证用例，以得到所述验证用例对应的参考结果。
在一些可选实施例中，所述预设用例模板包括：调度核指令流模板文件，包含所述验证用例的调度核指令流模板；所述系统还包括：调度指令编译器，连接所述用例生成器、所述待测芯片部件和所述比对部件，用于基于所述调度核指令流模板文件，生成所述待测设计的调度核指令流，其中，所述待测设计的调度核指令流包括在所述验证用例中；所述待测设计根据所述验证用例中的所述调度核指令流，对所述验证用例中的指令 流进行处理。
本公开实施例中，复用了针对1级DUT的AI芯片验证系统的其他模块。减少了芯片验证工作量，提高了对待测设计的验证效率，节省验证资源。
在一些可选实施例中，所述系统还包括性能统计器，与所述待测芯片部件连接，用于获取所述待测设计执行所述验证用例的性能参数，并基于所述性能参数得到性能统计结果。
本公开实施例中，性能统计器能够对待测设计执行神经网络模型的层级的验证用例的运算时间和带宽进行统计，也能够对待测设计执行整个神经网络模型的网络级的验证用例的运算时间和带宽进行统计，丰富了对待测设计验证的性能统计。
根据本公开实施例的第二方面，提供一种AI芯片验证方法，包括：基于待验证对象的信息和预设用例模板，生成所述待验证对象的验证用例，其中，所述验证用例满足所述预设用例模板定义的格式，所述待验证对象包括神经网络模型中的目标算子或者神经网络模型；通过待测设计执行所述验证用例，得到测试结果；将所述测试结果与所述验证用例对应的参考结果进行比对，获得所述待测设计对所述待验证对象的验证结果。
本公开实施例中，根据待验证对象的信息生成的验证用例，是基于预设用例模板中预先定义的格式生成的验证用例。不同类别的神经网络对应的待验证对象生成的验证用例具有统一的格式，所以本公开实施例中提供的AI芯片验证系统能够适配验证多种类别的神经网络。
在一些可选实施例中，所述预设用例模板包括下列中的至少一项：指令流模板文件，包含所述验证用例的指令流模板；初始化数据模板文件，包含所述验证用例的初始化数据模板；寄存器配置模板文件，包含所述验证用例的寄存器配置模板。
在一些可选实施例中，在所述基于待验证对象的信息和预设用例模板，生成所述待验证对象的验证用例之后，还包括：基于所述预设用例模板，将所述验证用例驱动到所述待测芯片部件中，以由所述待测设计执行所述验证用例。
在一些可选实施例中，所述预设用例模板包括：结果模板文件，包含所述验证用例的输出结果模板；在所述通过待测设计执行所述验证用例，得到测试结果之后，还包括：基于所述结果模板文件，从所述待测芯片部件采样所述测试结果，并将采样得到的所述测试结果传输至所述比对部件。
在一些可选实施例中，所述基于待验证对象的信息和预设用例模板，生成所述待验 证对象的验证用例，包括：获取所述待验证对象的参数文件，并通过解析所述待验证对象的所述参数文件中包含的对象参数，生成所述待验证对象的所述验证用例。
在一些可选实施例中，所述参数文件包括：定点化参数文件和网络结构文件。
在一些可选实施例中，所述预设用例模板包括：参考结果文件，包含所述验证用例的参考结果。
在一些可选实施例中，所述基于待验证对象的信息和预设用例模板，生成所述待验证对象的验证用例，包括：获取所述待验证对象的配置文件，基于所述配置文件随机生成待验证对象的对象参数，并基于生成的所述对象参数，生成所述待验证对象的所述验证用例。
在一些可选实施例中，所述方法还包括：通过参考模型部件执行所述验证用例，以得到对应的参考结果。
在一些可选实施例中，所述预设用例模板包括：调度核指令流模板文件，包含所述验证用例的调度核指令流模板；所述方法还包括：基于所述调度核指令流模板文件，生成所述待测设计的调度核指令流；基于所述调度核指令流，生成所述待验证对象的验证用例；所述待测设计根据所述验证用例中的所述调度核指令流，对所述验证用例中的指令流进行处理。
在一些可选实施例中，所述方法还包括：获取所述待测设计执行所述验证用例的性能参数，并基于所述性能参数得到性能统计结果。
根据本公开实施例的第三方面，提供一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现第二方面中任一项所述的AI芯片验证方法。
根据本公开实施例的第四方面，提供一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时，促使所述处理器实现第二方面中任一所述的AI芯片验证方法。
根据本公开实施例的第五方面，提供一种计算机程序产品，包括计算机程序，所述程序被处理器执行时，促使所述处理器实现第二方面中任一所述的AI芯片验证方法。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，而非限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。
图1是根据一示例性实施例示出的一种AI芯片验证系统示意图。
图2是根据一示例性实施例示出的一种预设用例模板示意图。
图3是根据另一示例性实施例示出的一种AI芯片验证系统示意图。
图4是根据另一示例性实施例示出的一种预设用例模板示意图。
图5是根据又一示例性实施例示出的一种AI芯片验证系统示意图。
图6是根据再一示例性实施例示出的一种AI芯片验证系统示意图。
图7是根据一示例性实施例示出的一种配置文件示意图。
图8是根据一示例性实施例示出的一种二级待测设计示意图。
图9是根据再一示例性实施例示出的一种AI芯片验证系统示意图。
图10是根据一示例性实施例示出的一种AI芯片验证方法流程图。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的具体方式并不代表与本公开相一致的所有方案。相反，它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。
在本公开使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本公开。在本公开和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本公开可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本公开范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可以被解释成为“在…… 时”或“当……时”或“响应于确定”。
为了使本公开提供的AI芯片验证方案更加清楚，下面结合附图和具体实施例对本公开提供的方案执行过程进行详细描述。
参见图1，图1是本公开提供的实施例示出的一种AI芯片验证系统的示意图。其中，如图1所示，该AI芯片验证系统可以包括：用例生成器101、待测芯片部件102和比对部件103，其中，用例生成器101用于生成验证用例，该验证用例被输入到待测芯片部件102中由待测设计进行处理，得到测试结果，该测试结果以及该测试结果对应的验证用例被输入到比对部件103进行处理，得到待测设计的验证结果。
在一些例子中，在AI芯片验证中，可以验证新设计的AI芯片是否能够实现神经网络模型中的某些操作(算子)，比如神经网络中的卷积、池化等，能否用该AI芯片成功执行这些操作。这种验证神经网络模型中某些算子的情况，是将神经网络模型中的算子作为待验证对象。在一些例子中，还可以验证新设计的AI芯片是否能够成功搭建完整的神经网络模型。这种验证完整的神经网络模型的情况，是将整个神经网络模型作为了待验证对象。其中，神经网络模型可以是用于图像识别的神经网络，比如卷积神经网络，可以是用于语音识别的神经网络或者用于文本识别的神经网络，本公开对此不进行限制。
如图1所示的AI芯片验证系统中，用例生成器101可以解析待验证对象的信息，得到用于生成该待验证对象的验证用例的参数。其中，待验证对象的信息例如可以参考待验证对象的特征，从待验证对象的设计规范中提取。待验证对象的信息可以是定点化参数文件和网络结构文件中包含的参数信息，也可以是对应该待验证对象的配置文件中的参数信息。定点化参数文件例如可以是针对AI网络的特性，将浮点参数进行定点化处理的参数。例如，将神经网络模型中的卷积操作作为待验证对象，用例生成器101可以解析该卷积操作对应的定点化参数文件和网络结构文件中的参数信息，得到用于生成该卷积操作的验证用例的参数。或者，将完整的神经网络模型作为待验证对象，用例生成器101可以解析该神经网络模型对应的定点化参数文件和网络结构文件，得到用于生成该神经网络模型的验证用例的参数。
用例生成器101得到用于生成待验证对象的验证用例的参数后，根据该参数并基于预设用例模板中定义的格式，生成待验证对象的验证用例。其中，预设用例模板定义了所要生成的验证用例中数据的格式。例如，预设用例模板可以定义所要生成的验证用例某一特定部分的数据的格式。具体的，预设用例模板可以定义所要生成的验证用例中指 令流数据和初始化数据的格式，也就是对特定部分(指令流数据和初始化数据)的格式进行定义。对于所要生成的验证用例中特定部分之外的数据格式，预设用例模板可以根据具体情况来定义。或者，预设用例模板可以对所要生成的验证用例的全部数据进行格式定义，本公开实施例对此不作限制。
在一些可选实施例中，所述预设用例模板包括下列中的至少一项：指令流模板文件，包含所述验证用例的指令流模板；初始化数据模板文件，包含所述验证用例的初始化数据模板；寄存器配置模板文件，包含所述验证用例的寄存器配置模板。
上述实施例中，用例生成器101可以根据预设用例模板中的各个模板文件，生成验证用例中不同的数据。用例生成器101可以根据指令流模板文件中的指令流模板，生成该指令流模板定义格式的指令流；根据初始化数据模板文件中的初始化数据模板，生成该初始化数据模板定义格式的初始化数据；根据寄存器配置模板文件中的寄存器配置模板，生成该寄存器配置模板定义格式的寄存器配置数据。用例生成器101根据预设用例模板定义的格式，生成待验证对象的验证用例。
示例性的，图2示出一种预设用例模板。用例生成器101可以根据“指令流”模板文件生成验证用例中的指令流；根据“图片/权重数据”模板文件和初始化列表(Initial list)模板文件，生成验证用例中的初始化数据：“图片/权重数据”和“Initial list”数据；根据寄存器配置(Register config)模板文件，生成验证用例中的寄存器配置数据。
用例生成器101生成验证用例后，待测芯片部件102可以获取验证用例并由待测设计(Design Under Test，DUT)执行该验证用例，得到测试结果。
在一些例子中，可以使用多个测试用例对待测芯片部件进行测试。多个测试用例中，可以有多个不同的指令流和/或不同的图片/权重数据。
在一些可选实施例中，如图3，所述系统还包括：环境驱动器301，用于基于所述预设用例模板，将所述验证用例驱动到所述待测芯片部件102中，以由所述待测设计执行所述验证用例。
在上述实施例中，可以由环境驱动器301解析验证用例中的具体数据，对解析后的数据进行激励驱动。比如，环境驱动器301可以将验证用例中的指令流、初始化数据和寄存器配置数据，将这些数据发送至和/或根据这些数据配置待测芯片部件102，从而待测设计可以执行该验证用例。
以图2所示的预设用例模板生成的验证用例为例进行说明。环境驱动器301可以解 析验证用例中的初始化数据、指令流和寄存器配置数据，通过UVM(Universal Verification Methodology,通用验证方法学)中的Driver组件对待测芯片部件102的四个部分进行激励驱动。
第一部分：利用验证用例中的寄存器配置数据对待测设计的寄存器进行配置。其中，寄存器配置操作需要严格遵循AHB(Advanced High performance Bus，高级高性能总线)和APB(Advanced Peripheral Bus，高级外围总线)等接口协议。
第二部分：对待测设计所连接的外部存储模型(如，vip model)进行初始化。其中，外部存储模型的初始化需要利用vip backdoor技术对外部存储模块使用的memory区域操作。
第三部分：对待测设计内部存储模型(如，memory model)进行初始化。其中，DUT内部存储模型的初始化需要封装DUT memory函数，对内部共享存储空间进行精确的地址写入操作。
第四部分：将验证用例中的指令流进行激励驱动。利用UVM Driver的时序特性，将验证用例指令流文件中的指令流按照规定的格式发送到DUT的指令接口上。
环境驱动器301将验证用例的数据驱动到待测芯片部件102中，由待测设计执行所述验证用例。该待测芯片部件102中包括的待测设计，可以是待验证的新设计的完整芯片，也可以是待验证的组成完整芯片的新设计的某一部分电路和/或代码，如完成某一特定功能的电路和/或代码，本公开不作限制。
待测芯片部件102中的待测设计执行验证用例后，得到验证用例对应的测试结果。比对部件103可以从待测芯片部件102中获取待测设计执行验证用例后的测试结果。
在一些可选实施例中，所述预设用例模板包括：结果模板文件，包含所述验证用例的输出结果模板。如图3所示，所述系统还包括：环境采样器302，用于基于所述结果模板文件，从所述待测芯片部件102采样所述测试结果，并将采样得到的所述测试结果传输至所述比对部件103。
参照图5示例的预设用例模板，其中转存文件列表(Dump list)模板文件可以作为结果模板文件，包含验证用例的输出结果模板。环境采样器302可以根据“Dump list”模板文件中的输出结果模板，从待测芯片部件102采样待测设计执行验证用例后得到的测试结果。
比如，环境采样器302通过UVM中的Monitor组件进行扩展，自定义结果数据 dump_data的采样任务，按照DUT输入/输出接口中的时序要求，将外部存储模型中的结果数据，采样到临时数组中。然后，根据结果模板文件中输出结果模板的格式，比如根据“Dump list”模板文件的格式，将临时数组存储到文件中。其中采样的结果数据包括AI网络的网络层级运算结果，采样条件是网络层结束的信号，如msg信号。
其中，图3所示的AI芯片验证系统中环境驱动器301和环境采样器302是独立于待测芯片部件102的，在其他可实现方式中，待测芯片部件102中也可以集成环境驱动器301和环境采样器302，本公开对此不进行具体限制。
比对部件103中用来与测试结果进行比对的参考结果，可以是比对部件103直接从验证用例中获取的结果数据，或者可以是验证用例通过参考模型部件执行验证用例后得到的结果数据。其中，参考模型部件是对待验证AI芯片或待测设计的各功能进行验证的模型，负责模拟待测设计DUT的逻辑行为。例如，可以用System C编码模型来实现参考模型部件，该参考模型部件可以负责生成用于比对的金(Golden)结果，在生成Golden结果时要保证和环境采样器采集数据的一致性。在一些可实现的方式中，参考模型部件可以是单独的一个组件。
在一些可实现的方式中，比对部件103负责将环境采样器302的测试结果和该验证用例对应的参考结果，进行正确性比对，且支持多种形式的结果数据比对。比如，支持DUT内部存储结果数据的比对；支持外部存储模型中的网络结果进行比对。可以在网络的每个网络层结束时进行数据比对，比对每一层的数据结果；可以在验证用例执行结束时比对所有网络层的数据结果。从而，研发人员可以根据该验证结果对当前验证的待测设计进行针对性的修改。
本公开实施例中，根据预先定义格式的预设用例模板生成验证用例，使得不同类别的神经网络对应的待验证对象生成的验证用例具有统一的格式，所以本公开实施例中提供的AI芯片验证系统能够适配验证多种类别的神经网络。
在一些可选实施例中，如图5所示，本公开又实施例中提供了一种AI芯片验证系统，该系统中的用例生成器101包括定向生成器401。定向生成器401可以获取待验证对象的参数文件，解析待验证对象的参数文件中包含的待验证对象的对象参数，生成待验证对象的验证用例。其中，参数文件中的对象参数，是生成待验证对象的验证用例时定向生成器401所需要的参数。比如，参数文件可以是定点化参数文件或网络结构文件中的一个或多个，或者可以是其他包含对象参数的参数文件，本公开实施例不作限制。
以参数文件包括定点化参数文件和网络结构文件两个文件为例，定向生成器401可以获取待验证对象的定点化参数文件和网络结构文件，解析两个文件中包含的对象参数，基于对象参数生成待验证对象的验证用例。
具体的，以C++作为编程语言为例进行说明。定向生成器401可以解析神经网络模型的定点化参数文件和网络结构文件，根据解析得到的对象参数调用Printf函数生成指令流。定向生成器401基于生成的指令流，按照预设用例模板中的指令流模板定义的格式，生成验证用例的指令流；进一步的，根据预设用例模板中的其他模板文件，生成验证用例中的其他数据，从而得到完整的验证用例。可以通过环境驱动器301(图5未示出)，将验证用例的数据驱动到待测芯片部件102，由待测设计DUT执行验证用例。通过环境采样器302(图5未示出)采样得到验证用例对应的测试结果。相关具体过程与前述实施例中相关描述类似，在此不再赘述。
本公开实施例中，定向生成器401生成的验证用例中可以包含该验证用例的参考结果。对应的，预设用例模板中包括：参考结果文件，包含验证用例的参考结果。以图4所示的预设用例模板为例，其中“层级数据”模板文件可以作为参考结果文件，包含定向生成器401生成的该验证用例的参考结果。这样，比对部件103可以直接从验证用例中获取参考结果，并将参考结果与从待测芯片部件102获得的测试结果进行比对。
在另一种可实现的方式中，定向生成器401在生成验证用例时，不论生成或不生成该验证用例的参考结果，都可以通过参考模型部件执行该验证用例得到参考结果，由参考模型部件将参考结果传输至比对部件103。其中，参考模型部件是对待验证芯片或待测设计的各功能进行验证的模型，负责模拟待测设计DUT的逻辑行为。
本实施例的AI芯片验证方法，比对部件可以直接从验证用例中获取该验证用例的参考结果，不需要参考模型部件执行验证用例，提高了比对装置的执行效率，从而提高了对待测设计的验证效率。
在一些可选实施例中，定向生成器401可以生成神经网络模型中某一层或某几层作为待验证对象的验证用例。比如，可以生成卷积神经网络模型中某一卷积层作为待验证对象的验证用例。神经网络模型中某一层或某几层作为待验证对象，可以称为层级验证对象。
以图5所示的AI芯片验证系统为例，在支持层级验证时，定向生成器401可以解析神经网络模型中目标网络层对应的参数文件，根据预设用例模板定义的格式，生成目 标网络层作为层级验证对象的验证用例。其中，目标网络层可以是神经网络模型中的待验证的某一个或多个网络层，比如某一个卷积层、池化层，或者也可以是多个卷积层、多个全连接层等。
具体的，比如要对卷积神经网络模型中的某一卷积层进行验证，以测试待测设计是否能够成功执行该卷积层的运算。本公开实施例中，定向生成器401通过解析该神经网络模型中该卷积层对应的定点化参数文件和网络结构文件，基于预设用例模板定义的格式，生成该卷积层作为层级验证对象的验证用例。
本公开实施例中，定向生成器401还可以生成某一完整神经网络模型作为待验证对象的验证用例。比如，可以生成卷积神经网络模型对应的验证用例。这种将整个神经网络模型作为待验证对象，可以称为网络级验证对象。
具体的，比如要对用于图像识别的卷积神经网络模型进行验证，以测试待测设计是否能够完整执行该卷积神经网络模型中的运算。本公开实施例中，定向生成器401通过解析该卷积神经网络模型对应的定点化参数文件和网络结构文件，基于预设用例模板定义的格式，生成该卷积神经网络模型作为网络级验证对象的验证用例。
本公开实施例提供的AI芯片验证方法，能够对神经网络模型的层级进行验证，还能够对整个神经网络模型进行验证，提高了待测设计对神经网络的验证深度和灵活性。从而，更精准定位芯片研发中产生的缺陷(Bug)并修正，加快芯片验证收敛。
在一些可选实施例中，如图6所示，本公开再一实施例中提供了一种AI芯片验证系统，该系统中用例生成器101中包括随机生成器501，验证用例与比对装置103之间还包括参考模型部件502。其中，参考模型部件502是对待验证AI芯片或待测设计的各功能进行验证的模型，负责模拟待测设计DUT的逻辑行为。例如，用System C编码模型来实现，负责生成用于比对的Golden结果，在生成Golden结果时要保证和环境采样器采集数据的一致性。
本实施例中，随机生成器501可以获取待验证对象的配置文件，基于配置文件中的参数利用随机化技术，生成待验证对象的对象参数，利用随机生成的对象参数生成验证用例。其中，配置文件是验证人员针对待验证的待测设计配置的文件，比如验证人员可以利用表格工具，如Excel，生成参数配置的表格文件。如图7示例了一种表格配置文件。
随机生成器501可以解析配置文件，利用配置文件配置的参数，随机生成待验证对 象的对象参数和/或改对象参数的数值。例如，利用SystemVerilog语言的random constraint技术，随机化产生源操作数范围、源操作数地址、目标操作数范围、目标操作数地址和算子的计算参数等。以图7示例的配置文件为例，随机生成器501可以解析表格配置文件中case-1中的n个参数，利用random constraint技术随机生成这n个参数中的一部分参数和/或这部分参数的数值。其中，“一部分参数”可以是n个参数中的一个，也可以是n个参数中的几个，对此本公开实施例不作限制。经过随机化以后，得到多组不同于case-1中的参数和/或参数值，作为待验证对象的对象参数，随机生成器501可以基于这些随机产生的对象参数和/或参数值生成验证用例。
图7示例了验证人员针对待验证的神经网络模型和待测设计配置的表格配置文件，该配置文件中包括生成多个特定验证用例的参数和/或参数值。比如序号1所在行，包括case-1的n个参数：param-1-1····param-1-n，相应的case-2、case-3······case-n等均由n个参数组成。每一个case中的n个配置参数及其参数值，对应该case的一个特定验证用例。随机生成器501可以利用随机化技术，在特定验证用例的基础上，生成覆盖范围更广的验证用例。
随机生成器501在生成验证用例的过程中，由于使用随机技术生成待验证对象的对象参数和/或参数的数值，所以生成的验证用例的参考结果未知。所以，本公开实施例中，如图6所示的AI芯片验证系统包括参考模型部件502。其中，参考模型部件502用于执行随机生成器501生成的验证用例，以获得该验证用例的参考结果。参考模型部件502是对待验证芯片或待测设计的各功能进行验证的模型，负责模拟待测设计DUT的逻辑行为。例如，用System C编码模型来实现，负责生成用于比对的Golden结果，在生成Golden结果时要保证和环境采样器采集数据的一致性。对于不同的待测芯片部件102，可以有不同的参考模型部件502。
上述实施例中，随机生成器501根据配置文件，基于特定验证用例，利用随机技术生成不同的验证用例，使得验证场景覆盖更全面，有效提升了验证完备性。
图6所示的AI芯片验证系统中，用例生成器101中只包括随机生成器501。在其他可实现的方式中，用例生成器101中可以同时包括定向生成器401和随机生成器501。图5和图6所示的AI芯片验证系统，为便于实施例描述清晰，将两者进行分别示出，并不进行任何形式限制。
对于定向生成器401生成的测试用例，可以使用参考模型部件502执行该测试用例，以获得该验证用例的参考结果；也可以不使用参考模型部件502，而直接使用定向生成 器401生成的该验证用例的参考结果。
在AI芯片验证过程中，可以将待验证的待测设计分为1级DUT和2级DUT。参照图8，2级DUT在1级DUT的基础上增加了调度核。其中，1级DUT可以执行验证用例中的指令流。2级DUT中的调度核可以执行调度核指令流，以对验证用例中的指令流进行搬运、调度、控制和分发，提高待测设计的灵活性。
以图4的预设用例模板生成的验证用例为例。1级DUT可以直接执行验证用例中根据指令流模板文件生成的指令流；2级DUT中的调度核可以执行验证用例中根据调度核指令流模板文件生成的调度核指令流，对该验证用例中的指令流进行搬运、调度、控制和分发。在以上实施例中提供的用例生成器(包括随机生成器和定向生成器)，由于不能生成“调度核指令流”，所以只能生成对应1级DUT的验证用例。
如图9所示，本公开再一实施例中提供了一种AI芯片验证系统。对应2级DUT，该AI芯片验证系统还包括调度指令编译器601。调度指令编译器601可以对指令控制流进行编译，得到2级DUT中调度核可执行的调度核指令流。其中，指令控制流是以某种编程语言描述的用于控制验证用例中指令流的指令。比如，调度指令编译器601，对以C语言代码描述的指令控制流进行编译，输出2级DUT中调度核可执行的二进制汇编文件，即得到调度核指令流。
对应图9所示的AI芯片验证系统，预设用例模板可以包括：调度核指令流模板文件，其中包含验证用例的调度核指令流模板。调度指令编译器601对指令控制流进行编译，根据调度核指令流模板定义的格式，生成验证用例中的调度核指令流。以图4所示的预设用例模板为例。用例生成器101还可以根据“调度核指令流模板文件”之外的其他模板文件，生成2级验证用例中“调度核指令流”之外的其他数据，从而生成对应2级DUT的完整验证用例。
本公开实施例中的AI芯片验证系统，相比于对应1级DUT的AI芯片验证系统，只是增加了调度指令编译器601，复用了针对1级DUT的AI芯片验证系统的其他模块。减少了芯片验证工作量，提高了对待测设计的验证效率，节省验证资源。
在一些可选实施例中，如图9所示，该AI芯片验证系统还包括与待测芯片部件102连接的性能统计器602。可以理解的是，这只是一种示例性的AI芯片验证系统的具体结构，还可以是其他的结构形式，比如，在前述图1、图3、图5或图6所述的验证1级DUT的系统中，也可以增加性能统计器602。本公开实施例中，性能统计器602可 以从待测芯片部件102中获取待测设计执行验证用例的性能参数，基于获得的性能参数进行性能统计。
在一些可实现的方式中，性能统计器602可以统计待测设计执行验证用例的时间。例如，性能统计器602可以以复位信号的触发，例如复位信号从低电平变为高电平，为起点，对神经网络模型的每一层(layer)运算的时间进行统计，或者对神经网络模型整个执行时间进行统计。
例如，待测设计执行的验证用例是层级验证对象的验证用例，则性能统计器602可以对神经网络模型中与验证用例对应的网络层在待测设计中执行的时间进行统计，比如对卷积层在待测设计中的运算时间进行统计。例如，待测设计执行的验证用例是整个神经网络模型对应的验证用例，则性能统计器602可以对待测设计运算整个神经网络模型的时间进行统计，比如对整个卷积神经网络模型在待测设计中的运算进行时间统计。
在一些可实现的方式中，性能统计器602还可以统计出待测设计的实际带宽。具体的，性能统计器602可以统计待测设计中所有接口的读写数据量，结合统计的待测设计对应该读写数据量的运算时间，计算出待测设计的实际带宽，如待测设计的AHB和APB的总线带宽。例如，待测设计执行的是针对层级的级验证用例，则性能统计器602可以计算待测设计执行神经网络模型对应网络层的实际带宽。比如，待测设计执行卷积层的验证用例的实际带宽。例如，待测设计执行的是针对网络级的验证用例，则性能统计器602可以计算待测设计执行整个神经网络模型的实际带宽。比如，待测设计执行整个卷积神经网络的实际带宽。
上述实施例中，AI芯片验证系统中增加了性能统计器，可以对待测设计执行验证用例的时间和带宽等性能参数进行统计。并且，性能统计器能够对待测设计执行神经网络模型的层级的验证用例的运算时间和带宽进行统计，也能够对待测设计执行整个神经网络模型的网络级的验证用例的运算时间和带宽进行统计，丰富了对待测设计验证的性能统计。
在上述实施例中，AI芯片验证系统可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。例如，上面各部件的功能对应的代码可以通过SystemVerilog语言实现，并将代码存储在服务器的存储器中，当验证待测芯片部件时，在服务器的处理器上运行该代码，从而完成对待测芯片部件的验证。又例如，上面各部件可以以电路的形式实现，如被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器 或其他电子元件实现。再例如，当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。
在一些可选实施例中，如图10所示，本公开的实施例示出了一种AI芯片验证方法。本公开实施列提供的AI芯片验证方法或AI芯片验证系统，可应用在进行AI芯片验证的服务器、终端设备或其他类型的电子设备中。如图10所示的AI芯片验证流程包括以下步骤。
步骤101，基于待验证对象的信息和预设用例模板，生成所述待验证对象的验证用例，其中，所述验证用例满足所述预设用例模板定义的格式，所述待验证对象包括神经网络模型中的目标算子或者神经网络模型。
本步骤中，可以解析待验证对象的信息，得到用于生成该验证对象的验证用例的参数。其中，待验证对象可以是神经网络模型中的算子，或者可以是整个神经网络模型。待验证对象的信息可以是定点化参数文件和网络结构文件中包含的参数信息，或者可以是对应该验证对象的配置文件中的参数信息。
在得到用于生成验证对象的验证用例的参数后，根据生成的参数并基于预设用例模板中定义的格式，生成待验证对象的验证用例。其中，预设用例模板定义了所要生成的验证用例中数据的格式，具体描述可以参见验证系统部分实施例的相关描述。
在一些可实现的方式中，可以获取待验证对象的参数文件，解析待验证对象的参数文件中包含的对象参数，生成待验证对象的验证用例。其中，相关描述可以参见验证系统实施例相关部分描述。这种实现方式，可以在验证用例中生成对应的参考结果，不需要参考模型部件执行验证用例得到，提高了对待测设计验证的效率。并且，能够对神经网络模型的层级进行验证，对整个神经网络模型进行验证，提高待测设计对神经网络的验证深度和灵活性。
在其他可实现的方式中，可以获取待验证对象的配置文件，基于配置文件中的参数利用随机化技术，随机生成待验证对象的对象参数，利用随机生成的对象参数生成验证用例。相关具体描述同样可以参见本公开验证系统实施例的相关部分的描述。该实现方式，基于特定验证用例，利用随机技术生成不同的验证用例，使得验证场景覆盖更全面，有效提升了验证完备性。
步骤102，通过待测设计执行所述验证用例，得到测试结果。
在生成待验证对象的验证用例后，可以通过待测设计来执行该验证用例，得到 执行后的测试结果。其中，待测设计可以包括在待测芯片部件中，并且基于预设用例模板，通过环境驱动器将验证用例的数据驱动到待测芯片部件中由待测设计执行。待测设计执行验证用例后，可以由环境采样器从待测芯片部件中采样该验证用例执行后的测试结果。相关描述与本公开中验证系统实施例的相关描述一致，在此不再赘述。
步骤103，将所述测试结果与所述验证用例对应的参考结果进行比对，获得所述待测设计对所述待验证对象的验证结果。
本步骤中与验证用例的测试结果进行比对的参考结果，可以是验证用例中已经存在的结果数据，或者可以是通过参考模型部件执行验证用例后得到的结果数据。其中，参考模型部件是对待验证AI芯片或待测设计的各功能进行验证的模型，负责模拟待测设计DUT的逻辑行为。
本公开实施例中，利用预设用例模板预先定义验证用例的格式，不同类别的神经网络对应的待验证对象生成的验证用例具有统一的格式，所以该AI芯片验证方法可以适配验证多种类别的神经网络。
在一些可选实施例中，可以对指令控制流进行编译，得到2级DUT中调度核可执行的调度核指令流。对应的预设用例模板包括：调度核指令流模板文件，其中包含验证用例的调度核指令流模板。
具体的，可以对指令控制流进行编译，根据调度核指令流模板定义的格式，生成验证用例中的调度核指令流。然后根据“调度核指令流模板文件”之外的其他模板文件，生成2级验证用例中“调度核指令流”之外的其他数据，从而生成对应2级DUT的完整验证用例。待测设计根据包括2级DUT的验证用例中的调度核指令流，对该验证用例中的指令流进行处理。这种生成2级DUT验证用例的方法，可以复用生成1级DUT验证用例的模块，减少芯片验证的工作量，提高验证效率。
在一些可选实施例中，可以获取待测设计执行验证用例的性能参数，基于获得的性能参数进行性能统计。比如，统计待测设计执行验证用例的时间，可以统计出待测设计的实际带宽。具体的统计过程可以参见验证系统实施例相关部分的描述，在此不再描述。本实施例的性能统计，可以对对待测设计执行神经网络模型的层级的验证用例的运算时间和带宽进行统计，也能够对待测设计执行整个神经网络模型的网络级的验证用例的运算时间和带宽进行统计，丰富了对待测设计验证的性能统计。
本公开还提供了一种计算机设备，包括存储器、处理器及存储在存储器上并可 在处理器上运行的计算机程序，所述处理器执行所述程序时能够实现本公开任一实施例的AI芯片验证方法。
本公开还提供了一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时，促使所述处理器实现本公开任一实施例的AI芯片验证方法。
其中，所述非临时性计算机可读存储介质可以是ROM、CD-ROM、磁带、软盘和光数据存储设备等，本公开并不对此进行限制。
本公开还提供了一种计算机程序产品，包括计算机程序，所述程序被处理器执行时实现本公开任一实施例的AI芯片验证方法。
本领域技术人员在考虑说明书及实践这里申请的发明后，将容易想到本公开的其它实施方案。本公开旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未申请的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本公开的真正范围和精神由下面的权利要求指出。
应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限制。
以上所述仅为本公开的较佳实施例而已，并不用于限制本公开，凡在本公开的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本公开保护的范围之内。

Claims (21)

  1. 一种人工智能AI芯片验证系统，其特征在于，所述系统包括：
    用例生成器，用于基于待验证对象的信息和预设用例模板，生成所述待验证对象的验证用例，其中，所述验证用例满足所述预设用例模板定义的格式，所述待验证对象包括神经网络模型中的目标算子或者神经网络模型；
    待测芯片部件，连接所述用例生成器，用于通过待测设计执行所述验证用例，得到测试结果；
    比对部件，连接所述待测芯片部件，用于将所述测试结果与所述验证用例对应的参考结果进行比对，获得所述待测设计对所述待验证对象的验证结果。
  2. 根据权利要求1所述的系统，其特征在于，所述预设用例模板包括下列中的至少一项：
    指令流模板文件，包含所述验证用例的指令流模板；
    初始化数据模板文件，包含所述验证用例的初始化数据模板；
    寄存器配置模板文件，包含所述验证用例的寄存器配置模板。
  3. 根据权利要求1或2中所述的系统，其特征在于，所述系统还包括：
    环境驱动器，连接所述用例生成器和所述待测芯片部件，用于基于所述预设用例模板，将所述验证用例驱动到所述待测芯片部件中，以由所述待测设计执行所述验证用例。
  4. 根据权利要求1至3中任一项所述的系统，其特征在于，
    所述预设用例模板包括：结果模板文件，包含所述验证用例的输出结果模板；
    所述系统还包括：环境采样器，连接所述待测芯片部件和所述比对部件，用于基于所述结果模板文件，从所述待测芯片部件采样所述测试结果，并将采样得到的所述测试结果传输至所述比对部件。
  5. 根据权利要求1至4中任一项所述的系统，其特征在于，所述用例生成器包括：
    定向生成器，用于获取所述待验证对象的参数文件，并通过解析所述待验证对象的所述参数文件中包含的对象参数，生成所述待验证对象的所述验证用例。
  6. 根据权利要求5所述的系统，其特征在于，所述参数文件包括：定点化参数文件和网络结构文件。
  7. 根据权利要求5或6中所述的系统，其特征在于，
    所述预设用例模板包括：参考结果文件，包含所述验证用例对应的参考结果；
    所述比对部件进行比对所利用的所述参考结果是从所述预设用例模板中获取的。
  8. 根据权利要求1至7中任一项所述的系统，其特征在于，所述用例生成器包括：
    随机生成器，用于获取所述待验证对象的配置文件，基于所述配置文件随机生成待验证对象的对象参数，并基于生成的所述对象参数，生成所述待验证对象的所述验证用例。
  9. 根据权利要求1至8中任一项所述的系统，其特征在于，所述系统还包括：
    参考模型部件，连接所述用例生成器和所述比对部件，用于执行所述用例生成器生成的所述验证用例，以得到所述验证用例对应的参考结果。
  10. 根据权利要求1至9中任一项所述的系统，其特征在于，
    所述预设用例模板包括：调度核指令流模板文件，包含所述验证用例的调度核指令流模板；
    所述系统还包括：调度指令编译器，连接所述用例生成器、所述待测芯片部件和所述比对部件，用于基于所述调度核指令流模板文件，生成所述待测设计的调度核指令流，其中，所述待测设计的调度核指令流包括在所述验证用例中；
    所述待测设计根据所述验证用例中的所述调度核指令流，对所述验证用例中的指令流进行处理。
  11. 根据权利要求1至10中任一项所述的系统，其特征在于，所述系统还包括：
    性能统计器，与所述待测芯片部件连接，用于获取所述待测设计执行所述验证用例的性能参数，并基于所述性能参数得到性能统计结果。
  12. 一种AI芯片验证方法，其特征在于，所述方法包括：
    基于待验证对象的信息和预设用例模板，生成所述待验证对象的验证用例，其中，所述验证用例满足所述预设用例模板定义的格式，所述待验证对象包括神经网络模型中的目标算子或者神经网络模型；
    通过待测设计执行所述验证用例，得到测试结果；
    将所述测试结果与所述验证用例对应的参考结果进行比对，获得所述待测设计对所述待验证对象的验证结果。
  13. 根据权利要求12所述的方法，其特征在于，所述预设用例模板包括下列中的至少一项：
    指令流模板文件，包含所述验证用例的指令流模板；
    初始化数据模板文件，包含所述验证用例的初始化数据模板；
    寄存器配置模板文件，包含所述验证用例的寄存器配置模板。
  14. 根据权利要求12至13中任一项所述的方法，其特征在于，在所述基于待验证对象的信息和预设用例模板，生成所述待验证对象的验证用例之后，还包括：
    基于所述预设用例模板中包括的至少一个模板文件，将所述验证用例驱动到所述待测芯片部件中，以由所述待测设计执行所述验证用例。
  15. 根据权利要求12至14中任一项所述的方法，其特征在于，所述预设用例模板包括：结果模板文件，包含所述验证用例的输出结果模板；
    在所述通过待测设计执行所述验证用例，得到测试结果之后，还包括：
    基于所述结果模板文件，从所述待测芯片部件采样所述测试结果，并将采样得到的所述测试结果传输至所述比对部件。
  16. 根据权利要求12至15中任一项所述的方法，其特征在于，所述基于待验证对象的信息和预设用例模板，生成所述待验证对象的验证用例，包括：
    获取所述待验证对象的参数文件，并通过解析所述待验证对象的所述参数文件中包含的对象参数，生成所述待验证对象的所述验证用例。
  17. 根据权利要求16所述的方法，其特征在于，所述预设用例模板包括：参考结果文件，包含所述验证用例的参考结果。
  18. 根据权利要求12至17中任一项所述的方法，其特征在于，所述基于待验证对象的信息和预设用例模板，生成所述待验证对象的验证用例，包括：
    获取所述待验证对象的配置文件，基于所述配置文件随机生成待验证对象的对象参数，并基于生成的所述对象参数，生成所述待验证对象的所述验证用例。
  19. 根据权利要求12至18中任一项所述的方法，其特征在于，所述预设用例模板包括：调度核指令流模板文件，包含所述验证用例的调度核指令流模板；
    所述方法还包括：
    基于所述调度核指令流模板文件，生成所述待测设计的调度核指令流；
    基于所述调度核指令流，生成所述待验证对象的验证用例；
    所述待测设计根据所述验证用例中的所述调度核指令流，对所述验证用例中的指令流进行处理。
  20. 一种计算机设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其特征在于，所述处理器执行所述程序时实现权利要求12-19任一所述的方法。
  21. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述程序被处理器执行时，促使所述处理器实现权利要求12-19任一项所述的方法。
PCT/CN2020/119406 2020-05-29 2020-09-30 人工智能芯片验证 WO2021238006A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2021557138A JP7270764B2 (ja) 2020-05-29 2020-09-30 人工知能チップ検証
KR1020217031299A KR20210149045A (ko) 2020-05-29 2020-09-30 인공 지능 칩 검증

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010476115.3A CN113742202A (zh) 2020-05-29 2020-05-29 Ai芯片验证系统、方法、设备及存储介质
CN202010476115.3 2020-05-29

Publications (1)

Publication Number Publication Date
WO2021238006A1 true WO2021238006A1 (zh) 2021-12-02

Family

ID=78724695

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/119406 WO2021238006A1 (zh) 2020-05-29 2020-09-30 人工智能芯片验证

Country Status (5)

Country Link
JP (1) JP7270764B2 (zh)
KR (1) KR20210149045A (zh)
CN (1) CN113742202A (zh)
TW (1) TWI778473B (zh)
WO (1) WO2021238006A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114996076A (zh) * 2022-07-01 2022-09-02 北京联盛德微电子有限责任公司 一种用于芯片仿真的遍历型用例验证方法、系统及电子设备
CN115047323A (zh) * 2022-08-17 2022-09-13 中诚华隆计算机技术有限公司 用于芯片的智能测试方法及系统
CN115495388A (zh) * 2022-11-17 2022-12-20 瀚博半导体(上海)有限公司 用于ai推理芯片的芯片验证方法、装置、设备及介质
CN115794503A (zh) * 2022-09-30 2023-03-14 湖南智存合壹信息科技有限公司 一种基于国产cpu主板的高性能测试装置及方法
CN115879409A (zh) * 2022-12-02 2023-03-31 南京金阵微电子技术有限公司 验证方法、验证装置、介质及电子设备
CN116431421A (zh) * 2023-06-13 2023-07-14 成都登临科技有限公司 神经网络生成方法及生成器、验证方法及系统、存储介质
CN117034824A (zh) * 2023-08-25 2023-11-10 广州市粤港澳大湾区前沿创新技术研究院 复用测试用例和验证环境的仿真验证系统、方法、终端及介质
CN117056897A (zh) * 2023-10-13 2023-11-14 沐曦集成电路(上海)有限公司 用于芯片验证的配置信息处理方法、电子设备和介质

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116302721A (zh) * 2021-12-20 2023-06-23 瑞昱半导体股份有限公司 芯片设计验证系统与方法及计算机可读取记录介质
CN114912413A (zh) * 2022-04-29 2022-08-16 上海阵量智能科技有限公司 芯片验证方法及平台
CN115792580B (zh) * 2023-01-30 2023-05-12 北京怀美科技有限公司 芯片检测系统及可编辑芯片检测系统
KR102662404B1 (ko) * 2023-07-22 2024-04-29 주식회사 플랜터 인공지능 기반 다목적 지능형 반도체 시스템
CN117743194B (zh) * 2024-02-19 2024-05-07 睿思芯科(深圳)技术有限公司 芯片验证用例的自动化生成方法、系统及相关设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080115114A1 (en) * 2006-11-10 2008-05-15 Sashank Palaparthi Automated software unit testing
CN102109572A (zh) * 2009-12-23 2011-06-29 中兴通讯股份有限公司 一种传输芯片的测试方法及测试控制方法
CN104268078A (zh) * 2014-09-23 2015-01-07 北京控制工程研究所 一种基于参数化ip 测试用例集合的芯片自动化验证方法
CN110489344A (zh) * 2019-08-02 2019-11-22 Oppo广东移动通信有限公司 引擎测试方法及相关产品
CN110941519A (zh) * 2019-12-17 2020-03-31 锐捷网络股份有限公司 一种网络设备的芯片测试方法及装置
CN111178512A (zh) * 2019-12-31 2020-05-19 中国科学院自动化研究所南京人工智能芯片创新研究院 器件运行神经网络的测试方法及装置

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW200745895A (en) * 2006-06-07 2007-12-16 Powerchip Semiconductor Corp Method and system for virtual metrology prediction and prediction model building in semiconductor manufacture
CN100389399C (zh) * 2006-11-20 2008-05-21 北京中星微电子有限公司 一种芯片验证的方法及系统
CN102402628B (zh) * 2010-09-07 2016-03-09 无锡中感微电子股份有限公司 一种生成SoC验证平台的方法及系统
US9448917B2 (en) * 2014-04-09 2016-09-20 Samsung Electronics Co., Ltd. System on chip and verification method thereof
CN104486169B (zh) * 2015-01-07 2018-07-20 北京华力创通科技股份有限公司 可重用自动检测及随机验证系统和方法
US11282695B2 (en) * 2017-09-26 2022-03-22 Samsung Electronics Co., Ltd. Systems and methods for wafer map analysis
US20190339884A1 (en) * 2018-04-16 2019-11-07 Lexmark International, Inc. Supply Chips and Methods for Restricting Read Access Thereof
CN109086199B (zh) * 2018-06-22 2022-06-07 网宿科技股份有限公司 一种自动化生成测试脚本的方法、终端和可存储介质
US20200042419A1 (en) * 2018-07-31 2020-02-06 Alibaba Group Holding Limited System and method for benchmarking ai hardware using synthetic ai model
CN109634843B (zh) * 2018-10-31 2021-09-21 中国科学院软件研究所 一种面向ai芯片平台的分布式自动化软件测试方法及平台
CN111045880A (zh) * 2019-12-17 2020-04-21 湖南长城银河科技有限公司 芯片测试方法、验证系统及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080115114A1 (en) * 2006-11-10 2008-05-15 Sashank Palaparthi Automated software unit testing
CN102109572A (zh) * 2009-12-23 2011-06-29 中兴通讯股份有限公司 一种传输芯片的测试方法及测试控制方法
CN104268078A (zh) * 2014-09-23 2015-01-07 北京控制工程研究所 一种基于参数化ip 测试用例集合的芯片自动化验证方法
CN110489344A (zh) * 2019-08-02 2019-11-22 Oppo广东移动通信有限公司 引擎测试方法及相关产品
CN110941519A (zh) * 2019-12-17 2020-03-31 锐捷网络股份有限公司 一种网络设备的芯片测试方法及装置
CN111178512A (zh) * 2019-12-31 2020-05-19 中国科学院自动化研究所南京人工智能芯片创新研究院 器件运行神经网络的测试方法及装置

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114996076A (zh) * 2022-07-01 2022-09-02 北京联盛德微电子有限责任公司 一种用于芯片仿真的遍历型用例验证方法、系统及电子设备
CN114996076B (zh) * 2022-07-01 2023-02-07 北京联盛德微电子有限责任公司 一种用于芯片仿真的遍历型用例验证方法、系统及电子设备
CN115047323A (zh) * 2022-08-17 2022-09-13 中诚华隆计算机技术有限公司 用于芯片的智能测试方法及系统
CN115047323B (zh) * 2022-08-17 2022-10-25 中诚华隆计算机技术有限公司 用于芯片的智能测试方法及系统
CN115794503A (zh) * 2022-09-30 2023-03-14 湖南智存合壹信息科技有限公司 一种基于国产cpu主板的高性能测试装置及方法
CN115495388A (zh) * 2022-11-17 2022-12-20 瀚博半导体(上海)有限公司 用于ai推理芯片的芯片验证方法、装置、设备及介质
CN115879409A (zh) * 2022-12-02 2023-03-31 南京金阵微电子技术有限公司 验证方法、验证装置、介质及电子设备
CN115879409B (zh) * 2022-12-02 2023-09-19 南京金阵微电子技术有限公司 验证方法、验证装置、介质及电子设备
CN116431421A (zh) * 2023-06-13 2023-07-14 成都登临科技有限公司 神经网络生成方法及生成器、验证方法及系统、存储介质
CN116431421B (zh) * 2023-06-13 2023-08-29 成都登临科技有限公司 神经网络生成方法及生成器、验证方法及系统、存储介质
CN117034824A (zh) * 2023-08-25 2023-11-10 广州市粤港澳大湾区前沿创新技术研究院 复用测试用例和验证环境的仿真验证系统、方法、终端及介质
CN117056897A (zh) * 2023-10-13 2023-11-14 沐曦集成电路(上海)有限公司 用于芯片验证的配置信息处理方法、电子设备和介质
CN117056897B (zh) * 2023-10-13 2023-12-26 沐曦集成电路(上海)有限公司 用于芯片验证的配置信息处理方法、电子设备和介质

Also Published As

Publication number Publication date
CN113742202A (zh) 2021-12-03
JP2022537620A (ja) 2022-08-29
TW202145046A (zh) 2021-12-01
TWI778473B (zh) 2022-09-21
JP7270764B2 (ja) 2023-05-10
KR20210149045A (ko) 2021-12-08

Similar Documents

Publication Publication Date Title
WO2021238006A1 (zh) 人工智能芯片验证
CN111428462B (zh) 通讯协议模板构建的方法及终端设备
US6816814B2 (en) Method and apparatus for decomposing and verifying configurable hardware
US8984349B2 (en) Method and system for automating the process of testing a device
US20080209405A1 (en) Distributed debugging for a visual programming language
US20070033552A1 (en) Method for detecting flaws in a functional verification plan
US10002069B2 (en) Automated testing of application program interface
WO2017159638A1 (ja) 能力付与データ生成装置
US10592703B1 (en) Method and system for processing verification tests for testing a design under test
US20100186003A1 (en) Per Group Verification
US9690681B1 (en) Method and system for automatically generating executable system-level tests
WO2020037475A1 (zh) 一种调试应用程序的方法及设备
CN111782207A (zh) 任务流代码生成方法、装置、设备及存储介质
GB2594269A (en) Emulation software and a method of emulating behaviour of a device containing one or more DSP cores
US11960385B2 (en) Automatic generation of integrated test procedures using system test procedures
CN116450431A (zh) Cpu参考模型的指令功能测试系统及其方法、计算机设备和存储介质
CN115562931A (zh) 处理器调试模块验证方法、装置、电子设备和存储介质
US9612870B2 (en) Inversion of control for executable extensions in a run-time environment
CN114461225A (zh) 编译方法及装置、电子设备和存储介质
US20120167037A1 (en) Software static testing apparatus and method
CN117591104B (zh) 模型的生成方法、装置、电子设备及存储介质
CN117112447B (zh) 一种数据传输方法、装置、电子设备及可读存储介质
CN117331565B (zh) 软件生成方法、装置、计算机设备及存储介质
CN117407229A (zh) 一种同步网络验证方法、装置、计算机设备及存储介质
Trout Testing safety-critical systems using Model-based Systems Engineering (MBSE)

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2021557138

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20937427

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20937427

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20937427

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 190523)

122 Ep: pct application non-entry in european phase

Ref document number: 20937427

Country of ref document: EP

Kind code of ref document: A1