WO2023125821A1

WO2023125821A1 - Chip test case generation method and apparatus, and storage medium

Info

Publication number: WO2023125821A1
Application number: PCT/CN2022/143439
Authority: WO
Inventors: 陈俊洁; 毛航宇; 郝建业; 孙月凤; 姜佳君
Original assignee: 华为技术有限公司
Priority date: 2021-12-31
Filing date: 2022-12-29
Publication date: 2023-07-06
Also published as: CN116432570A

Abstract

The present application relates to a chip test case generation method and apparatus, and a storage medium. The method may comprise: acquiring a set of input parameters, wherein the set of input parameters comprises M input parameters of a chip, M being an integer not less than 1; and inputting the set of input parameters into a first model, so as to generate a set of first test cases, wherein each first test case comprises values of the M input parameters, the first model is obtained by using a first test case training sample to perform training, and the values of the M input parameters in the first test case training sample satisfy a preset constraint. In the present application, a relatively high function coverage rate can be reached by means of generating a small-scale set of first test cases, such that the time required for testing is effectively shortened, thereby improving the testing efficiency, and saving on the testing costs.

Description

A chip test case generation method, device and storage medium

This application claims the priority of the Chinese patent application submitted to the China Patent Office on December 31, 2021, with the application number 202111663515.6, and the title of the invention is "a method, device and storage medium for generating test cases for a chip", all of which have been passed References are incorporated in this application.

technical field

The present application relates to the field of artificial intelligence, in particular to a chip test case generation method, device and storage medium.

Background technique

Chips are widely used in various electronic devices and are considered to be the heart of electronic devices. The generation process of the chip is complicated, and it needs to go through many links such as chip specification definition, chip design, physical layout, and tape-out production. Among them, chip design is particularly critical as the upstream basic link of the chip generation process. Usually, developers design the features, attributes, and functional architecture of the chip in the specification definition link, and then use the Hardware Description Language (Hardware Description Language, HDL) to write the Register-Transfer-Level (RTL) level (RTL) implemented by the chip design in the chip design link. ) code, and the subsequent links will be further developed based on this RTL code. Once the chip design is put into mass production, it will be difficult to change it, and the lack of chip design will bring catastrophic economic losses and even endanger the lives of users. Therefore, it is very important to ensure the quality of chip design. It is very necessary to fully test the chip design before putting the chip design into production.

At present, the constraint-driven random test method (Constraint Driven Random, CDR) is mainly used to test the chip design under test (Design Under Test, DUT). This method uses a constraint solver to correct randomly generated test cases, and uses the corrected test cases for testing. This method often requires a large-scale test case collection and a long period of testing to achieve high functional coverage. rate, the test efficiency is low, and the test cost is high.

Contents of the invention

In view of this, the present application proposes a chip test case generation method, device and storage medium.

In the first aspect, an embodiment of the present application provides a method for generating test cases for a chip, the method comprising: obtaining an input parameter set; the input parameter set includes M input parameters of the chip, and M is an integer not less than 1 ; Input the input parameter set into the first model to generate the first test case set; wherein, the first test case includes: the values of the M input parameters, and the first model utilizes the first test case training samples As obtained from the training, the values of the M input parameters in the training samples of the first test case satisfy preset constraints.

Based on the above technical solution, the value of the input parameters in the first test case training sample satisfies the preset constraints, and the first model is a model obtained by using the first test case training sample training, so that the first model obtained after training can be After learning the preset constraint, the effective value space can be selected from all the value spaces of the input parameters, thereby effectively reducing the value space of the input parameters to be explored in the process of generating the first test case; at the same time , the first model can automatically generate test cases whose values of input parameters meet the preset constraints without correction, which ensures the diversity of values of input parameters in the first test case set, so that by generating a smaller-scale first test A collection of use cases can achieve high functional coverage, effectively reducing the time required for testing, effectively improving testing efficiency, and saving testing costs.

According to the first aspect, in the first possible implementation manner of the first aspect, the method further includes: determining a target function coverage group, the target function coverage group including the target function to be tested; according to the function coverage group Associating information with input parameters, determine N target input parameters associated with the target function coverage group in the input parameter set, N is a positive integer not greater than M; wherein, the function coverage group and the input The correlation information between parameters is obtained by using the second test case training sample and the first function coverage information corresponding to the second test case training sample; the M input parameters in the second test case training sample The value satisfies the preset constraints, the first function coverage information indicates the tested functions and/or untested functions in each function coverage group; determine at least one first value of the N target input parameters value combination; using each of the at least one first value combination to replace a second value combination of the N target input parameters in the first test case set to obtain a second test case set, Wherein, the second test case includes values of the M input parameters.

Based on the above technical solution, the first function coverage information indicates the tested functions and/or untested functions in each function coverage group, and the association information between the function coverage group and the input parameters is that the values of the input parameters satisfy the predetermined Assuming that the training sample of the second test case constrained and the first functional coverage information corresponding to the second test case training sample are obtained through training, the association information between the functional coverage group and the input parameter can indicate the associated functional coverage group and the input parameter ; and furthermore, the target input parameter associated with the target function coverage group in the input parameter set can be determined according to the association information, and the target input parameter is an important parameter that affects whether the target function to be tested in the target function coverage group can be tested, Thereby further narrowing the value space of the exploration input parameters and improving the test efficiency; determining the value combination of the target input parameters and generating a second test case set containing the value combination, and then covering the target function coverage group to be tested The target function is tested more efficiently and the functional coverage is effectively improved. In some examples, for an uncovered functional coverage group, test cases that can cover the functional coverage group can be generated, thereby effectively improving the functional coverage rate, especially for the functional coverage group that is usually difficult to cover in the later stage of the test. Better coverage effect , reducing the manual intervention of domain experts, reducing the test cost, further improving the test efficiency, and effectively shortening the chip development cycle.

According to the first possible implementation of the first aspect above, in the second possible implementation of the first aspect, the determining at least one first value combination of the N target input parameters includes : Randomly generate a plurality of candidate value combinations according to the preset value ranges of the N target input parameters; use any value combination in the plurality of candidate value combinations as the value combination corresponding to the initial iteration round ; From the plurality of candidate value combinations, determine the value combination corresponding to the current iteration round; wherein, the value combination corresponding to the current iteration round has the largest difference from the determined value combination corresponding to each iteration round; The above operation of determining the value combination corresponding to the current iteration round is repeatedly performed until the preset iteration condition is met; and the value combination corresponding to each iteration round is used as the at least one first value combination.

Based on the above technical solution, considering that for any target input parameter, its corresponding value range may include more values, if all the values are traversed, the number of first value combinations generated is too large, therefore, Sampling is performed through a heuristic strategy. First, multiple candidate value combinations are generated, and then through multiple iterations, the value combination corresponding to each iteration round is determined, and the value combination corresponding to each iteration round is consistent with the determined iteration rounds. The corresponding value combinations have the largest difference, thus ensuring that the values of the N target input parameters in the value combinations corresponding to each iteration round are more uniform and diverse; thus, test cases with more uniform and diverse input parameter values can be generated. While ensuring the functional coverage, the test efficiency is further improved.

According to the various possible implementations above, in the third possible implementation of the first aspect, the method further includes: filtering out the values of the M input parameters in the second test case set A second test case that does not satisfy the preset constraints.

Based on the above technical solution, the second test cases whose values of the input parameters do not meet the preset constraints are filtered out, so as to ensure that the values of the input parameters of each second test case in the second test case set all meet the preset constraints.

According to various possible implementations of the above first aspect, in a fourth possible implementation of the first aspect, the determining the target function coverage group includes: obtaining the first test case corresponding to the first test case set Two function coverage information, the second function coverage information includes the number of untested functions in each function coverage group; according to the second function coverage information corresponding to the first test case set, it is determined that the first test case set corresponds to The heuristic score of each functional coverage group, the heuristic score is positively correlated with the number of untested functions in the functional coverage group; according to the heuristic score of each functional coverage group, determine the functional coverage group Corresponding probabilities: based on the probabilities corresponding to each functional coverage group, at least one target functional coverage group is determined.

Based on the above technical solution, the target function coverage group is selected through a heuristic strategy. The more untested functions in the function coverage group, the higher the corresponding heuristic score, the greater the corresponding probability, and the more likely to be selected. is the target functional coverage group; that is, for the functional coverage group that is not covered, it is more likely to be selected as the target functional coverage group; especially in the later stage of the test, the functional coverage group that is difficult to be covered is more likely to be selected as the target functional coverage group , and then can generate test cases in a targeted manner, thereby improving functional coverage, reducing manual intervention by domain experts, reducing testing costs, and improving testing efficiency.

According to the fourth possible implementation of the first aspect above, in the fifth possible implementation of the first aspect, according to the second function coverage information corresponding to the first test case set, determine the first A heuristic score for each functional coverage group corresponding to the test case set, including: determining the corresponding function of the first test case set according to the number of untested functions in each functional coverage group corresponding to the first test case set The total number of untested functions; according to the proportion of the number of untested functions in each functional coverage group to the total number of untested functions, determine the heuristic score of each functional coverage group.

According to the above first aspect or various possible implementations of the first aspect, in a sixth possible implementation of the first aspect, the method further includes: acquiring the first test case training samples; using The first test case training sample is used to train the first preset model to obtain the first model.

Based on the above technical solution, the first preset model is trained by using the first test case training sample whose value of the input parameter satisfies the preset constraint. During the training process, the first preset model can continuously learn the relationship between different input parameters. value constraints, and select an effective value space from the large-scale input parameter value space, so that the first model trained can generate the first test case whose input parameter value satisfies the preset constraint, and generates The value space of the input parameters to be explored in the process of the first test case is reduced, avoiding the generation of redundant or invalid test cases, without correction, and ensuring the diversity of the values of the input parameters in the first test case set, Therefore, by generating a smaller-scale test case set, a higher functional coverage rate can be achieved, which effectively reduces the time required for testing, improves testing efficiency, and saves testing costs.

According to the sixth possible implementation of the first aspect above, in the seventh possible implementation of the first aspect, the first preset model includes a generative adversarial network, and the generative adversarial network includes a generator and a discriminator; the training of the first preset model using the first test case training sample to obtain the first model includes: using the first test case training sample to generate an adversarial network Carry out multiple rounds of training; and determine the test case invariance rate corresponding to the generator when each round of training converges, the test case invariance rate represents that the values in the M input parameters of the test case generated by the generator meet the preset The ratio of the input parameters of the constraint; the generator corresponding to the maximum test case invariance rate is used as the first model.

Based on the above technical solution, use the first test case training sample to conduct multiple rounds of confrontation training on the discriminator and generator in the generative confrontation network. Through continuous training, the generator can tap into the potential value constraints between different input parameters, and The effective value space is selected from the value space of the large-scale input parameters, thereby reducing the value space of the input parameters to be explored in the process of generating test cases. The generator corresponding to the maximum test case invariance rate is used as the first model, so that the first model can be used to automatically generate test cases whose input parameter values meet the preset constraints, while achieving high functional coverage, effective The number of test cases that need to be generated is reduced, the test efficiency is improved, and the test cost is saved.

According to the seventh possible implementation of the first aspect above, in the eighth possible implementation of the first aspect, the method further includes: using the convergent discriminator to filter out the second test A second test case in which the M input parameters in the test case set do not satisfy the preset constraints.

According to various possible implementations of the first aspect above, in a ninth possible implementation of the first aspect, the method further includes: acquiring the second test case training sample and the second test case Use the first function coverage information corresponding to the training sample of the test case; use the second test case training sample and the first function coverage information to train the second preset model, and obtain the association between the function coverage group and the input parameters information.

Based on the above technical solution, use the second test case training sample and the first function coverage information corresponding to the second test case training sample to train the second preset model, so as to dig out the relationship between each function coverage group and the input parameters , that is, for any functional coverage group, dig out the important input parameters that affect whether the function in the functional coverage group can be tested, so as to obtain the correlation information between the functional coverage group and the input parameters; the correlation information can be used for further generation of test The use case process provides guidance to generate test cases that can cover the functional coverage points in the target functional coverage group in a targeted manner, thereby effectively improving the functional coverage. The value space effectively improves the test efficiency.

According to the ninth possible implementation manner of the above first aspect, in the tenth possible implementation manner of the first aspect, the second preset model includes a random forest model; the using the second test Using case training samples and the first function coverage information to train the second preset model to obtain the association information between the function coverage group and input parameters, including: for the target function coverage group, the The second test case training sample is input into the random forest model, and the random forest model is trained with the first functional coverage information as a label to obtain the correlation information between the functional coverage group and the input parameters.

Based on the above technical solution, for any functional coverage group, the first functional coverage information corresponding to the second test case training sample can be used as a label, and the random forest model corresponding to the functional coverage group can be trained by using the second test case training sample , so as to dig out the correlation between the functional coverage group and the input parameters of the test cases, that is, to dig out the important input parameters that affect whether the functions in the functional coverage group can be tested, so as to obtain the relationship between the functional coverage group and the input parameters The associated information; the associated information can provide guidance for the process of further generating test cases, so as to generate targeted test cases that can cover the functional coverage points in the target functional coverage group, thereby effectively improving the functional coverage rate and further reducing the test case size. The value space of the input parameters that needs to be explored during the use case generation process effectively improves the test efficiency.

According to various possible implementation manners of the first aspect above, in the eleventh possible implementation manner of the first aspect, the using each of the at least one first value combination replaces the A second value combination of the N target input parameters in the first test case set includes: for each first value combination, determining the first value combination whose similarity with the first value combination satisfies a preset condition A second value combination is used as a target second value combination of the first value combination; and the first value combination is used to replace the target second value combination.

Based on the above technical solution, the value of the input parameter in the second value combination satisfies the preset constraint, and the second value combination whose similarity with the first value combination satisfies the preset constraint is used as the target second value combination, so that It is ensured to the greatest extent that the values of the input parameters in the generated second test case containing the first value combination satisfy the preset constraints.

In a second aspect, an embodiment of the present application provides a device for generating test cases for a chip, the device comprising: a transmission module configured to obtain an input parameter set; the input parameter set includes M input parameters of the chip, and M is An integer not less than 1; a processing module, configured to input the input parameter set into the first model to generate a first test case set; wherein, the first test case includes: the values of the M input parameters, the first A model is obtained by training by using a first test case training sample, and values of the M input parameters in the first test case training sample satisfy preset constraints.

According to the second aspect, in the first possible implementation manner of the second aspect, the processing module is further configured to: determine a target function coverage group, where the target function coverage group includes the target function to be tested; The association information between the function coverage group and the input parameters, determine the N target input parameters associated with the target function coverage group in the input parameter set, N is a positive integer not greater than M; wherein, the function coverage The association information between the group and the input parameters is obtained by using the second test case training sample and the first function coverage information corresponding to the second test case training sample; the M in the second test case training sample The value of the input parameter satisfies the preset constraint, and the first function coverage information indicates the tested function and/or the untested function in each function coverage group; determine at least one of the N target input parameters The first value combination; using each of the at least one first value combination to replace a second value combination of the N target input parameters in the first test case set to obtain a second test A set of use cases, wherein the second test case includes the values of the M input parameters.

According to the first possible implementation of the second aspect above, in the second possible implementation of the second aspect, the processing module is further configured to: according to the preset of the N target input parameters Value range, randomly generating a plurality of candidate value combinations; using any value combination in the plurality of candidate value combinations as the value combination corresponding to the initial iteration round; from the plurality of candidate value combinations, Determine the value combination corresponding to the current iteration round; wherein, the difference between the value combination corresponding to the current iteration round and the determined value combination corresponding to each iteration round is the largest; repeat the above-mentioned determination of the value combination corresponding to the current iteration round operation until the preset iteration condition is satisfied; and the value combination corresponding to each iteration round is used as the at least one first value combination.

According to various possible implementations of the second aspect above, in a third possible implementation of the second aspect, the processing module is further configured to: filter out the A second test case in which the values of the M input parameters do not satisfy the preset constraints.

According to various possible implementations of the second aspect above, in a fourth possible implementation of the second aspect, the processing module is further configured to: acquire the second test case set corresponding to the first test case set Function coverage information, the second function coverage information includes the number of untested functions in each function coverage group; according to the second function coverage information corresponding to the first test case set, determine the first test case set corresponding The heuristic score of each functional coverage group, the heuristic score is positively correlated with the number of untested functions in the functional coverage group; according to the heuristic score of each functional coverage group, it is determined that each functional coverage group corresponds to probability; at least one target function coverage group is determined based on the probability corresponding to each function coverage group.

According to the fourth possible implementation of the second aspect above, in the fifth possible implementation of the second aspect, the processing module is further configured to: The number of untested functions in the function coverage group determines the total number of untested functions corresponding to the first test case set; according to the number of untested functions in each function coverage group accounts for the untested The proportion of the total number of functions tested determines the heuristic score for each functional coverage group.

According to the above second aspect or various possible implementation manners, in a sixth possible implementation manner of the second aspect, the processing module is further configured to: obtain the first test case training samples; use the The first test case training sample is used to train the first preset model to obtain the first model.

Based on the above technical solution, the first preset model is trained by using the first test case training sample whose value of the input parameter satisfies the preset constraint. During the training process, the first preset model can continuously learn the relationship between different input parameters. value constraints, and select an effective value space from the large-scale input parameter value space, so that the first model trained can generate the first test case whose input parameter value satisfies the preset constraint, and generates The value space of the input parameters to be explored in the process of the first test case is reduced, avoiding the generation of redundant or invalid test cases, without correction, and ensuring the diversity of the values of the input parameters in the first test case set, Therefore, by generating a smaller-scale test case set, a higher functional coverage rate can be achieved, which effectively reduces the time required for testing, effectively improves testing efficiency, and saves testing costs.

According to the sixth possible implementation of the second aspect above, in the seventh possible implementation of the second aspect, the first preset model includes a generative adversarial network, and the generative adversarial network includes a generator and a discriminator; the processing module is also used for: utilizing the first test case training sample to carry out multiple rounds of training to the generation confrontation network; and determining the test case invariance rate corresponding to the generator when each round of training converges , the test case invariance rate represents the ratio of the input parameters whose value meets the preset constraints among the M input parameters of the test cases generated by the generator; the generator corresponding to the maximum test case invariance rate is used as the Describe the first model.

According to the seventh possible implementation of the second aspect above, in the eighth possible implementation of the second aspect, the processing module is further configured to: filter out the A second test case in which the M input parameters do not satisfy the preset constraint in the second test case set.

According to various possible implementations of the second aspect above, in a ninth possible implementation of the second aspect, the processing module is further configured to: acquire the second test case training sample and the The first function coverage information corresponding to the second test case training sample; using the second test case training sample and the first function coverage information to train the second preset model to obtain the function coverage group and the input parameters related information between them.

According to the ninth possible implementation manner of the second aspect above, in the tenth possible implementation manner of the second aspect, the second preset model includes a random forest model; the processing module is further configured to : For the target functional coverage group, input the second test case training sample into the random forest model, and use the first functional coverage information as a label to train the random forest model to obtain the functional coverage Association information between groups and input parameters.

According to various possible implementation manners of the second aspect above, in the eleventh possible implementation manner of the second aspect, the processing module is further configured to: for each first value combination, determine and The second value combination whose similarity of the first value combination satisfies the preset condition is used as the target second value combination of the first value combination; use the first value combination to replace the target second value combination.

In a third aspect, an embodiment of the present application provides a device for generating test cases for a chip, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions A method for generating test cases for a chip in the above first aspect or various possible implementation manners of the first aspect is implemented.

On the fourth aspect, the embodiments of the present application provide a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above first aspect or various possibilities of the first aspect can be realized The test case generation method of the chip in the implementation manner.

In the fifth aspect, the embodiments of the present application provide a computer program product, which, when the computer program product runs on a computer, causes the computer to execute the above-mentioned first aspect or various possible implementation manners of the first aspect. A test case generation method for chips.

For the technical effects of the above-mentioned second aspect to the fifth aspect and various possible implementation manners, refer to the above-mentioned first aspect and the technical effects of various possible implementation manners of the first aspect.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the application and, together with the specification, serve to explain the principles of the application.

FIG. 1 shows a schematic diagram of chip testing in different steps.

FIG. 2 shows a schematic diagram of a scene for testing a chip design.

FIG. 3 shows a schematic diagram of the relationship among test cases, functional coverage points and functional coverage groups.

Fig. 4 shows a schematic flow diagram of a constraint-driven random testing method.

Fig. 5 shows a flowchart of a method for generating test cases for a chip according to an embodiment of the present application.

Fig. 6 shows a flow chart of another chip test case generation method according to an embodiment of the present application.

Fig. 7 shows a flow chart of another chip test case generation method according to an embodiment of the present application.

Fig. 8 shows a flow chart of another chip test case generation method according to an embodiment of the present application.

FIG. 9 shows a flowchart of another method for generating test cases for a chip according to an embodiment of the present application.

Fig. 10 shows a flowchart of a method for determining a target function coverage group according to an embodiment of the present application.

Fig. 11 shows a flowchart of a method for determining a first value combination according to an embodiment of the present application.

Fig. 12 shows a flow chart of another chip test case generation method according to an embodiment of the present application.

Fig. 13 shows a comparison chart of test results according to an embodiment of the present application.

Fig. 14 shows a structural diagram of an apparatus for generating test cases for a chip according to an embodiment of the present application.

FIG. 15 shows a schematic structural diagram of a device for generating test cases for a chip according to an embodiment of the present application.

Detailed ways

Various exemplary embodiments, features, and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures indicate functionally identical or similar elements. While various aspects of the embodiments are shown in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

In addition, in order to better illustrate the present application, numerous specific details are given in the following specific implementation manners. It will be understood by those skilled in the art that the present application may be practiced without certain of the specific details. In some instances, methods, means, components and circuits well known to those skilled in the art have not been described in detail in order to highlight the gist of the present application.

For ease of understanding, the following first introduces some concepts involved in the embodiments of the present application.

1. Generative adversarial networks (GAN)

The basic structure of generative confrontation network includes generator and discriminator. Inspired by the zero-sum game in game theory, during the training process, the generation problem is regarded as the confrontation and game between the two networks of the discriminator and the generator: the generator is used to exploit random noise (for example, uniform distribution or normal distribution) noise) to generate synthetic data, and the discriminator is used to distinguish the output data of the generator from the real data. The generator tries to produce data that is closer to the real data, and accordingly, the discriminator tries to more perfectly distinguish the real data from the data output by the generator. In this way, the generative confrontation network learns through the mutual game between the generator and the discriminator. The generator and the discriminator progress in the confrontation, and continue to fight after the progress. The performance of the generative confrontation network continues to improve, and the data generated by the generator becomes more and more perfect. , which is close to the real data, and then the generator can be used to obtain the expected data.

2. Random Forests (RF)

Random forest refers to a classification model constructed with a certain number of decision trees (Decision Tree) as the base learner; the random forest is obtained by training each decision tree through samples, and when using random forest for classification, each decision tree is Classification, and the final classification result can be determined by voting method according to the majority principle.

Among them, the decision tree is a tree structure, and the decision is completed by passing down from the root node. The decision tree generation algorithm can include iterative binary tree three generations (Iterative Dichotomiser 3, ID3), C4.5 (an extension of the ID3 algorithm) and classification regression Tree algorithm (Classification And Regression Tree, CART), etc., in which, during the generation of the decision tree, each node is split based on the selected optimal feature (feature), so that the samples contained on the end node are as much as possible of the same type category. The generation process of the decision tree in the random forest has two characteristics: random selection of sample sets and random selection of the characteristics of each node; among them, random selection of sample sets refers to the construction of sub-sample sets from the original sample set by sampling with replacement , the number of samples in the sub-sample set is the same as the original sample set, and a decision tree is trained through each sub-sample set; similar to randomly selected sample sets, each node splitting process of the decision tree does not use all the features, and It is to randomly select a certain number of features from all the features, and then select the optimal feature from the randomly selected features according to the evaluation index as the feature used when the node is split. Among them, the evaluation index can include Information Entropy (Information Entropy) , Gain Ratio, Gini Index, etc.

3. Hardware Description Language (Hardware Description Language, HDL)

A hardware description language is a language for describing digital circuits and systems in a formal way. Based on this language, chip developers can systematically describe chip design ideas. Common hardware description languages may include Very-High-Speed Integrated Circuit Hardware Description Language (VHDL) and Verilog Hardware Description Language.

4. Register-Transfer-Level (RTL)

The register conversion level is a level of chip design, and the chip design code at this level describes the registers of all levels in the chip and the conversion of signals between different registers.

5. Chip test

Chip testing runs through the chip design and production links. Figure 1 shows a schematic diagram of chip testing in different links. As shown in Figure 1, in the chip design link, chip testing can include: chip design testing in the chip design stage; , Chip testing may include: process technology testing at the wafer processing stage, wafer testing at the packaging stage, design testing at the mass production stage, and so on.

Among them, the chip design test in the chip design stage has the characteristics of long time ratio, high personnel investment, and high cost. From the perspective of time cost, in most chip projects, testing work occupies more than 60% of the development cycle. Development teams spend more time on chip design testing than chip design and development. From the perspective of labor costs, the proportion of testers in the chip development team has increased year by year, surpassing the number of chip design and development personnel. The test cost of chip design directly affects the entire development cycle of the chip.

Compared with other stages of testing, chip design testing has unique challenges. First, unlike the structural coverage commonly used in traditional software testing methods, functional coverage plays a more important role in chip design testing. Secondly, due to the many functions of the chip design to be tested, the scale of its input parameters is huge, and there are complex value constraints among different input parameters, so it is difficult for traditional software testing methods to test the large and complex input parameter value space. Explore efficiently.

Fig. 2 shows a schematic diagram of a scenario where a chip design is tested, as shown in Fig. 2, a test case is generated, the test case is input into a chip emulator, and a simulation result is obtained, wherein the simulation result may include functional coverage, etc., if If the simulation result does not meet expectations, a new test case can be generated and input to the above-mentioned chip emulator, and the above operations are repeated until the simulation result meets expectations.

5. Test cases

A test case, also called a test parameter use case or a test input, refers to a set of data provided to the chip design to be tested in order to test the chip design to be tested. Each test case may include multiple preset input parameters and the values of each input parameter, and multiple test cases are correspondingly generated based on different value combinations of the input parameters. Wherein, the preset input parameters are parameters designed during chip design, and the value of each input parameter is within a pre-designed value range. For example, as shown in Table 1, x1, x2...xK are K test cases, and these K test cases combine a test case set; p1, p2...pL are L input parameters contained in the test case, a11, a12...aKL are the values of the input parameters in the test case, for example, a31, a32...a3L in the row of the test case x3 are the values of the input parameters of the test case x3, where a32 represents the value of the input parameter p2 in the test case x3 value. For any two test cases, the value of at least one input parameter is different.

Table 1 - Test cases and corresponding input parameters

6. Functional Coverage (FCOV)

Functional coverage refers to the ratio of the functional coverage bins of the chip design to be tested to be covered by test cases. Achieving high functional coverage is one of the important goals of chip design and testing.

Wherein, each function coverage point represents a function to be tested in the chip design to be tested; the function coverage point being covered by the test case indicates that the function to be tested corresponding to the function coverage point has been tested. In the chip design test, multiple function coverage points can be divided based on the different functions of the chip design to be tested. Functional coverage points corresponding to similar functions can form a functional coverage group (Functional cover group), that is, functional coverage points in a functional coverage group represent multiple similar functions that need to be tested in the chip design to be tested; for any two A functional coverage group contains different functional coverage points.

When testing, after the test case is input into the chip emulator, the corresponding functional coverage information can be obtained after running in the simulation environment, such as the situation of the functional coverage group being covered, the situation of the functional coverage point being covered, and the functional coverage rate etc. Fig. 3 shows a schematic diagram of the relationship between a test case, a functional coverage point and a functional coverage group. As shown in Fig. 3, the functional coverage group includes two functional coverage points. After the test case runs in the simulation environment of the chip design to be tested, A functional coverage point in the functional coverage group is covered, that is, the function corresponding to the functional coverage point is tested; while another functional coverage point is not covered, that is, the function corresponding to the functional coverage point is not tested; and The ratio of the number of functional coverage points of the chip design to be tested covered by the test cases to the number of all functional coverage points of the chip design to be tested can be calculated to obtain the functional coverage.

7. Constraint Driven Random (CDR)-based random testing method

The random test method based on the constraint drive is the main test method used in the chip design test at the present stage. This method randomly selects the value of the corresponding input parameter within the value range of each input parameter, that is, randomly generates the test case of the chip design to be tested, but the value of each input parameter in the randomly generated test case may violate the relationship between different input parameters. The value constraints between different input parameters cannot be directly applied to the test, and then use the constraint solver that can solve the value constraints between the above different input parameters to correct the randomly generated test cases, so as to obtain the corrected test cases, and use the correction The following test cases are used to test the chip design under test. Among them, the value constraints between the above-mentioned different input parameters can be preset in the chip design process; for example, the input parameter a and the input parameter b are defined during chip design, wherein the value of the input parameter a represents the state of the interface A, and a = 1 means that interface A is open, a = 0 means that interface A is closed; the value of input parameter b indicates the state of interface B, b = 1 means that interface B is open, b = 0 means that interface B is closed; then the set input parameter a and The value constraint between the input parameter b can be: the value of the input parameter a and the number parameter b cannot be 1 at the same time, that is, the design interface A and interface B cannot be opened at the same time. For another example, if the input parameter c, input parameter d, and input parameter e are defined during chip design, the value constraints between the set input parameter a, input parameter b, and input parameter c can be: input parameter c, input The sum of the values of parameter d and input parameter e cannot be greater than a certain value.

Fig. 4 shows the schematic flow chart of the random test method based on constraint drive, as shown in Fig. 4, the main process includes: 1) adopting a random method to generate a large amount of test cases for the design of the chip to be tested, thereby obtaining a set of test cases; 2 ) The constraint solver can automatically modify the input parameter values of test cases that do not meet the preset constraints based on the value constraints between different input parameters defined in the chip design process, thereby generating test cases that are compared to random generation A test case set that more satisfies the preset constraints; 3) input the test case set that satisfies the value constraints between different input parameters into the chip emulator for simulation, obtain the number of covered function coverage points, and then compare with the pre-defined Compare the number of functional coverage points that need to be tested, so as to obtain the functional coverage.

For example, the K test cases shown in Table 1 above are input into the constraint solver, and the constraint solver automatically modifies the value of the input parameters that do not meet the value constraints between the input parameters in each test case, so that The corrected test cases are obtained; Table 2 shows the test cases corrected by the constraint solver, and the input parameter values of each test case. As shown in Table 2, x'1, x'2...x'K are the K test cases corrected by the constraint solver, p1, p2...pL are the L input parameters contained in the test cases, a11, a12, a'13...aKL is the value of the input parameter in the test case, where a'13, a'32, a'2L, etc. represent the value of the input parameter modified by the constraint solver, for example, a'13 represents the constraint solver The device modifies the value a13 of the input parameter p3 in the test case x1 in Table 1 above.

Table 2 - Test cases corrected by constraint solver and corresponding input parameter values

Input the test cases corrected by the constraint solver shown in Table 2 into the chip emulator to run, so that the functional coverage information corresponding to each test case can be obtained. Table 3 shows the test cases and the corresponding functional coverage information, as As shown in Table 3, x'1, x'2...x'K are K test cases corrected by the constraint solver, g1, g2...gR are R function coverage groups, b1, b2...bT are T functions Coverage points, c11, c12...cKT are function coverage parameters, indicating whether the corresponding function coverage points are covered by the corresponding test cases, that is, whether the functions that need to be tested represented by the function coverage points are tested; for example, c12, c22, c23 respectively indicates whether the functional coverage points b1, b2 and b3 in the functional coverage group g1 are covered by the test case x'2. Exemplarily, different values of c11, c12... or cKT can be used to distinguish that the corresponding function coverage point has been covered by the corresponding test case and the corresponding function coverage point has not been covered by the corresponding test case; for example, c11, c12... or cKT It can be 0 or 1. When it is 1, it means that the function coverage point corresponding to the column where the function coverage parameter is located is covered by the test case corresponding to the row. When it is 0, it means that the function coverage point corresponding to the column where the function coverage parameter is located is not covered by the row. It is covered by the corresponding test case; if c22=1, it means that the functional coverage point b2 is covered by the test case x'2, and if c22=0, it means that the functional coverage point b2 is not covered by the test case x'2.

Table 3 - Test cases and corresponding functional coverage information

Although the constraint-driven random testing method above can generate test cases satisfying the value constraints between different input parameters, since the design of the chip to be tested usually involves a large number of input parameters, the value space of the input parameters is huge, and there are There are a large number of complex value constraints between different input parameters. In the constraint-driven random testing method, the method of randomly generating test cases does not consider the value constraints between different input parameters, which will randomly generate a large number of redundant and even Invalid test cases, and then the constraint solver modifies the value of the input parameters that do not meet the requirements in the randomly generated test cases, which limits the value space of the input parameters explored to a certain extent, and the generated The diversity of input parameter values in the test case set becomes worse. Therefore, it is often necessary to generate a large-scale test case set and conduct a long-term test to achieve high functional coverage, low test efficiency, and low test cost. higher.

In order to solve the above technical problems, the present application provides a method for generating test cases for a chip. Exemplarily, the method may be executed by an entity having data processing capabilities such as a processor, a server, and a server cluster. In some examples, the above entities can also simulate the simulation environment of the chip design to be tested, so that the test cases can be run in the simulation environment and corresponding simulation results can be obtained.

Fig. 5 shows a flowchart of a method for generating test cases for a chip according to an embodiment of the present application. As shown in Figure 5, the method may include the following steps:

Step 501. Obtain an input parameter set.

Wherein, the input parameter set includes M input parameters of the chip, and M is an integer not less than 1.

Exemplarily, the chip can be designed for the chip to be tested; for example, it can be a system-on-chip (System on Chip, SOC) of a terminal device, a server chip, an artificial intelligence chip, a communication base station chip, a communication terminal chip or other dedicated chips, etc. Various types of chip designs to be tested are not limited.

Exemplarily, the input parameter set may include predefined input parameters of the chip design to be tested in the chip design stage, for example, the input parameter set may include the input parameters p1, p2 . . . pL in Table 1 above.

Exemplarily, the value range corresponding to each input parameter in the input parameter set may also be acquired. It can be understood that the value range corresponding to each input parameter may be preset during chip design.

Step 502: Input the input parameter set into the first model to generate the first test case set.

Wherein, the first test case includes: the values of the above M input parameters, the first model is obtained by using the training samples of the first test case, and the values of the M input parameters in the training samples of the first test case satisfy the preset constraints .

Exemplarily, the preset constraints may include value constraints between any two or more input parameters among the M input parameters. It can be understood that between any two or more input parameters among the M input parameters The value constraints of can be preset during chip design.

Wherein, the first test case training sample can be a pre-generated test case; the first test case training sample includes M input parameters in the above input parameter set, and the values of the M input parameters meet the preset constraints; exemplary Specifically, the first test case training samples may include test cases generated by using a constraint-driven random testing method; for example, they may include test cases x'1, x'2... or x'K in Table 2 above.

In this step, the first model is a model obtained by using the first test case training sample training, so that the first model can learn the value constraints between different input parameters, and input the input parameter set to the first model, the second A model can automatically generate one or more first test cases whose values of input parameters meet preset constraints, that is, a set of first test cases. Exemplarily, the first model may include a generator that generates an adversarial network, for example, may include GAN, Wasserstein GAN (Wasserstein GAN, WGAN), Wasserstein GAN-gradient penalty (Wasserstein GAN-gradient penalty , WGAN-GP), the generator of mutual information generation confrontation network (InfoGAN), and can also include variational autoencoder (Variational Autoencoder, VAE), conditional transformation autoencoder (Conditional Variational Autoencoders, CVAE) and other similar model, which is not limited.

Exemplarily, the M input parameters and the value range corresponding to each input parameter may be input into the first model, so that one or more test cases are automatically generated by using the first model, and each test case includes the M input The value of the parameter, the value of each input parameter is within the corresponding preset value range, and the values of the M input parameters meet the preset constraints.

Further, the generated first test case set can be input into the chip emulator to run, so as to obtain the corresponding power coverage. Exemplarily, the preset number of function coverage points can be obtained, and the number of function coverage points covered by the first test case set running in the simulation environment can be obtained, and then the difference between the number of covered function coverage points and the preset number of function coverage points can be calculated. The ratio of the number of functional coverage points is used to obtain the power coverage rate corresponding to the first test case set. Since the design of the chip to be tested usually involves a large number of input parameters, and the value range of each input parameter is usually large, resulting in a large scale of the value space of the input parameters, in addition, there are a large number of complex differences between different input parameters. Value constraints; in this step, the first model learns the value constraints between different input parameters, that is, the effective value space is selected from the value space of the large-scale input parameters, thereby reducing the number of first model generation. The value space of the input parameters that needs to be explored in the process of a test case; at the same time, since the value of the input parameters of the first test case generated by the first model meets the preset constraints, no correction is required, ensuring that the test case set The diversity of input parameter values can cover more functional coverage points. In this way, by generating a smaller test case set and spending less testing time, a higher functional coverage rate can be achieved, which effectively improves the Test efficiency. Compared with the method of randomly generating test cases in the constraint-driven random testing method, the number of required test cases is effectively reduced while achieving the same functional coverage.

Exemplarily, the operation of inputting the input parameter set into the first model and generating the first test case set may be repeatedly performed until a preset stop condition is satisfied; for example, the stop condition may include that the number of cumulatively generated test cases exceeds the first preset The set value or the function coverage exceeds the second preset value and so on. Wherein, the first preset value and the second preset value can be set according to the test requirement, which is not limited. As an example, after each operation of inputting the input parameter set into the first model, the first model can automatically generate a preset number of first test cases, calculate the cumulative number of first test cases generated by the first model, and compare them with the first A preset value is compared, and if the accumulatively generated number of first test cases exceeds the first preset value, the above operation of inputting the input parameter set into the first model is stopped. As another example, after each operation of inputting the input parameter set into the first model, the first model can automatically generate a preset number of first test cases, and the number of first test cases contained in the first test case set If the function coverage rate exceeds the second preset value, the above operation of inputting the input parameter set into the first model is stopped , thus completing the test.

In the embodiment of the present application, the value of the input parameter in the first test case training sample satisfies the preset constraints, and the first model is a model obtained by using the first test case training sample training. In this way, the first model obtained after training The preset constraint can be learned, that is, the effective value space can be selected from all the value spaces of the input parameters, thereby effectively reducing the value space of the input parameters to be explored in the process of generating the first test case; At the same time, the first model can automatically generate test cases whose input parameters meet the preset constraints without correction, which ensures the diversity of input parameter values in the first test case set, so that by generating smaller-scale first A collection of test cases can achieve high functional coverage, effectively reducing the time required for testing, effectively improving testing efficiency, and saving testing costs.

The manner of obtaining the first model through training by using the training sample of the first test case is exemplarily described below.

Fig. 6 shows a flow chart of another chip test case generation method according to an embodiment of the present application. As shown in Figure 6, the method may include the following steps:

Step 601. Obtain a training sample of a first test case.

Wherein, for the specific description of the first test case training samples, reference may be made to the relevant expressions above, and details will not be repeated here.

Exemplarily, according to the set of input parameters, a preset number of test cases can be randomly generated, for example, a small batch of test cases can be generated randomly, and then the above randomly generated test cases can be input into the constraint solver, The constraint solver can modify the value of the input parameters in the randomly generated test cases that do not meet the preset constraints, so as to obtain a preset number of test cases whose input parameter values meet the preset constraints, that is, the test case training sample set, The first test case training sample is any one in the test case training sample set. Wherein, the above-mentioned preset quantity can be set according to requirements, which is not limited.

Step 602: Using the training sample of the first test case to train the first preset model to obtain the first model.

Exemplarily, the first preset model may include a generation confrontation network, and the generation confrontation network includes a generator and a discriminator; for example, the first preset model may be GAN, WGAN, WGAN-GP, InfoGAN, Variational Autoencoder, Conditional Variational Autoencoders Wait, no limit on this.

In this step, a conventional training method may be used to train the first preset model by using the first test case training samples obtained above, so as to obtain a trained model, that is, the first model.

As an example, the

above steps

601 and 602 may be performed before the above step 501 . As another example, in the above-mentioned step 502, when the first test case set is generated by repeatedly performing the operation of inputting the input parameter set into the first model, after performing one or more operations, the newly generated The first test case trains the first model, so as to further improve the performance of the first model.

In this way, the first preset model is trained by using the first test case training samples whose input parameter values meet the preset constraints. During the training process, the first preset model can continuously learn the value constraints between different input parameters. , in some examples, the first preset model can continuously learn the adjustment of the input parameter values in the randomly generated test cases by the constraint solver, and select an effective value space from the large-scale input parameter value space, In this way, the trained first model can generate the first test case whose input parameter values meet the preset constraints, and the value space of the input parameters to be explored in the process of generating the first test case is reduced, thus avoiding the need to generate Redundant or invalid test cases do not need to be corrected, which ensures the diversity of input parameter values in the first test case set, so that more functional coverage points can be covered. In this way, by generating a smaller-scale first test case Aggregation can achieve high functional coverage, effectively reducing the time required for testing, effectively improving testing efficiency, and saving testing costs.

Taking the first preset model as an example of generating an adversarial network, the process of using the first test case training sample to train the first preset model in the above step 602 to obtain the first model will be described in detail below.

Fig. 7 shows a flow chart of another chip test case generation method according to an embodiment of the present application. As shown in Figure 7, the method may include the following steps:

Step 701 , using the first test case training sample to perform multiple rounds of training on the GAN; and determine the test case invariance rate corresponding to the generator when each round of training converges.

Among them, the test case invariance rate indicates the ratio of the input parameters whose values meet the preset constraints among the M input parameters of the test cases generated by the generator. Exemplarily, the test case invariance rate can be determined through the constraint solver; the test case generated by the generator is input into the constraint solver, and the constraint solver will modify the values of the input parameters that do not satisfy the preset constraints, In this way, the input parameter whose value has not been modified among the M input parameters of the test case is the input parameter that satisfies the preset constraint, so that the invariance rate of the test case can be obtained.

In a possible implementation, in each round of training, the input parameter set and the value range of each input parameter can be input into the generator to generate the target test case; the target test case and the first test case are trained The sample is input into the discriminator to obtain the similarity between the target test case and the first training test case; according to the similarity, the loss function value is obtained; the backpropagation is performed according to the loss function value, and the generator and the discriminator are trained until reaching convergence. Furthermore, the target test case generated by the generator at the time of convergence can be input into the constraint solver, and the constraint solver can modify the value of the input parameters in the target test case that do not meet the preset constraints; thus the target can be obtained The test case takes the number of input parameters whose value has not been modified before and after the constraint solver, and calculates the ratio of the number to the total number of all input parameters, which is the test case invariance rate corresponding to the generator when the current round of training converges. It can be understood that the higher the test case invariance rate corresponding to the generator, the more input parameters whose values satisfy the preset constraints in the test cases generated by the generator, that is, the better the performance of the generator.

Step 702, taking the generator corresponding to the maximum test case invariance rate as the first model.

Exemplarily, in the test case invariance rate corresponding to the generator obtained in each round of training above, the maximum test case invariance rate is selected. It can be understood that, compared with the generator obtained in other rounds of training, the maximum test case rate The generator corresponding to the invariant rate can better generate test cases whose values of input parameters meet the preset constraints. The invariant rate corresponds to the generator as the first model, which is used to generate test cases whose values of input parameters satisfy preset constraints.

In the embodiment of the present application, using the first test case training samples to conduct multiple rounds of confrontation training on the discriminator and generator in the generative adversarial network, during the training process, the update target of the generator is that the generated target test case can fool the discriminator , so as to be identified as the same as the corresponding first test case sample, the discriminator’s update goal is to be able to identify the target test case generated by the generator; through continuous training, the capabilities of the generator and the discriminator are getting stronger and stronger, generating The detector can mine the potential value constraints between different input parameters, and filter out the effective value space from the large-scale input parameter value space, thereby reducing the number of input parameters that need to be explored in the process of generating test cases. value space. In this way, using the first test case training samples to conduct multiple rounds of training on the generative adversarial network, the potential value constraints between different input parameters are mined, and the generator corresponding to the maximum test case invariance rate is used as the first model. Therefore, the first model can be used to automatically generate test cases whose values of input parameters meet the preset constraints. While achieving high functional coverage, it effectively reduces the number of test cases to be generated, improves test efficiency, and saves cost of testing.

Furthermore, due to the complexity of the chip design to be tested, the way the constraint solver modifies the value of the input parameters in the constraint-driven random testing method is unknown. Therefore, it is difficult for the constraint solver to target the uncovered functional coverage points. Generate corresponding test cases; there are usually many functional coverage points that are difficult to be covered in the later stage of the test. For these functional coverage points that are difficult to be covered, domain experts are often required to test by manually designing related test cases. It consumes a lot of time, thereby greatly increasing the test cost, reducing the test efficiency, and affecting the chip development cycle.

For this reason, the embodiment of the present application further narrows the value space for exploring the input parameters through the correlation information between the target function coverage group and the input parameters on the basis of the above-mentioned first test case where the value of the input parameter satisfies the preset constraints. , so as to generate test cases that can effectively test the untested functions in the target functional coverage group, and improve the functional coverage.

Fig. 8 shows a flow chart of another chip test case generation method according to an embodiment of the present application. As shown in Figure 8, the method may include the following steps:

Step 801. Obtain an input parameter set.

For a specific description of this step, reference may be made to relevant expressions in the above-mentioned step 501, which will not be repeated here.

Step 802: Input the input parameter set into the first model to generate the first test case set.

For a specific description of this step, reference may be made to relevant expressions in the above-mentioned step 502, which will not be repeated here.

Step 803, determine the target function coverage group.

Wherein, the target function coverage group includes the target function to be tested. Exemplarily, the target function coverage group can be any one of the preset function coverage groups; for example, the target function coverage group can be g1, g2... or gR in the above table 3; the target function is the target function coverage group The functionality that needs to be tested is represented by the included functional coverage points.

Step 804. Determine N target input parameters associated with the target function coverage group in the input parameter set according to the association information between the function coverage group and the input parameters, where N is a positive integer not greater than M. Wherein, the association information between the functional coverage group and the input parameters is obtained through training using the second test case training samples and the first functional coverage information corresponding to the second test case training samples.

The values of the M input parameters in the second test case training sample meet the preset constraints; the second test case training sample can be a pre-generated test case; illustratively, the second test case training sample can be the same as the above-mentioned first test case The training samples are the same.

The first function coverage information indicates tested functions and/or untested functions in each function coverage group. The first function coverage information corresponding to the second test case training sample can include the functions that have been tested by the second test case training sample in the target function coverage group and/or the functions that have not been tested by the second test case training sample in the target function coverage group function. Among them, the tested functions in the functional coverage group are the functions to be tested represented by the functional coverage points covered by the second test case training sample in the functional coverage group, and the untested functions in the functional coverage group are Functions that need to be tested are represented by functional coverage points in the functional coverage group that are not covered by the training samples of the second test case. Exemplarily, the first functional coverage information may include the values of the functional coverage parameters c11, c12...cKT in Table 3 above; for example, if the test case x'2 in Table 3 is the second test case training sample, then the corresponding first A function coverage information may include the values of the function coverage parameters c12, c22, c23 corresponding to the function coverage group g1, the values of the function coverage parameters c24, c25, c26, c27 corresponding to the function coverage group g2, etc. R function coverage groups The corresponding function overrides the value of the parameter.

The association information between the functional coverage group and the input parameters can indicate the associated functional coverage group and input parameters. For any functional coverage group, if an input parameter has an impact on the coverage of the functional coverage points contained in the functional coverage group or the impact is relatively large, it is considered that the functional coverage group is associated with the input parameter; otherwise, the functional coverage group is not associated with the input parameter. It can be understood that any functional coverage group can be associated with one or more input parameters; any input parameter can also be associated with one or more functional coverage groups.

In this step, the association information between the functional coverage group and the input parameters is the training of the second test case training sample using the value of the input parameter to meet the preset constraints, and the first functional coverage information corresponding to the second test case training sample, so The association information between each function coverage group and the input parameters is excavated; and then according to the association information, the target input parameters associated with the target function coverage group in the input parameter set are determined, and the target input parameters are the factors that affect the target function coverage group. An important parameter of whether the functional coverage point of is covered, that is, an important parameter that affects whether the target function to be tested in the target functional coverage group can be tested.

Step 805. Determine at least one first value combination of N target input parameters.

Wherein, the first value combination includes values of N target input parameters. Exemplarily, multiple first value combinations may be determined according to the value range corresponding to each target input parameter, and at least one target input parameter has a different value in any two first value combinations.

Since the target input parameter is associated with the target function coverage group, that is, the target input parameter has an impact on whether the target function to be tested in the target function coverage group can be tested, therefore, a certain number of first value combinations can be determined, so that The values of the target input parameters are as diverse as possible, so as to effectively improve the test efficiency while improving the functional coverage; for example, all the values in the value range of each target input parameter can be distributed, so as to generate multiple first Combination of values.

Step 806, using each of at least one first value combination to replace a second value combination of N target input parameters in the first test case set to obtain a second test case set; wherein, the second test case Including the values of M input parameters.

It can be understood that the values of the first value combination and the second value combination including the target input parameters may be the same or different; since the first value combination includes the values of the target input parameters associated with the target function coverage group value, using the first value combination to replace the second value combination, the obtained second test case set can more effectively cover the function coverage points in the target function coverage group, so that the target function to be tested in the target function coverage group Conduct effective testing and improve functional coverage.

Exemplarily, each first value combination can be used to replace the second value combination in the first test case set in the following manner.

Method 1. For each first value combination, determine a second value combination whose similarity with the first value combination satisfies a preset condition, as the target second value combination of the first value combination; use The first combination of values replaces the target second combination of values. Exemplarily, the second value combination of each first test case in the first test case set can be obtained, thereby obtaining a plurality of second value combinations, and for any first value combination, the first value combination is calculated respectively. The similarity between the value combination and each second value combination, and the second value combination corresponding to the highest similarity is used as the target second value combination, and then in the first test case where the target second value combination is located, use The first value combination replaces the target second value combination, so as to obtain the second test case. Since the value of the input parameter in the second value combination satisfies the preset constraint, the second value combination whose similarity with the first value combination satisfies the preset constraint is taken as the target second value combination, so as to ensure the generation of The value of the input parameter in the second test case containing the first value combination satisfies the preset constraint.

Mode 2: For each first value combination, replace the second value combination of each first test case in turn. Exemplarily, the second value combination of each first test case in the first test case set can be obtained, thereby obtaining a plurality of second value combinations, and for any first value combination, each second value combination is sequentially replaced. value combination, so as to obtain multiple second test cases, and so on, until the above substitution operation is completed for all the first value combinations.

Exemplarily, the above steps 803-806 may be repeatedly executed, and for each determined target function coverage group, a second test case for effectively testing the target function to be tested in the target function coverage group is generated until the preset condition is met; The preset conditions may include: the number of accumulatively generated test cases exceeds a preset number, the function coverage meets a preset requirement, or all preset function coverage point groups have been traversed, and the like.

In the embodiment of the present application, the first function coverage information indicates the tested functions and/or untested functions in each function coverage group, and the association information between the function coverage group and the input parameters is that the value of the input parameters satisfies The second test case training sample with preset constraints and the first functional coverage information corresponding to the second test case training sample are trained, and the association information between the functional coverage group and the input parameters can indicate the associated functional coverage group and the input parameter Parameters; furthermore, the target input parameter associated with the target function coverage group in the input parameter set can be determined according to the associated information, and the target input parameter is an important parameter that affects whether the target function to be tested in the target function coverage group can be tested , so as to further narrow the value space of the exploration input parameters and improve the test efficiency; determine the value combination of the target input parameters, and generate the second test case set containing the value combination, and then can target the target function coverage group to be tested Test the target functions more efficiently and effectively improve the functional coverage. In some examples, for an uncovered functional coverage group (or functional coverage point), test cases that can cover the functional coverage group can be generated, thereby effectively improving the functional coverage rate, especially for the functional coverage group that is usually difficult to cover in the later stage of the test (or functional coverage point) has better coverage effect, reduces the manual intervention of domain experts, reduces the test cost, further improves the test efficiency, and effectively shortens the chip development cycle.

The above-mentioned process of obtaining the association information between the function coverage group and the input parameters will be exemplarily described below.

FIG. 9 shows a flowchart of another method for generating test cases for a chip according to an embodiment of the present application. As shown in Figure 9, the method may include:

Step 901, obtain the second test case training sample and the first function coverage information corresponding to the second test case training sample.

Wherein, for the specific description of the second test case training sample and the first function coverage information, reference may be made to the above related expressions, and details will not be repeated here.

Exemplarily, a certain number of test cases can be randomly generated according to the set of input parameters, and the above randomly generated test cases can be input into the constraint solver, and the constraint solver can test the randomly generated test cases that do not satisfy the preset constraints The value of the input parameter in the test case is modified to obtain a test case whose value of the input parameter satisfies the preset constraint, that is, the test case training sample set, and the second test case training sample is any one of the test case training sample set; Furthermore, the second test case training sample can be input into the chip emulator to run, so as to obtain the first function coverage information corresponding to the second test case training sample.

Step 902 , using the second test case training sample and the first function coverage information to train the second preset model to obtain the association information between the function coverage group and the input parameters.

Exemplarily, the second preset model can be a random forest model, for example, it can be a random forest model constructed by decision tree generation algorithms such as ID3, C4.5, and CART; The model of the association relationship is not limited.

As an example, taking the random forest model as the second preset model, for the target function coverage group, the second test case training sample can be input into the random forest model, and the first function coverage information is used as a label to train the random forest model. Forest model, to obtain the association information between functional coverage groups and input parameters.

Exemplarily, the second test case training sample set is input into the random forest model, and the functions that have been tested by the second test case training samples in the target function coverage group and the target function coverage group that have not been trained by the second test case The function tested by the sample is used as a label to train the random forest model. During the training process, the influence of different input parameters on the classification in the training sample of the second test case is continuously mined, that is, whether different input parameters can affect the functions in the target function coverage group. Influenced by the test, determine the input parameters used when each node splits, so that the trained random forest model can accurately predict the coverage of the second test case training samples for the functional coverage points in the target functional coverage group; in the above training process , the input parameters used when splitting each node are the input parameters that affect whether the functions in the target function coverage group can be tested, and then after the training is completed, the importance of each influential input parameter can be determined, and according to the importance Select the N most important input parameters as the input parameters associated with the target function coverage group, wherein the value of N can be set according to actual needs. In the case of multiple preset functional coverage groups, for any preset functional coverage group, a corresponding random forest model can be trained in the above manner to obtain the input parameters associated with the preset functional coverage group, Therefore, the association information between the function coverage groups and the input parameters can be determined according to each preset function coverage group and the input parameters associated with each function coverage group.

As an example, after the above random forest model training is completed, for each input parameter used when splitting a node, the importance score of the input parameter can be calculated, and the importance score can evaluate the effect of the input parameter on the training sample of the second test case The role played by the set in the process of classifying, that is, the degree of influence of the input parameter on the coverage of the target function coverage group. It can be understood that the higher the importance score, the corresponding input parameter is more important to the realization of the second test case training The more important the accurate classification of the sample set is, the greater the impact of the corresponding input parameters on the coverage of the functional coverage points in the target functional coverage group; for example, the input parameters can be split in all decision trees of the random forest model. Such as, Gini index, information entropy, information gain, etc.) as the importance score of the input parameter; thus sorted according to the importance score, and the top N input parameters are used as the coverage group with the target function Associated input parameters.

For example, taking the random forest model corresponding to the training function coverage group g as an example, the test case training sample set is obtained. There are K test case training samples in the test case training sample set, and the label corresponding to each test case training sample is function coverage The functional coverage information corresponding to group g, each sample contains L input parameters; when training each decision tree, first, there are K times of random sampling in the test case training sample set, so that K test cases are selected Training samples, use the selected K test case training samples to train the decision tree, that is, as samples at the root node of the decision tree; then, when each node of the decision tree needs to be split, randomly select from L input parameters Select l input parameters, where l is a constant less than L; determine the evaluation index value corresponding to each input parameter in the l input parameters according to the functional coverage information corresponding to the functional coverage group g, and then select one of the l input parameters The input parameter with the best evaluation index value is used as the feature when the node is split. For example, taking the evaluation index as the Gini index as an example, for any input parameter, the Gini index corresponding to each possible value of the input parameter can be calculated, and then Get the Gini index corresponding to the input parameter, and so on, get the Gini index corresponding to all input parameters, so as to select the input parameter corresponding to the minimum Gini index as the feature used when the node is split; similarly, each node according to the The method determines the corresponding features and splits until the node can no longer be split or the number of nodes exceeds the preset value, then a decision tree is generated without pruning. In this way, by repeatedly performing the above operations of random sampling and randomly selecting the features of each node, a preset number of decision trees are generated, that is, the training of the random forest is completed. After the training is completed, the importance scores corresponding to the input parameters used for node splitting can be calculated, and some input parameters with the highest importance scores can be selected as being associated with the functional coverage group g, so that the functional coverage group g and the input Correlation information between parameters.

In this way, considering that the design of the chip to be tested may include large-scale input parameters, however, for any functional coverage group, not all input parameters may have an impact on the coverage of each functional coverage point in the functional coverage group, the embodiment of the present application In the second preset model, the second preset model is trained by using the second test case training sample and the first function coverage information corresponding to the second test case training sample, so as to dig out the correlation between each function coverage group and the input parameters, that is, for For any functional coverage group, dig out the important input parameters that affect whether the functions in the functional coverage group can be tested, so as to obtain the correlation information between the functional coverage group and the input parameters. As an example, for any functional coverage group, the first functional coverage information corresponding to the second test case training sample can be used as a label, and the second test case training sample is used to train the random forest model corresponding to the functional coverage group, In this way, the relationship between the functional coverage group and the input parameters of the test case is excavated, that is, the important input parameters that affect whether the functions in the functional coverage group can be tested are excavated, and the relationship between the functional coverage group and the input parameters is obtained. Associated information. The above association information can provide guidance for the process of further generating test cases, so that test cases that can cover the functional coverage points in the target functional coverage group can be generated in a targeted manner, thereby effectively improving the functional coverage rate. The value space of the input parameters to be explored effectively improves the test efficiency.

Furthermore, in each of the above embodiments, test cases can also be efficiently generated based on a predefined heuristic strategy, so as to further improve test efficiency.

As an example, the target function coverage group can be determined through a predefined heuristic strategy, for example, in the above step 803, the target function coverage can be determined through a predefined heuristic strategy; exemplarily, a roulette algorithm can be used , sequential selection and other heuristic strategies to determine the target function coverage group.

FIG. 10 shows a flowchart of a method for determining a target function coverage group according to an embodiment of the present application. As shown in FIG. 10 , the following steps may be included:

Step 1001. Obtain second function coverage information corresponding to the first test case set.

Wherein, the second function coverage information may include the number of untested functions in each function coverage group, that is, the number of function coverage points not covered by the first test case set in each function coverage group.

Step 1002, according to the second functional coverage information corresponding to the first test case set, determine the heuristic score of each functional coverage group corresponding to the first test case set.

where the heuristic score is positively correlated with the number of untested functions in the functional coverage group. It can be understood that the greater the number of untested functions in the functional coverage group, the greater the heuristic score of the functional coverage group.

Exemplarily, the total number of untested functions corresponding to the first test case set can be determined according to the number of untested functions in each function coverage group corresponding to the first test case set; The ratio of the number of tested functions to the total number of untested functions determines the heuristic score of each functional coverage group.

Exemplarily, the heuristic score of the functional coverage group can be shown as the following formula (1):

In the formula (1), S _g is the heuristic score of the functional coverage group g, u _g is the number of uncovered functional coverage points in the functional coverage group g, G is the functional coverage group set where the functional coverage group g belongs to, u _k is the number of uncovered functional coverage points in the kth functional coverage group in the functional coverage group set G.

Step 1003, according to the heuristic score of each functional coverage group, determine the probability corresponding to each functional coverage group.

Among them, the probability corresponding to the functional coverage group is positively correlated with the heuristic score of the functional coverage group. It is understandable that the higher the heuristic score for a functional coverage group, the greater the corresponding probability.

Exemplarily, based on the heuristic score obtained by the above formula (1), the probability corresponding to the functional coverage group can be shown in the following formula (2):

In formula (2), r _g is the probability corresponding to the functional coverage group g, S _g is the heuristic score of the functional coverage group g, G is the functional coverage group set where the functional coverage group g belongs to, and S _k is the functional coverage group set Heuristic score for the kth functional coverage group in G.

Step 1004: Determine at least one target function coverage group based on the probability corresponding to each function coverage group.

Exemplarily, a roulette algorithm can be used to select one or more functional coverage groups as the target functional coverage group based on the probability corresponding to each functional coverage group; the higher the probability corresponding to the functional coverage group, the more likely it will be selected as the target function Override group.

In the embodiment of this application, the target function coverage group is selected through a heuristic strategy. The more untested functions in the function coverage group, the higher the corresponding heuristic score, the greater the corresponding probability, and the more likely it is to be tested. Selected as the target functional coverage group; that is, for the functional coverage group that is not covered, it is more likely to be selected as the target functional coverage group; especially in the later stage of the test, the functional coverage group that is difficult to be covered is more likely to be selected as the target functional coverage group Groups, and then can generate test cases in a targeted manner, thereby improving functional coverage, reducing manual intervention by domain experts, reducing testing costs, and improving testing efficiency.

As an example, the first value combination may be determined based on a predefined heuristic strategy. For example, in the above step 805, at least one first value combination of N target input parameters may be determined according to the heuristic strategy; exemplary Specifically, an adaptive random sampling algorithm such as a distance sampling (T-wise) algorithm and a fixed-size candidate set (Fixed-Size-Candidate-Set, FSCS) can be used as a heuristic strategy to determine the first value combination.

Fig. 11 shows a flowchart of a method for determining a first value combination according to an embodiment of the present application. As shown in Fig. 11, the following steps may be included:

Step 1101, randomly generate multiple candidate value combinations according to the preset value ranges of N target input parameters.

Exemplarily, a larger number of candidate value combinations may be generated, for example, 30*k candidate value combinations may be generated, where k is a preset number of value combinations.

Step 1102, taking any value combination among multiple candidate value combinations as the value combination corresponding to the initial iteration round.

In this step, in the initial iteration round, a value combination may be randomly selected from the above multiple candidate value combinations.

Step 1103, from multiple candidate value combinations, determine the value combination corresponding to the current iteration round.

Among them, the value combination corresponding to the current iteration round has the largest difference from the determined value combination corresponding to each iteration round.

Exemplarily, for any candidate value combination, the total distance (such as Euclidean distance) between it and the value combination corresponding to the previous iterative rounds determined can be calculated; furthermore, the total distance corresponding to each candidate value combination For comparison, the candidate value combination corresponding to the maximum total distance is the value combination corresponding to the current iteration round; thus ensuring that the values of the target input parameters in the value combinations corresponding to each iteration round are more uniform and diverse.

Step 1104: Repeat the above operation of determining the value combination corresponding to the current iteration round until the preset iteration condition is met; and use the value combination corresponding to each iteration round as at least one first value combination.

Exemplarily, the preset iteration condition may be that the determined value combinations corresponding to each iteration round reach the preset number K of value combinations.

Considering that for any target input parameter, its corresponding value range can include more values, if all the values are traversed, the number of first value combinations generated is too large, for example, N target input parameters The preset value range of contains 512 values. If all the values are traversed, there are 512 ^N value combinations. In the embodiment of the present application, sampling is performed through a heuristic strategy, and multiple candidate value combinations are first generated, and then through multiple iterations, the value combination corresponding to each iteration round is determined, and the value combination corresponding to each iteration round is the same as The determined value combinations corresponding to each iterative round have the largest difference, thus ensuring that the values of the N target input parameters in the value combinations corresponding to each iterative round are more uniform and diverse; in turn, the values of the input parameters can be generated to be more uniform , Various test cases, while ensuring functional coverage, effectively reduce the number of required test cases and further improve test efficiency.

Further, considering that the value of some input parameters in the second test case has been modified relative to the first test case, therefore, in the obtained second test case set, there may be values of input parameters that do not meet the preset constraints The second test case; in a possible implementation, post-processing can be performed on each test case in the second test case set obtained above, and the values of the M input parameters in the second test case set are filtered out if they do not satisfy the preset Constrained second test case. Exemplarily, the above post-processing operation may be performed after the above step 806 .

As an example, the discriminator converged during the above training process of generating an adversarial network may be used to filter out the second test cases whose values of M input parameters do not meet the preset constraints in the second test case set. Exemplarily, in the multi-round training of the generative confrontation network, it can be determined to obtain a round of training of the generator corresponding to the maximum test case invariance rate, so as to use the discriminator when this round of training converges to filter out the second test A second test case whose values of the M input parameters in the test case set do not satisfy the preset constraints.

As another example, the above-mentioned constraint solver may be used to filter out the second test cases whose M input parameters do not satisfy the preset constraints in the second test case set. Exemplarily, the second test case set is input to the constraint solver, and the value of the input parameter does not meet the preset constraint. The second test case is corrected, thereby realizing filtering out M input parameters in the second test case set A second test case whose values do not satisfy the preset constraints.

In the embodiment of the present application, the second test cases whose values of input parameters do not meet the preset constraints are filtered out, so as to ensure that the values of the input parameters of each second test case in the second test case set all meet the preset constraints.

Taking the design D of a chip to be tested as an example, the method for generating test cases for the above-mentioned chip will be described as an example. The basic information of the chip design D to be tested is: the input parameter set includes 575 input parameters, 7219 function coverage points, and 681 corresponding function coverage groups.

FIG. 12 shows a flow chart of another chip test case generation method according to an embodiment of the present application. As shown in FIG. 12, the following stages may be included:

1. Information acquisition stage: In this stage, relevant information about the chip design to be tested can be obtained, for example, the set of input parameters, the value range of each input parameter, function coverage points, function coverage groups, and the corresponding relationship between function coverage groups and function coverage points etc. At this stage, test case sets and functional coverage information can also be obtained as training samples for training the generated confrontation network and random forest model described below.

Exemplarily, a small batch (for example, 10,000) of test case sets X1 is generated in a random manner, and the test case set X1 is input into the constraint solver, and the constraint solver will vary the values of the input parameters in the test case set X1 Correct the test cases that meet the preset constraints, and return the test case set X2 whose input parameter values meet the preset constraints, input the test case set X2 into the emulator F to run, and obtain the corresponding function coverage information, the function coverage The information can be a 10000*7219 functional coverage matrix M ^cover . In M ^cover , each row represents a test case in the test case set X2, each column represents a functional coverage point, and the value of each element is 0 or 1 , where 1 means that the test case corresponding to the row where the element is located covers the functional coverage point corresponding to the column where the element is located, and 0 means that the test case corresponding to the row where the element is located does not cover the functional coverage point corresponding to the column where the element is located point.

2. The information mining stage, which is used to learn the value constraints between different input parameters of test cases, and to mine the correlation between each function coverage group and input parameters.

Exemplarily, the generator and discriminator of the generative confrontation network are initialized, and the optimizer and loss function are defined. The test case set X2 obtained above is used as a training sample to train the generative confrontation network. In the training process, random noise is used as the input of the generator to generate fake samples, and the discriminator is used to distinguish true and false samples, and the discriminator and generator are trained end-to-end according to the loss value; so that the generator can compare the test case set X2 with The modeling ability of the constraint relationship C(X1, X2) between the test case sets X1 is getting better and better, that is, the value constraints between different input parameters of the test cases are continuously learned. Repeat the above training process for several rounds, and calculate the test case invariance rate corresponding to the test cases generated by the generator when each round of training converges, where the generator pair constraint relationship C(X1, X2) of the maximum test case invariance rate The modeling of is the best, and the model weight of the generator corresponding to the maximum test case invariance rate is reserved, so as to be used in the next stage to generate test cases whose input parameter values meet the preset constraints.

Exemplarily, for any functional coverage group g, the test case set X2 obtained above is used as a training sample, and the functional coverage matrix M ^cover corresponds to the functional coverage group g

As a label, use the input parameters as the characteristics of node splitting, train the random forest model, so that it can accurately predict the coverage of each test case in the test case set X2 for the functional coverage points in the functional coverage group g; for the trained random forest model , use the impurity of each input parameter to determine the correlation information R(g, p) between the functional coverage group g and the input parameter p, where the greater the impurity, the corresponding input parameters have a greater impact on the functional coverage point in the functional coverage group g The greater the impact of the coverage, the N input parameters that have the greatest impact on the coverage of the functional coverage points in the functional coverage group g are selected, and the set of N input parameters is denoted as P _g . In this way, for each functional coverage group, a corresponding random forest model is trained by the above method, and the correlation between the functional coverage group and the input parameters is mined, so as to obtain the correlation information between each functional coverage group and the input parameters; In the process of generating test cases in the subsequent stage, the associated information is used as a heuristic guide to quickly generate test cases with better coverage.

3. The test case generation stage, which is based on the heuristic strategy, uses the value constraints between the different input parameters of the test cases learned above, and digs out the relationship between each function coverage group and the input parameters to guide the direction towards Test cases are generated in a direction that is more likely to achieve higher functional coverage.

Exemplarily, first, use the generator corresponding to the maximum test case invariance rate in the above stage to generate a test case set X3 whose input parameter values satisfy preset constraints. Secondly, use a heuristic strategy to select the functional coverage group g that needs to be covered. For example, for each functional coverage group, you can refer to the above formula (1) to calculate the corresponding heuristic score, and refer to the above formula (2) to calculate the corresponding probability , and then use the roulette algorithm to sample the functional coverage group g; according to the correlation information between each functional coverage group and input parameters obtained in the above stage, determine the input parameter set P _g associated with the functional coverage group g, and according to each input The value range of the parameter p(p∈P _g ) randomly generates 30,000 possible value combinations as the candidate value combination set, and selects 1000 input parameter value distributions from the candidate value combination set. Uniform value combinations, and these 1000 value combinations replace the value combinations of corresponding input parameters in the 1000 test cases in the test case set X3 to generate the test case set X4. Finally, input the test case set X4 generated above into the simulator F, and obtain the corresponding function coverage information, and judge whether the test stop condition is satisfied, if not, repeat the above test case generation stage until the trigger Test the stop condition.

In this way, through the above-mentioned "model first, then generate" method, first, use the generative confrontation network to learn the value constraints between different input parameters, and use the random forest mining function to cover the relationship between the group and the input parameters; then, through One or more rounds of iterations are used to generate a set of test cases. In each round of iterations, the trained generator is used to generate test cases whose values of input parameters meet the preset constraints, and the function coverage of this round is selected through a heuristic strategy. Group, determine one or more input parameters associated with the function coverage group, and generate a variety of value combinations for each input parameter; combine the diversified value combinations and the values of the input parameters to meet the preset constraints The test cases are fused, and the test case sets are generated in a targeted manner, so as to test the functions to be tested in the selected functional coverage group more efficiently; in this way, by generating a smaller test case set, a higher The functional coverage rate, especially for the functional coverage group that is usually difficult to be covered in the later stage of the test, has a better coverage effect, effectively reducing the manual intervention of domain experts, so that 100% functional coverage can be achieved by automatically generating test cases, reducing the test cost. The cost reduces the time required for testing and effectively shortens the chip development cycle; at the same time, it greatly reduces the value space of input parameters that need to be explored when generating test cases, greatly improving test efficiency.

Fig. 13 shows a comparison diagram of test results according to an embodiment of the present application, wherein, the dotted line is the test result obtained by testing the above-mentioned chip design D to be tested by using the test case generation method of the chip in the embodiment of the present application, and the solid line is the test result obtained by using Test results obtained by testing the above-mentioned chip design D under test based on a constraint-driven random testing method. As shown in Figure 13, when 7176 functional coverage points are covered, only 10543 test cases are required for the method in the embodiment of this application, while 20000 test cases are required for the random test method based on constraint drive, that is, this The method in the embodiment of the application can save 94.57% of test cases. It can be seen that when covering the same number of functional coverage points, the method in the embodiment of the application can effectively reduce the number of test cases required. In addition, in the case of generating 20,000 test cases, the method in the embodiment of this application can cover 7217 functional coverage points, while the constraint-driven random testing method can only cover 7176 functional coverage points. It can be seen that when generating the same When there are a large number of test cases, the method in the embodiment of the present application can effectively improve the functional coverage.

Based on the same inventive concept of the above method embodiments, the embodiments of the present application also provide a chip test case generation device, the chip test case generation device is used to implement the technical solutions described in the above method embodiments. For example, each step of the method shown in any one of the above-mentioned Figs. 5-12 may be executed.

Fig. 14 shows a structural diagram of a device for generating test cases for a chip according to an embodiment of the present application. As shown in Fig. 14, the device may include: a transmission module 1401 for obtaining a set of input parameters; the set of input parameters It includes M input parameters of the chip, and M is an integer not less than 1; the processing module 1402 is configured to input the set of input parameters into the first model to generate a first set of test cases; wherein, the first test case includes: the The values of M input parameters, the first model is obtained by using the training samples of the first test case, and the values of the M input parameters in the first test case training samples satisfy the preset constraints.

In a possible implementation, the processing module 1402 is further configured to: determine the target function coverage group, the target function coverage group includes the target function to be tested; according to the association information between the function coverage group and the input parameters , determine N target input parameters associated with the target function coverage group in the input parameter set, N is a positive integer not greater than M; wherein, the association information between the function coverage group and the input parameters is utilized The second test case training sample and the first function coverage information corresponding to the second test case training sample are obtained through training; the values of the M input parameters in the second test case training sample satisfy the preset constraints , the first function coverage information represents the tested functions and/or untested functions in each function coverage group; determine at least one first value combination of the N target input parameters; use the at least one Each of the first value combinations replaces a second value combination of the N target input parameters in the first test case set to obtain a second test case set, wherein the second test case includes all Describe the values of the M input parameters.

In a possible implementation manner, the processing module 1402 is further configured to: randomly generate multiple candidate value combinations according to the preset value ranges of the N target input parameters; Any value combination in the value combination is used as the value combination corresponding to the initial iteration round; from the plurality of candidate value combinations, determine the value combination corresponding to the current iteration round; wherein, the value corresponding to the current iteration round The difference between the value combination and the determined value combination corresponding to each iteration round is the largest; repeat the operation of determining the value combination corresponding to the current iteration round until the preset iteration condition is satisfied; and combine the value combinations corresponding to each iteration round As the at least one first value combination.

In a possible implementation manner, the processing module 1402 is further configured to: filter out second test cases whose values of the M input parameters in the second test case set do not satisfy the preset constraints .

In a possible implementation manner, the processing module 1402 is further configured to: obtain the second function coverage information corresponding to the first test case set, the second function coverage information includes The number of functions tested; according to the second function coverage information corresponding to the first test case set, determine the heuristic score of each function coverage group corresponding to the first test case set, and the heuristic score is the same as that in the function coverage group The number of untested functions is positively correlated; according to the heuristic scores of each functional coverage group, determine the probability corresponding to each functional coverage group; based on the probability corresponding to each functional coverage group, determine at least one target functional coverage group .

In a possible implementation manner, the processing module 1402 is further configured to: determine the first test case according to the number of untested functions in each function coverage group corresponding to the first test case set The total number of untested functions corresponding to the set; according to the proportion of the number of untested functions in each functional coverage group to the total number of untested functions, determine the heuristic score of each functional coverage group.

In a possible implementation manner, the processing module 1402 is further configured to: obtain the first test case training sample; use the first test case training sample to train the first preset model to obtain the first model.

In a possible implementation manner, the first preset model includes a generative adversarial network, and the generative adversarial network includes a generator and a discriminator; the processing module 1402 is further configured to: utilize the first test case The training samples carry out multiple rounds of training to the generation confrontation network; and determine the test case invariance rate corresponding to the generator when each round of training converges, and the test case invariance rate represents the M input parameters of the test case generated by the generator The ratio of the input parameters whose values meet the preset constraints; the generator corresponding to the maximum test case invariance rate is used as the first model.

In a possible implementation, the processing module 1402 is further configured to: filter out the M input parameters in the second test case set that do not satisfy the preset constraints by using the discriminator during convergence. Second test case.

In a possible implementation manner, the processing module 1402 is further configured to: obtain the second test case training sample and the first function coverage information corresponding to the second test case training sample; use the second The test case training sample and the first function coverage information train the second preset model to obtain the association information between the function coverage group and the input parameters.

In a possible implementation manner, the second preset model includes a random forest model; the processing module 1402 is further configured to: for the target function coverage group, input the second test case training sample into The random forest model uses the first functional coverage information as a label to train the random forest model to obtain association information between the functional coverage group and input parameters.

In a possible implementation manner, the processing module 1402 is further configured to: for each first value combination, determine a second value combination whose similarity with the first value combination satisfies a preset condition, A target second value combination as the first value combination; using the first value combination to replace the target second value combination.

For the technical effects and specific descriptions of the device for generating test cases for the chip shown in FIG. 14 and its various possible implementations, please refer to the method for generating test cases for the chip above, which will not be repeated here.

It should be noted that it should be understood that the division of each module of the test case generation device for the above chip is only a division of logical functions, and may be fully or partially integrated into a physical entity or physically separated during actual implementation. And these modules can all be implemented in the form of software through scheduling of processing elements; they can also be implemented in the form of hardware; some modules can also be implemented in the form of processing element scheduling software, and some modules can be implemented in the form of hardware.

An embodiment of the present application provides a device for generating test cases for a chip, including: a processor and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions. Exemplarily, the test case generation method for the chip shown in any one of the above-mentioned FIGS. 5-12 can be implemented.

Fig. 15 shows a schematic structural diagram of a test case generation device for a chip according to an embodiment of the present application. As shown in Fig. 15, the test case generation device for the chip may include: at least one processor 1501, a communication line 1502, and a memory 1503 and at least one communication interface 1504.

The processor 1501 can be a general-purpose central processing unit (central processing unit, CPU), a microprocessor, a specific application integrated circuit (application-specific integrated circuit, ASIC), or one or more for controlling the execution of the application program program integrated circuit.

Communication line 1502 may include a pathway for communicating information between the above-described components.

The communication interface 1504 uses any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, RAN, wireless local area networks (wireless local area networks, WLAN) and so on.

The memory 1503 can be a read-only memory (read-only memory, ROM) or other types of static storage devices that can store static information and instructions, random access memory (random access memory, RAM) or other types that can store information and instructions It can also be an electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or can be used to carry or store desired program code in the form of instructions or data structures and can be programmed by a computer Any other medium accessed, but not limited to. The memory may exist independently and be connected to the processor through the communication line 1502 . Memory can also be integrated with the processor. The memory provided by the embodiment of the present application may generally be non-volatile. Wherein, the memory 1503 is used to store computer-executed instructions for implementing the solutions of the present application, and the execution is controlled by the processor 1501 . The processor 1501 is configured to execute computer-executed instructions stored in the memory 1503, so as to implement the methods provided in the foregoing embodiments of the present application. Exemplarily, each step of the method shown in any one of the above-mentioned Figs. 5-12 can be implemented.

Optionally, the computer-executed instructions in the embodiments of the present application may also be referred to as application program codes, which is not specifically limited in the embodiments of the present application.

Exemplarily, the processor 1501 may include one or more CPUs, such as CPU0 and CPU1 in FIG. 15 .

Exemplarily, the device for generating test cases of a chip may include multiple processors, such as processor 1501 and processor 1507 in FIG. 15 . Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (eg, computer program instructions).

In a specific implementation, as an embodiment, the device for generating test cases for a chip may further include an output device 1505 and an input device 1506 . Output device 1505 is in communication with processor 1501 and can display information in a variety of ways. For example, the output device 1505 may be a liquid crystal display (liquid crystal display, LCD), a light emitting diode (light emitting diode, LED) display device, a cathode ray tube (cathode ray tube, CRT) display device, or a projector (projector) wait. The input device 1506 communicates with the processor 1501 and can receive user input in various ways. For example, the input device 1506 may be a mouse, a keyboard, a touch screen device, or a sensory device, among others.

As an example, combined with the test case generating device of the chip shown in FIG. 15, the transmission module 1401 in FIG. 14 above can be realized by the communication interface 1504 in FIG. The processor 1501 is implemented.

An embodiment of the present application provides a non-volatile computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the foregoing method is realized. Exemplarily, each step of the method shown in any one of the above-mentioned Figs. 5-12 can be implemented.

An embodiment of the present application provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium bearing computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes the above method. Exemplarily, each step of the method shown in any one of the above-mentioned Figs. 5-12 can be implemented.

A computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer-readable storage media include: portable computer disk, hard disk, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), erasable Electrically Programmable Read-Only-Memory (EPROM or flash memory), Static Random-Access Memory (Static Random-Access Memory, SRAM), Portable Compression Disk Read-Only Memory (Compact Disc Read-Only Memory, CD - ROM), Digital Video Disc (DVD), memory sticks, floppy disks, mechanically encoded devices such as punched cards or raised structures in grooves with instructions stored thereon, and any suitable combination of the foregoing .

Computer readable program instructions or codes described herein may be downloaded from a computer readable storage medium to a respective computing/processing device, or downloaded to an external computer or external storage device over a network, such as the Internet, local area network, wide area network, and/or wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

Computer program instructions for performing the operations of the present application may be assembly instructions, instruction set architecture (Instruction Set Architecture, ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or in one or more source or object code written in any combination of programming languages, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as the “C” language or similar programming languages. Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In cases involving a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it can be connected to an external computer such as use an Internet service provider to connect via the Internet). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (Field-Programmable Gate Array, FPGA) or programmable logic arrays (Programmable Logic Array, PLA), the electronic circuit can execute computer-readable program instructions, thereby realizing various aspects of the present application.

Aspects of the present application are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It should be understood that each block of the flowcharts and/or block diagrams, and combinations of blocks in the flowcharts and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that when executed by the processor of the computer or other programmable data processing apparatus , producing an apparatus for realizing the functions/actions specified in one or more blocks in the flowchart and/or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause computers, programmable data processing devices and/or other devices to work in a specific way, so that the computer-readable medium storing instructions includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks in flowcharts and/or block diagrams.

It is also possible to load computer-readable program instructions into a computer, other programmable data processing device, or other equipment, so that a series of operational steps are performed on the computer, other programmable data processing device, or other equipment to produce a computer-implemented process , so that instructions executed on computers, other programmable data processing devices, or other devices implement the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.

The flowchart and block diagrams in the figures show the architecture, functions and operations of possible implementations of apparatuses, systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in a flowchart or block diagram may represent a module, a portion of a program segment, or an instruction that includes one or more Executable instructions. In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.

It should also be noted that each block in the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts, can be implemented with hardware (such as circuits or ASIC (Application Specific Integrated Circuit, application-specific integrated circuit)), or it can be realized by a combination of hardware and software, such as firmware.

Although the present invention has been described in conjunction with various embodiments herein, in the process of implementing the claimed invention, those skilled in the art can understand and Other variations of the disclosed embodiments are implemented. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that these measures cannot be combined to advantage.

Having described various embodiments of the present application above, the foregoing description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and alterations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The terminology used herein is chosen to best explain the principle of each embodiment, practical application or improvement of technology in the market, or to enable other ordinary skilled in the art to understand each embodiment disclosed herein.

Claims

A test case generation method for a chip, characterized in that the method comprises:

Obtain an input parameter set; the input parameter set includes M input parameters of the chip, and M is an integer not less than 1;

The input parameter set is input into the first model to generate a first test case set; wherein, the first test case includes: the values of the M input parameters, and the first model uses the first test case training sample training As a result, the values of the M input parameters in the training samples of the first test case satisfy preset constraints.
The method according to claim 1, further comprising:

determining a target function coverage group, the target function coverage group including the target function to be tested;

According to the association information between the function coverage group and the input parameters, determine N target input parameters associated with the target function coverage group in the input parameter set, N is a positive integer not greater than M; wherein the function The association information between the coverage group and the input parameters is obtained by using the second test case training sample and the first function coverage information corresponding to the second test case training sample; the M in the second test case training sample The value of each input parameter satisfies the preset constraint, and the first function coverage information indicates the tested functions and/or untested functions in each function coverage group;

determining at least one first value combination of the N target input parameters;

Using each of the at least one first value combination to replace one second value combination of the N target input parameters in the first test case set to obtain a second test case set, wherein the first test case set The second test case includes the values of the M input parameters.
The method according to claim 2, wherein said determining at least one first value combination of said N target input parameters comprises:

Randomly generate multiple candidate value combinations according to the preset value ranges of the N target input parameters;

Using any value combination in the plurality of candidate value combinations as the value combination corresponding to the initial iteration round;

From the plurality of candidate value combinations, determine the value combination corresponding to the current iteration round; wherein, the value combination corresponding to the current iteration round has the largest difference from the determined value combination corresponding to each iteration round;

The above operation of determining the value combination corresponding to the current iteration round is repeatedly performed until the preset iteration condition is met; and the value combination corresponding to each iteration round is used as the at least one first value combination.
The method according to claim 2 or 3, characterized in that the method further comprises:

Filtering out second test cases whose values of the M input parameters do not satisfy the preset constraints in the second test case set.
The method according to any one of claims 2-4, wherein said determining the target function coverage group comprises:

Obtaining the second function coverage information corresponding to the first test case set, the second function coverage information including the number of untested functions in each function coverage group;

According to the second functional coverage information corresponding to the first test case set, determine the heuristic score of each functional coverage group corresponding to the first test case set, and the heuristic score is consistent with the number of untested functions in the functional coverage group. number positive correlation;

determining the probability corresponding to each functional coverage group according to the heuristic score of each functional coverage group;

Based on the probability corresponding to each functional coverage group, at least one target functional coverage group is determined.
The method according to claim 5, wherein, according to the second functional coverage information corresponding to the first test case set, determining the heuristic score of each functional coverage group corresponding to the first test case set comprises:

Determine the total number of untested functions corresponding to the first test case set according to the number of untested functions in each function coverage group corresponding to the first test case set;

The heuristic score of each function coverage group is determined according to the ratio of the number of untested functions in each function coverage group to the total number of untested functions.
The method according to any one of claims 1-6, wherein the method further comprises:

Obtain the first test case training sample;

The first preset model is trained by using the first test case training samples to obtain the first model.
The method according to claim 7, wherein the first preset model includes a generative confrontation network, and the generative confrontation network includes a generator and a discriminator;

The first preset model is trained using the first test case training sample to obtain the first model, including:

Use the first test case training sample to carry out multiple rounds of training to the generation confrontation network; and determine the test case invariance rate corresponding to the generator when each round of training converges, and the test case invariance rate represents the generator generated by the test case. Among the M input parameters of the test case, the ratio of the input parameters whose values meet the preset constraints;

The generator corresponding to the maximum test case invariance rate is used as the first model.
The method according to claim 8, characterized in that, the method further comprises: using a convergent discriminator to filter out the M input parameters in the second test case set that do not satisfy the preset constraints Second test case.
The method according to any one of claims 2-9, wherein the method further comprises:

Acquiring the second test case training samples and the first function coverage information corresponding to the second test case training samples;

The second preset model is trained by using the second test case training sample and the first function coverage information to obtain the association information between the function coverage group and the input parameters.
The method according to claim 10, wherein the second preset model comprises a random forest model;

The step of using the second test case training sample and the first function coverage information to train the second preset model to obtain the association information between the function coverage group and input parameters includes:

For the target functional coverage group, input the second test case training sample into the random forest model, and use the first functional coverage information as a label to train the random forest model to obtain the functional coverage group Correlation information with input parameters.
The method according to any one of claims 2-11, wherein each of the at least one first value combination is used to replace the N target inputs in the first test case set A second combination of values for the parameter, including:

For each first value combination, determine a second value combination whose similarity with the first value combination satisfies a preset condition, as the target second value combination of the first value combination;

Using the first value combination to replace the target second value combination.
A test case generation device for a chip, characterized in that the device comprises:

The transmission module is used to obtain an input parameter set; the input parameter set includes M input parameters of the chip, and M is an integer not less than 1;

A processing module, configured to input the set of input parameters into a first model to generate a first set of test cases; wherein, the first test case includes: the values of the M input parameters, and the first model uses the first The values of the M input parameters in the first test case training sample satisfy preset constraints.
The device according to claim 13, wherein the processing module is further configured to: determine the target function coverage group, the target function coverage group includes the target function to be tested; according to the relationship between the function coverage group and the input parameters The associated information, determine the N target input parameters associated with the target function coverage group in the input parameter set, N is a positive integer not greater than M; wherein, the association between the function coverage group and the input parameters The information is obtained by using the second test case training sample and the first function coverage information corresponding to the second test case training sample; the values of the M input parameters in the second test case training sample satisfy the Preset constraints, the first function coverage information indicates the tested functions and/or untested functions in each function coverage group; determine at least one first value combination of the N target input parameters; use the For each of the at least one first value combination, replace a second value combination of the N target input parameters in the first test case set to obtain a second test case set, wherein the second test case The use case includes the values of the M input parameters.
The device according to claim 14, wherein the processing module is further configured to: randomly generate a plurality of candidate value combinations according to the preset value ranges of the N target input parameters; Any value combination in the candidate value combination is used as the value combination corresponding to the initial iteration round; from the plurality of candidate value combinations, determine the value combination corresponding to the current iteration round; wherein, the current iteration round The difference between the corresponding value combination and the determined value combination corresponding to each iteration round is the largest; repeat the above-mentioned operation of determining the value combination corresponding to the current iteration round until the preset iteration condition is satisfied; A combination of values is used as the at least one first combination of values.
The device according to claim 14 or 15, wherein the processing module is further configured to: filter out values of the M input parameters in the second test case set that do not satisfy the preset constraints The second test case for .
The device according to any one of claims 14-16, wherein the processing module is further configured to: obtain the second function coverage information corresponding to the first test case set, the second function coverage The information includes the number of untested functions in each function coverage group; according to the second function coverage information corresponding to the first test case set, determine the heuristic score of each function coverage group corresponding to the first test case set, so The heuristic score is positively correlated with the number of untested functions in the functional coverage group; according to the heuristic scores of the functional coverage groups, the probability corresponding to each functional coverage group is determined; based on the corresponding probability of each functional coverage group probabilistically, at least one target functional coverage group is identified.
The device according to claim 17, wherein the processing module is further configured to: determine the second function according to the number of untested functions in each function coverage group corresponding to the first test case set The total number of untested functions corresponding to a test case set; according to the ratio of the number of untested functions in each of the functional coverage groups to the total number of untested functions, determine the heuristic of each functional coverage group formula score.
The device according to any one of claims 13-18, wherein the processing module is further configured to: obtain the first test case training sample; use the first test case training sample to pair the first The preset model is trained to obtain the first model.
The device according to claim 19, wherein the first preset model includes a generative confrontation network, and the generative confrontation network includes a generator and a discriminator;

The processing module is also used to: use the first test case training sample to perform multiple rounds of training on the generative confrontation network; and determine the test case invariance rate corresponding to the generator when each round of training converges, the test The use case invariance rate represents the ratio of the input parameters whose values meet the preset constraints among the M input parameters of the test cases generated by the generator; the generator corresponding to the maximum test case invariance rate is used as the first model .
The device according to claim 20, wherein the processing module is further configured to: use a convergent discriminator to filter out the M input parameters in the second test case set that do not satisfy the predetermined A second test case with constraints.
The device according to any one of claims 14-21, wherein the processing module is further configured to: obtain the second test case training sample and the first test case corresponding to the second test case training sample. Functional coverage information: using the second test case training sample and the first functional coverage information to train a second preset model to obtain the correlation information between the functional coverage group and input parameters.
The device according to claim 22, wherein the second preset model includes a random forest model; and the processing module is further configured to train the second test case for the target function coverage group Samples are input into the random forest model, and the random forest model is trained by using the first functional coverage information as a label to obtain association information between the functional coverage group and input parameters.
The device according to any one of claims 14-23, wherein the processing module is further configured to: for each first value combination, determine that the similarity with the first value combination satisfies a predetermined Set the second value combination of the condition as the target second value combination of the first value combination; use the first value combination to replace the target second value combination.
A test case generation device for a chip, characterized in that it comprises:

processor;

memory for storing processor-executable instructions;

Wherein, the processor is configured to implement the method according to any one of claims 1-12 when executing the instructions.
A computer-readable storage medium, on which computer program instructions are stored, wherein the computer program instructions implement the method according to any one of claims 1-12 when executed by a processor.
A computer program product, characterized in that, when the computer program product is run on a computer, the computer is made to execute the method according to any one of claims 1-12.