CN113419959A - Method and equipment for generating coverage rate verification report - Google Patents
Method and equipment for generating coverage rate verification report Download PDFInfo
- Publication number
- CN113419959A CN113419959A CN202110744712.4A CN202110744712A CN113419959A CN 113419959 A CN113419959 A CN 113419959A CN 202110744712 A CN202110744712 A CN 202110744712A CN 113419959 A CN113419959 A CN 113419959A
- Authority
- CN
- China
- Prior art keywords
- coverage
- information
- uncovered
- coverage rate
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012795 verification Methods 0.000 title claims abstract description 142
- 238000000034 method Methods 0.000 title claims abstract description 81
- 230000006870 function Effects 0.000 claims description 86
- 238000012360 testing method Methods 0.000 claims description 30
- 238000004590 computer program Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 8
- 238000013515 script Methods 0.000 description 25
- 238000013461 design Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 7
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 235000008694 Humulus lupulus Nutrition 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012938 design process Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3676—Test management for coverage analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application relates to a method and a device for generating a coverage verification report, wherein the method comprises the following steps: acquiring a coverage rate file; traversing the coverage rate file, and searching the information of the functional points which are indicated by the coverage rate file and are not covered; and generating a coverage rate verification report according to the uncovered function point information. Based on the coverage rate file containing various coverage rate information and key identification information indicating various coverage rate information, coverage rate information which is not covered in the logic code can be searched out by traversing the coverage rate file, and a coverage rate verification report is further generated; the display format of the coverage rate verification report is adjusted and arranged, so that the coverage rate verification report with high readability can be generated.
Description
Technical Field
The present application relates generally to the field of software verification technology. More particularly, the present application relates to a method, apparatus, and readable storage medium for generating a coverage verification report.
Background
With the development of integrated circuit technology, the scale and complexity of chips are continuously improved, and especially with the rise of System on a Chip (SOC), the whole System is integrated on one System Chip, so that the Chip verification work is more and more complicated and difficult, the workload is also continuously improved, and the Chip verification work is an essential link in the Chip research and development process.
In order to improve the efficiency of chip verification and the success rate of first-time tape-out, a concept of function coverage is provided, wherein the function coverage means that the verified functions in the chip account for the percentage of all the functions of the chip, namely, the coverage is adopted to quantify the verification progress, and the success of first-time tape-out can be maintained only when all the functions in the chip are verified to be correct, so that the aim of chip verification work is to enable the function coverage to reach 100% as soon as possible.
In the chip design process, different chip design parties use different chip verification tools or different modes, for example, for chip verification, Synopsys VCS can be selected as a simulation verification environment, and functional verification is performed on logic codes in a chip. Functional verification of logic code in the chip is performed using a simulation tool VCS (compiled Verilog simulator), and then a corresponding coverage verification report file can be generated using URG commands (a command form of the VCS), the coverage verification report file includes coverage information of all functional points in the verified chip logic code, e.g., row coverage information, conditional coverage information, state machine coverage information and hop coverage information, and coverage information for non-covered to functional points (e.g., row coverage information, conditional coverage information, state machine coverage information, and hop coverage information) is identified by key character information, so that a technician can determine the condition of a certain chip verification job based on the coverage verification report file, and further guiding subsequent chip verification or other work according to the condition of the chip verification work.
Disclosure of Invention
In the chip verification process, the generated coverage verification report file not only contains uncovered coverage information but also covered coverage information, so that the coverage verification report file contains more information, but for a certain chip verification work, a technician may pay more attention to the coverage information of the uncovered function points and guide the coverage of the function points to reach 100% as soon as possible according to the coverage information of the uncovered function points. In addition, the coverage verification report file generated by using the URG command not only contains a webpage version file, but also contains a text file (for example, in a modinfo.
According to the method and the device, the uncovered function point information is extracted from the coverage rate verification report, the coverage rate verification report only containing the uncovered function point information and not containing the covered function point information is generated, for example, at least one page is arranged in the coverage rate verification report, each page contains one type of uncovered function point information, namely, all the uncovered function point information is only stored in one file, and one page is used for storing one type of uncovered function point information to present the uncovered function point information, so that the readability of the coverage rate verification report and the interaction quality and efficiency of the coverage rate verification report are improved.
According to a first aspect of the present application, there is provided a method of generating a coverage verification report according to the first aspect, comprising: acquiring a coverage rate file; traversing the coverage rate file, and searching the information of the functional points which are indicated by the coverage rate file and are not covered; and generating a coverage rate verification report according to the uncovered function point information.
According to a first method of generating a coverage verification report of a first aspect of the present application, there is provided a second method of generating a coverage verification report according to the first aspect, comprising: the uncovered function point information comprises: one or more types of row coverage information, conditional coverage information, state machine coverage information, and hop coverage information.
According to the first or second method of generating a coverage verification report of the first aspect of the present application, there is provided a method of generating a coverage verification report according to the third aspect, further comprising: the coverage verification report includes first information including a test item name and the test item profile.
According to a third method of generating a coverage verification report of the first aspect of the present application, there is provided a fourth method of generating a coverage verification report of the first aspect, matching the identification information in the coverage file, extracting a test item profile to add to the coverage verification report.
According to a third or fourth method of generating a coverage verification report of the first aspect of the present application, there is provided a method of generating a coverage verification report according to the fifth aspect of the first aspect, the first information further comprising: project information, validation engineer information, and module design engineer information.
According to one of the first to fifth methods of generating a coverage verification report of the first aspect of the present application, there is provided the sixth method of generating a coverage verification report of the first aspect, wherein the first information and the function point information are presented in a tabular form.
According to a sixth method of generating a coverage verification report of the first aspect of the present application, there is provided a method of generating a coverage verification report of the seventh aspect of the present application, traversing the coverage file, searching for uncovered function points indicated by the coverage file, comprising: traversing all modules in the coverage rate file, and identifying the module with the trip coverage rate not reaching 100%; traversing all the row coverage rate information of the module, and adding the information of the uncovered rows into the row coverage rate information.
According to a seventh method of generating a coverage verification report of the first aspect of the present application, there is provided the eighth method of generating a coverage verification report of the first aspect, wherein the information of the uncovered line includes an uncovered line ID, and the number of lines, code and reason for the uncovered line.
According to a sixth method of generating a coverage verification report of the first aspect of the present application, there is provided a method of generating a coverage verification report of the ninth aspect of the present application, traversing the coverage file, searching for uncovered function points indicated by the coverage file, comprising: traversing all modules in the coverage rate file, and identifying the module with the condition coverage rate not reaching 100%; and traversing all the condition coverage rate information of the module, and adding the information of the uncovered condition into the condition coverage rate information.
According to a ninth method of generating a coverage verification report of the first aspect of the present application, there is provided the tenth method of generating a coverage verification report of the first aspect, wherein the information of the uncovered condition includes an uncovered condition ID, and a number of lines, a code, and a reason for the uncovered condition involved.
According to a sixth method of generating a coverage verification report of the first aspect of the present application, there is provided a method of generating a coverage verification report of the eleventh aspect of the present application, traversing the coverage file, searching for uncovered function points indicated by the coverage file, comprising: traversing all modules in the coverage rate file, and identifying the module with the coverage rate of the state machine not reaching 100%; traversing all the coverage rate information of the state machines of the module, and adding the information of the uncovered state machines into the coverage rate information of the state machines.
According to a eleventh method of generating a coverage verification report of the first aspect of the present application, there is provided the twelfth method of generating a coverage verification report of the first aspect, wherein the information of the uncovered state machine comprises an uncovered state machine ID, and the number of rows, codes and uncovered information involved therewith.
According to a sixth method of generating a coverage verification report of the first aspect of the present application, there is provided a method of generating a coverage verification report of the thirteenth aspect of the present application, traversing the coverage file, searching for uncovered function points indicated by the coverage file, comprising: traversing all modules in the coverage rate file, and identifying the module with the jump coverage rate not reaching 100%; traversing all the jump coverage rate information of the module, and adding the information which does not cover the jump into the jump coverage rate information.
According to a thirteenth method of generating a coverage verification report of the first aspect of the present application, there is provided the method of generating a coverage verification report of the fourteenth aspect of the present application, wherein the information of the uncovered hops includes an uncovered port ID, a port name and a signal type, or an uncovered signal ID, a signal name and a signal type, and an uncovered reason.
According to one of the sixth to fourteenth methods of generating a coverage verification report of the first aspect of the present application, there is provided the method of generating a coverage verification report according to the fifteenth of the first aspect, in which one page contains all functional point information that is not covered each kind, wherein the coverage verification report includes at least one page.
According to a fifteenth method of generating a coverage verification report of the first aspect of the present application, there is provided the sixteenth method of generating a coverage verification report of the first aspect, each page containing the first information.
According to one of the methods of generating coverage verification reports of the first to sixteenth aspects of the present application, there is provided the method of generating a coverage verification report of the seventeenth aspect, further comprising: searching whether a homonymous report exists or not, and if so, backing up the homonymous report to form a backup report; and in the process of generating the coverage rate verification report, for an uncovered function point, matching the function point with the function point in the backup report, and if the matching is successful, adding the uncovered reason of the corresponding function point in the backup report into the generated coverage rate verification report.
According to a seventeenth method of generating coverage verification reports of the first aspect of the present application, there is provided a method of generating coverage verification reports according to the eighteenth of the first aspect, the backup reports being formed by changing file names.
According to a seventeenth method of generating a coverage verification report of the first aspect of the present application, there is provided a method of generating a coverage verification report according to the nineteenth of the first aspect, matching the functional point with a functional point in the backup report, comprising: and matching according to the test module name, the function point type and the corresponding code of the function point.
According to a eighteenth method of generating a coverage verification report of the first aspect of the present application, there is provided a method of generating a coverage verification report according to the twentieth of the first aspect, further comprising: and deleting the backup report after the coverage rate verification report is generated.
There is provided a method of generating a coverage verification report according to the twenty-first of the first aspect, the method being based on Python scripts.
According to one of the first to twenty-first methods of generating a coverage verification report of the first aspect of the present application, there is provided a method of generating a coverage verification report according to the twenty-second method of the first aspect, the verification report being in an Excel file format.
According to a second aspect of the present application, there is provided an apparatus for generating a coverage verification report according to the second aspect of the present application, comprising a processor and a memory, the memory storing a computer program which, when executed by the processor, implements any of the methods for generating a coverage verification report according to the first aspect of the present application.
According to a third aspect of the present application, there is provided a computer readable storage medium according to the third aspect of the present application, having stored thereon a computer program which, when executed, implements any of the methods of generating a coverage verification report of the first aspect of the present application.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art according to the drawings.
FIG. 1 is a flow diagram of a method of generating a coverage verification report according to an embodiment of the present application;
FIG. 2 is a screenshot of a line coverage information page of a coverage verification report using Excel according to an embodiment of the application;
FIG. 3 is a conditional coverage information page screenshot of a coverage verification report using Excel according to an embodiment of the application;
FIG. 4 is a screenshot of a state machine coverage information page of a coverage verification report using Excel according to an embodiment of the application;
FIG. 5 is a screenshot of a skip coverage information page of a coverage verification report using Excel according to an embodiment of the application;
FIG. 6 is a flow chart of script execution according to an embodiment of the present application;
FIG. 7 is a flowchart of step S206 in FIG. 6;
fig. 8 is a block diagram of a device that generates a coverage verification report according to an embodiment of the present application.
Detailed Description
The technical idea of the application is that uncovered function point information is extracted from a coverage rate file generated by using a URG command, the uncovered function point information is listed in a coverage rate verification report form, and the uncovered coverage rate information is analyzed and explained in the coverage rate verification report, so that the readable line of the coverage rate verification report is strong, and the related staff can conveniently check the coverage rate verification report.
The content of the work described above is a textual work, and therefore the application contemplates using computer technology (e.g., scripts) to detect coverage files generated by URG commands. The coverage rate file includes a web page file and/or a text file, for example, the text file is a modinfo. In the coverage rate file, the information of the functional points which are not covered by the specified keyword identification is marked, and the type of the functional points is not unique, so that the specified keyword and the functional points have corresponding relation. In order to extract the uncovered function point information from the coverage rate file, for example, the coverage rate file may be traversed to match the specified keyword, the coverage rate information uncovered by the logic code is identified, and then a coverage rate verification report only showing the uncovered function point information is generated.
Fig. 1 shows a flow of a method for generating a coverage verification report, which is described in detail below.
First, step S101 is executed to acquire a coverage file. For example, the coverage file is a modinfo.
Then, step S102 is executed to traverse the coverage file and search the uncovered function point information indicated by the coverage file.
Finally, step S103 is executed to generate a coverage verification report according to the uncovered function point information.
The method aims to present the uncovered function point information in the coverage rate file, generate the coverage rate verification report and use the coverage rate verification report as the project interaction document, so that the coverage rate verification report contains the uncovered function point information and also contains the project related information. By way of example, the coverage verification report further includes first information (item-related information), the first information mainly including: test project name, project information, validation engineer information, module design engineer information, and test project profile.
TABLE 1
Table 1 shows the composition of the first information by way of example. Wherein, the Project information may include a Project number (Project Code); the verification engineer information (verification Leader) and the module Design engineer information (Design Leader) may include a name and a mailbox; the test item profile (Summary) refers to information related to a test result, such as total score and functional point information that is not covered by each type, and specifically includes total score (total score), line coverage (line), condition coverage (condition), state machine coverage (fsm), and skip coverage (toggle), for example. In table 1, "xxx" indicates specific information of the corresponding field.
The test Project Name (IP Name), the Project Code (Project Code), the verification engineer information (verification lead), and the module Design engineer information (Design lead) are collectively referred to as configuration information, and may be imported from a configuration document or extracted from a coverage file, which is not limited in this application. A test item profile (Summary) is extracted from the coverage file. As will be explained in more detail below.
As shown in table 1, the first information is embodied in a form of table, which can clearly present the information, and is beneficial for the staff to view, so as to improve readability.
When the chip logic code is subjected to function verification, the corresponding function point types are various, so that the uncovered function point information in the coverage rate file after the chip verification comprises one or more types, wherein in an application scene, the uncovered function point information comprises but is not limited to: one or more types of row coverage information, conditional coverage information, state machine coverage information, and hop coverage information. For example, the uncovered function point information may also be presented in the form of a table as shown in table 1, specifically, as shown in tables 2, 3, 4, and 5 below.
TABLE 2
For example, table 2 shows a table corresponding to row coverage information, where a 1 st row in the table corresponds to a module (referred to as instance in this application), a 2 nd row includes a plurality of first fields, for example, the plurality of first fields include keywords such as total score, line, condition, fsm, toggle, and the like, a 3 rd row includes a coverage profile of the module indicated by each keyword in the 2 nd row, a 4 th row includes a plurality of second fields, for example, the plurality of second fields include keywords such as hole id, uncovered category, line number, code, uncovered replay, and the like, and a 5 th row includes specific row coverage information indicated by each keyword in the 4 th row. Wherein, uncovered category represents the uncovered function point type, wherein the uncovered function point type in table 2 is "line". Also in table 2 "xxx" denotes the specific information indicated by the key in row 2 or row 4, which can be extracted from the coverage file and filled in table 2. The extraction method comprises the following steps: first, traversing all modules (instances) in the project, when the line coverage (line coverage) of a module is not full (that is, does not reach 100%), extracting a coverage summary (coverage summary) of the module, and filling each information in the coverage summary into a position corresponding to the 3 rd row in table 2. Then, the module is traversed to obtain the line coverage information, if an uncovered line is matched, for example, the uncovered line may be referred to as a first uncovered line, a hole id is assigned to the uncovered line, the line number corresponding to the first uncovered line is filled in the column, and the code corresponding to the line number is filled in the column. The first uncovered line information fills in the 5 th line of table 2, if an uncovered line is matched, it may be called as a second uncovered line, when a hole id is given to the second uncovered line, it may be added on the basis of the hole id of the first uncovered line, then the related information is filled in the 6 th line of table 2, and so on, until all the uncovered line information is filled in table 2. In addition, the uncovered cause (uncovered reason) is also included in table 2, and may be filled in by a module design engineer or extracted from a legacy coverage verification report (described in detail below).
TABLE 3
For example, table 3 shows a table corresponding to the condition coverage information, where line 1 corresponds to a module (instance) name, line 2 also includes a plurality of first fields, for example, the plurality of first fields include keywords such as total score, line, condition, fsm, toggle, and the like, line 3 includes a coverage profile of the condition module indicated by each keyword in line 2, line 4 includes a plurality of second fields, for example, the plurality of second fields include keywords such as hole id, uncovered category, line number, code, uncovered replay, and the like, and line 5 includes specific condition coverage information indicated by each keyword in line 4. Wherein, uncovered category represents the uncovered function point type, and the uncovered function point type in table 3 is "condition". Also in table 3 "xxx" represents the specific information indicated by the key in row 2 or row 4, which can be extracted from the coverage file and filled into table 3. The extraction method comprises the following steps: first, all modules (instances) in the project are traversed, and when the conditional coverage (line coverage) of a module is not full (i.e., does not reach 100%), a coverage profile (coverage summary) of the module is extracted and filled in the row 3 of table 3. And then traversing the conditional coverage rate information of the module, if an uncovered condition is matched, namely a first uncovered condition, giving a hole id, filling the line number related to the first uncovered condition into the field corresponding to the line number, and filling the related code into the field corresponding to the code. If one more uncovered condition is matched, it may be referred to as a second uncovered condition. For the first uncovered condition, fill in line 5 of table 3, for the second uncovered condition, add on the previous hole id when assigning a hole id, then fill in the related information in line 6 of table 3, and so on until all uncovered condition information fills in table 3.
TABLE 4
For example, table 4 shows a table corresponding to the coverage information of the state machine, where line 1 corresponds to a module (instance) name, line 2 also includes a plurality of first fields, for example, the plurality of first fields include keys such as total score, line, condition, fsm, toggle, and the like, line 3 includes a coverage profile indicated by each key in line 2, line 4 includes a plurality of second fields, for example, the plurality of second fields include keys such as hole id, uncovered category, line number, code, uncovered replay, and the like, and line 5 includes specific coverage information of the state machine indicated by each key in line 4. Wherein, uncovered category indicates the uncovered function point type, the uncovered function point type is "fsm" in table 4, and "xxx" in table 4 indicates that the specific information indicated by the keyword in row 2 or row 4 can be extracted from the coverage file and filled in table 4. The extraction method comprises the following steps: first, traverse all the modules (instances) in the entry, and when the state machine coverage (fsm coverage) of a module is not full (i.e. not reaching 100%), extract the coverage profile (coverage summary) of the module and fill in the 3 rd row of table 4. And traversing the state machine coverage rate information of the module, if an uncovered state machine is matched, namely a first uncovered state machine, giving a hole id, filling the line number related to the first uncovered state machine into a column corresponding to the line number, and filling the related code into a column corresponding to the code. If one more uncovered state machine is matched, it can be called a second uncovered state machine. For the first uncovered state machine, fill in line 5 of table 4, for the second uncovered condition, add on the previous hole id when assigning hole id, then fill in the related information in line 6 of table 4, and so on until all the uncovered state machine information is filled in table 4.
TABLE 5
For example, table 5 shows a table corresponding to skip coverage information, where line 1 corresponds to a module (instance) name, line 2 also includes a plurality of first fields, and for example, the plurality of first fields include keywords such as total score, line, condition, fsm, toggle, and the like, line 3 indicates a coverage profile indicated by each keyword in line 2, and line 5 indicates specific skip coverage information indicated by each keyword in line 4. Wherein, the uncovered category represents the uncovered function point type, the uncovered category is represented as "toggle" in table 5, and "xxx" in table 5 represents the specific information indicated by the keyword in row 2 or row 4, which can be extracted from the coverage rate file and filled in table 5. The extraction method comprises the following steps: first, all modules (instances) in the item are traversed, and when the skip coverage (toggle coverage) of a module is not full (i.e. does not reach 100%), a coverage profile (coverage summary) of the module is extracted and filled in the 3 rd row of table 5. Then, traversing the jump coverage rate information of the module, if an uncovered jump is matched, namely a first uncovered jump, giving a hole id, filling the type (0 → 1 or 1 → 0) related to the first uncovered jump into the uncovered type field, and filling the corresponding port name or signal name into the field corresponding to the port/signal. If an uncovered jump is matched, it can be called a second uncovered jump. For the first uncovered jump, fill in line 5 of table 5, for the second uncovered jump, add on the previous hole id when assigning hole id, then fill in the related information into line 6 of table 5, and so on until fill in the uncovered jump information into table 5.
According to the embodiment, based on the coverage rate file containing various coverage rate information and indicating the key identification information of the various coverage rate information, coverage rate information which is not covered by the logic code can be searched by traversing the coverage rate file, and then a coverage rate verification report is generated; the display format of the coverage rate verification report is adjusted and arranged, so that the coverage rate verification report with high readability can be generated.
Fig. 2-5 show file formats for coverage verification reports using Excel. In the embodiments of fig. 2-5, an Excel file is used as the output format of the coverage verification report. The method comprises the steps of constructing a plurality of pages in an Excel file, and respectively displaying line coverage rate information, condition coverage rate information, state machine coverage rate information and skip coverage rate information by adopting different pages.
By way of example, table 1 and table 2 are placed on the same page (page identified as line, as shown in fig. 2) in which the line coverage information is presented; placing the table 1 and the table 3 in the same page (the page is identified as condition, as shown in fig. 3), and displaying the condition coverage rate information in the page; placing table 1 and table 4 in the same page (page id fsm, as shown in fig. 4), and displaying the coverage rate information of the state machine in the page; table 1 and Table 5 are placed on the same page (the page is labeled toggle, as shown in FIG. 5), and jump coverage information is shown in the page. In each page of fig. 2, 3, 4, and 5, table 1 (in which the first information is described) is a header of each page, and table 2, 3, 4, or 5 (in which the function point information is described) is a function point information part of each page. By the method, when the function point information is checked by switching the page, a worker can more intuitively see the types of the uncovered function points and the related information of the items contained in the current page, and the readability of the coverage rate verification report is improved. In other embodiments, table 1 may occupy one page separately, and tables 2, 3, 4, and 5 may occupy one page respectively.
In an embodiment, a Python script in a Linux environment is used to implement a method for generating a coverage verification report in an Excel format, which will be described in detail below. It should be noted that, for achieving the purpose of the invention of the present application, the selection of the operating environment, the selection of the scripting language, and the generated file form are arbitrary, and the present application is not limited.
For example, in the solution provided in the embodiment of the present application, a coverage file is generated based on Synopsys URG, coverage information of a test module is obtained from a modinfo. txt document of the coverage file, and various types of function point information are displayed in the previously agreed Excel format, a URG command generates a folder urgReport, which includes the previously mentioned modinfo. txt document, so that the script needs an input parameter to inform the urgReport of a folder path.
Txt lists all module coverage information, including the verification platform top module, and the module under test (DUT) is usually an example of the verification platform top module, so the coverage profile in the header of the page is the coverage information obtained for the top module in the verification platform. However, since the module names and the verification engineers are used to different names, the top module names of the verification platforms are different, so the script needs an input parameter to let the user tell the script the top module name of the corresponding verification platform used by the test. The specific input parameter requirements are as follows:
the input parameters include-C, -C, -d, -h, -t, -l, through which relevant information is passed to the script. For example, -C is used to cause the script to create an empty configuration document; c for reading a configuration document. The configuration document is used to describe information other than the test item profile in the first information, and for example, the configuration document includes configuration information such as a test item name, a item number, verification engineer information, and module design engineer information.
In the solution provided in the embodiment of the present application, the entry of the item information into each page may be parameterized by executing a script, or the contents of a fixed format input file (input cfg) may be imported into each page. Cfg files may be empty or non-empty, for example, when the corresponding profile is input during script execution. Cfg file is exemplified as follows: .
Cfg file example is as follows (corresponding to the configuration case of fig. 2-5):
the code represents that if the configuration document is empty, the Input _ cfg () task is executed; the Input _ cfg () task is used to manually Input configuration information; if the profile is not empty, a Load _ cfg _ info () task is performed, which is used to Load the already filled-in profile CfgFile (CfgFile is the profile name). The Input _ cfg () code is exemplified as follows:
and after the user fills the configuration document, executing the script and transmitting the configuration document into the script.
Further, in the solution provided by the embodiment of the present application, extracting uncovered function point information through a script, and generating a coverage verification report involve a write operation and a read operation on an Excel file. By way of example, in one embodiment, the script uses xlwt and xlrd modules, which primarily implement write and read operations on Excel files. In addition, as an example, in the solution provided in the embodiment of the present application, two categories are defined in the script, one category is used for managing engineering projects (project), and the code of the category is exemplified as follows:
another category for managing participant information, including verification engineers (verifier) and module design engineers (designer), has the following code examples:
fig. 6 shows a flow of script execution. As shown in fig. 6, first, step S201 is executed to declare variables that need to be used; such as the variable name, document path (for storing coverage verification report), test item (topmodule) name, and profile name corresponding to the above two categories, for example, the profile name is defined as CfgFile.
Then, step S202 is executed to analyze the input parameters and perform corresponding processing according to the input parameter types. For example, if the input parameter has a-h parameter, help information is output to prompt the user; if there is a-C parameter among the input parameters, an empty profile is created (for the following step S203 to show the empty profile to the user). If the input parameters include-c, -d, -l and other parameters, the obtained parameter information is assigned to the variable declared in step S201.
Next, step S203 is executed to process the configuration document, including: and if the configuration document is empty, calling an Input _ cfg () task, displaying the empty configuration document to the user, and asking the user to manually fill in the empty configuration document. If the configuration document is not empty, the Load _ cfg _ info () task is called to Load the configuration document cfgfefile.
Then, step S204 is executed to determine whether there is a same-name Excel file in the document path, and if so, the document name is modified to obtain a backup report.
Then, step S205 is executed to create a new Excel file, and set 4 pages in the new Excel file, where the four pages correspond to fig. 2, fig. 3, fig. 4, and fig. 5, respectively. An example of code is as follows:
Excel_headcfg(ws_line)
Excel_headcfg(ws_cond)
Excel_headcfg(ws_fsm)
Excel_headcfg(ws_tgl)
wherein the Excel _ headcfg () task is used to create pages in an Excel file.
And then, executing the step S206, analyzing the coverage rate file, capturing the uncovered function point information, and writing the uncovered function point information into a corresponding page in the new Excel file, thereby perfecting the new Excel file. Examples of relevant code for the grab function are as follows:
FileAnalysis(FilePath,ws_line,’line’)
FileAnalysis(FilePath,ws_cond,’cond’)
FileAnalysis(FilePath,ws_fsm,’fsm’)
FileAnalysis(FilePath,ws_tgl,’tgl’)
wherein, the FileAnalysis () task is used for obtaining the coverage information from the coverage file (modiinfo. txt) and writing the coverage information into the corresponding Excel page. For example, the first parameter FilePath identifies a path of a coverage file (modinfo.txt), the second parameter represents a corresponding Excel page, and the third parameter represents a type of a function point to be grabbed.
Finally, step S207 is executed to save the new Excel document as a coverage verification report. And if the backup document exists, deleting the backup document.
In the above flow, the significance of step S204 is that, in actual work, when the code coverage is analyzed, and some uncovered points are analyzed halfway, a supplementary test can be performed by adding a test case. After the supplementary test, the VCS regenerates the coverage file (the old coverage file and the old Excel file correspond to each other before the supplementary test). When the script is executed on the new coverage file, a new Excel file is generated (step S205), and many duplicate function point information exists in the new Excel file and the old Excel file. The purpose of this step is to determine whether there is an old Excel file, and then perform corresponding processing. If there is a same name Excel file in the document path (the same name as the Excel file to be created in step S205), it is indicated that the Excel file is an old Excel file, corresponding to the old coverage file. At this time, in order to avoid the same name as the Excel file to be created in step S205, the Excel file is renamed, and a backup report is generated. For example, the renaming rule is: code _ coverage _ ufc.xls (original name) → code _ coverage _ UFC _ bak.xls (new name). It can be seen that the new name is added with a "_ bak" suffix on the original name. In other embodiments, the name may not be changed, and other ways, such as changing the storage path, may be used for backup.
In the FileAnalysis () task in step S206, when an uncovered function point is captured, before writing into the corresponding page in the Excel file created in step S205, first performing matching in the backup report, and if the matching is successful, adding the uncovered reason of the corresponding function point in the backup report to the Excel file generated in step S205. The method is equivalent to copying the content of the uncovered reason (uncovered reason) in the backup report into a newly generated Excel file. The significance of this approach is: in actual work, with the addition of test cases, new Excel files can be generated continuously and iteratively, and uncovered reasons (uncovered reasons) which are marked before can be inherited by the new Excel files in such a way, so that the uncovered reasons (uncovered reasons) are prevented from being filled in repeatedly by manpower.
Finally, step S207 is executed to delete the backup report, so as to save storage space and avoid the occurrence of duplicate naming when the backup document is generated again.
The rule for matching a function point in the backup report includes: the module names are the same, the function point types (line \ cond \ fsm \ toggle) are the same, and the codes are the same. For example, if an uncovered line is found, the backup report is read first, and if a function point with the same module name, the same function point type, and the same code exists in the backup report, the uncovered reason (uncovered reason) corresponding to the function point is returned. Examples of relevant codes are as follows:
wherein, the Read _ Excel () task is used for realizing reading of the backup report. Wherein, script parameter-l corresponds to Line _ judge _ en ═ dis \ en, and when Line _ judge _ en is dis, the condition that Read _ Excel () returns the uncovered reason is: the function points are of the same type, the same codes and the same module names. This is the same as the matching rule described above; in addition, the code also exhibits a more specific point: when Line _ judge _ en is not dis, the condition for returning the uncovered reason is to add the Line number (Line number) equality on the basis of the above condition.
In addition, the FileAnalysis () task in step S206 is used to obtain coverage information from the coverage file (modiinfo. txt) and write the coverage information into the corresponding Excel page, which is an important link in the embodiment shown in fig. 6. Fig. 7 shows the process flow of the FileAnalysis () task. As shown in fig. 7, first, step S301 is executed to traverse the coverage file (modinfo. txt), and perform matching based on the keyword [ Module Instance: ] corresponding to the test item and the coverage profile keyword [ Instance' ssubtree ], so as to obtain the coverage profile (summary) of the test item.
Then, step S302 is executed to perform matching based on the keyword [ Module: ] indicated by the Module name, and obtain the Module name and the coverage profile. Wherein one module is divided into one and multiple cases. Once, the code is exemplified as follows:
wherein the module name is ufc _ assignment, and the coverage profile includes the content of the third row; the coverage information of the last row indicates the case once.
In the case of two times, the code is exemplified as follows:
wherein the module name is cmd _ status _ slot, and the coverage profile includes the contents of the third line; the coverage information of the last two rows indicates the case of two times. In step S302, the names of all modules are sequentially written into an array. Then, in step S303, each module is sequentially processed based on the above-described number group. If the uncovered function point type of a Module is 'Line' (Line Coverage is not full (i.e. does not reach 100%)), the Coverage profile (Coverage summary) of the Module is extracted, and then the Module is matched with the keyword [ Line Coverage For Module: or [ Line Coverage For Instance: row coverage information is extracted.
Similarly, if the function point type is 'continuation', the keyword [ Cond Coverage For Module: or [ Cond Coverage For Instance: extracting condition coverage rate information; if the function point type is 'fsm', the function point type is matched with a keyword [ fsm Coverage For Module: or [ fsm Coverage For Instance: extracting state machine coverage rate information; if the function point type is 'toggle', the function point type is matched with a keyword [ toggle Coverage For Module: or [ toggle Coverage For Instance: skip coverage information is extracted.
In the process of traversing the whole file, when an uncovered function point is matched, adding a row at the lowest part of the corresponding page of the Excel file, and filling related information. For example, each time an uncovered line is matched, a line is added below the "line" page of fig. 2 to fill in the information of the corresponding uncovered line.
The following description will exemplify the relationship between the coverage file and the script code. For example, part of the script code is:
always@(posedge clk or negedge rst_n)begin
if(！rst_n)begin
ctrl_state<＝2’h0；
end
else if(ctrl_en＝＝1’h1)begin
ctrl_state<＝2’h1；
end
else begin
ctrl_state<＝2’h2；
end
end
if ctrl _ en is not established at 1 'h 2, which results in that the line ctrl _ state < > 2' h2 cannot be executed, the corresponding line coverage (line coverage) indicates that the line is not covered, and is represented in the following format in the coverage file (modinfo.
Wherein, the digital part (100-110) at the left side is the number of lines of the code in the file; the key identifier 1/1 indicates that this line has been executed; key identifier Tests: T1T 2T 3 indicates test case numbers T1, T2 and T3 corresponding to the upper row being executed. The key identifier 0/1 indicates that this line has not been executed. Thus, the script may, via key identifier 0/1, grab out row coverage information that the corresponding module has not been executed.
Furthermore, as in the script code presented above, if ctrl _ en ═ 1' h1 does not hold, the condition of line 104 in the coverage file is not covered, and is represented in the coverage file (modinfo.txt) in the following format:
the condition that is Not Covered is identified by the keyword Not Covered (for example, the last line), so that the script only needs to identify the keyword Not Covered when reading the condition coverage rate of a certain module during file traversal.
Similarly, the state machine coverage information lists the states and transitions of the corresponding state machine in a coverage file (modinfo.txt), wherein the states can display whether coverage exists in a single state of the state machine, the corresponding state information, covered test cases and the like, the transitions can list jump scenes between states related in the code and identify whether coverage exists, and the format of the corresponding coverage file (modinfo.txt) is as follows:
the state or jump state Not Covered by the state machine in the code can also be sorted out through the keyword Not Covered (for example, the last line).
Similarly, jump coverage is also listed in a coverage file (modinfo. txt), for example:
the uncovered ports or signals are identified by the key No (for example, the 5 th line and the last line), so that the key No is only needed to be identified in the script processing process.
In summary, since all the coverage information is in the coverage file (modinfo. txt), and all types of coverage information have corresponding keywords for identification, the corresponding coverage information can be matched by using the keywords, and then the information is added to the generated coverage verification report (in the form of Excel file).
According to another aspect of the present application, an embodiment of the present application further provides an apparatus for generating a coverage verification report, where an internal structure diagram of the apparatus may be as shown in fig. 8. The equipment is computer equipment and comprises a processor, an internal memory, a communication interface, a display screen and an input device which are connected through a system bus. The processor of the computer equipment is used for providing calculation and control capability, and various varieties such as a CPU, a singlechip, a DSP or an FPGA can be selected. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The steps described in the above-described method embodiments may be performed when the computer program is executed. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program, when executed by a processor, implements a method of generating a coverage verification report. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 8 is merely a block diagram of some of the structures associated with aspects of the present application, and is not intended to limit the computing devices of the present application, as a particular computing device may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
According to yet another aspect of the present application, there is also provided a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the above-mentioned method of generating a coverage verification report. The storage medium may include various conventional storage media such as a nonvolatile and/or volatile memory.
It is noted that for the sake of brevity, this application describes some methods and embodiments thereof as a series of acts and combinations thereof, but those skilled in the art will appreciate that the aspects of the application are not limited by the order of the acts described. Accordingly, one of ordinary skill in the art will appreciate that certain steps may be performed in other sequences or simultaneously, in accordance with the disclosure or teachings herein. Further, those skilled in the art will appreciate that the embodiments described herein are capable of alternative embodiments, i.e., acts or modules referred to herein are not necessarily required for the implementation of the solution or solutions described herein. In addition, the description of some embodiments of the present application is also focused on different schemes. In view of the above, those skilled in the art will understand that portions that are not described in detail in one embodiment of the present application may also be referred to in the related description of other embodiments.
In particular implementation, based on the disclosure and teachings of the present application, one of ordinary skill in the art will appreciate that the several embodiments disclosed in the present application may be implemented in other ways not disclosed herein. For example, as for the units in the foregoing embodiments of the electronic device or apparatus, the units are split based on the logic function, and there may be another splitting manner in the actual implementation. Also for example, multiple units or components may be combined or integrated with another system or some features or functions in a unit or component may be selectively disabled. The connections discussed above in connection with the figures may be direct or indirect couplings between the units or components in terms of connectivity between the different units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection utilizing an interface, where the communication interface may support electrical, optical, acoustic, magnetic, or other forms of signal transmission.
While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application. It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. A method of generating a coverage verification report, comprising:
acquiring a coverage rate file;
traversing the coverage rate file, and searching the information of the functional points which are indicated by the coverage rate file and are not covered;
and generating a coverage rate verification report according to the uncovered function point information.
2. The method of claim 1, wherein,
the uncovered function point information comprises: one or more types of row coverage information, conditional coverage information, state machine coverage information, and hop coverage information.
3. The method of claim 1 or 2, further comprising,
the coverage verification report includes first information including a test item name and a test item profile.
4. The method of claim 3,
matching with the identification information in the coverage rate file, extracting the test item profile and adding the test item profile into the coverage rate verification report.
5. The method according to claim 3 or 4,
and displaying the first information and the function point information in a table form.
6. The method of any of claims 1-5, wherein traversing the coverage file, searching for uncovered function points indicated by the coverage file, comprises:
traversing all modules in the coverage rate file, and identifying the module with the trip coverage rate not reaching 100%; traversing all the row coverage rate information of the module, and adding the information of the uncovered rows into the row coverage rate information.
7. The method of claim 6,
the information of the uncovered line includes the uncovered line ID, and the line number, code and uncovered reason involved therewith.
8. The method according to any one of claims 2 to 6,
and one page contains all the uncovered function point information of each type in the coverage rate verification report, wherein the coverage rate verification report comprises at least one page.
9. The method according to any one of claims 1-8, further comprising:
searching whether a homonymous report exists or not, and if so, backing up the homonymous report to form a backup report;
and in the process of generating the coverage rate verification report, for an uncovered function point, matching the function point with the function point in the backup report, and if the matching is successful, adding the uncovered reason of the corresponding function point in the backup report into the generated coverage rate verification report.
10. An apparatus for generating a coverage verification report, comprising a processor and a memory, the memory storing a computer program which, when executed by the processor, implements the method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110744712.4A CN113419959A (en) | 2021-07-01 | 2021-07-01 | Method and equipment for generating coverage rate verification report |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110744712.4A CN113419959A (en) | 2021-07-01 | 2021-07-01 | Method and equipment for generating coverage rate verification report |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113419959A true CN113419959A (en) | 2021-09-21 |
Family
ID=77719940
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110744712.4A Pending CN113419959A (en) | 2021-07-01 | 2021-07-01 | Method and equipment for generating coverage rate verification report |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113419959A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114003513A (en) * | 2021-12-30 | 2022-02-01 | 云账户技术(天津)有限公司 | Code coverage rate processing method and device, electronic equipment and readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8826202B1 (en) * | 2013-05-09 | 2014-09-02 | Taiwan Semiconductor Manufacturing Company, Ltd. | Reducing design verification time while maximizing system functional coverage |
| CN112035376A (en) * | 2020-11-05 | 2020-12-04 | 四川科道芯国智能技术股份有限公司 | Method, device, equipment and storage medium for generating coverage rate report |
| CN112613257A (en) * | 2020-12-30 | 2021-04-06 | 海光信息技术股份有限公司 | Verification method, verification device, electronic equipment and computer-readable storage medium |
| CN112860556A (en) * | 2021-02-18 | 2021-05-28 | 中国工商银行股份有限公司 | Coverage rate statistical method, coverage rate statistical device, computer system and readable storage medium |
-
2021
- 2021-07-01 CN CN202110744712.4A patent/CN113419959A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8826202B1 (en) * | 2013-05-09 | 2014-09-02 | Taiwan Semiconductor Manufacturing Company, Ltd. | Reducing design verification time while maximizing system functional coverage |
| CN112035376A (en) * | 2020-11-05 | 2020-12-04 | 四川科道芯国智能技术股份有限公司 | Method, device, equipment and storage medium for generating coverage rate report |
| CN112613257A (en) * | 2020-12-30 | 2021-04-06 | 海光信息技术股份有限公司 | Verification method, verification device, electronic equipment and computer-readable storage medium |
| CN112860556A (en) * | 2021-02-18 | 2021-05-28 | 中国工商银行股份有限公司 | Coverage rate statistical method, coverage rate statistical device, computer system and readable storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114003513A (en) * | 2021-12-30 | 2022-02-01 | 云账户技术(天津)有限公司 | Code coverage rate processing method and device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8230370B2 (en) | Circuit design assisting apparatus, computer-readable medium storing circuit design assisting program, and circuit design assisting method | |
| CN108897724B (en) | Function completion progress determining method and device | |
| KR101477287B1 (en) | Test module generating apparatus, test sequence generating apparatus, generating method, program and test apparatus | |
| US20080307006A1 (en) | File mutation method and system using file section information and mutation rules | |
| CN106550038B (en) | Data configuration diagnosis system and method of digital control system | |
| EP3185027B1 (en) | Information processing method and device and computer storage medium | |
| US20060259891A1 (en) | System and method of generating an auto-wiring script | |
| US20120266121A1 (en) | Method to determine high level power distribution and interface problems in complex integrated circuits | |
| CN104834759B (en) | The implementation method and device of Electronic Design | |
| CN107451112B (en) | Form tool data checking method, device, terminal equipment and storage medium | |
| CN104809072A (en) | Automatic mensurability design method of Perl-based EDIF netlist-grade circuit automatic mensurability design system | |
| US6763360B2 (en) | Automated language and interface independent software testing tool | |
| CN100428153C (en) | Method and device for generating test script | |
| CN111400169A (en) | Method and system for automatically generating netlist file for testing software and hardware | |
| CN113419959A (en) | Method and equipment for generating coverage rate verification report | |
| CN117785723A (en) | Dynamic interface parameter association method and device and electronic equipment | |
| CN112463596B (en) | Test case data processing method, device and equipment and processing equipment | |
| CN112567375A (en) | Format verification method, information identification method, device and storage medium | |
| CN111124541A (en) | Configuration file generation method, device, equipment and medium | |
| JP3105279B2 (en) | Program unit test data generation method | |
| CN108334313A (en) | Continuous integrating method, apparatus and code management system for large-scale SOC research and development | |
| JP5163172B2 (en) | Software test item editing support apparatus and software test item editing support method | |
| CN106909511A (en) | A kind of automated testing method based on RedwoodHQ | |
| CN112597040A (en) | Interface automatic testing method and device and electronic equipment | |
| CN113419739B (en) | Node map difference detection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210921 |
|
| RJ01 | Rejection of invention patent application after publication |