JP5207314B2 - Test pattern compression method and test pattern compression system - Google Patents

Description

  The present invention relates to a test pattern compression method and a test pattern compression system for comprehensively testing a program without waste.

  In system LSI design, a test of a program executed in an upstream process is important in order to prevent a design error in required specifications or a function level from being mixed in a downstream process. The programs targeted by this test include not only programs that describe functions to be implemented as LSI hardware, but also embedded programs that operate in cooperation with LSI. In order to perform an efficient test, a simulation pattern (hereinafter referred to as a test pattern) that requires little waste and realizes a high coverage rate is required. That is, there is a demand for higher quality test patterns.

  The progress of verification is usually evaluated based on several comprehensive indexes. Ideally, it is desirable to achieve a high verification coverage even in the upstream process. However, in reality, the limitation of verification resources and the lack of methodology are the bottleneck, and the test of the upstream process tends to be more difficult than the downstream process. Therefore, a method for realizing high-quality verification at a low cost is indispensable to solve the verification omission in the upstream process.

  As a method for solving the above problem, a method for automatically generating a test pattern of a program by applying model checking which is one of formal methods has been proposed (Non-Patent Documents 1, 2, and 3). These convert a sequential processing program (C language or the like) into an intermediate representation suitable for analysis of a static control structure, and inspect the execution path of the program. If a specific execution path can be executed, a conditional pattern of the corresponding execution path can be formally solved to automatically generate a test pattern that increases the branch coverage rate as much as possible.

  The branch coverage is a coverage index that is executed at least once according to a test pattern in all conditional branches in a program, when true or false. It is defined as the ratio of the selected branch to the total number. Branch coverage is widely used not only in the field of software testing, but also in hardware functional verification.

Ball T., “Abstraction-guided Test Generation: A Case Study,” Technical Report MSR-TR-2003-86, Microsoft Research, 2003. D. Beyer, A.J.Chlipala, and R. Majumdar, “Generating Tests from Counterexample,” ICSE '04: Proceedings of the 26th International Conference on Software Engineering, pp.326-335, 2004. K. Sen, D. Marinov, and G. Agha, “CUTE: aconcolic unit testing engine for C,” ESEC / FSE-13: Proceedings of the 10th European software engineering conference, pp.263-272, 2005. G.C. Necula, S. Mcpeak, S.P. Rahul, and W. Weimer, “CIL: Intermediate language and tools for analysis and transformation of C programs,” In International Conference on Compiler Construction, pp.213-228, 2002. Tsutomu Hagio, “Logic Design-Switching Circuit Theory”, Modern Science, 2005.

  However, there is a problem that the test pattern generated by the method includes a redundant pattern. For the sake of explanation, FIG. 9 shows the result of generating the test pattern of the program shown in FIG. 8 using the method of Non-Patent Document 3. In Non-Patent Document 3, a restricted vertical search (Bounded Depth-First Search: hereinafter referred to as DFS) is used as a search method for searching for an unselected execution path.

  FIG. 9 shows the Depth value (hereinafter referred to as D) for limiting the DFS search depth on the horizontal axis, the test pattern generation count Iterations (hereinafter referred to as I), and the branch coverage rate on the vertical axis. Coverage (hereinafter referred to as BC) is shown.

  From FIG. 9, it can be seen that I is smaller as D is smaller, but BC is less than 100% when D <4. On the other hand, when D> 4, since BC is already saturated, a redundant test pattern is included in the generation result. Therefore, in this case, it is a good idea to generate a test pattern with D = 4 as a threshold value. When D = 4, I = 5, that is, five test patterns are generated. However, in this case, in order to set the branch coverage rate to 100%, 5 patterns are not necessary, and a minimum of 3 patterns is sufficient.

  The present invention has been made in consideration of the above-mentioned problems. The purpose of the present invention is to compress a redundant test pattern without compromising the branch coverage rate in order to efficiently test a program. The aim is to improve quality.

In order to achieve the above object, the invention according to claim 1 is a test pattern compression method for exhaustively testing a program without waste, and tests a program describing a function to be implemented as hardware as a system LSI. targeted program, a test pattern generating step of automatically generating by using a model checking the test pattern in the form approach to the test target program, branch control branches to extract the control branches to be branch coverage from the test target program An extraction step, and a coverage table configuration step that configures a coverage table in which the test pattern including the redundant pattern generated in the test pattern generation step is a row, and the branch control branch extracted in the branch control branch extraction step is a column. The singular column selection / deletion step for selectively deleting singular columns from the covering table. And a mandatory row selection / deletion step that saves the mandatory row with the row including the singular column of the coverage table as a mandatory row, and selectively deletes the mandatory row from the coverage table, and the remaining included in the mandatory row A mandatory row inclusion column selection / deletion step that deletes the columns of the covering table, and a set of columns covered by any of the remaining rows of the covering table includes a set of columns of the remaining other rows of the covering table, A controlled row selection / deletion step that selectively deletes other rows as controlled rows, and a row remaining in the covering table by the controlled row selection / deletion step is a set of rows covering a specific column. When a set of rows is included, a support array selection deletion step for selectively deleting the specific column as a support array, and a row remaining in the covering table, from the singular column selection deletion step to the support array selection deletion step. The row of the covering table disappears Comprising the steps repeated until the, and wherein the obtaining the required row as the final solution.
The invention according to claim 2 is a test pattern compression system for exhaustively testing a program without waste, and a program describing a function to be implemented as hardware as a system LSI is a test target program, and the test target program test pattern generation device, and a branch control branches extracting apparatus for extracting a control branches to be branch coverage from the test target program for automatically generating by using a model checking the test pattern in the form approach to the test pattern generation A covering table forming device for forming a covering table with a row of test patterns including redundant patterns generated by the device and having a branch control branch extracted by the branch control branch extracting device as a column; and a unique column from the covering table. Singular column selective deletion device for selective deletion, and a row including the singular column of the covering table as a mandatory row A mandatory row selection / deletion device that saves the mandatory row and selectively deletes the mandatory row from the covering table, a mandatory row inclusion column selection / deletion device that removes the remaining columns included in the mandatory row, When a set of columns covered by any of the remaining rows of the covering table includes a set of columns of the remaining other rows of the covering table, the selected row is selected and deleted as a controlled row. And the row remaining in the covering table by the deletion device and the controlled row selection / deletion device, when the set of rows covering a specific column includes a set of rows of another column, the specific column is arranged And a branch arrangement selection / deletion device that selectively deletes.

The invention according to claim 2 is a test pattern compression system for exhaustively testing a program without waste, a test pattern generation device that automatically generates a test pattern of a test target program,
A branch control branch extraction apparatus that extracts a control branch that is a target of the branch coverage from the test target program, and a branch control branch extracted by the branch control branch extraction apparatus using the test pattern generated by the test pattern generation apparatus as a row. A mandatory table, a mandatory table selecting / deleting device for selectively deleting a unique column from the coverage table, and a row including the unique column of the coverage table as an essential row. A mandatory row selection / deletion device that selectively deletes the mandatory row from the covering table, a mandatory row inclusion column selection / deletion device that deletes the remaining columns included in the mandatory row, and a remaining portion of the covering table. When a set of columns covered by any row includes a set of columns of the remaining other rows of the covering table, the controlled row selection / deletion device that selectively deletes the other row as a controlled row; By controlled row selection and deletion device A row arrangement selection / deletion device that selectively deletes the specific column as a branch arrangement when a set of rows covering a specific column includes a set of rows of another column for the rows remaining in the coverage table; It is characterized by providing.

  According to the present invention, after the test pattern is generated, the redundant test pattern is removed and an attempt is made to minimize. Therefore, if the test pattern is generated using the threshold value, the branch coverage rate is guaranteed. However, there is an effect that the number of test patterns can be reduced. As long as the algorithm on the test pattern generation side cannot completely eliminate redundancy, the compression effect according to the present invention can be expected. Even if the test pattern generation side algorithm is improved, it is possible to obtain the same number of candidate patterns as a result from a wider search range because the effect of reducing the number of redundant patterns is obtained. By applying the present invention to them, further improvement in quality can be expected. The present invention can be applied to general sequential processing programs regardless of the programming language to be tested.

It is a block diagram of the system which implements the test pattern compression method of this invention. It is a figure which shows CFG whole drawing in a present Example, and CFG which paid its attention only to the branch control branch. It is a figure which shows the coating | coated table | surface in a present Example. It is a figure which shows the process sequence of the test pattern compression method of this invention. It is a figure which shows the coating | covering table which simplified FIG. 3 in a present Example. It is a figure which shows the coating | covering table which simplified FIG. 5 in a present Example. It is a figure which shows the coating | covering table which simplified FIG. 6 in a present Example. It is a figure which shows the program used for the operation example of FIG. It is a figure which shows the operation example of a test pattern production | generation apparatus.

  Embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram of a system that implements the test pattern compression method of the present invention. The system includes a test pattern generation device 1 that automatically generates a test pattern of a test target program, a branch control branch extraction device 2 that extracts a control branch that is subject to a branch coverage from the test target program, and a test pattern generation device 1. A cover table forming device 3 that forms a cover table having the test pattern generated in step 1 as a row and the branch control branch extracted by the branch control branch extracting device 2 as a column, and a unique column in the obtained cover table Singular column selection / deletion device 4 for selective deletion, mandatory row selection / deletion device 5 for selective deletion of essential rows in the covering table, mandatory row inclusion column selection / deletion device 6 for deleting inclusion columns of essential rows, and controlled A controlled row selection / deletion device 7 that selectively deletes rows and a support array selection / deletion device 8 that selectively deletes support sequences are configured. Reference numeral 9 denotes a signal line. In the present invention, these devices are used to compress (delete) the test pattern according to a predetermined processing procedure. Hereinafter, the program shown in FIG. 8 will be specifically described as a test target program.

  FIG. 2A shows a control flow graph (hereinafter, CFG) of the program of FIG. A set of n nodes (ellipses) is represented by V = {v1, v2,..., vn}, and a set of m directed branches (arrows) is represented by E = {e1, e2,. CFG = (V, E). V corresponds to each sentence of the program of FIG. 8, and E represents the flow of control.

  FIG. 2B shows a state where attention is paid only to the “if” sentence in the CFG of FIG. Branches e1, e2,..., E8 branching from the nodes 12, 15, 20, and 22 are called branch control branches. In this case, the total number of branch control branches is eight. The branch coverage rate is a coverage index that represents the ratio of the number of branch control branches selected by a certain test pattern to the total number of branch control branches. When simulating with an arbitrary number of test patterns, all of the branch control branches e1, e2,..., E8 need to be selected at least once to achieve 100% branch coverage. is there.

  The test pattern generation device 1 includes means for generating a test pattern that increases the branch coverage rate as much as possible. This can be realized by using known techniques such as Non-Patent Documents 1 to 3, for example. Further, the branch control branch extraction apparatus 2 that extracts the control branch that is the target of the branch coverage from the test target program can be easily realized by using a known technique of Non-Patent Document 4.

  However, as described above, there is a problem that a redundant pattern is generated only by these conventional techniques. In order to remove duplicate test patterns on the branch coverage rate, in the present invention, the test pattern generated by the test pattern generation device 1 is used as a row, and the branch control branches extracted by the branch control branch extraction device 2 are arranged in a row. The covering table forming device 3 that forms the covering table is used.

  FIG. 3 shows the branch control branches e1, e2,... In FIG. 2B with the test pattern generated when D = 5 is designated as the Depth value using the method of Non-Patent Document 3. , E8 are shown in a column and shown in the form of a table.

  When there is a cross in the intersection (i, j) of the table, the row i covers the column j, and the table in FIG. 3 is called a covering table. For example, tp1 covers the branch control branch of e1. tp2 covers the branch control branches of e2 and e4 (the same applies hereinafter). In the present invention, at this time, the minimum number of test patterns necessary to cover all branch control branches is desired.

  As described above, this problem can be formulated as a minimum covering problem for obtaining a set of the minimum number of rows covering all the columns. The minimum coverage problem can often be expected to be solved in a practical time using the procedure in the latter half of the Quine-McCluskey (QM) method. The QM method can be easily realized by using a known technique as shown in Non-Patent Document 5, for example.

  In the present invention, the knowledge of the QM method is applied to devise a method of compressing (deleting) test patterns that are unnecessary for achieving the same branch coverage rate. This is called CRTP (Compression of Redundant Test Patterns) method. FIG. 4 shows a predetermined calculation procedure provided in the CRTP method.

  In the CRTP method, a test pattern is automatically generated using the test pattern generation device 1 in step S1 of FIG. 4, and a branch control branch is extracted from the CFG using the branch control branch extraction device 2 in step S2. In step S3, the cover table shown in FIG. Hereinafter, the processes of steps S4, S5, and S6 will be described in detail.

  In the covering table of FIG. 3, a column having only one cross mark at the intersection (i, j) is called a singular column. The singular row represents a unique branch control branch on the CFG. In this case, e1 and e4 are singular columns. A row including a cross column of a singular column is called a required row. An essential line means a test pattern that is indispensable for increasing the branch coverage rate. In this case, tp1 and tp2 correspond to essential rows. Using the singular column selection / deletion device 4 and the essential row selection / deletion device 5, after the check of the singular column and the essential row is completed, the essential rows tp1 and tp2 are stored in the minimum solution to be obtained, and the singular column and the essential row are deleted (step) S4, S5). Furthermore, the mandatory row inclusion column selection / deletion device 6 checks and deletes the columns contained by the mandatory rows (step S6). In this case, e2 is deleted. After completion of step S6, a simplified coverage table is shown in FIG.

  In FIG. 5, when the set of columns covered by row P includes the set of columns of row Q, row P dominates row Q. That is, when Q is a subset of P, Q is called a controlled row. In this case, the rows tp3, tp4, tp5, and tp6 are included in the row tp7. Therefore, the rows tp3, tp4, tp5, tp6 are checked using the controlled row deletion device 7 and deleted from the table of FIG. 5 (step S7).

  After the completion of step S7, a simplified coverage table is shown in FIG. In FIG. 6, when the set of rows covering column M includes the set of rows of column N, column M dominates column N. M is called a branch arrangement. When this is the case, the branch arrangement is checked and deleted using the branch arrangement selection / deletion device 8. In the present embodiment, the sets of rows covered by e3, e5, e6, e7, e8 are equal to each other. In such a case, all the columns can be interpreted as a branch arrangement, and therefore, deleting any column does not affect the result (step S8).

FIG. 7 shows a simplified coverage table after completion of step S8. FIG. 7 shows a case where the support arrays e5 to e8 in FIG. 6 are deleted. In FIG. 7, the procedures of steps S4 and S5 are repeated. The remaining e3 is a singular column, and therefore the row tp7 is an essential row. Using the singular column selection / deletion device 4 and the essential row selection / deletion device 5, the essential row tp7 is stored in the minimum solution to be obtained, and the essential row is deleted. Finally, the coverage table disappears and the minimum solution is TP = {tp1, tp2, tp7}
3 test patterns are obtained. This ends the procedure.

  At this time, it can be confirmed from FIG. 3 that all the branch control branches of e1, e2,..., E8 can be covered with the three test patterns of tp1, tp2, and tp7.

1: Test pattern generation device 2: Branch control branch extraction device 3: Cover table configuration device 4: Singular column selection deletion device 5: Mandatory row selection deletion device 6: Mandatory row inclusion column selection deletion device 7: Controlled row selection deletion device 8: Branch arrangement selection deletion device 9: Signal line

Claims (2)

A test pattern compression method for exhaustively testing a program,
A test pattern generation step for automatically generating a test pattern for the test target program by using a model check in a formal method as a program to be tested, which describes a hardware implementation function as a system LSI ;
A branch control branch extraction step for extracting a control branch subject to a branch coverage from the test target program;
A coverage table configuration step for configuring a coverage table with a row of the test pattern including the redundant pattern generated in the test pattern generation step and the branch control branch extracted in the branch control branch extraction step as a column;
A singular column selection and deletion step of selectively deleting singular columns from the covering table;
A mandatory row selection and deletion step of saving the mandatory row as a mandatory row, including the singular column of the coverage table, and selectively deleting the mandatory row from the coverage table;
A mandatory row inclusion column selection deletion step of deleting the remaining columns included in the mandatory row;
When a set of columns covered by any of the remaining rows of the covering table includes a set of columns of the remaining other rows of the covering table, the controlled row that selectively deletes the other row as a controlled row A selective deletion step;
For a row remaining in the covering table by the controlled row selection / deletion step, when a set of rows covering a specific column includes a set of rows of another column, the specific column is selectively deleted as a supporting array. A branch arrangement selection deletion step;
Repeating the steps from the singular column selection and deletion step to the branch arrangement selection and deletion step until the rows of the coverage table disappear for the rows remaining in the coverage table;
A test pattern compression method comprising obtaining the essential row as a final solution. A test pattern compression system for comprehensively testing programs without waste,
A test pattern generation device for automatically generating a test pattern for a test target program as a test target program by using a model check in a formal method as a program to be described as a system LSI hardware function ;
A branch control branch extraction device that extracts a control branch subject to a branch coverage from the test target program;
A coverage table constituting device that configures a coverage table in which the test pattern including the redundant pattern generated by the test pattern generation device is a row and the branch control branch extracted by the branch control branch extraction device is a column;
A singular column selective deletion device for selectively deleting a singular column from the covering table;
A mandatory row selection / deletion device that saves the mandatory row as a mandatory row, including the singular column of the coverage table, and selectively deletes the mandatory row from the coverage table;
A mandatory row inclusion column selective deletion device that deletes the remaining columns contained in the mandatory row;
When a set of columns covered by any of the remaining rows of the covering table includes a set of columns of the remaining other rows of the covering table, the controlled row that selectively deletes the other row as a controlled row A selective deletion device;
For a row remaining in the covering table by the controlled row selection / deletion device, when a set of rows covering a specific column includes a set of rows of another column, the specific column is selectively deleted as a supporting array. A support array selective deletion device;
A test pattern compression system comprising:

Images