JP5444939B2 - Software testing method and program - Google Patents

Description

  The present invention relates to a software test method and program.

  An integrated circuit mounted on an electronic device is generally designed using a hardware description language represented by Verilog-HDL or VHDL (VHSIC Hardware Description Language).

  An integrated circuit designed using a hardware description language is logically verified to perform an operation according to the specification in the state of a source code described in the hardware description language. Specifically, the verification worker creates a test case including input data and an expected output value for the input data from the specification. The verification device performs logic verification of the source code for the test case, and determines “good” or “bad”.

  That is, the verification apparatus determines whether or not the output data output by processing the test case input data by software is equal to the expected output value of the test case. The verification device determines “good” when the output data and the output expected value are equal, and conversely, the verification device determines “bad” when the output data and the output expected value are not equal.

  Then, the verification operator corrects (debugs) the source code based on the test case that is determined to be “defective”. The verification device performs logic verification on the corrected source code for the test case, and determines “good” or “bad”. Thereafter, the logic verification of the source code is performed until all the test cases are judged as “good”. When all the test cases are judged as “good”, the logic verification of the source code is completed assuming that the source code is correctly described according to the specification.

  In recent years, functions required for electronic devices have become complicated and multifunctional, and accordingly, functions of integrated circuits have also become complicated and multifunctional. As a result, in the logic verification of the source code of the integrated circuit, the number of test cases increases dramatically, and the time and cost for performing logic verification for all test cases and the time and cost required for debugging are enormous. It has become.

  Therefore, conventionally, the verification operator narrows down the test cases where the logic verification of the source code has been performed by the factor analysis table and the orthogonal table according to the experiment design method, and performs the logic verification of the source code for a small number of test cases. It was. As a result, only by performing logic verification of the source code for a small number of test cases, an effect equivalent to the effect of performing logic verification of the source code so far was obtained (for example, see Patent Document 1).

JP 2002-259463 A

  However, in the logic verification of the above source code, if the logic verification of the source code is performed for the test case and it is determined as “bad”, the logic verification of the corrected source code is performed in the same order for all the test cases. It was.

  Therefore, the logic verification of the source code is performed at the first stage for the test case that is determined to be “good” once by performing the logic verification of the source code. However, the test code that has been determined to be “good” once for the test cases that have been determined to be “good” from the ratio that is determined to be “bad” by performing logic verification of the source code for test cases that have not been subjected to logic verification of the source code. The rate at which logic verification is carried out to determine “bad” is low. For this reason, there is room for further shortening the time for performing the logic verification of the source code by considering the order of the test cases for performing the logic verification of the source code.

  The purpose of this software testing method and program is to shorten the test time.

According to one aspect of the present invention, testing the software, and input data, and the test case creation step of creating multiple test cases with an expected output value for the input data, based on the quantitative information and qualitative information for a category of classification of the test case, weighting ratio indicating the possibility that test results classified the test case is determined to be defective is based on the weighting ratio for each category of the set weighted tables respectively a category selection step of selecting the category to test the software, and software test step for selected the software test for classified the test case to the categories in the category selection process, the software test based on the bets result, the software te Calculating the weighting ratio of the weighting table for the category subjected to preparative, wherein the table change step of changing the weighting table based on the calculation result of the weighting ratio, one of the selected from a plurality of the test cases an output expected value of the test case, have a, a determination step of determining whether or not the output data outputted by processing is equal to the input data of the test cases selected by the software, the weighting table A predetermined reference weighting ratio, an upper limit value and a lower limit value of the reference weighting ratio, and a statistical weighting ratio calculated based on a test result of the software, and the category selection step includes the statistical weighting ratio When the ratio is not less than the lower limit and not more than the upper limit of the reference weight ratio, the reference weight Select the rate as the weighting ratio, the case statistical weighting ratio is higher than the lower or upper limit value than the lower limit of the reference weighting ratio, it selects the statistical weighting ratio as the weighting ratio.

  According to one aspect of the present invention, a test method and program for software can reduce test time.

It is a schematic block diagram of a verification apparatus. It is explanatory drawing of the logic verification of a source code. It is explanatory drawing of a factor analysis table. It is explanatory drawing of an orthogonal table. It is explanatory drawing of an orthogonal table. It is explanatory drawing of a test case file. It is explanatory drawing of a category weighting table. It is explanatory drawing of the logic verification of a source code.

Hereinafter, embodiments will be described with reference to FIGS.
FIG. 1 is a schematic configuration diagram of a computer system for performing logic verification of a source code of an integrated circuit described in a hardware description language.

  As shown in FIG. 1, the computer (verification device) 11 includes a central processing unit (hereinafter referred to as CPU) 12, a memory 13, a storage device 14, a display device 15, an input device 16, and a drive device 17. They exchange data with each other via a bus 18.

  The CPU 12 executes a program using the memory 13 and realizes processing necessary for logical verification of the source code. The memory 13 stores programs and data necessary for providing various processes. The memory 13 typically includes a cache memory, a system memory, and a display memory.

  The display device 15 is used for displaying various windows and data, and for this, a CRT, LCD, PDP, or the like is used. The input device 16 is used to input a request, an instruction, and a parameter from the user, and a keyboard and a mouse device (not shown) are used for this.

  The storage device 14 usually includes a magnetic disk device, an optical disk device, and a magneto-optical disk device. The storage device 14 stores program data and files for performing logic verification of the source code. As files, specifications 20, source code 21, factor analysis table 22, orthogonal table 23, test case file 24, category weighting table 25, and other files necessary for logical verification of source code are stored.

The specification 20 is data defined for the operation of the integrated circuit to be verified. The source code 21 is data described in a hardware description language based on the specification 20.
The factor analysis table 22 is data of a two-dimensional table of factors that are parameters when performing logical verification of the source code 21 and a level that is a value of the factor. In the present embodiment, as shown in FIG. 3, the factor analysis table 22 includes factors F1 to Fn extracted from the specification 20 and levels L1 to Ln.

  Further, in the factor analysis table 22, the levels L1 to Ln are classified into categories C1 to Cn based on quantitative information such as maximum values, minimum values, equivalence classes, boundary values, and the like. Here, the maximum value means the maximum input value at which the integrated circuit to be verified can operate normally. The minimum value is the minimum input value at which the integrated circuit to be verified can operate normally. Further, the equivalence class refers to a set of input values from which a similar output result can be obtained from the integrated circuit to be verified. Furthermore, the boundary value refers to a representative value near the boundary of each equivalence class.

  The orthogonal table 23 is data of an allocation table having a property that, for any two factors, all combinations of the levels appear the same number of times. Based on the orthogonal table 23, the verification operator narrows down all combinations of factors and levels to specific factor and level combinations. Then, the verification operator can obtain the same effect as when verification is performed for all combinations of factors and levels by verifying the combinations of specific factors and levels narrowed down based on the orthogonal table 23. .

  For example, as shown in Table 30 of FIG. 4, when two levels (0, 1) of factors A, B, and C are extracted from the specification 20, all combinations of factors and levels are “A1” to “A8”. There are 8 combinations. Therefore, the verification operator performs logical verification of the source code 21 for the eight input data created from the eight combinations “A1” to “A8”. However, by using an orthogonal table 31 called “L4” as shown in FIG. 5, the combinations of factors and levels can be narrowed down to four combinations of “B1” to “B4”.

  In this case, the orthogonal table 31 shows all combinations of levels (0, 1) between any two factors among the factors A, B, and C. Therefore, the verification operator simply performs logic verification of the source code 21 for the four input data created from the combinations “B1” to “B4”, and the eight inputs created from all combinations of factors and levels. It is possible to obtain the same effect as when the logic verification of the source code 21 is performed on the data.

  As shown in FIG. 6, the test case file 24 stores test cases Ta1 to Taj,..., Tn1 to Tnj including input data created based on the orthogonal table 23 and output expected values for the input data. ing.

  Test cases Ta1 to Taj,..., Tn1 to Tnj are classified into categories C1 to Cn, respectively. The CPU 12 reads the classified test cases Ta1 to Taj,..., Tn1 to Tnj from the test case file 24 in the order of category C1, category C2,.

  Further, in the test cases Ta1 to Taj,..., Tn1 to Tnj, the verification execution order P from “No. 1” to “No. j” is set for each category C1 to Cn. The CPU 12 reads the corresponding test cases Ta1 to Taj,..., Tn1 to Tnj from the test case file 24 in the order of “1st” to “jth” in the verification execution order P in each category C1 to Cn. It has become.

  That is, the CPU 12 classifies the test cases Ta1 to Taj classified into the category C1 as the test case Ta1 with the verification execution order P of “No. 1”, the test case Ta2 with the verification execution order P of “No. 2”,. Read in the order of the test cases Taj whose execution order P is “No. n”. Next, the CPU 12 categorizes the test cases Tb1 to Tbj classified in the category C2 into the test case Tb1 with the verification execution order P being “No. 1”, the test case Tb2 with the verification execution order P being “No. 2”,. The test execution order P is read in the order of the test case Tbj with the “n-th” execution order P.

  Then, for the test cases Tc1 to Tcj,..., And the test cases Tn1 to Tnj classified into the categories C3 to Cn, the CPU 12 follows the categories C1 and C2 to test cases Tc1,. The test cases Tn1,..., Tnj are read in this order.

  The category weighting table 25 is a table for determining which category C1 to Cn of the test cases Ta1 to Taj,..., Tn1 to Tnj stored in the test case file 24 is subjected to the logic verification of the source code 21. is there.

  As shown in FIG. 7, the category weighting table 25 has statistical weighting ratios RS1 to RSn, reference weighting ratios RK1 to RKn, upper limit values U1 to Un, and lower limit values LO1 to LOn for each category C1 to Cn. Is set. The higher the weighting ratio, the higher the possibility that the logical verification of the source code 21 is performed and the determination is “bad”, and the lower the value is, the logical verification of the source code 21 is performed and “bad” is determined. The possibility of being judged is low.

  The statistical weighting ratios RS1 to RSn are the numbers of test cases Ta1 to Taj,..., Tn1 to Tnj, in which the CPU 12 performs logic verification of the source code 21 and is determined to be “bad” in each category C1 to Cn. Is the ratio of the number of test cases Ta1 to Taj,..., Tn1 to Tnj in which the logic verification of the source code 21 is performed.

  That is, the statistical weight ratios RS1 to RSn are determined as “bad” by performing logic verification of the source code 21 for the test cases Ta1 to Taj,..., Tn1 to Tnj classified into the corresponding categories C1 to Cn by the CPU 12. It shows the possibility. Note that a category in which the logical verification of the source code 21 is not performed for the classified test cases Ta1 to Taj,..., Tn1 to Tnj cannot calculate the corresponding statistical weighting ratios RS1 to RSn. RS1 to RSn are not set.

  The reference weighting ratios RK1 to RKn are determined as “bad” by performing logic verification of the source code 21 for the test cases Ta1 to Taj,..., Tn1 to Tnj classified into the corresponding categories C1 to Cn by the CPU 12. The ratio is estimated by the verification operator in advance.

  Incidentally, the reference weight ratios RK1 to RKn are higher in the order of the reference weight ratio RK1, the reference weight ratio RK2,..., And the reference weight ratio RKn. In other words, when the verification operator performs logic verification of the source code 21 for the test cases Ta1 to Taj,..., Tn1 to Tnj, the category C1 corresponding to the reference weighting ratio RK1 and the category C2 corresponding to the reference weighting ratio RK2. ,... It is predicted that the classified test cases Ta1 to Taj,..., Tn1 to Tnj are likely to be determined as “bad” in the order of the category Cn corresponding to the reference weighting ratio RKn.

CPU12 selects either statistical weight ratio RS1-RSn or reference | standard weight ratio RK1-RKn as a category weight ratio in each category C1-Cn.
More specifically, in each category C1 to Cn, when the statistical weight ratios RS1 to RSn are lower than the lower limit values LO1 to LOn (<reference weight ratios RS1 to RSn), the CPU 12 sets the statistical weight ratios RS1 to RSn as the category weight ratios. select. In each category C1 to Cn, the statistical weight ratios RS1 to RSn are not less than the lower limit values LO1 to LOn (<reference weighting ratios RS1 to RSn) and not more than the upper limit values U1 to Un (> reference weighting ratios RS1 to RSn). In this case, the CPU 12 selects the reference weighting ratios RS1 to RSn as the category weighting ratio. Furthermore, in each category C1-Cn, when the statistical weight ratios RS1-RSn are higher than the upper limit values U1-Un (> reference weight ratios RS1-RSn), the CPU 12 selects the statistical weight ratios RS1-RSn as the category weight ratios. .

  That is, as a result of logical verification of the source code 21 for the test cases Ta1 to Taj,..., Tn1 to Tnj, the CPU 12 is in the vicinity of the predicted reference weighting ratio RS1 to RSn (from the upper limit value U1 to Un to the lower limit value LO1 to LOn ) Includes the statistical weight ratios RS1 to RSn, the reference weight ratios RS1 to RSn are selected as the category weight ratios. On the other hand, the CPU 12 performs logic verification of the source code 21 for the test cases Ta1 to Taj,... ), If the statistical weight ratios RS1 to RSn are not included, the statistical weight ratios RS1 to RSn are respectively selected as the category weight ratios.

  Therefore, the CPU 12 performs logic verification of the source code 21 from the test case file 24 and classifies the test cases Ta1 to Taj,... In the order of categories C1 to Cn that are likely to be determined as “bad”. Tn1 to Tnj are read out.

  The CPU 12 is stored in the storage device 14 in response to an instruction from the input device 16. The CPU 12 transfers the program data and file data to the memory 13 and executes them.

  The drive device 17 drives the recording medium 19 and accesses the stored contents. The CPU 12 reads program data from the recording medium 19 via the drive device 17 and stores it in the storage device 14.

  As the recording medium 19, any computer-readable recording such as magnetic tape (MT), memory card, flexible disk, optical disk (CD-ROM, DVD-ROM,...), Magneto-optical disk (MO, MD,...), Etc. Media can be used. The above-described program data can be stored in the recording medium 19 and loaded into the memory 13 for use as necessary.

  The recording medium 19 includes a medium and a disk device that record program data uploaded or downloaded via a communication medium. Furthermore, not only a recording medium that records a program that can be directly executed by a computer, but also a recording medium that records a program that can be executed once installed on another recording medium (such as a hard disk), or an encrypted program In addition, a recording medium on which a compressed program is recorded is also included.

  Next, processing for performing logic verification of the source code 21 described in a hardware description language such as Verilog-HDL or VHDL based on the specification 20 using the verification device 11 will be described with reference to FIG.

  First, the verification operator operates the input device 16 and creates the factor analysis table 22 shown in FIG. 3 from the specification 20 (step S1). That is, the verification operator extracts from the specification 20 parameters that affect the operation of the integrated circuit for which the logic verification of the source code 21 is to be performed, and what values the parameters have. Further, the verification operator classifies the extracted levels L1 to Ln into categories C1 to Cn based on quantitative information such as the maximum value, the minimum value, the equivalence class, and the boundary value. Next, the verification operator describes in the factor analysis table 22 which category C1 to Cn the extracted levels L1 to Ln are classified into. Then, the verification operator stores the created factor analysis table 22 in the storage device 14.

Then, the verification operator operates the input device 16 to create the orthogonal table 23 based on the factor analysis table 22 created in step S1 and stored in the storage device 14 (step S2).
That is, if input data is created for all combinations of the factors F1 to Fn and the levels L1 to Ln in the factor analysis table 22, the number becomes enormous. For this reason, if the logic verification of the source code 21 is performed on the enormous number of input data created, the verification time of the logic verification of the source code 21 increases. Therefore, the orthogonal table 23 is used to narrow down the combinations of the factors F1 to Fn and the levels L1 to Ln, and the number of input data for performing logic verification of the source code 21 is reduced. Then, the verification operator stores the created orthogonal table 23 in the storage device 14.

  Next, the verification operator operates the input device 16 to generate a test case based on the orthogonal table 23 generated in step S2 (step S3). That is, first, the verification operator operates the input device 16 and reads the orthogonal table 23 from the storage device 14. Next, the verification worker creates input data from combinations of factors and levels in the orthogonal table 23 and calculates an expected output value for the input data. Subsequently, the verification worker creates a test case including the input data and an output expected value for the input data.

  Further, the verification operator classifies the created test cases into categories C1 to Cn in which the levels included in the corresponding input data are classified. Specifically, the verification worker classifies the test cases Ta1 to Taj into the category C1, classifies the test cases Tb1 to Tbj into the category C2, and..., Categorizes the test cases Tn1 to Tnj into the category Cn. Classify.

  Next, the verification worker stores the test cases Ta1 to Taj, Tb1 to Tbj,..., Tn1 to Tnj classified into the categories C1 to Cn in the test case file 24 in the order shown in FIG. .

  Then, the CPU 12 performs the logical verification of the source code 21 for the test cases Ta1 to Taj,..., Tn1 to Tnj stored in the test case file 24 in step S3 based on the category weighting table 25 shown in FIG. (Verification execution category) is set (step S4, category selection step).

  At this time, since the logical verification of the source code 21 is not performed for the test cases Ta1 to Taj,..., Tn1 to Tnj, the statistical weighting ratios RS1 to RSn are not set. Thereby, CPU12 selects reference | standard weighting ratio RS1-RSn as a category weighting ratio in each category C1-Cn, respectively.

  Incidentally, when the logic verification of the source code 21 is performed for the test cases Ta1 to Taj,..., Tn1 to Tnj, the CPU 12 uses the reference weighting ratios RS1 to RSn or the statistics in the categories C1 to Cn that are the verification execution categories. Any one of the weighting ratios RS1 to RSn is selected as the category weighting ratio.

  Subsequently, the CPU 12 sets the categories C1 to Cn having the highest category weighting ratio in the category weighting table 25 as verification execution categories. That is, the CPU 12 performs the logic verification of the source code 21 for the classified test cases Ta1 to Taj,..., Tn1 to Tnj, and determines the category C1 to Cn that is most likely to be determined as “bad” as the verification execution category. It is set as.

  Next, the CPU 12 reads the test cases Ta1 to Taj,..., Tn1 to Tnj of the test case file 24 classified into the categories C1 to Cn, which are the verification execution categories, in descending order of the verification execution order P.

  At this time, the CPU 12 sets the category C1 having the highest category weighting ratio in the category weighting table 25 as the verification execution category. Next, the CPU 12 reads the test cases Ta1 to Taj of the test case file 24 classified into the category C1 that has become the verification execution category in the order of the high verification execution order P.

  The CPU 12 then provides source code for the test cases Ta1 to Taj,..., Tn1 to Tnj (in this case, test cases Ta1 to Taj classified in the category C1) classified into the categories C1 to Cn of the read verification execution categories. 21 is verified (step S5, software test process).

  Accordingly, the CPU 12 performs the logic verification of the source code 21 first from the test cases Ta1 to Taj,..., Tn1 to Tnj that are most likely to be determined as “bad” by performing the logic verification of the source code 21. Thus, test cases Ta1 to Taj,..., Tn1 to Tnj that are determined as “bad” with a small number of verifications are detected.

  As a result, the CPU 12 postpones the logic verification of the source code 21 for the test cases Ta1 to Taj,..., Tn1 to Tnj that are likely to be determined as “good”, and the test cases Ta1 to Taj,. ... Tn1 to Tnj need not be logically verified for the source code 21 that is not correctly described as in the specification 20. Accordingly, the CPU 12 can reduce the number of verifications of the logic verification of the source code 21 that is not correctly described as in the specification 20.

  Then, when the CPU 12 performs logical verification of the source code 21 for the test cases Ta1 to Taj,..., Tn1 to Tnj classified into the categories C1 to Cn of the verification execution category, the verification result is reflected in the category weighting table 25. (Step S6, table changing step).

  That is, the CPU 12 obtains the result of performing the logic verification of the source code 21 for the test cases Ta1 to Taj,. Reflected in the statistical weighting ratios RS1 to RSn for Cn.

  Specifically, the CPU 12 performs logic verification of the source code 21 on the test cases Ta1 to Taj,..., Tn1 to Tnj classified into the verification execution categories C1 to Cn, and is determined to be “bad”. When there are no test cases Ta1 to Taj,..., Tn1 to Tnj, the statistical weight ratios RS1 to RSn for the verification execution categories C1 to Cn are set to “0%”.

  On the other hand, the CPU 12 performs the logic verification of the source code 21 for the test cases Ta1 to Taj,..., Tn1 to Tnj classified into the categories C1 to Cn of the verification execution category, and as a result, is determined as “bad”. When there are Ta1 to Taj,..., Tn1 to Tnj, statistical weighting for the categories C1 to Cn of the verification execution category is performed according to the number of test cases Ta1 to Taj,. Increase or decrease the ratios RS1 to RSn.

  Specifically, as described above, the statistical weighting ratios RS1 to RSn are the test cases Ta1 to Taj,..., In which the CPU 12 performs logic verification of the source code 21 and is determined as “bad” in each category C1 to Cn. It is a ratio between the number of Tn1 to Tnj and the number of test cases Ta1 to Taj,..., Tn1 to Tnj in which the CPU 12 performs logic verification of the source code 21.

  Therefore, the CPU 12 increases the statistical weighting ratios RS1 to RSn for the categories C1 to Cn of the verification execution category as the number of test cases Ta1 to Taj,..., Tn1 to Tnj determined as “defective” increases. Conversely, the CPU 12 decreases the statistical weighting ratios RS1 to RSn for the categories C1 to Cn of the verification execution category as the number of test cases Ta1 to Taj,.

  At this time, the CPU 12 reflects the result of logical verification of the source code 21 for the test cases Ta1 to Taj classified in the category C1 that is the verification execution category in the statistical weight ratio RS1 for the category C1.

  Then, as a result of logical verification of the source code 21 for the test cases Ta1 to Taj,..., Tn1 to Tnj classified into the verification execution categories C1 to Cn in step S5, the CPU 12 is determined to be “bad”. It is detected whether there are test cases Ta1 to Taj,..., Tn1 to Tnj (step S7).

  When the CPU 12 detects the test cases Ta1 to Taj,..., Tn1 to Tnj determined to be “defective” (YES in step S7), the verification operator determines that the test cases Ta1 to Ta1 determined to be “bad”. The cause of the failure is determined from Taj,..., Tn1 to Tnj. Next, the verification operator determines whether or not the cause of the failure is due to the fact that the logic described in the source code 21 is not described as in the specification 20 (logic failure) (step S8). As a cause of failure other than a logical failure, there is an error in the contents of input data and output expected values of test cases Ta1 to Taj,..., Tn1 to Tnj, or an error in the contents of specification 20. There is.

  That is, the verification operator determines whether the cause of the failure is due to a description error in the source code 21 based on the test cases Ta1 to Taj,..., Tn1 to Tnj determined as “bad” or the test cases Ta1 to Taj. ,..., Tn1 to Tnj and the specification 20 are determined to be due to an error. In other words, based on the test cases Ta1 to Taj,..., Tn1 to Tnj determined as “defective”, the verification operator can specify the specification 20, the source code 21, or the test cases Ta1 to Taj,. It is determined which description or content of Tn1 to Tnj is to be corrected.

  If it is determined that the cause of failure is due to a logical failure (YES in step S8), the verification operator operates the input device 16 to read the source code 21 from the storage device 14 and there is an error in the description of the source code 21. The location is corrected (step S9).

  Subsequently, when the modified source code 21 is stored in the storage device 14, the CPU 12 returns to step S4, and again sets the categories C1 to Cn as the verification execution categories having the highest category weighting ratio based on the category weighting table 25. To do.

  At this time, the CPU 12 reflects the result of the logic verification of the source code 21 for the test cases Ta1 to Taj classified in the category C1 in the statistical weighting ratio RS1 for the category C1 in the category weighting table 25. In the category C1, the CPU 12 compares the statistical weighting ratio RS1 and the reference weighting ratio RK1, and selects either the statistical weighting ratio RS1 or the reference weighting ratio RK1 as the category weighting ratio according to the comparison result.

  The CPU 12 selects the reference weighting ratio RK1 as the category weighting ratio when the statistical weighting ratio RS1 is not less than the upper limit value U1 of the reference weighting ratio RK1 or the statistical weighting ratio RS1 is not more than the lower limit values LO1 to LOn of the reference weighting ratio RK1. . In this case, since the CPU 12 has selected the reference weighting ratio RK1 in the category C1, it does not change to the category weighting ratios in the categories C1 to Cn.

  That is, as a result of logical verification of the source code 21 for the test cases Ta1 to Taj classified in the category C1, the CPU 12 has the statistical weighting ratio RS1 within the range from the upper limit value U1 to the lower limit value LO1 of the reference weighting ratio RK1. Since the verification worker predicted, the category weighting ratio of the category C1 as the reference weighting ratio RK1 is not changed. In this case, since there is no change in the verification execution category, the CPU 12 performs the logic verification of the source code 21 again for the test cases Ta1 to Taj classified in the category C1 (step S5).

  On the other hand, when the statistical weighting ratio RS1 is lower than the upper limit value U1 of the reference weighting ratio RK1 or the statistical weighting ratio RS1 is higher than the lower limit value LO1 of the reference weighting ratio RK1, the CPU 12 selects the statistical weighting ratio RS1 as the category weighting ratio. To do.

  That is, as a result of logical verification of the source code 21 for the test cases Ta1 to Taj classified in the category C1, the CPU 12 falls within the lower limit LO1 from the upper limit U1 of the reference weighting ratio RK1 predicted by the statistical weighting ratio RS1. Since there was not, statistical weight ratio RS1 is selected as a category weight ratio.

  Thereby, when the category weighting ratio of the category C1 is higher than the category weighting ratio of the category C2, since the category weighting ratio of the category C1 is the highest, the category C1 is set again as the verification execution category. In this case, since there is no change in the verification execution category, the CPU 12 performs the logic verification of the source code 21 again for the test cases Ta1 to Taj classified in the category C1 (step S5).

  In other words, the CPU 12 performs the logic verification of the source code 21 on the test cases Ta1 to Taj classified in the category C1 of the verification execution category, and as a result, even if the category weighting ratio of the category C1 increases, the category weighting of the category C1 Since the ratio is the highest, the verification execution category as the category C1 is maintained.

  On the contrary, when the category weighting ratio of category C1 is lower than the category weighting ratio of category C2, the category weighting ratio of category C2 is the highest, so CPU 12 sets category C2 as the verification execution category. In this case, the CPU 12 performs logic verification of the source code 21 for the test cases Tb1 to Tbj classified in the category C2 of the verification execution category (step S5).

  That is, as a result of logical verification of the source code 21 for the test cases Ta1 to Taj classified into the category C1 of the verification execution category, the category weighting ratio of the category C1 is lower than the category weighting ratio of the category C2 that is the second highest. Thus, the CPU 12 changes the verification execution category from the category C1 to the category C2.

  If it is determined that the cause of failure is not due to a logical failure based on the test cases Ta1 to Taj,..., Tn1 to Tnj determined as “defective” (NO in step S8), the verification operator returns to step S1. Then, the input device 16 is operated, and the specification 20 or the factor analysis table 22 is created again based on the cause of the failure.

  That is, the verification operator corrects the specification 20 or the factor analysis table 22 because there is an error in the contents of the specification 20 or the factor analysis table 22. The verification operator creates the orthogonal table 23 and the test cases Ta1 to Taj,..., Tn1 to Tnj again based on the specifications 20 and the factor analysis table 22, and newly created test cases Ta1 to Taj,. , Tn1 to Tnj are subjected to logic verification of the source code 21.

  When the test cases Ta1 to Taj,..., Tn1 to Tnj determined as “bad” are not detected (NO in step S7), the CPU 12 stores the test cases Ta1 to Taj,. , Tn1 to Tnj are all determined to be “good” (step S10). Then, when there is a test case that is not determined as “good” (YES in step S10), the CPU 12 sets again the categories C1 to Cn of the verification execution category having the highest category weighting ratio based on the category weighting table 25. (Step S4).

  At this time, the CPU 12 reflects the result of logical verification of the source code 21 for the test cases Ta1 to Taj classified in the category C1 in the statistical weighting ratio RS1 for the category C1 in the category weighting table 25.

  In this case, as a result of performing logic verification of the source code 21 for the test cases Ta1 to Taj classified into the category C1, it is determined that all the test cases Ta1 to Taj are “good”. The ratio RS1 is set to “0%”. As a result, the CPU 12 selects the statistical weighting ratio RS1 of “0%” as the category weighting ratio in the category C1.

  Therefore, the category weight ratio of category C1 is lower than the category weight ratio of category C2. As a result, the CPU 12 sets the verification execution category to the category C2 (step S4), and performs logical verification of the source code 21 for the test cases Tb1 to Tbj classified into the category C2 (step S5).

  In other words, the CPU 12 performs the logic verification of the source code 21 for the test cases Ta1 to Taj classified into the category C1, and there is no problem. Therefore, the test case Tb1 classified into the category C2 having the second highest category weighting ratio. The logic verification of the source code 21 is performed for .about.Tbj.

  That is, the CPU 12 repeats the processing in the order of step S4 → step S5 → step S6 → step S7 → step S10, and performs logical verification of the source code 21 for the test cases Ta1 to Taj,..., Tn1 to Tnj. Thereafter, until the CPU 12 detects the test cases Ta1 to Taj,..., Tn1 to Tnj determined as “bad” (YES in step S7), the order of the categories C1 to Cn having the first category weighting ratio is high. Thus, the logic verification of the source code 21 is performed for the test cases Ta1 to Taj,..., Tn1 to Tnj corresponding to the categories C1 to Cn.

  In this case, the CPU 12 uses the verification results obtained by performing the logic verification of the source code 21 so far as the categories C1 to C1 in which the test cases Ta1 to Taj,. This is reflected in the statistical weighting ratios RS1 to RSn for Cn (step S6).

  Subsequently, the CPU 12 sets again the categories C1 to Cn having the highest category weight ratio of the categories C1 to Cn in the category weighting table 25 as verification execution categories (step S4). That is, the CPU 12 has the category weighting ratio next to the categories C1 to Cn into which the test cases Ta1 to Taj,... One of the categories C1 to Cn is set as the verification execution category.

  Thereafter, the CPU 12 repeats the processing in the order of step S4 → step S5 → step S6 → step S7 → step S10, and performs logical verification of the source code 21 for the test cases Ta1 to Taj,..., Tn1 to Tnj. Thereafter, the CPU 12 detects the test cases Ta1 to Taj,..., Tn1 to Tnj determined to be “bad” in the order of the categories C1 to Cn having the highest category weighting ratio in the first stage. The logic verification of the source code 21 is performed for the test cases Ta1 to Taj,..., Tn1 to Tnj corresponding to Cn.

  When the test cases Ta1 to Taj,..., Tn1 to Tnj stored in the test case file 24 are all determined to be “good” (NO in step S10), the CPU 12 uses the source code as specified in the specification 20. Judging that the code 21 is correctly described, the logic verification of the source code 21 is completed.

  Waveforms 50 and 51 shown in FIG. 8 are generally called “bug curves”, and errors in the specification 20 to be detected, the source code 21, and the factor analysis table 22 are detected during the logic verification period of the source code 21. The cumulative number of detected cases is shown in the figure.

  In software verification such as logic verification of the source code 21, many bugs (in the present embodiment, errors in the specification 20, the source code 21, and the factor analysis table 22) are detected at an initial stage of verification. The cumulative number of detections tends to converge gradually. With such a bug curve, it is possible to determine that the quality of the software is stable when a software test is performed and the target cumulative detection number D in the vicinity where the cumulative detection number of bugs becomes gentle is reached.

  Therefore, in the present embodiment, when logic verification of the source code 21 is performed and all of the test cases Ta1 to Taj,..., Tn1 to Tnj are determined to be “good”, the specification 20, the source code 21, and the factors The cumulative number of detections in which an error in the analysis table 22 is detected reaches the target cumulative detection number D.

Conventionally, when the logic verification of the source code 21 is performed without changing the order of the test cases Ta1 to Taj,..., Tn1 to Tnj, the bug curve has a waveform 50 shown in FIG.
In this embodiment, every time the source code 21 is logically verified for the test cases Ta1 to Taj,..., Tn1 to Tnj, which are classified into the categories C1 to Cn that are the verification categories, the category of the category weighting table 25 is used. A verification category is set from the categories C1 to Cn based on the category weighting ratios for C1 to Cn.

  As a result, the CPU 12 can perform logic verification of the source code 21 for the test cases Ta1 to Taj,..., Tn1 to Tnj that are most likely to be determined as “bad” from time to time. It is possible to verify errors in the specification 20, the source code 21, and the factor analysis table 22 by the number of times.

  Accordingly, the logic verification of the source code 21 can be completed with a smaller number of verifications than in the prior art, and the target cumulative detection number D is reached earlier than in the conventional case as shown by the waveform 51 in FIG. The verification time of the logic verification can be shortened.

As described above, according to the present embodiment, the following effects can be obtained.
(1) For categories C1 to Cn into which test cases Ta1 to Taj,..., Tn1 to Tnj are classified, statistical weighting ratios RS1 to RSn, reference weighting ratios RK1 to RKn, upper limit values U1 to Un, lower limit values LO1 to LO1 A category weighting table 25 in which LOn is set is provided. In each category C1 to Cn, the CPU 12 determines the reference weighting ratio according to whether or not the statistical weighting ratios RS1 to RSn are in the range from the upper limit values U1 to Un of the reference weighting ratios RK1 to RKn to the lower limit values LO1 to LOn. Any one of RK1 to RKn or statistical weighting ratios RS1 to RSn is set as the category weighting ratio. Next, the CPU 12 sets the category weight ratios C1 to Cn having the highest category weight ratio as the verification execution categories. Subsequently, the CPU 12 performs logic verification of the source code 21 for the test cases Ta1 to Taj,...

  Therefore, since the CPU 12 can perform logical verification of the source code 21 for the test cases Ta1 to Tnj that are highly likely to be determined as defective, the test cases Ta1 to Taj that are determined as defective with a small number of verifications. , Tn1 to Tnj can be detected. Along with this, the verification time of the logic verification of the source code 21 can be shortened.

(2) Further, the CPU 12 reflects the result of the logic verification of the source code 21 in the statistical weighting ratio of the category weighting table 25.
Therefore, each time the CPU 12 performs the logic verification of the source code 21 for the test cases Ta1 to Taj,..., Tn1 to Tnj classified into the categories C1 to Cn that are the verification execution categories, the verification result is reflected. A verification execution category can be set based on the category weighting table 25. As a result, the CPU 12 can detect the test cases Ta1 to Taj,..., Tn1 to Tnj that are determined to be defective with a smaller number of verifications. Accordingly, the verification time of the logic verification of the source code 21 can be further shortened.

In addition, you may implement the said embodiment in the following aspects.
In the present embodiment, the logic verification of the source code of the integrated circuit described in the hardware description language has been embodied, but since there is no difference in the test method as long as it is software, the software to be embodied is not particularly limited. .

  In the present embodiment, the verification operator classifies the levels L1 to Ln into the categories C1 to Cn based on quantitative information such as the maximum value, the minimum value, the equivalence class, and the boundary value. Not limited to this, the factors F1 to Fn may be classified into categories C1 to Cn based on qualitative information.

  In the present embodiment, the CPU 12 has set the verification execution category to the categories C1 to Cn having the highest statistical weighting ratios RS1 to RSn in the category weighting table 25. However, the verification category may be set to a plurality of categories C1 to Cn having high statistical weight ratios RS1 to RSn in the category weighting table 25.

11 Computer 20 Design specifications (specifications)
21 Software (source code)
25 Weighting table (category weighting table)
C1 to Cn Category LO1 to LOn Lower limit value RK1 to RKn Standard weighting ratio RS1 to RSn Statistical weighting ratio Ta1 to Taj, ..., Tn1 to Tnj Test cases U1 to Un Upper limit value

Claims (4)

And input data, and the test case creation step of creating multiple test cases with an expected output value for the input data,
For a category of classification of the test case based on the quantitative information and the qualitative information, each of the classified the test weighting table which case the test results weighting ratio indicating the likelihood that is determined to be defective is set respectively based on the weighting ratio for a category, the category selection step of selecting the category to test the software,
A software test step of testing the software for the test cases classified in the category selected in the category selection step;
Based on the test results of the software to calculate the weighting ratio of the weighting table for the categories tested of the software, the table change step of changing the weighting table based on the calculation result of the weighting ratio ,
Determine whether output expected value of one test case selected from a plurality of test cases is equal to output data output by processing the input data of the test case selected by the software A determination step to
I have a,
The weighting table is
A preset reference weighting ratio, an upper limit value and a lower limit value of the reference weighting ratio, and a statistical weighting ratio calculated based on a test result of the software,
The category selection step includes:
When the statistical weighting ratio is not less than the lower limit value and not more than the upper limit value of the reference weighting ratio, the reference weighting ratio is selected as the weighting ratio,
The software testing method , wherein the statistical weighting ratio is selected as the weighting ratio when the statistical weighting ratio is lower than a lower limit value or higher than an upper limit value of the reference weighting ratio . Before Symbol category selection process,
2. The software testing method according to claim 1, wherein a category having the highest weighting ratio among the weighting ratios for each category of the weighting table is selected. Before Symbol software,
3. The software test method according to claim 1 , wherein the source code is a source code described in a programming language or a hardware description language. Create multiple test cases with the input data and the expected output value for the input data,
A reference weight ratio set in advance indicating the possibility that the test result of the classified test case is determined to be defective for the category in which the test case is classified based on quantitative information and qualitative information; and the reference weight When the statistical weighting ratio of the weighting table having the upper limit value and the lower limit value of the ratio and the statistical weighting ratio calculated based on the test result of the software is not less than the lower limit value and not more than the upper limit value of the reference weighting ratio, the reference Select a weighting ratio as the weighting ratio,
When the statistical weighting ratio of the weighting table is lower than the lower limit value or higher than the upper limit value of the reference weighting ratio, the statistical weighting ratio is selected as the weighting ratio,
There line testing of the software for the test case that has been classified as selected has been the category,
Based on the test results of the previous SL software, it calculates the statistical weighting ratio of the weighting table for the categories tested of the software,
Changing the weighting table based on the calculation result of the statistical weighting ratio ;
Determine whether output expected value of one test case selected from a plurality of test cases is equal to output data output by processing the input data of the test case selected by the software Do
A program that causes a computer to execute processing .

Images