JP2003308226A - Test coverage system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a test coverage system which can efficiently evaluate a microprogram. <P>SOLUTION: The system is provided with a coverage storage device having storage areas corresponding to addresses of storage devices stored with instruction words of the microprogram one to one and a controller which writes flag information showing whether a branch condition is met when a conditional branch instruction is executed in a storage area divisionally as 1st and 2nd fields. When an instruction other than an unconditional instruction and a branch instruction is executed, the flag information is written to both the 1st and 2nd fields. <P>COPYRIGHT: (C)2004,JPO

Description

Description: BACKGROUND OF THE INVENTION [0001] 1. Field of the Invention [0002] The present invention relates to a test coverage method used for a program test of an information processing system, and more particularly, to a program in an information processing apparatus using a microprogram. The evaluation method. 2. Description of the Related Art A test of a created program must be performed so as to pass 100% of an execution path existing in the program. When performing a functional test on a microprogram, a test that evaluates the completeness, which is the ratio of the executed steps to all the steps of the microprogram, is used to determine how much of the test has been performed on all functions. Coverage is known. In the test coverage, a breakpoint for instructing the execution of a program to be stopped is set for each module of a program having a certain function, and the execution of the program is tested to determine whether or not the execution is stopped. Soft coverage to evaluate program coverage,
There is hard coverage that operates in parallel with the execution of instructions of a program, writes information for identifying the execution of instructions into a storage device, and analyzes the write information in the storage device to evaluate the coverage of the program. One of the hard coverage methods is to measure the ratio of the number of executed instructions to all the instructions described in Japanese Patent Laid-Open No. 61-26447.
C1 coverage, which measures the C0 coverage and the ratio of the number of executed branch directions to all branch directions, is known. [0004] C0 coverage is a method for knowing the ratio of the number of executed instructions to all the instructions of a program. The C0 coverage operates in parallel with the execution of the instructions, and a storage device having a one-to-one correspondence with the addresses where the program instruction words are stored. Then, when an instruction is executed, a writing means for setting an identification flag in the storage device corresponding to the address to "1" is provided. After the test, the coverage information written by the writing means is read out of the storage device, and by referring to the identification flag, the coverage method can determine the path of the executed program and the path of the unexecuted program. It is. [0005] C1 coverage is a method for determining the ratio of the number of executed branch directions to all branch directions. The C1 coverage is a conditional branch for each address in a storage device that corresponds one-to-one with the address at which a program instruction word is stored. A condition field and a condition branch not-taken field are provided, and information indicating whether or not a condition is satisfied for a conditional branch instruction having two execution paths is separately written into the condition branch-taken field and the condition branch-not-taken field in the storage device. Means. After the test is completed, the coverage information written by the writing means is read from the storage device,
This is a coverage method in which it is possible to determine whether or not a conditional branch instruction has passed an execution path of two existing programs by referring to an identification flag of an address corresponding to the conditional branch instruction. As described above, the C0 coverage can determine the executed program. Meanwhile, C1
The coverage cannot be determined by the C0 coverage, but can determine whether the branch condition in the conditional branch instruction is satisfied or not. FIG. 1 shows the relationship between all the coverage ranges and the corresponding coverage ranges of C0 and C1. In a model of a program in which an interrupt process occurs, a large number of execution paths exist depending on conditions, so it is difficult to cover the entire coverage range only with C0 and C1 coverage. The evaluation is performed using two coverage methods, and the coverage rate of both C0 and C1 is 100%.
If so, you can create a high-quality program. FIGS. 2 to 4 illustrate an embodiment of C0 and C1 coverage. FIG. 2 is a flow chart showing a part of a certain microprogram.
000, 1002, 1003, and 2000 indicate normal instructions other than branch instructions, address 1001 indicates a conditional branch instruction, and address 1004 indicates an unconditional branch instruction.
Path 1 indicates an execution path when the condition is not satisfied in the conditional branch instruction at the address 1001, and path 2 indicates an execution path when the condition is satisfied. FIG. 3 shows an address 1 when a test is performed in this program and C0 coverage is sampled.
It shows the values of bitmap information corresponding to 000 to 1004 and 2000 read from the coverage storage device. Referring to the written contents of the coverage information storage device of FIG. 3, the execution identification flags in the coverage information storage device corresponding to all the addresses are "1", and the instruction sequence of the program is at least once. It can be determined that it has been executed. However, the program address 1001
In, whether or not the program has run on the route 2 cannot be determined only by looking at the result of collecting the C0 coverage. FIG. 4 shows a similar test performed in the C1 coverage collection mode.
A value obtained by reading the bitmap information corresponding to 0 from within the coverage storage device is shown. As shown in FIG. 4, the C1 coverage includes a conditional branch taken field and a conditional branch not taken field in the coverage information storage device. Can be determined. In the conditional branch instruction at address 1001, the identification flag of the conditional branch taken field is "0", and the identification flag of the conditional branch not taken field is "1". In this case, the evaluator determines that the route 2 when the condition is satisfied has not been executed. In the unconditional branch instruction at address 1004, the identification flag of the conditional branch taken field is "1", and the identification flag of the conditional branch not taken field is "0". When only the bitmap information is referred to, the evaluator determines that the path of the program when the condition is not satisfied has not been executed. But C
In one coverage, if the identification flag as described above is indicated, there is a possibility that the instruction is an unconditional branch instruction. Therefore, the evaluator refers to the source code of the program, and finds that the instruction sequence of the program corresponding to the address is invalid. If the instruction is a conditional branch instruction, since there is no path of the program for which the branch determination is not taken, it is determined that the coverage for the address can be covered. If the instruction is a conditional branch instruction, the program for which the branch determination is not taken is given. Is determined not to have been executed. FIG. 5 shows a program evaluation flow using the C0 and C1 coverage method. In processing 500 to 501, test items are drafted and a test program is created. In processes 502 to 504, the test program created in process 501 is set to the operation mode of the hardware C
In step 504, the coverage result is read from the coverage information storage device. In step 505, the result of the C0 coverage is analyzed. If the identification flags corresponding to the instructions of all the programs are not "1", it can be determined that the instruction corresponding to the address has not been executed. Then, the process returns to the process 500 in order to improve the test items, and the test items are drafted again. If the identification flags corresponding to the instructions of all the programs are “1”, the hardware operation is performed in step 508 to determine whether or not the branch condition is satisfied in the conditional branch instruction that cannot be determined by C0 coverage. The mode is switched to the C1 coverage collection mode. At step 509, the same test program is executed again. At step 510, the C1 coverage result is read out from the coverage information storage device, and the result is analyzed at steps 511-5.
Step 13 is performed. In step 511, it is determined whether the identification flags corresponding to the addresses of all the conditional branch instructions are "1" in both the conditional branch taken field and the conditional branch not taken field. If all the identification flags are “1”, the evaluation is completed. In step 512, it is determined whether or not the program that passes when the branch condition is satisfied has been executed. If the identification flag of the conditional branch taken field is “0” and the identification flag of the conditional branch not taken field is “1”, the condition branch instruction corresponding to the address is executed by the program executed when the condition is taken. Since it can be determined that the route was not tested,
A test is performed on the path of this program, and the process returns to step 506 to improve the coverage ratio, and a test item is drafted again. In step 513, it is determined whether or not the program that passes when the branch condition is not satisfied has been executed. In the C1 coverage, when the identification flag of the conditional branch taken field is "1" and the identification flag of the conditional branch not taken field is "0", there is a possibility that the instruction is an unconditional branch instruction. It is necessary to determine whether the instruction is an unconditional branch instruction. If all the instructions at the addresses where the identification flag of the conditional branch taken field is "1" and the identification flag of the conditional branch not taken field is "0" are unconditional branches, the evaluation is completed. When a conditional branch instruction is included in the instruction indicating the identification flag, a test is performed on a path of a program executed when a condition is not satisfied in a conditional branch instruction corresponding to the address. Since it can be determined that the test has not been performed, a test is performed on the path of the program, and the process returns to the process 506 to improve the coverage ratio, and the test items are drafted again. [0020] The prior art described above is
By performing the test until the result of the C0 and C1 coverage reaches 100%, it can be determined that all the instructions in the program have been executed. A model of a program in which an interrupt process occurs cannot cover the entire coverage range, but is effective as a means for improving the quality of the program. However, there are the following problems. As shown in FIG. 1, the C0 and C1 coverages have different coverage ranges that can be covered by the respective coverage methods. If the coverage rate in one coverage method is only 100%, the overall coverage rate can be increased. In other words, it is necessary to increase the coverage rate by switching the operation mode of the hardware in order to further improve the quality. Since the same test must be executed again by switching the operation mode of the hardware, the time required for the program evaluation increases, and the processing flow becomes complicated. An object of the present invention is to solve the above-mentioned problems and to provide a method for simplifying processing in program evaluation by coverage. In order to solve the above-mentioned problem, the present invention changes the write rule of C0 and C1 coverage to reduce the coverage range obtained by C0 and C1 coverage once. Provide a coverage method that can be fully covered. In the coverage method according to the present invention, means for writing information indicating whether a condition used in C1 coverage is satisfied or not in a coverage information storage device into a condition branch taken field and a condition branch not taken field is also applied to C0 coverage. . That is, when an instruction other than the branch instruction is executed, the identification flag of the address corresponding to both the conditional branch taken field and the conditional branch not taken field is set to “1”. Further, even when an unconditional branch instruction is included in a program, the analysis of the coverage information is performed only by referring to the result of collecting the coverage information.
When the unconditional branch instruction is executed, the identification flag of the address corresponding to both the conditional branch taken field and the conditional branch not taken field is set to “1”. That is, in the coverage system according to the present invention, a storage device, which is also used as a conventional technique, is provided in a one-to-one correspondence with an address at which a program instruction word is stored, and operates in parallel with the execution of the instruction. And a coverage information control device that writes information indicating whether the condition in the conditional branch instruction is taken or not taken into the storage device into a condition branch taken field and a condition branch not taken field in the storage device,
In a conditional branch instruction having two execution paths, a conditional branch instruction is executed, and when a branch condition is satisfied, the identification flag of the address corresponding to the instruction execution address in the conditional branch satisfied field is set to “1”. Means for executing the conditional branch instruction, and when the branch condition is not satisfied, in addition to the writing means for setting the identification flag of the address corresponding to the instruction execution address of the conditional branch unsatisfied field to "1", When an instruction is executed, a writing unit that sets an identification flag of an address corresponding to both the conditional branch taken field and the conditional branch not taken field to “1”, and when an instruction other than the branch instruction is executed, Writing means for setting an identification flag of an address corresponding to both the branch taken field and the conditional branch not taken field to '1'; Based on the contents of the written memory by stages, a coverage method to evaluate the completeness of the execution process of the microprogram. That is, the writing rule of the coverage information in the present invention is as follows. (1) When a conditional branch instruction is executed and a branch condition is satisfied, “1” is written only to the address corresponding to the instruction execution address in the conditional branch taken field. (2) If a conditional branch instruction is executed and the branch condition is not satisfied, '1' is written only to the address corresponding to the instruction execution address in the conditional branch not satisfied field. (3) When an unconditional branch instruction is executed, '1' is written to an address corresponding to both the conditional branch taken field and the conditional branch not taken field. (4) When an instruction other than a branch instruction is executed, '1' is written to an address corresponding to both the conditional branch taken field and the conditional branch not taken field. By reading the coverage data written based on the above rules, the evaluator can determine the following. (1) The identification flag of the conditional branch taken field corresponding to all addresses and the identification flag of the conditional branch not taken field are “1”
Indicates that the coverage rate of C0 and C1 is 100
% Can be determined. (2) If the identification flag of the conditional branch taken field corresponding to a certain address is "1", and the identification flag of the conditional branch not taken field is "0", the branch condition is not satisfied in the conditional branch instruction at the address. In this case, it can be determined that the executed program could not be evaluated. (3) If the identification flag of the conditional branch taken field corresponding to a certain address is "0" and the identification flag of the conditional branch not taken field is "1", the branch condition is satisfied in the conditional branch instruction at the address. In this case, it can be determined that the executed program could not be evaluated. (4) If the identification flag indicates 0 in both the condition branch taken field and the condition branch not taken field corresponding to an address, it can be determined that the instruction corresponding to the address has not been executed. FIG. 6 shows a program evaluation flow according to the present invention. In processing 600 to 601, test items are drafted and a test program is created. Process 602
In steps 604 to 604, the test program created in the process 601 is executed after setting the operation mode of the hardware to the coverage collection mode, and the coverage result is read from the coverage information storage device. Process 605
Analyzes the coverage results. That is, in the coverage method of the present invention, the coverage range that can be covered by each of the conventional coverage methods differs between the conventional C0 and C1 coverages. Since the rate cannot be increased, the coverage evaluation, which was performed by switching the hardware operation mode to further improve the quality, is now covered by changing the coverage collection rule to cover the coverage range of C0 and C1 at once. Since the collected coverage information can be collected, the coverage evaluation processing can be simplified. Even if the program contains an unconditional branch instruction, the coverage information can be analyzed only by referring to the coverage information collection result. It is not necessary to refer to the source code of a simple program to determine whether or not the branch is unconditional, and the coverage evaluation processing can be simplified. An embodiment of the present invention will be described below with reference to FIG. 2 and FIGS. FIG. 7 is a block diagram of a coverage information processing apparatus according to one embodiment of the present invention. In FIG. 7, a program control circuit 700 includes a command 710 indicating an execution address of a microprogram, an instruction execution signal 711 that becomes “1” when an instruction corresponding to the address is executed, and a signal “1” when an execution instruction is a branch instruction. Branch instruction signal 7 which becomes 1 '
12, a branch taken signal 713 that becomes “1” when the branch instruction is taken and the branch is taken, and an unconditional branch signal 714 that becomes “1” when the execution instruction is an unconditional branch instruction. . The coverage information storage device 702 has signal lines 716 and 7 connected to the write enable terminals WE0 and WE1.
When 18 is “1”, the coverage information is written to the address in the coverage information storage device corresponding to the address indicated by the signal line 710 connected to the write address terminal WA. The data to be written is the data indicated by the signal lines 715 and 717 connected by WD0 and WD1, and the signal line 715 connected to WD0 is the write data of the conditional branch satisfied field and the signal line 717 connected to WD1.
Indicates write data in the conditional branch unsatisfied field. The coverage information control circuit 701 includes signal lines 711 and 71 output from the program control circuit 700.
Depending on the conditions of 2, 713 and 714, '1' is written to the coverage data corresponding to the execution address of the conditional branch taken field of the coverage information storage device 702, or '1' is written to the coverage data corresponding to the execution address of the conditional branch not taken field. It controls whether to write 1 'or to write 1' to the coverage data corresponding to the execution addresses of both fields.
5, 717 and write enable signals 716, 718, respectively. That is, the coverage information control circuit 701 executes a conditional branch instruction in a conditional branch instruction having two execution paths, and when a branch condition is satisfied, the coverage information control circuit 701 corresponds to the instruction execution address in the conditional branch taken field. First writing means 703 for setting the address identification flag to '1'
If the conditional branch instruction is executed and the branch condition is not satisfied, a second writing unit 704 that sets the identification flag of the address corresponding to the instruction execution address of the conditional branch unsatisfied field to “1”; When the instruction is executed, the third writing unit 705 that sets the identification flag of the address corresponding to both the conditional branch taken field and the conditional branch unsatisfied field to “1”, and when the instruction other than the branch instruction is executed, Comprises fourth writing means 706 for setting the identification flag of the address corresponding to both the conditional branch taken field and the conditional branch not taken field to '1'. If the microprogram being executed by the program control circuit 700 of FIG. 7 is a conditional branch instruction and the condition is satisfied, the program control circuit 700 of FIG.
The instruction execution signal 711 is output as “1”, the branch instruction signal 712 is output as “1”, the branch establishment signal 713 is output as “1”, and the unconditional branch instruction signal 714 is output as “0”. In this case, since the output of the AND circuit 730 of the first writing unit 703 is “1”, the output of the OR circuit 734 is also “1”, which corresponds to the address indicated by the signal line 710 indicating the execution address of the microprogram. '1' is written only in the conditional branch satisfied field in the coverage information storage device thus set. When the microprogram being executed by the program control circuit 700 of FIG. 7 is a conditional branch instruction and the condition is not satisfied, the program control circuit 700 of FIG. ', The branch instruction signal 712 is output as'1', the branch taken signal 713 is output as' 0 ', and the unconditional branch instruction signal 714 is output as'0'. In this case, since the output of the AND circuit 731 of the second writing unit 704 is “1”, the output of the OR circuit 735 is also “1”, which corresponds to the address indicated by the signal line 710 indicating the execution address of the microprogram. '1' is written only in the conditional branch unsatisfied field in the coverage information storage device. Further, when the microprogram being executed by the program control circuit 700 of FIG. 7 is an unconditional branch instruction, the program control circuit 700 of FIG.
711 is output as "1", the branch instruction signal 712 is output as "1", the branch taken signal 713 is output as "1", and the unconditional branch instruction signal 714 is output as "1". In this case, the third writing means 7
Since the output of the AND circuit 732 of "05" is "1",
The outputs of the R circuits 734 and 735 are both "1", and "1" is set in both the conditional branch established field and the conditional branch established field in the coverage information storage device corresponding to the address indicated by the signal line 710 indicating the execution address of the microprogram. 'Is written. Further, when the microprogram being executed by the program control circuit 700 of FIG. 7 is an instruction other than the branch instruction, the program control circuit 700 of FIG. 7 sets the instruction execution signal 711 to “1”, the branch instruction signal 712 is '0',
The branch taken signal 713 is set to '0', and the unconditional branch instruction signal 71
4 is output as '0'. In this case, since the output of the AND circuit 733 of the fourth writing unit 706 is “1”, the outputs of the OR circuits 736 and 737 are both “1”,
Signal line 710 indicating the execution address of the microprogram
"1" is written to both the conditional branch taken field and the conditional branch taken field in the coverage information storage device corresponding to the address indicated by. FIG. 8 shows a value obtained by performing a test in the program shown in FIG. 2 and reading out bitmap information corresponding to addresses 1000 to 1004 from the coverage storage device when the coverage information is collected by the coverage method according to the present invention. Is shown. In the bitmap information shown in FIG. 8, the identification flag of the condition branch unsatisfied field corresponding to the address 1001 is “0”, and the program passes through the path 2 in FIG. 2 executed when the condition is not satisfied at this address. It was not shown. Therefore, the evaluator makes a test at the address 1001 so that the branch condition is not satisfied, and executes the test again. As described above, since the coverage ranges of the conventional C0 and C1 coverages are different from each other, the coverage evaluation performed by switching the operation mode of the hardware is described in the present invention. By using the method, it is possible to collect coverage information covering both the coverage ranges of C0 and C1, so that there is an effect that the coverage evaluation processing can be simplified. Even when an unconditional branch instruction is included in a program, the coverage information can be analyzed only by referring to the result of collecting the coverage information. Therefore, the source code of the program is referred to. In addition, there is no need to determine whether or not the branch is unconditional, and the coverage evaluation processing can be simplified.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 shows all coverage ranges and their corresponding C0, C
FIG. 4 is a diagram illustrating a relationship between one coverage range. FIG. 2 is a diagram showing a part of a certain program in a flowchart. FIG. 3 is a diagram for explaining an embodiment of C0 coverage using an instruction sequence of a microprogram and a value read from a coverage information storage device. FIG. 4 is a diagram for explaining an embodiment of C1 coverage using an instruction sequence of a microprogram and a value read from a coverage information storage device. FIG. 5 is a diagram illustrating a program evaluation process using a conventional C0, C1 coverage method. FIG. 6 is a diagram illustrating a program evaluation process using a coverage method according to the present invention. FIG. 7 is a block diagram of a coverage control device according to an embodiment of the present invention. FIG. 8 is a diagram for explaining an embodiment of the present invention using an instruction sequence of a microprogram and a value read from a coverage information storage device. [Description of Signs] 700 Program control circuit 701 Coverage information control circuit 702 Coverage information storage device 703 Coverage information first writing unit 704 Coverage information second writing unit 705 Coverage information third writing unit 706 Coverage information fourth writing unit 710 Program execution Address signal 711 Instruction execution signal 712 Branch instruction signal 713 Branch taken signal 714 Unconditional branch taken signal 715 Conditional branch taken field write data signal 716 Conditional branch taken field write enable signal 717 Conditional branch not taken field write data signal 718 Conditional branch not taken field write Enable signal 730, 731, 732, 733 AND circuit 734, 735 OR circuit

Claims (1)

Claims: 1. A test coverage method for evaluating the completeness of an execution process of a microprogram stored in a storage device, the test coverage method comprising: one-to-one correspondence with an address of the storage device in which a microprogram instruction word is stored. A coverage storage device having a storage area corresponding to a, and a control device that writes flag information indicating the satisfaction / non-satisfaction of a branch condition at the time of execution of a conditional branch instruction in the storage area separately into a first field and a second field, A test coverage method wherein the flag information is written into both the first field and the second field when an instruction other than an unconditional branch instruction and a branch instruction is executed.