JPS63307546A - Method and device for evaluation of program test - Google Patents

Abstract

PURPOSE:To realize the evaluation for perfection of a program test by applying an optional test data and measuring the access frequency to a data area for each type of access. CONSTITUTION:An optional test data 40 is given to a program 10 to be tested and consisting of a program data area 11 and a program control area 12. The access frequency to the area 11 is measured for each type of access. As a result, the type-based access frequency can be known together with a data area which does not receive an entire access. In this respect, the test data having the perfection even with the data area can be obtained by setting again such test items and test data that ensure the accesses of all data areas. Furthermore the perfection of a program test is improved by giving a test to a program to be tested based on the obtained test data.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a program test evaluation method and apparatus for evaluating the sufficiency of a program test using test data.

[Conventional technology]

Once you have created a program to be executed by a computer, you need to verify whether it operates according to the functional specifications.

Verification methods are broadly divided into static testing, in which verification is performed manually on a desk without running the program under test, and dynamic testing, in which verification is performed by actually running the program under test.

For dynamic testing, the important factor is how to select effective test data and perform effective verification.

For example, [Hitachi Review J (VoQ, 66Nα3
, PP7-10) and rb i tJ (Vo Q
.

18, Na7, PP831-839)
As shown in , the program under test is used as an indicator for selecting test items and test data in program testing work, such as full text coverage and all branch coverage, which indicate how comprehensively the program under test is executed. Standards based on control structures are known.

According to these coverage criteria, the results of the test work,! Ill
If the pass rate is low, it means that there is a path that has not been executed yet, so it is possible to improve the sufficiency of the test by setting new test items and test data for executing that path.

Here, test data is a general term for test data given to a program under test, and test items are classifications of test contents that are distinguished by the type of test data.

[Problem that the invention seeks to solve]

However, according to the above-mentioned coverage standard, since it focuses only on the control structure of the program under test, even if the coverage rate of the execution path with the test items and test data given in advance is high, the data area of the program under test is There are cases where the test may not be sufficient, such as not necessarily accessing all of these areas, and in some cases, a specific data area among multiple data areas may not be accessed at all. 'L rz' is occurring.

FIG. 8 is a diagram for explaining this.

In the data area 100 of the program under test, as shown in FIG.
As shown in a), the areas 106 to 10B of variables x, y, z
is set in the control area 200, and a program as shown in the flowchart of FIG.

The program shown in Figure 8(b) depends on whether the variable , which has a function similar to a so-called up-down counter, but in order to test this program, for example, x =
5 and x=-5, then x
If =5, one path is passed in the order of steps 208→209→210. Further, in the case of x=-5, the second path is passed in the order of steps 208→208→211.

Therefore, if we calculate the full text coverage rate and all branch coverage rate.

To pass all 3 paths in total, the full text is #! Both Ira rate and total branch coverage rate are 3/3 x 100 (%) = 100
%, and the test is sufficient.

However, regarding the data area 100.

Although variable 2 is set in addition to variable I don't know. That is, there is a problem in that it cannot be determined whether the testing of the program under test is sufficient using only the indicators of full text coverage and all branch coverage.

An object of the present invention is to provide a program test evaluation method and apparatus that can further improve the sufficiency of testing a program under test.

[Means for solving problems]

The present invention provides arbitrary test data to a program under test consisting of a data area and a control area, measures the frequency of access to the data area by type of access,
The measurement results are used to evaluate the sufficiency of the program test using the test data.

[Effect]

Since the program under test is broadly divided into a control area and a data area, the frequency of accesses for definition and reference to the data area during program execution is measured for each type of access such as definition and reference. As a result of this measurement, it is possible to know the frequency of access for each type of access and data areas that are never accessed. Therefore, by resetting the test items and test data so that all data areas are accessed based on this evaluation result, it is possible to obtain test data that is sufficient for the data area. By testing the program under test using , the sufficiency of the test can be further improved.

〔Example〕

FIG. 1 is a functional block diagram showing an embodiment of an evaluation apparatus according to the present invention, in which a data access mechanism 20 that accesses a data area 11 of a program under test 10 consisting of a data area 11 and a control area 12; The test data 40 is given to the test program 10, and the control area 12
The test mechanism 30 executes processing according to the contents of the test and updates the data area 11 according to the execution results.

Here, when the test mechanism 30 accesses the data area 11, the data access mechanism 20 monitors and measures the access frequency by type of access, that is, by content of the access such as data definition or reference. in this case,
If two or more programs are set in the control area 12, the access frequency is measured for each program.

On the other hand, in addition to the access mechanism 20 and the test mechanism 3o, there is an access information storage mechanism 50 that stores the access frequency measured by the access mechanism 20 for each type of access, and an access information storage mechanism 50 that stores the access frequency measured by the access mechanism 20 by type of access, and stores inaccessible areas etc. based on the accumulated access frequency. It includes an access information analysis mechanism 60 that performs analysis, and an access information output mechanism 70 that visually outputs the analysis results and access frequencies accumulated in the accumulation mechanism 50 in a format such as printing or display.

In the above configuration, as a method for the test mechanism 30 to execute the program under test 10, the program under test 1
There is an interbrid method in which the instruction words written in 0 are executed while sequentially interpreting them, and a compilation method in which the instruction words are translated into machine language and the machine language is executed.

When applying Interpretation 1 one-sided method, test mechanism 3
0 executes processing according to the contents while sequentially reading the contents of the control area 12, and when the data area 11 is accessed during the execution, the access mechanism 20 measures the access frequency.

When the compilation method is applied, the test mechanism 30 reads all the contents of the control area 12, translates it into machine language, stores the machine language inside the test mechanism 30, and uses this accumulated machine language. Execute processing according to the word. In this case, when translating into machine language, the test mechanism 30 inserts a counter function to measure the access frequency by type of access in the part where the data area 11 is accessed, and calculates the access frequency obtained by this counter function. Access mechanism 2
Provided to 0.

Note that since the data area 11 is for storing various variables, etc., it is specifically composed of a plurality of areas. Therefore, the access frequency is measured for each area.

Further, the storage mechanism 50 is constituted by known storage means such as a magnetic tape memory, a magnetic disk memory, an optical disk memory, and an IC memory. Furthermore, the analysis mechanism 60 is composed of an IC memory that stores analysis procedures, a microprocessor, and the like. Furthermore, the output mechanism 70 is composed of a CRT display device, a printer, or the like.

FIG. 2 shows, as an example of the tested progera 1110, a program called PASCAI that divides and counts input characters into three types: A-Z, O-9, and other characters.

It is expressed in language. This program is roughly divided into a data area and a control area. In the data area, the variable c
tbll, i is used. The structure of the variable ctb n is divided into two, the array variable counter and the variable ah.
It is divided into ter (0) to (2).

Therefore, the data area 11 in this example is shown in FIG. 3(a).
As shown in the figure, it is composed of a total of five areas, 101 to 105. In this case counter (0)
~(3) and the area of the variable ah belongs to the variable ctbQ, so ctbα・counter
(0] to (2), expressed as ctb12-ch.

FIG. 3(b) is a flowchart of the program shown in FIG. 2, but as shown in FIG. , test item 303 for reading character 2 is set, the storage mechanism 50 stores each area 101 to 105 of the data area 11 shown in FIG. 3(a) and the test item. An accumulation region 50 having a structure as shown in FIG. 5 corresponds to the combinations 301 to 303.
is being prepared.

Therefore, when the test mechanism 30 reads the test data 40 shown in FIG. 4 to test the program under test 10 and executes the process shown in FIG. The process ends in the path of steps 201→202→203→206→207 in FIG. 3(b).

At this time, access to the data area 11 is performed in step 20.
1, define input data to ctbQ-ch,
In the next step 202, the data of ctb12-ch is referred to, and in the first step 203, the data (1,
2, O).

Furthermore, in step 206, the data of variable i is referred to twice to update ctbQ-counter (1).

Therefore, the access frequency of each data area for the test item 301 is as shown in the column of the test item 301 in FIG.

Next, in the case of the test item 302, the process ends in the path of steps 201→202→205→206→207 in FIG. 3(b).

Therefore, the access frequency of each data area for the test item 302 is as shown in the column of the test item 302 in FIG.

Next, in the case of test item 303, the processing ends through the exact same path as test item 301, so the access frequency of each data area is as shown in the column of test item 303 in Figure 5. .

The access frequency data thus measured by the access mechanism 20 and stored in the storage mechanism 50 is subjected to analysis processing such as summation by the analysis mechanism 60. As a result, an analysis result including the total column in Figure 5 is obtained, and the output mechanism 7
Output from 0.

As is clear from FIG. 5, the data area of i=2 (=ct
bQ -counter (2)) For 103, the total number of definitions and references to the data area is 0.
It can be seen that it is not accessed at all.

Therefore, it can be evaluated that by resetting test items such as accessing this data area 103, the test of the data area and control area can be completed.

By the way, if the program in Figure 2 is pl.

There is a control area for p2 that operates independently of pl and is separate from the control area for pl, and the control area for pl and the data area 1 in FIG. 3(a) exist.
If there is a program p2 that commonly uses 1, an area may be added to the storage mechanism 5o to store the frequency of access by the program p2 for each test item, as shown in FIG.

Note that the access frequency data shown in FIG. 5 can also be obtained by the following procedure.

That is, the areas 101 to 1 necessary as the data area 11
05 from the program under test 10, first scans all the control areas, and stores the locations where each data area 101 to 105 is accessed. Then, when the test is executed, it is compared whether the current execution location corresponds to an access location of the data area, and if so, the access frequency data of the area prepared in advance in the a product mechanism 50 is updated.

For example, for the program shown in FIG. 2, the storage area 50' having the structure shown in FIG. 7 may be used. i Prepare the stacker groove mechanism 50 in advance. Therefore, when the test item 301 is executed by the test mechanism 30, the data area 101 is defined and referenced once each, 102 is defined and referenced 0 times, and 103 is defined and referenced 0 times.
It is possible to obtain data that 105 is defined and referenced once each, and 105 is referenced twice and defined once.

Although the above embodiment has been explained using an example of a PASCAL language program, it can be applied to other high-level languages, machine language, etc. in exactly the same way.

〔Effect of the invention〕

As described above, according to the present invention, it is possible to supplement the full text coverage rate and all branch coverage rate when testing a program, so that the sufficiency of the test can be further improved.

Furthermore, sufficient test data can be easily set.

[Brief explanation of drawings]

FIG. 1 is a functional block diagram showing an embodiment of an evaluation device to which the present invention is applied, FIG. 2 is a program list showing an example of a program under test, and FIG. 3(a) is a structure showing the structure of a program data area. 3(b) is a flowchart showing the processing of the program in FIG. 2, FIG. 4 is an explanatory diagram showing an example of test data, FIG. 5 is an explanatory diagram of the volumetric structure of the access information storage mechanism, Figure 6 shows access information storage V
FIG. 7 is an explanatory diagram showing another example of the & structure storage mechanism, and FIG. 7 is an explanatory diagram showing still another example of the access information storage mechanism. FIG. 8 is a structural diagram of a data area and a flowchart of a program under test for explaining a conventional evaluation method. DESCRIPTION OF SYMBOLS 10... Program under test, 11... Data area, 12... Control area, 20... Data access mechanism, 30... Test mechanism, 40... Test data, 5
0...Access information storage mechanism, 60...Access information analysis mechanism, 70...Access information output mechanism.

Claims (1)

[Claims] 1. Give arbitrary test data to a program under test consisting of a data area and a control area, measure the frequency of access to the data area for each type of access, and use the measurement results to determine whether the test A program test evaluation method characterized by evaluating the sufficiency of a program test using data. 2. Give arbitrary test data to the program under test, which consists of a data area and a control area, execute processing according to the contents of the control area, execute processing according to the contents of the data area, and change the contents of the data area. A test mechanism that updates according to the execution results, an access information storage mechanism that stores the frequency of access to data areas by this test function for each type of access, and an output mechanism that visually outputs the stored access frequency for each type of access. A program test evaluation device.