JPH0452835A - Program test evaluation system - Google Patents

Abstract

PURPOSE:To rapidly and efficiently sample and edit a program inclusion rate and a module inclusion rate in real time by outputting an executed/unexecuted program name list, an executed/unexecuted module name list, etc., based upon module execution/unexecution information. CONSTITUTION:An executed module deciding part 7 determines a program and a module including an execution instruction address or a branched address stored in an execution instruction address storing part 6 based upon a table stored in a module constitutional information storing part 2 and decides an execution module. After determining the execution module, a table execution/ unexecution indicator 14 corresponding to the determined module and stored in the storing part 2 is set up to '1'. A module inclusion rate calculating part 3 finds out a program inclusion rate from the number of programs including at least one execution indicator of value '1'/the total number of programs based upon the indicator 14 stored in the storing part 2 and finds out the module inclusion rate from the number of modules including the execution indicator of value '1'/the total number of modules.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a method for measuring program test coverage in an information processing system, and particularly to a method for measuring program coverage and module coverage.

[Conventional technology]

In recent years, the scale of information processing systems has increased significantly and the fields of application have expanded significantly, and the impact that their reliability has on society is increasing. For this reason, the importance of ensuring the reliability of software has become even more important.

Conventionally, program module coverage, which is one of the measures to measure the test sufficiency of a program, is based on the program execution address trace, the branch destination address trace when a branch instruction is taken, or a subset of a program module divided into a fixed size. The trace of a certain unit was stored in a storage device, and after testing, the stored trace information was used for analysis. Additionally, a special command was inserted at the entrance of the module, and the history of the executed module was collected using that command.

Note that a method for collecting this kind of program module coverage rate is disclosed in Japanese Unexamined Patent Application Publication No. 1983-1999. -135555, JP-A-1-
93841 etc. are mentioned.

[Problem to be solved by the invention]

The above-mentioned conventional technology does not take into account collecting and editing the test coverage of the programs and modules of the entire system in testing a large-scale information processing system. That is, in order to obtain the execution module history from the various types of trace information mentioned above, there is a problem that a large amount of storage device is required and it takes a long time to store and retrieve the trace information from the storage device. Furthermore, depending on the type of trace information, there is an accuracy problem in that an error occurs in the calculated program module coverage rate.

Additionally, it was necessary to insert special instructions to record the module execution history in all the modules that make up the program. Furthermore, when shipping the product, it was necessary to delete the instructions inserted to record the execution history.

The present invention also applies to large-scale information processing systems.
With a small amount of hardware resources and without changing the contents of the program, it is possible to quickly and efficiently collect and edit the program coverage rate and module coverage rate in real time.

[Means to solve the problem]

In order to achieve the above object, the present invention acquires the program name and loading address of the program when a program of the system under test is loaded, and acquires the loading address storage area of the corresponding program under test in the module configuration information storage section. and a loading detection means for storing a loading address.

Execution instruction address acquisition means for storing an execution instruction address of the system under test or a branch destination address when a branch of a branch instruction is taken in an execution instruction address storage section; and an execution instruction address or branch destination stored in the execution instruction address storage section. Execution module determination means that reads an address and determines an execution module based on the loading address of the program in the module configuration information storage, the location of the module in each program, and the base module length; and the program name and module in the module configuration information storage. The program coverage rate, module coverage rate, list of executed/unexecuted program names, and executed/unexecuted program name list are determined by the execution status information of the module represented by
The present invention is characterized by comprising module coverage calculation means for outputting a list of unexecuted module names to an output device.

[Effect]

In the present invention, the executed instruction address acquisition means does not stack all executed instruction addresses or all branch addresses when a branch of a branch instruction is taken. The execution instruction address or branch destination address is only temporarily stored in the execution instruction address storage unit until the next instruction is executed or the next branch occurs. Therefore, the capacity of the execution instruction address storage section may be equal to one instruction address.

Furthermore, when the execution instruction address or branch destination address is stored in the execution instruction address storage section, the program name of the loading program acquired by the loading detection means, its loading address, and the execution instruction stored in the execution instruction address storage section The execution program and module are determined based on the instruction address or branch destination address, the program name stored in the module configuration information storage unit, the module name that makes up the program, the location of the module in the program, and the module length. . That is, after the test is completed, all stacked execution instruction addresses or all branch destination addresses are read out and analyzed to calculate the program module coverage rate.

Therefore, the executed/unexecuted modules are determined at the end of the test.

〔Example〕

An embodiment of the present invention will be described below with reference to FIGS. 1 and 2. FIG. 1 shows the configuration of the embodiment, and FIG. 2 shows the structure and contents of a table in the module configuration information storage section 2 shown in FIG.

In FIG. 1, 1 is a loading detection section, 2 is a module configuration information storage section, 3 is a module coverage calculation section, and 4 is a loading detection section.
5 represents an output device, 5 represents an execution instruction address acquisition section, 6 represents an execution instruction address storage section, and 7 represents an execution module determination section.

When a program constituting the system under test is coded into the memory, the loading detection means 1 acquires the program name and loading address of the program, and stores the loading address area 11 of the corresponding program in the module configuration information storage unit 2. Store the obtained loading address in . Furthermore, each time a module is loaded, the module name, rotation 12, and module length 13 of that module are stored in the table.

When a program is executed, an execution instruction address acquisition section 5 acquires an execution instruction address or a branch destination address when a branch of a branch instruction is established, and stores it in an execution instruction address storage section 6. Next, the execution module determining unit 7 determines in which module of which program the execution instruction address or branch destination address stored in the execution instruction address storage unit 6 is located by checking the address in the module configuration information storage unit 2. The execution module is determined based on the table. That is, if the execution instruction address or branch destination address in the execution instruction address storage section 6 is A, and the loading address 11, module location 12, and module length 13 in the module configuration information storage section 2 are B, C, and D, respectively, A program module that satisfies the following equation is an execution module.

(B+C)≦A<(B+C+D) When the execution module is determined, the execution status indicator 14 of the table in the module information storage unit 2 corresponding to the module is set to 1 to indicate that the module has been executed. show. The initial value of the execution status indicator is O,
Indicates that it is an unexecuted module.

The process is repeated for each execution instruction during test execution,
When the test is completed, the module coverage rate calculation unit 3 uses the execution status indicator stored in the module information storage unit 2 to calculate the program coverage rate as the number of programs containing at least one execution indicator with value 1/total number of programs. , the module coverage rate is calculated as the number of modules of the execution indicator with the value 1/the total number of modules, and is output to the output device 4. Furthermore,
If necessary, output the module coverage rate for each program, a list of executed/unexecuted program names, and a list of executed/unexecuted module names for each program! Output to 4.

〔Effect of the invention〕

According to the present invention, even in a large-scale information processing system, it is possible to quickly and efficiently collect and edit program module coverage rates in real time with a small amount of hardware resources and without changing the program contents. be.

In addition, it is possible to obtain not only the program coverage rate and module coverage rate, but also a list of executed/unexecuted program names and a list of executed/unexecuted module names.

[Brief explanation of drawings]

FIG. 1 is a block diagram of an embodiment of the present invention, and FIG. 2 is a diagram showing an example of the structure and contents of data in a module configuration information storage section in FIG. 1. DESCRIPTION OF SYMBOLS 1... Loading detection unit, 2... Module configuration information storage unit, 3... Module coverage calculation unit, 4...
・Output device, ... Execution instruction address acquisition unit, 6...
Execution instruction address storage section, . . . execution module reading section. No. ■ Father

Claims (1)

[Claims] 1. As a measure for quantitatively measuring the test sufficiency and comprehensiveness of a program composed of multiple modules, it indicates the ratio of executed modules to all modules constituting the program. In the method expressed by program module coverage (SO coverage), when a program constituting the system under test is loaded into memory, a loading detection means detects the program name and loading address of the program, and an execution instruction address of the program, Alternatively, an execution instruction address detection means for storing a branch destination address when a branch of a branch instruction is taken into an execution instruction address storage section, and a program name, a loading address, an execution instruction address, or a branch destination address obtained by each of the means, and Execution module determining means that determines which module has been executed based on a module configuration information storage unit that includes a program name that configures the system under test, a module name that configures the program, a location in the program, and a length of the module. Then, the program coverage rate and module coverage rate are calculated based on the module execution and non-execution information determined by the execution module determination means, and the program coverage rate, module coverage rate, executed/unexecuted program name list, and executed/unexecuted program name list are output to the output device. 1. A program test evaluation method, comprising: module coverage calculation means for outputting a list of execution module names.