JPH02230434A - Inspecting method for performance of test program - Google Patents

Abstract

PURPOSE:To improve the test coverage and the performance of a test program by measuring the test coverage of the test program at debugging carried out via a simulator for the test program and making clear the incomplete test items to compensate this incompleteness. CONSTITUTION:A test program 9 is carried out under the control of a simulation system production means 1 of a computer to be tested, and the input information related to the execution of the program 9 and the output information 2 on the executing result are collected for each executing step. The function items 5 of a test subject are collated with each other based on the collected information 2. The attributes 3 related to the items 5 are inspected, and the ratio of the test enable function items and the attribute limit 4 of the program 9 are inspected to the items 5 and the attributes 3. Thus the performance of the program 9 can be inspected.

Description

[Detailed Description of the Invention] [Summary] Regarding a method for measuring Test 1 coverage for evaluating the performance of a test program of a computer system, a test program performance inspection method that can accurately measure the test coverage of a test program is provided. A performance testing method for testing the performance of a test program that tests the functionality of a computer system, the purpose of which is to provide a method for testing the performance of a test program that tests the functionality of a computer system, the test program managing a means for generating a simulated system that simulates the computer system that is the target of the test. The test program is executed below, and when executing each execution step of the test program, input information related to the step execution and output information of the execution result are collected, and the computer system uses the collected input/output information. In addition to collating the functional items to be tested in the computer system, the attributes related to the verified items are verified, and the testable functional items of the test program are verified against the functional items and attributes set as the test target of the computer system. Configure to check ratios and attribute limits.

[Field of Use in Industry] The present invention relates to a method for measuring Test 1 Kahaleshi for evaluating the performance of a test program J for a computer system.

In recent years, data processing by computers has permeated every field of society, and as the performance and functionality of computer systems has been enhanced, test programs that can adequately test and verify the performance and functionality have become increasingly important. It has become to. Test program 1: In order to understand the performance of 1 gram, it is necessary to objectively measure the extent to which the test program can test among all the functions of the computer hardware. Therefore, it is necessary to improve the test program so that the test I and capacity can be improved and computer hardware functions can be universally tested. Therefore, there is a need for an inspection method that objectively measures the Test 1 coverage of Test Program 1.

[Conventional technology]

The conventional method of measuring capacity is to measure test items extracted during the function design process, which determines which functions are to be tested and how, before creating a test program, and functions such as computer instructions and interrupt mechanisms. Approximate values for the test program's test program performance were determined by comparison with the range of the hardware logic specifications that defined the test program.

[Problem to be solved by the invention]

According to the conventional method, test items extracted during functional design are compared with the range of hardware logical specifications to set test program test 1, so the actually created test program does not match the logical specifications. It is not possible to objectively measure what kind of events occur on the actual hardware and over what range for the functions specified in . In particular, in large-scale computer systems where operating modes, instructions, functions, etc. are increased, it is becoming increasingly difficult to measure test program test I/Kahlesi, and hardware design errors and failures cannot be detected due to omission of tests. There was a point.

SUMMARY OF THE INVENTION An object of the present invention is to provide a method for testing the performance of a test program that can accurately measure the test program's Test 1 Kahaleshi.

[Means to solve the problem] FIG. 1 shows a flow chart of the principle of the present invention.

In the figure: ■ Creation of a simulated system that simulates the computer system to be tested Execute a test program 9 in which multiple types of instructions for the computer system J to be tested are arranged in a predetermined order under the control of Sendan 1 let

(2) When the simulated system generating means 1 executes one execution step of the test program 9, it collects input information related to the execution and output information 2 of the execution result.

■Verify the functional item 5 to be tested in the computer system based on the collected input/output information 2, ■Verify the attribute 3 related to the verified functional item 5, and ■Functional item 1 set to be tested in the computer system. ]5
and attribute 3, the ratio of testable functional items of test program 9 and attribute limit 4 are examined.

[Operation] According to the present invention, the test program 9 is executed under the control of the simulated system generator 1 of the test target computer, and the input information related to the execution and the output information 2 of the execution result are collected for each execution step. , collate the test target function item 5 with the collected input/output information 2, verify the attribute 3 related to the function item 5, and determine the testable function items of the test program 9 for the test target function item 5 and attribute 3. The performance of the test program 9 can be tested by checking the ratio and attribute limits 4.

〔Example〕

Embodiments of the present invention will be described below with reference to FIGS. 2 to 5. The whole country is ilI, and the same symbol - 3 indicates the same object. Components in FIG. 2 that correspond to those in FIG. 1 are surrounded by dashed dotted lines.

In FIG. 2, a test program 9a is loaded and executed on a computer system to be tested or a system with equivalent functions and environment to test its hardware functions. In other words, in order to universally test the functions defined in the hardware logic specification C, instructions with various address specification methods, registers to be used, operand values, etc., and combinations are varied for all instructions. LL is executed, and the calculation result is compared with the JRI1 waiting value to test the -C hardware function.

The simulator 1a is a simulator that realizes the functions of the computer hardware and hardware to be tested by generating a program to be executed on another computer system.・Programs created by Uni, for example,
Execute test program 9a in a simulated manner.

The collection unit 2a is configured as a part of the simulator 1a, and collects the state of the hardware 1 and the software in the process of executing one instruction of the test program 1 and 9a as an occurrence event.
event file 21]. That is, for example, when the addition (Δadd) instruction included in the test program 9,) is executed, data as shown in the event data collection example in FIG. 4 is collected, and the executed instruction call is tested. Number of times, number of occurrences of status code CC (0~;3) that reflects the operation result, general-purpose register (G11?0~GR]5)
The number of times the program is used, the maximum/minimum value stored, the value of the operand, the number of times a program interrupt occurs, the code of the interrupt, etc. are collected.

The disk space 3a stores item data of all events that may occur due to the execution of each instruction based on hardware logic specifications. That is, as shown in FIG. 5, for each instruction, (1) the possible values of the instruction code and CC;
C". R O-GR15's possible range and maximum/minimum value, operation result and maximum/minimum value of the operand, presence or absence of emitted η2 for each interrupt type, interrupt code 1, etc., arrangement Q
(2) Program state commands (psW) that control the system
), system control registers (CRO to C R n )
, the possible contents of the memory protection control register, etc. are stored.
There is ζ.

- When the execution of the luck test program 9a is completed, the inspection unit 4a extracts event data that occurred for each event item L1 by instruction code from the event data in the event file 2b based on the event item data in the data header 3a; The maximum/minimum value and the 41 month = L rate are calculated and displayed or printed on the output section.

The operation of this embodiment will be explained with reference to flowchart 1 in FIG.

■ Execute the test program 9a under the control of the simulator 1a, ■ Execute, for example, an 8dd instruction in the test program 9a, and ■ When an interrupt such as a 1-gram exception occurs, the collection unit 2
First, as shown in FIG. 4, the number of program interrupts is accumulated, and the interrupt': food is collected.

■When no interrupt occurs, the collection unit 2a collects the number of executions of the Alcl instruction, CC(0~!) as shown in FIG.
}), the number of times GRO-GR 15 is used, the maximum/minimum value stored, the value of the operand, etc. are collected and totaled.

■Execute the processing of notes ■ to ■ until test programs J and 9a are completed.

■When the execution of the test program 9a is completed, the inspection unit 4a checks whether all CCO values have occurred or not, based on the event item data in the data field 3a, regarding the event data collected in the event file 2b. Evaluate whether G R O to GR15 are used, test 1 data uses the highest/minimum value that can be specified, etc., and check the incidence rate of CC1 to CC3 and the GR used in the operand.
O~3 usage rate, test 1~data and calculation results, test 1
- Calculate the occurrence rate of the maximum/minimum value of data, the occurrence rate of the maximum/minimum value of the 61i calculation results, the incidence rate of program interrupts for each interrupt code, etc.

(2) The inspection section 4a displays or prints out the above-mentioned incidence rate and usage rate on the output section 4b.

Therefore, the test program 9a is executed under the control of the simulator 18, the collection section 2a collects the events that have occurred, and the inspection section 4a collects all the events that can occur (or be used) based on the hardware logic specifications. The system is configured to calculate the occurrence ratio of the event data collected by the collection unit 2a and measure the test program's Test 1 coverage.

〔Effect of the invention〕

As explained above, according to the present invention, the test program 9
When dehugging using the simulator 1a of test a, the test capacity can be measured, so the items that are lacking in test I become clear, and by making up for these deficiencies, test 1 and
It is possible to improve the performance of the test program 9a by increasing storage capacity, and it has the effect of correcting and repairing hardware design errors and failures before shipping the computer to the user. , the number of man-hours required to develop the test program 9a is significantly reduced by improving deficiencies and deficiencies in the test program 9a that were discovered during system operation at the user's site due to hardware failures due to design errors, before shipping the computer to the user. The effect is that it can be done.

[Brief explanation of the drawing]

Fig.] is a principle flowchart of the present invention, Fig. 2 is a block diagram showing an embodiment of the present invention, Fig. 3 is a flowchart 1 of the embodiment of the present invention, Fig. 4 is an example of event data collection, Figure 5 shows an example of the contents of the database. In H, 1 is a simulation rate means, 1a is a simulator section, 2 is a collection section, 2a is a collection section, 2b is an event file, 3 is a storage means, 3a is a data base, 4 is an inspection means, 4a is an inspection section , 4b is an output section, and 9 and 9a are test programs.

Claims (1)

[Claims] Test program for testing the functionality of a computer system (9)
A performance testing method for testing the performance of a test program (9), wherein the test program (9) is executed under the control of a simulated system generation means (1) that simulates a computer system to be tested. The test program (9)
When executing each execution step, input information related to the step execution and output information (2) of the execution result are collected, and the functional items to be tested of the computer system are determined based on the collected input/output information (2). (5) and verify the attribute (3) related to the verified function item (5) to determine the function item (5) set as the test target of the computer system.
) and attribute (3), the ratio of testable functional items and attribute limit (4) of the test program (9) are checked.