JPH02103644A - Method for testing software module - Google Patents

Abstract

PURPOSE:To perform a perfect module test by using the information delivered to the interface of a module to be tested (SUT) for calling the SUT and calling information of the SUT to another module as the information representing the behavior of the SUT. CONSTITUTION:A test attendant 101 gives the interface information 103 of an SUT 109 to an automatic deriver stab preparing program 105. The information 103 contains a list of parameters for calling the SUT 109 and another list of parameters for discriminating global variables used by the SUT 109 and of modules called by the SUT 109 and used for describing input-output interfaces performed between the SUT 109 and its higheer- and lower-rank modules. Therefore, the behavior of the SUT 109 viewed from the lower-rank side interface can be observed in addition to the behavior viewed from the higher-rank side interface without changing the SUT 109 itself. Thus a perfect module test can be performed.

Description

DETAILED DESCRIPTION OF THE INVENTION [Technical Field of the Invention] The present invention relates to a software testing method, and particularly to a testing method for each module.

[Prior art and its problems]

One way to increase the reliability of software is to perform sufficient testing as early as possible in the development lifecycle.

Software that is larger than a certain level is composed of many modules that form a hierarchical structure. Methods for testing software having such a structure include a top-down method, a bottom-up method, a sandwich method, and a big-bang testing method.

In the top-down method, the top-level module is first tested, and then the modules immediately below it are added to the tested module and tested, and the procedure is repeated until the lowest-level module is added and tested. That is,
At the end of this test, all modules have been integrated.

In contrast to the top-down method, the bottom-up method starts with the lowest-level module and proceeds with testing while adding higher-level modules.

The Sand Inch method is a compromise between top-down and bottom-up methods, in which both methods are carried out simultaneously.

In these three types of tests, in order to perform module creation and testing in parallel, the order in which modules are created is determined. For example, in the top-down method, lower-level modules cannot be tested unless higher-level modules are completed. Therefore, schedule delays in some modules are likely to affect the entire project. Also, in this method of simultaneous integration and testing,
As integration progresses, large and complex software must be tested, making it increasingly difficult to detect and isolate failures.

On the other hand, in the big punch method, each module is tested individually, so the above-mentioned problems of flexibility in the development schedule and finding and isolating failures are resolved. However, this method has problems as described below, so it is not suitable for application to the development of software containing a large number of modules.

The modules that make up the software are called by higher-level modules, which also call lower-level modules.
In other words, it operates in conjunction with other modules.

Since the big bread method tests all modules individually, it is necessary to prepare a test environment for each module, such as a group of dummy modules that directly call the module or replace the modules that will be called. The test environment that must be prepared becomes enormous as the number of modules increases, and the amount of cost and time required for testing becomes acceptable.

Furthermore, in conventional module testing, so-called white-box testing was often employed, which acquired information about the control flow paths and branches within the module as substitute characteristics for the module's functions. However, in white box testing, the flow of control is visible but the flow of information is not, so it is not always satisfactory. On the other hand, in conventional black box testing of modules, when the module under test (hereinafter referred to as SUT) is called from the upper side, the response from the SUT is recorded and compared with the expected response. However, no particular processing was performed, such as recording information regarding calls from the SUT to its lower-level side. Therefore, it was insufficient as a black box test to observe the behavior of the SUT from the outside and check whether it was moving correctly.

[Purpose of the invention]

SUMMARY OF THE INVENTION It is an object of the present invention to solve the problems of the prior art described above and to provide a module testing method that can examine as much behavior of an SUT as possible with respect to the environment.

Another object of this module testing method is to automatically generate a test environment to reduce test costs.

[Summary of the invention]

According to one embodiment of the present invention, a module testing method is provided to achieve the above objectives.

For this reason, in embodiments, the SUT
The information represents the behavior of T.

This makes it possible to more fully understand the behavior of the SUT with respect to test data etc. given from the test environment than with conventional testing methods.

Furthermore, in order to automatically create a test environment for the above-mentioned test, this embodiment creates a test driver (test environment) from SUT interface information (including interface information with lower modules called by the SUT). (the side that calls the SUT) and the stub (the side that calls the SUT
(the side called by T) is automatically created. In the simulate mode, instead of actually operating the SUT, test data (values given to the SUT via the interface) of interactions such as input/output and calls to other modules between the SUT and the test environment are generated. input to 3UT)
The user provides expected data (value passed by StJT to the outside via the interface = output from SUT). The run mode executes the SUT based on previously provided test data and compares its effects on the test environment (outputs, calls to other modules, etc.) with the expected data also provided previously. If the data differs from the expected data, the SUT is corrected and operated in execution mode again, and the cycle is repeated until a match is found.

[Embodiments of the invention]

FIG. 1 conceptually shows an embodiment of the present invention. Further, for comparison, a module test based on the conventional method is conceptually shown in FIG.

In the conventional example shown in FIG. 3, a tester 301 creates a test driver and stub 303 by himself, and compiles/links them with the 5tJT 305 to create an executable test program 307. Next, the test program 307 is given test data 309 and executed. The test driver linked in the test program 307 writes the output (return value of a function, parameter passed by reference, writing to a global variable, etc.) from the SUT also linked therein to a file as an execution result. The tester 311 compares this execution result with expected data, and if there is a discrepancy, debugs the 5UT 305 assuming that there is a problem. After debugging is completed, the above process is repeated until the execution results match the expected data.

On the other hand, in the embodiment of the present invention shown in FIG.
give to The interface information 103 includes a list of parameters when calling the SUT, what kind of global variables the SUT uses, a list of parameters of the module called by the SUT, etc. It describes the input/output interface between the two.

Next, the automatically generated test driver and stub 10
7 and 5UT 109 are compiled and linked to create a test program 111. Test program 111
FIG. 2 shows a diagram illustrating an outline of the internal operation of the system. The test program 110 differs from the prior art shown in FIG. 3 in that it has two operating modes: a simulation mode and an execution mode.

In the simulate mode, test program 111 operates as illustrated in FIG. 2(a). In the figure, the test driver 201 provides the tester with an input to the SUT to call 5UT109'' (add `` to the reference number to indicate that it is a compiled and linked SUT), and when the SUT is The tester requests the input of the output returned as the execution result.
Corresponding test data 113 is provided in response to a request for input data to the UT, and expected data 115 is provided in response to a request for output data from the SUT, for example, from a console. These are each test data file 1
19 and is written to the expected data file 117.

Test driver 201 then transfers control to dummy module 201-1, which is part of the test driver.

Here, an inquiry is made to the tester regarding which module the 5UT 109' calls. When the tester specifies the module that should be called first, the dummy module creates the corresponding stub 2
Call 03-207. The called stub, in simulate mode, contains the data to be passed from the SUT to the module to be called (output data from the SUT), just as the test driver 201 did earlier.
and the data that the module returns to the SUT (SU
The tester requests input data (input data to T) from a console or the like, and when the input is completed, control is returned to the dummy module. At this time, the information identifying the module to be called and the data given by the tester to be passed from the SUT to that module are given to the expected data file 117 and by this module to be passed to the SUT. The data is written to test data file 119.

When control is returned to dummy module 201-1, a request is made to the tester to specify the module to be called next, and the above-described operations are repeated. After performing this procedure for all modules to be called, the dummy module is notified of the end of inputting information regarding module calling. This completes the operation in simulate mode.

Next, the execution mode is entered as shown in FIG. 2(b). In this mode, the test driver 201 saves the data to be input to the 5UT 109' into the test data file 11.
5UT 109', and actually transfers control to the 5UT 109'. When a lower module is called during SUT operation, the corresponding stubs 203 to 20, which look exactly the same as the module to be called from the SUT, are
Control is transferred to 7.

The stub called in execution mode writes the data passed from the SUT to the execution result file 121, reads the data to be passed to the SUT from the test data file 119, and sets the required parameters and variables. After that, control is transferred to the SUT.

When all of the internal operations of the 5UT 109' and calls to other modules are completed, control returns to the test driver 201. After the test driver performs the necessary post-processing, it either stops its operation or transfers control to a routine that performs the next stage of processing.

If ``5UT109'' operated as expected, the contents of the expected data file 117 and the execution result file 121 should be the same at this point, so by comparing the two, it is possible to determine whether 5UT109 It can be determined whether or not it has been created correctly (FIG. 1 123).

Alternatively, this comparison may not be performed to determine whether or not there is a complete match, but to determine whether or not the values fall within a certain range, as necessary.

Below, the SUT is Pa5ca 1 with some expanded specifications.
A more specific explanation will be provided assuming that it is a procedure or function. It should be noted that, of course, the present invention can be similarly applied to functional languages, and the module may be any independently testable unit other than individual procedures/functions.

Consider a procedure with the following procedure head as an SOT.

procedure GetADC12(pinLis
t: PinListType; offset:
integer; resultNun+: int
egerHvar resultArray:
Re5ultArrayType); Here, Pin
ListType is a type defined by the user and is other than an array or pointer type. Also, Re5ultA
rrayType is also a type defined by the user as a real array. Further, the third parameter resultNu- represents the size of the array resultArray.

This procedure GetADC12 interface information (
By passing the information (including type definition information used on interfaces) through the driver stub automatic generation program, the test driver for module testing and, if this procedure calls other modules, that module's A replacement stub will be automatically generated. Table 1 shows the source code of the procedure, which is the main part of the test driver generated for this procedure.

1: PROCEDtlRE Table 1 proc539; EGIN pros+ptln(″====GetA port Cl2==
==');pfistReadln('pinList', PIist[1]);1va
rReadln ('offset', 1var [1]); 1var
Readln ('resultNus+ber', 1var[2
) ;IF simulateMode <>
I THENGetADC12(P1istL1),
1var[1].

1var[2], rvararray[13);I
F simulateMode = I TH
ENdums+yModule; IF simulateMode = I T
HENrvararrayReadIn ('res
uI tArray15: 1var[2], rvararray(-1]); r
varrayWritelnResult('re
sultArray', 1var[21°rvar
array (Mon): 16: END.

Note that when generating such a test driver, SU
In order to instruct the interface for calling GetADC12, which is T, parameter information in the format shown in Table 2 is given to the driver stub automatic generation program.

539 GetADC12 PinList 1 offset T resultNua+ T resultArray 0 Table 2 procedure PASCAL 4a
tom PinListType Btt)lIIinteger atO@ integer array real 316384In the parameter information shown in Table 2, the items placed in the first row are, from the left, the identification number given to the SUT, the name of the SUT, and whether the SUT is a function. Identifier to indicate (if it is a function, this will be function), S
This is the UT description language and the number of parameters. The second to fifth lines are information regarding individual parameters. Identifier indicating that it is a parameter from the left (if the value is passed through a global variable other than the parameter in the procedure head, the line where G is written instead of P is also included here) , a sequence number indicating the position of this parameter in the procedure head, an identifier indicating whether a value is input through this parameter (T) or a value is output (0), the name of the parameter, this Whether the parameter is a single variable (atom) or an array (arr
ay) or the type of the identifier or parameter (in the case of an array, its element) indicating the pointer (pointer). If the parameter is an array that can take any size, an indicator indicating the position in the parameter list of the number specifying the execution-time size of the array and an indicator indicating the maximum size of the array. A child is placed.

In addition, instead of creating and providing the information shown in Table 2 manually, by parsing the subprogram head of the subprogram under test (SUT), the driver stub automatic generation program generates the corresponding information. myself is SUT
It may be extracted from.

Note that when doing this, since information about user-defined types and global variables is external to SOT, it is necessary to embed SOT in a dummy program that only contains these external definition information. It is possible to use a method such as giving the information to an automatic driver stub generation program or giving only such information in the form shown in Table 2.

As described above, the driver stub automatic generation program also automatically generates stubs. GetADC1 which is SUT
2 calls two subprograms ProcA and FuncB whose heads are as follows: procedure ProcA(paraml, pa
ram2: integer); function
FuncB (param3: integer):rea
l: At this time, parameter information as shown in Table 3, for example, is given as in the case of SUT.

$601 pH 2T Table 3 ProcA procedure PASCA
L 2paraml atom integer
paraw2 atom integer 602
FuncB function PASCA
L IP 1 1 paraIII3 ato
m integerP OOFuncB atom
realHere, FuncB parameter information 3
The sequence number in line 1 is 0 to indicate that this is the return value of this function. GetAD
As mentioned regarding how to provide C12 parameter information, the information for creating a stub is not in the form of a table like this, but can be written, for example, by writing only the subprogram head into the SUT source code file. good.

A stub is generated by providing parameter information in this manner. Table 4 shows an example of the main parts of the stub for FuncB.
Shown below. Note that the stub generated for ProcA is similar, so a specific example thereof will be omitted.

Table 4 FUCNCTION FuncB (param3:
integer) : real; AR ReturnValue : real; BEGIN promptIn (' ====FuncB====
');IP (simulateMode <>
1) THEN BEGINivarWriteRe
sult('param3', para+*3);E
ND ELSE BEGIN ivarReadln('param3'', par
am3):1varWriteResult('par
as+3', param3); END.

rvarReadln('ReturnVarRetu
rnValue); FuncB := ReturnValue; END.

Further, dummy modules forming another part of the test driver generated based on the above-mentioned interface information are as shown in Table 5 below, for example.

table procedure dun+wyModule;AR procID: integer; BEGIN EPEAT ivarReadln ('Dummy odule No.

9'proclD); ASE procID 0F 601: ProcA (ivar[1]+1var[2J); 602: rvar[1]:= func
B (ivar (1 piece); 0 THERWISE: IF proclD > OTHEN promptln ('Wrong nu++ber!'); END.

UNTIL proclD <= 0; END.

Note that the automatic generation of test drivers and test stubs using this interface information is itself a technique for parsing, interpreting, code generation, etc. in compilers, interpreters, and translators, and is well known to those skilled in the art. Therefore, a description of the method for implementing the test driver stub automatic generation program will be omitted.

Below, the procedure for conducting a test using the test driver and stub generated in this way will be explained with reference to the above table.

In the code in Table 1, the value of the global variable simulateMode is set to 1 when in simulation mode, and set to a value other than 1 when in execution mode.

This set operation is performed in a portion outside the procedure proc539 shown in Table 1 in the test driver.

The procedure promptIn in row 3 of Table 1 stores the string of its parameters and the return/linefeed code to the expected data file.

The procedure plistReadln in line 4 is SUT G
pinLis, the first parameter of etADC12
This is related to the incorporation of t. This procedure prompts with its first parameter: pinList? to the console (in simulated mode)
After telling the tester what test data to enter, input from standard input is typed PinLisTy.
Set the second parameter to be pe. Standard input is connected to the console in simulate mode and to the test data file in run mode. Also, a character string of the first parameter and a character string representation of the value set to the second parameter are output. Note that in the simulate mode, the character string of the first parameter and the character string representation of the data set in the second parameter are also written to the test data file at the same time.

Procedure 1varRead In on line 5.6 is also similar to pli, except that the type of the variable it handles is integer.
The operation is exactly the same as stReadln, and GetAD
Import the values of the second and third parameters of C. Other procedures named xxxReadln operate similarly. Note that in the procedure rvararrayRead In, which performs a similar operation for arrays, the second parameter specifies the size of the array into which the values are to be taken.

In the case of simulate mode, the dummy module dus++syModule is called on line 12, and as described above, it requests the input of the identification number of the lower-level module to be called from the SUT, and calls the stub corresponding to the specified module. The module collects and stores expected value data and test data. This du
The operation of mmyModule and the operation of the stub in simulated mode are clear from Tables 4 and 5, respectively.

Also, regarding the output from the SUT, in simulate mode, line 14 displays its output (here the 4 passed by reference).
The tester is requested to input the data that is supposed to be the data written by GetADC 12 in the array that is the th parameter. The data given by this is written to the expected data file along with the parameter name in line 15.

The operation of the simulate mode is as described above. Note that setting several types of test data at once can be easily done by repeating procedure proc 539 in Table 1 as many times as necessary.

After the test data and expected data have been accumulated as described above, the test program is operated in the execution mode.

In this mode, in row 1 of Table 1, the string GetADC12 is stored in the execution result file by promptIn to indicate that this SUT has been tested.

After that, in lines 4 to 6, the test data previously set in the test data file is read out, and in line 9
These values are set as actual parameters with Get
ADC 12 is actually called.

Output from GetADC12 (resultArray
) is stored in the execution result file on line 15.

When the GetADC 12 attempts to call a lower module in execution mode, the corresponding stub is actually called. As shown in Table 4, the operation of the stub in execution mode is based on the passed data (parameters, etc.).
In some cases, global variables, etc.) are stored in the execution result file.

Also, the value to be input to GetADC12 which is SUT (
Here, the function FuncBO value) is also read out from the test data file and set.

Values to be input to the SUT may be passed through Goopal variables or parameters passed by reference, but the method for setting values in that case is the same.

After the execution mode operation is completed as described above, by comparing the expected data file and the execution result file, it can be determined whether the SUT is operating as expected.

Note that instead of comparing after all execution operations are completed, comparison may be made every time individual execution result data is obtained.

If there is a discrepancy as a result of the comparison, make corrections to the SUT. Unless there are changes to the SUT's interface here, all that is required for the next module test is to compile the new SUT, link it with the previous test drivers and stubs, and then use the test data already created.
All you have to do is run the test program in immediate run mode with the file and expected data file. The test is completed by repeating this until the expected data matches the execution results.

〔Effect of the invention〕

As explained above, according to the present invention, it is possible to observe the behavior seen from the interface on the lower side in addition to the upper side without making any changes to the SUT itself, making it possible to perform more complete module tests. can. Furthermore, automatic generation of test drivers and stubs makes it easier to test large-scale programs on a module-by-module basis, which was previously thought to be difficult. In addition, by answering the questions in order and inputting the necessary data while the test program is running in simulation mode, these input data can be combined into the test data file and the expected data file. Since the data is automatically sorted into files and written in the correct order, errors in inputting this data are reduced. Additionally, once you create both files, you can use the same files each time you modify and test the SUT, allowing you to retest automatically. By retesting, you can not only check whether the defect has been corrected correctly, but also check whether the correction has affected other functions. This improves the overall reliability of the software. Furthermore,
Data is written to both files in the correct order, so
Comparison of execution results and expected data can also be automated.

[Brief explanation of the drawing]

1 and 2 are diagrams for explaining one embodiment of the present invention, and FIG. 3 is a diagram for explaining the prior art. 105: Test driver stub automatic generation program 107: Test driver/stub 109: Module under test 117: Expected data file 119: Test data file 121: Execution result file 201: Test driver 201-1: Dummy・Module 203.205.207: Stub

Claims (3)

[Claims] (1) A software module testing method characterized in that interaction information with a test environment, including information regarding the module under test calling other modules, is obtained from the module under test. (2) A test driver having a first mode and a second mode and a stub used in place of the other module called by the module under test are provided, and the interaction information is used in the first mode. the second mode executes the module under test based on the interaction information and records or compares actions made on the test environment during this execution with the interaction information; The software module according to claim 1.
Test method. (3) The software module testing method according to claim 2, further comprising means for automatically creating the test driver and the stub based on interface information of the module under test.