JPH09244921A - Automation system for software test - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently execute the test of a software and to improve software quality characteristics. SOLUTION: A test item extraction function part 1 is provided and the test route and test item of a software test are automatically extracted from a detailed design book and a source file. A test procedure preparation function part 2 is provided and a test procedure model is automatically outputted based on a variable and a break point extracted from the detailed design book, the test route and the test item. Also, a test procedure editing function part 3 is provided, a set value and an expected value are inputted to the procedure model and a test procedure is automatically edited. A test procedure instruction file preparation function part 4 is provided and the test procedure instruction file corresponding to an optional debugger is automatically generated from the test procedure. Also, a tested result judgement function part 5 is provided, the result of the test is compared with the expected value, normal/defective conditions are automatically judged and the test procedure with the result is outputted. Also, a printing function part 6 is provided and the test procedure and the test procedure with the result are routine-printed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a software test automation system for efficiently carrying out software tests and supporting the improvement of software quality.

Software is generally developed according to the waterfall model. This is to conduct individual, combined, and comprehensive tests after the design-manufacturing process, and to check the software quality characteristics (functionality, reliability, usability, efficiency, maintainability, portability). It is a task to check each functional unit.

Particularly, the unit test has a large role to play since it is a test immediately after manufacturing. The unit test requires a large number of test items and a large number of test steps because it is necessary to carry out a white-box test for each module. However, the delay of the design process
In many cases, due to the delivery date, it is not possible to secure sufficient man-hours.

Therefore, the test is often carried out while the extraction of the test item, the preparation of the procedure manual, the preparation of the test data, the judgment of the test result, etc. are insufficient. As a result, due to quality problems, the man-hours of the post-process test are squeezed,
A simple malfunction of the software after operation may cause a problem such as a system stop.

Therefore, it is necessary to efficiently perform the software test including the unit test which is the test immediately after the software is manufactured (the period and the man-hours are greatly reduced) and improve the software quality characteristic. A software test automation system capable of

[0006]

2. Description of the Related Art Conventionally, regarding a method and system (apparatus) for automatically testing software, the efficiency of the test is improved (the number of steps is reduced) by automating the test itself.
And many techniques for improving the reliability of software (evaluation judgment of results) are already known (for example, Japanese Patent Laid-Open No. Sho 5
8-205266 "Automatic software testing method", JP-A-63-8948 "Exchange software automatic testing method", JP-B-5-16220 "Task module automatic verification method", JP-A-3-38956 ""Automatic test device", JP-A-3-156645 "Software automatic test device", JP-A-3-179963 "Information processing device interactive test system", JP-A-3-198457 "Automatic call processing program for exchange processing" Test apparatus ", Japanese Patent Laid-Open No. 5-
No. 244272 "Automatic test method for exchange software",
JP-A-5-233349 "Automatic test tool for program module", JP-A-5-182176 "Automatic test execution device", JP-A-6-208484 "Automatic test method for exchange software", JP-A-7-93185 "Software automatic test system", JP-A-7-200355
No. "Software automatic test equipment").

However, merely automating the test itself does not lead to a true improvement in test efficiency. For example, the tester manually sets the test items so that the program created by him / herself operates, and creates the procedure manual and test data according to the items. Human error is often introduced.

[0008] In particular, unless the purpose of the test is clearly defined and the test is carried out, the test becomes meaningless.

Further, the technique for automatically determining the test result only notifies the tester of the availability of the test result,
The examiner himself must determine which part of the program is defective.

[0010]

If the tester manually extracts the test items or prepares the test procedure as in the conventional software test method, the occurrence of human error is completely eliminated. Cannot be prevented.

Therefore, even if the software test itself can be made efficient, the quality of the software cannot be improved unless the above problems are solved.

The present invention is intended to solve the problems of the prior art as described above, and automates a series of operations from the extraction of software test items to the determination of test results. The objective is to provide an automated software test system that can improve the efficiency of the test and ensure the software quality characteristics by automatically totalizing the test items carried out by and conducting the actual management. .

[0013]

FIG. 1 shows a principle configuration of the present invention, which is a "software test automation system" which automates a series of operations from test item extraction to test result determination. 2 shows an example of the configuration of a test support system (TSS) as an example.

In FIG. 1, 1 is a test item extraction function unit (TS) for automatically generating test items from a detailed design document or source, and 2 is a detailed design document and test item extraction function unit 1.
A test procedure manual creation function unit (TP) that automatically creates a test procedure manual based on the test items created by the 3 can be edited as desired based on the test procedure manual created by the test procedure manual creation function unit 2. Then, the test procedure manual editing function unit (TE) 4 for notifying a message such as omission of the test procedure manual is designated by 4 from the test procedure manual edited by the test procedure manual editing function unit 3 to indicate a test procedure corresponding to an arbitrary debugger. A test procedure instruction file creation function unit (TF) 5 for automatically generating a file executes the test procedure instruction file created by the test procedure instruction file creation function unit 4 on the debugger, and performs a test from the obtained test result file. Test result judgment function unit (TC) for judging the result, 6
Prints the document (or data) generated by each functional unit,
Alternatively, it is a print function unit (TR) that makes a file.

A test support system control unit 7 sets the operating environment of each functional unit and controls all the functional units as a whole. Test item extraction function section 1,
The test procedure document creation function unit 2, the test procedure document edit function unit 3, the test procedure instruction file creation function unit 4, the test result determination function unit 5, and the print function unit 6 can operate independently as tools. Under the supervision of the test support system supervising section 7, it is possible to operate in individual combinations. Therefore, it is possible to schedule the test items to be executed and continuously execute the functions from the test item extracting function unit 1 to the printing function unit 6.

2 and 3 are views for explaining the operation of the present invention.
(1/2), (2/2), which shows a comparison between a conventional test work item and a work item in the software test automation system of the present invention. As shown, the system allows
By repeating item selector, procedure manual creation and editing, command file creation, debugger, and result judgment repeatedly, a series of test work including each work that was done manually in the past is automated. By solving the above problem, it becomes possible to improve the efficiency of software testing and improve the quality of software.

Specific means for solving the problems of the present invention will be given below.

(1) The software test automation system of the present invention is based on a detailed design chart (or source file) of software, and has all test items consisting of a combination of routes on a program that operates under given conditions. The test item extraction function unit 1 that automatically extracts, and the variables that are referred to in the test route automatically based on the detailed design chart and the extracted test items, and the breakpoints are set based on the detailed design chart. A test procedure document creation function part 2 that automatically extracts the test procedure as a procedure manual template to a file and a specified format based on these variables and breakpoints, and set values in the procedure manual template. Enter the expected value data or breakpoints and select the route that is not automatically output by the test procedure document creation function. A test procedure command editing function unit 3 that automatically edits the test procedure document by adding the necessary procedure command for the test and the test command corresponding to any debugger is automatically created based on the test procedure document. Test is performed, the lower module used by the upper module to be tested is automatically connected to perform the bottom-up test, and the test procedure instruction file creation function unit 4 for selecting the test method of the lower module and the test result are displayed. Test result judgment function section 5 that automatically judges pass / fail by comparing with expected value and outputs test procedure with result, and specified test procedure and test procedure with result file start up existing printing tool And a print function unit 6 for performing standard printing.

(2) In the case of (1), the test item extraction function unit 1
Scheduling is performed according to the test items obtained in step 1, and the test procedure document creation function unit 2, the test procedure document editing function unit 3, the test procedure instruction file creation function unit 4, the test result determination function unit 5, and the print function unit 6 The treatment of is continuously performed.

(3) In the case of (1) or (2), the test item extraction function unit 1, the test procedure document creation function unit 2, the test procedure document editing function unit 3, the test procedure instruction file creation function unit 4, the test An additional function is provided to automatically output the content of the abnormality as a message for the abnormality processing during setting of the test environment and during the test by the result determination function unit 5, the print function unit 6, and the test support system control unit 7.

(4) In the case of (1), in the test support system supervision section 7 which supervises the operation of each functional section, the test procedure manual used before is compared with the newly created test procedure manual, and changes are made. In the test item extraction function unit 1, the test procedure manual creation function unit 2, the test procedure instruction file creation function unit 4, and the test result determination function unit 5 while performing the process of inheriting the setting value information other than the part having A difference information collection unit 72 that can back up the previous state and extract the part specified by the user or the difference between the previous time and this time and reflect or restore the information, and the condition by recognizing the specified debugger type A reverse conversion unit 73 that reverses the original procedure, such as converting a debugger command that conforms to the procedure manual language to generate a test procedure instruction file as a test procedure manual.
And a test status management unit 74 that grasps the test progress status and schedules the test based on the test result status table output from the test result determination function unit 5.

[0022]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings. [1] Test item extraction function unit (TS) The test item extraction function unit combines all the test items by combining the route to be tested (the route of the program operating under a certain condition) based on the detailed design chart or the source file. Is automatically extracted. That is, the test item is analyzed by referring to the detailed design document or source file to be tested, and the “conditional expression” on the design document or source is analyzed.
And branch information such as "conditional statement" are extracted, and the test items of the function to be tested are automatically extracted according to the combination of these conditions, and output to the specified file or specified format. After the extraction of the test item, the user can arbitrarily edit (delete, add, change) by using the test procedure manual editing function unit or the like.

The test item extraction function section is composed of the following functional blocks.・ Branch position judgment unit ・ Branch information collection unit ・ Test item output unit

FIG. 4 shows the input / output of the test item extraction function unit. One or both of the detailed design document and the source file are input to the test item extraction function unit 1, It is shown that either or both of the test route table and the test item table can be output.

FIG. 5 is a block diagram showing a configuration example of detailed functions of the test item extraction function unit. Under the test item extraction function unit main 10, a branch position determination unit 11, a branch information collection unit 12, It is shown to have a test item output unit 13.

FIG. 6 shows an outline of an operation flow of the test item extracting function section, and shows an example of a flow of generating a test route and test contents. In the figure, ◇ indicates a judgment sentence, and □ indicates processing.

The information collection elements performed by the branch information collection section and the information output elements performed by the test item output section will be described below. There are two types of information output elements, a test item table and a test route table, which can be arbitrarily output to a specified file or specified format. The information elements needed to generate these are as follows:

(1) Test item table Regarding the test item table, the following items are arbitrarily extracted and output. 1. Test target function name (alphabetic name, Japanese name, comment, etc.) 2. Test item number 3. Test item name 4. Test content 5. Test data name 6. Test item list creation date 7. Test item list Creator's name 8. Other (remarks column)

FIG. 7 shows an output example of the test item list. As a list of test (test) items, it is shown that the test item name, test content, test data name, and remarks are output.

(2) Test route table Regarding the test route table, the following items are arbitrarily extracted and output. 1. Test target function name (alphabetic name, Japanese name, comment, etc.) 2. Test item number 3. Test item name 4. Test content 5. Test data name 6. Test route table creation date 7. Branch point content 8 Data for passing / non-passing of branch points 9. Positions of conditional statements (source upper line numbers, etc.) 10. Others (remarks column)

FIG. 8 shows an output example of the test route table. As a test (test) route table, it is shown that the test item number and the test item name are output for each line number together with the conditional expression. In the figure, the double-lined frame is the test route information, and if there is no true or false notation, the route does not pass. Also, IF statement ELSE
If it is not true even if there is no, a false notation is made.

[2] Test Procedure Document Creation Function Unit (TP) The test procedure document creation function unit automatically extracts symbols (variables) to be referenced in the test route based on the detailed design chart (or source file) and test items. In addition, the breakpoints are automatically extracted based on the detailed design chart, and based on the symbols (variables) and breakpoints referenced in the test route, the test procedure is used as a test procedure template.
It automatically outputs to a file and a specified format, and is composed of the following functional block elements.

-Operating environment analysis unit-Test route analysis unit-Format standardization conversion unit-Syntax element analysis unit-Variable information collection unit-Branch information collection unit-Procedure manual template output unit

FIG. 9 is a block diagram showing a configuration example of detailed functions of the test procedure document creating function unit. Under the test procedure document creating function unit main 20, an operating environment analyzing unit 21 and a test route analyzing unit are provided. 22, a source analysis unit 23, and a procedure manual template output unit 24 are shown.

FIG. 10 is a block diagram showing an example of the functional configuration of the source analysis unit. The source analysis unit 23 has a format standardization conversion unit 231 and a syntax element analysis unit 232.
Further, the variable information collection unit 2 is provided under the syntax element analysis unit 232.
33, a branch information collecting unit 234 is shown.

FIGS. 11 and 12 are diagrams (1/2) and (2/2) showing the outline of the operation flow of the test procedure document creating function section, in which ⋄ indicates a judgment sentence, □ indicates processing.

Further, FIGS. 13 and 14 are diagrams (1/2) and (2/2) showing the generation flow of the test route and the test content, and the test procedure document shown in FIGS. 11 and 12. Process in each functional block shown in (a) ~ (e) in the creation flow,
An example applied to realistic processing is shown. In the figure,
Indicates a judgment sentence, and □ indicates a process.

The process of determining whether to register in the "variable information table" as the set value variable or the expected value variable when the test procedure manual is created will be described below. Registering here means putting it in the entry as a set value variable (expected value variable), and considering the order of registration, it is set as a set value variable (expected value variable) in the FIF on the test procedure document.
Expand in O format.

A test procedure manual including human thought can be constructed by automatically extracting a set value or an expected value that satisfies these conditions.

(1) There are the following four cases as conditions for registering as a set value variable. Note that the duplicated variables are
Do not register. 1. Variable on the right side of the assignment statement 2. Variable of conditional expression (only when the conditional expression of the conditional statement is a variable and the corresponding variable before the conditional expression has never been used) 3. Static It is determined that setting conditions are required such as various pointer variables 4. It is possible to select whether to display the entry of the setting information for lower-level functions

(2) The conditions to be registered as the expected value variable are as follows. In this case, it is possible to select whether to remove the duplicated variable or to operate it, depending on the information in the operating environment analysis of the test procedure document creating function unit. 1. Variable on the left side of the assignment statement 2. It is also possible to select whether or not to register the confirmation information variable for lower-level functions

FIG. 15 shows an example of the flow for registering variables. In the figure, ◇ indicates a judgment sentence, and □ indicates processing.

An example of the processing procedure when the test procedure manual is created will be described below. A. Example of processing procedure (1) Generate a source that matches the test items for each test item (generate as a work file).

(2) The variable name accessed on the test route is extracted for each test item based on the test route table. 1. Search the storage class or type name of the variable from the variable name and register it in the setting value or expected value information table. 2. If the extracted variable name is a duplicate variable name, delete it in the information table. Even if the structure member name and the simple variable name match, they are not regarded as the same.

3. When the union declaration is made, the type of the variable is analyzed and the one pointing to the same area is deleted. 4. If there is a Japanese name tag (Japanese comment) corresponding to the extracted variable name, retrieve the Japanese name from the Japanese definition information file and register it in the information table. 5. For the array variable name, register the maximum number of array subscripts in the information table.

(3) Based on the information table, set values and expected values are output in a procedure manual template. 1. A variable whose entry is registered as an information table for set value variables is output in the form of a procedure manual as an initial set value in FIFO format. In order to faithfully verify the same movement as the source logic, the set value processing is performed in the FIFO format.

2. The variable whose entry is registered as the information table for the expected value variable is output to the procedure manual template as an expected value in the FIFO format or an arbitrary format. When the check value duplication check function is enabled, it is possible to delete or comment the lower-level function part of the expected value that overlaps between the lower-level function and the higher-level function, and only check the upper-level function. There is.

3. When outputting in the above 1. and 2., AUTO variables used in the function under test (variables used only in the function being tested), or parameters, unions ， Variables used in the loop are marked with distinction and output to the procedure manual template.
For variables that have this AUTO variable mark information (the variable type is also shown to show the range of numerical values that can be set to the user), ignore this variable in the test procedure manual import information for lower-level functions. It expands to a command file and realizes the confirmation function of the calling logic to the subordinate function. Also, by clarifying the variable attributes used in the union and loop, it helps the user to describe the test data as the set value.

(4) Consideration on testing to pointer variables 1. READ operation is performed using the pointer variables used in the function under test (module) (only when global variables are referred to by pointers). When a pointer reference statement is detected, in order to set a value to the pointer reference variable, a breakpoint is set where the value is written to the corresponding pointer variable, and a value is set to the pointer reference variable. The processing procedure and setting value description are output in the procedure manual template.

2. When an AUTO variable pointer is used to find a pointer-related bug that tends to exist on the source, even if the READ operation similar to the above 1 is performed, a breakpoint is set. Do not output the settings and setting value descriptions of in the procedure manual template.

(5) When a lower-level function (subroutine) is present on the test route, the lower-level procedure manual import command is output to the procedure manual template (default operation). Also,
If this procedure manual import command is unnecessary, it can be easily invalidated by deleting the line.

(6) The following information is extracted from the test route table, the test item list, and the YPS source (a program or design document described in the YAC chart) and output as the header of the procedure manual template. To do. 1. Test item name (test procedure file name) 2. Test content and effect confirmation method 3. Procedure creator name 4. Function name (module name) 5. Test procedure creation date

(7) Based on the specified test route table, break points for AUTO variables and pointer variables used in the function called in the test route for each test item are used in the module under test. Number of break points (maximum number depends on the maximum number of branches at one point in the source) + 1 position, and it is used as a pair with breakpoint clear to minimize the number. .

(8) It has a function of extracting the following information and setting the operating environment. This function is performed by the operating environment analysis of the test procedure document creation function unit. Or the same thing can be done in the general department.

1. Designation of input source of detailed design chart and source line number correspondence information file 2. Designation of input source of name tag information file (variable name information file) 3. Designation of output destination of template file of test procedure document 4 . Input source specification of variable type declaration file (source base) 5. Input source specification of test item list file of item selector output 6. Input source specification of test route table file of item selector output 7. Include file name for setting value Specify the input source for the expected value. 8. Specify the input source for the include file name for the expected value. 9. Specify the source language. 10. Specify the function that does not import the test procedure. 11. Specify the function that imports the test procedure.

B. TP process error warning message The process performed by the user when creating the procedure manual (setting procedure, expected value procedure, test procedure manual editing function section (TE) such as procedure addition, test procedure instruction file creation function section (TF)) If the operation is incorrect for (output), inform the information to that effect by a message so that the set value, expected value, procedure description, etc. when creating the test procedure manual will be properly omitted. Reportedly.

The main messages output by the test procedure manual edit function unit (TE) and the test procedure instruction file creation function unit (TF) are as follows. The setting value of variable XXXX is not described. The expected value of variable XXXX is not described. The parameter of procedure language XXX is abnormal.

C. Test Procedure Document Format The test procedure document is output in the following format.

FIG. 16 shows an example of the format of the test procedure manual. FIG. 17 shows an example of the format of the sub procedure manual. 18 and 19 show
Figures (1/2) and (2/2) showing an example of the test procedure manual.

D. Test Procedure Document Language The test procedure document is described using commands and comments according to the command syntax as follows.

(1) Command syntax In the test procedure manual, the procedure part is described after the header part. By describing the procedure manual command in this procedure section, the command of the simulated debugger can be automatically generated.

The procedure manual command is described by the following description method. <Command>[<Paramater>] △ <Paramater> △ ｜ [= Valu
e] △ ｜ [＝ Value] △

Search for a command character string. Detect “△” after the command character example. (1) After the first parameter, the parameter is detected according to the format of each command. Detect the end mark. Search for the next command string. Ignore all strings in the middle.

(A) Handling of Numerical Values Numerical values correspond to the following formats. All single-byte numbers.・ Decimal number nnnnnn ・ Hexadecimal number 0xnnnn ・ Character string "aaaa"

(B) Handling of variables Variables correspond to the following formats. -Variable aaa-Array variable aaa [bbb] -Structure variable aaa [bbb]. ccc [ddd]
[Eeee]

(2) A line having ";" at the beginning of the comment line is a comment line.

(3) Command In the present embodiment, the following various types of procedure manual commands can be recognized.・ Breakpoint command ・ Execution command ・ Setting command ・ Check command ・ Capture command ・ Direct command

Various commands will be described below.

(A) Breakpoint command This is a command for temporarily stopping the operation of the program under test for the purpose of confirming data and setting data, and there are the following various types.・ STOP ・ CLEAR

1. STOP (Break Point), NONS
TOP (Non Break Point) This is a command for temporarily stopping a running program or for confirming whether or not a certain point is passed. 2. CLEAR (Clear Break Point) This command clears the set break point.

(B) Execution / Stop Command This is a command for executing or stopping the program under test, and there are the following various types.・ GO ・ STEP

1. GO This command starts execution from the stopped program counter. 2. STEP This is a command to execute only one line from the stopped program counter.

(C) Setting command This is a command for setting a value in the memory specified by a variable or address, and there are the following various types.・ SET ・ MEM ・ FIL

1. SET (Set Valiable) This is a command for setting values for variables, arrays, and structures (unions, etc.). 2. MEM (Set Memory) This is a command to set a value at a specified address. 3. FIL (Fill) This command sets the specified value by the specified number from the specified address.

(D) Confirmation command This is a command for displaying the values of variables and addresses. The value displayed by this command is read by the result determination tool, and compared with the expected value to check whether the value is OK or NG. There are the following various types.・ CHK ・ DMF ・ RET ・ REG

1. CHK (Check Valiable) This command checks the values of variables, arrays, and structures (unions, etc.). 2. CHK (Check Valiable) [Extended] This command checks the values of variables, arrays, and structures (unions, etc.). 3. DMP (Monitor Address) This is a command to check the value of the specified address. 4. DMP (Monitor Address) [Extended] It is a command to check the value of the specified address.

5. RET (Return Value Disply) This is a command to be expanded in the command file by directly writing the command (the command to confirm the return value of the function) designating the beginning of the test procedure manual. 6. REG (Register) This is a command that expands to a command file by directly writing a command that specifies the beginning of the test procedure manual (a command that confirms the value of the specified register).

(E) Import command This is a command for importing the procedure manual of the lower-level function and the procedure manual describing the common initialization and common confirmation information, and there are the following various types.・ INC ・ FILE ・ CALL

1. INC (Include File) This is a command to include the procedure manual of the lower function. 2. FILE The common items (initial values, expected values, procedures) of each procedure manual can be made into a common file. This is a command to import this common file into the procedure manual. 3. This is a command for directly calling the CALL function.

(F) Direct command The following are the procedure manual commands for directly describing the commands of the simulator / debugger.・ DR

1. DR (Direct) This is a command for expanding the command of the simulated debugger into a command file by directly writing it.

(G) Special Command The special command is a command used to indicate the range of the procedure block shown below. -Procedure manual classification (START-END) -Pre-setting classification (PSET-PEND) -Initial setting classification (INIT-IEND) -Expected value classification (EXPC-EEND) -Stop classification (STOP-SEND) These classifications are tests It is incorporated into the procedure manual by the procedure manual creating function unit (TP). In addition, the test procedure document editing function unit (TE) allows arbitrary editing.

1. Procedure manual classification In the procedure manual, the range in which the procedure manual language can be used is specified. 2. Pre-setting category In the procedure manual, specify the setting process description range before calling the function under test. This block is START
~ Described outside END. 3. Initial setting category In the procedure manual, specify the range for initial setting. 4. Expected value classification In the procedure manual, specify the range for checking expected values. 5. Stop category Specify the stop point. In this, the operation (setting, confirmation) to be performed when stopped can be described.

(H) Others In addition, there are the following various types of procedure manual languages that are used for a special meaning. -Indicates continuation line ($)-Indicates variable dependency (AT, PT, UN, for) -Variable type information (ss, si, ui, ul) These categories that indicate the range that can be set (confirmed) by the variable Is incorporated into the procedure manual by the test procedure manual creation function unit.

1. Continuation line notation In the procedure manual, the range of one line of the procedure manual language is specified. 2. Variable attribute Specify the attribute of the variable in the procedure manual.

[3] Test Procedure Manual Editing Function Unit (TE) The procedure manual editing function unit edits the procedure manual as follows.

(1) Input the following data to the procedure manual template created by the test procedure manual creating function section. 1. Setting value 2. Expected value 3. Test item number of function to be called

(2) In the procedure manual creating function shown below, a procedure manual command (procedure manual language) that cannot be automatically output is added as necessary. 1. Variable that is not accessed in the route of the test item
When adding confirmation (including array and structure) ・ Input variable name (Japanese, English name) ・ Input variable type ・ Input setting value or expected value

2. When setting a breakpoint at each branch point to confirm the route passage ・ Breakpoint specification (line number, function name specification) ・ Nonstop breakpoint specification (line number, function name specification)

3. A command is added so that the set value or expected value of the variable (including array or structure) accessed in the while statement or the for statement can be checked for the variable for the number of loops. Case ・ Input variable name (Japanese, English name) ・ Input variable type ・ Input setting value or expected value

4. When adding a command so that the function called in the while statement or for statement can be accessed for the number of loops: -Access-Breakpoint input-Execution command (GO) input-Called function Enter the test item number of

5. When setting a breakpoint at the position where the function ends in the middle of the module ・ Specifying the line number of the breakpoint 6. Calling the function by table jump (calling the corresponding function corresponding to the test item Add commands so that you can) ・ Enter the test item number of the calling function 7. Import the procedure manual for setting the initial value for the test 8. Enter the test result confirmer name

[4] Test procedure instruction file creation function unit (TF) The test procedure instruction file creation function unit automatically generates a test command corresponding to an arbitrary debugger based on the test procedure manual and executes the test. , The lower module used by the upper module to be tested is automatically connected to perform the bottom-up test, and the testing method (white box or black box) of the lower module is selected as necessary. It is composed of each functional block element.・ Operating environment analysis unit ・ Lower function import unit ・ Procedure language conversion unit ・ Test procedure instruction file creation unit

FIG. 20 is a block diagram showing a configuration example of detailed functions of the test procedure instruction file creation function section, in which the operating environment analysis section 41 and the lower-level function import are provided under the test procedure document creation function section main 40. Part 42, procedure manual language conversion part 43,
It is shown to have a test procedure instruction file creation unit 44.

FIG. 21 shows an outline of the operation flow of the test procedure instruction file creation function section.
Indicates a judgment sentence, and □ indicates a process.

The test procedure instruction file (command
An example of the processing procedure when creating a file will be described.

(1) A test procedure instruction file (command file) is automatically converted from the test procedure instruction file created by the operation of the test procedure instruction editing function unit into an execution command suitable for the specified symbolic debugger. To generate. The test procedure manual is written using a versatile and convenient TSS test procedure manual language, and conversion is performed by associating this procedure manual language with the debugger command for the designated debugger.

(2) The following items described in the procedure manual are output to the command file as comments. 1. Test item name (command file name) 2. Procedure file generation date 3. Creator name 4. Japanese comment for variable name

(3) When there is a subordinate function, it is possible to select whether the test method is a white box test or a black box test.

1. When the white box test is specified, the setting values of the dynamic variables used in the test procedure for lower-level function (module) are deleted. The expected values of the dynamic variables used in the test procedure for lower-level functions (modules) are retained. The setting values of static variables used in the test procedure for lower-level functions (modules) are retained. The expected values of static variables used in the test procedure for lower-level functions (modules) are retained.

1. When the black box test is specified, the setting value of the dynamic variable used in the test procedure for lower-level function (module) is deleted. Delete the expected value of the dynamic variable used in the test procedure for lower-level function (module). The setting values of static variables used in the test procedure for lower-level functions (modules) are retained. Delete the expected value of the static variable used in the test procedure for lower-level function (module).

This low-level function import function makes it possible to create test execution data for inter-function connection tests as well as function unit tests. Also, bottom-up and top-down tests can be used.

[5] Test Result Judgment Function Unit (TC) The test result judgment function unit compares the test result with an expected value,
The automatic determination is performed and includes the following functional blocks.・ Operating environment analysis unit ・ Test procedure document analysis unit ・ Test result file analysis unit ・ Test procedure document output unit with results ・ Test status table output unit

FIG. 22 is a block diagram showing a configuration example of detailed functions of the test result judging function unit. Under the test result judging function unit main 50, an operating environment analyzing unit 51 and a test procedure document collating unit 52 are provided. , A test procedure manual output unit 53 with results and a test status output unit 54 are shown.

FIG. 23 is a block diagram showing an example of the functional configuration of the test procedure document collating unit. The test procedure document collating unit 52 has a test procedure document analyzing unit 521 and a test result file analyzing unit 522. Is shown.

FIG. 24 is a diagram showing an outline of the operation flow of the test result judging function section. In the figure, ⋄ indicates a judgment sentence and □ indicates a process.

(1) It is checked whether or not the test result recorded in the test result file (journal file, etc.) generated by executing the test on the debugger matches the expected value. The NG display and the result value are automatically entered next to the expected value column in the procedure manual for the test item corresponding to the case. (2) As a result of (1), the test date and test result (OK / NG) are automatically input from the journal file.

(3) In the confirmer column, the operator checks the test result and enters the name in the editor as proof of approval (when this column is not described, the test is not completed). (4) As a function linked to the progress management tool, the date, the number of test runs, the number of NGs, and the number of OKs are output. (5) Output the test item number of the item that became date, OK, and NG as a function linked to the question management tool.

25, 26, and 27 are diagrams (1/3), (2/3), and (3/3) showing an example of the test procedure manual with results.

[6] Print Function Unit (TR) The print function unit prints the files of the designated test procedure manual, test procedure manual with results, etc. by the standard printing by activating the existing print tool.

[7] Test Support System Control Section The test support system control section is composed of the following functional block elements. Each of these functional blocks is independent.・ Operating environment analysis unit ・ Differential information collection unit ・ Reverse conversion unit ・ Test status management unit

FIG. 28 is a block diagram showing a configuration example of detailed functions of the test support system control section. Under the test support system control section 7, an operating environment analysis section 71, a difference information collection section 72, and a reverse information collection section 72 are provided. It is shown to have a conversion unit 73 and a test status management unit 74.

(1) Operating environment analysis unit Sets the operating environment of the test support system. In this part, the test item extracting function unit 1, the test procedure document creating function unit 2, the test procedure document editing function unit 3, the test procedure instruction file creating and creating function unit 4, the test result judging function unit 5, the printing function unit 6 The operating environments may be set in parallel.

(2) Difference information collection section When creating a test procedure manual for retesting, etc., the test procedure manual used previously is compared with the test procedure manual newly created this time, and there are changes The setting value information other than is inherited.
It also has a function of backing up the previous state in each functional unit of TS, TP, TF, and TC so that it can be restored.

(3) Reverse conversion unit This is a function to generate a test procedure instruction file as a test procedure manual. Grasp the specified debugger type and convert the debugger command that meets the condition into procedure language.

(4) Test status management section Based on the test result status table output by the test result determination function section, the test progress status is grasped and the scheduling is performed.

29 and 30 are diagrams (1/2) and (2/2) showing the outline of the operation flow of the difference information collecting section, in which □
Indicates processing. Further, FIG. 31 is a diagram showing an outline of the operation flow of the reverse conversion unit, and in the figure, □ indicates processing.

FIGS. 32, 33, and 34 are diagrams (1/1) showing the outline of the data cooperation relating to the test support system, respectively.
3), (2/3), (3/3). As shown in the figure, various files related to the procedure manual creation tool (TP), the procedure manual editing work (TE), the command file creation tool (TF), and the result determination tool (TC) are shown.

FIG. 35 illustrates the difference in work contents between the conventional technique and the method of the present invention. (A) shows the conventional work, and (b) shows the work by the test support system of the present invention. Shows. The shaded portion in the conventional work indicates a portion that requires manual work (the portion where human error may occur), and the hatching portion in the work according to the present invention indicates a portion where the work is automated.

FIG. 36 shows the number of man-hours in the software test, and shows the number of man-hours of the conventional test and the number of man-hours of the test in this system in comparison with each other for each test item. According to this system, it is shown that the number of test steps can be halved as compared with the case of the conventional system.

FIG. 37 shows the defect detection rate in the software test, showing the case of the conventional system and the case of the present system in comparison for each test item. In the figure, CT is the compotest, IT indicates an integration test and ST indicates a system test. It has been shown that this system can detect a defect by a test at a relatively early stage as compared with the case of the conventional system.

[0122]

As described above, according to the present invention, it is possible to improve the efficiency of the entire work of the software test, and to improve the quality of the software at a relatively early stage of the test. The development period of the entire software can be shortened. Further, it becomes possible to provide software that sufficiently satisfies the software quality characteristics.

[Brief description of drawings]

FIG. 1 is a diagram showing a basic configuration of the present invention.

FIG. 2 is a diagram (1/2) explaining the operation of the present invention.

FIG. 3 is a diagram (2/2) explaining the operation of the present invention.

FIG. 4 is a diagram showing inputs and outputs of a test item extraction function unit.

FIG. 5 is a block diagram showing a configuration example of detailed functions of a test item extraction function unit.

FIG. 6 is a diagram showing an outline of an operation flow of a test item extraction function unit.

FIG. 7 is a diagram showing an output example of a test item list.

FIG. 8 is a diagram showing an output example of a test route table.

FIG. 9 is a block diagram illustrating a configuration example of detailed functions of a test procedure document creating function unit.

FIG. 10 is a block diagram illustrating a functional configuration example of a source analysis unit.

FIG. 11 is a diagram (1/2) showing an outline of an operation flow of the test procedure document creating function unit.

FIG. 12 is a diagram (2/2) showing the outline of the operation flow of the test procedure document creation function section.

FIG. 13 is a diagram (1/2) showing a flow of generating a test route and test contents.

FIG. 14 is a diagram (2/2) showing a flow of generating a test route and test contents.

FIG. 15 is a diagram showing an example of a flow for registering variables.

FIG. 16 is a diagram showing an example of a format of a test procedure manual.

FIG. 17 is a diagram showing an example of a format of a sub procedure manual.

FIG. 18 is a diagram (1/2) showing an example of the test procedure manual.

FIG. 19 is a diagram (2/2) showing an example of the test procedure manual.

FIG. 20 is a block diagram illustrating a configuration example of detailed functions of a test procedure instruction file creation function unit.

FIG. 21 is a diagram showing an outline of an operation flow of a test procedure instruction file creation function section.

FIG. 22 is a block diagram illustrating a configuration example of detailed functions of a test result determination function unit.

FIG. 23 is a block diagram showing a functional configuration example of a test procedure document collating unit.

FIG. 24 is a diagram illustrating an outline of an operation flow of a test result determination function unit.

FIG. 25 is a diagram (1/3) showing an example of the test procedure manual with results.

FIG. 26 is a diagram (2/3) showing an example of the test procedure manual with results.

FIG. 27 is a diagram (3/3) showing an example of the test procedure manual with results.

FIG. 28 is a block diagram illustrating a configuration example of detailed functions of a test support system control unit.

FIG. 29 is a diagram showing an outline of an operation flow of a difference information collection unit.
(1/2).

FIG. 30 is a diagram showing an outline of an operation flow of a difference information collection unit.
It is (2/2).

FIG. 31 is a diagram illustrating an outline of an operation flow of a reverse conversion unit.

FIG. 32 is a diagram (1/3) showing an outline of data cooperation relating to the test support system.

FIG. 33 is a diagram (2/3) showing an outline of data cooperation relating to the test support system.

FIG. 34 is a diagram (3/3) showing an outline of data cooperation relating to the test support system.

FIG. 35 is a diagram for explaining the difference in work content between the conventional technique and the method of the present invention, (a) showing the conventional work, and (b) showing the work by the test support system of the present invention.

FIG. 36 is a diagram showing work man-hours in a software test.

FIG. 37 is a diagram showing a defect detection rate in a software test.

[Explanation of symbols]

 1 Test item extraction function unit (TS) 2 Test procedure document creation function unit (TP) 3 Test procedure document edit function unit (TE) 4 Test procedure instruction file creation function unit (TF) 5 Test result judgment function unit (TC) 6 Print Function Department (TR) 7 Test Support System Management Department

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Shinichi Ishizaki 3-9-18 Shin-Yokohama, Kohoku Ward, Yokohama City, Kanagawa Prefecture Fujitsu Communication Systems Ltd.

Claims (4)

[Claims] 1. A test item extraction function unit that automatically extracts all test items consisting of a combination of routes on a program that operates under given conditions based on a software detailed design chart, and a source file. The variables referenced in the test route are automatically extracted based on the detailed design chart including the above and the extracted test items, and the breakpoints are automatically extracted based on the detailed design chart including the source file. A test procedure document creation function unit that automatically outputs the test procedure as a procedure manual template to a file and a specified format based on variables and breakpoints, and setpoint and expected value data or breakpoints in the procedure manual template. And the procedure manual for route selection that cannot be automatically output by the test procedure manual creation function. Test procedure manual edit function that automatically edits the test procedure manual by adding commands as necessary, and based on the test procedure manual, automatically creates test commands corresponding to any debugger and executes the test. Then, the lower module used by the upper module to be tested is automatically connected to perform the bottom-up test, and the test procedure instruction file creation function unit for selecting the test method of the lower module and the expected value and the passing of the test result. / The test result judgment function that automatically judges pass / fail by comparing with the non-passing break point and outputs the test procedure with result, and the specified test procedure and the file with test procedure with result are existing printing tools. An automatic system for software testing, comprising a print function unit for starting up and performing standard printing. 2. Scheduling according to the test item obtained by the test item extracting function unit, the test procedure document creating function unit, the test procedure document editing function unit, the test procedure instruction file creating function unit, and the test result determination 2. The software test automation system according to claim 1, wherein the processing of the functional unit and the processing of the printing functional unit are successively performed. 3. The software test automation system according to claim 1 or 2, further comprising an additional function for automatically outputting the content of an abnormality as a message with respect to abnormality processing during setting of the test environment and during the test. A featured software testing automation system. 4. The automated system for software testing according to claim 1, wherein the test support system supervision section 7 supervising the operation of each of said functional sections has a test procedure manual used before and a test procedure manual newly created. And performs processing for inheriting the setting value information other than the changed portion, and also the test item extraction function unit, the test procedure document creation function unit, the test procedure instruction file creation function unit, and the test result determination function. Difference information collection part that backs up and restores the previous state of the part and the specified debugger type and converts the debugger command that matches the condition into the procedure manual language, and the test procedure instruction file The test conversion status is generated based on the reverse conversion unit that generates the test procedure document and the test result status table output from the test result determination function unit. A software test automation system comprising: a test status management unit that schedules tests.