JPH09218805A - Test control method for communication software - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform the automatic test of a communication software. SOLUTION: A test item management tool 1 prepares a test item management file 8 and a procedure preparation tool 2 prepares a procedure management file 9 (S21). A test item selection tool 3 prepares a procedure file 11 (S22). An automatic test tool 4 performs the automatic test for all the test items of the procedure file 11 and writes a test log in a test log file 12 (S23 and S24) and a test result file is prepared (S25). The test item management tool 1 prepares a test report list 14 or the like (S26). Test confirmation and problem registration are performed (S27), a problem processing is performed (S28 and S29), file updating is performed (S30) and control to perform repetition until the test is ended for all the test items is performed (S31).

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is applied to a test operation of communication software, and in the test of communication software, a test item management tool which is a test-related tool, a procedure manual tool, a test item selection tool, an automatic tool. The present invention relates to a communication software test control method for improving the efficiency of the entire test work by providing a function of interlocking a test tool, a test result collation tool, a report tool, and a problem processing management tool without human intervention.

[0002]

2. Description of the Related Art Conventionally, techniques in such a field include:
For example, there is one described in the following literature. Reference 1: IEICE Technical Report, SSE92-85, 1992, Takada et al., "Software verification method using automatic test system", P. 13-18 Ref. 2; IEICE Technical Report, SSE94-89, 1994, Sugawara et al., "Exchange Software Testing Support System Focusing on Information Sharing, 1994, P.49-54 REFERENCE 3; Report, SSE94-106, 1994,
Murakami et al., "Individual test support system focusing on objectivity of test", 1994, p. 93-98 With the increase of additional functions and changes in electronic exchanges,
The software is also increasing in scale and complexity. On the other hand, since the exchange is required to have a high degree of reliability and a large number of man-hours are required for the verification test, various tools and methods have been conventionally used for the purpose of improving the efficiency of the test.

FIG. 2 is a diagram showing a communication software test control method described in the documents 1 to 3. A conventional test control method for communication software will be described below with reference to FIG. As shown in FIG.
In step 1, create a procedure manual describing the test contents using the procedure manual creation tool. In step S2, a test item selection work is performed as a preparatory stage of the test work. In step S3, the test of the selected test item is performed using the automatic test tool, and in step S4,
It is judged whether or not there is a remaining test, and if there is a remaining test, the process returns to step S4 to carry out the remaining test. In step S5, it is analyzed whether or not the test result is correct, and in step S6, the test result report is created from the analysis result of the test by using the report tool and the progress management tool, and the problem processing management tool is executed. Create a problem processing management file to manage the problems that occurred in the test. In step S7, it is determined whether or not the problem processing remains, and if the problem processing remains, the problem processing is performed in step S8. In step S9, the problem processing management file for the result of the problem processing is updated. If the problem processing remains, in step S10, it is determined whether or not all the test items are completed. If not completed, the process returns to step S1. If completed, the communication software test is completed.

[0004]

However, the conventional communication software test control method has the following problems. For the test item selection work, which is the test preparation work, manually complete the test item with the progress management tool, check the problem processing / not yet started item, and check the problem processing item with the problem processing management tool. It was time-consuming to check the analyzed items, select the problem-solved items and untestable testable items as test items, and confirm the preparation of the procedure manual. Regarding the verification after the test result, the test result collation tool can confirm the normality of the test, but the work report creation for the test result, the registration of the progress result, and the problem process registration work when a problem occurs Everything was done manually.

[0005]

[Means for Solving the Problems] In order to solve the above problems, the first invention provides information on whether or not each test item can be tested for all test items and the progress of the test items. To link with a problem processing management file that stores information indicating the test execution status, the problem processing number given when the test is executed, the content of the problem, and the processing status of the problem Information describing the test item management process for creating a test item management file for storing the problem process number managed by the problem process management file generated by performing the test, and the test procedure for all the test items A procedure manual management file creation process for creating a procedure manual management file for storing is executed.
Then, for each test item of the test item management file, information on whether the test of the test item can be performed, the test execution state, and the problem processing indicated by the problem number generated in the test of the test item. Based on the processing status stored in the management file, it is determined whether or not the test can be performed. A test log that creates a test log file of test items by performing an automatic test that performs the test of the test item according to the test item selection process of registering the contents in the procedure file and the procedure manual of the test item of the procedure file. Executes file creation processing. Since the test control method of the communication software is configured as described above, the test item management file has information for judging the progress of the test and whether the test can be performed. The test procedure is registered in the procedure manual management file, the test item management file is used to judge whether the test can be performed, and the test item is automatically selected to create the procedure manual file. Then, an automatic test is performed using the procedure manual file.

[0006]

1 is a diagram showing a communication software test control method according to an embodiment of the present invention. The test control method of the communication software of the present embodiment is different from the test control method of the conventional communication software in that the test item selection work using the test item selection tool, the test execution by the automatic test tool, the test result verification tool This is the automation of a series of operations from result analysis to report of test results using a report tool. As shown in FIG. 1, the communication software test control method includes a test item management tool 1,
Procedure manual creation tool 2, test item selection tool 3, automatic test tool 4, test result collation tool 5, report tool 6,
Using the problem processing management tool 7, test item management file 8, procedure manual management file 9, problem processing management file 1
0, procedure manual file 11, test log file 12, test result file 13, test result list 14 creation / reference
Implement while updating. The test item management tool 1 creates and updates a test item management file 8 for managing test items through a user interface.
The procedure manual creating tool 2 refers to the test item management file 8 and refers to the test item management file 8 to show the test procedure of the test item.
Is created and updated.

The test item selection tool 3 refers to the test item management file 8, the procedure manual management file 9, and the problem handling management file 10 to judge whether or not to carry out the test, and carry out the test. For the test items to be performed, the procedure manual file 11 indicating the test procedure is created. The automatic test tool 4 performs a test according to the test procedure of the test item shown in the procedure manual file 11 and writes the test result in the test log file 12. The test result collation tool 5 refers to the test log file 12 and the correct data in the test item management file 8 and writes the test results of the test items into the test result file 13. The report tool 6 refers to the test result file 13, creates a test report list, and updates the test item management file 8 and the problem processing management file 10 from the test result. The problem processing management tool 7 identifies the test items that need to be processed from the test results, performs the problem processing, and creates and updates the problem processing management file 10.

3A to 3F are block diagrams of the test item management file. FIG. 3A is a schematic configuration diagram of the test item management file. As shown in FIG. 3A, the test item management file 8 is composed of a large item table, a medium item table, and a small item table, and manages in three stages of large / medium / small items. Shows. FIG. 3B is a diagram showing the contents of the large item table. As shown in FIG. 3B, in the large item table, the number of large items in the header section, the maximum item number, and the presence / absence of the large item mounting display for each large item (to set this, It is necessary to check whether or not the test item is missing from all the items, which is to prevent the test item from being missed), and a pointer to the middle item table is stored. FIG. 3C is a diagram showing the contents of the middle item table. FIG.
As shown in (c), in the medium item table, the number of medium items belonging to the large item in the header part, the maximum item number, and the presence / absence of the medium item implementation display for each medium item (the same as the implementation of the large item implementation) It is a purpose), and a pointer to a small item table belonging to the middle item is stored.

FIG. 3D shows the contents of the small item table. As shown in FIG. 3D, in the small item table, the number of small items in the header part, the maximum item number, the presence / absence of the small item mounting display, and the test execution availability display (test item However, if the hardware is not ready, etc.), the test date, the test completion date (the date when the test was completed normally), and the total number of problems that occurred in the test for that test item. , Pointer to the question number table, correct answer data implementation flag indicating whether correct answer data is registered,
Pointer to the correct answer data, test execution display (new test that tests the test item for the first time from now on, correction test that the problem has occurred due to the test execution and correction check test will be performed after that problem has been addressed, specified by the start option of the user interface The test of this test item has been completed normally, but a problem has occurred in another test item, and whether or not this test item has not been degraded) is stored. FIG. 3E is a diagram showing the contents of the problem processing number table. As shown in FIG. 3E, in the problem processing number table, the number of problems generated in the test and the problem processing number managed by the problem processing management tool for each problem number are stored in the header part. The question number is a number managed by the problem management tool 7, and by this, the test item and the problem processing are related. FIG. 3F is a diagram showing the content of the correct answer data. As shown in FIG. 3 (f), the correct answer data stores transmission / reception data of a signal assumed as data for determining whether the test result is correct.

FIGS. 4A to 4F are configuration diagrams of the procedure manual management file. FIG. 4A is a schematic configuration diagram of the procedure manual management file. As shown in FIG. 4A, the procedure manual management file 9 is composed of a large item table, a medium item table, a small item table, a procedure manual control data and a procedure manual, and has three stages of large / medium / small items. Indicates that it is managed by. This indicates that the test item and the procedure manual correspond to each other by the test item number. FIG. 4B is a diagram showing the contents of the large item table. As shown in FIG. 4B, in the large item table, the number of items of the large item is stored in the header portion, and a pointer to the middle item table is stored for each large item. FIG. 4C is a diagram showing the contents of the middle item table. As shown in FIG. 4C, in the middle item table, the number of middle items belonging to the large item is stored in the header portion, and for each middle item, a pointer to the small item table belonging to the middle item is stored. . FIG. 4D is a diagram showing the contents of the small item table. As shown in FIG. 4D, in the small item table, the number of small items in the header part, the test group name, the person in charge of the test, the presence or absence of the procedure manual implementation for each small item, and the procedure manual control data The pointer is stored.

FIG. 4 (e) is a diagram showing the contents of the procedure manual control data. As shown in FIG. 4 (e), the procedure manual control data includes a procedure manual control name in the header portion, the number of procedure manuals for performing a test, a procedure manual name for each procedure manual, and a test procedure manual in time series. The pointer is stored. Figure 4 (f) shows
It is a figure which shows the content of the procedure manual. The hardware-embedded device such as a communication device includes a case where a test is performed using the device main body and a simulation test where a workstation is used. Testing using the device itself involves reading the program to be tested by an in-circuit emulator (hereinafter referred to as ICE) to the signal device, reading and writing to memory, program trace control procedures, telephone off-hook, etc. There are three types of signal input / output operation procedures by a pseudo signal device (hereinafter, pseudo call) to be performed and an operation / maintenance command input procedure such as registration of a telephone number. The test using the Worth Station is the loading of the problem target program by the simulation debugger, the reading and writing to the memory etc., the control procedure of the program trace etc., the input / output control of the signal of the communication device by the signal device simulator, the operation / maintenance command. There are three types of input procedure. Therefore, as shown in FIG. 4F, each test procedure is defined in the procedure manual data,
It stores the control procedure of the ICE or the simulation debugger, the input / output control of the signal of the communication device by the pseudo call or the signal device simulator, and the operation / maintenance command input procedure.

FIGS. 5A to 5C are configuration diagrams of the problem processing management file. FIG. 5A is a schematic configuration diagram of the question management file. As shown in FIG. 5A, the question management file 10 is composed of a question management table and a related test number table. FIG. 5B is a diagram showing the contents of the question management table. As shown in FIG. 5B, in the problem management table, the number of problem registrations in the header part, the content of the problem, the processing status (unprocessed, corrected, pending, dead,
(Confirmation), a pointer to the related test number table, etc. are stored. FIG. 5C is a diagram showing the contents of the related test number table. As shown in FIG. 5C, the number of test items related to each question number and the related test item number (large / large)
Stores medium / small items. As a result, the problem number is associated with the test item, the problem processing status of the test item is known, and it is possible to select the test item and automatically manage the progress of the test.

FIGS. 6A to 6D are block diagrams of the procedure manual management file. FIG. 6A is a schematic configuration diagram of the procedure manual file. As shown in FIG. 6A, the procedure manual file 10 is composed of a test table, procedure manual control data, and a procedure manual. FIG. 6B is a diagram showing the contents of the test table. As shown in FIG. 6B, in the test table, the header part has the procedure manual control data name, the number of tests to perform the automatic test, the test item number for each test management data, and the pointer to the procedure manual control data. Etc. are stored. FIG. 6 (c)
FIG. 6 is a diagram showing the contents of procedure manual control data. FIG.
The procedure sheet control data in (c) is the same as the procedure sheet control data in FIG. 5 (e). FIG. 6D is a diagram showing the contents of the procedure manual. The procedure manual of FIG. 6D has the same contents as the procedure manual of FIG.

FIG. 7 is a configuration diagram of the test result log file. The test result log file 12 is. This is a file in which test log information is written sequentially. As shown in FIG. 7, the test information and the log information are stored in the test result log file 12, and the test information includes the test item number and the test item number.
For test start date / time, test end date / time, normal route passage / procedure document abnormal route passage / abnormal end display, and log information, log information of ICE or simulation debugger (contents of memory contents, trace information, etc.), pseudo call Alternatively, log information of the signal device simulator (contents of input / output signals of the signal device, etc.) and log information of operation / maintenance commands (input command and output message, etc.) are stored.
The test items that are defined at the top and the test start date and time are keywords of the test items in the test result matching tool 5. FIG. 8 is a configuration diagram of a test result file. The test result file 13 is sequentially created in the order of tests, like the test log file 12. As shown in FIG. 8, the test information and auxiliary information are stored in the test result file 13. For the test information, the test item number, the test result (normal / abnormal / unmatchable), etc.
It stores the pointer to the log file corresponding to the test item, and information at the time of abnormality (information at the time of mismatch with correct answer data, information at abnormal route of procedure manual, information at abnormal end).

FIG. 9 shows the contents of the test report list. As shown in FIG. 9, the test report list includes a test start date, a test end date, a confirmation file name, a test target program name, a procedure manual file name, a test log file name, a test result file name, and a new test as headings. The test results are printed in the order of the list, the degradation test result list, the correction confirmation test result list, and the test result confirmation waiting list of the test items other than these. As a result, the test progress status of all test items can be confirmed in a list. For new tests, the total number of new test items, the number of normally confirmed items (test item number that was normally confirmed), the number of items that did not match the test collation (the test item number that did not match the test collation, a pointer to the test result file),
The number of items passed through the abnormal route of the test procedure (the number of the test item that passed the abnormal route of the test procedure and the pointer to the test result file) and the number of items that ended the test abnormally (the abnormal test item and the pointer to the test result file) are printed. .

Regarding the degrading test, the total number of items of the degrading test, the number of normally confirmed items (test item number of normally confirmed), and the number of items of mismatched test collation (the number of items of the test collation disagreement, a pointer to the test result file) , The number of items passed through the abnormal route in the test procedure manual (the number of the test item that passed the abnormal route in the test procedure manual, a pointer to the test result file) and the number of items ended abnormally in the test (test abnormal item, a pointer to the test result file) are printed. There is. Regarding the modification test, the total number of modification test items, the number of normally confirmed items (test item number that was normally confirmed), the number of items that do not match the test collation (the test item number when the test collation does not match, a pointer to the test result file), the test procedure document Number of items that passed abnormal route (test item number that passed abnormal route of test procedure, pointer to test result file), number of items that ended abnormally (test abnormal item,
A pointer to the test result file) is printed. As for the test result confirmation waiting, the number of test result confirmation waiting items (the number of test items waiting for confirmation, a pointer to the test result file) is printed. FIG. 10 is a flowchart showing a test item selection flow.

FIG. 11 is a diagram showing a test form using a communication system main body for carrying out the communication software test control method of the present invention. As shown in FIG. 11, the communication software test control method using the communication system main body includes a communication system main body 21, an ICE 22, a pseudo call device 23, a console 24, and a workstation 25. The workstation 25 has an automatic test tool 4,
A test log file 5 and a procedure manual file 11 are installed. The workstation 25 is connected to the communication system main body 21 such as a PBX, the IE 22, the pseudo call device 23, and the console 24, and is connected through these for testing. FIG. 12 is a diagram showing a simulation test form by a workstation for carrying out the communication software test control method of the present invention. As shown in FIG. 12, the test control method of the communication software includes a communication system program 31, a simulation debugger 32, a signal device simulator 33, a pseudo console program 34, an automatic test tool 4, a test log file 12 installed in the workstation 30. , Procedure file 1
1. The program communication system program 31 to be tested is the simulation debugger 3
2. The test is performed on the workstation through the signal device processing simulator 33 and the pseudo console program 34.

FIG. 12 is a flow chart showing the test result collation flow. Hereinafter, the communication software control method according to the embodiment of the present invention will be described with reference to these drawings. (A) Test management process The test item management tool 1 creates a large / medium / small item table and correct answer data from the test item management file 8 shown in FIG. Number, maximum item number, small item mounting display, small item name, test execution availability display,
Correct data mounting display, if correct data is already set, only set the pointer to the correct data. (B) Procedure manual management process The procedure manual creation tool 2 creates a large / medium / small item table of the procedure manual management file 9, procedure manual control data, and a procedure manual. The large / medium / small item table is created and managed based on the test item management file 8, and whether or not the procedure manual control data and the procedure manual are created is managed by the procedure manual mounted display.

(C) Creation / Update of Procedure Manual File 11 The operation flow of the test item selection tool 3 will be described with reference to FIG. In step S41, the test item number is set to the smallest number of the large item, the medium item, and the small item. In step S42, it is determined whether or not a small item in the management table of the test item management file 8 can be tested and the procedure manual corresponding to the test item number is mounted in the procedure manual management file 9, and the test cannot be performed. If so, the process proceeds to step S50, and if the test is possible, the process proceeds to step S43. In step S43, it is determined whether or not the implementation date of the item management table of the test item management file 8 is not set. If it is already set, the new test is finished, so the process proceeds to step S44, and it is not set. For example, since this is a new test for the first time, the process proceeds to step S45. In step S44, the completion date of the small item table of the test item management file 8 has been set (the test item has been completed normally), and the start-up parameter set by the test item selection tool 3 through the user interface is the degraded test. (The applicable test item has been completed normally, but the test item for which other problems have been dealt with should be tested to see if the applicable test item has been degraded.) In the case, the process proceeds to step S46, and in the case of performing the degradation test, the process proceeds to step S47.

In step S45, a new test is set in the test execution display, and the flow advances to step S49. In step S46, one of fixed / solved / diminished is set in the problem processing data of the problem processing management file 10 corresponding to the unsolved problem number in the problem processing number table of the test item management file 8 (in these cases, there is a problem. It is being processed and is in the state of confirmation of correction)
If the problem has been dealt with, the process proceeds to step S48 to set the correction confirmation test for the test item for which the problem has been dealt with. If the problem has not been dealt with, the correction confirmation test cannot be performed, so the process proceeds to step S50. In step S47, in order to carry out the degradation test, the degradation test is set in the test execution display, and the process proceeds to step S49. Step S
At 48, since the problem processing is confirmed, the correction confirmation test is set in the test execution display of the small item table.

In step S49, the test execution display is set in the small item management table of the test item management file 8, the procedure manual control data of the procedure manual management file 9 and the contents of the procedure manual are registered in the procedure manual file, and step S50 is performed. Proceed to. As a result, a procedure manual for performing a new test, a degradation test, and a problem correction confirmation test is registered. This procedure manual is not created because a new procedure manual is created every time a test is performed and it is meaningless to test a test item awaiting revision confirmation. Furthermore, even if the problem is resolved to other test items after normal termination, I do not want to perform a degrading test on the normally completed test item (for example, after finishing all the test items for the time being, I will (If you want to do it), if you do not perform the degrade with the startup option, the procedure manual will not be created for the test items that completed normally, and the test will not be performed. In step S50, the small item number is counted up to indicate the next small item, and the process proceeds to step S51. In step S51, it is determined whether or not all the small items in the middle item unit have been completed. If not, the process returns to step S42 to set the remaining small items in the middle item.

In step S52, the middle item number is incremented to indicate the next middle item. Step S5
In 3, it is determined whether or not all the middle items of the large item have been completed. If not, the process returns to step S42 to set the remaining middle items of the large items. Step S
At 54, the major item number is counted up to indicate the next major item, and the process proceeds to step S55. Step S5
In step 5, it is determined whether all major items have been completed. If not, the process returns to step S42 to set the next major item. Through the above steps S41 to S55, for the large items / medium items / small items of the test item management file 8, the test execution date, test completion date, and start parameter According to the management table of the degrading test execution / problem processing file 10, the display of the new test, the degrading test, and the correction confirmation are set in the test execution display. Will be registered in the procedure manual file 11, and the test of each small item will be carried out.

By the above processes (a) to (c), the test is ready. (D) Test Implementation A test using the communication software body shown in FIG. 11 will be described. The automatic test tool 4 uses the procedure file 11
Based on the test control data of the procedure manual, first, the control command for the ICE22 is transmitted and received according to the ICE procedure manual, and the contents of the transmission and reception are sequentially output to the test log file 12, and then the console 24 operates according to the procedure manual.・
The transmission and reception of the maintenance command and the contents of the transmission and reception are sequentially output to the test log file 12. Then, according to the pseudo call procedure manual, for the pseudo call, the procedure manual is transferred, the pseudo call is activated, the transmission / reception of the pseudo call and the signal transmission / reception data of the communication system 21, and the contents of the transmission / reception are sequentially output to the test log file 12. Run in order and test. Next, a simulation test by the workstation shown in FIG. 12 will be described. Based on the test control data of the procedure manual of the procedure manual file 11, the automatic test tool 4 first transmits / receives control commands to / from the simulation debugger 32 according to the procedure manual of the simulation debugger 32, and the contents of transmission / reception to the test log file 12 sequentially. Then, according to the procedure manual of the pseudo console program 34, the transmission / reception of the operation / maintenance command and the contents of the transmission / reception are sequentially output to the test log file 12.

Then, in accordance with the signal device simulator procedure manual, the procedure manual is transferred to the signal device simulator 33, the signal device simulator 33 is activated, the signal transmission / reception data of the signal device simulator 33 and the communication system program 31 is transmitted / received, and the data is transmitted / received. Test log file 1
Perform the test by sequentially executing the output in 2 order. Then, the automatic test tool 4 detects the occurrence of a problem such as passing a normal route during the operation of the procedure manual, passing an abnormal route of the procedure manual, and an abnormal end display based on the received data, and after collecting the data, shifts the control to the next test. The test is repeated by the number of tests registered in the test table of the procedure manual file 11.

(F) Test Result Analysis In step S25, the test result collation tool 5 performs the test result analysis, in accordance with the flow shown in FIG. 13, of the correct answer data and the test log file 12 registered in the test item management file 8 in advance. Match. Hereinafter, a detailed description will be given with reference to FIG. In step S61, a test item number is searched from the test log file 12, and in step S62, it is determined whether or not there is a test item number. If there is a test number, the test result of the test number is analyzed. Therefore, the process proceeds to step S62, and if there is no test item number, the process ends. In step S62, it is determined from the test information of the corresponding test item number in the test log file 12 whether or not the test is abnormally terminated during the test. If the test is not abnormally terminated, the process proceeds to step S64 and abnormally terminated. If so, the process proceeds to step S65. In step S64, it is determined during the test whether or not the procedure manual has passed an abnormal route (for example, when the transmission data cannot be received, the condition for passing the abnormal route is set in the procedure manual).
If the vehicle has not passed the abnormal route, the process proceeds to step S66. If the vehicle has passed the abnormal route, the process proceeds to step S67.

In step S65, the abnormal end is set in the item of the test result corresponding to the test item number in the test result file 13, and the abnormal end information and the corresponding in the test log file 12 at the time of the abnormal end are set in the auxiliary information items. A pointer to the log is set and the process proceeds to step S72. Step S
At 66, it is judged whether or not the correct answer data is registered in the test management file 8 corresponding to the test item number,
If the correct answer data is registered, the process proceeds to step S68, and if not registered, the process proceeds to step S69. In step S67, since the test item has passed the abnormal route, the test result of the test result file 13 is set to pass the abnormal route in the procedure manual, and the corresponding log in the test log file 12 when the abnormal route is passed is set. The pointer is set and the process proceeds to step S72. In step S68,
The registered correct answer data and the log information item of the test log file 12 are compared to determine whether or not there is a mismatch. If there is no mismatch, the process proceeds to step S70, and if there is a mismatch, It proceeds to step S71. However, for example, specific data, for example, data that is grouped by lines such as PBX and selected by rotation that connects lines in the order of groups is normal if the group exists. To consider.

In step S69, since the correct answer data is not registered, the item of the test result of the test result file 13 is set to "unmatchable", and the item of the auxiliary information is set to the pointer to the log information corresponding to the test item. Then, the process proceeds to step S72. In step S70, the test result item in the test result file 13 is set to the normal test result, and the process proceeds to step S72. In step S71,
The test result item in the test result file 13 is set to be inconsistent with the correct data, and the auxiliary information item is set to a pointer to log information corresponding to the test item to the inconsistent portion and a comparison list (repeated by the number of inconsistencies). Then, step S72
Proceed to. In step S72, test result file 1
3 is the test item number, test groove name, test person,
If there is a test result and auxiliary information, the auxiliary information is set, the process returns to step S62, the next test item is searched, and the test result of the test item is output to the test result file 13. Through the above steps S61 to S72, for each test item, it is judged from the test information and the log information in the test log file 12 whether the normal route is passed, the abnormal route of the procedure manual is passed, or the abnormal end is determined, and the test result is obtained. And the auxiliary information is written in the test result log file 12 to create the test result log file.

(G) Test Result Report After the test result analysis, in step S26 in FIG. 1, the report creation tool 6 uses the test result file 13 as a basis.
The following processing is performed on the test item management file 8, the problem processing file 10, and the test report list 14. -The test item management file 8 sets the test completion date of the small item table for normal items in units of test item numbers, updates the number of problems that occur when a test error occurs (adds and accumulates), and the problem processing number table. Then, the problem number registered in the next problem processing management file 10 is set. When the test execution display set in the sub-item table is a new test, when the test start date is set and the test is abnormal, it is created for the test item number registered in the next problem processing file 10. Set the problem number in the problem number table and the pointer to the related problem number table.

The problem processing management file 10 updates the number of problems in the problem management table when a test is abnormal, and sets the updated number of problems as a problem number in the problem processing data, the problem content, the problem occurrence date, the problem process registrant. Create a name and related test number table, set the pointer to the test number table, and set the test item number in the related test number table. As a result, it is possible to deal with the small items in the test item management file 8 from the question test number table. -The test report list 14 prints a test start date, a test end date, a confirmation file name, a procedure manual file name, and a test result file name as headings.
Print the degradation test result list, the correction confirmation result list, and the test result confirmation waiting list in this order. As a result, it is possible to know how many test items such as a new test, a degradation test, a correction confirmation test, and a waiting list for correction confirmation other than these, and to grasp the progress of the test.

(H) Test Confirmation / Problem Registration In step S27, based on the test result list 14, the test result confirmation waiting items are judged manually by checking the contents of the test result file 13 as normal or abnormal. Then, the test item management file 1 is updated by the test item management tool 1 (for example, if it is determined to be normal, the test completion date or the like is updated, and if it is determined to be abnormal, the problem processing number table is updated). The problem processing management tool 10 updates the problem processing management file 10 (for example, the processing status is corrected if it is determined to be normal). Furthermore, for abnormal test items, the problem processing management tool 7
The problem contents of the problem management file 10 are detailed and the problem process target process name is set.

(I) Problem processing In step S28, the person in charge of the problem processing target process name registered in the problem processing management file 10 analyzes the problem contents from the test result file 13 and the test result log file 12, and the information is If insufficient, retest to collect necessary information and analyze problem content. When a problem is found, the program is corrected (the unit test is performed) and the problem processing management tool 7 corrects the analysis contents of the problem management table of the problem processing management file 10, the name of the problem processing analyst, and the processing status. Set. If the problem is another process, the problem process management tool 7 changes the problem process target process name in the problem process management file 10. If the content of the problem occurs due to an error in the test procedure or the like, there is a problem in the procedure manual instead of the test items. Therefore, it is necessary to correct the procedure manual and perform the test again. Set the processing status to "Immortal". When a problem occurs in this way, the processing status of the problem that has occurred is set, and the processing status is linked by the problem processing number table of the small items in the test management file 8 so that the problem processing status is displayed when selecting a test. The procedure manual will be created or not created depending on the problem processing status. (J) File update In step S30, the program is corrected by a problem process or the like to perform a unit test or the like, and the process returns to step S22. Then, the procedure manual file 11 is newly created and the test is executed in order to perform the correction confirmation test and the degradation test on the test item in which the problem is corrected.

(L) Judgment of End of All Test Items In step S31, it is judged whether or not some of the small items in the test item management file 8 have no mounting display or the test cannot be executed. If there is not, then it is judged that all the test items have been completed, and the test is ended. As described above, according to the present embodiment, it is possible to automatically perform a series of tasks from test item selection of a communication software test task to test result report without human intervention. As compared with the conventional automatic work from test execution to test result analysis, the test work effect can be significantly enhanced. Regarding the selection of test items, the test item selection tool 3 automatically selects the items that can be tested including the execution / non-execution of the degradation test and automatically creates the procedure manual file 11 that is the input of the automatic test tool 4. The test items can be selected efficiently with. Regarding the test result report, the work can be shortened by creating a report of the test result by the report list 14 and automatically updating the test item management file and the problem processing management file regarding the test progress and problem processing.

[0033]

As described in detail above, the first to third embodiments
According to the invention, it becomes possible to automatically perform a series of tasks from the selection of test items of communication software test tasks to the report of test results without human intervention. Compared with automatic work up to test result analysis, the test work effect can be greatly enhanced.

[Brief description of drawings]

FIG. 1 is a diagram showing a communication software test control method according to an embodiment of the present invention.

FIG. 2 is a diagram showing a conventional communication software test control method.

FIG. 3 is a configuration diagram of a test item management file.

FIG. 4 is a configuration diagram of a procedure manual management file.

FIG. 5 is a configuration diagram of a problem processing management file.

FIG. 6 is a configuration diagram of a procedure manual file.

FIG. 7 is a diagram showing the contents of a test result log file.

FIG. 8 is a diagram showing the contents of a test result file.

FIG. 9 is a diagram showing the contents of a test report list.

FIG. 10 is a flowchart showing a test item selection flow.

FIG. 11 is a diagram showing a test configuration using a communication system main body.

FIG. 12 is a diagram showing a simulation test form by a workstation.

FIG. 13 is a flowchart showing a test result matching flow.

[Explanation of symbols]

1 Test Management Tool 2 Procedure Manual Creation Tool 3 Test Item Selection Tool 4 Automatic Test Tool 5 Test Result Matching Tool 6 Report Tool 7 Problem Handling Management Tool 8 Test Item Management File 9 Procedure Manual Management File 10 Problem Handling Management File 11 Procedure Manual File 12 Test log file 13 Test result file 14 Test report list

Claims (3)

[Claims] 1. For all test items, information indicating whether or not a test can be performed for each test item, information indicating a test execution state indicating the progress of the test item, and information that occurs when the test is executed A test item management file for storing the given problem handling number, the problem handling number of the test to link with the problem handling management file that stores information indicating the content of the problem and the processing status of the problem About the test item management process to create, a procedure document management file creation process to create a procedure document management file for storing information describing the test procedure of all the test items, and each test item of the test item management file , The information indicating whether or not the test of the test item can be performed, the state of the test execution, and the problem processing management flag indicated by the problem number generated in the test of the test item. Based on the processing status stored in the file, it is determined whether or not the test can be performed, and for the test item that can be tested, the contents of the procedure manual of the test item from the procedure manual management file. And a test item selection process of registering the test item in a procedure file,
A test control method for communication software, comprising: performing an automatic test for performing the test of the test item, and performing a test log file creation process of creating a test log file of the test item. 2. The test item information in the test item management file is given the correct answer data of the test, and the test log file and the correct answer data of the test are collated, and the test result is normal or abnormal. Test result collation processing to create a test result file indicating any of the following, based on the test result file, the test item in which a problem occurred at the time of a test abnormality is assigned a problem number, and the content of the problem and its test item number Is stored in the question management file, the question number is stored as the test item information in the test management item file, and the test that has been completed normally is the test item information in the test management item file. Stores information that indicates normal termination, classifies test items that ended normally and test items that ended abnormally, and creates a test report list And management, according to claim 1 Test method for controlling a communication software, wherein the run. 3. In the test item management process, the test execution status is a new test for the first time to test or abnormally ended, but a problem has been dealt with and a correction confirmation test or problem processing for confirming the problem is performed. Expressed as one of the degrading tests when confirming whether the degrading is performed.The report creation process is registered in the test item management file for the new test, the correction confirmation test, the degrading test, and other than these. The test control method for communication software according to claim 2, wherein all the test items that are set are classified to create the test report list.