JP2002251297A - Test process management method and system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide test progress management method and system by which the change of the transition relation of test items is suppressed to absolute minimum even in the case that the test items are changed during a test and a person easily performs the operations of increasing/decreasing the test items and changing a performance order or the like. SOLUTION: This method for managing the test progress of the test items having a hierarchical structure is provided with the step of inputting the information on the test items to be managed and the information on the performance order of the test items to a system, the step of preparing a transition chart for each hierarchy from the information of the test items and the information on the performance order that are inputted and the step of displaying the prepared transition chart for each hierarchy.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a software debugging method for a computer system, and more particularly to a test progress management method and system effective for improving debugging efficiency after a combination test.

[0002]

2. Description of the Related Art Various methods have been developed for managing the progress of a program test in software development of a computer system.
No. 4758 discloses a program test progress management method using a “program transition table” created in advance.

[0003]

However, in the above-described program test progress management method using the program transition table, all the test items are set in order to set the transition relation for each test item before the test starts. It is necessary to create a transition diagram as a node. Therefore, the transition diagram becomes complicated as the number of test items increases. Also, since the purpose is to automate the entire test process, it is difficult for humans to intervene and change the transition diagram when test items increase or decrease during the test. If the change is made, the effect may be significantly different from the one before the change over the entire area. In that case, the test must be restarted from the beginning, and the previous test results may not be useful.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems of the prior art, and minimizes the change in the transition relation of test items even when the test items change during the test. It is another object of the present invention to provide a test progress management method and system in which a person can easily perform operations such as increase and decrease of test items and change of an execution order.

[0005]

According to the first aspect of the present invention,
A method for managing a test progress of a test item having a hierarchical structure, comprising: inputting information of a test item to be managed and information of an execution order of the test item to a system; and information of the input test item. And a step of creating a transition diagram for each layer from the information on the execution order and displaying the created transition diagram for each layer.

According to a second aspect of the present invention, in the first aspect, the transition diagram is represented by a node and a path, and the node and the path graphically change according to the state of the test item. Features.

According to a third aspect of the present invention, there is provided a system for managing a test progress of a test item having a hierarchical structure, comprising means for reading information on the test items to be managed and information on the execution order of the test items. And a means for creating a transition diagram for each hierarchy from the read test item information and the execution order information, and a means for displaying the created transition diagram for each hierarchy.

According to a fourth aspect of the present invention, in the third aspect of the invention, there is provided a G for operating a test item in a transition diagram.
It has a UI.

According to a fifth aspect of the present invention, in the third aspect of the present invention, there is provided a method for inspecting a transition relation between test items.

According to a sixth aspect of the present invention, in the third aspect of the present invention, there is provided a device for retrieving a test item necessary for trial of a test item.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of a test progress management method and system according to the present invention will be described with reference to the drawings. Here, consider a test having a plurality of test items. The test items are test contents. The test is performed by trying each of the test items in a preset order. When all the test items have been tried, the test ends.

Hereinafter, the test items will be described. Test items shall have a hierarchical structure. The hierarchical relationship in the hierarchical structure indicates an inclusion relationship, and a child test item (hereinafter, referred to as “child test item”) is included in a parent test item (hereinafter, referred to as “parent test item”). However, the parent test item does not have its own test content, but is a group of child test items. FIG. 1 is a diagram showing an example of a hierarchical structure of test items. Test items C21, C22, and C23 are grouped, and 1
One test item C2 and one test item C are grouped together with the test item C2 and other test items C1 and C3.

The execution order of the test items in the test is as follows.
It is represented as a transition diagram. The transition diagram has a start point and an end point, and the test is performed by trying a test item corresponding to a node from the start point along a route (path). Here, as for the transition of the trial of the test item, when the trial of the test item corresponding to the previous process is completed (obtains the expected result), the test item corresponding to the next process is subjected to the trial. When all the test items in the preceding process at the end point are completed, the test ends.

Regarding the test and the transition of the test item,
This will be described with reference to FIG. 2 showing an example of a transition diagram. First, test item A is tried. When the trial of the test item A ends, the target of the trial moves to the test item B and the test item C. Since the test item B and the test item C are in a parallel relationship, the result of one test item does not affect the result of the other test item. Test item D cannot be tried unless both test item B and test item C are completed. However, there is a case where a trial is forcibly performed as described later. When the trial of test item D is completed,
The test is over.

Next, the parent-child relationship of the test items will be described. A parent test item having a test item has a transition diagram with child test items as nodes. This indicates the execution order of the child test items included in the parent test item. FIG.
FIG. 4 is a transition diagram of the test item C in FIGS. According to FIG. 1, since the test item C2 has a test item, the test item C2 also has a transition diagram. If all child test items are not completed, the parent test item is not considered completed. That is, in FIG. 3, the child test items C1,
When all of C2 and C3 are completed, the parent test item C is also completed.

The transition diagram can include test items of other hierarchies. For example, it is assumed that the test item D is a test item that can be started when the trial of the test item C23 is actually completed. In this case, considering the ease of test management, it is desirable to try the test item D after the trial of the test item C including the test item C23 is completed. However, as shown in FIG. 4, although the test item C23 has been completed, a failure has occurred in another test item C1 in the test item C (that is, the test item C23 has not been completed). Has stopped. This is because the transition diagram specifies that the trial of the test item D cannot be started unless all the test items in the test item C are completed.

Here, when it is desired to start the trial of the test item D without waiting for the handling of the failure of the test item C1, the problem is solved by including the test item C23 in the transition diagram of the test item D. For example, when the test item D2 in the test item D requests the end of the test item C23, as shown in FIG. 5, in the transition diagram, the test item C2 is placed before the test item D2.
3 (the parent test item C of the test item C23).
2 has also been completed, so the test item C2 may be arranged before the test item D2). Note that test item C23
Are arranged in two places in the test item C2 and the test item D, but the test item C23 itself is one and is referred to only from two places. Therefore, if the process is completed at one of the two locations, the other references are also completed, and the trial is performed only once.

As described above, the distribution of references to the next test item when the test is stopped due to a failure in the test item can be automated as follows. First, a required test item list is created for each test item. The necessary test item list is a list of test items required to start the trial of the test item (preferably a terminal test item having no transition diagram) (test item D2 shown in FIG. 5).
Is a list including the test item C23). By dynamically changing the transition relation based on the list when the test item in the previous process does not end due to a failure, the test can proceed based on the changed transition relation.

However, even if all the test items in the transition diagram are terminated due to the change of the transition relation, the parent test item is terminated with "suspicion". This “suspicion” means that the test item was terminated due to a change in the transition relationship,
That is, it indicates that a failure has occurred in the test item in the previous process. Further, the information of “suspicious” is propagated when the test of the next process is completed. Therefore, even if the end point of the transition diagram is reached in the “suspicious” end state, the parent test item is not regarded as the end.

Next, interference between test items will be described. FIG. 6 shows the relationship between test item B and test item C waiting for each other to end. In this case,
If the transition relation between the test item B and the test item C is as shown in FIG. 7, deadlock occurs.
Therefore, the transition relationship between the test item B and the test item C in the case of the relationship shown in FIG. 6 must be constructed as shown in FIG. Of course, it is not permissible to construct a looping transition relationship.

Hereinafter, a test progress management system according to the present invention will be described. FIG. 9 is a diagram showing an embodiment of the configuration of the test progress management system according to the present invention.
Reference numeral 1 indicates a test progress management system (hereinafter, referred to as “system”) according to the present invention. The system 1 includes a test result determiner 2, a next candidate searcher 3, a necessary item searcher 4, and a transition diagram creator 5 therein.

The test result determiner 2 determines the result of the trial of the test item and the state of the test item. The status of the test item includes "test not possible", which is not the stage where trial can be performed, "test waiting", which is the stage where trial can be performed, and "testing", which is the stage where trial is being performed. "Failure" where the result of the trial is different from the expected one, and "end" where the result of the trial is the same as the expected one. As an example of the determination, when all the trials of the child test items are completed, the test result determiner 2 determines that the parent test item is “finished”.

When the trial of the test item is completed, the next candidate searcher 3 searches for a test item to be tried next.
Further, the next candidate searcher 3 examines the transition relation when the test item table and the transition relation table are read, or when a test item change operation is performed. As an inspection method, the corrected test item is marked, the next candidate test item is traced along the path, and whether or not there is a test item marked at the tracing destination is taken up. A loop can be determined.

The necessary item searcher 4 searches for a test item to be performed from a list of required test items of the test item before attempting a test item.
This is used together with the transition diagram creator 5 when forcibly trying a test item in the “waiting for test” state.

Here, the state of the test item when the test item is forcibly tried will be described. In the case where the transition relationship between the test items A, B, and C is A → B → C, the next test item B is forcibly executed and “End” even though the test item A is “failure”. ", The state of the test item B is set to an" end "state to which" suspicious "information is added. The information of “with suspicion” indicates that there is a remaining problem (unsupported obstacle) in the previous process. Test item B with "suspicious" information added
Is in the "finished" state, the next test item C can be tried. However, the information of "with suspicion" is propagated, and the information of "with suspicion" is added even when the test item C is "finished". When the result of the test item A is “finished” and the result of the retry of the test item B is “finished”, the “suspicious”
Is canceled. Similarly, for the test item C, when the result of the retry is “end”, the information of “suspicious” is released.

The transition diagram creator 5 is used when a test item table and a transition relationship table are read, and when there is a change in the transition relationship such as an increase or decrease in test items, and creates a transition diagram. The transition diagram creator 5 creates a transition diagram so that test items that do not interfere with each other, that is, have no transition relation, can be executed in parallel.

Here, the execution order of each test item is A = 1, B = 5, C = 5, D = 10 for each hierarchy, and an absolute order relation is given by assigning a numerical value to each test item. Or B = A, C = A, D = B
After C and C, a relative order relation may be given.

Hereinafter, a test progress management method according to the present invention using the system 1 will be described. First, the system 1 uses a test item table 10
And the transition relation table 20 are read as initial input information. Here, the test item table 10 is a table having information on all test items managed by the system 1 for progress. The transition relation table 20 is a table having information on the execution order of test items, and refers to child test items of each test item (that is, reference to test items in the test item table 10), Manage the transition state of items. After reading the test item table 10 and the transition relation table 20, the system 1 uses the transition diagram creator 5 to create a transition diagram for each layer.

The system 1 provides a user of the system 1 (hereinafter, referred to as “user”) with a function described below via a graphical user interface (GUI).

The system 1 displays the transition diagram of the uppermost hierarchy in the main window using nodes and paths using a means (not shown) so that the user can browse the transition relationship. Nodes represent test items and are displayed as icons in the main window,
The path indicates a transition relationship between test items, and is displayed as a line in the main window. The system 1 makes it easier for the user to grasp the progress of the test by changing the display of the nodes and paths graphically (including the color change) according to the state of the test item and displaying it.

According to the above-described embodiment, there is provided a method for managing the test progress of test items having a hierarchical structure, wherein information on the test items to be managed and information on the execution order of the test items are stored in a system. 1; a step of creating a transition diagram for each layer from the input test item information and the execution order information; and a step of displaying the created transition diagram for each layer. It is possible to manage the test items by regarding the test items within the test item as one test item, and to grasp an overview of the progress of the test.

Hereinafter, operations that can be performed on the system 1 by the user via the user interface of the system 1 will be described.

When the user selects a node displayed on the main window using a pointing device, the system 1
Displays a child window, and displays the transition diagram of the test item corresponding to the selected node in the child window. The contents displayed on the child window and the method of operating the child window by the user are the same as those of the main window.

When the user selects a node in a window (general term for a main window and a child window) and drags and drops it to another node, the system 1 creates a new path.

The user selects the path displayed on the window,
When a test item addition instruction is input, the system 1 adds a new test item on the path.

When the user selects a node displayed on the window and inputs a node deletion instruction, the system 1 deletes a test item corresponding to the selected node.

When the user drags and drops a node on the path displayed in the window, the system 1 changes the order of trial of the corresponding test item.

The operation of the transition diagram by the user is not limited to the operation within one window, and the node can be moved or copied between the transition diagrams in the two windows.

A required test item list is associated with each node. In the required test item list, the user can directly write the name of the test item, or the user can add a node to the required item list by dragging and dropping the node into the required item list.

According to the above-described embodiment, there is provided a system for managing the test progress of test items having a hierarchical structure, in which information on the test items to be managed and information on the execution order of the test items are read. Means, a transition diagram creator 5 for creating a transition diagram for each hierarchy from the read test item information and execution order information, and a means for displaying the created transition diagram for each hierarchy 1
By using the test item, the test items in the hierarchy can be regarded as one test item, the test items can be managed, and an overview of the progress of the test can be grasped.

The system 1 has a GUI for operating the test items in the transition diagram.
It is easy to perform operations such as increasing or decreasing the test items and changing the execution order. Since test items have a hierarchical structure,
Even when a test item changes during the test, it is possible to minimize the change in the transition relation of the test item. That is, the test can be restarted by taking over the previous test results.

Further, since the system 1 has the next candidate searcher 3 for examining the transition relation of the test item, if the transition relation of the test item is looped, it can be determined.
It is possible to avoid a deadlock of a test item after the start of the test.

Further, since the system 1 has the necessary item searcher 4 for searching for a test item necessary for trial of a test item, if a failure occurs in the test item, another test item is searched. Even if a forcible attempt is made, the test can be completed without overlooking the occurrence of the failure.

[0044]

According to the first aspect of the present invention, there is provided a method for managing the test progress of a test item having a hierarchical structure, wherein information on the test items to be managed and information on the execution order of the test items are provided. To the system, a step of creating a transition diagram for each hierarchy from the information of the input test items and information of the execution order, and a step of displaying the created transition diagram for each hierarchy, Test items in the hierarchy can be regarded as one test item and the test items can be managed, and an overview of the progress of the test can be grasped.

According to the second aspect of the present invention, in the first aspect of the present invention, the transition diagram is represented by a node and a path, and the node and the path graphically change according to the state of the test item. This makes it easier to grasp the progress of the test.

According to the third aspect of the present invention, there is provided a system for managing the test progress of test items having a hierarchical structure, wherein information on the test items to be managed and information on the execution order of the test items are read. Means for creating a transition diagram for each layer from the read test item information and execution order information, and means for displaying the created transition diagram for each layer. The test items can be managed by regarding the test item as one test item, and an overview of the progress of the test can be grasped.

According to the fourth aspect of the present invention, in the third aspect of the present invention, a GUI for operating the test items in the transition diagram is provided, so that operations such as increase / decrease of the test items and change of the execution order are performed. Easy to do. Since the test items have a hierarchical structure, even if the test items change during the test, it is possible to minimize the change in the transition relation of the test items. In other words, the test can be resumed with the previous test result taken over

According to the fifth aspect of the present invention, in the third aspect of the present invention, by determining the transition relation of the test item, it is possible to determine if the transition relation of the test item is looped. It is possible to prevent a test item from becoming deadlocked after the start of the test.

According to the sixth aspect of the present invention, in the third aspect of the present invention, when a failure has occurred in a test item, a means for searching for a test item necessary for trial of the test item is provided. Even if another test item is forcibly attempted, the test can be completed without overlooking the occurrence of the failure.

[Brief description of the drawings]

FIG. 1 is a diagram showing an example of a hierarchical structure of test items used in a test progress management method and system according to the present invention.

FIG. 2 is a diagram showing an example of a transition diagram of test items used in the test progress management method and system according to the present invention.

FIG. 3 is a transition diagram of a test item C shown in FIGS. 1 and 2;

FIG. 4 is a diagram illustrating an example of a hierarchical structure of test items in which a test of a parent test item is stopped because a part of a child test item has a failure.

FIG. 5 is a diagram illustrating an example of a transition diagram of a test item when a test item is moved to a transition diagram of another test item.

FIG. 6 is a diagram illustrating an example of a hierarchical structure of test items having a mutual interference relationship;

FIG. 7 is a diagram illustrating an example of a transition relationship between test items in which a deadlock occurs.

FIG. 8 is a diagram showing an example of a transition diagram of test items having a hierarchical structure shown in FIG. 6;

FIG. 9 is a diagram showing an example of an embodiment of a structure of a test progress management system according to the present invention.

[Explanation of symbols]

 1 Test Progress Management System 2 Test Result Judgment Device 3 Secondary Candidate Search Device 4 Necessary Item Search Device 5 Transition Diagram Creator 10 Test Item Table 20 Transition Relationship Table

Claims (6)

[Claims] 1. A method for managing a test progress of a test item having a hierarchical structure, comprising: inputting information on a test item to be managed and information on an execution order of the test item to a system; A step of creating a transition diagram for each hierarchy from the information on the test items and the information on the execution order, and a step of displaying the created transition diagram for each hierarchy. 2. The test progress management method according to claim 1, wherein the transition diagram is represented by a node and a path, and the node and the path graphically change according to a state of the test item. 3. A system for managing the test progress of test items having a hierarchical structure, comprising: means for reading information on test items to be managed and information on the execution order of the test items; A test progress management system comprising: means for creating a transition diagram for each hierarchy from the information of the above and execution order information; and means for displaying the created transition diagram for each hierarchy. 4. The test progress management system according to claim 3, further comprising a GUI for operating a test item in the transition diagram. 5. The test progress management system according to claim 3, further comprising means for examining a transition relation between test items. 6. The test progress management system according to claim 3, further comprising means for searching for a test item necessary for trial of the test item.