JPH09223040A - System test supporting device for software and test scenario generator to be used for the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily execute the system test of software of a graphical user interface(GUI) base, to prevent the contents of the test from being omitted and to enable test design even without completing the mount of the software. SOLUTION: The test scenario generating means 4 is provided for generating a test scenario according to a structure specification file 1, state transition specification file 2 and individual state specification file 3 which are defined by a user in order to exactly expressing the software accompanying a GUI. Then, since the test scenario required for covering all the states and all the state transitions provided for the GUI part of this software is generated while using this state transition specification file 2, the test scenario is automatically generated only with necessity for the user to define the specification information of the software, and it is made not necessary to manually actually operate a mouse or a keyboard and to record the history of operations in advance.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a software system test support device and a test scenario generation device used for the system test support device. For example, a GUI (Graphical User I)
It is suitable for use as a test tool to support the system test of software with an nterface).

[0002]

2. Description of the Related Art In recent years, computer systems have been used in a wide variety of applications throughout the world, and their range of use is expanding more and more. The performance of CPUs used in this computer system has been dramatically improved, and high-precision displays such as bitmap displays have become popular.

Under such a background, software design is performed by using a character-based user interface.
There is a growing focus on things like building user interfaces that are easier to use. Therefore, recently, GUI-based systems have become mainstream.

Particularly, as the standard window system represented by the X window system has spread, the importance of GUI in software has become higher and higher. Therefore, the evaluation of the entire software by the user often depends on the evaluation of the GUI.

This GUI-based software has visual parts such as buttons, and the operations for them are mainly performed by an input device such as a mouse. Therefore, it is easy to obtain an intuitive understanding and is very easy for the user to use. Also,
The software has a high degree of freedom that can change the input state according to the relationship between the position and position of the pointer and the position and type of parts that appear on the screen.

[0006]

By the way, in the software development stage, a system test is generally performed to confirm whether the created software operates according to the specifications. For example, in a system test of software including a GUI, a mouse and a keyboard are manually operated to verify whether or not the output of the software conforms to the specifications. In some cases, a history of the above operation is recorded so that it can be used when a test is required again due to version upgrade or the like.

That is, the system test support device conventionally used in the X window system temporarily records the operation of the mouse and keyboard manually,
It was a so-called capture & playback that reproduces it and verifies the specifications. For this reason, the conventional system test support device has no application other than a category of test called a regression test.

However, in GUI-based software, the more complex the GUI function, the more enormous the combinations of mouse and keyboard operations and the layout and types of windows that dynamically change due to these operations. . Therefore, when the developer first records the history of mouse and keyboard operations, it is very difficult to execute and record all combinations without omission, and the test contents may be omitted.

If the software itself is complicated,
The system test also becomes very complicated. For this reason, there is also a problem that a great deal of effort is required for the developer to perform the system test.

Furthermore, in order to record the history of mouse and keyboard operations by the developer, in order to actually perform mouse and keyboard operations, the designed software program itself is already mounted and in a state of operation. That is the premise. Therefore, it was not possible to implement the program and test design (and creation) in parallel.

Therefore, at this point, it is considered that the manual testing of GUI-based software has reached the limit. Moreover, as the functions required for the above-mentioned GUI become more sophisticated, the software programs including the GUI tend to become more complicated, so the above-mentioned troubles during system testing will become more serious in the future. It is thought to go.

The present invention has been made in view of the above circumstances, and makes it possible to easily perform a system test of software including a GUI, to prevent omission of test contents, and to provide a program. The purpose is to enable test design even at the stage where is not yet implemented. This aims to shorten the development period of the software as a whole.

[0013]

A software system test support device of the present invention is a software system test support device for supporting a system test of software with a GUI, wherein the software with the GUI is replaced by the GUI part of the software. It is defined by three types of specification information: structural specification information indicating the structure regarding the software, state transition specification information indicating the state transition regarding the GUI section of the software, and individual state specification information indicating the attribute value in each individual state regarding the GUI section of the software. Generates a test scenario that defines an operation procedure for performing the specification verification of the software, based on the specification defining means for defining according to the format and the structural specification information and state transition specification information defined by the specification defining means. On the test scenario generation means, A test scenario execution means for manipulating the test program of the software in accordance with the test scenarios generated by the test scenario generation unit,
Execution result collating means for collating whether or not the attribute value in each state of the program under test being executed in the test scenario executing means matches the value described in the individual state specification information is provided. It is a feature.

Another feature of the present invention is that, in a software system test support device for supporting a software system test, the software to be subjected to the system test is structured specification information indicating the structure of the software. , Specification defining means for defining in accordance with a format defined by three types of specification information of state transition specification information indicating a state transition of the software and individual state specification information indicating an attribute value in each individual state of the software, and the above specification. Based on the structural specification information and state transition specification information defined by the definition means, a test scenario generation means for generating a test scenario that defines an operation procedure for performing the specification verification of the software, and a test scenario generation means for generating the test scenario. The above software according to the tested scenario The test scenario executing means for operating the test program of the target and the attribute value in each state of the test program being executed in the test scenario executing means match the value described in the individual state specification information. An execution result collating means for collating whether or not it is provided.

Further, the test scenario generation device of the present invention is a test scenario generation device used when performing a system test of software accompanied by a GUI,
The software with I is structural specification information that represents the structure of the GUI part of the software, state transition specification information that represents the state transition of the GUI part of the software, and an individual state that represents the attribute value in each individual state of the GUI part of the software. When the specification verification of the software is performed based on the specification defining means for defining in accordance with the format defined by the three types of specification information, and the structural specification information and the state transition specification information defined by the specification defining means. And a test scenario generation means for generating a test scenario that defines the operation procedure described in 1. above.

Since the present invention comprises the above technical means, if the user defines the structural specification information and the state transition specification information,
A test scenario that defines the operation procedure for verifying the software specifications will be automatically generated by the test scenario generation means, and the operation history will be recorded by actually operating the mouse or keyboard manually in advance. There is no need to leave it.

At this time, since the test scenario is generated based on the specifications of the software to be tested, specifically, the structural specification information and the state transition specification information, the generated test scenario is the GUI part of the software. It covers all the states that it has and all state transitions.

Then, based on the test scenario automatically generated as described above, the system test is automatically executed by operating the mouse and keyboard by the test scenario executing means instead of manually, and the execution result comparing means is executed. As a result, the verification of whether the system test execution result conforms to the specifications is also automatically executed.

[0019]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings. The software system test support apparatus according to the present embodiment automatically generates a test scenario for specification verification based on the specifications given to software executed in an X window system, for example. Then, using the generated test scenario, a GUI operation is automatically executed for the target software, and during execution of this GUI operation, it is a device that verifies whether the target software conforms to the specifications.

As will be described in detail below, in the present embodiment, a mouse for manual operation is devised by devising a specification for accurately defining GUI-based software and providing a test scenario generating means capable of interpreting the specification. Also, it is possible to create a test scenario based on the above specifications and automatically execute it without recording keyboard operations.

FIG. 1 is a block diagram showing an example of the configuration of the system test support device according to the present embodiment described above.
The configuration and operation of the system test support device of this embodiment will be described below with reference to FIG. In FIG. 1, the X window system to which the system test support device 20 of the present embodiment is applied is generally realized by a client server system including an X server 21 and an X client 22.

In the configuration of the system test support device 20 of the above embodiment, 1 is a structural specification file, 2 is a state transition specification file, and 3 is an individual state specification file.

The structure specification file 1 defines a structure related to the GUI part of software. Also,
The state transition specification file 2 is a software GUI.
It defines the state of the state transition regarding the part. The individual state specification file 3 is the software G
The attribute value of the GUI component in each individual state regarding the UI unit is defined.

The structure specification file 1, the state transition specification file 2 and the individual state specification file 3 are all in text file format and are created by the user. 3 to 8 are diagrams showing an example of each of these files, FIG. 3 shows an example of the structural specification file 1, and FIG.
Shows an example of the state transition specification file 2, and FIGS. 5 to 8 show an example of the individual state specification file 3.

As shown in FIG. 3, the structure specification file 1
What GUI is the software to be tested?
Describes what kind of combination of parts is configured, and for one GUI part, the part ID
And the part class and the part name are described.

In the example of FIG. 3, the component ID indicates the parent-child relationship from the highest component. That is, when the component ID is divided into the right and left by the rightmost period, the left shows the component ID of the parent component of the component, and the right shows the order in the sibling components. For example, G with a part ID of 1.1.1.1
The UI component has a component ID of 1.1.1 as its parent component, and indicates that it is the first child component.

The component class indicates the class to which each GUI component belongs. Further, the component name is described so that the same GUI components respectively belonging to different aggregate components (component classes indicated by component IDs without periods) can be distinguished from each other.

As shown in FIG. 4, the state transition specification file 2 is such that, when an input event relating to a GUI occurs, the state of the GUI portion of the software to be tested is changed from before the event occurs to after the event occurs. It shows whether to transition to. The state transition specification file 2 describes the types of input events and the states before and after the event occurrence in association with each other.

For example, explaining the state transition described at the top of FIG. 4, (SAMPLE.Meters) is shown in FIG.
Since the toggle button is shown from the structural specification file 1 of 1., it means that when the toggle button is clicked in the state “0”, the state transits to the state “1”. in this way,
The state transition specification file 2 of this embodiment describes the state transition for each minimum unit in which a person operates a GUI using a mouse or a keyboard.

As is apparent from FIG. 4, in the present embodiment, there are four states of "0" to "3", and by receiving eight kinds of input events, the state of transition between those states is shown. This state transition specification file 2 is shown.

Further, the individual state specification file 3 is such that each state (0 to 3) that can be taken by the GUI section of the software to be tested is a component (see the structural specification file 1 in FIG. 3) that constitutes the GUI section. It describes how the attribute value of each component shown) will be. FIG.
Is the state “0”, FIG. 6 is the state “1”, FIG. 7 is the state “2”,
FIG. 8 describes the attribute values for the state “3”.

The attribute names handled by the individual state specification file 3 of this embodiment are basically based on the window attributes managed by the X window system. That is, the map attribute (map_state), the x coordinate value (x) based on the parent component, the y coordinate value (y) based on the parent component, the component width (width), and the component height (height).
And border width (border_width).

The above-mentioned three types of specifications are newly devised by the applicant of the present invention in order to define GUI-based software. In particular, it is characterized by incorporating the state transition specification. By using these three types of specifications, it is possible to accurately express the specifications of GUI-based software, which has been difficult to describe in the past, in accordance with a predetermined format. By unifying the specification description methods in this way, it becomes possible to realize an automatic test scenario generation function and an automatic test result determination function, which will be described later.

Returning to FIG. 1, the description will be continued. Reference numeral 4 denotes a test scenario generation means, which is a test scenario 5 according to the structure specification file 1 and the state transition specification file 2.
Generate The test scenario is a GU for verifying the specifications related to the GUI part of the software to be tested.
I operation procedure is defined.

Here, this test scenario generation means 4
Uses the above-mentioned state transition specification file 2 to automatically generate a test scenario 5 necessary to cover all states and all state transitions of the GUI part of the software. That is, the test scenario 5 that always executes all the events (input events) shown in FIG. 4 at least once is automatically generated. This test scenario 5
It is in a text file format so that the user can easily edit it.

FIG. 2 is a flow chart showing a procedure for generating the test scenario 5 in the test scenario generating means 4. The operation of the test scenario generating means 4 will be described below with reference to FIG.

First, in step P1 of FIG. 2, the state transition specification file 2 illustrated in FIG. 4 is developed into a graph that is information of a structure that is easy to operate on a computer. The graph is
Each of the nodes is composed of a plurality of nodes and a plurality of edges connecting the nodes, and each node has a state of "0"-
Each "3" corresponds 1: 1 and each edge corresponds 1: 1 to each of the various input events.

The expansion into the above graph is performed on the memory,
The state transition specification file 2 illustrated in FIG. 4 is replaced with a node list representing a plurality of nodes and an adjacent edge list representing a plurality of edges connected to each node.

Next, in step P2, the leading node of a certain edge is set as the transition source node, and in step P3 the state is moved to that transition source node. For example, state “0”
The current state is moved to “0” with the node of 1 as the head node. Then, in step P4, it is determined whether or not there is an edge without a completion mark in the adjacent edge list for the current transition source node. The completion mark mentioned here is a mark that is added to an edge corresponding to an event when the event is added to the test scenario.

If it is determined in this step P4 that there is an edge without a completion mark, the process proceeds to step P5.
An event corresponding to the first edge found is added to test scenario 5. Then, in step P6, a completion mark is added to the edge found first. Then, in step P7, the terminal node of the first edge found is set as the transition source node, and in step P8 it is checked again whether or not there is an edge not marked as complete in the adjacent edge list of the transition source node. to decide.

If it is determined that there is no edge without the completion mark, the transition source node is marked with the completion mark in step P9, and the process proceeds to step P10. The completion mark attached to the transition source node indicates that, for the transition source node, all the events corresponding to the related edges have been added to the test scenario 5. On the other hand, if it is determined in step P8 that there is an edge without a completion mark, the process jumps to step P10.

If it is determined in step P4 that there is no edge without a completion mark, step P11
Proceed to and determine if there are any unmarked nodes in the node list. If it is determined that there is a node without a completion mark, the process proceeds to step P12, and the first node found is set as the transition destination node. On the other hand, when it is determined that there is no node without the completion mark, the test scenario generation process ends.

After the processing of the step P12 is completed, it is determined in the next step P13 whether or not there is a shortest edge path connecting the transition source node and the transition destination node set at that time. Here, when it is determined that there is a shortest path, the process proceeds to step P14, the event corresponding to the edge along the shortest path found first is added to the test scenario 5, and if necessary,
Mark the edges and nodes in the middle of the shortest path as complete. Then, it progresses to step P10.

In step P10, the transition destination node is set as a new transition source node. When the process of step P10 ends, the process returns to step P3, and the state is moved to the transition source node set at that time. Then, the above-described operation is repeated through the processing of the next step P4.

On the other hand, when it is determined in step P13 that there is no shortest path, the process proceeds to step P15, and it is determined whether the transition source node at that time is the head node. When it is determined that the node is the head node, the test scenario generation processing is abnormally terminated, and when it is determined that the node is not the head node, the transition source node is set as the head node of the edge in step P16, and the process of step P13 is performed. Return. The test scenario 5 illustrated in FIG. 9 is generated by the above processing.

As described above, in this embodiment, if the user gives the structural specification information and the state transition specification information, it is necessary to cover all the states and all the state transitions of the GUI part of the software. A different test scenario 5 is automatically generated by the test scenario generation means 4. Therefore, the mouse 23 and the keyboard 2 are manually
It is not necessary to actually operate 4 to record the operation history.

As a result, manpower required for manual operation can be significantly reduced. In addition, it is not necessary to install a software program as a premise for creating a test scenario for actually operating the mouse 23 and the keyboard 24, so that the program installation and the test design can be performed in parallel. Become.

If the test scenario 5 is simply generated, it can be generated without using the state transition specification. However, the state transition specification is used when the test scenario 5 is generated as in this embodiment. As a result, it becomes possible to generate the complete test scenario 5 that covers all the state transitions based on the specifications. For this reason, it is possible to prevent omissions in the test contents.
Since unnecessary state transitions are not duplicated, excellent test results can be obtained.

Reference numeral 6 is a test scenario executing means, which reads the test scenario 5 and directly generates an event message based on the read content of the test scenario 5 (generated directly from an actual input device such as the mouse 23 or the keyboard 24). Create an event message of the same format as in. And the created direct event message is X
By supplying to the server 21, the GUI part of the software to be tested is indirectly operated according to the test scenario 5.

Numeral 7 is an execution result collating means, which reads the structure specification file 1 and the individual state specification file 3 and each component attribute value in each state of the program under test being executed by the test scenario executing means 6 is an individual state. Check whether it matches the value described in the specification. Then, a test result report 8 in the form of a text file is created based on the contents of the collation result.

For example, based on the test scenario 5, the test scenario executing means 6 executes the input event (clicking the toggle button in the state “0”) at the top of the state transition specification file 2 shown in FIG. If executed, the state should transition to "1". At this time, the execution result collating means 7 outputs the result of the execution of the input event,
Whether or not the state really transits to "1" is verified by collating the attribute value of the program being started with the attribute value of the individual state specification file 3 immediately after the execution of the input event.

As described above, the test scenario executing means 6 and the execution result collating means 7 operate in the test scenario 5 described above.
According to the above, the system test is automatically executed by operating the mouse and the keyboard instead of manually, and the test result report file 8 as the execution result is automatically created.

In the present embodiment, the mouse 23 and the keyboard 24 are manually operated by the configuration described above.
3 specification files 1-3 without recording the operation of
It is possible to create a test scenario 5 based on the above and automatically execute it. On the other hand, the system test support device of this embodiment is also provided with the message exchange means 9 and the test scenario recording means 10 as shown in FIG. 1, and the test scenario 5 can be manually created.

That is, the message exchange means 9 is provided between the X server 21 and the X client 22, and exchanges various messages based on the X protocol which is the protocol of the X window system. The message exchanging means 9 has three modes of standard, recording, and reproducing, and is configured to deliver the X protocol message according to each mode.

In the standard mode, the message exchange means 9 functions as an intermediate buffer between the X server 21 and the X client 22 and has no other function. That is, the direct event message generated by the operation of the mouse 23 or the keyboard 24 is not recorded or reproduced, and X
The window system is used normally.

In the recording mode, the X client 22 sends a direct event message generated by the operation of the mouse 23 or the keyboard 24 and sent from the X server 21.
And the test scenario recording means 10. In this recording mode, other X protocol messages have no function other than the intermediate buffer.

Further, in the reproduction mode, at the request of the test scenario executing means 6, the event message sent from the X server 21 according to the contents of the test scenario 5 is supplied to the X client 22. At that time, the content of the message is modified so that the X client 22 receives the above event message as being generated by an actual event from the mouse 23 or the keyboard 24. In this playback mode, other X protocol messages have no function other than the intermediate buffer.

The test scenario recording means 10 adds the control information such as the event interval time to the content of the event message sent from the message exchange means 9. Then, a series of input event sequences to which this control information is added is recorded as the test scenario 5 in the text file format. The control information is used when the test scenario executing means 6 reads the test scenario 5 and directly reproduces the event message.

The GUI 11 is a user interface for instructing various operations related to the system test support device according to this embodiment. For example, the operation of the mouse 23 and the keyboard 24 from the user is accepted,
A process of passing the command converted into an internal command to the message exchange means 9 is performed. Also, when recording the event message, you can instruct the start and end of the operation,
Processing for instructing automatic generation or automatic execution of the test scenario 5 is performed.

As described above, according to the present embodiment, the event message based on the test scenario 5 is generated by the X client 2 as if it were generated by manually operating the mouse 23 or the keyboard 24.
2 can be supplied. Therefore, X client 2
In the case of No. 2, it becomes possible to receive the reproduced direct event message even if the special extended function means is not incorporated in the X server 21.

By the way, in the conventional system test support device, in order to introduce this into the X window system,
It is necessary to add a special extended function to the server 21 or configure the software to be tested by a specific library depending on the system test support device to be introduced.

On the other hand, in the system test support device of the present embodiment, as described above, the special expansion means X is used.
It is not necessary to incorporate it in the server 21. Therefore, the labor and time for incorporating the expansion means can be completely eliminated. Moreover, conventionally, some side effects may be exerted on the basic means due to an operator's human error when incorporating the expansion means, but according to the present embodiment, such a concern is eliminated. be able to.

Further, according to this embodiment, it is not necessary to use a specific library depending on the system test support device when creating the software under test. For this reason, there is no need to make any changes when testing software that has already been created (for example, software before version upgrade) before introducing the system test support device. Also, regarding newly created software,
Industry standard libraries can be used. Further, there is an advantage that no problem occurs even if the library used is replaced in the middle of the development cycle.

In the above embodiment, the software accompanied by the GUI is described, but the present invention is not limited to this, and it is needless to say that it is suitable for the system test of the software without the GUI. .

[0065]

As described above, the present invention provides specification defining means for defining software with a GUI according to a format defined by three types of specification information, structural specification information, state transition specification information and individual state specification information. Since the test scenario generation means for generating a test scenario based on the defined specification information is provided, the user can automatically generate the test scenario simply by defining the specification information. It is possible to eliminate the need to manually record the operation history by manually operating the mouse or keyboard in advance.

As a result, the labor required for manual operation can be significantly reduced. Further, it is not necessary to implement a program as a premise for creating a test scenario for actually operating a mouse or keyboard, and it becomes possible to perform program implementation and test design in parallel. Furthermore, since the test scenario necessary to cover all the states and all the state transitions is generated based on the state transition specification information, it becomes possible to generate a test scenario in which there are no missing events. It is possible to prevent omissions in the test contents.

As a result of the above, there is an extremely high possibility that an initially occurring bug of software can be automatically and quickly found.

Further, in the software system test support device according to the present invention, the test scenario executing means for executing the system test of the program under test based on the contents of the test scenario generated by the test scenario generating means, and the test execution result thereof. Is provided with an execution result collating means for collating whether or not the specification conforms to the individual state specification information, so that the system test of the program under test is automatically executed based on the test scenario automatically generated as described above. Thus, the labor required for manual operation can be further reduced.

That is, the test scenario can be automatically generated based on the above-mentioned specification information only by the user by giving the specification information for defining the GUI-based software, and the test scenario can be automatically generated. Based on the above, the system test can be automatically executed, and the system test of the software including the GUI can be performed very easily and highly accurately.

Further, the present invention can obtain the above-mentioned effects not only in the software with GUI but also in the system test of the software without GUI.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration example of a software system test support device according to an embodiment of the present invention.

FIG. 2 is a flowchart showing an operation of a test scenario generation means.

FIG. 3 is a diagram showing an example of a structural specification file.

FIG. 4 is a diagram showing an example of a state transition specification file.

FIG. 5 is a diagram showing an example of an individual state specification file.

FIG. 6 is a diagram showing a continuation of an individual state specification file.

FIG. 7 is a diagram showing a continuation of an individual state specification file.

FIG. 8 is a diagram showing a continuation of an individual state specification file.

FIG. 9 is a diagram showing an example of a generated test scenario.

[Explanation of symbols]

 1 Structural Specification File 2 State Transition Specification File 3 Individual State Specification File 4 Test Scenario Generation Means 5 Test Scenario File 6 Test Scenario Execution Means 7 Execution Result Matching Means 8 Test Result Report File

Claims (3)

[Claims] 1. A software system test support apparatus for supporting a system test of software with GUI, wherein the software with GUI is structural specification information indicating a structure with respect to a GUI part of the software, and status with respect to the GUI part of the software. Specification defining means for defining in accordance with a format defined by three types of specification information of state transition specification information indicating transition and individual state specification information indicating attribute values in each individual state related to the GUI part of the software, and the above specification defining means Generated by the test scenario generation means and the test scenario generation means for generating a test scenario that defines an operation procedure for performing the specification verification of the software based on the structural specification information and the state transition specification information defined by According to test scenario The attribute value in each state of the test scenario executing means for operating the test program of the software and the test program executing in the test scenario executing means matches the value described in the individual state specification information. A system test support device for software, characterized by comprising an execution result collation means for collating whether or not. 2. A test scenario generation device used when performing a system test of software including a GUI, wherein the software including the GUI is structural specification information indicating a structure related to a GUI part of the software, and a GUI part of the software. Specification definition means for defining in accordance with a format defined by three types of specification information of state transition specification information indicating state transition regarding the software and individual state specification information indicating attribute values in each individual state regarding the GUI part of the software, and the above specification. And a test scenario generating means for generating a test scenario that defines an operation procedure for performing the specification verification of the software based on the structural specification information and the state transition specification information defined by the defining means. Test scenario generator. 3. A software system test support device for supporting a software system test, wherein the software to be subjected to the system test is structured specification information representing the structure of the software, and status representing the state transition of the software. Specification defining means for defining in accordance with a format defined by three types of specification information of transition specification information and individual state specification information representing attribute values in each individual state of the software, and structural specification information defined by the above specification defining means And a test scenario generation means for generating a test scenario that defines an operation procedure for verifying the specifications of the software based on the state transition specification information, and a target of the software according to the test scenario generated by the test scenario generation means. Operate the test program Execution result to check whether the test scenario execution means to be created and the attribute value in each state of the program under test being executed in the test scenario execution means match the value described in the individual state specification information A system test support device for software, characterized in that a collating means is provided.