JP2014232514A - Method and device for automatically operating graphical user interface - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology for a comprehensive automatic operation performed regarding the state of a graphical user interface (GUI), which technology reduces processing time for creating a GUI state transition graph when a large number of terminals running multiple applications to be tested are prepared to automatically and concurrently operate the GUI.SOLUTION: A comprehensive operation control unit 5 extracts a path for reaching from a state of a displayed GUI to a state (target state) of the GUI in which a non-operated GUI element exits. Then, the comprehensive operation control unit 5 determines whether a non-operated GUI element surely exits when the state is changed to the target state. If it is determined that no non-operated GUI element exits, the comprehensive operation control unit 5 extracts a path to another target state, performs such determination processing again, and automatically operates a GUI element along with the extracted path. In this way, the GUI is changed to a state in which a non-operated GUI element exits.

Description

  The present invention relates to a technique for generating a state transition graph representing an operation of a graphical user interface.

  Most of the recent software includes a graphical user interface (hereinafter referred to as GUI), and testing that the GUI changes as expected by operating the GUI is an essential task in software development. Many GUIs employ a window system, and the GUI changes by operating GUI elements such as buttons and text boxes in the window. This is called GUI state transition. Specifically, the operations here are mouse clicks, keyboard inputs, and the like. If the GUI after the change due to such an operation is not in an expected state, this is called a GUI error.

  In order to find a GUI error, it is necessary to execute all operation procedures for the GUI and check the GUI after the operation. However, since the GUI is composed of many GUI elements, there are a huge number of combinations of the operation order. Therefore, if this confirmation is performed manually, the work requires a huge amount of man-hours.

  As a conventional technique, a technique has been proposed in which a GUI element is automatically operated to actually change the GUI, and the automatic operation is recursively repeated for the changed GUI. As a result of such an automatic operation, it is possible to generate a state transition graph that represents a transition of the GUI with respect to the operation of the GUI element. Using such a state transition graph, it is possible to detect a GUI error and generate a test case for testing the GUI. Here, the prior art of these automatic operations and their problems will be described.

  In Non-Patent Document 1, GUI elements in a plurality of windows displayed as initial GUIs are automatically and sequentially operated. When a GUI element is operated, a new window may be displayed (or in the case of a GUI based on a browser or the like, the display is switched to a GUI configured with another GUI element). The same processing is recursively repeated for the GUI element. For example, consider the case where the initial GUI is composed of one window, as shown in FIG. 1, and two radio buttons and one button exist as GUI elements in the window. When the button is pressed, the GUI is switched to (a), (b) or (c) according to the selection state of the radio button. When the technique of Non-Patent Document 1 is applied to such a GUI, automatic operation is performed in the following procedure.

1. Operate radio button 1 (radio button 1 is selected)
2. Operate radio button 2 (radio button 2 is selected. Radio button 1 is deselected)
3. Operate button 1 (GUI switches to (c))
4). Manipulate GUI elements (buttons with captions Yes and No) in GUI (c).

  The problem of Non-Patent Document 1 is that each GUI element that appears in the process of automatic operation is operated only once. As a result, it is not possible to transition to the GUI of (a) or (b) as in the previous automatic operation procedure, and the generated state transition graph also has a lack of transition to (a) or (b). It will be complete.

  In response to such a problem of Non-Patent Document 1, a technique for generating a more precise GUI state transition graph by comprehensively manipulating the GUI state as in Non-Patent Document 2 has been proposed. Yes. The GUI state is information expressing the structure of the GUI element displayed as the GUI. The GUI element holds properties such as captions, active states, and selection states, and information on their values. The state of the elements constituting the GUI can be expressed by a set of a set of GUI element name, property, and property value. FIG. 2 shows an example of the state of each element of the initial GUI of FIG. The GUI is composed of a plurality of windows. Therefore, the state of each window is a set of states of GUI elements constituting the windows. The GUI state can be expressed by a set of states of each window.

For example, the initial GUI of FIG. 1 can be expressed as different states depending on the radio button selection. FIG. 3 shows the state of the GUI when neither radio button 1 nor radio button 2 is selected, when radio button 1 is selected, or when radio button 2 is selected. Since there is only one window in this GUI, if the state of each window is w ₀ , w ₁ , w ₂ , the state of each GUI is {w ₀ }, {w ₁ }, {w ₂ }. .

  In Non-Patent Document 2, an application to be tested is first activated to display an initial GUI. The GUI state is acquired from the initial GUI, and the operable GUI elements are automatically operated. Since the GUI state displayed by this process changes, the same process is recursively repeated for the changed GUI state. By doing so, for example, if the button 1 is automatically operated in each state of FIG. 3, it is possible to transition to the GUI (a) (b) (c) of FIG. A more precise state transition graph can be constructed.

  In order to automatically operate GUI elements comprehensively for all the GUI states that appear in this way, the GUI states that appear during the automatic operation and the GUI elements that are operated in the GUI states are stored. It is necessary to keep. If not stored, when the same GUI state appears, a GUI element that has been operated in the past may be repeatedly operated, resulting in an infinite loop. Therefore, in Non-Patent Document 2, the state of the GUI that appears and the state transition that has occurred are added to and stored in the set of the GUI state S and the state transition set T, respectively.

  Specifically, the state of the currently displayed GUI is acquired, and it is checked whether the GUI state has appeared in the past, that is, whether the current GUI state is included in the set S of GUI states. If not, the GUI state is added to the GUI state set S. Further, possible state transitions in the current GUI state are added to the state transition set T. State transition can be expressed formally as a set of a state before operation, a GUI element to be operated, and a state after operation.

For example, consider a case where an automatic operation according to Non-Patent Document 2 is performed on the initial GUI of FIG. First, since the initial GUI is displayed on the display, the initial GUI state s ₀ = {w ₀ } is acquired. Since the GUI state s ₀ is not yet registered in the state set S, the GUI state s ₀ is added to the set S. Further, since the radio button 1, the radio button 2, and the button 1 can be operated in this GUI state, (s ₀ , radio button 1, TBD), (s ₀ , radio button 2, TBD), (s ₀ , Button 1, TBD) are added to the set T (FIG. 4). Each row of the table indicating the state transition set T in FIG. 4 corresponds to the added state transition. The state after the operation in the state transition is TBD means that the state after the operation is not yet determined, and therefore the state after the operation is not yet determined (To Be Determined). Hereinafter, the state of the GUI is simply referred to as a state.

Next, one transition that has not been operated in the current state s ₀ is selected, and the GUI element is automatically operated. For example, here, it is assumed that the radio button 1 is automatically operated from the set T in FIG. As a result, the GUI changes as shown in FIG. If the GUI state is s ₁ , the previously selected state transition (s ₀ , radio button 1, TBD) is replaced with (s ₀ , radio button 1, s ₁ ). Thereafter, the processing so far is recursively repeated for the current state s ₁ . That is, it is checked whether the state s ₁ is included in the set S. Since it is not included, s ₁ is added to the set S. Further, a possible transition for the state s ₁ is added to the set T. Then, _one possible transition is selected in the state s ₁ and, for example, the radio button 1 is automatically operated. FIG. 6 shows a state set S and a state transition set T at this time.

FIG. 7 shows the state transition of the GUI when the automatic operation process is repeated from the initial GUI. The state of the GUI changes as s ₀ , s ₁ , s ₁ , s ₂ , s ₁ , s ₃ , and continues until the application is terminated by pressing the end button (button 2) in state s ₃ . FIG. 8 shows a set S of GUI states and a set T of state transitions at this time. When the application is terminated, Non-Patent Document 2 restarts the application, displays the initial GUI, and repeats the process.

Non-Patent Document 2 can generate a state transition graph as a result of such automatic operation. The state transition graph is formally expressed by a pair <S, T> of a set of GUI states and a set of state transitions. When the state transition graph is expressed in a diagram form, the states included in the set S are expressed by nodes, and the state transition is expressed as a directed edge between the nodes. That is, in the transition included in T, an edge is drawn from the node corresponding to the state before the operation to the node corresponding to the state after the operation. Further, a transition GUI element is described as a label in the edge. FIG. 9 shows a diagram format of the state transition graph of the GUI state set S and state transition T in FIG. In the figure, an edge that does not indicate a node, for example, an edge labeled as radio button 2 or button 1 of node s ₀ means a state transition in which the state after the operation is TBD.

If automatic operation processing is continued in this way, there may be no GUI element that has not been operated in the displayed GUI state s. At this time, if there is an unoperated GUI element in the state s ′ of another GUI, it is necessary to change the GUI from the state s to the state s ′ by an automatic operation and operate the unoperated GUI element. For example, when the state of the displayed GUI is s ₁ , there is no unoperated state transition in s ₁ as shown in the state transition graph of FIG. Therefore, for example, it is necessary to transition the GUI to the state s ₂ where there is an unoperated state transition.

In Non-Patent Document 2, an execution path is used to transition the GUI from the state s to the state s ′. An execution path is a finite sequence of state transitions generated by a series of automatic operations by the above processing. For example t ₁ state transition occurred _{is, t 2, t 3, ...} , if a t _n, the execution path can be represented as _{t 1 t 2 t 3 ... t} n. In Non-Patent Document 2, the state transition sequence from the initial GUI state until the application is terminated by automatic operation is saved as one execution path (if the application is restarted again and automatically operated, another execution path is stored). Save as). Paths (partial state transition sequences) t _i ... T _j are extracted from the set of execution paths accumulated in this way. However, the state before the operation of t _i must be s, and the state after the operation of t _j must be s ′. Furthermore, for any 1 ≦ i ≦ n−1, the state after the operation of the state transition t _i needs to be equal to the state before the operation of t _{i + 1} . For example, the path from the state s ₁ to s ₂ is (s ₁ , radio button 2, s ₂ ). The GUI state can be transitioned to s ′ by automatically automatically manipulating the GUI elements from the path state transitions t _i to t _j . Non-patent document 2 can acquire a more precise state transition graph than non-patent document 1 by the processing described above.

  In Non-Patent Document 2, a precise state transition graph is acquired by exhaustively manipulating GUI elements with respect to a GUI state appearing in the course of automatic operation. Accordingly, the processing time until the state transition graph is obtained increases in proportion to the number of GUI states that appear. The number of GUI states that can appear exponentially increases with respect to the number of GUI elements that constitute the GUI. As a result, the processing time increases exponentially with respect to the number of GUI elements, and the number of GUI elements increases. Measures are required to apply to the above. Non-Patent Document 2 describes parallelization of automatic operation processing as a response to this increase in processing time. The parallelization of automatic operation will be described with reference to FIG. In order to make parallel, first, a plurality of terminals on which the test target application operates are prepared. The automatic operation device of Non-Patent Document 2 acquires the GUI state of an application from a plurality of prepared terminals, and automatically operates an unoperated GUI element in accordance with the GUI state of each terminal. CPUs and OSs in recent years are compatible with multi-threading, and GUI status acquisition and automatic operation of a plurality of terminals can be performed in parallel. By increasing the number of terminals on which the test target application operates in this way, comprehensive automatic operations can be performed in parallel, and the processing time can be shortened.

A. Memon, I. Banerjee, A. Nagarajan, "GUI Ripping: Reverse Engineering of Graphical User Interfaces for Testing", In proc. Of the 10th Working Conference on Reverse Engineering (WCRE 2003), IEEE, pp.260-269, 2003 Nakajima, Masuda, Takahashi, “Proposal of automated GUI test method by comprehensive operation and operation log search”, IEICE Tech., Vol.112, No.378, IEICE, pp.13-18, 2013

  The problem with the parallelization of Non-Patent Document 2 is that the processing time cannot be shortened with the increase in the number of terminals by such a simple parallelization. The cause lies in the processing in the case where there is no GUI element not yet operated in the displayed GUI state s. In Non-Patent Document 2, at this time, the path from the state s to the state s ′ where an unoperated GUI element exists is extracted from the execution paths accumulated so far. For example, when there are a plurality of terminals whose GUI state is “s”, if the automatic operation process is simply made parallel by multithreading, the same path is extracted for those terminals. Since the same path has the same state s ′ where there is an unoperated GUI element that is the final target state, when the GUI of each terminal is automatically operated along the path, the GUI state changes to s ′. Become. If there are enough unoperated GUI elements in the state s', it is possible to operate the unoperated GUI elements at each terminal. However, if there are not enough unoperated GUI elements, some terminals can operate the unoperated GUI elements, but the remaining terminals no longer have the unoperated GUI elements, Path extraction and automatic operation processing along the path are wasted.

For example, this will be specifically described with reference to FIG. Now, assume that there are three terminals on which the test target application operates for parallel processing, and the test target application of each terminal is in the state s ₁ . At this time, if the processing is simply performed by multithreading, a path (s ₁ , radio button 2, s ₂ ) is extracted for each terminal as a path to a state in which an unoperated GUI element exists. However, as shown in FIG. 9, there are only two GUI elements that have not been operated in the state s ₂ . Therefore, even if the GUI is in the state s ₂ by automatically operating along the path at each terminal, the GUI elements that are not operated can be automatically operated on the two terminals, but there are GUI elements that are not operated on the remaining one terminal. This is a wasteful process. Even if the number of terminals is increased in this way, the number of terminals that cannot automatically operate unoperated GUI elements increases, and the processing time cannot be shortened.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to prepare a large number of terminals on which a plurality of test target applications operate in a technology for comprehensively automatically operating the GUI state. The purpose is to shorten the processing time for generating a GUI state transition graph in the case of automatic operation in parallel.

  In order to achieve the above object, the present invention is a method for comprehensively and automatically operating the state of a graphical user interface (GUI) of a test target application operating on a plurality of terminals, which is stored in advance in a storage means. A first step of starting a test target application in each terminal by executing the file on each of the plurality of terminals; and further, the GUI state of the test target application displayed on the display means of the terminal for each terminal The second step of acquiring the GUI, the third step of acquiring all the GUI elements that can be operated in the GUI state when the displayed GUI state has never been acquired, and the test target application ends. If this is the case, run the startup file to test The fourth step of restarting the application and recursively processing the displayed GUI state, and when there is an unoperated GUI element in the displayed GUI state, A fifth step in which the GUI of the test target application is automatically transitioned and the process is recursively continued with respect to the state after the transition, and there is no GUI element not yet operated in the displayed GUI state In addition, when there is an unoperated GUI element in the other GUI states acquired so far, the path that reaches the GUI state (target state) where the unoperated GUI element exists is extracted from the displayed GUI state. When the state is changed to the target state, it is determined whether an unoperated GUI element is surely present. If it is not determined that there is not, a path to another target state is determined. And the sixth step of performing the determination process again and automatically operating the GUI element along the extracted path to change the GUI to a state in which there is an unoperated GUI element. A seventh step for recursively continuing the process and an eighth step for terminating the process when there is no unoperated GUI element in all the obtained GUI states are processed in parallel between the terminals. It is characterized by doing.

  As an example of a preferred aspect of the present invention, in the sixth step, when a path to a target state is extracted, target state management indicating that a state transitions to the target state in a terminal related to its own processing The information is stored in the shared storage means in each parallel process, and the writing process to the storage means of the target state management information is exclusively controlled between the parallel processes during the period from the storage process to the determination process. Is mentioned.

  Furthermore, as an example of a preferred aspect of the present invention, the target state management information includes a pair of a terminal identifier and a target state, and after the first step, a pair of a terminal identifier and a null value is set for each terminal after the first step. In the sixth step, when the path to the target state is extracted, the target management information is rewritten to the set of the identifier of the terminal related to the self-processing and the target state, as target management information. What is characterized in that the determination process is performed based on the number of identifiers of terminals having the same target state as the target state related to the self-processing and the number of unoperated GUI elements in the target state.

  According to the present invention, in the technology for comprehensively automatically operating the GUI state, a GUI state transition graph in the case where a large number of terminals on which a plurality of test target applications operate is prepared and automatically operated in parallel is provided. The generation processing time can be shortened.

An example of GUI Example of the state of each element of the initial GUI of FIG. Example of GUI state transition Example of GUI state transition Example of GUI state transition Example of GUI state transition Example of GUI state transition Example of GUI state transition Example of state transition graph System configuration diagram explaining parallelization of automatic operation Configuration diagram of state device of graphical user interface according to the present embodiment Example of target state management department data Flow chart for explaining the operation of the comprehensive operation control unit Flow chart for explaining the operation of the comprehensive operation control unit Flow chart for explaining the operation of the comprehensive operation control unit Flow chart for explaining the operation of the comprehensive operation control unit Flow chart for explaining the operation of the comprehensive operation control unit Example of target state management department data System configuration example of automatic operation apparatus according to embodiment An example of data of the target state management unit according to the embodiment An example of data of the target state management unit according to the embodiment An example of data of the target state management unit according to the embodiment

  An automatic operation apparatus that implements an exhaustive parallel automatic operation method for a graphical user interface according to an embodiment of the present invention will be described with reference to FIG. Here, an example realized on a known computer is shown. In the figure, 1 is an input device, 2 is a computer main body, 3 is a storage device, and 4 is a display device.

  The input device 1 includes a keyboard, a mouse, and the like, and a user can input predetermined information, here, a startup file to the computer main body 2. The input device 1 can register and delete the identifier of the terminal on which the test target application operates. The terminal identifier may be a host name or an IP address if the terminal is connected via an IP network. When the identifier of this terminal is registered from the input device, it is added to the storage device 3, and when it is deleted from the input device, it is deleted from the storage device 3.

  The storage device 3 includes a list of terminal identifiers input from the input device 1, a set of GUI states S and a set of state transitions T that are output results of the procedures of FIGS. 13 to 17 performed by the covering operation control unit 5. The apparatus stores and saves a set P of execution paths, and temporarily stores information necessary for various processes. As will be described later, the processing by the covering operation control unit 5 is executed in multi-threads for each terminal that executes the test target application. It should be noted that the GUI state set S, the state transition set T, and the execution path set P stored and stored in the storage device 3 are shared resources shared by the threads. It should also be noted that when each thread adds, deletes, or refers to the data of the shared resource, an exclusive lock is applied so that a plurality of threads cannot be operated on the shared resource at the same time.

  The display device 4 includes an image display device such as a liquid crystal display and a CRT, and the contents of a list of terminal identifiers, a GUI state set S, a state transition set T, and an execution path set P stored in the storage device 3. Can be displayed. Further, the state transition graph <S, T> can be displayed in a diagram form.

  The computer main body 2 controls each device and constitutes a comprehensive operation control unit 5, a GUI state acquisition unit 6, a GUI automatic operation unit 7, a path set search unit 8, and a target state management unit 9. Each unit of the computer main body 2 may be configured by hardware, or may be configured by installing a program in the computer main body 2. The program can be recorded on a recording medium or provided through a network.

  The covering operation control unit 5 controls processing for comprehensively acquiring the states of all GUIs of the test target application. The comprehensive operation control unit 5 executes the activation file given from the input device 1 on each terminal connected to the computer main body 2 via a network or the like, thereby activating the test target application on the terminal, and the GUI state acquisition unit 6 The GUI state is acquired by the above, and the GUI automatic operation unit 7 automatically operates the unoperated GUI element. Since the GUI state transitions by this automatic operation, these processes are recursively repeated for the transitioned GUI state. The processing by the covering operation control unit 5 is executed in multithread for each terminal that executes the test target application. The operation of the covering operation control unit 5 will be described later.

  The GUI state acquisition unit 6 can acquire the state of the GUI of the test target application that runs on each terminal connected to the computer main body 2 via a network or the like. The GUI status acquisition unit 6 is called in steps S5, S696, and S69954 when the covering operation control unit 5 performs the procedures of FIGS.

  The GUI automatic operation unit 7 can automatically operate the GUI element specified by the covering operation control unit 5 on the GUI of the test target application that operates on each terminal connected to the computer main body 2 via a network or the like. The GUI automatic operation unit 7 is called in steps S695 and S69953 when the covering operation control unit 5 performs the procedures of FIGS.

  The path set search unit 8 searches for a path from a GUI state designated by the covering operation control unit 5 to a state where an unoperated GUI element exists. The path set search unit 8 is called in step S6991 when the covering operation control unit 5 performs the procedures of FIGS. In this step, it is described that a path is searched from a set of execution paths. For example, a similar path may be searched from the state transition graph <S, T> accumulated so far. The path search method from the execution path and the state transition graph is a known technique.

  The target state management unit 9 holds a pair of terminal identifiers and target states registered in the storage device 3 in a list format. An example of the data structure of the list held in the target state management unit 9 is shown in FIG. FIG. 12 shows an example when the terminal identifier is an IP address and the number of terminals is 3, and the target state of each terminal is a null value. When the comprehensive operation control unit 5 executes the procedures of FIGS. 13 to 17 in a multi-thread manner for each terminal connected to the computer main body 2, the operation after the last state transition of the path p found by each thread in step S6991 is performed. Is registered as the target state of the identifier of the corresponding terminal in step S6993. Here, the list held by the target state management unit 9 is shared by each thread in the same manner as the GUI state set S, state transition set T, and execution path set P stored and stored in the storage device 3. Note that this is a resource. It should also be noted that when each thread adds, deletes, or refers to the data of the shared resource, an exclusive lock is applied so that a plurality of threads cannot be operated on the shared resource at the same time.

The comprehensive operation control unit 5 performs the procedure of FIG. In the procedure of FIG. 13, by executing the activation file given from the input device 1 for each terminal registered in the storage device 3, the test target application is activated on each terminal (step S1), Are displayed on each terminal. Further, the GUI state set S, the state transition set T, and the execution path set P are set as empty sets (step S2). Thereafter, the pair of the identifier of the terminal registered in the storage device 3 and the target state is registered in the target state management unit 9. At this time, the target state is registered as a null value. As described above, the information (a pair of terminal identifier and target state) registered in the sets S, T, and P and the target state management unit 9 becomes a shared resource when performing automatic operation processing in parallel with multithreading. , Access the same information from each thread. In addition, since the subsequent processing is processed in a multi-thread manner for each terminal, variables other than information registered in the sets S, T, P and target state management unit 9 are prepared for each thread, and each thread has its own Note that the value of is assigned. In addition, when a thread adds, deletes, or references data to the above-described shared resource, an exclusive lock is applied so that a plurality of threads cannot operate on the shared resource at the same time. Next, a constant ε representing a zero-length column is set in the execution path π (step S4). The GUI state acquisition unit 6 is used to acquire the currently displayed GUI state, and this is set as s ₀ . Further, s ₀ is substituted for a variable s representing the current state (step S5). Finally, the procedure of FIG. 14 is performed (step S6).

  In the procedure of FIG. 14, it is first checked whether or not the current state s is included in the state set S (step S61). If not, the current state s is added to the set S (step S62). Further, a set E of operable GUI elements is extracted from the current state s (step S63). When the GUI element included in the set E is e, state transitions (s, e, TBD) are added to the state transition set T for all GUI elements (steps S64, S65, S66). When such state transition addition processing is completed, or when the current state s is included in the state set S in step S61, the state after operation in the state transition set T is TBD. It is checked whether there is a state transition (step S67). If it does not exist, it means that the comprehensive automatic operation has been completed. Therefore, the accumulated GUI state set S, state transition set T, and execution path set P are output to the storage device (step S68). End the procedure. If there is a state transition that is TBD in step S67, the procedure of FIG. 15 is executed (step S69).

In the procedure of FIG. 15, it is first checked whether the application is terminated and no GUI is displayed (step S691). If completed, the application is restarted using the startup file (step S692). Further, the execution path π is added to the set P of execution paths, a constant ε meaning a column having a length of 0 is substituted for π, and the initial state s ₀ is substituted for the variable s indicating the current state (step S693). ). And it returns to step S61 of the procedure of FIG. If the application has not ended in step S691, it is checked whether there is an unoperated GUI element in the current state indicated by the variable s. That is, a state transition t in which the state before the operation is s and the state after the operation is TBD is searched for in the state transition set T (step S694). If the GUI element of the state transition t is e, it can be expressed as t = (s, e, TBD). When such a state transition t exists, the GUI automatic operation unit 7 automatically operates the GUI element e of t (step S695), and makes the GUI transition. Further, the state of the transitioned GUI is acquired by the GUI state acquisition unit 6 and substituted into the variable s ′ (step S696). Since the state after the operation of the state transition t is determined by this process, t is replaced with (s, e, s ′) (step S697). A state transition t is added to the end of the execution path π, and the variable s indicating the current state is replaced with s ′ (step S698). And it returns to step S61 of the procedure of FIG. If the state transition t cannot be found in step S694, the procedure of FIG. 16 is performed (step S699).

In the procedure of FIG. 16, the path set search unit 8 searches for a similar path p from the set of execution paths P∪ {π} (step S6991). Next, it is examined whether or not the path p has been found (step S6992). If it cannot be found, the GUI cannot be changed from the state indicated by the variable s (but not the initial state after the application is restarted) to the state s ′. Therefore, in order to restart the application and continue the procedure, the process returns to step S692 in FIG. If the path p can be found, the last state transition (s _l , e _l , _sl ′) of the path p is set, and the pair of the terminal identifier and the target state managed by the target state management unit 9 is set. rewrites the identifier of the terminal and the state s _l '(step S6993). For example, when the pair of the terminal identifier and the target state in the target state management unit 9 is in the state of FIG. 12, this processing is now executed in the thread for the terminal having the terminal identifier 192.168.1.1. In this case, the pair of 192.168.1.1 and the null value is rewritten with the pair of 192.168.1.1 and the state _sl ′, and the state shown in FIG. 18 is obtained. Thereafter, the number of pairs whose target state of the target state management unit 9 is the state _sl ′ is N, the number of state transitions before the operation of the set T is _sl ′ and the state after the operation is a constant TBD is N _TBD is set (step S6994), and N ≦ N _TBD is determined (step S6995). If N ≦ N _TBD , the number of terminals (N) trying to make a transition to the state s _l ′ using the path discovered by the path set search unit 8 is an unoperated GUI in the state s _l ′. This is less than the number of elements (N _TBD ). This is because when there is an automatic operation along the path and the state s _l ′ is reached, an unoperated GUI element is surely present, and the unoperated GUI element can be automatically operated. Therefore, the procedure proceeds to the procedure of FIG. By performing such processing, the problems of Non-Patent Document 2 can be solved. If N> N _TBD , there is a case where an unoperated GUI element does not exist even if the state is changed to s _l 'as it is, so that the terminal identifier rewritten to the target state management unit 9 in step S6993. and a pair of states s _l ', rewrites the pair of the identifier and an empty value for the terminal (step S6996). Then, the process returns to step S6991 again to search for another path. When the series of processes of steps S6993, S6994, and S6995 described above is performed, the exclusive lock on the pair of the identifier and the target state, which is the shared resource of the target state management unit 9, is not released by each process. The exclusive lock is released when all these processes are completed.

In the procedure of FIG. 17, first, it is checked whether the length of the path p is 0 (step S69951). If nonzero, removed state transition beginning of the path _{p t = (s t, e} t, s t ') as (step S69952), GUI automatic operation unit 7 operates the GUI element e _t (step S69953) . Accordingly, since the GUI transitions, the GUI state acquisition unit 6 is used to acquire the GUI state, and is substituted into the variable s indicating the current state (step S69954). Further state transition at the end of the path _{π (s t, e t,} s) by adding a state transition to the set T of the state transition _{(s t, e t, s} ) to add (step S69955). At this time, it is checked whether the current state indicated by the variable s is included in the GUI state set S or whether there is an unoperated GUI element in that state (step S69956). If this condition is not met, the process returns to step S69951, the next head of the path p is taken out, and steps S69951 to S69956 are executed. Steps S69951 to S69956 are executed for all the state transitions included in the path p. If the length of the path is 0 in the step S69951, the GUI element is automatically operated along the path p. However, it is impossible to reach a state having an unoperated GUI element. Therefore rewritten terminal identifier and a pair of states s _l 'to the target state management unit 9 in step S6993 of FIG. 16, rewrites the pair of the identifier and an empty value for the terminal (step S69957). Then, the process returns to step S692 in FIG. 16 in order to restart the test target application and continue the processing.

If the beginning of the state transition t = path _{p (s t, e t,} s t ') perform the steps S69951 S69956 respect, by operating the GUI element e _t GUI state is included in the set S no, or if the non-operation GUI element is present in the state of the GUI is rewritten pairs terminal identifier and state s _l 'to the target state management unit 9 in step S6993 of FIG. 16, the The terminal identifier is rewritten to a null value pair (step S69958). Then, the process returns to step S61 in FIG. 14 to automatically operate the GUI element that has not been operated recursively.

  As described above, according to the present invention, in the technology for comprehensively automatically operating the GUI state, when preparing a terminal on which a plurality of test target applications operate and performing automatic operation in parallel, no operation is performed on each terminal. The GUI element can be automatically operated reliably, and the processing time for generating the GUI state transition graph can be shortened as the number of terminals increases.

  In the above-described embodiment, an example is shown in which the comprehensive operation control unit 5, the GUI state acquisition unit 6, the GUI automatic operation unit 7, the path set search unit 8, and the target state management unit 9 are realized on the computer main body 2. It may be realized by hardware. Each of these devices exists in a distributed manner on a plurality of pieces of hardware connected to each other via an appropriate communication path, and can be executed while communicating with each other. Furthermore, a plurality of terminals that operate the test target application may be prepared as separate hardware, or a plurality of terminals may be virtually installed on one or a plurality of hardware using a virtual terminal, a virtual OS, or the like. Or may be mounted on the same hardware as the automatic operation device of the present invention. That is, the mounting form is not required within the scope of the present invention.

  2 illustrates an example of comprehensive GUI automatic operation according to the present invention. Here, it is assumed that the screen of the test target application changes as shown in FIG. In this embodiment, as shown in FIG. 19, the number of terminals operated in parallel is set to 3, and each terminal is connected to a computer main body 2 as an automatic operation device by an IP network. The IP address of each terminal is 192.168.1.1, 192.168.1.2, and 192.168.1.3, respectively.

  In the present invention, first, a file (script, batch file) for starting the application to be tested of FIG. 1 from the input device 1 and identifiers of each terminal as 192.168.1.1, 192.168.1.2, 192.168. .1.3 is input, and the identifier of the terminal is added to the storage device 3.

  In the present invention, the covering operation control unit 5 performs the procedures of FIGS. First, in step S1 in FIG. 13, the activation file is executed to activate the test application on each terminal. In step S2, the GUI state set S, the state transition set T, and the execution path set are initialized as empty sets. In step S <b> 3, the identifier / empty value pair of each terminal stored in the storage device 3 is registered in the target state management unit 9. The registered contents are shown in FIG. Subsequent procedures are executed in parallel by multithreading for each terminal.

In the present embodiment, it is assumed that the sets S and T are in the state of FIG. 8 as a result of the cover operation control unit 5 performing the procedures of FIGS. 13 to 17 in parallel at each terminal. Further, it is assumed that the GUI state of the displayed test application of each terminal is s ₁ . At this time, as shown in FIG. 8, the state before the operation of the set T is s ₁ and the state after the transition is TBD, that is, there is no GUI element that has not been operated. Accordingly, the thread for each terminal proceeds to step S6991 in FIG. 16, and searches for a path from the state s ₁ to a state having an unoperated GUI element. Since each thread searches for a path with the same algorithm, the same path can be acquired here for each thread, and here the path is (s ₁ , radio button 2, s ₂ ).

If each thread proceeds to step S69951 in FIG. 18 as it is and automatic operation is performed along this path, the displayed GUI status is s ₂ for all three terminals. As shown by the state transition set T in FIG. 8, there are only two transitions in which the state before the operation is s ₂ and the state after the operation is TBD. Thus, for example, when the GUI state of the 192.168.1.1 and 192.168.1.2 terminals changes to s ₂ before the 192.168.1.3 terminal, 192.168.1. 1 and 192.168.1.2 terminals can operate radio button 2 and button 1 which are non-operated GUI elements, but 192.168.1.3 terminals automatically operate unoperated GUI elements at s ₂ Even if an attempt is made to perform an operation, since the corresponding GUI element does not exist, processing automatically operated along the path is wasted.

Therefore, in the present invention, step S6993 in FIG. 16 is executed before the automatic operation along the path. That is, the state s ₂ after the operation of the last state transition of the path (s ₁ , radio button 2, s ₂ ) is registered in the target state management unit as a pair with the terminal identifier. In the multithreading process, the processing of each thread is not performed at the same time, but strictly, it is performed sequentially. Therefore, when the thread for 192.168.1.1 executes step S6993 for the first time, the list of terminal identifier / target state pairs in the target state management unit 9 is shown in FIG. Since the exclusive lock on the pair of the identifier of the terminal and the target state in the target state management unit 9 is not released, other threads cannot wait for step S6993 and wait. Therefore, the thread for 192.168.1.1 further executes step S6994. Since the number N of pairs whose target state is the state s ₂ of the target state management unit 9 and the number N _{TBD of} unoperated GUI elements in the state s ₂ are N _TBD = 2, in this case, N ≦ N _TBD is determined in step S6995. Satisfies and proceeds to step S69951 in FIG. As a result, unoperated GUI elements can be automatically operated by automatically operating along the path. When the thread of 192.168.1.1 proceeds to step S69951 in FIG. 17, the exclusive lock on the pair of the terminal identifier and the target state in the target state management unit 9 is released.

Next, when the thread of 192.168.1.2 executes step S6993, the list of terminal identifier / target state pairs in the target state management unit 9 is as shown in FIG. 21 (a) or (b). (A) shows a case of a state where the thread 192.168.1.1 has not yet operated the GUI elements unmanipulated in state s _2. If the 192.168.1.1 thread is operating an unoperated GUI element in state s ₂ , (b) is obtained. The case of (a), the number N = 2, GUI unmanipulated state s ₂ pairs thread 192.168.1.2 target state of the target state management section 9 in step S6994 is in the state s ₂ Since the number of elements N _TBD = 2, the determination of N ≦ N _TBD is satisfied in step S6995, and the process proceeds to step S69951 in FIG. As a result, unoperated GUI elements can be automatically operated by automatically operating along the path. Even In the case of (b), the target state of the target state management section 9 at step S6994 the number N = 1,192.168.1.1 thread pair is a state s ₂ has not been operated in the state s ₂ Since the GUI element is operated, the number N _{TBD of} unoperated GUI elements in the state s ₂ becomes N _TBD = 1, and the determination of N ≦ N _TBD is satisfied in step S6995, and the process proceeds to step S69951 in FIG. As a result, by automatically operating along the path, the 192.168.1.2 thread can also automatically operate the GUI element that has not been operated. When the 192.168.1.2 thread proceeds to step S69951 in FIG. 17, the exclusive lock on the pair of the terminal identifier and the target state in the target state management unit 9 is released.

Finally, when the thread of 192.168.1.3 executes step S6993, a list of terminal identifier / target state pairs in the target state management unit 9 is shown in FIG. However, it is assumed that the threads of 192.168.1.1 and 192.168.1.2 have not yet operated an unoperated GUI element in the state s ₂ . In step S6994, the thread of 192.168.1.3 sets the number N of pairs whose target state of the target state management unit 9 is the state s ₂ N = 3, the number of unoperated GUI elements in the state s ₂ N _TBD = 2. Therefore, the determination of N ≦ N _TBD is not satisfied in step S6995. Therefore, the routine proceeds to step S6996, and rewrites the target state management unit 9 own terminal identifier 192.168.1.3 and target state s ₂ pairs to a pair of 192.168.1.3 and null. The pair of the terminal identifier and the target state managed by the target state management unit 9 at this time is shown in FIG. Then, the process returns to step S6991 to search again for a path to a state having another unoperated GUI element.

As described above, the thread for the terminal of 192.168.1.3 is surely set to an unoperated GUI element without causing the GUI of the application to be tested to transition to a state s _{2 in} which an unoperated GUI element may not exist. It is possible to make a transition to a state where there exists. As a result, it is possible to shorten the time for generating the state transition graph by comprehensive automatic operation.

  DESCRIPTION OF SYMBOLS 1 ... Input device, 2 ... Computer main body, 3 ... Memory | storage device, 4 ... Display apparatus, 5 ... Coverage operation control part, 6 ... GUI state acquisition part, 7 ... GUI automatic operation part, 8 ... Path set search part, 9 ... Target state management department.

Claims (6)

A method of comprehensively and automatically operating the state of a graphical user interface (GUI) of a test target application operating on a plurality of terminals,
A first step of starting a test target application in each terminal by executing a startup file stored in advance in a storage unit on each of the plurality of terminals;
Furthermore, for each terminal,
A second step of acquiring the GUI state of the test target application displayed on the display means of the terminal;
A third step of obtaining all GUI elements operable in the GUI state when the displayed GUI state has never been obtained;
A fourth step of restarting the test target application by executing the startup file and continuing the processing recursively with respect to the displayed GUI state when the test target application is in a finished state;
When there is a GUI element that has not been operated in the displayed GUI state, the GUI element of the test target application is automatically transitioned by automatically operating the GUI element, and processing is recursively continued for the state after the transition. A fifth step;
When there is no GUI element that has not been operated in the displayed GUI state and there is an GUI element that has not been operated in the other GUI states acquired so far, the GUI element that has not been operated from the displayed GUI state. When a path that reaches the state (target state) of the GUI in which is present is extracted, it is determined whether there is a GUI element that is not yet operated when the state is transitioned to the target state, and it is not determined that it does not exist A sixth step of extracting a path to another target state and performing the determination process again;
A seventh step of automatically changing the GUI element along the extracted path to change the GUI to a state in which an unoperated GUI element exists, and continuing the processing recursively for the changed state;
An eighth step of ending the process when there is no unoperated GUI element in all the obtained GUI states;
A graphical user interface automatic operation method characterized by processing in parallel between terminals. In the sixth step, when a path to the target state is extracted, target state management information indicating that the state is changed to the target state in the terminal related to the self-process is stored in a shared storage unit in each parallel process. 2. The method for automatically operating a graphical user interface according to claim 1, wherein during the period from the storage process to the determination process, exclusive writing control of the target state management information to the storage means is performed between the parallel processes. . Target state management information consists of a set of terminal identifier and target state,
After the first step, a pair of terminal identifier and null value is stored in the storage means as target state management information for each terminal,
In the sixth step, when the path to the target state is extracted, the target management information is rewritten to a set of the identifier of the terminal related to the self-processing and the target state, and the same target as the target state related to the self-processing as the target management information The automatic operation method for a graphical user interface according to claim 2, wherein the determination processing is performed based on the number of identifiers of terminals having a state and the number of unoperated GUI elements in the target state. The path to the target state is extracted based on a set of execution paths, which is a sequence of state transitions executed in the processing so far, in the sixth step. Automatic operation method of graphical user interface. The path to the target state is extracted based on the state transition graph representing the transition of the GUI state acquired in the previous processing in the sixth step. Automatic operation method of graphical user interface. An apparatus that comprehensively and automatically operates the state of a graphical user interface (GUI) of a test target application that runs on a plurality of terminals,
GUI status acquisition means for acquiring the GUI status from the GUI displayed on the display means of the terminal;
GUI automatic operation means for operating the GUI element to change the state of the GUI,
Path set search means for extracting a path from the first state of the GUI to the second state;
The test target application is started in each terminal by executing the startup file stored in advance in the storage means on each of the plurality of terminals, and the GUI of the test target application displayed on the display means of the terminal for each terminal. The state is acquired by the GUI acquisition unit, and when the displayed GUI state has not been acquired so far, all GUI elements that can be operated in the GUI state are acquired, and the test target application has ended. In this case, the test target application is restarted by executing the startup file, and the process continues recursively with respect to the displayed GUI state, and there is an unoperated GUI element in the displayed GUI state. In this case, the GUI element is automatically operated by the GUI automatic operation means to test the target object. The state of the application GUI is changed, the process is recursively continued for the state after the transition, and there is no GUI element not yet operated in the displayed GUI state and the state of the other GUI acquired so far When there is an unoperated GUI element, the path reaching the GUI state (target state) where the unoperated GUI element exists from the displayed GUI state is extracted by the path set search means, and the target state is obtained. It is determined whether there is an unoperated GUI element when the state is changed, and if it is not determined that it does not exist, a path to another target state is extracted by the path set search means and the determination process is performed. The GUI element is automatically operated by the GUI automatic operation means along the extracted path to change the GUI to a state in which there is an unoperated GUI element. And a comprehensive operation control means for executing a process for terminating the process in parallel between terminals when there is no GUI element not yet operated in all the obtained GUI states. A graphical user interface automatic operation device characterized by that.

Images