JP2017010088A - Program, device and method for automating screen transition of gui - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a program, device and method for realizing the automatization of operation management by configuring with a highly reusable script for even screen transitions of a GUI.SOLUTION: The present invention makes a computer function as: path search means for searching, for each diagraph using a script that executes a transition process, a path from a current display screen to a transition destination screen on the basis of a digraph when the transition destination screen is designated by a user, the digraph consisting of screens displayable by a user operation as vertexes and a transition to other screens transitionable from each screen by one instance of operation as a directed link; script selection means for selecting a script based on one or more directed links passing through a path; and script execution means for executing selected scripts in order of path.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a technique for automating GUI (Graphic User Interface) operations.

  A communication carrier (communication carrier, service provider, cloud service, etc.) provides computer resources and cloud services to a user-operated terminal via a high-speed, large-capacity network. A large amount of communication business facilities (communication equipment and server devices) are installed in such a network, and specifically, about 100,000 to 200,000 nodes are managed and managed in one data center. It is important for telecommunications carriers to operate and manage these large-scale telecommunications business facilities automatically at low cost and automatically (efficiently).

  A function corresponding to a standardized automation protocol is installed in a group of communication business facilities. In many cases, for example, a CLI (Command Line Interface) such as Telnet or SSH (Secure Shell) or SNMP (Simple Network Management Protocol) is used in order to uniformly operate and manage the entire large-scale system from the operator.

For example, in the data centers of Google (registered trademark) and Facebook (registered trademark), advanced operation management is performed for large-scale communication business facilities. According to Google, the communication business equipment itself is developed for its own use, and is configured so that it can be uniformly operated and managed (for example, see Non-Patent Document 1).
According to Facebook, OCP (Open Compute Project, see Non-Patent Document 2, for example) is proposed, and hardware such as a server / storage / data center that is most efficient for scalable computing is designed.
Of course, these efforts are effective in the case where a single telecommunications carrier can purchase a large amount of telecommunications business equipment. On the other hand, the communication business facilities of other general communication carriers are generally multi-vendor / multi-device environments in which different device vendors and communication devices are mixed. This is in consideration of the risk of equipment introduction stability and cost reduction, and it is difficult to construct a uniform communication business facility.

In addition, there is an automation technique that performs centralized operation management from a management server by installing a client function in advance in a large number of server devices that are operation target devices (see, for example, Non-Patent Document 3). According to this technology, the management server registers a ruby script for operation management, and the operation management information is reflected on each device by a client function. In particular, it has been actively adopted by service providers.
Also, unified operation management is being realized for network devices driven by an OS (Operating System) by adopting a general-purpose OS such as Linux (registered trademark). However, additional installation of client functions for operation management cannot be applied to existing (less expandable) network devices.

  Furthermore, there are standardization techniques for setting management protocols for network devices (see, for example, Non-Patent Document 4) and operation automation techniques for multi-vendor / multi-device environments using the same (for example, see Non-Patent Document 5). These technologies are also premised on equipment that supports an automation protocol represented by NETCONF and operation management by the above-described CLI or SNMP.

S. Jain et al., “B4: Experience with a Globally-Deployed Software Defined WAN,” proc. Of the ACM SIGCOMM 2013, pp. 3-14 Open Compute Project (OCP), [online], [Search June 3, 2015], Internet <URL: http://opencompute.org/> Chef, [online], [Search June 3, 2015], Internet <URL: https://www.chef.io/chef/> IETF RFC6241 NETCONF (Network Configuration Protocol), [online], [Search June 3, 2015], Internet <URL: https://tools.ietf.org/html/rfc6241> Cisco, [online], [Search June 3, 2015], Internet <URL: http://www.tail-f.com/> Sikuli, [online], [Search June 3, 2015], Internet <URL: http://www.sikuli.org/> T.H. Chang et al., “GUI Testing Using Computer Vision,” proc. Of the SIGCHI Conference on Human Factors in Computing Systems 2010 (CHI ’10), pp. 1535-1544 Pywinauto, [online], [Search June 3, 2015], Internet <URL: https: //code.google.com/p/pywinauto/> A. Memon et al., “The first decade of GUI ripping: Extensions, applications, and broader impacts,” proc. Of 20th Working Conference on Reverse Engineering (WCRE), 2013 pp. 11-20

  Some communication devices can be remotely operated and managed by a dedicated GUI to facilitate setting and status confirmation from the user. However, each type of communication device is individually managed and operated by using a dedicated GUI. It ’s difficult. However, in reality, there is a communication device that permits an operation only through a dedicated GUI as a user interface for an operator. Even with such communication devices, there are many cases in which they must be operated as communication business facilities from the viewpoint of introduction costs. Here, it is important to automate operation management for a communication device that permits operation only through such a dedicated GUI.

  FIG. 1 is an explanatory diagram showing a transition of operation management comparing a command line interface and a graphic user interface.

FIG. 1A shows three basic work steps for automating operator management.
S1: Status check (checks the current setting status of the device)
S2: Setting (changing device settings)
S3: Status check (check the current setting status of the device again)

  According to FIG.1 (b), the operation content based on CLI like Telnet and SSH is represented corresponding to Fig.1 (a). When using CLI, a command can be input in association with a work step. Here, by creating a module (script) composed of one or more command strings for each setting and status confirmation work step, the module can be reused (same processing is executed).

  FIG. 1C shows a GUI screen (page) corresponding to FIG. Similarly, a screen associated with the work step can be displayed.

However, in order to automatically open the target screen desired by the operator, the GUI needs state management that can be changed for each page (stateful). In the case of GUI, the transition path to the “target screen” differs depending on the “current display screen”. Therefore, a script that automates the transition path to the “target screen” needs to be created for each “current display screen”, and its reusability is low. The script can be reused only with the same combination of “current display screen” and “target screen”. That is, it cannot be expected that the quality of the automated procedure is improved by repeatedly using the reusable script.
In addition, it is necessary to program the transition from the unintended screen to the target screen and the transition from all screens to the target screen. In particular, when a pop-up screen for displaying an alert or an error is displayed according to the setting state of the device, it is necessary to consider transitions from these screens.

  FIG. 2 is an explanatory diagram showing screen transition switching in the graphic user interface.

According to FIG. 2, in order to open the screen as shown in FIG.
Top screen A-> Screen 1->"Status check screen B"-> Screen 1-> Screen 2-> Screen 3
->"Setting screen C"-> Screen 2->"Status confirmation screen B"
Here, even when the “state confirmation screen B” is displayed, there may be a transition from the screen 1 or a transition from the screen 2. In addition, when displaying the “setting screen C”, it is necessary to make a transition from the screen 3 without fail. Thus, in the case of GUI, it has the state which can be changed for every screen. It is difficult to reuse the screen transition processing module like the CLI in FIG. 1B that does not require state management.

  Therefore, an object of the present invention is to provide a program, an apparatus, and a method that realize automation of operation management by configuring GUI screen transitions with highly reusable scripts.

According to the present invention, in a user interface program that causes a computer to function as a graphic user interface that transitions between a plurality of screens,
Executes transition processing for each directed edge for a directed graph with all the screens that can be displayed by user operations as vertices and transitions from each screen to other screens that can be transitioned with a single operation. Using a script to
A route search means for searching a route from the current display screen to the transition destination screen based on the directed graph when the transition destination screen is designated by the user;
Script selection means for selecting a script based on one or more directed edges passing through the route;
The computer is caused to function as script execution means for executing the selected scripts in the order of paths.

According to another embodiment of the user interface program of the present invention,
Directed graph generation means for generating a directed graph with all the screens that can be displayed by user operation as vertices and transitions from each screen to other screens that can be transitioned by one operation as directed edges;
It is also preferable to further cause the computer to function as script generation means for generating a script for executing a transition process for each directed side.

According to another embodiment of the user interface program of the present invention,
Assign a command to each transition destination screen,
The route search means preferably causes the computer to function so as to search for a route from the current display screen to the transition destination screen corresponding to the command when a command is issued by the user.

According to another embodiment of the user interface program of the present invention,
It is also preferred that the computer function as a command line interface so that transitions between screens of the graphic user interface are controlled.

According to another embodiment of the user interface program of the present invention,
The route search means preferably causes the computer to search for the shortest route based on a breadth first search.

According to another embodiment of the user interface program of the present invention,
The script generation means preferably causes the computer to function so as to use Sikuli (registered trademark) in order to automate the operation in the screen and the operation of the screen transition.

According to the present invention, in an apparatus for displaying a graphic user interface for transitioning a plurality of screens to a user,
Executes transition processing for each directed edge for a directed graph with all the screens that can be displayed by user operations as vertices and transitions from each screen to other screens that can be transitioned with a single operation. Using a script to
A route search means for searching a route from the current display screen to the transition destination screen based on the directed graph when the transition destination screen is designated by the user;
Script selection means for selecting a script based on one or more directed edges passing through the route;
Script execution means for executing the selected script in the path order.

According to the present invention, in a screen transition method of a device having a graphic user interface for transitioning a plurality of screens,
Executes transition processing for each directed edge for a directed graph with all the screens that can be displayed by user operations as vertices and transitions from each screen to other screens that can be transitioned with a single operation. Using a script to
Searching for a route from the current display screen to the transition destination screen based on the directed graph when the transition destination screen is designated by the user;
Selecting a script based on one or more directed edges passing through the path;
And executing the selected script in the order of the paths.

  According to the program, apparatus and method of the present invention, automation of operation management can be realized by configuring GUI screen transitions with highly reusable scripts.

It is explanatory drawing showing the transition of the operation management which compared the command line interface and the graphic user interface. It is explanatory drawing showing switching of the screen transition in a graphic user interface. It is a functional block diagram in the graphic user interface program of this invention. It is explanatory drawing showing the directed graph of a screen transition. It is explanatory drawing showing the screen transition of the directed side script in this invention. It is a program code showing the process of the screen transition of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 3 is a functional configuration diagram of the graphic user interface program of the present invention.

  According to FIG. 3, the terminal 1 operates and manages a target device having a dedicated GUI, and is operated by an operator. The GUI screen is for operating a remote target device, and may be described in a text markup language such as HTML (Hypertext Markup Language).

  According to FIG. 3, the graphic user interface program installed in the terminal 1 includes a directed graph generation unit 11, a directed graph storage unit 110, a script generation unit 12, a script storage unit 120, a route search unit 13, and a script selection. Unit 14 and script execution unit 15. These functional components are realized by executing a program that causes a computer mounted on a terminal (device) to function. The processing flow of these functional components can be understood as a screen transition method.

[Directed Graph Generation Unit 11 / Directed Graph Accumulation Unit 110]
The directed graph generation unit 11 generates a directed graph that connects GUI screens.
All the screens (pages) that can be displayed by user operation are set as “vertices”.
A transition from one screen to another screen that can be transitioned with a single operation is defined as a “directed side”.
The directed graph here is simply an edge connecting vertices and may have no weight.

The directed graph itself can be generated using an existing GUI ripping technique (see, for example, Non-Patent Document 9). According to this technique, screen transition is traced by reverse engineering while sequentially clicking all the buttons that allow page transition in the screen, and finally a directed graph (test scenario) of the screen is generated. This technology is a test tool for a developer who mainly develops a GUI application to comprehensively check the screen display.
In addition, the directed graph storage unit 110 stores the directed graph generated by the directed graph generation unit 11.

  FIG. 4 is an explanatory diagram showing a directed graph of screen transition.

  According to FIG. 4, seven screens (vertices) are connected by eleven directed sides. This is automatically generated by the GUI ripping of the directed graph generation unit 11.

[Script generation unit 12 / script storage unit 120]
The script generation unit 12 generates a script for executing a transition process for each directed side. A “script” refers to a simple program that does not need to be compiled (converted) into machine language. In general, executable programs need to compile source code into machine language, whereas scripts do not require compilation.

  Here, the script generation unit 12 uses, for example, Sikuli (registered trademark) in order to automate the operation in the screen and the operation of the screen transition. This technique can automate operations such as “input a word in the search field and press the search button”. In order to automatically find the user interface component to be operated, an “image recognition” function is installed. An image (screen shot) of the user interface component and an operation method for the component can be described together as a script. For example, Jython or JRuby is used as the script description language. According to Sikuli, various operations such as starting, stopping, moving the cursor, clicking the mouse, dragging and dropping, and keyboard input can be automated.

  According to the GUI, for example, by clicking a page transition button displayed on a page (screen), it is possible to transition to the next page. Here, the script generation unit 12 is programmed with Sikuli to click the “page transition button” on the screen for each directed side. Thereby, the screen transition itself can be automated by the script.

  There are other GUI control techniques for determining page contents (see, for example, Non-Patent Documents 7 and 8). In these techniques, a program interprets GUI display contents, and mouse clicks and character input can be executed from the program. The technique described in Non-Patent Document 7 can determine the display content of the screen based on image recognition. The technique described in Non-Patent Document 8 can determine the display contents by a window controller that is internally allocated for each page by the Windows (registered trademark) system.

[Route search unit 13]
When the transition destination screen is designated by the user, the route search unit 13 searches for the shortest route from the current display screen to the transition destination screen based on the directed graph. If the directed graph has no weight for edges, the simplest, for example, breadth first search algorithm can be used. This algorithm searches all adjacent nodes from one node (vertex / screen) in a brute force manner. By repeating this also for the adjacent nodes, the target node (target screen) is searched exhaustively. Since there is no weight for the edge, the path with the least number of directed edges is selected.

[Script selection unit 14]
The script selection unit 14 selects a script based on one or more directed sides that pass through the shortest path.

  FIG. 5 is an explanatory diagram showing screen transition of the directed side script in the present invention.

(S1) First, the shortest path from the current display screen A (top screen) to the next transition destination screen B (state confirmation screen) is searched using a directed graph.
Screen A- (Directed edge 1)-> Screen 1- (Directed edge 2)-> Screen B
Here, the scripts of directed sides 1 and 2 are executed in the order of the path, and the screen transits to screen B (state confirmation screen).
(S2) Next, the shortest path from the current display screen B (state confirmation screen) to the next transition destination screen C (setting screen) is searched using the directed graph.
Screen B- (Directed edge 3)-> Screen 1- (Directed edge 5)-> Screen 2- (Directed edge 10)-> Screen 3
-(Directed side 11)-> Screen C
Here, the scripts for directed sides 3, 5, 10 and 11 are executed in the order of the paths, and the screen transitions to screen C (setting screen).
(S3) Next, the shortest path from the current display screen C (setting screen) to the next transition destination screen B (state confirmation screen) is searched using the directed graph.
Screen C- (Directed side 9)-> Screen 2- (Directed side 6)-> Screen B
Here, the scripts of the directed sides 9 and 6 are executed in the order of the paths, and the screen transits to the screen B (state confirmation screen).

[Script execution unit 15]
The script execution unit 15 displays the transition destination screen to the user by executing the selected script in the order of the route. The script largely executes the following two processes.
(1) Screen transition processing for a directed edge (2) Operation processing for a vertex (screen itself) According to the present invention, in particular, by configuring the screen transition processing for a directed edge as a reusable script, It is possible to automate the operation on the devices that you have.

  FIG. 6 is a program code representing the screen transition process of the present invention.

  FIG. 6 shows a route search process (S13), a script selection process (S14), and a script execution process (S15). By executing this program code, screen transition and screen operation scripts are executed, and automation of operation management can be realized even with a dedicated GUI.

  As another embodiment, according to the present invention, as in the CLI in FIG. 1B, a command is assigned to each transition destination screen, and GUI operation management can be automated. That is, as a command line interface, transition between screens of the graphic user interface can be controlled. When a command is issued by the user, only the shortest route from the current display screen to the transition destination screen corresponding to the command is searched (route search unit 13).

  As described above in detail, according to the program, apparatus, and method of the present invention, automation of operation management can be realized by configuring GUI screen transitions with highly reusable scripts. For example, by using Sikuli, screen operations and screen transitions for the GUI can be generated as scripts. This script is configured as a reusable module like CLI even if it is a GUI.

  Various changes, modifications, and omissions of the above-described various embodiments of the present invention can be easily made by those skilled in the art. The above description is merely an example, and is not intended to be restrictive. The invention is limited only as defined in the following claims and the equivalents thereto.

DESCRIPTION OF SYMBOLS 1 Graphic user interface program 10 Ripping detection part 11 Directed graph generation part 110 Directed graph storage part 12 Script generation part 120 Script storage part 13 Path search part 14 Script selection part 15 Script execution part

Claims (8)

In a user interface program that causes a computer to function as a graphic user interface that transitions between multiple screens,
For each directed edge, a transition process is performed for each directed edge, with all the screens that can be displayed by user operation as vertices and transitions from each screen to other screens that can be transitioned by a single operation as directed edges. Using a script to execute
A route search means for searching for a route from the current display screen to the transition destination screen based on the directed graph when a transition destination screen is designated by the user;
Script selection means for selecting a script based on one or more directed edges passing through the path;
A user interface program that causes a computer to function as script execution means for executing the selected scripts in order of path. Directed graph generation means for generating a directed graph with all the screens that can be displayed by user operation as vertices and transitions from each screen to other screens that can be transitioned by one operation as directed edges;
The user interface program according to claim 1, further causing a computer to function as script generation means for generating a script for executing a transition process for each directed side. Assign a command to each transition destination screen,
The route search means causes the computer to function to search for a route from a current display screen to the transition destination screen corresponding to the command when the command is issued by a user. The user interface program according to 1 or 2. 4. The user interface program according to claim 3, wherein the computer functions as a command line interface so that transition between screens of the graphic user interface is controlled. 5. The user interface program according to claim 1, wherein the route search means causes the computer to search for the shortest route based on a breadth first search. 6. The said script production | generation means makes a computer function so that Sikuli (trademark) may be used in order to automate the operation in a screen, and the operation of a screen transition, The any one of Claim 1 to 5 characterized by the above-mentioned. User interface program. In an apparatus for displaying a graphic user interface for transitioning a plurality of screens to a user,
For each directed edge, a transition process is performed for each directed edge, with all the screens that can be displayed by user operation as vertices and transitions from each screen to other screens that can be transitioned by a single operation as directed edges. Using a script to execute
A route search means for searching for a route from the current display screen to the transition destination screen based on the directed graph when a transition destination screen is designated by the user;
Script selection means for selecting a script based on one or more directed edges passing through the path;
An apparatus comprising script execution means for executing the selected scripts in order of paths. In a screen transition method of a device having a graphic user interface for transitioning a plurality of screens,
For each directed edge, a transition process is performed for each directed edge, with all the screens that can be displayed by user operation as vertices and transitions from each screen to other screens that can be transitioned by a single operation as directed edges. Using a script to execute
Searching for a route from the current display screen to the transition destination screen based on the directed graph when a transition destination screen is designated by the user;
Selecting a script based on one or more directed edges passing through the path;
And a step of executing the selected script in the order of the path.

Images