JP6756019B1 - Information processing program, information processing method and information processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing system capable of efficiently automating processing for a web page. SOLUTION: A first display unit including an editorial unit for editing contents to be automated and a second display unit for displaying a web page to be automated as a web browser are displayed, and a first display unit and a second display unit are displayed. The display unit and the display unit are displayed at the same time, and the content for automating the processing for the web page displayed on the second display unit is incorporated into the editorial unit. [Selection diagram] Fig. 4

Description

The present invention relates to an information processing program, an information processing method, and an information processing system for automating the processing of a web page.

In recent years, a technique has been used in which a user's operation procedure on a computer is recorded and the operation procedure is automatically executed to automate the processing on the computer. Such a technique is called Robotic Process Automation (RPA).

The screen display of the web page to be automated is acquired, converted into an image, the image is displayed on the screen, the operation content performed while the image is displayed is recorded, and the recorded operation content is recorded. An operation automation device that is automatically executed is disclosed (Patent Document 1).

An operation automation device that displays a captured image of a web page and options for selectable elements on the captured image, records the operation content selected from the option, and automatically executes the recorded operation content is disclosed. (Patent Document 2).

Japanese Unexamined Patent Publication No. 2015-76042 JP-A-2019-75084

By the way, in the prior art, the operation content is recorded for the captured image of the web page, and the operation content in response to the change or update of the state of the web page due to the user's operation may not be recorded. Further, when the web page is updated, it is necessary to reopen the URL in order to capture the web page again, and it is not possible to smoothly record the operation contents.

One aspect of the present invention is an information processing program for automating the processing of a web page, the first display unit including an editorial unit for editing the contents to be automated, and a target for automation as a web browser. A second display unit that displays a web page, and a display processing means for displaying the web page, the first display unit and the second display unit are displayed at the same time, and the web displayed on the second display unit. It is an information processing program characterized by incorporating the processing for a page into the editorial unit as the content for automating the page.

Here, the second display unit does not incorporate the first mode into the editorial unit as the content for automating the processing for the element selectable on the web page, and the editorial unit as the content for automating the processing. It is preferable to have a switching selection unit for switching between the second mode and the switching selection unit, and to switch between the first mode and the second mode in response to a selection input from the user to the switching selection unit.

Further, in the first mode, a menu showing processing options is displayed for selectable elements on the web page displayed on the second display unit, and the processing selected from the options is automated. It is preferable to incorporate it into the editorial unit.

Further, in the second mode, it is preferable to enable normal web browsing on the web page.

Further, it is preferable that the editorial unit can capture the processing in the native application as the content to be automated.

Further, it is preferable that the automated processing is performed in the cloud. Further, it is preferable that the automated processing is performed on a local server. Further, it is preferable that the automated processing is performed in a cloud or a local server, and the cloud and the local server can be switched.

Another aspect of the present invention is an information processing method for automating a process for a web page, and the content to be automated is described in order to describe the process for the web page performed by a user as data that can be processed by a computer. A display processing step for displaying a first display unit including an editorial unit to be edited and a second display unit for displaying a web page to be automated as a web browser is provided, and the first display unit and the second display unit are provided. It is an information processing method characterized in that the parts are displayed at the same time and the processing for the web page displayed on the second display part is taken into the editorial part as the content to be automated.

Another aspect of the present invention is an information processing system for automating the processing of a web page, the first display unit including an editorial unit for editing the contents to be automated, and a target of automation as a web browser. A second display unit that displays a web page, and a display processing means for displaying the web page, the first display unit and the second display unit are displayed at the same time, and the web displayed on the second display unit. It is an information processing system characterized in that processing for a page is taken into the editorial unit as a content for automating the page.

Another aspect of the present invention is an information processing device for automating the processing of a web page, the first display unit including the editorial unit for editing the content to be automated, and the web page to be automated as a web browser. A second display unit for displaying the above and a display processing means for displaying the above are provided, the first display unit and the second display unit are displayed at the same time, and processing for a web page displayed on the second display unit is performed. It is an information processing apparatus characterized in that it is taken into the editorial unit as the content to be automated.

According to the present invention, it is possible to provide an information processing program, an information processing method and an information processing system capable of efficiently automating the processing of a web page. Other objects of embodiments of the present invention will become apparent by reference to the entire specification.

It is the schematic of the whole structure of the information processing system in embodiment of this invention. It is a figure which shows the structure of the server in embodiment of this invention. It is a figure which shows the structure of the client in embodiment of this invention. It is a flowchart which shows the information processing method in embodiment of this invention. It is a figure which shows the display example of the 1st display part in embodiment of this invention. It is a figure which shows the display example of the 2nd display part in embodiment of this invention. It is a figure which shows the display example of the menu image in embodiment of this invention. It is a figure explaining the process which selects the process option in embodiment of this invention. It is a figure explaining the process of adding RPA data in embodiment of this invention. It is a figure explaining the 1st mode in the modification of this invention. It is a figure explaining the 2nd mode in the modification of this invention.

As shown in FIG. 1, the information processing system 100 according to the embodiment of the present invention includes a server 102, a client 104, and an information communication network 106. The server 102 and the client 104 are connected via the information communication network 106 (for example, the Internet), and the client 104 can access the web page provided by the server 102.

Although only one server 102 is shown in FIG. 1, a plurality of servers 102 may be connected to the information communication network 106, and a web page may be provided from each of the plurality of servers 102. ..

The server 102 includes a processing unit 10, a storage unit 12, an input unit 14, an output unit 16, and a communication unit 18. The processing unit 10 includes means for performing arithmetic processing such as a CPU. The processing unit 10 has a function of providing a web page to the client 104 in the information processing system 100 according to the present embodiment by executing a web page providing program (for example, a server program) stored in the storage unit 12. To realize. The storage unit 12 includes storage means such as a semiconductor memory and a memory card. The storage unit 12 is accessiblely connected to the processing unit 10 and stores the web page providing program and information necessary for the processing. The input unit 14 includes means for inputting information. The input unit 14 includes, for example, a keyboard, a touch panel, buttons, and the like that receive input from the administrator. The output unit 16 includes means for outputting a processing result on the server 102, such as a user interface screen (UI) for receiving input information from the administrator. The output unit 16 includes, for example, a display that presents an image to the administrator. The communication unit 18 is configured to include an interface for communicating information with the client 104 via the information communication network 106. Communication by the communication unit 18 may be wired or wireless.

As shown in FIG. 3, the client 104 includes a processing unit 20, a storage unit 22, an input unit 24, an output unit 26, and a communication unit 28. The processing unit 20 includes means for performing arithmetic processing such as a CPU. By executing the client program stored in the storage unit 22, the processing unit 20 realizes the automation of processing for the web page in the information processing system 100 in the present embodiment. The storage unit 22 includes storage means such as a semiconductor memory and a memory card. The storage unit 22 is accessiblely connected to the processing unit 20 and stores a client program and information necessary for the processing. The input unit 24 includes means for inputting information. The input unit 24 includes, for example, a keyboard, a touch panel, buttons, and the like that receive input from the user. The output unit 26 includes means for outputting information necessary for processing on the client 104, such as a user interface screen (UI) for receiving input information from the user. The output unit 26 includes, for example, a display that presents an image to the user. The communication unit 28 includes an interface for communicating information with the server 102 via the information communication network 106. Communication by the communication unit 28 may be wired or wireless.

The server 102 and the client 104 can be a commonly used computer, that is, a PC (personal computer), a mobile terminal such as a smartphone, or the like. Further, a part or all of the processing of the server 102 in the following may be realized in the client 104, or a part or all of the processing of the client 104 may be realized in the server 102.

In the information processing system 100, in order to automate the processing for a web page, it is necessary to store the processing as a program, code, data, etc. in a format that can be automatically executed by a computer, and to execute the processing based on these. Is. Hereinafter, programs, codes, data, and the like that can be automatically executed by a computer are referred to as RPA data (Robotic Process Automation Data). For example, processing for a web browser can be stored as RPA data that can be executed by an application such as Selenium.

Hereinafter, the process of storing RPA data in order to automate the process for the web page by the information processing system 100 in the present embodiment will be described with reference to the flowchart of FIG. The processing for a web page means an operation that can be performed on the web page such as an operation (for example, cursor movement, click, etc.) for an element constituting the web page, an input of information such as text, and a processing associated therewith.

In step S10, an application startup process for storing RPA data is performed. By the processing in the step, the information processing system 100 functions as an activation processing means. The client 104 starts an application for storing RPA data by executing a client program stored in the storage unit 22.

In step S12, a process of displaying the first display unit including the editorial unit for editing the content to be automated is performed. By the processing in the step, the information processing system 100 functions as the first display unit display means. Along with the activation of the application in step S10, the output unit 26 of the client 104 displays the first display unit including the editing unit for editing the content to be automated.

FIG. 5 shows a display example of the first display unit 200. The first display unit 200 includes a choice display area 202, a processing content editing area 204, and a processing content input area 206.

The option display area 202 is an area indicating options for processing to be automated. The user can add a new process to be automated by selecting one of the options displayed in the option display area 202 using the input unit 24 (for example, clicking the option with the mouse). it can. Processing options include processing associated with operations on web browsers and web pages. The processing for the web browser and the web page includes the operation of opening the web page specified by the URL entered by the user and the processing thereof, the processing of updating the web page of the active window, and the area specified by the user in the web page. Examples include an operation of moving the cursor to and its processing, an operation of clicking the area where the cursor is located on a web page, and its processing. However, the processing for the web browser and the web page is not limited to these, and other processing may be an option.

Further, the processing option may be processing for a native application used locally. Here, the native application in the present specification means an application that performs arithmetic processing in the arithmetic processing means inside the terminal. Specifically, there are word processor applications, spreadsheet applications and the like.

The processing content editing area 204 is an area for displaying and editing the content of the processing to be automated. That is, the processing content editing area 204 is an area for displaying the processing stored as RPA data. It is preferable that the processing content editing area 204 displays the processing stored as RPA data in the form of a flowchart for each step. The content of the process to be automated is stored as RPA data arranged as steps in the order in which each operation or each process is automatically performed. For example, the process corresponding to the selected option in the option display area 202 is displayed in the process content editing area 204 and is added to the RPA data as a process to be automated. Further, as will be described later, the operation performed by the user on the second display unit displaying the web page and the processing performed in the second display unit are displayed in the processing content editing area 204 and are automated as RPA. Added to the data. In addition, the user can edit the processing content displayed in the processing content editing area 204 by using the input unit 24. Editing of the contents of processing includes changing the order of processing, changing the data used for processing, changing the parameters required for processing, and the like. However, editing the contents of the process is not limited to these edits, and may be any one necessary for setting the process to be automated.

The process content input area 206 is an area for inputting information necessary for automation such as data and parameters when editing the content of the process to be automated. In the processing content input area 206, an input interface image for inputting information necessary for the processing to be edited in the processing content editing area 204 is displayed. The user inputs the necessary information in the processing content input area 206 using the input unit 24, and then confirms the information to set the processing to be stored as RPA data. The confirmation of the information is performed, for example, by clicking the "execute" button.

For example, when registering a process for opening a web page in RPA data, select (click) "Open web page" in the option display area 202 as illustrated in FIG. 5 (indicating that the selection is made in the figure). Therefore, it is displayed as a process to be newly stored in the RPA data in the process content editing area 204. At the same time, as a process (step) currently being edited in the process content edit area 204, an input interface image for inputting information related to the process is displayed in the process content input area 206. Here, a name given to the process of "opening a web page", a URL for identifying the target web page, and an image for inputting an option for the process of opening the web page are displayed. When the execute button is clicked with the name "Open page A" and the URL "https: /www.xxx", the process (step) of "Opening the web page" to be edited in the process content edit area 204 is performed. On the other hand, the name "Open page A" was given, and https: / www. The process of opening a web page called xxx is stored as RPA data.

By repeating such an operation, the RPA data arranged in chronological order with the process to be automated as a step is stored in the storage unit 22. Hereinafter, in the present embodiment, a process of opening a web page and storing the process on the web page as RPA data will be described.

In step S14, a source code acquisition process for displaying a web page to be automated is performed. By the processing in the step, the information processing system 100 functions as a code acquisition means. When the URL of the web page to be automated in step S12 is specified, the processing unit 10 accesses the server 102 via the information communication network 106 to access the server 102 and the source of the web page specified by the specified URL. Get the code.

In step S16, a process of embedding a code for displaying selectable processing options on a web page to be automated is performed. By the processing in the step, the information processing system 100 functions as a code embedding means. The processing unit 10 analyzes the source code of the web page acquired in step S14, and embeds the code for displaying the processing performed on the tabs that can be processed on the web page in the source code in the menu format. Perform processing.

In step S18, a process of displaying a second display unit for displaying a web page to be automated is performed. By the processing in the step, the information processing system 100 functions as a second display unit display means. In step S16, the processing unit 10 displays a web page corresponding to the source code in which the code for displaying the menu is embedded on the web browser as the second display unit. The web browser for displaying the web page may be an existing browser or a dedicated browser in the information processing system 100.

FIG. 6 shows a display example of the second display unit 300. When the processing unit 10 receives the source code specified by the URL (https://www.xxx in the example) from the server 102, the processing unit 10 embeds the code for displaying the menu in the source code and then the second display unit. Display as 300. In the second display unit 300, the cursor 302 is moved to an image area (an item on the web page called "news 3" in the figure) that can be processed (or the mouse is right-clicked on the image area). , As shown in FIG. 7, a menu image 304 for displaying possible processes in the area as options based on the embedded code is displayed. The user can select a desired process from the options displayed as the menu image 304 using the input unit 24.

Here, it is preferable to display the first display unit 200 and the second display unit 300 at the same time. In this case, the window for displaying the first display unit 200 and the window for displaying the second display unit 300 may be displayed side by side, or a part or all of them may be displayed in an overlapping manner. That is, by switching the windows, the first display unit 200 and the second display unit 300 may be displayed in such a manner that they can be immediately operated.

In step S20, a process of storing the process for the web page is performed. By the processing in the step, the information processing system 100 functions as an RPA data storage means. When one of the options of the menu image 304 is selected, the process represented by the option is stored as RPA data.

Specifically, when the user selects one of the options from the menu image 304 displayed on the second display unit 300 using the input unit 24, the processing content editing area 204 of the first display unit 200 is used as RPA data. Add and display the stored process. For example, as shown in FIG. 8, the cursor 302 is moved to the option "click" in the menu image 304, and the mouse is clicked. As a result, as shown in FIG. 9, a new operation of "clicking" and the processing thereof are added as RPA data to the processing content editing area 204 of the first display unit 200, and the information indicating the processing content is the processing content. It is displayed in the input area 206. Therefore, the user can store the processing for the web page displayed on the second display unit 300 as RPA data by editing and confirming the content displayed on the first display unit 200.

When a new process is added to the RPA data, the process can be selected from the options displayed in the option display area 202 of the first display unit 200 and added to the RPA data as described above.

By repeating the above steps S18 and S20, the processes to be automated can be sequentially stored as RPA data.

Based on the RPA data stored in this way, processing including operations on the website can be automated.

The automation process based on the RPA data may be a process by the cloud via the information communication network 106 or a process by a local server. When processing is performed by the cloud, processing that does not depend on the performance of the local server can be realized. Further, when the processing is performed by the local server, the processing for the web page such as an in-house dedicated site that cannot be accessed from the outside via the information communication network 106 can be automated. In the information processing system 100, it is preferable to be able to switch between processing by the cloud and processing by the local server.

<Modification example>
In the above embodiment, the menu image 304 is displayed on the web page of the second display unit 300, and is incorporated into the first display unit 200 as a process to be automated by selecting a process for a selectable element on the web page. And said. Along with this, the process of registering in RPA data is set as the process of automation.

On the other hand, it may be possible to switch to a mode in which the processing for the web page displayed on the second display unit 300 is not captured by the first display unit 200.

FIG. 10 shows an example of a second display unit 300 provided with a switching unit 306 for switching between a first mode in which the menu image 304 is displayed on the web page and a second mode in which the menu image 304 is not displayed on the web page. .. Here, the toggle button is displayed on the switching unit 306, and the user can exclusively switch between the first mode and the second mode by selecting the toggle using the input unit 24.

When the first mode is selected, the display process of the second display unit 300 is performed based on the source code in which the code is embedded in step S16. After that, the process to be automated by performing the processes of steps S18 and S20 is taken into the first display unit 200 and stored as data in the RPA.

When the second mode is selected, as shown in FIG. 11, the process of embedding the code is not performed in step S16, and the display process of the second display unit 300 is performed based on the source code acquired in step S14. .. Therefore, the menu image 304 is not displayed on the second display unit 300. In addition, the second display unit 300 is in a state in which the normal browsing function of the web browser is exhibited as it is for the displayed web page. Therefore, by selecting a link on a web page, normal browsing such as opening a new web page or acquiring data on the web page can be performed.

For example, the process of automating in the state set to the first mode is registered in the RPA data, then the mode is switched to the second mode to perform normal browsing, and the state obtained as a result of the browsing is returned to the first mode. It is possible to register the process to be automated while going back and forth between the first mode and the second mode, such as registering the process to be automated in the RPA data again. Therefore, it is possible to improve the operability of the user when registering the process to be automated in the information processing system 100.

10 processing unit, 12 storage unit, 14 input unit, 16 output unit, 18 communication unit, 20 processing unit, 22 storage unit, 24 input unit, 26 output unit, 28 communication unit, 100 information processing system, 102 server, 104 client , 106 Information and communication network, 200 1st display area, 202 Choice display area, 204 Processing content editing area, 206 Processing content input area, 300 2nd display area, 302 Cursor, 304 Menu image, 306 Switching unit.

Claims (9)

An information processing program for automating the processing of web pages.
Computer,
It functions as a display processing means for displaying a first display unit including an editorial unit for editing the contents to be automated and a second display unit for displaying a web page to be automated as a web browser.
Causes display and said second display unit and the first display unit at the same time, Captures the editorial processing for a web page displayed on the second display unit as the contents of the automated,
The second display unit has a first mode in which processing for elements selectable on the web page is imported into the editorial unit as content for automating the processing, and a second mode in which the processing is not imported into the editorial unit as content for automating the processing. It has a switching selection unit that switches between
An information processing program characterized by switching between the first mode and the second mode in response to a selection input from a user to the switching selection unit . The information processing program according to claim 1 .
In the first mode, a menu showing processing options is displayed for selectable elements on the web page displayed on the second display unit, and the processing selected from the options is edited as the content for automating the processing. An information processing program characterized by being incorporated into a department. The information processing program according to claim 1 or 2 .
In the second mode, an information processing program characterized in that normal web browsing on the web page is possible. The information processing program according to any one of claims 1 to 3 .
The editorial unit is an information processing program characterized in that processing in a native application can be imported as the content to be automated. The information processing program according to any one of claims 1 to 4 .
An information processing program characterized in that the automated processing is performed in the cloud. The information processing program according to any one of claims 1 to 4 .
An information processing program characterized in that the automated processing is performed on a local server. The information processing program according to any one of claims 1 to 4 .
An information processing program characterized in that the automated processing is performed in a cloud or a local server, and the cloud and the local server can be switched. An information processing method for automating the processing of web pages.
In order to describe the processing performed by the user on the web page as data that can be processed by a computer.
A display processing step for displaying a first display unit including an editorial unit for editing the contents to be automated and a second display unit for displaying a web page to be automated as a web browser is provided.
Causes display and said second display unit and the first display unit at the same time, Captures the editorial processing for a web page displayed on the second display unit as the contents of the automated,
The second display unit has a first mode in which processing for elements selectable on the web page is imported into the editorial unit as content for automating the processing, and a second mode in which the processing is not imported into the editorial unit as content for automating the processing. It has a switching selection unit that switches between
An information processing method characterized by switching between the first mode and the second mode in response to a selection input from a user to the switching selection unit . An information processing system for automating the processing of web pages.
Computer,
It functions as a display processing means for displaying a first display unit including an editorial unit for editing the contents to be automated and a second display unit for displaying a web page to be automated as a web browser.
Causes display and said second display unit and the first display unit at the same time, Captures the editorial processing for a web page displayed on the second display unit as the contents of the automated,
The second display unit has a first mode in which processing for elements selectable on the web page is imported into the editorial unit as content for automating the processing, and a second mode in which the processing is not imported into the editorial unit as content for automating the processing. It has a switching selection unit that switches between
An information processing system characterized in that it functions to switch between the first mode and the second mode in response to a selection input from a user to the switching selection unit .

Images