WO2019074125A1 - Système, procédé et programme d'automatisation de processus commercial impliquant le fonctionnement d'un navigateur web - Google Patents

Système, procédé et programme d'automatisation de processus commercial impliquant le fonctionnement d'un navigateur web Download PDF

Info

Publication number
WO2019074125A1
WO2019074125A1 PCT/JP2018/038221 JP2018038221W WO2019074125A1 WO 2019074125 A1 WO2019074125 A1 WO 2019074125A1 JP 2018038221 W JP2018038221 W JP 2018038221W WO 2019074125 A1 WO2019074125 A1 WO 2019074125A1
Authority
WO
WIPO (PCT)
Prior art keywords
web page
captured image
server
display
web browser
Prior art date
Application number
PCT/JP2018/038221
Other languages
English (en)
Japanese (ja)
Inventor
光敏 嶋田
剛 袖山
Original Assignee
ＢｉｚｔｅＸ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ＢｉｚｔｅＸ株式会社 filed Critical ＢｉｚｔｅＸ株式会社
Publication of WO2019074125A1 publication Critical patent/WO2019074125A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/30—Creation or generation of source code
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/10—Office automation; Time management

Definitions

  • the present invention relates to a system, method and program for automating business processes, and more particularly to a system, method and program for automating business processes involving the operation of a web browser.
  • robots which have been introduced in factories and so on, has also spread to office work. It stores operations performed by humans on a computer to automate business processes, and is called robotic process automation (RPA). If the task can stylize the operation, for example, even the operation across multiple application software can be automated, it is faster than human being, and it can be repeated accurately without making a mistake, so much attention Bathed in
  • the present invention has been made in view of such problems, and an object thereof is to provide a system, method, and program for automating a business process involving operation of a web browser.
  • a first aspect of the present invention is a method for automating a business process involving operation of a web browser, wherein an operation specified by human being is computer readable data.
  • Displaying one or more selectable operations associated with the element receiving as an indication a selection for any of the one or more operations, and for the selected one or more elements Generating one or more data describing the specified operation.
  • each element in the web page corresponds to a position on the captured image corresponding to a position on the display of the web page of each element. It is characterized in that it can be selected by arranging an object.
  • the object has a rectangular shape surrounding a corresponding element.
  • the object is transparent before selection, and a border is displayed after selection.
  • any one of the first to fourth aspects when the specified operation causes a change in display on the web page, a captured image of the web page to be operated To display again.
  • the specified operation is a click or a tap of a link or a pull-down menu.
  • the specified operation is an input to an input form.
  • the specified operation is output to a file or input from a file.
  • the ninth aspect of the present invention is characterized in that in any one of the first to eighth aspects, an operation can be designated to a plurality of elements of the same or different web page.
  • a tenth aspect of the present invention is a program for causing a computer to execute a method for automating a business process involving operation of a web browser, the method comprising: Displaying on the web browser a captured image of the web page to be manipulated, describing as readable data, receiving a selection for a selectable element on the captured image, and Displaying the selectable one or more operations associated with the selected element; and receiving as an indication a selection for any of the one or more operations, selected one Or one or more describing the specified operation for multiple elements And transmitting the data.
  • An eleventh aspect of the present invention is a method for automating a business process involving the operation of a web browser, wherein the server describes an operation specified by a human as computer readable data.
  • Each element in the page is characterized by being selectable by arranging a corresponding object at a position on the capture image corresponding to the position on the display of the web page of each element.
  • the twelfth aspect of the present invention is characterized in that, in the eleventh aspect, the position in the source code is indicated by an absolute path.
  • the position on the display is a position when the width of the web page is determined as a predetermined width.
  • the position on the display is indicated by coordinates based on a predetermined point in the web page. Do.
  • the fifteenth aspect of the present invention is characterized in that, in the fourteenth aspect, the predetermined point is the left end or the right end of the upper end of the web page.
  • a sixteenth aspect of the present invention is characterized in that, in any one of the eleventh to fifteenth aspects, the one or more pieces of data are in JSON format.
  • a seventeenth aspect of the present invention is characterized in that, in the sixteenth aspect, the server converts the one or more data into a block style YAML format and stores it.
  • An eighteenth aspect of the present invention is characterized in that, in any one of the eleventh to seventeenth aspects, the server is one or more instances on a cloud.
  • a nineteenth aspect of the present invention is a program for causing a server and a user terminal to execute a method for automating a business process involving operation of a web browser, wherein the method is a human-specified operation.
  • the server For the server to receive the captured image of the web page to be manipulated and the source code, and interpreting the source code to describe each of the elements in the source code, in order to describe the data as computer readable data. Identifying the location on the display of the web page of each element, and transmitting the captured image and the location of the element in the source code and the location on the display to the user terminal.
  • a twentieth aspect of the present invention is a system for automating a business process involving operation of a web browser, which is an operation target for describing an operation designated by a human as computer readable data.
  • the selected Display one or more selectable operations associated with the mentment, receive a selection for any of the one or more operations as a designation, and designate a designated operation for the selected one or more elements
  • a user terminal that generates one or more data describing them and transmits the data to the server, and each element in the web page corresponds to the display position of each element on the web page in the user terminal It is characterized in that it can be
  • the user can designate an operation to each element to be automated through the captured image of the web page to be operated, and the web page which can not acquire the captured image is limited. This makes it possible to easily target a wide range of web pages for RPA.
  • FIG. 1 shows a system for automation according to a first embodiment of the present invention. It is a figure showing the flow of the user's work procedure for robot creation concerning a 1st embodiment of the present invention. It is a figure which shows the flow of the robot preparation procedure concerning the 1st Embodiment of this invention. It is a figure which shows the flow of the robot preparation procedure concerning the 1st Embodiment of this invention. It is a figure which shows the flow of the robot preparation procedure concerning the 1st Embodiment of this invention. It is a figure which shows the flow of the robot preparation procedure concerning the 1st Embodiment of this invention. It is a figure which shows the flow of the robot preparation procedure concerning the 1st Embodiment of this invention. It is a figure which shows the flow of the robot preparation procedure concerning the 1st Embodiment of this invention.
  • robot There are two steps in creating a so-called “robot” and its execution in order to have a computer automatically execute a business process involving the operation of a web browser.
  • robot has many meanings, such as a computer-executable program or code describing an operation performed by a human or data describing an operation performed by a human-readable program executable by the computer. In this specification, it is used only for the purpose of schematic explanation.
  • RPA data data interpretable or readable by a computer
  • the present invention allows a computer to interpret human-operated or designated operations to enable automation of operations on various websites, even those who do not have high-level programming skills. Make it easy to describe as In other words, the present invention can facilitate the creation of so-called "robots”.
  • FIG. 1 shows a configuration for automation according to the present embodiment.
  • the server system 100 includes a first server 110 that communicates with the user terminal 130, and a second server 120 that accesses a target website based on an instruction from the first server 110.
  • the website is provided by the third server 140 on the Internet.
  • the first server 110 and the second server 120 may each be an instance on the cloud, and may generate an instance corresponding to the second server 120 for each of one or more robots, as described later.
  • the second server 120 may be the same server without being separate from the first server 110, and in this case, between the servers. The exchange of data is not performed.
  • the server system 100 and the user terminal 130 may be collectively referred to as a “system”.
  • the first server 110 includes a communication unit 111 such as a communication interface, a processing unit 112 such as a processor and a CPU, and a storage unit 113 including a storage device such as a memory and a hard disk or a storage medium. It can be configured by executing a program for performing each process described later.
  • the first server 110 may include one or more servers or instances, and the program may include one or more programs. It can be a program product of transient.
  • first server 110 Although devices, servers or instances other than the first server 110 are not illustrated individually, like the first server 110, a communication unit, a processing unit and a storage unit are provided, and one or more programs are processing units The required functions are realized by being executed by
  • RPA data creation The procedure of RPA data creation in the present embodiment will be described.
  • the user accesses the website 300 provided by the first server 110 from the user terminal 130 (S201).
  • login is performed as needed.
  • the user terminal 130 receives a creation program for creating RPA data from the first server 110 or the system 100 using the web browser at the first terminal 130.
  • the creation program may be, for example, JavaScript, and causes an operation performed by the user to be stored in the user terminal 130, and updates the display of the website 300 as shown in FIG. 4 and later according to the user's operation. .
  • the user gives a title to the newly created robot on the website 300, and clicks or taps a "Create” button (S202).
  • the title “test robot” is given, and an Excel (registered trademark) file “test” is used as an output file for outputting data acquired as a result of an operation on a target website.
  • the robot output file “.xlsx” is associated with the robot.
  • the first server 110 that receives the creation request of the new robot from the user terminal 130 creates the second server 120 as an instance for the robot. Instances may be created for each robot, or instances being created may be assigned, or a predetermined number of robots such as four or five may be assigned to each instance. In this case, if there is no assignable instance, a new instance will be created. Further, the first server 110 associates the associated output file with the robot and stores the output file in a storage device or storage medium accessible from the storage unit 113 or the first server 110 via the network.
  • the user inputs the URL of the target web page into the URL input field 401, and clicks or taps the "display" button 402 (S203).
  • an image of a news page of a website of a company is displayed in the display section 310 of the website 300.
  • each element of the news page is selectable on the image based on the analysis result of the source code.
  • An operation field 320 is displayed on the website 300 in addition to the display field 310. In the operation field 320, the history of the operation performed by the user is left.
  • Each operation is composed of the type of operation and an object, and here, the type of “open URL” and the URL that is the object are shown.
  • the first server 110 that receives the display request of the web page from the user terminal 130 causes the second server 120 to start the browser and display the URL.
  • the second server 120 can operate the browser through an application such as Selenium.
  • Selenium it is not always necessary to activate the browser in the second server 120 at this time, and although the second server 120 displays the URL on the browser, it is possible to display it on the display. It is not necessary.
  • the second server 120 captures the displayed web page as an image, and delivers it to the first server 110 along with its source code.
  • the first server 110 transmits the received image and the analysis result of the source code to the user terminal 130, and the user terminal 130 updates the display field 310 of the website 300 as shown in FIG.
  • the update is performed by the generation program described above based on the image and the analysis result.
  • the analysis may be performed by the second server 120, and the analysis result may be passed to the first server 110. Details of the analysis will be described later.
  • an element having a path identical or similar to the selected element 501 is selected based on the analysis result of the source code.
  • elements 502 to 506 having div tags of the same class are automatically selected.
  • the target of the repeat operation is determined (S205-2).
  • an element having a path identical or similar to that of the selected element 501 is automatically selected, but is shown by, for example, highlighting a candidate element which is repeated for the selected element 501.
  • an element having an absolute path common to both of them can be repeatedly selected.
  • the screen transitions if the link is clicked, and the option is displayed if the pull-down menu is clicked.
  • the URL is newly reopened and the display section 310 of the website 300 is updated.
  • another element may be selected to similarly designate the type and target of the operation.
  • the user selects the “output to file” operation shown in FIG. 8 when acquiring the text as shown in this example, and outputs it to the output file associated in advance.
  • the output format is specified (S206). Output file association may be done here. As shown in FIG. 9, when the output file is a spreadsheet, although the output start cell of the extracted text can be specified, the specification of the output format is not necessarily required. Also, the output file provided in advance may be used as a template, and the storage location, file name, etc. may be separately specified.
  • an operation history is displayed in the operation field 320 of the website 300.
  • the user taps or clicks on the "save" button in the operation field 320 to complete the creation of the robot (S207).
  • the user terminal 130 generates RPA data corresponding to the operation history of the operation on the selected element and transmits it to the first server 110, and the first server 110 converts the data into the data as necessary. Apply and memorize.
  • RPA data transmitted from the user terminal 130 may be in JSON format, and data after conversion in the first server 110 or the system 100 may be in block-style YAML format. Also, the transmission from the user terminal 130 may be performed each time an operation is performed, instead of waiting for the specification of all the operations to be completed.
  • the first server 110 can convert the received array data into the following YAML format data and store it as a character string.
  • each method, program, terminal, device, server or system may have aspects that perform operations different from the operations described in the present specification.
  • the aspect is directed to the same operation as any of the operations described herein, and the existence of an operation different from the operations described herein means that the respective methods of the present invention It is added that it is not out of the scope of the embodiment.
  • the source code analysis of the web page to be performed in the first server 110 includes identification of the position of each element in the source code and identification of the position of each element on the display of the web page.
  • at least one of tag name and attribute of each element and height and width on display can be included.
  • the generation program In the user terminal 130, the generation program generates, based on the position information, for each element in the target web page, a position on the captured image corresponding to the position on the display in the web page of each element. Place the corresponding object.
  • the object may be, for example, a rectangular frame enclosing the corresponding element, as shown in FIG. 5, may be transparent prior to selection, and may have a border displayed after selection. .
  • the position of each element in the source code can be indicated by an absolute path, and the display position of each element on the corresponding web page is the position when the width of the web page is defined as a predetermined width, for example, 1280 pixels. can do.
  • the position can be represented by coordinates with the upper left or upper right when the web page is scrolled to the top, in other words, the left end or the right end of the upper end of the web page as the origin.
  • the coordinates may be coordinates of any one of the top left, top right, etc. of each element.
  • the same operation is repeated for the elements appearing repeatedly, but different specified operations may be repeated for the same element.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Software Systems (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Human Resources & Organizations (AREA)
  • Strategic Management (AREA)
  • Quality & Reliability (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Operations Research (AREA)
  • Marketing (AREA)
  • Computer Hardware Design (AREA)
  • Economics (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Information Transfer Between Computers (AREA)
  • Stored Programmes (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

L'invention concerne un système d'automatisation d'un processus commercial qui implique le fonctionnement d'un navigateur web. Un utilisateur accède à un site web (300) fourni par un premier serveur (110). À ce moment, un terminal utilisateur (130) reçoit, en provenance du premier serveur (110), un programme de création pour créer des données RPA dans un premier terminal (130). L'utilisateur entre l'URL d'une page web souhaitée dans une colonne d'entrée et clique sur un bouton « Affichage ». Dans l'exemple de la Figure 4, une image de la page d'actualités d'une société donnée est affichée dans une colonne Affichage (310) du site web (300), et chaque élément de la page d'actualités devient sélectionnable dans l'image sur la base d'un résultat d'analyse de code source. Lorsque la désignation de toutes les opérations nécessaires est achevée, un historique d'opérations est affiché dans une colonne Opération (320) du site web (300). Du fait que l'utilisateur clique sur le bouton « Sauvegarder » dans la colonne Opération (320), des données RPA qui correspondent à l'historique d'opérations sont générées.
PCT/JP2018/038221 2017-10-13 2018-10-12 Système, procédé et programme d'automatisation de processus commercial impliquant le fonctionnement d'un navigateur web WO2019074125A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017-199890 2017-10-13
JP2017199890A JP2019074889A (ja) 2017-10-13 2017-10-13 ウェブブラウザの操作を伴う業務プロセスを自動化するためのシステム、方法及びプログラム

Publications (1)

Publication Number Publication Date
WO2019074125A1 true WO2019074125A1 (fr) 2019-04-18

Family

ID=66100710

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/038221 WO2019074125A1 (fr) 2017-10-13 2018-10-12 Système, procédé et programme d'automatisation de processus commercial impliquant le fonctionnement d'un navigateur web

Country Status (2)

Country Link
JP (1) JP2019074889A (fr)
WO (1) WO2019074125A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111047147A (zh) * 2019-11-20 2020-04-21 深圳市法本信息技术股份有限公司 一种业务流程自动采集方法及智能终端
WO2023105696A1 (fr) * 2021-12-08 2023-06-15 日本電信電話株式会社 Dispositif d'attribution d'informations, procédé d'attribution d'informations et programme d'attribution d'informations
WO2023135772A1 (fr) * 2022-01-14 2023-07-20 日本電信電話株式会社 Dispositif, procédé et programme de traitement d'informations

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6651178B1 (ja) 2019-09-11 2020-02-19 有限会社Ｃ−Ｒｉｓｅ コンピュータプログラム、情報処理方法、情報処理装置及び情報処理システム
KR102234332B1 (ko) * 2019-10-14 2021-03-31 (주) 이든티앤에스 업무 프로세스 자동화 방법 및 그를 위한 장치
KR102337270B1 (ko) * 2020-12-11 2021-12-09 주식회사 이노룰스 Rpa 스크립트 개발을 위한 웹 기반의 통합개발환경 제공 시스템 및 그 방법

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008084665A1 (fr) * 2006-12-28 2008-07-17 International Business Machines Corporation Procédé et programme de prise en charge d'entrée de données pour gestion administrative
JP2012108740A (ja) * 2010-11-18 2012-06-07 Hitachi Ltd 入力支援方法及びプログラム
US20170001308A1 (en) * 2015-07-02 2017-01-05 Accenture Global Solutions Limited Robotic Process Automation

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008084665A1 (fr) * 2006-12-28 2008-07-17 International Business Machines Corporation Procédé et programme de prise en charge d'entrée de données pour gestion administrative
JP2012108740A (ja) * 2010-11-18 2012-06-07 Hitachi Ltd 入力支援方法及びプログラム
US20170001308A1 (en) * 2015-07-02 2017-01-05 Accenture Global Solutions Limited Robotic Process Automation

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"Field-initiated scenario design' that determines the success or failure of robots' that can be utilized at call centers of RPA solutions", CALLCENTER JAPAN, vol. 20, no. 9, 20 August 2017 (2017-08-20), pages 72 - 76 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111047147A (zh) * 2019-11-20 2020-04-21 深圳市法本信息技术股份有限公司 一种业务流程自动采集方法及智能终端
CN111047147B (zh) * 2019-11-20 2023-09-26 深圳市法本信息技术股份有限公司 一种业务流程自动采集方法及智能终端
WO2023105696A1 (fr) * 2021-12-08 2023-06-15 日本電信電話株式会社 Dispositif d'attribution d'informations, procédé d'attribution d'informations et programme d'attribution d'informations
WO2023135772A1 (fr) * 2022-01-14 2023-07-20 日本電信電話株式会社 Dispositif, procédé et programme de traitement d'informations

Also Published As

Publication number Publication date
JP2019074889A (ja) 2019-05-16

Similar Documents

Publication Publication Date Title
WO2019074125A1 (fr) Système, procédé et programme d'automatisation de processus commercial impliquant le fonctionnement d'un navigateur web
JP6452882B1 (ja) ウェブブラウザの操作を伴う業務プロセスを自動化するためのシステム、方法及びプログラム
US20210318851A1 (en) Systems and Methods for Dataset Merging using Flow Structures
JP6699669B2 (ja) 分析情報管理システム
US10514910B2 (en) Automatically mapping data while designing process flows
US10963138B1 (en) Webpage testing tool
Ruiz et al. Astrotaverna—building workflows with virtual observatory services
KR101951719B1 (ko) 그래픽 인터페이스 기반으로 이벤트 동작을 프로그래밍 하는 웹 레포팅 디자인 시스템
JP6045050B2 (ja) ユーザインタフェース一貫性チェック方法、装置およびプログラム
CN108431799B (zh) 分析信息管理系统
US20160292067A1 (en) System and method for keyword based testing of custom components
JP6539432B2 (ja) 解析サーバ装置、解析プログラム、ヒートマップ解析システム
JP2021163504A (ja) 情報処理プログラム、情報処理装置及び情報処理方法
JP2016224583A (ja) 解析サーバ装置、解析プログラム
US10362090B2 (en) Automating a process associated with a web based software application
Chang et al. A development of a web-based and user-centered process analysis system for quality improvement
JP6101880B1 (ja) Ｗｅｂページを表示するためのシステム、プログラムおよび記録媒体
Chakraborty et al. CRUD Operation on WordPress Posts From C# Over REST API
Yardimci Agriculture Insurance LLM Conversational Assistant
JP6423671B2 (ja) ウェブページ間複写装置、ウェブページ間の複写方法、およびプログラム
Jaschob et al. Mason: a JavaScript web site widget for visualizing and comparing annotated features in nucleotide or protein sequences
US20230124408A1 (en) An approach for analysis of logs from a complex physical equipment
Subramaniam et al. An Interactive Visualization Web Application for Industrial-Focused Statistical Process Control Analysis
Holzmüller-Laue et al. A BPM-based approach of human task support within automated workflows
Huang Visual Regression Testing in Practice: Problems, Solutions, and Future Directions

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18866499

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18866499

Country of ref document: EP

Kind code of ref document: A1