CN116185411A - RPA interface determination method, RPA interface determination device, computer equipment and storage medium - Google Patents

RPA interface determination method, RPA interface determination device, computer equipment and storage medium Download PDF

Info

Publication number
CN116185411A
CN116185411A CN202310290945.0A CN202310290945A CN116185411A CN 116185411 A CN116185411 A CN 116185411A CN 202310290945 A CN202310290945 A CN 202310290945A CN 116185411 A CN116185411 A CN 116185411A
Authority
CN
China
Prior art keywords
target
element object
rpa
determining
preset
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310290945.0A
Other languages
Chinese (zh)
Other versions
CN116185411B (en
Inventor
陶晓宝
庄建伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Fengzhiding Information Technology Co ltd
Original Assignee
Suzhou Fengzhiding Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Fengzhiding Information Technology Co ltd filed Critical Suzhou Fengzhiding Information Technology Co ltd
Priority to CN202310290945.0A priority Critical patent/CN116185411B/en
Priority claimed from CN202310290945.0A external-priority patent/CN116185411B/en
Publication of CN116185411A publication Critical patent/CN116185411A/en
Application granted granted Critical
Publication of CN116185411B publication Critical patent/CN116185411B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/30—Creation or generation of source code
    • G06F8/38—Creation or generation of source code for implementing user interfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/40—Transformation of program code
    • G06F8/41—Compilation
    • G06F8/42—Syntactic analysis
    • G06F8/427—Parsing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/10—Office automation; Time management
    • G06Q10/103—Workflow collaboration or project management
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Abstract

The application relates to the technical field of flow automation processing and discloses a method, a device, computer equipment and a storage medium for determining an RPA interface, wherein the method comprises the steps of determining a target command corresponding to a preset script based on the preset script and a script analysis grammar; acquiring a target element object according to the type of the target command; and executing the target command corresponding to the target element object in a preset execution mode according to the type of the target element object so as to realize the determination of the robot flow automation (RPA) interface. By the method, after the preset script is analyzed through the preset script analysis grammar, the target command is determined, the target element object is determined according to the type of the target command, and then the command corresponding to the target element object is executed according to the preset implementation mode by determining the type of the target element object, so that the determination of the RPA interface is completed, and the efficiency of RPA automatic implementation under the composite scene is improved.

Description

RPA interface determination method, RPA interface determination device, computer equipment and storage medium
Technical Field
The present disclosure relates to the field of process automation processing technologies, and in particular, to a method and an apparatus for determining an RPA interface, a computer device, and a storage medium.
Background
The RPA is known as robotic process automation (robotics), and the main function is to interact work information with services through the robot to follow a previously designed process. Therefore, if the interaction between the working information and the service is excessive, the RPA can efficiently solve the complex processes, and the labor cost is saved. Today, RPA has become the most widely used, most effective, and highly mature intelligent software. Many enterprises wish to deploy RPAs that fit themselves.
Common RPA technical implementations cannot support automation of composite scenarios or support limitations and cannot adapt to various changing situations. If the window is written by VB, but the button is realized by Flash, and the size of the window can be dragged; the UIAutomation cannot click a button, and the operating system keyboard and mouse event method cannot process button position movement after window dragging; the Selenium cannot support Windows. Therefore, how to improve the efficiency of RPA automation implementation in a composite scenario is a technical problem to be solved.
Disclosure of Invention
The application provides a method, a device, computer equipment and a storage medium for determining an RPA interface so as to improve the efficiency of RPA automation realization in a composite scene.
In a first aspect, the present application provides a method for determining an RPA interface, where the method for determining an RPA interface includes:
determining a target command corresponding to a preset script based on the preset script and a script analysis grammar;
acquiring a target element object according to the type of the target command;
and executing the target command corresponding to the target element object in a preset execution mode according to the type of the target element object so as to realize the determination of the robot flow automation (RPA) interface.
Further, before determining the target command corresponding to the preset script based on the preset script and the script parsing grammar, the method comprises the following steps:
analyzing the target element object of the preset script through a preset analysis tool, and determining an automation technology supported by the target element object.
Further, analyzing the target element object of the preset script through a preset analysis tool, determining an automation technology supported by the target element object, including:
acquiring the target element object through MSAA/UIIAutomation grammar or a simulated keyboard and mouse technology;
if the automation technology is the simulated keyboard and mouse technology, judging whether a target window corresponding to the target element object supports zooming or not;
and if the target window supports zooming, adding a movable identifier to the target window.
Further, if the target window supports scaling, adding the movable identifier to the target window includes:
capturing a screen of the target interface, and acquiring the current position of the target element object based on a preset OpenCV image recognition technology algorithm library;
and determining a preset number of pixel coordinates at preset adjacent positions of the current position, and correcting coordinate parameters of the target element object to determine the element type of the target element object.
Further, if the target window supports scaling, adding the movable identifier to the target window includes:
matching the current parameter information of the target window with preset parameter information;
and if the current parameter information is not matched with the preset parameter information, determining that the target window supports scaling.
Further, if the target window supports scaling, adding the movable identifier to the target window includes:
and if the target window does not support zooming, executing the target command through the simulated keyboard and mouse technology to realize the determination of the RPA interface.
Further, after the target element object is acquired according to the type of the target command, the method includes:
judging whether an unprocessed element object exists or not;
if the unprocessed element object exists, acquiring the element object to be processed based on the command type corresponding to the unprocessed element object.
In a second aspect, the present application further provides a device for determining an RPA interface, where the device for determining an RPA interface includes:
the target command acquisition module is used for determining a target command corresponding to a preset script based on the preset script and a script analysis grammar;
the element object acquisition module is used for acquiring a target element object according to the type of the target command;
and the command execution module is used for executing the target command corresponding to the target element object in a preset execution mode according to the type of the target element object so as to realize the determination of the robot flow automation (RPA) interface.
In a third aspect, the present application also provides a computer device comprising a memory and a processor; the memory is used for storing a computer program; the processor is configured to execute the computer program and implement the method for determining the RPA interface as described above when the computer program is executed.
In a fourth aspect, the present application further provides a computer readable storage medium storing a computer program, which when executed by a processor causes the processor to implement a method for determining an RPA interface as described above.
The application discloses a method, a device, computer equipment and a storage medium for determining an RPA interface, wherein the method comprises the steps of determining a target command corresponding to a preset script based on the preset script and a script analysis grammar; acquiring a target element object according to the type of the target command; and executing the target command corresponding to the target element object in a preset execution mode according to the type of the target element object so as to realize the determination of the robot flow automation (RPA) interface. By the method, after the preset script is analyzed through the preset script analysis grammar, the target command is determined, the target element object is determined according to the type of the target command, and then the command corresponding to the target element object is executed according to the preset implementation mode by determining the type of the target element object, so that the determination of the RPA interface is completed, and the efficiency of RPA automatic implementation under the composite scene is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flowchart of a method for determining an RPA interface according to a first embodiment of the present application;
FIG. 2 is a schematic diagram of a script execution flow in the present embodiment;
fig. 3 is a schematic flowchart of a method for determining an RPA interface according to a second embodiment of the present application;
FIG. 4 is a schematic flow chart of editing scripts in the present application;
FIG. 5 is a schematic block diagram of a determining device of an RPA interface provided by an embodiment of the present application;
fig. 6 is a schematic block diagram of a computer device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
The flow diagrams depicted in the figures are merely illustrative and not necessarily all of the elements and operations/steps are included or performed in the order described. For example, some operations/steps may be further divided, combined, or partially combined, so that the order of actual execution may be changed according to actual situations.
It is to be understood that the terminology used in the description of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
The embodiment of the application provides a method, a device, computer equipment and a storage medium for determining an RPA interface. The method for determining the RPA interface can be applied to a server, a target command is determined after the preset script is analyzed through a preset script analysis grammar, a target element object is determined according to the type of the target command, and further the command corresponding to the target element object is executed according to a preset implementation mode by determining the type of the target element object, so that the determination of the RPA interface is completed, and the efficiency of RPA automation implementation under a composite scene is improved. The server may be an independent server or a server cluster.
Some embodiments of the present application are described in detail below with reference to the accompanying drawings. The following embodiments and features of the embodiments may be combined with each other without conflict.
Referring to fig. 1, fig. 1 is a schematic flowchart of a method for determining an RPA interface according to a first embodiment of the present application. The method for determining the RPA interface can be applied to a server, and is used for determining a target command after the preset script is analyzed through the preset script analysis grammar, determining a target element object according to the type of the target command, further executing the command corresponding to the target element object according to a preset implementation mode by determining the type of the target element object, completing the determination of the RPA interface, and improving the efficiency of RPA automatic implementation under a composite scene.
As shown in fig. 1, the method for determining the RPA interface specifically includes steps S10 to S30.
Step S10, determining a target command corresponding to a preset script based on the preset script and a script analysis grammar;
step S20, obtaining a target element object according to the type of the target command;
and step S30, executing the target command corresponding to the target element object in a preset execution mode according to the type of the target element object so as to realize the determination of the robot flow automation (RPA) interface.
In a specific embodiment, as shown in fig. 2, fig. 2 is a schematic flow chart of a script execution in the present embodiment, and the script execution process includes the following steps:
(1) Parsing the script: and reading the saved script and analyzing the script into executable commands according to a script grammar format. Software has defined a set of instruction sets and grammar rules such as click, getElementById, input and the like. For example, the script content is getElementById61723 loginbtuclicklogingbtu, and the process after program analysis is processed as "acquire id is 61723 element, defined as variable loginBtu, click loginBtu".
(2) Acquiring an interface window object handle: and executing an operation of acquiring the interface window object handle to provide window context for subsequent actions.
(3) Acquisition element: the get element operation is performed in a different manner depending on the automation technique used. The acquisition mode is (byId, byName, byAutomationName, byHandle, byClass, byXy) so that the specified element object can be acquired.
(4) For elements supporting MSAA/UIIAutomation technology: the get element and element action script commands are executed using MSAA/UIIAutomation technology.
(5) For elements supporting the Selenium technology: the get element and element action script commands are executed using a Selenium technique.
(6) For elements that are not movable and that use keyboard-mouse simulation techniques: element action script commands are executed using a manner simulating keyboard and mouse inputs.
(7) For elements that employ keyboard-mouse simulation techniques but are movable: firstly, screen capturing is carried out on a current interface, then element positions are obtained through an integrated OpenCV image recognition technology algorithm library, then 10 pixel coordinates are obtained at adjacent positions (smaller than 10 pixels) according to the positions, and the element types are analyzed. And obtaining element type coordinates in the first matching script record. The method can correct the coordinate parameters in the script to be executed, and finally execute the element action script command in a mode of simulating keyboard and mouse input.
(8) Recording elements and executing action scripts: the method comprises the steps of carrying out a first treatment on the surface of the And recording elements and executed actions as scripts by using a simulated keyboard and mouse input mode for other scripts.
(9) Determining whether the window is movable or zoomed: the position and size of the movable or scaled window may be different when executing than when editing the script, and its position and size parameters need to be processed dynamically, so the marking element needs to be a movable object when recording the script.
(10) After all the elements are processed, the execution script is completed.
The embodiment discloses a method, a device, computer equipment and a storage medium for determining an RPA interface, wherein the method comprises the steps of determining a target command corresponding to a preset script based on the preset script and a script analysis grammar; acquiring a target element object according to the type of the target command; and executing the target command corresponding to the target element object in a preset execution mode according to the type of the target element object so as to realize the determination of the robot flow automation (RPA) interface. By the method, after the preset script is analyzed through the preset script analysis grammar, the target command is determined, the target element object is determined according to the type of the target command, and then the command corresponding to the target element object is executed according to the preset implementation mode by determining the type of the target element object, so that the determination of the RPA interface is completed, and the efficiency of RPA automatic implementation under the composite scene is improved.
Based on the embodiment shown in fig. 1, in this embodiment, the step S10 includes:
analyzing the target element object of the preset script through a preset analysis tool, and determining an automation technology supported by the target element object.
Referring to fig. 3, fig. 3 is a schematic flowchart of a method for determining an RPA interface according to a second embodiment of the present application. The method for determining the RPA interface can be applied to a server, and is used for determining a target command after the preset script is analyzed through the preset script analysis grammar, determining a target element object according to the type of the target command, further executing the command corresponding to the target element object according to a preset implementation mode by determining the type of the target element object, completing the determination of the RPA interface, and improving the efficiency of RPA automatic implementation under a composite scene.
Based on the above embodiment, in this embodiment, the above steps specifically include steps S01 to S03.
Step S01, obtaining the target element object through MSAA/UIAIutomation grammar, selenium grammar or simulated keyboard and mouse technology;
step S02, judging whether a target window corresponding to the target element object supports zooming or not if the automation technology is the simulated keyboard and mouse technology;
and S03, if the target window supports zooming, adding a movable identifier to the target window.
In a specific embodiment, as shown in fig. 4, fig. 4 is a schematic diagram of an editing script flow in the present application, and the editing script process includes the following steps:
(1) Acquiring an interface window object handle: and acquiring an interface window object handle for executing the automation action in a script recording or editing tool, and recording an acquisition mode.
(2) Analysis element: and sequentially analyzing each element to be subjected to the automation action in an element-by-element manner, and acquiring the characteristics and the supported automation technology types. Element feature identification information, such as id, className, automationId, handles, etc., is grabbed by Uispy, spyXX, etc. tools. The elements that can be grabbed include buttons, input boxes, drop down selectors, text, and the like.
(3) And judging supported automation technology: and judging according to the type of the supported automation technology obtained by the analysis in the last step, such as elements with the type of buttons, analyzing the characteristics of the elements, and representing the elements as clickable and double-clickable. And collecting the identified characteristic data, such as id and name, from the element attribute through UIAutomation
(4) Recording elements and executing action scripts: for elements supporting the MSAA/UIIAutomation technology, recording the elements according to an MSAA/UIIAutomation grammar format and recording the executed actions which are written and hidden manually as scripts; for elements supporting the Selenium technology, recording the elements and executing actions as scripts according to a Selenium grammar format; and recording elements and executed actions as scripts by using a simulated keyboard and mouse input mode for other scripts.
(5) Determining whether the window is movable or zoomed: the position and size of the movable or scaled window may be different when executing than when editing the script, and dynamic processing of element position and size parameters is required, so that the marking element is required to be a movable object when recording the script. The program simulation automatic execution can calculate the x and y coordinates of the system desktop where the element is positioned according to the relative position of the window where the element is positioned, and can perform corresponding operation on the element
(6) If all the elements are processed completely, the final composite action script is saved.
The embodiment discloses a method, a device, a computer device and a storage medium for determining an RPA interface, wherein the method comprises the steps of obtaining the target element object through MSAA/UIAIutomation grammar, selenium grammar or a simulated keyboard and mouse technology; if the automation technology is the simulated keyboard and mouse technology, judging whether a target window corresponding to the target element object supports zooming or not; and if the target window supports zooming, adding a movable identifier to the target window. By the method, after the preset script is analyzed through the preset script analysis grammar, the target command is determined, the target element object is determined according to the type of the target command, and then the command corresponding to the target element object is executed according to the preset implementation mode by determining the type of the target element object, so that the determination of the RPA interface is completed, and the efficiency of RPA automatic implementation under the composite scene is improved.
Based on the embodiment shown in fig. 3, before the step S03, the method includes:
capturing a screen of the target interface, and acquiring the current position of the target element object based on a preset OpenCV image recognition technology algorithm library;
and determining a preset number of pixel coordinates at preset adjacent positions of the current position, and correcting coordinate parameters of the target element object to determine the element type of the target element object.
Based on the embodiment shown in fig. 3, before the step S03, the method includes:
matching the current parameter information of the target window with preset parameter information;
and if the current parameter information is not matched with the preset parameter information, determining that the target window supports scaling.
Based on the embodiment shown in fig. 3, before the step S03, the method includes:
and if the target window does not support zooming, executing the target command through the simulated keyboard and mouse technology to realize the determination of the RPA interface.
Based on all the above embodiments, in this embodiment, after step S20, the method includes:
judging whether an unprocessed element object exists or not;
if the unprocessed element object exists, acquiring the element object to be processed based on the command type corresponding to the unprocessed element object.
Specifically, the technical implementation of RPA is MSAA, UIAutomation, operating system keyboard and mouse events, selenium, etc. The automatic login operation execution has the following implementation modes:
1. the user name, password and login button element are obtained through a GetWindow, getElement method of the UIAIutomation technology, and then the user name, password and button are set through a SetValue mode, an InvokePattern mode, an Invoke mode and the like, and the button is clicked.
2. And setting a window focus through the movement of the simulated mouse, inputting and filling in a user name and a password through the simulated keyboard, and clicking a login button on the simulated mouse.
If the Web page is a Web page, the elements such as a user name, a password, a login button and the like can be acquired through a Selenium framework, and then the methods such as SetText, click of the Selenium are called to set the user name and the password and click the button.
Referring to fig. 5, fig. 5 is a schematic block diagram of an RPA interface determining apparatus according to an embodiment of the present application, where the RPA interface determining apparatus is configured to execute the foregoing RPA interface determining method. The determining device of the RPA interface may be configured in a server.
As shown in fig. 5, the RPA interface determining device 400 includes:
the target command acquisition module 10 is configured to determine a target command corresponding to a preset script based on the preset script and a script parsing grammar;
an element object obtaining module 20, configured to obtain a target element object according to the type of the target command;
and the command execution module 30 is configured to execute the target command corresponding to the target element object in a preset execution manner according to the type of the target element object, so as to determine an RPA interface of the robot flow automation.
Further, the determining device of the RPA interface further includes:
and the automation technology determining module is used for analyzing the target element object of the preset script through a preset analysis tool and determining the automation technology supported by the target element object.
Further, the automated technology determination module includes:
the grammar screening unit is used for acquiring the target element object through MSAA/UIAIutomation grammar, selenium grammar or simulated keyboard and mouse technology;
the zoom judging unit is used for judging whether the target window corresponding to the target element object supports zooming or not if the automatic technology is the simulated keyboard and mouse technology;
and the movable identification unit is used for adding the movable identification to the target window if the target window supports scaling.
Further, the automated technology determination module includes:
the image recognition unit is used for capturing a screen of the target interface and acquiring the current position of the target element object based on a preset OpenCV image recognition technology algorithm library;
and the element correction unit is used for determining a preset number of pixel coordinates at preset adjacent positions of the current position and correcting the coordinate parameters of the target element object to determine the element type of the target element object.
Further, the automated technology determination module includes:
the parameter matching unit is used for matching the current parameter information of the target window with preset parameter information;
and the support scaling judging unit is used for determining that the target window supports scaling if the current parameter information is not matched with the preset parameter information.
Further, the automated technology determination module includes:
and the non-support scaling judging unit is used for executing the target command through the simulated keyboard and mouse technology to realize the determination of the RPA interface if the target window does not support scaling.
Further, the determining device of the RPA interface further includes an element checking module, which specifically includes:
an unprocessed element unit for judging whether an unprocessed element object exists;
an unprocessed element obtaining unit, configured to obtain, if an unprocessed element object exists, an element object to be processed based on a command type corresponding to the unprocessed element object.
It should be noted that, for convenience and brevity of description, the specific working process of the apparatus and each module described above may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
The apparatus described above may be implemented in the form of a computer program which is executable on a computer device as shown in fig. 6.
Referring to fig. 6, fig. 6 is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device may be a server.
With reference to FIG. 6, the computer device includes a processor, memory, and a network interface connected by a system bus, where the memory may include a non-volatile storage medium and an internal memory.
The non-volatile storage medium may store an operating system and a computer program. The computer program comprises program instructions that, when executed, cause the processor to perform any of the methods of determining the RPA interface.
The processor is used to provide computing and control capabilities to support the operation of the entire computer device.
The internal memory provides an environment for the execution of a computer program in a non-volatile storage medium that, when executed by a processor, causes the processor to perform any of the methods of determining the RPA interface.
The network interface is used for network communication such as transmitting assigned tasks and the like. It will be appreciated by those skilled in the art that the structure shown in fig. 6 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
It should be appreciated that the processor may be a central processing unit (CentralProcessingUnit, CPU), but may also be other general purpose processors, digital signal processors (DigitalSignalProcessor, DSP), application specific integrated circuits (ApplicationSpecificIntegratedCircuit, ASIC), field programmable gate arrays (Field-ProgrammableGateArray, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Wherein in one embodiment the processor is configured to run a computer program stored in the memory to implement the steps of:
determining a target command corresponding to a preset script based on the preset script and a script analysis grammar;
acquiring a target element object according to the type of the target command;
and executing the target command corresponding to the target element object in a preset execution mode according to the type of the target element object so as to realize the determination of the robot flow automation (RPA) interface.
In one embodiment, based on a preset script and a script parsing grammar, before determining a target command corresponding to the preset script, the method is used for realizing:
analyzing the target element object of the preset script through a preset analysis tool, and determining an automation technology supported by the target element object.
In one embodiment, the automation technology supported by the target element object is determined by analyzing the target element object of the preset script through a preset analysis tool, and is used for realizing:
acquiring the target element object through MSAA/UIIAutomation grammar or a simulated keyboard and mouse technology;
if the automation technology is the simulated keyboard and mouse technology, judging whether a target window corresponding to the target element object supports zooming or not;
and if the target window supports zooming, adding a movable identifier to the target window.
In one embodiment, if the target window supports scaling, before adding the movable identifier to the target window, the method is used for realizing:
capturing a screen of the target interface, and acquiring the current position of the target element object based on a preset OpenCV image recognition technology algorithm library;
and determining a preset number of pixel coordinates at preset adjacent positions of the current position, and correcting coordinate parameters of the target element object to determine the element type of the target element object.
In one embodiment, if the target window supports scaling, before adding the movable identifier to the target window, the method is used for realizing:
matching the current parameter information of the target window with preset parameter information;
and if the current parameter information is not matched with the preset parameter information, determining that the target window supports scaling.
In one embodiment, if the target window supports scaling, before adding the movable identifier to the target window, the method is used for realizing:
and if the target window does not support zooming, executing the target command through the simulated keyboard and mouse technology to realize the determination of the RPA interface.
In one embodiment, after the target element object is acquired according to the type of the target command, the method is used for realizing:
judging whether an unprocessed element object exists or not;
if the unprocessed element object exists, acquiring the element object to be processed based on the command type corresponding to the unprocessed element object.
Embodiments of the present application further provide a computer readable storage medium, where the computer readable storage medium stores a computer program, where the computer program includes program instructions, and the processor executes the program instructions to implement a method for determining any RPA interface provided in the embodiments of the present application.
The computer readable storage medium may be an internal storage unit of the computer device according to the foregoing embodiment, for example, a hard disk or a memory of the computer device. The computer readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk provided on the computer device, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash memory card (FlashCard), etc.
While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. A method for determining an RPA interface, comprising:
determining a target command corresponding to a preset script based on the preset script and a script analysis grammar;
acquiring a target element object according to the type of the target command;
and executing the target command corresponding to the target element object in a preset execution mode according to the type of the target element object so as to realize the determination of the robot flow automation (RPA) interface.
2. The method for determining an RPA interface according to claim 1, wherein before determining the target command corresponding to the preset script based on the preset script and the script parsing syntax, the method comprises:
analyzing the target element object of the preset script through a preset analysis tool, and determining an automation technology supported by the target element object.
3. The method for determining an RPA interface according to claim 2, wherein the analyzing the target element object of the preset script by a preset analysis tool, and determining an automation technology supported by the target element object, comprises:
acquiring the target element object through MSAA/UIIAutomation grammar or a simulated keyboard and mouse technology;
if the automation technology is the simulated keyboard and mouse technology, judging whether a target window corresponding to the target element object supports zooming or not;
and if the target window supports zooming, adding a movable identifier to the target window.
4. A method of determining an RPA interface according to claim 3, wherein if the target window supports scaling, before adding a movable identifier to the target window, the method comprises:
capturing a screen of the target interface, and acquiring the current position of the target element object based on a preset OpenCV image recognition technology algorithm library;
and determining a preset number of pixel coordinates at preset adjacent positions of the current position, and correcting coordinate parameters of the target element object to determine the element type of the target element object.
5. A method of determining an RPA interface according to claim 3, wherein if the target window supports scaling, before adding a movable identifier to the target window, the method comprises:
matching the current parameter information of the target window with preset parameter information;
and if the current parameter information is not matched with the preset parameter information, determining that the target window supports scaling.
6. A method of determining an RPA interface according to claim 3, wherein if the target window supports scaling, before adding a movable identifier to the target window, the method comprises:
and if the target window does not support zooming, executing the target command through the simulated keyboard and mouse technology to realize the determination of the RPA interface.
7. The method for determining an RPA interface according to any one of claims 1 to 6, wherein after the obtaining the target element object according to the type of the target command, the method comprises:
judging whether an unprocessed element object exists or not;
if the unprocessed element object exists, acquiring the element object to be processed based on the command type corresponding to the unprocessed element object.
8. A RPA interface determining device, comprising:
the target command acquisition module is used for determining a target command corresponding to a preset script based on the preset script and a script analysis grammar;
the element object acquisition module is used for acquiring a target element object according to the type of the target command;
and the command execution module is used for executing the target command corresponding to the target element object in a preset execution mode according to the type of the target element object so as to realize the determination of the robot flow automation (RPA) interface.
9. A computer device, the computer device comprising a memory and a processor;
the memory is used for storing a computer program;
the processor being configured to execute the computer program and to implement the method of determining an RPA interface according to any one of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program, which when executed by a processor causes the processor to implement the method of determining an RPA interface according to any one of claims 1 to 7.
CN202310290945.0A 2023-03-23 RPA interface determination method, RPA interface determination device, computer equipment and storage medium Active CN116185411B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310290945.0A CN116185411B (en) 2023-03-23 RPA interface determination method, RPA interface determination device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310290945.0A CN116185411B (en) 2023-03-23 RPA interface determination method, RPA interface determination device, computer equipment and storage medium

Publications (2)

Publication Number Publication Date
CN116185411A true CN116185411A (en) 2023-05-30
CN116185411B CN116185411B (en) 2024-04-30

Family

ID=

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200401431A1 (en) * 2019-06-19 2020-12-24 Sap Se Adaptive web-based robotic process automation
CN112231033A (en) * 2019-12-23 2021-01-15 北京来也网络科技有限公司 Software interface element matching method and device combining RPA and AI
CN114090155A (en) * 2021-11-17 2022-02-25 中国建设银行股份有限公司 Robot process automation interface element positioning method and device and storage medium
US20220083181A1 (en) * 2020-09-14 2022-03-17 UiPath, Inc. User interface (ui) mapper for robotic process automation
CN114895884A (en) * 2022-05-20 2022-08-12 深圳市比量科技传媒有限公司 Method and system for compiling and executing RPA script of terminal equipment
CN114995816A (en) * 2022-06-24 2022-09-02 中电金信软件有限公司 Business process configuration method and device, electronic equipment and readable storage medium
CN115437621A (en) * 2021-06-03 2022-12-06 北京中关村科金技术有限公司 Process editing method and device based on robot process automation
WO2022252637A1 (en) * 2021-06-01 2022-12-08 平安科技（深圳）有限公司 Browser-based rpa implementation method and apparatus, device, and medium

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200401431A1 (en) * 2019-06-19 2020-12-24 Sap Se Adaptive web-based robotic process automation
CN112231033A (en) * 2019-12-23 2021-01-15 北京来也网络科技有限公司 Software interface element matching method and device combining RPA and AI
US20220083181A1 (en) * 2020-09-14 2022-03-17 UiPath, Inc. User interface (ui) mapper for robotic process automation
WO2022252637A1 (en) * 2021-06-01 2022-12-08 平安科技（深圳）有限公司 Browser-based rpa implementation method and apparatus, device, and medium
CN115437621A (en) * 2021-06-03 2022-12-06 北京中关村科金技术有限公司 Process editing method and device based on robot process automation
CN114090155A (en) * 2021-11-17 2022-02-25 中国建设银行股份有限公司 Robot process automation interface element positioning method and device and storage medium
CN114895884A (en) * 2022-05-20 2022-08-12 深圳市比量科技传媒有限公司 Method and system for compiling and executing RPA script of terminal equipment
CN114995816A (en) * 2022-06-24 2022-09-02 中电金信软件有限公司 Business process configuration method and device, electronic equipment and readable storage medium

Similar Documents

Publication Publication Date Title
US20210256708A1 (en) Interactive image matting using neural networks
US11886892B2 (en) Machine learned retraining for detection of user interface controls via variance parameters
US11966758B2 (en) Method and system for accessing table content in a digital image of the table
JP2018535459A (en) Robotic process automation
US11726802B2 (en) Robust user interface related robotic process automation
US7805008B2 (en) Structural content filtration of hypotheses in a cognitive control framework
CN110851299A (en) Automatic flow exception eliminating method, device, equipment and storage medium
US20170322637A1 (en) Mouse action capture and replay system and method, recording medium, and program product
CN111143188A (en) Method and equipment for automatically testing application
JP2021018104A (en) Automatic determination processor, method for automatic determination processing, inspection system, program, and recording medium
CN111208998A (en) Method and device for automatically laying out data visualization large screen and storage medium
KR102011212B1 (en) Method for Collecting and Saving object that is used as training data of Neural network for Artificial Intelligence
CN116185411B (en) RPA interface determination method, RPA interface determination device, computer equipment and storage medium
CN116185411A (en) RPA interface determination method, RPA interface determination device, computer equipment and storage medium
KR102439574B1 (en) System for Searching Object Based on Image Search for Robot Process Automation
US20230169399A1 (en) System and methods for robotic process automation
CN114490320A (en) Automatic testing method and device, electronic equipment and storage medium
Pan et al. Research on functional test of mobile app based on robot
CN117112446B (en) Editor debugging method and device, electronic equipment and medium
WO2024023948A1 (en) Analysis device, analysis method, and analysis program
JP2019125128A (en) Information processing device, control method and program
CN112015650B (en) Event testing method and device based on computer vision
CN108667911B (en) Data acquisition method and system for application program
CN115658531A (en) Automatic testing method and system applied to VR (virtual reality) all-in-one machine platform application layer
CN115858079A (en) Data processing method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant