CN114895884A - Method and system for compiling and executing RPA script of terminal equipment - Google Patents

Abstract

The invention provides a method and a system for compiling and executing terminal equipment RPA scripts, wherein the method comprises the steps of obtaining target page information and defining RPA script relations on each target page respectively; after the RPA script runs, running a terminal APP on the terminal equipment by using an ADB tool and monitoring page information of the equipment in real time; after the RPA script is stable in the page, matching the page according to the priority and executing the page action, wherein the page action comprises the following steps: when the RPA script receives the latest equipment page information, a page state object is constructed; matching the target page objects according to the priority order after the page state objects are stable; and executing the target action corresponding to the matched target page object. The invention can solve the problems of complicated RPA script compiling, untimely page response, difficult multi-step task coping and the like in the prior art, thereby improving the RPA script compiling efficiency, the script execution efficiency and the script execution accuracy.

Description

Method and system for compiling and executing RPA script of terminal equipment

Technical Field

The invention relates to the technical field of computers, in particular to a compiling and executing method of terminal equipment RPA scripts and a compiling and executing system applying the method.

Background

At present, with the development of computer technology, script files are widely used in various scenes. Scripting languages, such as Python (object-oriented, interpreted programming language), TCL (Tool command language), etc., are also called dynamic languages, which are interpreted languages that do not require compilation and can be run directly. The script language is interpreted and executed by the interpreter, is easy to master, has good rapid development efficiency, and is widely applied to the fields of computers, electronic information and the like.

Currently, the existing scripts written on the market have the following disadvantages:

executing the program: the execution of the action waits until a specific element appears on the page layout before executing the next action. In this process, scripts are written tediously, and repetitive codes frequently appear.

RPA recording action: can only execute simple tasks and is ineffective for NAF. The conventional approach is to find a specific NAF tag, with the help of a visual analysis tool, and perform the corresponding action.

By means of polling: and waiting for the completion of page loading, so that the execution action cannot be timely.

Disclosure of Invention

In order to overcome the defects of the prior art, the invention aims to provide a method and a system for compiling and executing an RPA script of a terminal device, which can solve the problems that the RPA script is complicatedly compiled, a response page is not timely, multi-step tasks are difficult to deal with and the like in the prior art, and can improve the efficiency of compiling the RPA script, the efficiency of executing the script and the accuracy of executing the script.

In order to solve the problems, the technical scheme adopted by the invention is as follows:

a method for compiling and executing terminal equipment RPA script comprises the following steps: acquiring target page information, and defining an RPA script relation on each target page respectively; after the RPA script runs, running a terminal APP on the terminal equipment by using an ADB tool and monitoring page information of the equipment in real time; after the RPA script is stable in the page, matching the page according to the priority and executing the page action, wherein the page action comprises the following steps: when the RPA script receives the latest equipment page information, a page state object is constructed; matching the target page objects according to the priority order after the page state objects are stable; and executing the target action corresponding to the matched target page object.

The further scheme is that target page information is obtained, and an RPA script relation is defined on each target page; after the RPA script runs, running a terminal APP on the terminal equipment by using an ADB tool and monitoring page information of the equipment in real time; after the RPA script is stable in the page, matching the page according to the priority and executing the page action, wherein the page action comprises the following steps: when the RPA script receives the latest equipment page information, a page state object is constructed; matching the target page objects according to the priority order after the page state objects are stable; and executing the target action corresponding to the matched target page object.

Further, the obtaining of the target page information and defining the RPA script relationship on each target page respectively includes: constructing a corresponding standard atlas of all key page NAF nodes; and defining each target page object, wherein the target page object is the interface state of the corresponding terminal equipment when the RPA script executes the action.

Further, the method for constructing all page key NAF nodes corresponds to a standard atlas, and comprises the following steps: and converting the screenshot corresponding to the NAF node on the key page into a standard graph, and giving a globally unique picture number, wherein the NAF node refers to a node with attribute NAF of true on a terminal equipment page layout tree.

Further, the parameters of the target page object at least include: the node elements are node elements on a page layout tree of the terminal equipment; the top activity matching relationship is expressed as that the top activity of the current interface state is matched with the top activity of the target page object; the target action is an action to be executed on the terminal equipment by the RPA script.

A further scheme is that after the RPA script runs, an ADB tool is used for starting the terminal APP, the terminal APP acquires information such as a page layout tree and a screenshot of the terminal equipment in real time, and the information is transmitted to the RPA script in real time through a TCP (transmission control protocol); the method comprises the steps that a terminal APP obtains a page layout tree by using a UI (user interface) Automator, and marks activated subtrees; and after finding that the screen shot is changed, the terminal APP transmits a new screen shot to the RPA script.

In a further aspect, the building a page status object includes: if the RPA script configures NAF node replacement requirements, traversing NAF nodes in the node tree, sequentially executing screenshot by each NAF node and searching matched standard graph operation in a standard graph library, and updating the value of the specific attribute to be the picture number for the NAF node matched with the standard graph; if the RPA script configures the node cutting requirement, traversing the node tree and deleting the subtrees meeting the requirement; the attributes of the page state object at least comprise a screenshot, a node tree and a top-level activity.

Further, the matching the target page object according to the priority order after the page status object is stable includes: the page state object is stabilized as the description of the terminal equipment page which is completely loaded, and when the RPA script which defines the NAF node element, the basis for judging the stability of the page state object is as follows: all key page NAF nodes match the upper standard graph.

Further, the matching the target page object according to the priority order after the page status object is stable includes: comparing the page state object with the target page object, judging whether the page state object and the target page object meet the matching relation, if so, taking the target page object as a final result, finishing the subsequent matching operation, and directly executing the target action; wherein the comparing the page state object with the target page object specifically comprises: judging whether the top activity is matched; and judging whether all the node elements are matched.

Further, the constructing the page state object also performs NAF node element replacement: after the RPA script obtains a layout tree and a screen capture of a terminal APP, the RPA script finds a NAF node in the layout tree, extracts a node capture, searches a standard graph library by using a picture matching algorithm, and adds an attribute img _ tag which is a picture number in the NAF node by using the number of the hit picture.

Therefore, compared with the prior art, the method and the device can find out the target page and respectively define the RPA scripts, then use the ADB to run the terminal APP on the android device to monitor the device page information in real time, match the page according to the priority and execute the page action after the page is stabilized by the RPA scripts, complete the algorithm execution of script self-definition, improve the script execution efficiency, use the standard picture to replace XPath grammar expression, and simplify the RPA script writing logic.

Therefore, the invention can simplify the RPA script compiling complexity and reduce the code compiling threshold; responding to the page in time; and the RPA script code multiplexing rate is improved.

A compiling and executing system of terminal equipment RPA script comprises: the RPA script definition module is used for defining a target page object; the terminal module is used for monitoring page information of the terminal equipment; and the RPA script execution module is used for matching the target page according to the priority and executing the page action.

Therefore, the invention completes the writing and the execution of the RPA script through the execution system consisting of the RPA script definition module, the terminal module and the RPA script execution module, and can solve the problems that the writing of the RPA script is complicated, the page change cannot be responded in time and the multi-step task is difficult to deal with in the prior art, thereby improving the writing efficiency and the execution efficiency of the RPA script and being beneficial to the popularization of the RPA on android equipment.

The present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.

Drawings

Fig. 1 is a flowchart of a first embodiment of a method for writing and executing an RPA script of a terminal device according to the present invention.

Fig. 2 is a table diagram of node element definitions in an embodiment of a method for writing and executing an RPA script of a terminal device according to the present invention.

Fig. 3 is a table diagram of target action definitions of a page object in an embodiment of a method for writing and executing an RPA script of a terminal device according to the present invention.

Fig. 4 is a table diagram of page object definition in an embodiment of a method for writing and executing an RPA script of a terminal device according to the present invention.

Fig. 5 is a schematic diagram of a NAF node standard graph in the second embodiment of the method for writing and executing the RPA script of the terminal device of the present invention.

Fig. 6 is a table diagram of node element definitions in a second embodiment of a method for compiling and executing an RPA script of a terminal device according to the present invention.

Fig. 7 is a table diagram of target action definitions of page objects in a second embodiment of the method for writing and executing an RPA script of a terminal device according to the present invention.

Fig. 8 is a table diagram of page object definition in a second embodiment of a method for writing and executing an RPA script of a terminal device according to the present invention.

Detailed Description

The technical solutions in the embodiments of the present application are described below clearly and completely with reference to the accompanying drawings, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.

First, a part of words and related technologies referred to in the present application will be explained with reference to the accompanying drawings so as to be easily understood by those skilled in the art.

NAF: NAF is known collectively as Not access availability friend, a node that is Not easily accessible.

The terminal equipment: the terminal devices may each include, but are not limited to, other portable devices such as smartphones, tablets, wearable devices, and the like.

RPA: the RPA changes the service into one script, and can expand the automation range which can not be achieved by the prior art by flexibly using the RPA, thereby obviously improving the service efficiency.

The embodiment of a method for compiling and executing terminal equipment RPA script includes:

as shown in fig. 1, a method for writing and executing an RPA script of a terminal device includes the following steps:

step S1, acquiring target page information, and defining an RPA script relationship on each target page, respectively, including:

step S1-1, constructing all key page NAF nodes corresponding to standard atlas;

and step S1-2, defining each target page object, wherein the target page object is the interface state of the corresponding terminal device when the RPA script executes the action.

And step S2, after the RPA script runs, using the ADB tool to run the terminal APP on the terminal equipment and monitoring the page information of the equipment in real time.

After the RPA script runs, the ADB tool can be used to start the terminal APP, the terminal APP acquires information such as a page layout tree and a screenshot of the terminal equipment in real time, and the information is transmitted to the RPA script in real time through a TCP.

The method comprises the steps that a terminal APP obtains a page layout tree by using a UI (user interface) Automator, and marks activated subtrees;

and after finding that the screen shot is changed, the terminal APP transmits a new screen shot to the RPA script.

Step S3, when the RPA script is stable, matching the page according to the priority and executing the page action, which includes:

step S3-1, when the RPA script receives the latest device page information, a page state object is constructed.

And step S3-2, matching the target page objects according to the priority order after the page state objects are stable.

And step S3-3, executing the target action corresponding to the matched target page object.

In the above step S1-1, constructing all page key NAF node correspondence standard atlas, including:

and converting the screenshot corresponding to the NAF node on the key page into a standard graph, and giving a globally unique picture number, wherein the NAF node is a node with attribute NAF of true on a page layout tree of the terminal equipment.

In step S1-2, the parameters of the target page object at least include:

and the node elements are node elements on the page layout tree of the terminal equipment.

And the top activity matching relationship is expressed as that the top activity of the current interface state is matched with the top activity of the target page object.

The target action is an action to be executed on the terminal equipment by the RPA script.

In the above step S3-1, the page status object is constructed, including: if the RPA script configures NAF node replacement requirement, traversing NAF nodes in the node tree, each NAF node sequentially executing screenshot and searching matched standard graph operation in a standard graph library, and updating the value of the specific attribute to be the picture number for the NAF node matched with the standard graph.

And if the RPA script configures the node cutting requirement, traversing the node tree and deleting the subtrees meeting the requirement.

The attributes of the page state object at least comprise a screenshot, a node tree and a top-level activity.

In the above step S3, the page status object is stabilized as a description of the terminal device page being completely loaded, and when the RPA script defining the NAF node element is determined to be stabilized according to the following: all key page NAF nodes match the upper standard graph.

In the step S3-2, the page status object and the target page object are compared to determine whether they satisfy the matching relationship, and if so, the target page object is used as the final result to end the subsequent matching operation and directly execute the target action.

Wherein the comparing the page state object with the target page object specifically comprises:

judging whether the top activity is matched;

and judging whether all the node elements are matched.

In the above step S3-1, when building the page state object, NAF node element replacement is also performed: after the RPA script obtains a layout tree and a screen capture of a terminal APP, the RPA script finds a NAF node in the layout tree, extracts a node capture, searches a standard graph library by using a picture matching algorithm, and adds an attribute img _ tag which is a picture number in the NAF node by using the number of the hit picture.

Further, in step S1-1, the NAF node is a node with an attribute NAF of "true" on the android device layout tree, and converts a screenshot corresponding to the NAF node on the key page into a standard graph, and gives a globally unique picture number, so as to improve the speed of picture matching and enhance the robustness of matching. The implementation of the method can comprise the methods of image graying processing, image scaling, image contour extraction and the like. Preferably, the NAF screenshot can be converted to a standard graph by graying and scaling to 64 x 64 pixels.

Further, in step S1-2, the target page object refers to an interface state of the corresponding android device when the RPA script performs an action, and includes an interface layout tree, a top-level activity, a screenshot, and the like.

Further, in step S1-2, the target page object can be described in the RPA script system by 3 necessary parameters:

1) a node element;

2) top level activity matching relationship;

3) and (6) target action.

Further, in step S1-2, the node element refers to a node element on the device page layout tree, such as an icon, a button, and a radio box on the android device page. Preferably, the unique node element is expressed using the xpath method. Preferably, the logic of AND AND NOT is used, wherein the former expresses that the page object must contain the node element, AND the latter expresses that the page object cannot contain the node element.

Further, in step S1-2, if the node element is a NAF attribute node and the NAF node screenshot exists in the standard gallery of step S1-1, then preferably, in the xpath expression of the node element, the selection of the NAF node may be expressed using a method of "img _ tag ═ picture number".

Further, in step S1-2, the activity matching relationship expresses that the top-level activity of the current interface state matches the top-level activity of the target page object. Preferably, the condition to be satisfied by the activity is expressed using a regular expression.

Further, in step S1-2, the target action refers to an action that the RPA script will perform on the android device. The action comprises two major types, one is simple action such as clicking, long pressing, sliding, text setting and the like, and the other is complex action defined by script codes, such as executing continuous multiple times of clicking on a target node.

Further, in step S1-2, the target page object may include other parameters in addition to the 3 necessary parameters.

Further, in step S1-2, the target page object preferably uses an inheritance relationship to express that the child page object includes the parent page object; using inheritance relationships, the definition of the target page object can be simplified.

Further, in step S2, the terminal APP acquires the page layout tree by using a dumpWindowHierarchy method of uiautoamator, and labels the activated subtree. Preferably, the device layout tree is stored using XML format; and after finding that the equipment layout tree is changed, the terminal APP transmits a new equipment layout tree to the RPA script.

Further, in step S2, it is preferable that the terminal APP transmits a new screen shot to the RPA script after finding that the screen shot is changed.

Further, in step S3, the attributes of the page status object include a screenshot, a node tree, a top level activity, and the like.

Preferably, in step S3-1, if the RPA script configures NAF node replacement requirement, traversing NAF nodes in the node tree, each NAF node sequentially executes screenshot, searches for a matched standard graph in the standard graph library, and updates the value of the specific attribute to be the picture number for the NAF node matched with the standard graph.

Preferably, in step S3-1, if the RPA script configures the node clipping requirement, the node tree is traversed and the subtree satisfying the requirement is deleted. For example, deleting a sub-tree corresponding to the non-top-level activity, deleting a blocked sub-tree, and so on.

Further, in step S3-2, the page state object is stable and refers to the description of the phenomenon that the android device page is completely loaded. Different RPA scripts are inconsistent in judgment standard, and one standard for judging whether the page state object is stable is that the layout tree is unchanged and the screen capture content is unchanged. In this embodiment, for RPA scripts that define NAF node elements, a stable criterion may be that all key NAF nodes match the upper criteria graph.

Further, in step S3-2, matching the target page object means comparing the page status object with the target page object, determining whether the two objects satisfy a matching relationship, and if so, taking the target page object as a final result, ending the subsequent matching operation, and directly executing step S3-3. The page state object is compared with the target page object, and the specific steps comprise:

1) whether the top level activity is matched;

2) whether all node elements match.

It should be noted that, the page status object may be compared with the target page object, and matching of other relationships may also be performed, which is specifically determined according to the attribute of the target page object.

Further, in step S3-3, the RPA script may directly execute the final action on the android device by calling the terminal APP, calling the ADB command, and the like.

Further, the target page object defined in step S1: the execution is performed 0 times and n times.

The RPA script can define an initialization action and define an exit action, wherein the exit action can be a target action of a target page object or a timeout action. Watchdog actions may also be defined, destroying the no response state.

In practical application, the compiling and executing method of the terminal device RPA script provided by the embodiment is explained by combining a deep app (the deep app is a marketing application for mobilizing the consciousness of the owner of the enterprise, realizing the marketing, crisis coordination and task statistics of the enterprise, and adopting three technologies to mobilize the enthusiasm of staff and effectively improve the office efficiency of the staff), and the method specifically comprises the following steps:

and obtaining login account information on the deep APP.

Finding out a target page and respectively defining RPA scripts, wherein the target page relates to three target pages: a deep-heart login page, a deep-heart account page, and other pages. The deep-heart login page refers to a deep-heart APP login interface and indicates that the account is not logged in; the deep-heart account page is a login user information page and can extract login account information; other pages, referring to pages other than the first two pages.

And constructing a standard atlas corresponding to all page key NAF nodes. Since the target page and node elements involved in the first embodiment do not contain NAF nodes, all NAF node standard galleries do not need to be constructed.

Each target page object is defined. The target page object at least comprises three parts of a node element, a top-level activity and a target action.

As shown in fig. 2, three key node elements are defined to which the target page object relates. Preferably, the node element positioning is expressed using an xpath syntax; preferably, the presence AND absence of a node element are expressed using an AND relation AND a NOT relation, respectively. The deep login page of the first embodiment includes three key node elements, namely, a mobile phone login text, a mobile phone number input, and an authentication code acquisition. The deep account page also comprises three key node elements of the personnel, the account ID and the software version. Other pages, the universal node of xpath ═// node' is used to express the range of the page.

As shown in FIG. 3, the target actions to which three target page objects relate are defined. And the deep-heart login page and the deep-heart account page respectively use a custom function as a target action. And the other pages use the built-in start _ deep _ link action type and take the deep _ link of the deep account page as a parameter to execute the action of automatically opening the deep account page.

As shown in fig. 4, three target page objects are completely defined. The page objects can use rules to express relationships between pages, thereby simplifying the definition of target objects.

In this embodiment, an inheritance relationship rule of a target page object is defined: for target page objects A and B, if B inherits from A, that is, the parent page of B is A, then: 1) if the page state is matched with the target page object B, the page state must be matched with the target page object A; 2) the matching priority of the target page object B is higher than A.

In this embodiment, the deep-heart login page and the deep-heart account page are both inherited from other pages, and only when the deep-heart login page and the deep-heart account page are not matched, other pages can be matched. In this way, the definition between three pages is simplified.

In fig. 4, three key properties of the node element, the top-level activity and the target action of the three target page objects are completely defined.

And then, running a terminal APP on the android device by using the ADB to monitor the page information of the device in real time. Wherein, through the ADB mode, start terminal APP on android device, the purpose lets terminal APP can acquire necessary permission. The terminal APP provides at least two functions: 1) acquiring a page layout tree by using a dumpWindowHierarchy method of UiAutomator; 2) a screenshot is obtained.

In this embodiment, in order to reduce the operation load of the android terminal, only when the RPA script notifies that the page layout tree and the screenshot begin to be received, the terminal APP begins to collect information such as the page layout tree and the screenshot. After judging that the page layout tree changes, the terminal APP sends a new page layout tree to the RPA script; and sending a new screenshot to the RPA script after judging that the screenshot is changed. Both the page layout tree and the screen capture need real-time performance, namely, the page layout tree and the screen capture in the current state are obtained as much as possible.

The page layout tree acquired by the terminal APP needs to mark the node tree corresponding to the top activity as an activated state.

Then, the RPA script matches the page according to the priority and executes the page action after the page is stable.

And after the RPA script receives the latest equipment page information, constructing a page state object. The page state object at least comprises three attributes: screenshot, node tree, top level activity.

After the page state object is stable, the target page object is matched according to the priority order, the matching order depends on the rules, but it can be clear that the priority of the deep login page and the deep account page is higher than that of other pages due to the adoption of the inheritance rule.

And executing the target action corresponding to the matched target page object. For example, if the android device interface is in a deep-heart account page, the deep-heart account page target page object is matched, and the RPA script executes an action corresponding to the deep-heart account page: and acquiring account information, and executing and exiting successfully.

For an embodiment corresponding to an RPA task, a conventional RPA script construction method is as follows:

1) opening a deep account page deeplink by an RPA script;

2) if the layout tree has specific node information of a deep account page after waiting for a plurality of time lengths, acquiring the account information and executing exit;

3) if the layout tree has specific node information of the deep login page, the account number is failed to be acquired, and the exit is executed;

4) and if the waiting time exceeds the specified time length and the specific information of the deep account and the deep login page does not appear, trying to open the deep account page deeplink again.

Compared with the traditional method, the RPA task based on the first embodiment has the following advantages:

1) in the RPA script compiling process, a large amount of logic for waiting for specified time is saved, whether the specific node element appears on the current page is judged, and the code amount is reduced.

2) Based on the target page object thinking, the page matching logic and the target action are easy to organize.

The embodiment of a method for compiling and executing terminal equipment RPA script comprises the following steps:

referring to fig. 5 to 8, the present embodiment executes an RPA script on a chat application, and the task of the present embodiment is to select two users specified in an address book.

As shown in fig. 5, two key NAF node elements, that is, a search icon and a multi-selection icon of a page to be selected appear on the chat application address book interface in the second embodiment. If the two NAF nodes do not exist, the NAF nodes may not be in the address book interface of the chat application at present, or the address book interface of the chat application is not completely rendered.

Since the RPA script in this embodiment has NAF node elements, according to step S1-1, it is necessary to extract the screenshots corresponding to these two NAF node elements, generate the standard picture shown in the upper part of fig. 5, and respectively assign picture numbers img _001 and img _002 to the picture.

As shown in fig. 6, when two NAF node elements, i.e., a to-be-selected page search icon and a multi-selection icon, are written, @ img _ tag ═ img _001 and @ img _ tag ═ img _002 are used in xpath, which expresses that a screenshot corresponding to the to-be-selected page-search icon node element must match an img _001 picture of a standard gallery, and a screenshot corresponding to the to-be-selected page-multi-selection icon node element must match an img _002 picture of the standard gallery. In fig. 6, two node elements, namely, the unchecked contact a and the unchecked contact B, express the unchecked meaning by using a NOT relationship.

In the embodiment, four target page objects of a page to be selected, a contact A page selection, a contact B page selection and a contact AB page selection are defined. The existence of the four target page objects can ensure that two contacts can be correctly selected to the maximum extent.

In step S3, the RPA script of the present embodiment needs to specifically define a judgment option for whether the page status object is stable. Optionally, when the top activity corresponding to the page state object is regularly matched, ". contact.controller.selectactivity" (that is, the top activity is an address book interface of the chat application) and all NAF nodes are matched by the standard gallery picture, the page state object may be considered to be stable.

In step S3-1, the RPA script performs NAF node element replacement based on the configuration in addition to conventional layout tree clipping. For example, for an address book interface of a chat application, after an RPA script acquires a layout tree and a screen capture of a terminal APP, the RPA script finds a NAF node in the layout tree, extracts a node capture, searches a standard gallery by using a picture matching algorithm, and adds an attribute img _ tag, namely a picture number, in the NAF node by using the number of a hit picture.

Based on the first embodiment and the second embodiment, it can be seen that compared with the conventional method, the method provided by the invention has the following advantages:

1) based on the target page object organization matching logic and the target function, the complexity of the RPA script is reduced, and the efficiency is improved;

2) the definition rules of the node elements and the target page objects have wide applicability;

3) the problem that NAF node elements are difficult to utilize is solved.

The embodiment of the system for compiling and executing the terminal equipment RPA script comprises the following steps:

a compiling and executing system of terminal equipment RPA script comprises:

and the RPA script definition module is used for defining the target page object.

And the terminal module is used for monitoring the page information of the terminal equipment.

And the RPA script execution module is used for matching the target page according to the priority and executing the page action.

Therefore, in the embodiment, the writing and the execution of the RPA script are completed through the execution system consisting of the RPA script definition module, the terminal module and the RPA script execution module, so that the problems that the writing of the RPA script is complicated, the page change cannot be responded in time and the multi-step task is difficult to deal with in the prior art can be solved, the writing efficiency and the execution efficiency of the RPA script are improved, and the popularization of the RPA on android equipment is facilitated.

It can be seen that the embodiments in this specification are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The method and system provided by the present application are introduced in detail, and specific examples are applied herein to explain the principle and the implementation of the present application, and the descriptions of the above embodiments are only used to help understand the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

The above embodiments are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited thereby, and any insubstantial changes and substitutions made by those skilled in the art based on the present invention are within the protection scope of the present invention.

Claims (10)

1. A method for compiling and executing terminal equipment RPA script is characterized by comprising the following steps:

acquiring target page information, and defining an RPA script relation on each target page respectively;

after the RPA script runs, running a terminal APP on the terminal equipment by using an ADB tool and monitoring page information of the equipment in real time;

after the RPA script is stable in the page, matching the page according to the priority and executing the page action, wherein the page action comprises the following steps:

when the RPA script receives the latest equipment page information, a page state object is constructed;

matching the target page objects according to the priority order after the page state objects are stable;

and executing the target action corresponding to the matched target page object.

2. The method according to claim 1, wherein the obtaining of the target page information and defining an RPA script relationship on each target page respectively comprises:

constructing a corresponding standard atlas of all key page NAF nodes;

and defining each target page object, wherein the target page object is the interface state of the corresponding terminal equipment when the RPA script executes the action.

3. The method of claim 2, wherein the constructing all page key NAF nodes corresponding to a standard atlas comprises:

and converting the screenshot corresponding to the NAF node on the key page into a standard graph, and giving a globally unique picture number, wherein the NAF node refers to a node with attribute NAF of true on a terminal equipment page layout tree.

4. The method of claim 2, wherein:

the parameters of the target page object at least comprise:

the node elements are node elements on a page layout tree of the terminal equipment;

the top activity matching relationship is expressed as that the top activity of the current interface state is matched with the top activity of the target page object;

the target action is an action to be executed on the terminal equipment by the RPA script.

5. The method of claim 1, wherein:

when the RPA script runs, starting a terminal APP by using an ADB tool, acquiring information such as a page layout tree and a screenshot of the terminal equipment in real time by the terminal APP, and transmitting the information to the RPA script in real time through a TCP (transmission control protocol);

the method comprises the steps that a terminal APP obtains a page layout tree by using a UI (user interface) Automator, and marks activated subtrees;

and after finding that the screen shot is changed, the terminal APP transmits a new screen shot to the RPA script.

6. The method of claim 1, wherein building a page state object comprises:

if the RPA script configures NAF node replacement requirements, traversing NAF nodes in the node tree, sequentially executing screenshot by each NAF node and searching matched standard graph operation in a standard graph library, and updating the value of the specific attribute to be the picture number for the NAF node matched with the standard graph;

if the RPA script configures the node cutting requirement, traversing the node tree and deleting the subtrees meeting the requirement;

the attributes of the page state object at least comprise a screenshot, a node tree and a top-level activity.

7. The method of claim 6, wherein matching the target page objects in order of priority after the page status object is stable comprises:

the page state object is stabilized as the description of the terminal equipment page which is completely loaded, and when the RPA script which defines the NAF node element, the basis for judging the stability of the page state object is as follows: all key page NAF nodes match the upper standard graph.

8. The method of claim 2, wherein matching the target page objects in order of priority after the page status object is stable comprises:

comparing the page state object with the target page object, judging whether the page state object and the target page object meet the matching relation, if so, taking the target page object as a final result, finishing the subsequent matching operation, and directly executing the target action;

wherein the comparing the page state object with the target page object specifically comprises:

judging whether the top activity is matched;

and judging whether all the node elements are matched.

9. The method of claim 6 wherein the constructing a page state object further performs NAF node element replacement:

after the RPA script obtains a layout tree and a screen capture of a terminal APP, the RPA script finds a NAF node in the layout tree, extracts a node capture, searches a standard graph library by using a picture matching algorithm, and adds an attribute img _ tag which is a picture number in the NAF node by using the number of the hit picture.

10. A compiling and executing system of terminal equipment RPA script is characterized by comprising:

the RPA script definition module is used for defining a target page object;

the terminal module is used for monitoring page information of the terminal equipment;

and the RPA script execution module is used for matching the target page according to the priority and executing the page action.

Images