CN114897296A

CN114897296A - RPA flow labeling method, execution process playback method and storage medium

Info

Publication number: CN114897296A
Application number: CN202210338838.6A
Authority: CN
Inventors: 唐琦松; 林平; 吴鑫; 靳志业
Original assignee: Shanghai I Search Software Co ltd
Current assignee: Shanghai I Search Software Co ltd
Priority date: 2022-04-01
Filing date: 2022-04-01
Publication date: 2022-08-12

Abstract

The invention discloses an RPA flow labeling method, an execution process playback method and a storage medium, wherein the labeling method comprises the following steps: and inserting a service label into the RPA flow to be labeled by utilizing the RPA flow labeling model to generate the RPA flow containing the service label. The execution process playback method comprises the following steps: acquiring a service flow chart and an RPA flow chart; acquiring an execution timestamp, a screen video and an event log of the RPA flow through the RPA flow execution process; associating the service flow chart, the RPA flow chart, the screen video and the event log; and synchronously playing the associated service flow chart, RPA flow chart, screen recording and event log according to the execution timestamp. The invention adds the service label which is convenient for service personnel to understand to the RPA flow, and simultaneously displays the execution process of the RPA flow in an intuitive and convenient mode, thereby reducing the understanding difficulty of the RPA flow and improving the positioning efficiency of the flow playback.

Description

RPA flow labeling method, execution process playback method and storage medium

Technical Field

The invention relates to an RPA flow labeling method, an execution process playback method and a storage medium, belonging to the technical field of RPA flows.

Background

The RPA is an automatic process technology, and a computer/robot can automatically and repeatedly execute the RPA after a developer writes the RPA, so that specific operations can be executed manually. Because the RPA is efficient and accurate, the RPA flow can achieve the purposes of saving manpower and improving efficiency and employee satisfaction, so the RPA is widely applied. In order to ensure that the process executed by the RPA process is consistent with the expectations of developers and business personnel, the developers and the business personnel need to check the specific execution process of the RPA after the RPA process is written, so as to ensure the accuracy of the execution process and the result.

The general method for checking the RPA execution process is to record the complete screen content when the RPA process is executed, and then view the screen content in a video playback manner. This approach is simple and intuitive, but still has some problems: 1. when the operation steps of the RPA flow are too many or very complicated, the specific execution step corresponding to the picture cannot be accurately positioned through video playback, the situation of missing execution step or multiple execution steps may exist, the check is not facilitated, when the flow has an error, some error information can appear in the flow log, but the error information cannot be quickly corresponding to the video picture, and the error situation cannot be intuitively understood; 2. technical requirements on inspectors are high, when a service person without programming capability is responsible for inspecting the RPA execution process, the service person is difficult to visually correspond the video picture with the RPA flow steps and the log thereof, so that the inspection work is difficult to be smoothly and accurately performed, and if all developers perform the inspection operation, the labor cost is high.

Disclosure of Invention

Aiming at the problems that an RPA execution process playback method in the prior art is inconvenient to locate and difficult to understand, the invention provides an RPA process labeling method, an execution process playback method and a storage medium.

In order to solve the technical problems, the invention adopts the following technical means:

in a first aspect, the present invention provides a labeling method for an RPA process, including the following steps:

acquiring an RPA process to be marked;

inserting a service label into the RPA flow to be labeled by utilizing the trained RPA flow labeling model to generate an RPA flow containing the service label;

and the RPA process labeling model adopts a neural network model based on a machine learning algorithm.

With reference to the first aspect, further, the training method of the RPA process labeling model includes:

acquiring a plurality of groups of RPA flows containing service labels marked manually to form a marked data set;

and training the RPA process labeling model by using the labeling data set to obtain the trained RPA process labeling model.

With reference to the first aspect, further, a service tag has a mapping relationship with one or more steps in the RPA process, and the service tag is inserted before the one or more steps mapped by the service tag.

With reference to the first aspect, further, the service tag includes a tag number, a tag insertion location, and service information; the service label is not executable and will not affect the execution of the RPA flow.

In a second aspect, the present invention provides a playback method for an RPA process execution process, including the following steps:

inserting a service label into the RPA process by the marking method of the first aspect;

obtaining an RPA flow chart and a service flow chart with a mapping relation according to the RPA flow and a service label thereof;

obtaining an execution timestamp, a screen video and an event log of the RPA flow through the RPA flow execution process;

according to the mapping relation among the execution timestamp, the RPA flow chart and the service flow chart, the RPA flow chart, the screen video and the event log are associated on a time axis;

and synchronously playing the associated service flow chart, RPA flow chart, screen recording and event log according to the execution timestamp.

With reference to the second aspect, further, a service flow graph is formed by using the service labels as nodes, and an RPA flow graph is formed by using each step of the RPA flow as a node; and the service flow chart and the screen video are played in a service mode page, and the RPA flow chart, the screen video and the event log are played in a development mode page.

With reference to the second aspect, further, in the synchronous playing process, in response to the play jump command, the target time after the jump is obtained, the playing pictures of the service flow chart, the RPA flow chart, the video screen and the event log are all jumped to the target time, and the contents of the service flow chart, the RPA flow chart and the event log at the target time are highlighted.

With reference to the second aspect, further, the play skip command includes a play skip command based on the play progress bar and a play skip command based on keyword retrieval, where the keywords include a service flow keyword, an RPA flow keyword, and an event log keyword.

In a third aspect, the invention proposes a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method according to the first aspect of the invention.

In a fourth aspect, the invention proposes a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method according to the second aspect of the invention.

The following advantages can be obtained by adopting the technical means:

the invention provides an RPA flow marking method, an execution process playback method and a storage medium, firstly, inserting a service label for an RPA flow through the RPA flow marking method, and generalizing one or more steps in the RPA flow into one service label, thereby reducing the flow complexity, simultaneously, the service label information is simpler and easier to understand, thereby facilitating business personnel to understand the operation process of the RPA flow and reducing the understanding difficulty of the RPA flow. Secondly, according to the RPA flow containing the service label, the screen video and the event log in the RPA flow execution process are synchronized with the RPA flow chart and the service flow chart and then played back, so that not only can the picture on the screen when the RPA flow executes each step be conveniently and visually checked, but also the event log corresponding to the screen picture, the RPA flow step and the upper summarized service step can be simultaneously checked, the positioning efficiency of flow playback is improved, and accurate analysis and positioning errors in playback are facilitated. Finally, the invention provides 2 playback modes aiming at different requirements of developers and service personnel, and in the service playback mode, the content of the RPA flow is displayed by using a simplified service flow chart, so that the service personnel can conveniently understand the RPA flow, the technical requirement for checking the RPA flow execution process is reduced, and the labor cost for checking the RPA execution process is further reduced.

In addition, the invention can also search and change the content of playback by dragging the video progress bar or the log, and quickly position the execution process of the RPA flow according to the requirements of users, thereby further simplifying the playback operation of the execution process of the RPA flow and improving the use efficiency of the RPA flow.

Drawings

FIG. 1 is a flow chart illustrating the steps of a labeling method for RPA process according to the present invention;

FIG. 2 is a flow chart illustrating the steps of a playback method of an RPA process according to the present invention;

FIG. 3 is a schematic view of playback in a business mode page according to an embodiment of the present invention;

FIG. 4 is a schematic diagram illustrating adjustment of a display size of a business flow chart in a business mode page according to an embodiment of the present invention;

fig. 5 is a schematic view of playback in the development mode page in the embodiment of the present invention.

Detailed Description

The technical scheme of the invention is further explained by combining the accompanying drawings as follows:

example 1:

the invention provides a labeling method for an RPA process, which specifically comprises the following steps as shown in FIG. 1:

step 1, establishing and training an RPA process labeling model.

In the embodiment of the invention, the RPA process labeling model adopts a neural network model based on a machine learning algorithm, and specifically can adopt a sequence labeling model LSTM + CRF. The specific steps for training the RPA process labeling model are as follows:

step 101, acquiring a plurality of groups of RPA flows containing service labels labeled manually to form a labeled data set, wherein the labeled data set comprises a training set, a testing set and a verification set.

In the embodiment of the invention, the service label can be directly inserted into the compiled RPA flow, and is specifically inserted between the steps of the RPA flow to become one piece of information in the RPA flow. Each service label may summarize one or more steps in the RPA process, that is, one service label has a mapping relationship with one or more steps in the RPA process. Each service tag is inserted before the step or steps of its mapping, so that the RPA procedure step between two service tags is associated with the service tag located at the front, and the service tag is inserted and then the corresponding RPA procedure operation step is searched backwards. In the marking training set, the insertion position of the business process is determined by business personnel according to the business content.

The service label mainly comprises a label number, a label inserting position and service information. The label number is used for distinguishing different service labels and marking the sequence of the plurality of service labels; the label inserting position refers to a specific position where the service label is inserted into the RPA flow, and the label inserting position is related to the label mapping relation, so the label inserting position can also represent the relation between the service label and the RPA flow; the service information is information in which a service tag is inserted and displayed in the RPA flow, and the service information is generally easily understood text content, such as "start", "submit order", "confirm order", "end", and the like. In addition to the service label having a specific meaning, there is a default service label, denoted by "O", which represents that this operation does not belong to any service.

In the embodiment of the invention, the specific operation steps in the RPA flow can be changed during the execution, the operation steps also comprise a timestamp, the service tag is a null operation and cannot be executed, the execution of the RPA flow cannot be influenced, and the service tag is mainly used during the playback of the execution process of the RPA flow.

And 102, converting each RPA process in the labeled data set into sequence labeled data.

The original RPA flow operation steps are expressed by json, and the RPA flow operation steps are converted into numbers through a fixed mapping, so that a sequence is formed. The simplest number mapping mode is to select a plurality of fields from the RPA process, then judge whether the fields belong to the same event, map the fields to the same number if the fields belong to the same event, and record the mapping relation between the RPA process fields and the number so as to perform RPA process restoration in the following. Assume that an RPA process includes: action = 'click', name = 'submit', parent = 'window _ xxx', xpath = '// div/p/a', three fields are selected: action, name, xpath, if their three values are consistent, then map to a number s 1. Meanwhile, the service label can also be represented by a number, and the number can directly follow the label number of the service label.

Assume that the sequence data of the RPA procedure is: s1, s2, s3, s4, s5, s4, s6, and the sequence annotation data of the RPA flow after the service tag is manually annotated is: < t1> s1, s2, < t2>, s3, s4, s5, < t3>, s4, s6, which can be expressed as: < s1, t1>, < s2, t1>, < s3, t2>, < s4, t2>, < s5, t2>, < s4, t3>, < s6, t3>, wherein < t1>, < t2>, < t3> are the numbers of the service tags in the sequence annotation data, service tag < t1> summarizes operations s1 and s2, service tag < t2> summarizes operations s3, s4 and s5, and service tag < t3> summarizes operations s4 and s 6.

And 103, training the RPA process labeling model by using the sequence labeling data converted in the step 102, and continuously updating model parameters until the trained RPA process labeling model is obtained. The input of the RPA flow annotation model is sequence data of the RPA flow, for example, s1, s3, s4, s5, and the output of the RPA flow annotation model is a sequence obtained by adding a label to each operation, for example, < s1, t1>, < s3, t2>, < s4, t2>, < s5, t2>, and the model prediction result is compared with the sequence annotation data of the RPA flow after the service label is manually labeled, so that the model can be automatically learned.

And 2, acquiring the RPA flow to be labeled, wherein the RPA flow to be labeled only has an operation step written by a developer and does not have a service label.

And 3, inserting a service label into the RPA flow to be labeled by using the trained RPA flow labeling model to generate the RPA flow containing the service label. Specifically, the RPA process to be labeled is converted into sequence data, meanwhile, the mapping relation between the serial number in the sequence data and the RPA process operation step is recorded, then the sequence data is input into an RPA process labeling model for labeling, the sequence labeling data output by the RPA process labeling model is obtained, and finally, the sequence labeling data is converted into the RPA process with the service label according to the mapping relation between the serial number in the sequence labeling data and the RP process operation step or the service label.

Example 2:

the invention provides a playback method of an RPA flow execution process, which specifically comprises the following steps as shown in FIG. 2:

step 1, an RPA process is obtained, and a service tag is inserted into the RPA process by the tagging method in embodiment 1. And installing an RPA flow execution program, a screen recorder, a log recorder and the like.

And 2, obtaining an RPA flow chart and a service flow chart with a mapping relation according to the RPA flow and the service label thereof.

And forming a business flow chart by using the business label as a node, specifically, using the business information in the business label as the step content in the business flow chart. Each step of the RPA flow is used as a node to form an RPA flow chart, and the step content in the RPA flow chart is the content edited by a programmer. Since the service label is a summary of one or more steps in the RPA flow, and the service information belongs to easily understood text content, the steps of the service flow chart are fewer than those of the RPA flow chart, and are better understood.

And step 3, obtaining the execution timestamp, the screen video and the event log of the RPA flow through the RPA flow execution process.

Calling an RPA flow executive program, running the RPA flow, and simultaneously starting a screen recorder and a log recorder, wherein when the RPA flow executive program runs, the starting time and the ending time of each step in the RPA flow are automatically recorded, RPA flow executive data are generated, the RPA flow executive data can be expressed as < timestamp, step, rect >, wherein the timestamp is a timestamp, the step is an ID of a corresponding step in the flow, the rect is a box where the corresponding step is located in a flow chart, and an executive timestamp corresponding to the RPA flow step is obtained from the RPA flow executive data; the screen content is automatically recorded by using a screen recorder, and the screen recorder and an RPA process executive program synchronously start to run, so that a screen video corresponding to an execution timestamp can be generated, and the data of the screen video can be expressed as < timestamp, frame >, wherein the frame is a video frame; all events occurring in the process of executing the RPA flow are recorded by a log recorder, each event contains a timestamp, and event log data can be expressed as < timestamp, line >, wherein line is an event line, each event can have multiple lines, and each line is provided with the timestamp.

And 4, associating the service flow chart, the RPA flow chart, the screen video and the event log on a time axis according to the execution timestamp and the mapping relation between the RPA flow chart and the service flow chart.

Because the service label and the RPA flow have a mapping relation and the execution timestamp records the starting time and the ending time of each step, the corresponding starting time and the ending time of each service label can be determined according to one or more steps mapped by each service label, and the time also exists in the screen recording and the event log, so that the invention can associate all information by executing the timestamp,

and step 5, loading the player, and synchronously playing the associated service flow chart, RPA flow chart, screen recording and event log according to the execution timestamp.

The player page is as shown in fig. 3-5, the service flowchart and the screen recording are played in the service mode page, and the RPA flowchart, the screen recording and the event log are played in the development mode page. Under the default condition, the left column of the player page is general information of the current RPA process, including information of an operator, operation start and end time, step number and the like; the main window of the player page comprises two parts under the service mode page, wherein the middle part is a screen video, and the lower right corner is an animation of a service flow chart; the main window of the player page comprises three parts under a development mode page, wherein the event log corresponding to the current operation is arranged in the middle, the screen video is arranged at the upper right corner, and the animation of the RPA flow chart is arranged at the lower right corner; the bottommost part of the player page is a play controller which can control the play progress of log highlight display, screen video and flow chart animation at the same time. In the invention, a user can switch the playback mode at will according to requirements, and can adjust the position and the size of each window in the page according to the requirements.

In the synchronous playing process, the highlight display of the flow chart animation and the event log is promoted at the same time, and in addition, in order to facilitate a user to check the RPA flow execution process, the user can jump to the playing progress through a playing jump command. And responding to the play jump command, acquiring the jumped target time, jumping the playing pictures of the service flow chart, the RPA flow chart, the screen video and the event log to the target time, and highlighting the contents of the service flow chart, the RPA flow chart and the event log at the target time. The play skip command comprises a play skip command based on a play progress bar (namely, a user directly drags the progress bar to skip) and a play skip command based on keyword retrieval (namely, the user inputs keywords in a page to perform retrieval and then skips according to a retrieval structure), wherein the keywords comprise contents such as service process keywords, RPA process keywords, event log keywords and the like.

Example 3:

the present invention also proposes a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of the RPA flow labeling method in embodiment 1.

Example 4:

the present invention also proposes a computer-readable storage medium on which a computer program is stored, which when executed by a processor, implements the steps of the RPA procedure execution process playback method in embodiment 2.

Compared with the prior art, the marking method can insert the service label into the RPA process, thereby reducing the complexity of the process, simultaneously, the service label information is simpler and easier to understand, facilitating the service personnel to understand the operation process of the RPA process, and reducing the difficulty of understanding the RPA process. The playback method of the invention adds the RPA flow chart, the service flow chart and the event log on the basis of recording and playing back the screen video of the RPA execution process, synchronizes the data, facilitates the understanding of the specific steps and log information corresponding to the video picture in the playback execution process, improves the positioning efficiency of the flow steps, facilitates the analysis of problems and positioning errors, and simultaneously reduces the technical requirements for checking the RPA flow execution process through the service flow chart, thereby reducing the labor cost for checking the RPA execution process. In addition, the invention can also search and change the content of playback by dragging the video progress bar or the log, and quickly position the execution process of the RPA flow according to the requirements of users, thereby further simplifying the playback operation of the execution process of the RPA flow and improving the use efficiency of the RPA flow.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.

Claims

1. A labeling method for RPA process is characterized by comprising the following steps:

acquiring an RPA process to be marked;

2. The labeling method for the RPA process according to claim 1, wherein the training method for the RPA process labeling model comprises:

3. The method of claim 1, wherein a service tag is mapped to one or more steps in the RPA process, and the service tag is inserted before the step or steps mapped.

4. The labeling method for the RPA process according to claim 1 or 3, wherein the service label includes a label number, a label insertion position, and service information; the service label is not executable and will not affect the execution of the RPA flow.

5. A playback method of an RPA flow execution process is characterized by comprising the following steps:

inserting a service label into an RPA process by the labeling method of claims 1-4;

6. The RPA flow execution process playback method according to claim 5, wherein the service flow graph is composed using the service label as a node, and the RPA flow graph is composed using each step of the RPA flow as a node; and the service flow chart and the screen video are played in a service mode page, and the RPA flow chart, the screen video and the event log are played in a development mode page.

7. The RPA process execution process playback method of claim 5, wherein in the synchronous play process, in response to a play jump command, a target time after jumping is obtained, the play pictures of the service flow chart, the RPA flow chart, the video screen and the event log are all jumped to the target time, and the contents of the service flow chart, the RPA flow chart and the event log at the target time are highlighted.

8. The RPA flow execution process playback method of claim 7, wherein the play jump command comprises a play jump command based on a play progress bar and a play jump command based on keyword retrieval, wherein the keywords comprise a service flow keyword, an RPA flow keyword, and an event log keyword.

9. Computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 4.

10. Computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 5 to 8.