KR20230165086A - Method for recording and executing for rpa task - Google Patents

Abstract

A recording method and execution method of RPA tasks are provided. A recording method of an RPA task according to an embodiment of the present disclosure is, in a recording situation of an RPA task, in response to a user input for selecting an object inside a window area of an external desktop, from an agent of the external desktop, to the user input. It may include receiving object information about the object selected by and adding action information to the specification data of the RPA task using the received object information.

Description

Method for recording and executing RPA tasks {METHOD FOR RECORDING AND EXECUTING FOR RPA TASK}

This disclosure relates to Robotics Process Automation (RPA) technology. More specifically, it is about a method of recording and executing a series of actions performed by RPA technology.

Robotic Process Automation (RPA), a technology that automates tasks repeatedly performed by users, is one of the IT (Information Technology) technologies that has recently received great attention. A series of actions performed by RPA technology may be referred to as RPA TASK. RPA tasks can run in various process instances. For example, an RPA task may be executed on an operating system (OS) installed on a server, or may be executed on the operating system of a user terminal.

Meanwhile, as the situation in which users can use various terminal devices becomes established, external desktop access technology that supports users to perform tasks by accessing the computing device they were using as an external computing device is provided. For example, remote desktop connection functionality is provided. In other words, the computing device that creates the desktop provided by the operating system and the computing device that displays the desktop screen and manipulates user input can be separated from each other. External desktop access technology has evolved into Virtual Desktop Infrastructure (VDI) technology and is widely used.

However, there are inconveniences at the point where RPA technology and external desktop access technology are combined. For example, when selecting an object inside an external desktop window in an RPA task recording situation, detailed information about the selected object cannot be known, so actions related to the object inside the external desktop window are difficult to include in the RPA task. These limits on the actions that can be included in an RPA task limit the range of repetitive tasks that can be automated by defining them as RPA tasks.

Korean Patent No. 2337270 Korean Patent No. 2227687 US Patent Publication No. 2022-0100539

The technical problem to be solved in some embodiments of the present disclosure is to provide an RPA task recording method that can accurately record actions using objects on an external desktop and a computing system to which the method is applied.

The technical problem to be solved in some embodiments of the present disclosure is to provide an RPA task execution method that can accurately execute RPA tasks, including actions using objects on an external desktop, and a computing system to which the method is applied.

The technical problem to be solved in some embodiments of the present disclosure is to accurately execute the RPA task even if an object related to an action included in the RPA task exists on an external desktop that is not yet connected to the desktop on which the RPA task is running. It provides an executable RPA task execution method and a computing system to which the method is applied.

The technical problems of the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by those skilled in the art from the description below.

In order to solve the above technical problem, the RPA task recording method according to an embodiment of the present disclosure includes, in response to a user input of selecting an object inside the window area of an external desktop in a recording situation of an RPA task, an agent of the external desktop. It includes receiving object information about the object selected by the user input from , and adding action information to the specification data of the RPA task using the received object information.

The step of receiving the object information includes, in response to detecting that a cursor enters a window area of the external desktop in a recording situation of the RPA task, sending an object extraction mode switch request to the agent; It may include at least part of the step of receiving the object information generated by the agent switched to the object extraction mode.

Additionally, the step of receiving the object information may further include receiving execution environment information of the external desktop created by the agent.

In order to solve the above technical problem, an RPA task execution method according to another embodiment of the present disclosure includes the steps of acquiring specification data of the RPA task, and each action constituting the RPA task using the obtained specification data. Identifying, determining an execution environment for the identified action, and selecting an execution device for the identified action using the determined execution environment among the computing device and an external device corresponding to an external desktop connected to the computing device. determining one of the following, and when the execution device of the identified action is determined to be an external device, transmitting data for execution of the identified action to an agent of an external desktop connected to the computing device, and creating the execution environment. It may include repeating the steps of determining, determining the execution device of the identified action as one of the computing device and an external device, and transmitting the action in the order of each identified action.

The RPA task execution method may further include receiving execution environment information from an agent of each external desktop connected to the computing device. At this time, the step of determining the execution device of the identified action as one of the computing device and an external device determines whether the identified action can be executed on the computing device using the execution environment of the identified action. and, if it is determined that the identified action can be executed on the computing device, determining that the computing device is an execution device for the identified action, and determining that the computing device cannot execute the identified action. Once determined, it may include determining the execution device of the identified action as one of the respective external desktops using the received execution environment information.

The RPA task execution method may further include receiving a user input indicating execution of the RPA task. At this time, the step of receiving the execution environment information may include receiving the execution environment information in response to receiving the user input.

The execution environment information may include execution environment type information, installed application information, resolution information, and hardware available resource information. At this time, the step of determining the execution device of the identified action as one of the external desktops using the received execution environment information includes, among the received execution environment information, the execution environment type information, installed application information, determining an external desktop having execution environment information in which both resolution information and hardware available resource information match the execution environment of the identified action as an execution device of the identified action; and, from among the received execution environment information, the execution environment information. If there is no external desktop having execution environment information in which all of the environment type information, installed application information, resolution information, and hardware available resource information match the execution environment of the identified action, the execution environment type information, installed application information, It may include determining an external desktop that satisfies only some essential information among resolution information and hardware available resource information as an execution device for the identified action.

The step of determining the execution device of the identified action as one of the external desktops using the received execution environment information includes displaying a warning message if there is no external desktop matching the received execution environment information. and may include stopping execution of the RPA task.

The step of determining the execution device of the identified action as one of the respective external desktops using the received execution environment information may be performed by the user of the computing device when an external desktop matching the received execution environment information does not exist. It may include determining, among accessible desktops registered in advance, an external desktop matching the received execution environment information as an execution device for the identified action. At this time, the step of determining an external desktop matching the received execution environment information among the accessible desktops registered in advance by the user of the computing device as an execution device of the identified action may include determining an external desktop matching the received execution environment information as an execution device of the identified action. When a desktop does not exist, a step of newly connecting to an accessible desktop registered in advance by a user of the computing device, and selecting an external desktop matching the received execution environment information from among the newly connected external desktops by the identified action. It may include a step of determining the execution device.

In order to solve the above technical problem, a computing system according to another embodiment of the present disclosure may include a communication interface connected to a user terminal, a processor, and a memory that stores a computer program loaded into the processor. You can. At this time, the computer program includes an instruction for receiving specification data of the RPA task through the communication interface, an instruction for receiving an execution command for the RPA task from the user terminal, and an instruction for receiving an execution command for the RPA task. In response, it may include instructions for executing the RPA task. At this time, the instructions for executing the RPA task include an instruction for identifying each action constituting the RPA task using the obtained specification data, an instruction for determining an execution environment for the identified action, and the determined execution. An instruction for determining the execution device of the identified action as one of the computing device and an external device corresponding to an external desktop connected to the computing device using the environment, and when the execution device of the identified action is determined to be an external device, An instruction for transmitting data for execution of the identified action to an agent of an external desktop connected to the computing device, determining the execution environment, and selecting an execution device of the identified action as either the computing device or an external device. It may include instructions for repeating the step of determining one and the step of transmitting according to the order of each identified action.

The computing system may be a system that provides a remote desktop service to the user terminal.

In order to solve the above technical problem, an RPA task execution method according to another embodiment of the present disclosure includes the steps of acquiring specification data of an RPA task, and using the obtained specification data, each action constituting the RPA task Identifying, determining an execution environment of the identified action, and using the determined execution environment, executing the identified action device to an external device corresponding to the computing device and an external desktop connected to the computing device. repeating for each identified action the steps of determining one of the devices, determining an execution environment, and determining an execution device of the identified action to be one of the computing device and an external device; If the determined execution device of the identified action is the computing device, executing the identified action, and if the determined execution device of the identified action is an external device, data for execution of the identified action transmitting to an agent corresponding to the external device, executing the identified action, and transmitting data for execution of the identified action to the agent corresponding to the external device. It may include repeating steps according to the order of actions.

The step of determining the execution device of the identified action as one of the computing device and an external device corresponding to an external desktop connected to the computing device includes: displaying a notification message when the execution device of the identified action cannot be determined; It may include stopping execution of the RPA task.

The step of determining the execution device of the identified action as one of the computing device and an external device corresponding to an external desktop connected to the computing device includes the received execution environment among the accessible desktops registered in advance by the user of the computing device. It may include determining an external desktop matching the information as an execution device for the identified action, and newly connecting to the external desktop determined as the execution device.

In order to solve the above technical problem, an RPA task execution method according to another embodiment of the present disclosure includes the steps of acquiring specification data of an RPA task, and using the obtained specification data, each action constituting the RPA task Identifying, using the execution environment of the identified action, determining a generic execution environment of the RPA task, and if the execution environment of the computing device matches the determined generic execution environment, the RPA task determining an execution device as the computing device, and if at least a part of the execution environment of the computing device does not conform to the determined comprehensive execution environment, selecting an external desktop connected to the computing device that matches the determined comprehensive execution environment. It may include determining an external desktop as an execution device of the RPA task and transmitting specification data of the RPA task to the execution device of the RPA task.

The step of determining the comprehensive execution environment of the RPA task may include determining the comprehensive execution environment by combining all execution environments of the identified actions with an 'AND condition'.

1 to 2 are configuration diagrams of an RPA system according to an embodiment of the present disclosure.
Figure 3 is a signal flow diagram of an RPA task recording method according to another embodiment of the present disclosure.
FIG. 4 is a diagram illustrating exemplary execution environment information configured by an agent in some embodiments of the present disclosure.
5 to 7 are signal flow diagrams for explaining RPA task execution methods according to different embodiments of the present disclosure.
8 is a hardware configuration diagram of a computing system described in some embodiments of the present disclosure.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the attached drawings. The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the technical idea of the present invention is not limited to the following embodiments and may be implemented in various different forms. The following examples are merely intended to complete the technical idea of the present invention and to be used in the technical field to which the present invention pertains. It is provided to fully inform those skilled in the art of the scope of the present invention, and the technical idea of the present invention is only defined by the scope of the claims. In describing the present disclosure, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description will be omitted. Hereinafter, several embodiments of the present disclosure will be described with reference to the drawings.

An RPA system according to an embodiment of the present disclosure will be described with reference to FIGS. 1 and 2.

The RPA system according to this embodiment may be composed of one or more computing systems to which RPA technology is applied for automation of repetitive tasks. Additionally, the RPA system according to this embodiment may be linked to one or more computing systems that support the operation of an external desktop.

As shown in FIG. 1, the RPA system according to this embodiment may include a user terminal 20, external devices 11 and 12, and cloud resources 200.

The user terminal 20 is a computing device used by a user to display an external desktop and manipulate the displayed external desktop. On the other hand, the external devices 11 and 12 and the cloud resource 200 are computing devices installed with an operating system (OS) that provides the external desktop. That is, the external desktop has separate execution-side computing devices and operation-side computing devices.

As described above, an 'external desktop' can be understood as a software instance that supports user operation in the form of a graphical user interface (GUI). For example, when a user using a first computing device remotely accesses a second computing device, a window of the desktop screen provided by the operating system of the second computing device is displayed on the desktop screen provided by the operating system of the first computing device. It may be displayed, and at this time, the desktop screen provided by the operating system of the second computing device may be understood as an example of the 'external desktop'.

As will be explained later, the 'external desktop' does not necessarily have to be displayed. In other words, the computing device that executes the RPA task or records the RPA task and the agent installed on the 'external desktop' are 'connected' to transmit and receive data with each other, so that at the time of RPA task execution, the computing device is assigned to the agent on the external desktop as the execution target. Any instance of software that supports user interaction in the form of a graphical user interface that satisfies those conditions, provided that it can transmit information about an action, or an agent on an external desktop at the time of RPA task recording, can transmit information about a selected object to the computing device. It can be an ‘external desktop’.

In some embodiments, the external desktop may be provided using Virtual Desktop Infrastructure (VDI) technology. At this time, the user terminal 20 can create and display a window of the external desktop on the desktop screen of the user terminal 20 by connecting to the VDI service server 100 that services provision of the external desktop. .

An operating system installed on a virtual machine (VM) provisioned on the cloud resource 200 may create the external desktop. Additionally, a container created on the operating system of the virtual machine or the operating system of the external device 11 may create the external desktop.

The cloud resource 200 may be located inside a closed network 10 where packet transmission and reception are filtered by an external network and one or more firewall systems (not shown). In other words, the cloud resource 200 may be a hardware resource that constitutes a private cloud system for enterprises.

The external device 11 may be a desktop PC or laptop PC located inside the closed network 10. The operating system installed on this external device 11 can create the external desktop. In some use cases, an employee located outside the company may be able to create the desktop screen of his or her desktop PC 11 located inside the company as an external desktop on the desktop screen of his or her laptop PC.

Additionally, the external device 12 may be a desktop PC or laptop PC located outside the closed network 10. The operating system installed on this external device 12 can create the external desktop. In some use cases, an employee located outside the company may be able to display the desktop screen of the desktop PC 12 installed at his or her home on the desktop screen of his or her laptop PC and create it as an external desktop.

Meanwhile, as shown in FIG. 2, a remote desktop window 30 may be created on the desktop screen 21 of the operating system installed on the user terminal 20 without using VDI technology. 'External desktop' described in this specification may be understood to have substantially the same meaning as the generally used 'remote desktop'. As described above, the remote desktop window 30 may be a desktop screen of an operating system installed on the external device 12, or may be a desktop screen of an operating system installed on a virtual machine (not shown) provisioned on the cloud resource 200. .

Additionally, in some embodiments, the OS desktop 21 described herein may itself also be an external desktop. For example, an external desktop window can be created once again from an external desktop created using VDI technology. In this way, it can be understood that the concept of an external desktop is a relative concept according to the parent-child relationship with the subject who created the screen of the external desktop.

Hereinafter, in this specification, computing devices that provide an external desktop will be collectively referred to as 'external devices'. For example, the external device may be a desktop PC or laptop PC inside the above-described closed network, a desktop PC or laptop PC outside the closed network, a virtual machine provisioned on a cloud resource, or a container environment created in the virtual machine. There will be.

The RPA system according to this embodiment can support recording of RPA tasks including actions using objects included in an external desktop and execution of the RPA tasks.

The RPA task is a full remote RPA task in which all actions use objects from the remote desktop, or some actions use objects from the remote desktop and the remaining actions use objects from the desktop of the operating system of the user terminal. It may be a hybrid RPA task that is used.

In other words, the RPA system according to this embodiment provides excellent scalability by supporting recording and execution of actions included in the RPA task regardless of which desktop object is manipulated. This scalability will allow users of RPA systems to automate a wider variety of repetitive tasks.

The external device of the RPA system according to this embodiment may always run an agent. For example, the agent may always run in the background when the operating system of the external device boots. To this end, agent binaries for each type of operating system can be deployed and executed in advance. Additionally, the VDI service server 100 described with reference to FIG. 1 may manage distribution and execution of the agent binary.

The operation of the RPA system according to this embodiment will be described.

First, the RPA task recording-related operations of the RPA system according to this embodiment will be described. In a recording situation of an RPA task, the user terminal 20 responds to a user input for selecting an object inside the area of the window 30 of the external desktop, from the agent of the external desktop, to the object selected by the user input. Receive object information about Additionally, the user terminal 20 adds action information to the specification data of the RPA task using the received object information. Of course, if there is a user input for selecting an object on the desktop 21 of the user terminal's operating system, the user terminal 20 generates action information including information about the selected object and stores it in the specification data of the RPA task. We will add action information.

While recording of the RPA task is in progress, the user terminal 20 adds action information using object information received from the agent of the external desktop or action information using self-generated object information to the specification data of the RPA task. can be repeated. When recording is terminated through user input to the user terminal 20, the specification data of the RPA task will also be completed.

The specification data of the RPA task may be text in a predefined format. For example, the specification data of the RPA task may have a format based on a predefined mark-up language. For example, the specification data of the RPA task may be an XML-based script, and each action defining the RPA task may be hierarchically defined as a tag of a specific level. Additionally, since each action defining an RPA task may have an execution order, the tag defining each action will be placed within the specification data of the RPA task according to its execution order.

First, operations related to RPA task execution of the RPA system according to this embodiment will be described.

The user terminal 20 may receive an execution command for a specific RPA task from the user. The user terminal 20 may obtain specification data of the specific RPA task and identify each action constituting the RPA task. Additionally, the user terminal 20 may determine an execution environment for the identified action. In some embodiments, information about the execution environment of each action may be included in the specification data in a predefined format, and the user terminal 20 executes each action using the information included in the specification data. You will be able to determine the environment.

The user terminal 20 may use the determined execution environment to determine the execution device of the identified action as one of the computing device and an external device corresponding to an external desktop connected to the computing device.

In some embodiments, the external device may be any computing device that provides an external desktop connected to the computing device, regardless of whether an external desktop window is displayed on the desktop screen of the computing device.

In some other embodiments, the external device is an external desktop connected to the computing device, and may be limited to a computing device that provides an external desktop in which a window of the external desktop is displayed on the desktop screen of the computing device.

When the execution device of the identified action is determined to be an external device, the user terminal 20 transmits data for execution of the identified action to the agent of the external desktop displayed on the computing device, and executes the identified action. If the device is determined to be the user terminal 20, the identified action can be performed.

In addition, the user terminal 20 determines an execution environment, determines an execution device of the identified action as one of the computing device and an external device, and transmits data for execution of the identified action. It can be repeated according to the order of actions.

As described above, the RPA system according to this embodiment determines the performer for each action constituting the RPA task, and if the performer is a computing device that executes the RPA task, it executes and performs the action on its own. If the subject is an external desktop, the action can be executed by sending data for execution of the action to the agent of the external desktop. That is, the RPA system according to this embodiment can process RPA tasks by dividing them into individual action units.

Meanwhile, in a specific use case, a specific action of an RPA task may not have an execution device that matches its execution environment. In other words, both the operating system of the user terminal 20 and any external desktop connected to the user terminal 20 may not satisfy the execution environment required for the specific action. In this case, the user terminal 20 may display a notification message indicating that execution of the specific action is impossible and terminate execution of the RPA task.

Meanwhile, in a situation where the execution of this specific action is impossible, among the actions included in the RPA task, even the actions preceding the specific action that cannot be executed will be executed normally. In other words, a situation may arise where only some actions of the RPA task are executed. It is desirable for RPA tasks to be executed in their entirety or not in their entirety. In other words, similar to the atomicity that transactions in the database field must have, it is desirable that the execution of RPA tasks also satisfy atomicity.

To ensure that atomicity is satisfied in the execution of an RPA task, the RPA system according to this embodiment can check in advance whether each action included in the RPA task is executable before starting execution of the RPA task. To this end, the user terminal 20 may use the execution environment of each action of the RPA task to determine the execution device of each action as one of the user terminal 20 and an external device as the prior confirmation. At this time, if there is any action whose execution device cannot be determined, the user terminal 20 may display a notification message and stop execution of the RPA task.

To ensure that atomicity is satisfied in the execution of the RPA task, the user terminal 20 of the RPA system according to this embodiment identifies each action constituting the RPA task using the specification data of the RPA task, and Using the execution environment of the identified action, the generic execution environment of the RPA task is determined, and if the execution environment of the computing device matches the determined generic execution environment, the execution device of the RPA task is determined to be the computing device. And, if at least a part of the execution environment of the computing device does not conform to the determined comprehensive execution environment, select an external desktop that matches the determined comprehensive execution environment among the external desktops of the external desktop windows displayed on the computing device for the RPA task. It may be determined as an execution device and transmit the specification data of the RPA task to the execution device of the RPA task.

Meanwhile, the RPA system according to this embodiment may expand the external desktop used to execute RPA tasks to ones that are not currently instantiated. Assume that as a first user executes an RPA task, there is a first external desktop to which the first user has access. Even if the first external desktop is not instantiated on the desktop screen of the user terminal used by the first user, the first action included in the RPA task that the first user wants to execute uses the object of the first external desktop. If so, the first action can be executed through manipulation of the first external desktop. To this end, the RPA system according to this embodiment can automatically instantiate the first external desktop on the user terminal 20. That is, the user terminal 20 may automatically create a window of the first external desktop by connecting to the first external desktop to execute the RPA task.

Next, an RPA task recording method according to another embodiment of the present disclosure will be described with reference to FIG. 3. The RPA task recording method according to this embodiment may be performed by one or more computing devices. That is, in the RPA task recording method according to this embodiment, all operations may be performed by one computing device, or some operations may be performed by another computing device. For example, some operations may be performed by a first server system, and other operations may be performed by a second server system.

Additionally, as the server system is implemented on a cloud computing node, operations performed by one server system may also be performed separately on a plurality of cloud computing nodes. Hereinafter, in describing the RPA task recording method according to this embodiment, description of the performer of some operations may be omitted. In this case, the performer may be the user terminal 20 described with reference to FIG. 1. .

Additionally, technical ideas that can be recognized with reference to the description of the above-described RPA system may also be applied to the RPA task recording method according to this embodiment, even if not separately specified in this embodiment.

This will be explained with reference to Figure 3. 'Computing device' shown in FIG. 3 refers to all types of computing devices used by users. For example, the computing device shown in FIG. 3 may be the user terminal 20 described with reference to FIG. 1 . Additionally, the 'agent installed on an external device' shown in FIG. 3 refers to software that is installed on an 'external device', which is a computing device that provides an external desktop, and runs at all times in the background. Hereinafter, ‘agent installed on an external device’ will be abbreviated as ‘agent’.

The computing device receives an RPA task recording start command from the user (S101) and waits for the user's mouse input (S102). In addition to the mouse input, various user inputs such as keyboard or touch input may be monitored.

As a result of the monitoring, the computing device detects whether the cursor enters the area of the external desktop window (S103). If the cursor enters the area of the external desktop window (S103), the computing device generates an object extraction mode switch request (S104) and transmits the object extraction mode switch request to the agent (S105). As described above, since the agent is always running on the operating system of the external device, it will be able to receive the object extraction mode switch request. To prevent unnecessary load from being applied to the agent, if an object extraction mode switch request has already been sent within a predetermined time, the computing device does not generate and transmit the object extraction mode switch request (S104, S105). won't

When the agent receives the object extraction mode switch request (S105), the agent switches its operation mode to the object extraction mode (S106). An agent in object extraction mode detects the user's manipulation of the external desktop. The agent in object extraction mode detects a user input for selecting a specific object (S107) and processes the selected object information (S108). The agent in object extraction mode can process the object information to include attribute information such as an image of the selected object and text corresponding to the selected object. Table 1 below shows exemplary object information processed in the manner described above.

In some embodiments, the agent may additionally process the execution environment information of the external desktop on which the agent is running in addition to the object information (S109). Referring to FIG. 4, the execution environment information may include execution environment type information, installed application information, resolution information, and hardware available resource information. The agent may process the execution environment information to have a predetermined format.

Execution environment type information may include operating system information. Additionally, if the external desktop is provided by an operating system installed in a virtual machine, the execution environment type information may include virtual machine type information. The execution environment type information is added to the specification data of the recorded RPA task in order to satisfy the RPA producer who recorded the RPA task that the recorded RPA task is executed using an external desktop of a specific execution environment type. It can be included as information.

If the recorded RPA task includes a first action for manipulating a specific application, the first action may be executed on a desktop on which the specific application is installed, and information on the application installed on the desktop on which the action was performed may be stored in the execution environment. By being included in the information, an appropriate execution environment in which the first action for manipulating a specific application can be executed can be secured.

In addition, the screen resolution of the external desktop in the recording situation of the RPA task and the screen resolution of the external desktop when executing the RPA task later should be as identical as possible to accurately find objects related to the action. Considering this, resolution information may also be included in the execution environment information.

Additionally, the execution environment information may also include hardware available resource information. Later, at the time of execution of the RPA task, external desktops with available resources that do not meet the minimum requirements can be excluded from the execution device of the RPA task by referring to the hardware available resource information.

Additionally, in some embodiments, the available resource status of the external desktop in the recording situation of the RPA task and the available resource status of the external desktop when executing the RPA task later may be made to be the same as possible. For example, in the recording situation of the RPA task, a second action is executed on the first external desktop, and if the CPU usage rate at that time is 30% and the memory usage rate is 50%, in the later execution situation of the RPA task, the second action The execution device that executes the action will be given a higher degree of suitability as the CPU usage rate of 30% and the memory usage rate of 50% are similar to the available resource situation.

Table 2 below shows exemplary execution environment information generated in the manner described above.

Description will be made again with reference to FIG. 3 . The agent transmits object information and execution environment information to the computing device (S110). The operations of steps S102 to S110 described above are continuously repeated until the user inputs an RPA task recording end command, thereby completing the RPA task specification data.

So far, the recording method of the RPA task according to this embodiment has been described. Only one agent is shown in Figure 3, and this embodiment has been described assuming this situation. However, in a situation where two or more external desktop windows are displayed on the desktop screen of the computing device, objects displayed in two or more external desktop windows are used. So, of course, the actions of the RPA task may be performed. In this case, the specification data of the RPA task will include information about objects on two or more external desktops.

Hereinafter, RPA task execution methods according to further embodiments of the present disclosure will be described with reference to FIGS. 5 to 7. The RPA task execution method according to this embodiment may be performed by one or more computing devices. That is, in the RPA task execution method according to this embodiment, all operations may be performed by one computing device, or some operations may be performed by another computing device. For example, some operations may be performed by a first server system, and other operations may be performed by a second server system.

Additionally, as the server system is implemented on a cloud computing node, operations performed by one server system may also be performed separately on a plurality of cloud computing nodes. Hereinafter, in describing the RPA task execution method according to this embodiment, description of the performer of some operations may be omitted. In this case, the performer may be the user terminal 20 described with reference to FIG. 1. .

Additionally, technical ideas that can be recognized with reference to the description of the above-described RPA system may also be applied to the RPA task recording method according to this embodiment, even if not separately specified in this embodiment.

This will be explained with reference to FIG. 5 . 'Computing device' shown in FIG. 5 refers to all types of computing devices used by users. For example, the computing device shown in FIG. 5 may be the user terminal 20 described with reference to FIG. 1 . Additionally, the 'agent installed on an external device' shown in FIG. 5 refers to software that is installed on an 'external device', which is a computing device that provides an external desktop, and runs at all times in the background. An 'agent installed on an external device' will be abbreviated as 'agent'.

The computing device receives an RPA task execution command from the user (S200) and analyzes specification data of the RPA task to be executed (S201). The specification data may be a script file based on the markup language of the RPA task. The computing device may be able to identify object information and execution environment information for each action constituting the RPA task through analysis of the specification data.

Next, the computing device can identify the agent of each external desktop where an external desktop window is created on the desktop screen provided by the operating system of the computing device (S202).

As described above, in some other embodiments, the computing device may identify agents for all external desktops connected to the computing device, regardless of whether an external desktop window is displayed on the desktop screen of the computing device. .

The computing device may transmit a request to provide execution environment information to the identified agent, and the agent may receive the request to provide execution environment information, process the execution environment information (S202-1), and transmit the processed execution environment information (S202-1). S202-2). Regarding the agent processing execution environment information, refer to the above-described details with reference to FIGS. 3 and 4.

Next, the computing device begins executing each action of the RPA task sequentially (S203).

In step S204, the computing device finds an execution device suitable for execution of the current action, which is the action to be executed, and determines whether the execution device exists. In the process of finding the execution device, the execution environment information of the current action will be referenced. In other words, the computing device will find an execution device that has an execution environment that matches the execution environment information of the current action written in the RPA task specification data. The execution device is the computing device, or an external device that provides an external desktop with each external desktop window created on the desktop screen provided by the operating system of the computing device, or provides an external desktop connected to the computing device. It can be any one of the external devices.

If both the computing device and the external device match the execution environment information of the current action, the computing device will be determined as the execution device. This is because executing RPA tasks on the computing device's own desktop screen has a higher probability of success than on an external desktop.

Meanwhile, if both the computing device and the external device satisfy only some of the items constituting the execution environment information, the computing device determines whether there is a computing device or an external device that satisfies all essential items among the items of the execution environment information. can be judged. The computing device may determine a computing device or an external device that satisfies all required items among the execution environment information items as the execution device.

As described above, the execution environment information may include execution environment type information, installed application information, resolution information, and hardware available resource information, of which the essential items that must be satisfied in order to become an execution device are installed application information. and resolution information. Additionally, in some embodiments, the required items may be installed application information, resolution information, and hardware available resource information.

If the execution device does not exist, the computing device will display a user notification message (S214) and end RPA task execution. In other words, both the operating system of the computing device and the external desktop connected to the computing device may not satisfy the execution environment required for the current action. In this case, the user terminal 20 may display a notification message indicating that execution of the specific action is impossible and terminate execution of the RPA task.

On the other hand, in some embodiments, when there is no external desktop matching the execution environment information, an external desktop matching the received execution environment information is provided among the accessible desktops registered in advance by the user of the computing device. An external device may be determined as the execution device of the identified action.

To this end, if there is no external desktop matching the received execution environment information, the computing device additionally instantiates an external desktop by creating a new window of an accessible desktop registered in advance by the user of the computing device. (instantiation) You will be able to do it. For this instantiation, the computing device may store login information for an accessible desktop that the user has previously registered.

Additionally, in some embodiments, if there is no external desktop matching the received execution environment information, the computing device newly connects to an accessible desktop registered in advance by the user of the computing device, and newly connects to the accessible desktop. An external device that provides an external desktop may be determined as the execution device of the identified action.

In step S205, the computing device checks whether the execution device is an external device or the computing device itself, and if the execution device is the computing device itself, it will execute the current action by itself.

If the execution device is an external device, the computing device transmits a request to switch to the automated execution mode to the agent installed in the execution device (S207). The agent will receive the request to switch to the automated execution mode and switch the execution mode to the automated execution mode (S209).

After transmitting a request to switch to the automated execution mode, the computing device transmits data for executing the current action to the agent (S208). Data for executing the current action may include data defining the action and data defining the object that is the target of the action. The agent receives data for executing the current action (S210) and executes the action as defined in the received data (S211). For example, the agent may be able to find an object that is the target of an action on the desktop screen and create a click operation for the found object. The computing device transmits data for executing the current action to the agent and then waits for the agent's response (S212). When the computing device receives the action execution result from the agent, the waiting state will be released.

The computing device repeatedly performs steps S204 to S212 while moving to the next action (S214) until the remaining actions included in the RPA task no longer exist.

The RPA task execution method according to this embodiment described so far with reference to FIG. 5 processes RPA tasks by dividing them into individual action units.

Meanwhile, in a situation where a specific action is impossible to execute, among the actions included in the RPA task, even the actions preceding the specific unexecutable action will be executed normally. In other words, a situation may arise where only some actions of the RPA task are executed. It is desirable for RPA tasks to be executed in their entirety or not in their entirety. In other words, it is desirable that the execution of RPA tasks also satisfy atomicity.

To ensure that atomicity is satisfied in the execution of an RPA task, the RPA system according to this embodiment can check in advance whether each action included in the RPA task is executable before starting execution of the RPA task. This will be described with reference to FIG. 6. For convenience of understanding, the explanation will focus on the modified operation compared to Figure 5.

When an RPA task execution command is input (S200), the computing device performs RPA task specification data analysis (S201) and agent identification (S202). Next, before starting execution of each action, the computing device determines whether an execution device suitable for executing the RPA task exists (S204). More specifically, the computing device uses the obtained specification data to identify each action constituting the RPA task, determines an execution environment for the identified action, and uses the determined execution environment to identify the determining an execution device of the identified action to be one of the computing device and an external device corresponding to an external desktop connected to the computing device, determining the execution environment, and selecting the execution device of the identified action to be one of the computing device and an external device corresponding to an external desktop connected to the computing device. The step of deciding on one of the above can be repeated for each of the identified actions.

If there is at least one action for which an execution device cannot be determined (S204), the computing device may display a notification message and stop execution of the RPA task (S214). On the other hand, in some embodiments, when there is an action for which an execution device cannot be determined, an external device that provides an external desktop matching the received execution environment information among accessible desktops registered in advance by the user of the computing device. may be determined by the execution device. To this end, if there is no external desktop matching the received execution environment information, the computing device additionally instantiates an external desktop by creating a new window of an accessible desktop registered in advance by the user of the computing device. (instantiation) may be possible. Additionally, in some embodiments, if there is no external desktop matching the received execution environment information, the computing device newly connects to an accessible desktop registered in advance by the user of the computing device, and newly connects to the accessible desktop. An external device that provides an external desktop may be determined as the execution device.

Next, the computing device starts executing each action (S203). The operation of executing each action may be described with reference to FIG. 5 .

Next, an RPA task execution method according to another embodiment of the present disclosure will be described with reference to FIG. 7. This embodiment may be understood as the computing device executing the entire RPA task itself or an external device executing it.

When an RPA task execution command is input (S300), the computing device performs RPA task specification data analysis (S301) and agent identification (S302). In the agent identification process, the agent will process execution environment information (S302-1) and transmit it to the computing device (S302-2).

Next, before starting the execution of each action, it is determined whether an execution device suitable for executing the entire RPA task exists (S304). More specifically, the computing device uses the obtained specification data to identify each action constituting the RPA task, and uses the execution environment of the identified action to determine a comprehensive execution environment of the RPA task. , if the execution environment of the computing device matches the determined comprehensive execution environment, the execution device of the RPA task may be determined to be the computing device. At this time, the computing device may determine the comprehensive execution environment by combining all execution environments of the identified actions with an 'AND condition'.

If the execution device cannot be determined (S304), the computing device may display a notification message and stop execution of the RPA task (S314). On the other hand, in some embodiments, when the execution device cannot be determined, an external device that provides an external desktop conforming to the comprehensive execution environment among accessible desktops pre-registered by the user of the computing device may be determined as the execution device. It may be possible.

To this end, if an external desktop matching the comprehensive execution environment does not exist, the computing device additionally instantiates an external desktop by creating a new window of an accessible desktop registered in advance by the user of the computing device. ) It might be possible. Additionally, in some embodiments, if there is no external desktop matching the comprehensive execution environment information, the computing device newly connects to an accessible desktop registered in advance by a user of the computing device, and newly connects to an external desktop that matches the comprehensive execution environment information. An external device that provides a desktop may be determined as the execution device.

Next, the computing device executes the RPA task. There may be cases where RPA task execution must be performed by a computing device (S305, S312), and there may be cases where RPA task execution must be performed by an external device (S306 to S311). However, in this embodiment, unlike what was explained with reference to FIGS. 5 and 6, information for action execution is not transmitted to the agent for each action, and when an external device is the execution device, the specification data of the RPA task itself may be transmitted to the external device. Accordingly, the agent receives execution data of the entire RPA task (S309) and executes the entire RPA task (S310).

8 is a hardware configuration diagram of a computing system according to some embodiments of the present disclosure. The computing system 1000 of FIG. 8 includes one or more processors 1100, a system bus 1600, a communication interface 1200, and a memory (loading) a computer program 1500 executed by the processor 1100. 1400) and a storage 1300 that stores a computer program 1500. For example, the computing system in FIG. 8 may be the user terminal 20 described with reference to FIGS. 1 and 2 .

The processor 1100 controls the overall operation of each component of the computing system 1000. The processor 1100 may perform operations on at least one application or program to execute methods/operations according to various embodiments of the present disclosure. The memory 1400 stores various data, commands and/or information. The memory 1400 may load one or more computer programs 1500 from the storage 1300 to execute methods/operations according to various embodiments of the present disclosure. Storage 1300 may non-temporarily store one or more computer programs 1500. The computer program 1500 may include one or more instructions implementing methods/operations according to various embodiments of the present disclosure. When the computer program 1500 is loaded into the memory 1400, the processor 1100 can perform methods/operations according to various embodiments of the present disclosure by executing the one or more instructions.

The computer program 1500 for executing the RPA task includes instructions for receiving specification data of the RPA (Robotic Process Automation) task through a communication interface, instructions for receiving an execution command of the RPA task from the user terminal, and execution of the RPA task. In response to receiving a command, it may include instructions for executing the RPA task. In addition, the instructions for executing the RPA task include an instruction for identifying each action constituting the RPA task using the obtained specification data, an instruction for determining an execution environment for the identified action, and an instruction for determining the determined execution environment. An instruction for determining the execution device of the identified action as one of the computing device and an external device corresponding to an external desktop connected to the computing device. If the execution device of the identified action is determined to be an external device, the identified Determining an instruction and an execution environment for transmitting data for execution of an action to an agent of an external desktop displayed on the computing device, and determining an execution device of the identified action as one of the computing device and an external device. It may include an instruction to repeat the transmitting step in the order of each identified action.

In addition, the computer program 1500 for recording the RPA task, in response to a user input for selecting an object inside the window area of the external desktop in a recording situation of the RPA task, selects an object selected by the user input from the agent of the external desktop. It may include an instruction for receiving object information about an object, and an instruction for adding action information to the specification data of the RPA task using the received object information.

In some embodiments, the computing system described with reference to FIG. 8 may be configured using one or more physical servers included in a server farm based on cloud technology, such as a virtual machine. In this case, at least some of the processor 1100, memory 1400, and storage 1300 among the components shown in FIG. 6 may be virtual hardware, and the communication interface 1200 may also be a virtual switch. It may be composed of virtualized networking elements such as switches.

Although operations are shown in the drawings in a specific order, it should not be understood that the operations must be performed in the specific order shown or sequential order or that all illustrated operations must be performed to obtain the desired results. In certain situations, multitasking and parallel processing may be advantageous. Although embodiments of the present disclosure have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention can be implemented in other specific forms without changing the technical idea or essential features. I can understand that there is. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. The scope of protection of the present invention should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be construed as being included in the scope of rights of the technical ideas defined by this disclosure.

Claims (19)

In a method performed by a computing device,
In a recording situation of an RPA task, in response to a user input for selecting an object inside a window area of an external desktop, receiving object information about the object selected by the user input from an agent of the external desktop; and
Comprising the step of adding action information to the specification data of the RPA task using the received object information,
RPA task recording method. According to claim 1,
The step of receiving the object information is,
In a recording situation of the RPA task, in response to detecting a cursor entering inside a window area of the external desktop, sending an object extraction mode switch request to the agent,
RPA task recording method. According to clause 2,
The step of receiving the object information is,
Further comprising receiving the object information generated by the agent switched to the object extraction mode,
RPA task recording method. According to claim 1,
The step of receiving the object information is,
Comprising the step of further receiving execution environment information of the external desktop created by the agent,
RPA task recording method. In a method performed by a computing device,
Obtaining specification data of the RPA task;
Identifying each action constituting the RPA task using the obtained specification data;
determining an execution environment for the identified action;
Using the determined execution environment, determining an execution device of the identified action as one of the computing device and an external device corresponding to an external desktop connected to the computing device;
When the execution device of the identified action is determined to be an external device, transmitting data for execution of the identified action to an agent of an external desktop connected to the computing device; and
Repeating the steps of determining the execution environment, determining an execution device of the identified action as one of the computing device and an external device, and transmitting the steps in the order of each identified action. ,
How to run RPA tasks. According to clause 5,
Further comprising receiving execution environment information from an agent of each external desktop connected to the computing device,
The step of determining the execution device of the identified action as one of the computing device and an external device,
determining whether the identified action can be executed on the computing device using an execution environment of the identified action;
If it is determined that the computing device can execute the identified action, determining the computing device as an execution device for the identified action; and
When it is determined that the identified action cannot be executed on the computing device, determining the execution device of the identified action as one of the respective external desktops using the received execution environment information,
How to run RPA tasks. According to clause 6,
Further comprising receiving a user input indicating execution of the RPA task,
The step of receiving the execution environment information is,
In response to receiving the user input, comprising receiving the execution environment information,
How to run RPA tasks. According to clause 6,
The execution environment information is,
Containing execution environment type information, installed application information, resolution information, and hardware available resource information,
How to run RPA tasks. According to clause 8,
The step of determining the execution device of the identified action as one of the respective external desks using the received execution environment information,
Among the received execution environment information, the execution environment type information, installed application information, resolution information, and hardware available resource information all match the execution environment of the identified action to an external desktop having execution environment information of the identified action. deciding on an execution device; and
Among the received execution environment information, if there is no external desktop having execution environment information in which the execution environment type information, installed application information, resolution information, and hardware available resource information all match the execution environment of the identified action, Comprising the step of determining an external desktop that satisfies only some essential information among the execution environment type information, installed application information, resolution information, and hardware available resource information as the execution device of the identified action,
How to run RPA tasks. According to clause 6,
The step of determining the execution device of the identified action as one of the respective external desktops using the received execution environment information,
If there is no external desktop matching the received execution environment information, displaying a warning message and stopping execution of the RPA task,
How to run RPA tasks. According to clause 6,
The step of determining the execution device of the identified action as one of the respective external desktops using the received execution environment information,
If there is no external desktop matching the received execution environment information, the external desktop matching the received execution environment information among the accessible desktops registered in advance by the user of the computing device is used as the execution device of the identified action. Including the step of deciding,
How to run RPA tasks. According to claim 11,
The step of determining, among accessible desktops pre-registered by the user of the computing device, an external desktop matching the received execution environment information as an execution device for the identified action,
If there is no external desktop matching the received execution environment information, connecting to an accessible desktop registered in advance by the user of the computing device; and
Comprising the step of determining, among the newly connected external desktops, an external desktop matching the received execution environment information as an execution device of the identified action.
How to run RPA tasks. In computing systems,
A communication interface connected to a user terminal;
processor; and
Includes a memory that stores a computer program loaded on the processor,
The computer program is,
Instructions for receiving specification data of an RPA (Robotic Process Automation) task through the communication interface;
An instruction that receives an execution command of the RPA task from the user terminal;
In response to receiving an execution command of the RPA task, it includes instructions for executing the RPA task,
The instructions for executing the RPA task are:
Instructions for identifying each action constituting the RPA task using the obtained specification data;
Instructions for determining an execution environment for the identified action;
Instructions for determining an execution device of the identified action as one of the computing device and an external device corresponding to an external desktop connected to the computing device using the determined execution environment;
When the execution device of the identified action is determined to be an external device, an instruction for transmitting data for execution of the identified action to an agent of an external desktop displayed on the computing device; and
Instructions for repeating the steps of determining the execution environment, determining an execution device of the identified action as one of the computing device and an external device, and transmitting the steps in the order of each identified action. ,
RPA task execution device. According to claim 13,
The computing system is a system that provides a remote desktop service to the user terminal,
RPA task execution device. In a method performed by a computing device,
Obtaining specification data of the RPA task;
Identifying each action constituting the RPA task using the obtained specification data;
determining an execution environment for the identified action;
Using the determined execution environment, determining an execution device of the identified action as one of the computing device and an external device corresponding to an external desktop connected to the computing device;
repeating the steps of determining the execution environment and determining an execution device of the identified action as one of the computing device and an external device for each identified action;
if the determined execution device of the identified action is the computing device, executing the identified action;
When the determined execution device of the identified action is an external device, transmitting data for execution of the identified action to an agent corresponding to the external device; and
Comprising the steps of repeating the steps of executing the identified action and transmitting data for executing the identified action to an agent corresponding to the external device in the order of each identified action,
How to run RPA tasks. According to claim 15,
The step of determining the execution device of the identified action as one of the computing device and an external device corresponding to an external desktop connected to the computing device,
If the execution device of the identified action cannot be determined, displaying a notification message and stopping execution of the RPA task,
How to run RPA tasks. According to claim 15,
The step of determining the execution device of the identified action as one of the computing device and an external device corresponding to an external desktop connected to the computing device,
determining an external desktop matching the received execution environment information among accessible desktops registered in advance by a user of the computing device as an execution device of the identified action;
Comprising the step of newly connecting to an external desktop determined as the execution device,
How to run RPA tasks. In a method performed by a computing device,
Obtaining specification data of the RPA task;
Identifying each action constituting the RPA task using the obtained specification data;
Using the execution environment of the identified action, determining a comprehensive execution environment of the RPA task;
If the execution environment of the computing device matches the determined comprehensive execution environment, determining the computing device as the execution device of the RPA task; and
If at least a part of the execution environment of the computing device does not conform to the determined comprehensive execution environment, determine an external desktop that matches the determined comprehensive execution environment among external desktops connected to the computing device as an execution device of the RPA task, Comprising the step of transmitting specification data of the RPA task to the execution device of the RPA task,
How to run RPA tasks. According to clause 18,
The step of determining the comprehensive execution environment of the RPA task is:
Comprising the step of determining the comprehensive execution environment by combining all execution environments of the identified actions with an 'AND condition',
How to run RPA tasks.

Images