KR102299657B1 - Key Input Virtualization System for Robot Process Automation - Google Patents

Abstract

A key input virtualization system for robot processor automation according to an aspect of the present invention that can virtualize key input of string data for application of robot processor automation in a program requiring actual keyboard input is a robot process automation (Robot Process Automation) Script execution unit that executes the generated automation script (Script); a virtual key input signal generator for generating a virtual key input signal based on a key code on a real keyboard mapped to a key of each character constituting the character string data included in the automation script; and a keyboard driving driver generating an actual key input message for input of the character string data based on the virtual key input signal.

Description

Key Input Virtualization System for Robot Process Automation

The present invention relates to robotic process automation, and more particularly, to data input for robotic process automation.

With the development of information and communication technology, the number of tasks performed using a computer (hereinafter referred to as "computing task") has increased significantly, and the number of tasks that are repeatedly processed in a certain pattern using a computer has also increased.

Accordingly, a robot process automation (RPA, hereinafter referred to as 'RPA') solution that can automatically process routinely performed standardized tasks without user involvement has been proposed.

RPA is a solution that records a user's work pattern, stores it as an automation automation script (Script), and plays it back to replace a human's work. , refers to software that re-executes repetitive or standardized tasks as automated automation scripts.

In a typical RPA solution, keyboard virtualization that virtually replaces the user's keystrokes for task automation is very important. To this end, the RPA solution uses a keyboard virtualization method through an application programming interface (API, hereinafter referred to as 'API') provided by the operating system to automate the user's key input.

However, the general keyboard virtualization method using the API provided by the operating system cannot be applied to programs that require actual keyboard input to prevent illegal use, such as banking programs, ERP, business systems, etc. There is a problem with being limited.

The present invention is to solve the above problems, and to provide a key input virtualization system for robot processor automation capable of virtualizing key input of character string data for robot processor automation application in a program requiring actual keyboard input. make it a technical task.

In addition, the present invention provides a key input virtualization system for robot processor automation that can virtualize the string data included in the automation script into a key input signal in the same form as the actual keyboard input without modification of the automation script for robot process automation. other technical tasks.

A key input virtualization system for robot processor automation according to an aspect of the present invention for achieving the above object includes: a script execution unit that executes an automation script generated for robot process automation; a virtual key input signal generator for generating a virtual key input signal based on a key code mapped to a character key of each character on a real keyboard for each character constituting the character string data included in the automation script; and a keyboard driving driver generating an actual key input message for input of the character string data based on the virtual key input signal.

A key input virtualization system for robot processor automation according to another aspect of the present invention for achieving the above object sets a task composed of at least one event and a process composed of at least one task, and performs the task and the process At least one script generator for generating an automation script for; at least one script execution unit for executing the automation script by executing at least one event constituting the task; at least one key input virtualization unit for generating a virtual key input signal from string data to be input to an object activated according to the execution of the automation script, and generating an actual key input message based on the generated virtual key input signal; and at least one control unit interworking with the at least one script execution unit to control execution of the automated script.

As described above, according to the present invention, since character string data included in an automation script for robot process automation can be virtualized into a signal of the same type as a key input signal by an actual keyboard input, even in a program requiring an actual keyboard input As robot process automation can be applied, it has the effect of expanding the scope of application of robot process automation.

In addition, according to the present invention, without modification of the automation script for robot process automation, the string data included in the automation script can be virtualized into a signal of the same type as the key input signal by the actual keyboard input, so that the user's convenience is increased. It works.

1 is a block diagram showing the configuration of a key input virtualization system for robot processor automation according to an embodiment of the present invention.
FIG. 2 is a block diagram showing the configuration of the key input virtualization unit shown in FIG. 1 .
3 is a diagram showing a data structure of a virtual key input signal.
4 is a diagram illustrating a virtual key input signal for inputting a lowercase letter “a”.
5 is a diagram illustrating a virtual key input signal for input of a capital letter “S”.
6 is a diagram illustrating an example in which virtual key input signals for character string data “start” are stored in a queue.
7 is a block diagram showing the configuration of a key input virtualization system for robot processor automation according to an embodiment of the present invention.

Like reference numerals refer to substantially identical elements throughout. In the following description, detailed descriptions of configurations and functions known in the art and cases not related to the core configuration of the present invention may be omitted. The meaning of the terms described herein should be understood as follows.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in a variety of different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the technical field to which the present invention belongs It is provided to fully inform the possessor of the scope of the invention, and the present invention is only defined by the scope of the claims.

The shapes, sizes, proportions, angles, numbers, etc. disclosed in the drawings for explaining the embodiments of the present invention are illustrative and the present invention is not limited to the illustrated matters. Like reference numerals refer to like elements throughout. In addition, in describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

When 'including', 'having', 'consisting', etc. mentioned in this specification are used, other parts may be added unless 'only' is used. When a component is expressed in the singular, the case in which the plural is included is included unless otherwise explicitly stated.

In interpreting the components, it is interpreted as including an error range even if there is no separate explicit description.

In the case of a description of the positional relationship, for example, when the positional relationship of two parts is described as 'on', 'on', 'on', 'beside', etc., 'right' Alternatively, one or more other parts may be positioned between the two parts unless 'directly' is used.

In the case of a description of a temporal relationship, for example, 'immediately' or 'directly' when a temporal relationship is described with 'after', 'following', 'after', 'before', etc. It may include cases that are not continuous unless this is used.

Although the first, second, etc. are used to describe various components, these components are not limited by these terms. These terms are only used to distinguish one component from another. Accordingly, the first component mentioned below may be the second component within the spirit of the present invention.

The term “at least one” should be understood to include all possible combinations from one or more related items. For example, the meaning of “at least one of the first, second, and third items” means each of the first, second, or third items as well as two of the first, second and third items. It may mean a combination of all items that can be presented from more than one.

Each feature of the various embodiments of the present invention may be partially or wholly combined or combined with each other, technically various interlocking and driving are possible, and each of the embodiments may be implemented independently of each other or may be implemented together in a related relationship. may be

Hereinafter, an embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically showing the configuration of a key input virtualization system for robot processor automation according to an embodiment of the present invention. As shown in FIG. 1 , the key input virtualization system 100 for robot processor automation includes a script generation unit 110 , a script execution unit 120 , a key input virtualization unit 130 , and an object activation unit 140 . includes

The script generation unit 110 generates an automation script (Script) for robot process automation. An automation script means a document written to enable a robot to automatically perform repetitive or standardized tasks performed by a user. By executing the automation script by the robot, the task to be performed by the user can be reproduced in the same way as the user does. In one embodiment, the script generating unit 110 provides a tool (eg, an automated script editor) for generating a script to a user to generate an automated script, and the user uses the script editor to generate a processor (Process), a task (Task). ), and by performing debugging after setting at least one of the events in detail, it is possible to generate an automation script.

In one embodiment, an event set through an automation script means that a user performs an arbitrary action on a specific object or an operating system (OS) is generated regardless of the user's will. As a meaning of a trigger event, it may include window object detection, web object detection, UI (User Interface) object detection, image detection, OCR (Optical Character Reader) detection, key input, or mouse input.

In addition, a task set through an automation script means a unit execution action of a user composed of one or more events, and a process means a unit computing task of a user composed of one or more tasks.

To this end, the script generating unit 110 may include a process setting unit 112 , a task setting unit 114 , and a script generating unit 116 as shown in FIG. 1 .

The process setting unit 112 provides a function for setting and editing a process to the user. For example, the process setting unit 112 provides a process editor capable of setting and editing a process to the user, and the user creates or selects one or more tasks through the process editor, and sets properties, execution conditions, execution order, etc. Decide to set up the process. To this end, the process editor may include a task creation manager that manages creation of one or more tasks, a task property manager that manages properties of one or more tasks, a task connection manager that manages interconnection of one or more tasks, and the like.

According to this embodiment, the user creates one or more tasks through the process editor provided by the process setting unit 112 and sets their properties, execution conditions, execution order, and the like. In one embodiment, the attribute of the task may be at least one of the name of the task, input data, input method, output data, output method, processing time, and delay time, and the execution condition is determined when a preset preceding task is terminated. At least one of execution, execution when preset input data is received, and execution at a preset execution time may be included. In the execution order, each task is continuously input and output, and a plurality of tasks are interconnected to create one process.

The task setting unit 114 provides a function for setting and editing a task to the user. For example, the task setting unit 114 provides a task editor capable of setting and editing a task to the user, and the user creates or selects one or more events through the task editor and sets properties, execution conditions, execution order, etc. Decide and set the task. To this end, the task editor may include an event generation manager managing generation of one or more events, an event property manager managing properties of one or more events, an event connection manager managing interconnection of one or more events, and the like.

According to this embodiment, the user creates one or more events through the task editor provided by the task setting unit 114 and sets their properties, execution conditions, execution order, and the like. In one embodiment, the properties of the event include the name of the event, a target (eg, text, image, object, file), an execution method (eg, object detection, key input, mouse input), and execution result (eg, simple execution, text storage, image storage, file generation), and at least one of an execution time, and the execution condition is executed when a preset precedent event ends, execution after a predetermined time has elapsed when a preset precedent event ends, and a preset execution time It may include at least one of the executions at execution time. In the execution order, each event becomes an input and output relationship in succession, so that a plurality of events are interconnected, so that one task can be created.

In one embodiment, the above-described process setting unit 112 and task setting unit 114 records the behavior pattern of the user executing the process and task, process recording material and task recording material (for reference, computing environment information reflected) and extract process and task attributes, execution conditions, execution order, etc. based on process/task recording data.

The script production unit 116 produces an automated script so that the processes and tasks set by the user through the process setting unit 112 and the task setting unit 114 can be actually automatically executed. For example, the script production unit 116 produces a process automation script for performing the process set by the process setting unit 112 , and similarly produces a task automation script for performing the task set by the task setting unit 114 . In addition, the script production unit 116 interworks with the script execution unit 120 to verify that one or more events constituting the task can operate normally in the actual execution order, and debugs, and similarly, one or more tasks constituting the process. Debugging is verified by verifying that the program can operate normally in the actual execution order.

The script production unit 116 transmits the generated process automation script or task automation script to the script execution unit 120 for actual execution, and may store the process automation script or task automation script produced for later reuse and management. have.

As described above, the script generating unit 110 transmits the generated automated script to the script executing unit 120 so that the automated script can be automatically executed by the script executing unit 120 .

The script execution unit 120 automatically processes the computing task of the user by actually executing the automated script generated by the script generation unit 110 . Specifically, the script execution unit 120 receives the process automation script or task automation script from the script generation unit 110, interprets the received process automation script or task automation script, and attributes for each event, task, and process. , execution conditions, priorities, etc. are determined and the execution order is scheduled. In addition, the script execution unit 120 executes the task by sequentially executing each event according to the execution order, and also executes the process by executing each task according to the execution order.

The key input virtualization unit 130 generates a virtual key input signal from the string data to input the string data included in the automation script executed by the script execution unit 120 to the object activated according to the execution of the automation script, and , generates an actual key input message based on the generated virtual key input signal.

The reason that the key input virtualization system 100 for robot processor automation according to the present invention includes the key input virtualization unit 130 is that a virtual keyboard that virtually performs a user's key input is required for robot process automation. In order to automate tasks such as banking or ERP, it is necessary to detect the actual keyboard input, but the existing virtual keyboard cannot generate the same signal as the actual keyboard input, so robot process automation cannot be applied to tasks such as banking or ERP. because it can't Therefore, the key input virtualization system 100 for robot processor automation according to the present invention includes the key input virtualization unit 130 to generate the same signal as the actual keyboard input without the actual keyboard.

Hereinafter, the configuration of the key input virtualization unit 130 according to the present invention will be described in more detail with reference to FIG. 2 .

2 is a block diagram showing the configuration of a key input virtualization unit according to an embodiment of the present invention. As shown in FIG. 2 , the key input virtualization unit 130 according to an embodiment of the present invention includes a character string data extraction unit 210 , a virtual key input signal generation unit 220 , a queue 230 , and a key input unit ( 240 ), and a keyboard drive driver 250 .

The string data extraction unit 210 extracts string data from the automated script executed by the script execution unit 120 . The string data extraction unit 210 transmits the extracted string data to the virtual key input signal generation unit 220 .

The virtual key input signal generating unit 220 generates a virtual key input signal for each character constituting the character string data extracted by the character string data extracting unit 210 . Specifically, the virtual key input signal generating unit 220 generates a virtual key input signal by using a key code mapped to the corresponding key on the real keyboard for each key corresponding to each character of the character string data.

In this case, the character string data may be expressed as an input sequence of keys corresponding to each character and a DOWN or UP state of each key, and a key input corresponding to one character is a DOWN of the corresponding key ) and the press release (UP) process is completed when sequentially occurring. Accordingly, string data composed of a plurality of characters may be input by sequentially performing processes of a depressed state (DOWN) and a depressed state (UP) of a corresponding key for each key corresponding to each character. Accordingly, the virtual key input signal generating unit 220 according to the present invention may convert the string data into a virtual key input signal by using the key code of the key corresponding to each character and the state information of the corresponding key.

To this end, the virtual key input signal generating unit 220 divides the character string data into units of each character. For example, when the text string data extracted from the automation script is "start", the virtual key input signal generating unit 220 converts the string data into "s", "t", "a", "r", "t" Split by character like ".

Thereafter, the virtual key input signal generating unit 220 generates a first field in which a key code mapped to a key corresponding to the corresponding character on the real keyboard for each divided character is recorded, and the DOWN or UP of the corresponding key. ) generates a virtual key input signal composed of a second field in which information indicating the state is recorded. Here, the down information of the corresponding key indicates that the corresponding key is pressed by the user, and the up information of the corresponding key indicates that the pressing of the corresponding key is released. is recorded, and when the corresponding key is in an up state, a value of “0” may be recorded.

In an embodiment, the virtual key input signal generated by the virtual key input signal generating unit 220 may further include a third field in which delay time information between character input is recorded. In the present invention, the reason that the virtual key input signal additionally includes a third field in which delay time information between character inputs is recorded is that, when there is no delay time between each character input, adjacent characters may be input too quickly. , This is because, in this case, character input may not be processed normally in a specific program.

An example of a virtual key input signal composed of first to third fields is shown in FIG. 3 . As shown in FIG. 3 , the key code of the key corresponding to each character is recorded in the first field, status information on the down or up of the corresponding key is recorded in the second field, and the character input interval is recorded in the third field. It can be seen that delay time information is recorded.

For example, when the character to be input is “a”, the virtual key input signal generating unit 220 records “0x04”, which is the key code value of “a”, in the first field as shown in FIG. 4 , In field 2, information indicating the “DOWN” status of the corresponding key, “1” is recorded, and in field 3, the first virtual key input in which “20” corresponding to “20ms” is recorded as delay time information between character input. Generates a signal 410. The first virtual key input signal 410 shown in Fig. 4 indicates that the next virtual key input signal should be input 20 ms after the key corresponding to the letter "a" is pressed In addition, as shown in FIG. 4 , in the virtual key input signal generating unit 220 , “0x04”, which is the key code value of “a”, is recorded in the first field, and the “UP” state of the corresponding key is recorded in the second field. A second virtual key input signal 420 is generated in which “0”, which is information representing The illustrated second virtual key input signal 420 indicates that the next virtual key input signal should be input 20 ms after the key corresponding to the letter “a” is released.

As described above, it can be seen that the virtual key input signal generating unit 220 generates two virtual key input signals for inputting one character.

In the above-described embodiment, a case has been described in which the input of a function key (or a special key, hereinafter referred to as a 'function key') is not included in the character string data extracted by the character string data extraction unit 210 . However, when a function key input is included in the character string data, such as a capital letter input, the virtual key input signal generating unit 220 additionally generates a virtual key input signal for the function key in addition to the character key. Hereinafter, a method for generating a virtual key input signal by the virtual key input signal generating unit 220 when a function key input is included in the character string data will be described.

First, the virtual key input signal generating unit 220 divides character string data into character units, and determines whether input of a function key is required to input each character. For example, the virtual key input signal generating unit 220 may determine that input of the function key is required when the divided characters include uppercase letters.

For example, when the extracted string data is "Start", since the letter "S" is a capital letter, the "Shift" key as the function key and the "s" key as the letter key must be simultaneously selected in order to input the capital letter "S". Accordingly, when it is determined that the function key input is required for input of the corresponding character, the virtual key input signal generating unit 220 generates a virtual key input signal for the function key using a key code mapped to the function key on the actual keyboard. After creation, a virtual key input signal for a key corresponding to the corresponding character is generated.

Specifically, as shown in FIG. 5 , when a capital letter “S” is required to be input, the virtual key input signal generating unit 220 generates key code values “0xE1” and “Shift” for the function key “Shift”. A third virtual key input signal 510 is generated in which "1", which is information indicating the "DOWN" state of the key, and "20" corresponding to "20 ms" are recorded as delay time information between character input. Thereafter, the virtual key input signal generating unit 220 generates '0x16' which is a key code value for the character key "s", "1" which is information indicating the "DOWN" state of "s", and delay time information between character input. The fourth virtual key input signal 520 in which "20" corresponding to "20ms" is recorded, '0x16', which is the key code value for the character key "s", information indicating the "UP" state of "s" A fifth virtual key input signal 530 in which "0" and "20" corresponding to "20 ms" are recorded as delay time information between character input is generated. Finally, the virtual key input signal generating unit 220 generates “0xE1” which is a key code value for “Shift” which is a function key, “0” which is information indicating the “UP” state of the “Shift” key, and a delay between character input. By generating the sixth virtual key input signal 540 in which "20" corresponding to "20 ms" is recorded as time information, the generation of the virtual key input signal for the letter "S" is completed.

The virtual key input signal generating unit 220 stores the virtual key input signal generated for each character constituting the character string data in the queue 230 .

The virtual key input signal generated by the virtual key input signal generating unit 220 is sequentially inserted into the queue 230 according to the input order of each character constituting the character string data. As an example, when the string data extracted from the automation script is “start”, as shown in FIG. 6 , in the queue 230 , two virtual key input signals 600 and 610 for “s”, 't' Two virtual key input signals 620 and 630 for "a", two virtual key input signals 640 and 650 for "r", two virtual key input signals 660 and 670 for "r", and " It can be seen that the two virtual key input signals 680 and 690 for t" are sequentially inserted in the order of occurrence of characters.

The key input unit 240 sequentially retrieves virtual key input signals from the queue 230 and inputs them to the keyboard driving driver 250 . That is, the key input unit 240 inputs the virtual key input signal stored in the queue 230 to the keyboard driving driver 250, so that the keyboard driving driver 250 determines that the key input signal is input from the real keyboard. make it possible

As described above, according to the present invention, as the key input unit 240 inputs the virtual key input signal stored in the queue 230 to the keyboard driving driver 250, the keyboard driving driver 250 causes the corresponding string data to use the actual keyboard. to generate a key input message of the same type as when it was input.

When the retrieval of all virtual key input signals stored in the queue 230 is completed, the key input unit 240 may determine that the input of the corresponding character string data is completed.

The keyboard driving driver 250 generates an actual key input message for inputting character string data based on the virtual key input signal input by the key input unit 240 . As described above, since the keyboard driving driver 250 determines that the virtual key input signal input from the key input unit 240 is the same as that input by the user using the real keyboard, the keyboard driving driver 250 operates the virtual When a key input signal of , an actual key input message corresponding to the corresponding key input signal is generated based on this.

The keyboard driving driver 250 transmits the generated actual key input message to the operating system (OS) so that the operating system can input the actual key input message into the object activated according to the execution of the automation script.

As described above, according to the present invention, unlike the virtual keyboard implemented based on an application programming interface (API) supported by the existing operating system, by using the keyboard driving driver 250 that supports the driving of the actual keyboard, It is possible to generate a key input message generated when using the real keyboard from the virtual key input signal, so that robot processor automation can be applied to programs that require real keyboard input.

In addition, in the present invention, since the character string data included in the automation script is automatically converted into a virtual key input signal for transmission to the keyboard driving driver 250 while using the keyboard driving driver 250, a separate automation script modification is required. This has the effect of increasing user convenience.

Referring back to FIG. 1 , the object activator 140 detects a corresponding object on the computer to be performed on the basis of the information on the object included in the automated script executed by the script execution unit 120, and the detected Activate the object on the computer. As the actual key input message generated by the above-described key input virtualization unit 130 is input into the object activated by the object activation unit 140, the execution of the corresponding automation script can be completed.

To this end, the object activation unit 140 may include an object information acquisition unit 142 and an object detection unit 144 as shown in FIG. 1 .

The object information acquisition unit 142 acquires information on an object on which a keyboard input is to be performed among objects included in the automated script executed by the script execution unit 120 . In an embodiment, the object information obtaining unit 142 may obtain the two-dimensional coordinate information of the object as information on the object on which the keyboard input is to be performed. The object information acquisition unit 142 transmits the acquired two-dimensional coordinate information to the object detection unit 144 .

The object detection unit 144 detects an object to which a keyboard input is to be performed on the computer based on the two-dimensional coordinate information of the object provided from the object information acquisition unit 142 . When an object mapped to two-dimensional coordinates is detected on the computer, the object detection unit 144 activates the detected object on the computer so that an actual key input message generated by the key input virtualization unit 130 can be input to the corresponding object. let it be

In the above-described embodiment, it has been described that the object detection unit 144 detects the corresponding object based on the two-dimensional coordinate information of the object to which the keyboard input is to be performed. However, in a modified embodiment, the object detection unit 144 uses the attribute information and hierarchical affiliation information of each object (eg, executable file name, title bar text, window control type, etc.) The object can be detected based on the calculated eigenvalue. Specifically, the object detection unit 144 detects whether an object having the same eigenvalue as the eigenvalue of the corresponding object exists on the computer, and when an object having the same eigenvalue as the eigenvalue of the object is detected, the object is detected. Activate it.

As another example, the object detector 144 may detect an object using image information of each object. Specifically, the object detection unit 144 detects whether an object having the same image information as image information obtained in advance for the object exists on the computer, and the object having the same image information as the image information of the object is detected. When this happens, the object is activated.

As another example, when determining whether the same object exists based on the image information of the object, the image of the object may be different depending on the state of the object even though it is actually the same object, so even in this case, the same object is accurately detected In order to do this, the object detection unit may detect the same object as the corresponding object by additionally using the standardized deformed image obtained in advance for each object.

To this end, the object activator 140 according to the present invention may further include an image generator 146 . The image generator 146 generates a standardized modified image for each object. In one embodiment, the standardized deformed image generated by the image generator 146 is an image when the corresponding object is in an enabled state, an image when the corresponding object is in a disabled state, Image when the object is pressed, image when the object is selected, image when the object is in focus, and the input tool is hovered on the object ) may include at least one of the images in the state.

According to this embodiment, when an object having the same image information as the image information of the object is not detected, the object detection unit 144 may be identical to the standardized modified image of the object generated by the image generation unit 146 . It is further detected whether an object having an image exists, and when an object having the same image as the standardized modified image of the corresponding object is detected, the detected object is activated.

7 is a block diagram schematically showing the configuration of a key input virtualization system for robot processor automation according to another embodiment of the present invention. 7, the key input virtualization system 700 for robot processor automation according to another embodiment of the present invention includes at least one script generating unit 710a to 710n, and at least one script executing unit 720a to 720n), at least one key input virtualization unit 730a to 730n, at least one control unit 740a to 740n, and a script management unit 750 .

In the key input virtualization system 700 for robot processor automation shown in FIG. 7 , at least one script execution unit 720a to 720n and at least one control unit 750 are interlocked in a one-to-one manner to form one control unit 750 ) is a structure that controls one script execution unit 720a to 720n. In addition, in the key input virtualization system 700 for robot processor automation shown in FIG. 7 , the script management unit 750 manages at least one script generation unit 710 and at least one script execution unit 720a to 720n. .

At least one script generating unit (710a to 710n) generates an automation script (Script) for robot process automation. To this end, each script generating unit 710a to 710n includes a process setting unit 712 , a task setting unit 714 , and a script manufacturing unit 716 . The process setting unit 712 , the task setting unit 714 , and the script manufacturing unit 716 included in the script generating units 710a to 710n and the script generating units 710a to 710n include the script generating unit 110 shown in FIG. 1 . ) and the process setting unit 112 , the task setting unit 114 , and the script production unit 116 have the same functions, and thus a detailed description thereof will be omitted.

The at least one script execution unit 720a to 720n automatically processes the user's computing task by actually executing the automated script generated by the at least one script generation unit 710 . Specifically, each script execution unit 720a to 720n receives an automation script from the script generation unit 710a to 710n or the script management unit 750, and executes an execution command from the control unit 740a to 740n or the script management unit 750. Upon receipt, it executes the corresponding automation script to handle the user's computing tasks.

Each script execution unit 720a to 720n receives an execution command for the received automation script, interprets the automation script to determine attributes, execution conditions, priority, etc. for each event, task, and process, and executes the sequence to schedule In addition, the script execution units 720a to 720n execute a task by sequentially executing each event according to the execution order, and also execute a process by executing each task according to the execution order.

In particular, each of the script execution units 720a to 720n according to the present invention interlocks the control units 740a to 740n and the script management unit 750 between the control units 740a to 740n and the script management unit 750 to interlock the control units 740a to 740n) allows the script management unit 750 to control the remote script execution unit.

In an embodiment, the script execution units 720a to 720n refer to computing environment information at the time of recording when the script is executed so that the current computing environment (screen resolution, screen arrangement, screen size, etc.) is the same as the computing environment at the time of recording. You can adjust your current computing environment to suit your needs.

The at least one key input virtualization unit 730a to 730n extracts string data from the automated script executed by the at least one script execution unit 720, and generates a virtual key input signal from the extracted string data, based on this to generate the actual key input message. The key input virtualization units 730a to 730n transmit the actual key input message generated by the keyboard driving driver 250 to the operating system (OS), so that the operating system transmits the actual key input message to the automated script execution. Allows input within the activated object according to

To this end, the key input virtualization system 700 for robot processor automation may further include an object activator (not shown) for detecting and activating an object included in the automation script. Since the function of the object activator is the same as that of the object activator shown in FIG. 1 , a detailed description thereof will be omitted. In another embodiment, detection and activation of the object may be directly performed by the above-described script execution units 720a to 720n.

Since the configuration of the key input virtualization units 730a to 730n is the same as that shown in FIG. 2 , a detailed description thereof will be omitted.

The key input virtualization unit 730 as described above may be provided for each script execution unit 720a to 720n, but in a modified embodiment, one key input virtualization unit 730 includes a plurality of script execution units ( 720a ~ 720n) may be implemented to be interlocked.

The at least one control unit (740a to 740n) is a device for the user to control the at least one script execution unit (720a to 720n). At least one control unit (740a to 740n) is interlocked with at least one script execution unit (720a to 720n) that is a control object, selects an automation script to be processed by the corresponding script execution units (720a to 720n), and commands execution. In addition, the execution result of the automation script is received from the corresponding script execution units 720a to 720n.

To this end, each of the controllers 740a to 740n provides a user interface so that the user can control the desired script execution units 720a to 720n through the corresponding controllers 740a to 740n. For example, the control units 740a to 740n provide a GUI (Graphic User Interface) for at least one of a user's login/logout, a script execution unit selection, and execution/stop of an automated script through a user interface, and execute the corresponding script The progress status of the process included in the script being executed in the unit 720, the execution result, etc. are displayed.

In addition, the control units 740a to 740n provide the user with an automated script that can be processed by the script execution units 720a to 720n. In addition, when a user registers or deletes an arbitrary automation script for the script execution units 720a to 720n, the corresponding automation script can be executed or canceled accordingly.

In addition, the control units 740a to 740n instruct the script execution units 720a to 720n to execute the corresponding automated script, receive the execution result and notify the user through the user interface.

In particular, when each control unit (740a ~ 740n) according to the present invention is linked one-to-one with each script execution unit (720a ~ 720n), the user is the first control unit that is interlocked with the first script execution unit (720a) When the execution of the automation script is registered in the second script execution unit 720b that is not linked with the first control unit 740a as 740a, the first control unit 740 is the first script execution unit 720a, the script A corresponding automation script may be executed through the management unit 750 and the second script execution unit 720b.

Specifically, the first control unit 740a first instructs the first script execution unit 720a to execute the requested automation script. Accordingly, the first script execution unit 720a transmits it to the script management unit 750, and the script management unit 750 commands execution while transmitting the corresponding automation script to the second script execution unit 720b. The second script execution unit 720b executes the automated script received from the script management unit 750 and delivers the result to the script management unit 750 . The script management unit 750 transmits the execution result of the corresponding automation script to the first script execution unit 720a, and the first script execution unit 720a transmits the execution result of the automation script received from the script management unit 750 to the first It is transmitted to the control unit (740a).

Through this, each control unit (740a ~ 740n) is not only the script execution unit (720a ~ 720n) directly interlocked with itself, but also the script management unit (750) through the remote script execution unit (720a ~ 720n) for the desired automation You can command the execution of the script.

Meanwhile, the system according to the present invention may further include a script management unit 750 .

The script management unit 750 interworks with at least one script generating unit 710a to 710n and/or at least one script execution unit 720a to 720n to manage them. The script management unit 750 interworks with at least one script generation unit 710a to 710n, and receives and manages an automated script from the at least one script generation unit 710a to 710n. In addition, the script management unit 750 is interlocked with at least one script execution unit 720a to 720n, commands the execution of an automation script to at least one script execution unit 720a to 720n, and receives and manages the execution result. .

In particular, the script management unit 750 according to the present invention is a second script execution unit 720b at a remote location according to the request of the first script execution unit 720a among the plurality of script execution units 720a to 720n. The execution of the automation script may be commanded, and the execution result may be received and transmitted to the first script execution unit 720a.

That is, the script management unit 750 includes not only the script execution units 720a to 720n in which each control unit 740a to 740n is directly interlocked with itself, but also to the other remote script execution units 720a to 720n that are not interlocked with themselves. In order to command the execution of the desired automation script, the transmission of the automation script, the transmission and reception of the execution command of the automation script, and the transmission and reception of the execution result of the automation script are performed between the script execution units 720a to 720n.

Those skilled in the art to which the present invention pertains will understand that the above-described present invention may be embodied in other specific forms without changing the technical spirit or essential characteristics thereof.

Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

100: key input virtualization system 110: script generation unit
120: script execution unit 130: key input virtualization unit
140: object activation unit 210: string data extraction unit
220: virtual key input signal generator 230: queue
240: key input unit 250: keyboard driving driver

Claims (20)

Script execution unit for executing the automation script (Script) generated for robot process automation (Robot Process Automation);
a virtual key input signal generator for generating a virtual key input signal based on a key code mapped to a character key of each character on a real keyboard for each character constituting the character string data included in the automation script; and
a keyboard driving driver for generating an actual key input message for input of the character string data based on the virtual key input signal;
The virtual key input signal generating unit divides the string data into character units, and for each divided character, a first field in which a key code mapped to the character key is recorded on the real keyboard and a pressed state (DOWN) of the character key ) A key input virtualization system for robot processor automation, characterized in that it generates the virtual key input signal consisting of a second field in which down information indicating the down information or information indicating the depressed state (UP) of the character key is recorded. According to claim 1,
The keyboard driving driver transmits the real key input message to an operating system (OS), and the real key input message is input to an object executed by the automation script through the operating system. keystroke virtualization system for delete delete According to claim 1,
The virtual key input signal is a key input virtualization system for robot processor automation, characterized in that it further comprises a third field disposed after the second field and in which delay time information between character input is recorded. According to claim 1,
When a function key input is required for input of the character, the virtual key input signal generating unit uses a key code mapped to the function key on a real keyboard and information indicating a pressed or released state of the function key. A key input virtualization system for robot processor automation, characterized in that generating a virtual key input signal for the function key. According to claim 1,
The virtual key input signal generating unit continuously generates a first virtual key input signal indicating a pressed state of a character key for each character and a second virtual key input signal indicating a depressed state of the corresponding character key. A keystroke virtualization system for robot processor automation. According to claim 1,
The virtual key input signal generating unit may include a third virtual key input signal indicating a pressed state of the function key, and a third virtual key input signal indicating a pressed state of the character key when an additional input of a function key is required to input the character. 4 A robot characterized in that it continuously generates a virtual key input signal, a fifth virtual key input signal indicating a press release state of the character key, and a sixth virtual key input signal indicating a release state of the function key A keystroke virtualization system for processor automation. According to claim 1,
Key input virtualization for robot processor automation, characterized in that it further comprises a queue in which the virtual key input signal generated by the virtual key input signal generator is sequentially inserted according to the input order of each character constituting the character string data. system. 10. The method of claim 9,
The key input virtualization system for robot processor automation, characterized in that it further comprises a key input unit for sequentially fetching the virtual key input signal from the queue and inputting the virtual key input signal to the keyboard driving driver. According to claim 1,
A key input virtualization system for robot processor automation, characterized in that in the automation script, tasks or processes to be executed for automatic task execution are sequentially recorded according to the execution order. According to claim 1,
and an object detection unit for detecting and activating an object corresponding to the information of the object on the terminal based on the information of the object included in the executed automation script. 13. The method of claim 12,
The object detection unit detects and activates an object having the same eigenvalue as the eigenvalue calculated using at least one of attribute information and hierarchical affiliation information of the object included in the automation script. Keystroke virtualization system. 13. The method of claim 12,
The object detection unit detects and activates an object having the same image as the image of the object included in the automation script and activates the key input virtualization system. 15. The method of claim 14,
Further comprising a deformation image generating unit for generating a standardized deformation image of the object included in the automation script,
The object detection unit detects and activates an object having the same image as the deformed image of the object when an object having the same image as the image of the object is not detected and activated. 16. The method of claim 15,
The standardized deformation image includes an image when the object is in an enabled state, an image when the object is in a disabled state, an image when the object is in a pressed state, and the object It characterized in that it includes at least one of an image when is in a selected state, an image when an object is in a focused state, and an image when an input tool is hovered on the object. A keystroke virtualization system for robot processor automation with According to claim 1,
an object information acquisition unit configured to acquire, in the form of two-dimensional coordinates, a position of an object to which the actual key input message corresponding to the string data is to be input from the executed automation script; and
Key input virtualization system for robot processor automation, characterized in that it further comprises an object detection unit for detecting and activating the object based on the two-dimensional coordinates obtained by the object information acquisition unit. at least one script generator configured to set a task composed of at least one event and a process composed of at least one task, and to generate an automation script for performing the task and the process;
at least one script execution unit for executing the automation script by executing at least one event constituting the task;
at least one key input virtualization unit for generating a virtual key input signal from string data to be input to an object activated according to the execution of the automation script, and generating an actual key input message based on the generated virtual key input signal; and
Comprising at least one control unit interlocked with the at least one script execution unit to control the execution of the automated script,
The at least one key input virtualization unit divides the string data into character units, and for each divided character, a first field in which a key code mapped to a character key of each character on the actual keyboard is recorded and the character key is pressed A key for robot processor automation, characterized in that it generates the virtual key input signal composed of a second field in which down information indicating a state (DOWN) or information indicating a press release state (UP) of the character key is recorded Input virtualization system. 19. The method of claim 18,
The at least one control unit corresponds to the at least one script execution unit, respectively, and commands the at least one script execution unit to execute the automation script, and the at least one script execution unit executes the corresponding automation script. A keystroke virtualization system for robot processor automation. 19. The method of claim 18,
Further comprising a script management unit that interworks with the script execution unit to manage the script execution unit,
When the script management unit receives an execution command of the target automation script to be executed by the second script execution unit from the control unit through the first script execution unit, the script management unit delivers the execution command of the target automation script and the target automation script to the second script execution unit and receiving the target automation script execution result from the second script execution unit and transferring the result to the control unit through the first script execution unit.

Images