CN113592636A

CN113592636A - Service flow processing method, device, equipment and medium based on RPA and AI

Info

Publication number: CN113592636A
Application number: CN202110923975.1A
Authority: CN
Inventors: 殷星; 王瑞丰; 汪冠春; 胡一川; 褚瑞; 李玮
Original assignee: Beijing Laiye Network Technology Co Ltd; Laiye Technology Beijing Co Ltd
Current assignee: Beijing Laiye Network Technology Co Ltd; Laiye Technology Beijing Co Ltd
Priority date: 2021-08-12
Filing date: 2021-08-12
Publication date: 2021-11-02

Abstract

The embodiment of the invention discloses a service flow processing method, a device, equipment and a medium based on RPA and AI, wherein the method comprises the following steps: s1, when receiving the operation instruction of the flow, executing the flow content corresponding to each flow block in sequence according to the sequence of each flow block; and S2, in the process of executing the flow content, if the flow auxiliary equipment needs to be started, sending a control instruction to the flow auxiliary equipment based on the communication connection between the flow auxiliary equipment and the flow auxiliary equipment, wherein the control instruction is used for instructing the flow auxiliary equipment to execute the flow action corresponding to the control instruction. By adopting the technical scheme, the problem of low execution efficiency of the automatic flow caused by shielding of the automatic script is solved.

Description

Service flow processing method, device, equipment and medium based on RPA and AI

Technical Field

The embodiment of the invention relates to the technical field of flow automation, in particular to a service flow processing method, a device, equipment and a medium based on RPA and AI.

Background

RPA (robot Process Automation) simulates human operations on a computer through specific "robot software" and automatically executes Process tasks according to rules.

AI (Artificial Intelligence) is a new technical science for studying and developing theories, methods, techniques and application systems for simulating, extending and expanding human Intelligence.

RPA has unique advantages: low code, non-intrusive. The low code means that the RPA can be operated without high IT level, and business personnel who do not know programming can also develop the flow; non-invasively, the RPA can simulate human operation without opening the interface with a software system. However, conventional RPA has certain limitations: can only be based on fixed rules and application scenarios are limited. With the continuous development of the AI technology, the limitation of the traditional RPA is overcome by the deep fusion of the RPA and the AI, and the RPA + AI is a Hand work + Head work, which greatly changes the value of the labor force.

At present, when the RPA has inputs related to passwords, the RPA generally depends on a keyboard API (Application Programming Interface) function Keybd _ event function provided by Windows. Keybd _ event can trigger a key event, that is, a WM _ KEYDOWN or WM _ KEYUP message is generated, and the effect of simulating keyboard input is realized through the message.

However, in some application scenarios, for example, in the financial field, operation scenarios such as bank stream downloading, tax declaration, etc., some websites or application programs may purposely prevent the automation script and only allow a service person to input key information such as an account password based on security considerations, considering that the above-mentioned automated operation manner is easily utilized and attacked by hackers. Common practice for these websites is to mask out the Keybd _ event function or block execution of the automation script using a hook mechanism. Therefore, the RPA process of simulating keyboard input by using the API can not be regenerated. In practice, a "human-computer interaction" method has to be adopted to solve the problem, so that the execution efficiency of the automation process is greatly reduced, and the customer is doubtful about the automation capability of the product, which is not beneficial to the sales of the product and the development of the RPA automation capability.

Disclosure of Invention

Embodiments of the present invention provide a service flow processing method, device, equipment and medium based on RPA and AI, which solve the problem of low execution efficiency of an automation flow caused by shielding an automation script.

In a first aspect, the present invention provides a service flow processing method based on RPA and AI, which is applied to an RPA robot, and includes:

s1, when receiving the operation instruction of the flow, executing the flow content corresponding to each flow block in sequence according to the sequence of each flow block;

and S2, in the process of executing the flow content, if the flow auxiliary equipment needs to be started, sending a control instruction to the flow auxiliary equipment based on the communication connection between the flow auxiliary equipment and the flow auxiliary equipment, wherein the control instruction is used for instructing the flow auxiliary equipment to execute the flow action corresponding to the control instruction.

Optionally, the method further includes:

s3, receiving a return message of the flow auxiliary equipment for completing the flow action corresponding to the control instruction;

and S4, executing the flow contents corresponding to the rest of the flow blocks in sequence according to the sequence of the unexecuted flow blocks.

Optionally, the process content includes logging in to the internet bank; correspondingly, the control instruction is a login information input instruction;

correspondingly, the S2 specifically includes:

s21, opening an online bank login interface;

s22, sending the login information input instruction to the process auxiliary equipment based on the communication connection with the process auxiliary equipment, wherein the login information input instruction is used for instructing the process auxiliary equipment to respectively input the account and the password corresponding to the online bank into the corresponding positions of the login dialog box;

wherein the password is stored in the process aid in an encrypted form.

Optionally, the method further includes:

s23, in the process of logging in the online bank, if the verification code is detected to exist, identifying the picture where the verification code is located by adopting an Optical Character Recognition (OCR) method;

and S24, inputting the recognition result into the verification code filling place of the login dialog box.

In a second aspect, an embodiment of the present invention further provides a service flow processing method based on RPA and AI, which is applied to a flow auxiliary device, and the method includes:

s5, receiving a control instruction sent by the RPA robot based on the communication connection with the host where the RPA robot is located in the process of executing the process content corresponding to each process block by the RPA robot;

and S6, executing corresponding flow action according to the control command.

Optionally, the communication connection between the process auxiliary device and the host where the RPA robot is located is established through a USB interface.

correspondingly, the S6 specifically includes:

and respectively inputting the stored account number of the online bank and the decrypted password corresponding to the account number into a login dialog box of an online bank login interface according to the login information input instruction.

Optionally, the verification code required to be input in the internet bank login process is obtained by identifying the picture where the verification code is located by the RPA robot by using an Optical Character Recognition (OCR) method.

In a third aspect, an embodiment of the present invention further provides a service flow processing apparatus based on RPA and AI, including:

a flow execution module configured to: when an operation instruction of the process is received, sequentially executing the process content corresponding to each process block according to the sequence of each process block;

a control instruction sending module configured to: in the process of executing the process content, if a process auxiliary device needs to be started, sending a control instruction to the process auxiliary device based on the communication connection between the process auxiliary device and the process auxiliary device, wherein the control instruction is used for instructing the process auxiliary device to execute a process action corresponding to the control instruction.

Optionally, the apparatus further comprises:

a return message receiving module configured to: receiving a return message of the flow auxiliary equipment for completing the flow action corresponding to the control instruction;

a remaining block execution module configured to: and executing the flow contents corresponding to the rest flow blocks in sequence according to the sequence of the flow blocks which are not executed.

correspondingly, the control instruction sending module 420 is specifically configured to:

opening an online bank login interface;

sending the login information input instruction to the process auxiliary equipment based on the communication connection with the process auxiliary equipment, wherein the login information input instruction is used for instructing the process auxiliary equipment to respectively input an account and a password corresponding to the online bank into corresponding positions of the login dialog box;

wherein the password is stored in the process aid in an encrypted form.

Optionally, the apparatus further comprises:

a passcode identification module configured to: in the process of logging in the online bank, if the verification code is detected to exist, identifying the picture where the verification code is located by adopting an Optical Character Recognition (OCR) method;

a recognition result input module configured to: and inputting the identification result into the verification code filling part of the login dialog box.

In a fourth aspect, an embodiment of the present invention further provides a service flow processing apparatus based on RPA and AI, including:

a control instruction receiving module configured to: in the process of executing the process content by the RPA robot, receiving a control instruction sent by the RPA robot based on the communication connection with a host where the RPA robot is located;

an instruction execution module configured to: and executing corresponding flow actions according to the control instructions.

correspondingly, the instruction execution module is specifically configured to:

In a fifth aspect, an embodiment of the present invention further provides a computing device, including:

a memory storing executable program code;

a processor coupled with the memory;

the processor calls the executable program code stored in the memory to execute the RPA and AI-based business process processing method applied to the RPA robot provided by any embodiment of the invention.

In a sixth aspect, an embodiment of the present invention further provides a computing device, including:

a memory storing executable program code;

a processor coupled with the memory;

the processor calls the executable program code stored in the memory to execute the service flow processing method based on RPA and AI provided by any embodiment of the invention and applied to the flow auxiliary equipment.

In a seventh aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the RPA and AI-based business process processing method applied to an RPA robot, provided in any embodiment of the present invention.

In an eighth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the RPA and AI-based business process processing method applied to the process assistance device, provided by any embodiment of the present invention.

According to the technical scheme provided by the embodiment of the invention, in the process of executing the automation flow by the RPA, under the application scene that the RPA tool is forbidden due to shielding of the automation script, the action of the RPA can be completed by the flow auxiliary equipment of the drive layer. The RPA is based on the communication connection with the process auxiliary equipment, and the process auxiliary equipment can execute the process action corresponding to the control instruction by sending the control instruction to the process auxiliary equipment, so that the efficient, accurate and reliable process automation operation is realized in the scene that the RPA tool is forbidden.

The innovation points of the embodiment of the invention comprise:

1. in an application scenario where the automation script is shielded and an RPA tool cannot be used, the flow auxiliary device of the driver layer is called to execute the flow automation action, so that the efficiency and reliability of the flow automation are improved, and the method is one of the innovative points of the embodiment of the present invention.

2. The password stored in the flow auxiliary equipment is encrypted, so that the safety of the password of the user is improved, and the method is one of the innovation points of the embodiment of the invention.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1a is an interaction diagram of an RPA robot and a process aid in an online banking login scenario;

FIG. 1b is a flow diagram set up using a process automation platform UiBot;

FIG. 1c is a screenshot of relevant instructions for logging on to a bank using a process aid;

FIG. 1d is a screenshot of password storage in the procedural assistant device management software;

fig. 2 is a flowchart of a service flow processing method based on RPA and AI according to an embodiment of the present invention;

fig. 3 is a flowchart of a service flow processing method based on RPA and AI according to a second embodiment of the present invention;

fig. 4 is a flowchart of a service flow processing method based on RPA and AI according to a third embodiment of the present invention;

fig. 5 is a block diagram of a service flow processing apparatus based on RPA and AI according to a fourth embodiment of the present invention;

fig. 6 is a block diagram of a service flow processing apparatus based on RPA and AI according to a fifth embodiment of the present invention;

fig. 7 is a schematic structural diagram of a computing device according to a sixth embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention.

It is to be noted that the terms "comprises" and "comprising" and any variations thereof in the embodiments and drawings of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

In the description of the embodiments of the present invention, "flow block" is used to indicate a process block in which a flow in a flowchart runs;

in the description of the embodiment of the present invention, the "asynchronous instruction" means that a process does not need to wait all the time, but continues to perform the following operation regardless of the states of other processes. When a message is returned, the system will notify the process to process. In the embodiment of the invention, after the RPA robot sends out the control signal, the flow auxiliary equipment can sequentially execute a series of designed flow actions according to the control signal. After the flow auxiliary equipment finishes each designed instruction in turn, the flow auxiliary equipment returns an answer signal or an end signal.

In order to clearly and clearly explain the contents of the embodiments of the present invention, the following briefly introduces the basic operation principle of the embodiments of the present invention.

Robot Process Automation (RPA) is a Process task that simulates human operations on a computer through specific robot software and automatically executes according to rules.

Ai (intellectual intelligence) is an english abbreviation for artificial intelligence, which is a new technical science for studying and developing theories, methods, techniques and application systems for simulating, extending and expanding human intelligence.

In the existing service, when there is an input scenario involving a password in the RPA, a flow designer typically designs the following way: 1. and writing the password into the code by adopting a hard coding mode, and finally converting the password into a ciphertext and writing the ciphertext into the code by using an algorithm built in the product when generating the flow. When the user executes the RPA process through the actuator, the ciphertext data is restored into a plaintext password through a corresponding decryption algorithm and is filled in the target element. 2. And adopting a storage certificate of the management center, namely a stored account password. When a process designer encounters a scene needing to input an account password, the process designer firstly sends the corresponding account password through an ID or name request management center, and the management center usually returns a user name and a password field after receiving a request of a user. And filling the returned account password into the corresponding scene by the user. The input of the account password of the two methods depends on a keyboard API function Keybd _ event function provided by Windows. Keybd _ event can trigger a key event, that is, a WM _ KEYDOWN or WM _ KEYUP message is generated, and the effect of simulating keyboard input is realized through the message.

Based on security considerations, there are currently some websites/applications that prevent the execution of automation scripts at the application layer, e.g., existing Keybd _ event APIs are masked. In such application scenarios, a service person usually needs to input key information such as an account password by manually operating a keyboard. In consideration of the problem that manual operation easily causes low efficiency of an automatic process, the technical scheme provided by the embodiment of the invention adopts actual physical hardware, namely, the process auxiliary equipment provided by the embodiment of the invention, as input equipment of key information such as account numbers and passwords. The process auxiliary equipment operates on a driving layer, and the process auxiliary equipment can realize the simulation key pressing of a driving level.

Wherein, the flow auxiliary equipment includes: an MCU (micro controller Unit), a USB (Universal Serial Bus) interface, and an EEPROM (Electrically Erasable Programmable read only memory). The process aid is a U-disk like structure. Each process aid is provided with a fixed equipment ID (Identity Document) that can be imprinted on the outside of the process aid to make it clearly visible to distinguish between different process aids.

Considering the scenario that partial RPA tools are disabled, the operation capability of partial automatic clicking and automatic inputting needs to be performed in the flow auxiliary equipment, so that relevant instructions of 'asynchronous execution' are designed for the flow auxiliary equipment, including 'submit asynchronous action', 'asynchronous simulated clicking' and 'asynchronous input password', and the like, and the instructions are integrated into a 'code block' to be written into the flow auxiliary equipment. When the RPA process is executed, if a scene that the automation script is disabled is encountered, the RPA can activate the corresponding code in the process auxiliary equipment in advance, and at the moment, the process auxiliary equipment can execute a preset series of process actions. Because the operation of the process auxiliary device is drive layer, the process auxiliary device can realize drive-level simulation key pressing in the application scene that the automation script of the application layer is shielded. Compared with the mode of inputting information by manually operating the keyboard in the scene in the prior art, the method and the device for inputting information by adopting the process auxiliary equipment have higher efficiency and better accuracy. .

In the following, referring to fig. 1a to 1d, the operation of the flow auxiliary device will be briefly described by taking an application scenario of logging in online banking as an example.

Fig. 1a is an interaction diagram of an RPA robot and a process auxiliary device in an online banking login scene. Fig. 1b is a flow chart set up by using a process automation platform (UiBot), and fig. 1c is a screenshot of related instructions when logging in to the internet bank by using a process auxiliary device. As shown in FIG. 1a, during the development phase, a process aid may first be inserted into the developer's computer. The passwords required for logging in each internet bank can be written into the process auxiliary equipment through the process auxiliary equipment management software. FIG. 1d is a screenshot of password storage in the procedural assistant device management software. As shown in fig. 1d, multiple sets of passwords that need to be used may be stored in a process auxiliary device (KeyBox for short) with a device number "10000268", that is, a binding relationship between an ID of the process auxiliary device and the stored passwords is established. In addition, in the process development phase, for a certain bank login page scene, a group of passwords needs to be specified for the ID of the process auxiliary equipment to be used, and the passwords are used as input passwords to be used in the internet bank login. In addition, as shown in fig. 1c, corresponding code segments of asynchronous instructions "enter account 123456789", "simulate keyboard Tab performs single click", and "enter password" that the flow auxiliary device needs to execute may also be written into the flow auxiliary device.

After the process design is completed and released, in the process that the user uses the process, the RPA robot can sequentially execute the process contents corresponding to the process blocks according to the sequence of the process blocks. As shown in fig. 1b, when the RPA robot runs to the flow block of KeyBox logging in to the internet bank, it will execute the instructions shown in fig. 1 c. According to the behavior example of logging in a certain target network, the RPA robot can start an IE browser, open the website of a bank to be logged in, click 'personal bank popular version' through a mouse, and 'log in by using an account password'. The RPA robot may place the cursor into the "account" input box by moving (1190,328) the mouse to the pixel location of the "account" of the login box and then clicking the left button with the mouse. In this application scenario, since the RPA automation script is shielded by the website, the operation of inputting the account and the password may be performed by the process assistance device. As shown in fig. 1a, the RPA robot sends a control instruction for inputting an account number and a password to the flow auxiliary device according to the ID number of the flow auxiliary device inserted into the host, and the flow auxiliary device executes an asynchronous command, that is, sequentially executes the pre-designed flow actions after receiving the control instruction. As shown in fig. 1c, the flow auxiliary device first inputs an account number "123456789", then presses the Tab key to perform a single-click operation, i.e., switches to the password input box, and then inputs the stored first group of passwords into the located password input box.

Because the shielding of the website for logging in the internet bank to the automation script is in an application layer, and the flow auxiliary equipment is in communication connection with the host through the USB interface in the embodiment of the invention, namely, the operation of the flow auxiliary equipment is in a driving layer, the flow auxiliary equipment can be used for assisting the RPA robot to complete the automation flow operation under the application scene that the automation script cannot normally run, and the operation is accurate and reliable, the efficiency is higher, and the reliability is better.

And after finishing the action corresponding to the preset instruction, the flow auxiliary equipment returns a message of finishing the action. After receiving the message returned by the flow assistant device, the RPA robot may then perform the operation of the flow content, for example, in the application scenario of the above-mentioned internet banking login, if the verification code is detected to exist, the RPA may call a component with AI capability, for example, identify a picture where the verification code is located through OCR (Optical Character Recognition), so as to obtain the content of the verification code. If the content of the verification code needs to be analyzed, for example, the number in the verification code needs to be calculated, the result of the verification code can be obtained through NLP (Natural Language Processing), and the result is input to the verification code filling part.

Next, the service flow processing procedure is described in detail from the perspective of the RPA robot and the perspective of the flow auxiliary device, respectively.

Example one

Fig. 2 is a flowchart of a service flow processing method based on RPA and AI according to an embodiment of the present invention, where an execution main body of the method is an RPA robot. Typically, the method can be applied to an application scenario in which the process automation tool is disabled, and can be performed by an RPA and AI-based business process processing method device, which can be implemented by software and/or hardware. As shown in fig. 2, the method includes:

and S110, when the operation instruction of the flow is received, sequentially executing the flow content corresponding to each flow block according to the sequence of each flow block.

In the flowcharts, the execution order of the blocks may be represented by the directions indicated by arrows. In the flowchart, there are a flow start block for indicating the start of the flow, an end block for indicating the end of the flow, and a flow block for performing different operation processing between the flow start block and the flow end block. As shown in fig. 1b, when receiving the operation instruction of the flow, the RPA robot executes KeyBox login to the internet bank, view and download the contents of the pipeline and the mailing list in sequence from the flow start block to the flow end block.

In the process diagram development stage, research and development staff can set different instructions for different operations for process blocks which are subjected to different operation processing between a process starting block and a process ending block, and the process content corresponding to the executed process block is the operation instruction corresponding to the executed process block, so that different functions are realized. For example, by executing the respective instructions as shown in fig. 1c, automatic login of the internet bank can be realized.

And S120, in the process of executing the flow content, if the flow auxiliary equipment needs to be started, sending a control instruction to the flow auxiliary equipment based on the communication connection with the flow auxiliary equipment, wherein the control instruction is used for instructing the flow auxiliary equipment to execute the flow action corresponding to the control instruction.

The process auxiliary device is a device for realizing process automation operation instead of a shielded RPA tool in an application scene where an automation script is shielded, for example, in the financial field, in a scene of downloading bank water or in tax declaration, and the like. The actions of the flow auxiliary equipment can be controlled by the MCU in the flow auxiliary equipment. Since the process auxiliary device is a communication connection established with the host through the USB interface, that is, the operations of the process auxiliary device are all driver-level, and the shielded scene of the general RPA tool generally shields an automation script of an application layer, the RPA robot can be assisted by the process auxiliary device to complete the automation process operation.

In addition, the flow auxiliary equipment also has a storage function, and the storage function can be realized through an EEPROM device in the flow auxiliary equipment or a flash space built in the MCU. Each process aid has its unique ID number. The use and management of the process assistance device can be generally implemented by process assistance device management software. For example, for the password stored in the flow auxiliary device, the user can modify the account password stored in the flow auxiliary device through the management software without updating the flow, and such design can greatly increase the flexibility of the RPA flow and avoid that the user frequently updates the flow software version. Also, a user may upgrade the process kit firmware through the process kit management software to extend its capabilities, taking into account hardware compatibility and extensibility capabilities.

In this embodiment, whether the process auxiliary device needs to be started or not may be determined by a service scenario of the process.

For example, during the process of the automated process operation performed by the RPA robot, it may be detected whether the automated script is masked, that is, in a case that the automated process cannot be normally performed, for example, during an internet banking login process, when the RPA robot simulates a keyboard to input an account of a user, the account cannot be successfully input into the login dialog box, at this time, it indicates that the bank masks a message sent by the automated software through some technical means, and at this time, it may be determined to start the process auxiliary device.

For example, the published flow is generally developed and tested by developers before being used by users, that is, some application scenarios that cannot use the RPA tool are determined in the testing process. After the process is released, in the process of using the process by the user, for the operation of the RPA in the application scenario where the RPA tool cannot be used, for example, the operation of inputting a bank password or the like, the process auxiliary device needs to be started, so as to complete the automation operation of the process by means of the process auxiliary device.

In this embodiment, when the process auxiliary device needs to be started, a control instruction may be sent to the process auxiliary device based on the communication connection with the process auxiliary device. Different control instructions carry corresponding identification information, for example, the instruction is a password input instruction or a simulated click instruction, and if the instruction is the password input instruction, information such as an input password number and the like is embodied in the identification information in the control instruction. After receiving the control instruction, the flow auxiliary device triggers the code segment corresponding to the control instruction, so as to execute a series of flow actions corresponding to the control instruction.

Further, the flow auxiliary device may return an end message to the RPA robot after completing the flow action corresponding to the control instruction. And after receiving the return message sent by the flow auxiliary equipment, the RPA robot continues to execute the flow content corresponding to the residual unexecuted flow block. For example, as shown in fig. 1b, after logging in the internet bank through the KeyBox, the RPA robot will continue to perform the viewing and downloading processes and send the report through the mail.

According to the technical scheme provided by the embodiment of the invention, in the process of executing the automation flow by the RPA, under the application scene that the RPA tool is forbidden due to shielding of the automation script, the action of the RPA can be executed by the flow auxiliary equipment. The RPA is based on the communication connection with the process auxiliary equipment, and the process auxiliary equipment can execute the process action corresponding to the control instruction by sending the control instruction to the process auxiliary equipment, so that the efficient, accurate and reliable process automation operation is realized under the condition that the RPA tool is forbidden.

Example two

Fig. 3 is a flowchart of a service flow processing method based on RPA and AI according to a second embodiment of the present invention, where the embodiment is refined based on the foregoing embodiment, an application scenario is refined into an online banking login scenario, and an automated flow operation in the scenario is introduced, as shown in fig. 3, the method includes:

and S210, when the operation instruction of the flow is received, sequentially executing the flow content corresponding to each flow block according to the sequence of each flow block.

In this embodiment, the process content is login internet banking.

And S220, opening an online bank login interface.

And S230, sending a login information input instruction to the process auxiliary equipment based on the communication connection with the process auxiliary equipment, wherein the login information input instruction is used for instructing the process auxiliary equipment to respectively input the account and the password corresponding to the internet bank into the corresponding positions of the login dialog box.

In this embodiment, in an application scenario of online banking login in which the RPA tool is disabled, a user may insert a process auxiliary device into a host currently operating through a USB interface in advance to establish a communication connection between the host and the process auxiliary device. After the communication connection is established, the RPA may read the ID number of the flow process aid and send a login information input instruction to the flow process aid. In the application scenario of this embodiment, the login information input instruction is an account and password input instruction. In the process development stage, different passwords corresponding to different internet banking need to be specified for the process auxiliary equipment to log in. As shown in fig. 1c, the process aid needs to enter group 1 password.

Wherein the password may be stored in the process aid in an encrypted form. The process auxiliary equipment can determine the password to be input according to the received login information input instruction, decrypt the password and input the decrypted password into the corresponding position of the login dialog box. Wherein, the encryption and decryption operations can be realized by the MCU in the process auxiliary equipment.

And S240, receiving a return message of the flow action corresponding to the control instruction completed by the flow auxiliary equipment.

And S250, if the verification code is detected to exist, identifying the picture where the verification code is located by adopting an Optical Character Recognition (OCR) method.

And S260, inputting the identification result into a verification code filling part of the login dialog box.

For example, if the recognition result is a letter or a number, the recognition result may be directly input to the verification code filling place of the login dialog box. Further, if there are four arithmetic operations of numbers in the recognition result, NLP can be adopted to perform semantic analysis and obtain the result of the four arithmetic operations. The RPA may enter the result into the authentication code writing place of the login dialog.

According to the technical scheme provided by the embodiment, in the online bank login process, the login account and the password of the online bank are input by calling the process auxiliary equipment, and the automatic operation of the online bank login is realized in the application scene that the RPA tool is forbidden. Compared with a mode of manually inputting an account number and a password through man-machine interaction, the method is higher in efficiency and better in accuracy.

EXAMPLE III

Fig. 4 is a flowchart of a service flow processing method based on RPA and AI according to a third embodiment of the present invention, where an execution main body is a flow auxiliary device. The method is typically applicable in application scenarios where the process automation tool is disabled. The method can be executed by a service flow processing method device based on RPA and AI, and the device can be realized by software and/or hardware. As shown in fig. 4, the method includes:

and S310, receiving a control command sent by the RPA robot based on the communication connection with the host computer where the RPA robot is located in the process of executing the process content corresponding to each process block by the RPA robot.

The communication connection between the process auxiliary equipment and the host where the RPA robot is located is established through a USB interface. For a specific interaction process between the RPA robot and the procedure auxiliary device, reference may be made to the description of the above embodiments, and details are not described here.

And S320, executing corresponding flow actions according to the control command.

In this embodiment, considering a scenario in which partial RPA tools are disabled, the operational capability of partial automated clicking and automated inputting needs to be implemented in the flow auxiliary device, so that relevant instructions for "asynchronous execution" are designed in the flow auxiliary device, including "submit asynchronous action", "asynchronous simulated clicking", and "asynchronous input password", and these instructions are integrated into one "code block" and written into the flow auxiliary device. When the RPA process is executed, if a scene that the RPA tool is disabled is encountered, the RPA will activate the corresponding code in the process auxiliary equipment in advance, and at this time, the process auxiliary equipment will execute a preset series of process actions. Since the disabled is the process automation operation at the application layer, and the operations of the process auxiliary devices are all driven layers, the process auxiliary devices can assist the RPA to realize the complete process automation operation in this scenario.

Specifically, when the process content executed by the RPA is login internet banking, the instruction sent by the RPA to the process auxiliary device is a login information input instruction. After receiving the instruction, the flow assistant device may execute the instruction shown in fig. 1c according to the instruction, so as to input the online banking account number and the password into the login dialog box. If the password stored in the process auxiliary equipment exists in an encrypted form, the password can be decrypted firstly, and the decrypted password is input into a login dialog box of the online bank login interface.

Further, the flow auxiliary device may return an action end message to the RPA after completing the flow action corresponding to the control instruction, and the RPA may continue to execute the corresponding automated flow operation after receiving the message.

According to the technical scheme provided by the embodiment, in the process of executing the automation flow by the RPA, under the application scenario that the automation script is shielded to disable the RPA tool, the action of the RPA can be executed by the flow auxiliary equipment. The RPA is based on the communication connection with the process auxiliary equipment, and the process auxiliary equipment can execute the process action corresponding to the control instruction by sending the control instruction to the process auxiliary equipment, so that the efficient, accurate and reliable process automation operation is realized under the condition that the RPA tool is forbidden.

Example four

Fig. 5 is a block diagram of a service flow processing apparatus based on RPA and AI according to a fourth embodiment of the present invention, and as shown in fig. 5, the apparatus includes: a flow execution module 410 and a control instruction sending module 420; wherein the content of the first and second substances,

a flow execution module 410 configured to: when an operation instruction of the process is received, sequentially executing the process content corresponding to each process block according to the sequence of each process block;

a control instruction sending module 420 configured to: in the process of executing the process content, if a process auxiliary device needs to be started, sending a control instruction to the process auxiliary device based on the communication connection between the process auxiliary device and the process auxiliary device, wherein the control instruction is used for instructing the process auxiliary device to execute a process action corresponding to the control instruction.

Optionally, the apparatus further comprises:

opening an online bank login interface;

wherein the password is stored in the process aid in an encrypted form.

Optionally, the apparatus further comprises:

The service flow processing device based on the RPA and the AI provided by the embodiment of the invention can execute the service flow processing method based on the RPA and the AI applied to the RPA robot provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method. For details of the service flow processing method based on RPA and AI applied to the RPA robot, reference may be made to the technical details not described in detail in the above embodiments.

EXAMPLE five

Fig. 6 is a block diagram of a service flow processing device based on RPA and AI according to a fifth embodiment of the present invention, and as shown in fig. 6, the device includes: a control instruction receiving module 510 and an instruction executing module 520; wherein the content of the first and second substances,

a control instruction receiving module 510 configured to: in the process of executing the process content by the RPA robot, receiving a control instruction sent by the RPA robot based on the communication connection with a host where the RPA robot is located;

an instruction execution module 520 configured to: and executing corresponding flow actions according to the control instructions.

accordingly, the instruction execution module 520 is specifically configured to:

The service flow processing device based on the RPA and the AI provided by the embodiment of the invention can execute the service flow processing method based on the RPA and the AI applied to the flow auxiliary equipment provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method. For details of the service process based on RPA and AI, which are not described in detail in the above embodiments, reference may be made to the service process processing method based on RPA and AI, which is provided in any embodiment of the present invention.

EXAMPLE six

Referring to fig. 7, fig. 7 is a schematic structural diagram of a computing device according to a sixth embodiment of the present invention. As shown in fig. 7, the computing device may include:

a memory 701 in which executable program code is stored;

a processor 702 coupled to the memory 701;

the processor 702 calls the executable program code stored in the memory 701 to execute the service flow processing method based on RPA and AI applied to the RPA robot according to any embodiment of the present invention.

An embodiment of the present invention further provides a process auxiliary device, where the process auxiliary device includes:

one or more processors;

a storage device for storing one or more programs,

when the one or more programs are executed by the one or more processors, the one or more processors implement the RPA and AI-based business process processing method applied to the process auxiliary device according to any embodiment of the present invention.

The embodiment of the invention discloses a computer readable storage medium which stores a computer program, wherein the computer program enables a computer to execute a service flow processing method based on RPA and AI, which is applied to an RPA robot.

The embodiment of the invention discloses a computer readable storage medium which stores a computer program, wherein the computer program enables a computer to execute a service flow processing method based on RPA and AI, which is applied to flow auxiliary equipment.

In various embodiments of the present invention, it should be understood that the sequence numbers of the above-mentioned processes do not imply an inevitable order of execution, and the execution order of the processes should be determined by their functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

In the embodiments provided herein, it should be understood that "B corresponding to A" means that B is associated with A from which B can be determined. It should also be understood, however, that determining B from a does not mean determining B from a alone, but may also be determined from a and/or other information.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated units, if implemented as software functional units and sold or used as a stand-alone product, may be stored in a computer accessible memory. Based on such understanding, the technical solution of the present invention, which is a part of or contributes to the prior art in essence, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for causing a computer device (which may be a personal computer, a server, a network device, or the like, and may specifically be a processor in the computer device) to execute part or all of the steps of the above-described method of each embodiment of the present invention.

It will be understood by those skilled in the art that all or part of the steps in the methods of the embodiments described above may be implemented by hardware instructions of a program, and the program may be stored in a computer-readable storage medium, where the storage medium includes Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM), or other Memory, such as a magnetic disk, or a combination thereof, A tape memory, or any other medium readable by a computer that can be used to carry or store data.

Those of ordinary skill in the art will understand that: the figures are merely schematic representations of one embodiment, and the blocks or flow diagrams in the figures are not necessarily required to practice the present invention.

Those of ordinary skill in the art will understand that: modules in the devices in the embodiments may be distributed in the devices in the embodiments according to the description of the embodiments, or may be located in one or more devices different from the embodiments with corresponding changes. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A business process processing method based on RPA and AI is applied to the RPA robot, which is characterized in that the method comprises the following steps:

2. The method of claim 1, wherein the process content comprises logging into an online bank; correspondingly, the control instruction is a login information input instruction;

correspondingly, the S2 specifically includes:

s21, opening an online bank login interface;

wherein the password is stored in the process aid in an encrypted form.

3. The method of claim 2, further comprising:

4. A service process processing method based on RPA and AI is applied to process auxiliary equipment, and is characterized by comprising the following steps:

and S6, executing corresponding flow action according to the control command.

5. A business process processing device based on RPA and AI is applied to RPA robot, which is characterized in that it includes:

6. A business process processing device based on RPA and AI is applied to process auxiliary equipment, which is characterized in that the device comprises:

7. A computing device, wherein the computing device comprises:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the RPA and AI based business process method of any one of claims 1-3 as applied to an RPA robot.

8. A procedure assistance device, the procedure assistance device comprising:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the RPA and AI based business process method as recited in claim 4, applied to a process aid.

9. A computer-readable storage medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the RPA and AI-based business process processing method applied to an RPA robot as described in any one of 1 to 3.

10. A computer-readable storage medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the RPA and AI-based business process handling method applied to a process aid of claim 4.