CN111324440A - Method, device and equipment for executing automation process and readable storage medium - Google Patents
Method, device and equipment for executing automation process and readable storage medium Download PDFInfo
- Publication number
- CN111324440A CN111324440A CN202010100850.4A CN202010100850A CN111324440A CN 111324440 A CN111324440 A CN 111324440A CN 202010100850 A CN202010100850 A CN 202010100850A CN 111324440 A CN111324440 A CN 111324440A
- Authority
- CN
- China
- Prior art keywords
- terminal
- intention
- analysis model
- data
- intention analysis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Abstract
The invention discloses an execution method, a device, equipment and a readable storage medium of an automation process, which relate to the field of financial science and technology, and the method comprises the following steps: obtaining multi-modal data corresponding to a terminal, and taking the multi-modal data as the input of a deep learning model to obtain an intention analysis model; determining a behavior intention corresponding to an automatic process in the terminal through the intention analysis model; and executing a target operation instruction corresponding to the automatic flow according to the behavior intention so as to execute the automatic flow. The method and the device realize the analysis of the behavior intention corresponding to the automatic flow through the intention analysis model, execute the automatic flow according to the behavior intention, and avoid the occurrence of the situation that the execution of the automatic flow fails due to the change of the execution environment corresponding to the automatic flow, thereby improving the adaptability of the automatic flow to different execution environments, namely improving the adaptability of the automatic flow and improving the execution success rate of the automatic flow.
Description
Technical Field
The invention relates to the technical field of data processing of financial technology (Fintech), in particular to an execution method, device and equipment of an automatic process and a readable storage medium.
Background
With the development of computer technology, more and more technologies are applied in the financial field, the traditional financial industry is gradually changing to financial technology (Fintech), and the data processing technology is no exception, but due to the requirements of the financial industry on safety and real-time performance, higher requirements are also put forward on the technology.
In the current process automation scenario, the automation process can only be executed mechanically according to a preset instruction, and if the automation process is successfully executed, it is necessary to ensure that the execution environment of the automation process is completely consistent with the set environment. For example, in an RPA (software Process Automation) Automation working Process, a corresponding RPA robot can only execute according to a preset instruction sequence, but an actual execution environment in which the RPA robot executes an Automation Process is always different from a preset execution environment by a greater or lesser amount, that is, some dynamic factors may exist in the actual execution environment to cause the RPA robot to be unable to successfully execute the Automation Process, or cause the Automation Process to be unable to execute according to a normal Process, and dynamic factors such as an advertisement pop-up window and automatic updating of a background program may exist in an actual scene.
Therefore, in the current process automation scene, the applicability and the success rate of the automation process are poor.
Disclosure of Invention
The invention mainly aims to provide an execution method, device and equipment of an automatic process and a readable storage medium, and aims to solve the technical problems of poor applicability and low success rate of the automatic process in the existing process automation scene.
In order to achieve the above object, the present invention provides an execution method of an automation process, including:
obtaining multi-modal data corresponding to a terminal, and taking the multi-modal data as the input of a deep learning model to obtain an intention analysis model;
determining a behavior intention corresponding to an automatic process in the terminal through the intention analysis model;
and executing a target operation instruction corresponding to the automatic flow according to the behavior intention so as to execute the automatic flow.
Preferably, after the step of obtaining the multi-modal data corresponding to the terminal and using the multi-modal data as the input of the deep learning model to obtain the intention analysis model, the method further includes:
acquiring local use data of a user corresponding to a terminal, performing federal learning based on the local use data, and updating the intention analysis model according to a learning result of the federal learning to obtain an updated intention analysis model;
the step of determining the behavior intention corresponding to the automatic process in the terminal through the intention analysis model comprises the following steps:
and determining the behavior intention corresponding to the automatic process in the terminal through the updated intention analysis model.
Preferably, the step of obtaining the local usage data of the terminal corresponding to the user, performing federal learning based on the local usage data, and updating the intention analysis model according to a learning result of the federal learning includes:
the method comprises the steps of obtaining local use data of a terminal corresponding to a user, carrying out federal learning based on the local use data, and obtaining a gradient value obtained by the federal learning;
and updating the model parameters in the intention analysis model according to the gradient values to obtain an updated intention analysis model.
Preferably, after the step of obtaining the multi-modal data corresponding to the terminal and using the multi-modal data as the input of the deep learning model to obtain the intention analysis model, the method further includes:
acquiring interactive data of interaction between a terminal and a server, performing federal learning based on the interactive data, and updating the intention analysis model according to a learning result of the federal learning to obtain an updated intention analysis model;
the step of determining the behavior intention corresponding to the automatic process in the terminal through the intention analysis model comprises the following steps:
and determining the behavior intention corresponding to the automatic process in the terminal through the updated intention analysis model.
Preferably, after the step of determining the behavior intention corresponding to the automated process in the terminal through the intention analysis model, the method further includes:
detecting whether non-process factors which do not belong to the automatic process are detected in the process of executing the automatic process in the terminal;
if the non-process factor is detected, the step of executing the target operation instruction corresponding to the automatic process according to the behavior intention so as to execute the automatic process comprises the following steps:
and executing a target operation instruction corresponding to the automatic flow according to the behavior intention and the non-flow factors so as to execute the automatic flow.
Preferably, the obtaining of the multi-modal data corresponding to the terminal, and the using the multi-modal data as the input of the deep learning model to obtain the intention analysis model includes:
obtaining multi-modal data corresponding to a terminal and obtaining model sample data corresponding to the terminal;
and taking the multi-modal data and the model sample data as the input of a deep learning model to obtain an intention analysis model.
Preferably, after the step of executing the target operation instruction corresponding to the automated process according to the behavior intention to execute the automated process, the method further includes:
detecting the execution success rate of the automatic flow;
and if the execution success rate is less than the preset success rate, updating the intention analysis model.
In addition, to achieve the above object, the present invention further provides an execution device of an automation process, including:
the system comprises an acquisition module, a target module and a target module, wherein the acquisition module is used for acquiring multi-modal data corresponding to a terminal and taking the multi-modal data as the input of a deep learning model so as to obtain an intention analysis model;
the determining module is used for determining a behavior intention corresponding to an automatic process in the terminal through the intention analysis model;
and the execution module is used for executing the target operation instruction corresponding to the automatic flow according to the behavior intention so as to execute the automatic flow.
In addition, in order to achieve the above object, the present invention further provides an execution device of an automation process, where the execution device of an automation process includes a memory, a processor, and an execution program of an automation process stored in the memory and executable on the processor, and when the execution program of an automation process is executed by the processor, the execution program of an automation process implements steps of an execution method of an automation process corresponding to a federal learning server.
In addition, to achieve the above object, the present invention further provides a computer-readable storage medium having stored thereon an execution program of an automation process, which when executed by a processor, implements the steps of the execution method of the automation process as described above.
According to the method, the intention analysis model is obtained by taking multi-modal data corresponding to the terminal as the input of the deep learning model, the behavior intention corresponding to the automatic process in the terminal is determined through the intention analysis model, and the target operation instruction corresponding to the automatic process is executed according to the behavior intention so as to execute the automatic process. The behavior intention corresponding to the automatic flow is analyzed through the intention analysis model, the automatic flow is executed according to the behavior intention, and the situation that the automatic flow fails to be executed due to the fact that the automatic flow corresponds to the execution environment is avoided, so that the adaptability of the automatic flow to different execution environments is improved, namely the adaptability of the automatic flow is improved, and the execution success rate of the automatic flow is improved.
Drawings
FIG. 1 is a schematic flow chart diagram illustrating a first embodiment of a method for performing an automated process according to the present invention;
FIG. 2 is a schematic flow chart diagram of a second embodiment of a method for performing an automated process of the present invention;
FIG. 3 is a schematic flow chart diagram illustrating a third embodiment of a method for performing an automated process according to the present invention;
FIG. 4 is a block diagram of a preferred embodiment of an apparatus for performing an automated process according to the present invention;
fig. 5 is a schematic structural diagram of a hardware operating environment according to an embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention provides an execution method of an automation process, and referring to fig. 1, fig. 1 is a schematic flow chart of a first embodiment of the execution method of the automation process of the invention.
While a logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in an order different than presented herein.
The execution method of the automation process is applied to a server or a terminal, and the terminal may include a mobile terminal such as a mobile phone, a tablet computer, a notebook computer, a palm computer, a Personal Digital Assistant (PDA), and the like, and a fixed terminal such as a Digital TV, a desktop computer, and the like. In the embodiments of the execution method of the automation flow, for convenience of description, the execution subject is omitted to illustrate the embodiments. The execution method of the automation process comprises the following steps:
and step S10, obtaining multi-modal data corresponding to the terminal, and taking the multi-modal data as the input of the deep learning model to obtain the intention analysis model.
And when an acquisition instruction for acquiring the multi-mode data corresponding to the terminal is detected, acquiring the multi-mode data corresponding to the terminal according to the acquisition instruction. The acquisition instruction can be set by a user according to needs, and can also be triggered by preset timing task timing. In this embodiment, multimodal data of one terminal may be acquired, or multimodal data corresponding to a plurality of terminals may be acquired. The multimodal data includes, but is not limited to, operation data corresponding to an input operation of inputting text, a mouse click operation of clicking a mouse, and a mouse movement operation of moving the mouse, which are triggered in the terminal by a terminal user. By means of the multi-modal data, it can be determined what function the operation instruction of the user is specifically to implement, namely, the behavior intention corresponding to the operation instruction is determined. In this embodiment, without limiting how many pieces of multi-modal data corresponding to the terminals are acquired and the acquisition duration of the multi-modal data, multi-modal data of the terminal in the last 10 days or multi-modal data generated by the terminal in the last 15 days may be acquired.
After the multi-modal data are obtained, the multi-modal data are used as the input of a preset deep learning model, namely the multi-modal data are used as training sample data of the deep learning model, and the deep learning model is trained through the training sample data to obtain an intention analysis model through training. The Deep learning model includes, but is not limited to, DBN (Deep Belief Network), CNN (convolutional Neural Network), and RNN (recurrent Neural Network). In the process of obtaining the intention analysis model, which deep learning model is specifically used can be preset by a user according to needs. It can be understood that, by using the multi-modal data as training sample data of the deep learning model, the obtained user intention analysis can analyze the user behavior intention of the operation instruction (input operation, mouse click operation, mouse movement operation, and the like) of each terminal.
Further, the step S10 includes:
step a, obtaining multi-modal data corresponding to the terminal and obtaining model sample data corresponding to the terminal.
Furthermore, in order to improve the accuracy of the analysis behavior intention of the obtained intention analysis model, after multi-modal data corresponding to the terminal is obtained, model sample data corresponding to the terminal is obtained. The model sample data may be screen capture data corresponding to the terminal, the screen capture data may be captured by a screen capture program preset in the background in the operation terminal for each terminal user, and in the screen capture process, the screen capture frequency may be set according to specific needs, for example, 1 frame of screen capture image may be acquired every second, or 1 frame of screen capture image may be acquired every 2 seconds, which is not limited in the embodiment. It is understood that at least two screen shots are included in the screen shot data. If the screen capture data corresponding to the plurality of terminals are acquired, each screen capture image in the screen capture data can carry the terminal identification and the screen capture time of the corresponding terminal, the terminal identification can be used for determining which terminal the screen capture image belongs to, and the screen capture time can be used for determining the generation time of the screen capture image. Further, the model sample data may also be screen data shot by each terminal user through a camera of the terminal during operation of the terminal.
And b, taking the multi-modal data and the model sample data as the input of a deep learning model to obtain an intention analysis model.
And after obtaining the model sample data and the multi-mode data, taking the multi-mode data and the model sample data as the input of the deep learning model, namely taking the multi-mode data and the model sample data as training sample data to train the deep learning model, and obtaining the intention analysis model. It can be understood that the change condition of the corresponding terminal screen interface can be recorded through the model sample data, so that the intention analysis model obtained by combining the multi-mode data and the model sample data training has higher accuracy when analyzing the behavior intention of the operation instruction. It can be understood that, when analyzing the user behavior intention corresponding to the operation instruction through the intention analysis model, only the operation instruction needs to be used as the input of the intention analysis model, and at this time, the output of the intention analysis model is the behavior intention corresponding to the operation instruction.
And step S20, determining the behavior intention corresponding to the automation process in the terminal through the intention analysis model.
And after the intention analysis model is obtained, determining the behavior intention corresponding to the automatic process in the terminal through the intention analysis model. The automatic process is a work process which is preset in the terminal and automatically executed by the terminal, and in the automatic process, the work process is composed of a series of operation instructions, and the operation number of the operation instructions in each automatic process can be equal or unequal. The action intention is the function to be realized by the automatic flow, such as opening a webpage or realizing a retrieval function. If a certain automation process comprises three operation instructions, the operation instruction A is executed firstly, then the operation instruction B is executed, and finally the operation instruction C is executed. The target operation instruction is each preset operation instruction in the automation process.
And step S30, executing a target operation instruction corresponding to the automation process according to the behavior intention so as to execute the automation process.
And after the behavior intention of the target operation instruction corresponding to the automatic flow is determined, executing the target operation instruction corresponding to the automatic flow according to the determined behavior intention so as to execute the automatic flow. It can be understood that the behavioral intention corresponding to the automated process can be identified through the intention analysis model, so that the situation that the automated process fails to be executed or is mistakenly executed due to the fact that the behavioral intention corresponding to the automated process is not considered when the automated process is executed according to the originally set operation instruction is avoided. If a certain automatic flow is to open a certain webpage, at this time, the corresponding operation instruction is an opening instruction, if an advertisement popup window is automatically popped up in the terminal at this time, the opening instruction may open an advertisement page corresponding to the advertisement popup window without opening the webpage in the automatic flow under the condition of no intention analysis model; the intention analysis model can analyze that the current operation aims at opening the webpage, if the advertisement popup window appears in the process of automatically opening the webpage through the opening instruction, the advertisement popup window can be ignored, the webpage is directly opened according to the opening instruction, or the advertisement popup window is firstly closed, and then the webpage is opened according to the opening instruction, so that the success rate of automatically opening the webpage is improved.
Further, the method for executing the automated process further includes:
and c, detecting whether non-process factors which do not belong to the automatic process are detected in the process of executing the automatic process in the terminal.
Further, after the automatic process which is currently required to be executed in the terminal is determined, whether non-process factors which do not belong to the automatic process are detected in the process of executing the automatic process is detected. It can be understood that the non-process factors may affect the execution of the automated process, such as the fluctuation of the execution environment corresponding to the automated process, for example, in the automated retrieval process for automatically implementing the retrieval function, the position of the search box and the retrieval content corresponding to the user retrieval instruction are preset. Under normal conditions, the retrieval function can be automatically realized by inputting retrieval contents at the position of the search box, but if the position of the search box changes due to the change of the execution environment of the terminal, the change of the position of the search box is a non-flow factor.
If the non-process factor is detected, the step of executing the target operation instruction corresponding to the automatic process according to the behavior intention so as to execute the automatic process comprises the following steps:
and d, executing a target operation instruction corresponding to the automatic flow according to the behavior intention and the non-flow factors so as to execute the automatic flow.
And if non-process factors which do not belong to the automatic process are detected in the process of executing the automatic process, executing a target operation instruction corresponding to the automatic process through the behavior intention and the non-process factors determined by the intention analysis model so as to successfully execute the automatic process. It can be understood that when a non-flow factor occurs in an automated flow, the target operation instruction of the automated flow needs to be adaptively changed according to the non-flow factor, so that the automated flow can be successfully executed. As explained in the following example, since the position of the search box is determined to have changed, during the execution of the automated search process, since the action is intended to implement the search function first, during the execution of the automated search process, the position of the search box in the terminal is determined first, and then the search content is output in the determined position of the search box, so that the automated search process is successfully executed, instead of directly inputting the search content in the original position of the search box. Further, if no non-process factor which does not belong to the automation process is detected in the process of executing the automation process, a target operation instruction corresponding to the automation process is directly executed. It is understood that the same automated process, corresponding to different non-process factors, may have different target operating instructions.
The embodiment obtains an intention analysis model by taking multi-modal data corresponding to the terminal as the input of the deep learning model, determines a behavior intention corresponding to the automatic process in the terminal through the intention analysis model, and executes a target operation instruction corresponding to the automatic process according to the behavior intention so as to execute the automatic process. The behavior intention corresponding to the automatic flow is analyzed through the intention analysis model, the automatic flow is executed according to the behavior intention, and the situation that the automatic flow fails to be executed due to the fact that the automatic flow corresponds to the execution environment is avoided, so that the adaptability of the automatic flow to different execution environments is improved, namely the adaptability of the automatic flow is improved, and the execution success rate of the automatic flow is improved.
Further, after determining the behavior intention corresponding to the automated process, in the process of executing the automated process, whether the auxiliary software needs to be started or not can be determined according to the behavior intention. And if the auxiliary software is determined to be required to be started according to the behavior intention, the auxiliary software is started simultaneously in the process of executing the automatic flow so as to improve the execution success rate of the automatic flow. For example, in the process of executing the automatic translation process, the translator can be started at the same time so as to improve the translation efficiency. Whether the specific behavior intent requires the auxiliary software to be started may be preset. And if the auxiliary software does not need to be started according to the behavior intention, directly executing the automatic process.
Furthermore, the embodiment does not need to independently develop corresponding solutions for various scenes of the execution environment, thereby reducing the development cost and improving the utilization rate of the storage space of the terminal.
Further, a second embodiment of the method for executing an automation process of the present invention is provided. The second embodiment of the method for executing an automated process differs from the first embodiment of the method for executing an automated process in that, with reference to fig. 2, the method for executing an automated process further includes:
and step S40, local use data of a user corresponding to the terminal are obtained, federal learning is carried out based on the local use data, the intention analysis model is updated according to the learning result of the federal learning, and the updated intention analysis model is obtained.
For ease of understanding, federal learning is explained first. Federal Learning (Federal Learning) is an emerging artificial intelligence basic technology, and the design goal of the technology is to guarantee the information security during big data exchange, protect terminal data and personal privacy data, and develop efficient machine Learning among multiple participating or computing nodes on the premise of legal compliance. In the system architecture of federal learning, the present embodiment describes the system architecture of federal learning by taking a scenario including two data owners (i.e., enterprise a and enterprise B) as an example. The framework is extensible to scenarios involving multiple data owners. Suppose that enterprise a and enterprise B want to jointly train a machine learning model, and their business systems have the relevant data of their respective users. In addition, enterprise B also has label data that the model needs to predict. For data privacy protection and security, enterprise a and enterprise B cannot directly exchange data, and a federal learning system can be used for establishing a model.
The method for establishing the model by using the federal learning comprises two parts of contents, wherein the first part is as follows: the encrypted samples are aligned. Because the user groups of the two enterprises are not completely overlapped, the system confirms the users shared by the two enterprises on the premise that the enterprise A and the enterprise B do not disclose respective data by using an encryption-based user sample alignment technology, and does not expose the users which are not overlapped with each other, so that the modeling is performed by combining the characteristics of the users. The second part is as follows: and (5) training an encryption model. After the common user population is determined, the machine learning model can be trained using these data. In order to ensure the confidentiality of data in the training process, the third-party collaborator C needs to be used for encryption training. Taking the linear regression model as an example, the training process can be divided into the following 4 steps:
at step ①, collaborator C distributes the public key to enterprise A and enterprise B for encrypting the data to be exchanged during the training process.
At step ②, the intermediate results used to compute the gradient are interacted with each other in encrypted form between Enterprise A and Enterprise B.
At ③, enterprise A and enterprise B calculate based on the encrypted gradient values, respectively, while enterprise B calculates the loss based on its tag data and summarizes the results to collaborator C.
And ④, the collaborator C respectively transmits the decrypted gradient values back to the enterprise A and the enterprise B, and the enterprise A and the enterprise B update the parameters of the respective models according to the gradient values.
And iterating the steps until the loss function corresponding to the linear regression model converges, wherein the loss function is a preset function, and thus the whole training process is completed. In the sample alignment and model training process, the data of the enterprise A and the enterprise B are kept locally, and data privacy is not leaked due to data interaction in the training process. Thus, both parties are enabled to collaboratively train the model with the help of federal learning.
For different data sets, federal Learning is divided into horizontal federal Learning (horizontal federal Learning), vertical federal Learning (vertical federal Learning), and federal transfer Learning (FmL).
In the case of more user feature overlap and less user overlap, the horizontal federal learning is to divide the data sets according to the horizontal direction (i.e. user dimension) and extract the part of data with the same user feature but not identical user for training. For example, if two banks in different regions exist, their user groups are respectively from the regions where they are located, and the intersection of the user groups is very small. However, their services are very similar and therefore the recorded user characteristics are the same. At this point, the federated model may be constructed using horizontal federated learning.
The longitudinal federated learning is to divide the data sets according to the longitudinal direction (namely feature dimension) under the condition that the users of the two data sets overlap more and the user features overlap less, and take out the part of data which is the same for both users and the user features are not completely the same for training. For example, if there are two different organizations, where one is a bank in one location and the other is an e-commerce in the same location, their user group will likely contain a large portion of the residents of that location and thus the intersection of users will be large. However, the bank records the user's income and expense behavior and credit rating, and the e-commerce maintains the user's browsing and purchasing history, so the intersection of the user characteristics is small. Longitudinal federal learning is to aggregate these different features in an encrypted state to enhance model capabilities. At present, numerous machine learning models such as logistic regression models, tree structure models and neural network models have been gradually proven to be capable of being established on the joint system.
Federal transfer learning is to overcome the situation of insufficient data or labels by using transfer learning without segmenting data under the condition that the overlap of users and user characteristics of two data sets is less. For example, there are two different institutions, one being a bank located in china and the other being an e-commerce located in the united states. Due to regional limitation, the user population intersection of the two organizations is very small. Meanwhile, due to the difference of mechanism types, the data characteristics of the two are only partially overlapped. Under the condition, migration learning must be introduced to solve the problems of small scale of unilateral data and few label samples so as to improve the effect of the model for effective federal learning.
It should be noted that, in the process of training the intention analysis model, daily behavior data of each terminal user is needed, and the daily behavior data is stored locally in the terminal held by each user and is used as local use data of the terminal corresponding to the user. In the local use data, part of the data belongs to the private data of the user, and if the authorization of the user does not exist, the private data of the user cannot be acquired, namely, the private data of the user cannot be used as training sample data to train the deep learning model to obtain the intention analysis model. Currently, part of applications in a terminal will usually force to insert terms that require a user to agree to use private data in advance into a user agreement, or obtain the private data of the user in other unknown ways, which obviously reduces the privacy of the private data of the user. And by adopting federal learning, the intention analysis model can be updated on the basis of not touching the original local use data of the user.
The method comprises the steps of obtaining local use data of a terminal corresponding to a user, wherein the local use data comprises but is not limited to multi-mode data, and relative to the multi-mode data, the local use data further comprises some privacy data of the user and some statistically analyzed use data, such as starting frequency of a certain application started by the user, use duration of the application used each time and the like. After the local use data of the user corresponding to the terminal is obtained, federal learning is carried out based on the local use data to obtain a learning result of the federal learning, and the intention analysis model is updated according to the learning record, so that the updated intention analysis model is obtained. It should be noted that the specific use of horizontal federal learning, vertical federal learning, or federal migratory learning is determined by the nature of the local usage data.
Further, the step S40 includes:
and e, acquiring local use data of a user corresponding to the terminal, performing federal learning based on the local use data, and acquiring a gradient value obtained by the federal learning.
And f, updating the model parameters in the intention analysis model according to the gradient values to obtain an updated intention analysis model.
Further, local use data of a user corresponding to the terminal is obtained. It can be understood from the above explanation of federal learning that, in order to ensure the privacy of the local use data of each terminal, the local use data of the user corresponding to each terminal is encrypted by the public key sent by the federal server (collaborator), that is, in the process of federal learning, the original local use data of the user corresponding to each terminal cannot be known after the adopted local use data are encrypted. In this embodiment, in order to perform federal learning, the present embodiment may be configured to at least obtain local usage data of users corresponding to the two terminals. After the federal server uses the local use data to carry out federal learning, a learning result of the federal learning can be obtained, the learning result is a gradient value corresponding to the federal learning, and at the moment, the federal server returns the gradient value to each terminal.
Specifically, the gradient value may be directly substituted for the model parameter in the intent analysis model, or weights between each model parameter and the gradient value in the intent analysis model may be set, then the model parameter is multiplied by the corresponding weight to obtain a first parameter, the gradient value is multiplied by the corresponding weight to obtain a second parameter, a sum of the first parameter and the second parameter is calculated, and the sum of the first parameter and the second parameter is determined as the updated model parameter of the intent analysis model.
Step S20 includes:
and step S21, determining the behavior intention corresponding to the automation process in the terminal through the updated intention analysis model.
And after the updated intention analysis model is obtained, determining a behavior intention corresponding to an automatic flow preset in the terminal according to the updated intention analysis model.
In the embodiment, after the intention analysis model is obtained, the local use data of the terminal users are obtained through federal learning, the limitation of data islands among terminals is broken through, the intention analysis model can be updated by combining the local use data of the terminal users, the defect caused by less training sample data of the intention analysis model is overcome on the basis of protecting the privacy of the local use data of the terminal users and the data ownership of a data owner (terminal), and the accuracy of intention analysis of the obtained intention analysis model is further improved.
Further, a third embodiment of the method for executing an automation process of the present invention is provided.
The third embodiment of the method for executing an automated process differs from the first and/or second embodiment of the method for executing an automated process in that, with reference to fig. 3, the method for executing an automated process further includes:
and step S50, acquiring interactive data of the terminal and the server, performing federated learning based on the interactive data, and updating the intention analysis model according to the learning result of the federated learning to obtain an updated intention analysis model.
Step S20 further includes:
and step S21, determining the behavior intention corresponding to the automation process in the terminal through the updated intention analysis model.
The method comprises the steps of obtaining interactive data of a terminal and a server, wherein the interactive data comprises effective operation data in the interactive process of the terminal and the server, such as search engine retrieval operation, webpage form entry operation and the like. The server in this embodiment may be a cloud server and a local server corresponding to the terminal. And after the interactive data are obtained, carrying out federal learning based on the interactive data, updating the intention analysis model according to a learning result of the federal learning to obtain an updated intention analysis model, and determining a behavior intention corresponding to a preset automatic flow in the terminal through the updated intention analysis model. It should be noted that the principle of federal learning based on interactive data is the same as the principle of federal learning based on local usage data of the end user, and will not be repeated herein.
It should be noted that, in the execution method of the automation process, federal learning can be performed only by using data locally to update the intention analysis model; federal learning can also be performed only through interactive data to update the intention analysis model; or may be learned by local usage data along with interaction data to update the intent analysis model.
According to the method and the device, the intention analysis model is updated through federal learning of the interactive data of the terminal and the server, so that the intention analysis model can be applied to various interactive scenes, the intention analysis model is updated through the interactive data under the condition that privacy of various data of the terminal is not affected, the intention analysis model can understand behavior intention of users in the interactive scenes, an automatic interactive process is executed through the intention analysis model, flexibility of the intention analysis model is improved, applicability of the automatic process is improved, execution efficiency and success rate of the automatic process corresponding to the interactive scenes are improved, and execution results of the automatic process in the terminal really meet expected operation results of developers.
Further, a fourth embodiment of the method for executing an automation process of the present invention is provided.
The fourth embodiment of the method for executing an automated process differs from the first, second and/or third embodiment of the method for executing an automated process in that the method for executing an automated process further comprises:
and j, detecting the execution success rate of the automatic process.
And h, if the execution success rate is less than the preset success rate, updating the intention analysis model.
After the automatic flows are executed, an execution success rate of the automatic flows in the terminal may be detected, specifically, the execution success rate is equal to the number of the automatic flows successfully executed within the preset time length divided by the number of all the automatic flows executed within the preset time length. After the execution success rate is obtained through calculation, it is determined whether the execution success rate is smaller than a preset success rate, where the size of the preset success rate may be set according to specific needs, and the size of the preset success rate is not specifically limited in this embodiment. When the execution success rate is determined to be smaller than the preset success rate, updating the intention analysis model, at the moment, re-acquiring multi-mode data corresponding to the terminal to update the intention analysis model, or locally updating the intention analysis model by using the data, interactive data and the like; if the execution success rate is determined to be greater than or equal to the preset success rate, the intention analysis model does not need to be updated.
According to the embodiment, the accuracy of analyzing the behavior intention by the intention analysis model is further improved by detecting the execution success rate of the automatic flow and updating the intention analysis model when the execution success rate is smaller than the preset success rate.
In addition, the present invention also provides an apparatus for executing an automation process, and referring to fig. 4, the apparatus for executing an automation process includes:
the system comprises an acquisition module 10, a target analysis module and a target analysis module, wherein the acquisition module is used for acquiring multi-modal data corresponding to a terminal and taking the multi-modal data as the input of a deep learning model to obtain an intention analysis model;
a determining module 20, configured to determine a behavior intention corresponding to an automation process in the terminal through the intention analysis model;
and the execution module 30 is configured to execute the target operation instruction corresponding to the automation process according to the behavior intention, so as to execute the automation process.
Further, the obtaining module 10 is further configured to obtain local usage data of a user corresponding to the terminal;
the execution device of the automatic process comprises:
a first federated learning module configured to perform federated learning based on the local usage data;
the first updating module is used for updating the intention analysis model according to the learning result of the federal learning to obtain an updated intention analysis model;
the determining module 20 is further configured to determine a behavior intention corresponding to an automated process in the terminal through the updated intention analysis model.
Further, the first federal learning module is further configured to perform federal learning based on the local use data, and obtain a gradient value obtained by federal learning;
the first updating module is further used for updating the model parameters in the intention analysis model according to the gradient values to obtain an updated intention analysis model.
Further, the execution device of the automation process comprises:
the second federated learning module is used for performing federated learning based on the interactive data;
the second updating module is used for updating the intention analysis model according to the learning result of the federal learning to obtain an updated intention analysis model;
the determining module 20 is further configured to determine a behavior intention corresponding to an automated process in the terminal through the updated intention analysis model.
Further, the execution device of the automation process comprises:
the first detection module is used for detecting whether non-process factors which do not belong to the automatic process are detected in the process of executing the automatic process in the terminal;
the execution module 30 is further configured to execute a target operation instruction corresponding to the automated process according to the behavior intention and the non-process factor if the non-process factor is detected, so as to execute the automated process.
Further, the obtaining module 10 is further configured to obtain model sample data corresponding to the terminal; and taking the multi-modal data and the model sample data as the input of a deep learning model to obtain an intention analysis model.
Further, the execution device of the automation process comprises:
the second detection module is used for detecting the execution success rate of the automatic process;
and the third updating module is used for updating the intention analysis model if the execution success rate is less than a preset success rate.
The specific implementation of the apparatus for executing an automation process of the present invention is substantially the same as the embodiments of the method for executing an automation process, and will not be described herein again.
In addition, the invention also provides an execution device of the automatic process. As shown in fig. 5, fig. 5 is a schematic structural diagram of a hardware operating environment according to an embodiment of the present invention.
It should be noted that fig. 5 is a schematic structural diagram of a hardware operating environment of an execution device of an automation process. The execution device of the automatic flow of the embodiment of the invention can be a terminal device such as a PC, a portable computer and the like.
As shown in fig. 5, the apparatus for executing the automated process may include: a processor 1001, such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the automated process performing apparatus configuration shown in FIG. 5 does not constitute a limitation on the automated process performing apparatus and may include more or fewer components than shown, or some components in combination, or a different arrangement of components.
As shown in fig. 5, the memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and an execution program of an automation flow. The operating system is a program for managing and controlling hardware and software resources of the execution device of the automation process, and supports the execution program of the automation process and the operation of other software or programs.
In the execution apparatus of the automation flow shown in fig. 5, the user interface 1003 is mainly used for connecting other terminals and performing data communication with the other terminals; the network interface 1004 is mainly used for the background server and performs data communication with the background server; the processor 1001 may be configured to call an execution program of the automation process stored in the memory 1005 and execute the steps of the execution method of the automation process as described above.
The specific implementation of the apparatus for executing an automation process of the present invention is substantially the same as the embodiments of the method for executing an automation process, and will not be described herein again.
Furthermore, an embodiment of the present invention further provides a computer-readable storage medium, where an execution program of an automation process is stored on the computer-readable storage medium, and when the execution program of the automation process is executed by a processor, the steps of the execution method of the automation process described above are implemented.
The specific implementation manner of the computer-readable storage medium of the present invention is substantially the same as that of each embodiment of the above-mentioned method for executing an automation process, and will not be described herein again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. An execution method of an automation process is characterized by comprising the following steps:
obtaining multi-modal data corresponding to a terminal, and taking the multi-modal data as the input of a deep learning model to obtain an intention analysis model;
determining a behavior intention corresponding to an automatic process in the terminal through the intention analysis model;
and executing a target operation instruction corresponding to the automatic flow according to the behavior intention so as to execute the automatic flow.
2. The method for executing an automation process according to claim 1, wherein the step of obtaining multi-modal data corresponding to the terminal and using the multi-modal data as an input of a deep learning model to obtain an intention analysis model further comprises:
acquiring local use data of a user corresponding to a terminal, performing federal learning based on the local use data, and updating the intention analysis model according to a learning result of the federal learning to obtain an updated intention analysis model;
the step of determining the behavior intention corresponding to the automatic process in the terminal through the intention analysis model comprises the following steps:
and determining the behavior intention corresponding to the automatic process in the terminal through the updated intention analysis model.
3. The method for executing an automation process according to claim 2, wherein the step of obtaining local usage data of a user corresponding to the terminal, performing federal learning based on the local usage data, updating the intention analysis model according to a learning result of the federal learning, and obtaining the updated intention analysis model includes:
the method comprises the steps of obtaining local use data of a terminal corresponding to a user, carrying out federal learning based on the local use data, and obtaining a gradient value obtained by the federal learning;
and updating the model parameters in the intention analysis model according to the gradient values to obtain an updated intention analysis model.
4. The method for executing an automation process according to claim 1, wherein the step of obtaining multi-modal data corresponding to the terminal and using the multi-modal data as an input of a deep learning model to obtain an intention analysis model further comprises:
acquiring interactive data of interaction between a terminal and a server, performing federal learning based on the interactive data, and updating the intention analysis model according to a learning result of the federal learning to obtain an updated intention analysis model;
the step of determining the behavior intention corresponding to the automatic process in the terminal through the intention analysis model comprises the following steps:
and determining the behavior intention corresponding to the automatic process in the terminal through the updated intention analysis model.
5. The method for executing the automated process according to claim 1, wherein after the step of determining the behavior intention corresponding to the automated process in the terminal through the intention analysis model, the method further comprises:
detecting whether non-process factors which do not belong to the automatic process are detected in the process of executing the automatic process in the terminal;
if the non-process factor is detected, the step of executing the target operation instruction corresponding to the automatic process according to the behavior intention so as to execute the automatic process comprises the following steps:
and executing a target operation instruction corresponding to the automatic flow according to the behavior intention and the non-flow factors so as to execute the automatic flow.
6. The method for executing the automated process according to claim 1, wherein the step of obtaining multi-modal data corresponding to the terminal and using the multi-modal data as an input of a deep learning model to obtain an intention analysis model comprises:
obtaining multi-modal data corresponding to a terminal and obtaining model sample data corresponding to the terminal;
and taking the multi-modal data and the model sample data as the input of a deep learning model to obtain an intention analysis model.
7. The method for executing the automated process according to any one of claims 1 to 6, wherein after the step of executing the target operation instruction corresponding to the automated process according to the behavioral intention to execute the automated process, the method further comprises:
detecting the execution success rate of the automatic flow;
and if the execution success rate is less than the preset success rate, updating the intention analysis model.
8. An apparatus for executing an automated process, the apparatus comprising:
the system comprises an acquisition module, a target module and a target module, wherein the acquisition module is used for acquiring multi-modal data corresponding to a terminal and taking the multi-modal data as the input of a deep learning model so as to obtain an intention analysis model;
the determining module is used for determining a behavior intention corresponding to an automatic process in the terminal through the intention analysis model;
and the execution module is used for executing the target operation instruction corresponding to the automatic flow according to the behavior intention so as to execute the automatic flow.
9. An automated process execution apparatus comprising a memory, a processor, and an automated process execution program stored on the memory and executable on the processor, wherein the automated process execution program, when executed by the processor, implements the steps of the automated process execution method recited in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon an execution program of an automated procedure, which when executed by a processor implements the steps of the execution method of an automated procedure according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010100850.4A CN111324440A (en) | 2020-02-17 | 2020-02-17 | Method, device and equipment for executing automation process and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010100850.4A CN111324440A (en) | 2020-02-17 | 2020-02-17 | Method, device and equipment for executing automation process and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111324440A true CN111324440A (en) | 2020-06-23 |
Family
ID=71165317
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010100850.4A Pending CN111324440A (en) | 2020-02-17 | 2020-02-17 | Method, device and equipment for executing automation process and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111324440A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112183982A (en) * | 2020-09-21 | 2021-01-05 | 中国建设银行股份有限公司 | Workflow creating method and device, computer equipment and storage medium |
| CN112199424A (en) * | 2020-09-07 | 2021-01-08 | 深圳市法本信息技术股份有限公司 | Method for pulling communication information island through RPA, readable storage medium and RPA robot |
| CN112256786A (en) * | 2020-12-21 | 2021-01-22 | 北京爱数智慧科技有限公司 | Multi-modal data processing method and device |
| CN112819443A (en) * | 2021-02-08 | 2021-05-18 | 上海交通大学 | Workflow-oriented execution method and system based on intelligent form robot |
| WO2022012129A1 (en) * | 2020-07-17 | 2022-01-20 | 华为技术有限公司 | Model processing method for cloud service system, and cloud service system |
| CN114800545A (en) * | 2022-01-18 | 2022-07-29 | 泉州华中科技大学智能制造研究院 | Robot control method based on federal learning |
| CN116719911A (en) * | 2023-08-10 | 2023-09-08 | 成都不烦智能科技有限责任公司 | Automatic flow generation method, device, equipment and storage medium |
-
2020
- 2020-02-17 CN CN202010100850.4A patent/CN111324440A/en active Pending
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022012129A1 (en) * | 2020-07-17 | 2022-01-20 | 华为技术有限公司 | Model processing method for cloud service system, and cloud service system |
| CN112199424A (en) * | 2020-09-07 | 2021-01-08 | 深圳市法本信息技术股份有限公司 | Method for pulling communication information island through RPA, readable storage medium and RPA robot |
| CN112183982A (en) * | 2020-09-21 | 2021-01-05 | 中国建设银行股份有限公司 | Workflow creating method and device, computer equipment and storage medium |
| CN112256786A (en) * | 2020-12-21 | 2021-01-22 | 北京爱数智慧科技有限公司 | Multi-modal data processing method and device |
| CN112256786B (en) * | 2020-12-21 | 2021-04-16 | 北京爱数智慧科技有限公司 | Multi-modal data processing method and device |
| CN112819443A (en) * | 2021-02-08 | 2021-05-18 | 上海交通大学 | Workflow-oriented execution method and system based on intelligent form robot |
| CN114800545A (en) * | 2022-01-18 | 2022-07-29 | 泉州华中科技大学智能制造研究院 | Robot control method based on federal learning |
| CN114800545B (en) * | 2022-01-18 | 2023-10-27 | 泉州华中科技大学智能制造研究院 | Robot control method based on federal learning |
| CN116719911A (en) * | 2023-08-10 | 2023-09-08 | 成都不烦智能科技有限责任公司 | Automatic flow generation method, device, equipment and storage medium |
| CN116719911B (en) * | 2023-08-10 | 2023-10-31 | 成都不烦智能科技有限责任公司 | Automatic flow generation method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111324440A (en) | Method, device and equipment for executing automation process and readable storage medium | |
| CN109460664B (en) | Risk analysis method and device, electronic equipment and computer readable medium | |
| CN109479061A (en) | Compliance violates detection | |
| Verkooij et al. | Mobile business intelligence: Key considerations for implementations projects | |
| CN110569377A (en) | Media file processing method and device | |
| CN103984626B (en) | A kind of method and device for generating test case script | |
| CN110909229A (en) | Webpage data acquisition and storage system based on simulated browser access | |
| Iqbal et al. | Advancing automation in digital forensic investigations using machine learning forensics | |
| CN105956087A (en) | Data and code version management system and method | |
| CN113627086A (en) | Method, apparatus, medium, and program product for optimizing horizontal federated learning modeling | |
| CN111259446B (en) | Parameter processing method, device and storage medium based on federal migration learning | |
| WO2022156084A1 (en) | Method for predicting behavior of target object on the basis of face and interactive text, and related device | |
| CN113627085A (en) | Method, apparatus, medium, and program product for optimizing horizontal federated learning modeling | |
| CN109947635A (en) | A kind of data reporting method, device, storage medium and terminal device | |
| US20220291966A1 (en) | Systems and methods for process mining using unsupervised learning and for automating orchestration of workflows | |
| Singh | Systematic review of data-centric approaches in artificial intelligence and machine learning | |
| CN109635993A (en) | Operation behavior monitoring method and device based on prediction model | |
| CN110443044A (en) | Block chain client bug excavation method, device, equipment and storage medium | |
| CN115544558A (en) | Sensitive information detection method and device, computer equipment and storage medium | |
| CN111459774A (en) | Method, device and equipment for acquiring flow of application program and storage medium | |
| US20220086183A1 (en) | Enhanced network security based on inter-application data flow diagrams | |
| Le et al. | Theoretical perspective of deep domain adaptation | |
| Bhatia et al. | CFRF: cloud forensic readiness framework–A dependable framework for forensic readiness in cloud computing environment | |
| Alrammal et al. | Using heuristic approach to build Anti-malware | |
| US20240144198A1 (en) | Machine Learning-based Knowledge Management for Incident Response |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |