CN116974785A - Data acquisition method, device, equipment and storage medium - Google Patents
Data acquisition method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN116974785A CN116974785A CN202310914146.6A CN202310914146A CN116974785A CN 116974785 A CN116974785 A CN 116974785A CN 202310914146 A CN202310914146 A CN 202310914146A CN 116974785 A CN116974785 A CN 116974785A
- Authority
- CN
- China
- Prior art keywords
- data acquisition
- data
- proxy service
- task
- response information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 230000004044 response Effects 0.000 claims abstract description 47
- 230000000977 initiatory effect Effects 0.000 claims abstract description 21
- 238000012546 transfer Methods 0.000 claims description 36
- 238000012216 screening Methods 0.000 claims description 23
- 230000000903 blocking effect Effects 0.000 claims description 21
- 238000013480 data collection Methods 0.000 claims description 14
- 230000002159 abnormal effect Effects 0.000 claims description 11
- 239000002574 poison Substances 0.000 claims description 7
- 231100000614 poison Toxicity 0.000 claims description 7
- 238000009877 rendering Methods 0.000 claims description 7
- 238000011084 recovery Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 description 13
- 230000009466 transformation Effects 0.000 description 12
- 230000004048 modification Effects 0.000 description 10
- 238000012986 modification Methods 0.000 description 10
- 238000012545 processing Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 239000003795 chemical substances by application Substances 0.000 description 4
- 238000004801 process automation Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 238000013515 script Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 239000003999 initiator Substances 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 239000000243 solution Substances 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013501 data transformation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000007943 implant Substances 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 239000003826 tablet Substances 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/547—Remote procedure calls [RPC]; Web services
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention belongs to the field of computers and discloses a data acquisition method, a device, equipment and a storage medium. The method comprises the following steps: when a data acquisition task is received, starting a back-end proxy service, and initiating a back-end data acquisition request, wherein the back-end proxy service is a newly added proxy service in a robot flow automation program; receiving response information corresponding to the back-end data acquisition request through the back-end proxy service; and extracting back-end data from the response information according to the data acquisition task. The response information corresponding to the back-end data acquisition request is received through the back-end proxy service; and extracting back-end data from the response information according to the data acquisition task. Compared with the mode that the existing robot flow automation program can only acquire data from the user interface, the mode can acquire the back-end data through the back-end proxy service, and the data which cannot be acquired from the user interface is acquired.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a data acquisition method, apparatus, device, and storage medium.
Background
The existing RPA technology adopts a data acquisition mode that data is directly acquired from the UI. Similar personnel directly read the data, two major methods of OCR character recognition or page structured data analysis are used, but the data form finally collected is structured text or picture. There is no direct acquisition approach to other forms of information (e.g., styles, scripts, raw json data, etc.). And sometimes for some unstructured non-text front-end UI built-in data (such as acquiring coordinate points on a map, time series data in a stock price graph, etc.), the current RPA technology does not cope with policies. Therefore, how to acquire back-end data that cannot be acquired on a user interface by using RPA technology is a technical problem to be solved.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
The invention mainly aims to provide a data acquisition method, a device, equipment and a storage medium, and aims to solve the technical problem that in the prior art, a robot flow automation program can only acquire user interface data and cannot acquire back-end data.
In order to achieve the above object, the present invention provides a data acquisition method, which includes the steps of:
when a data acquisition task is received, starting a back-end proxy service, and initiating a back-end data acquisition request, wherein the back-end proxy service is a newly added proxy service in a robot flow automation program;
receiving response information corresponding to the back-end data acquisition request through the back-end proxy service;
and extracting back-end data from the response information according to the data acquisition task.
Optionally, the step of starting the back-end proxy service and initiating the back-end data acquisition request when the data acquisition task is received includes:
when a data acquisition task is received, initializing a transfer Actor, and actively entering a blocking state;
creating and starting a back-end proxy service through the transfer Actor;
and after the back-end proxy service is successfully started, exiting the blocking state, and initiating a back-end data acquisition request.
Optionally, the step of starting the back-end proxy service and initiating the back-end data acquisition request when the data acquisition task is received includes:
when receiving a data acquisition task, starting a back-end proxy service;
and initiating a back-end data acquisition request through a robot flow automatic sub-thread.
Optionally, after the step of extracting the back-end data from the response information according to the data acquisition task, the method further includes:
determining a callback strategy according to the data acquisition task;
modifying the back-end data according to the callback strategy to obtain target data;
and performing data rendering on the user interface based on the target data.
Optionally, after the step of starting the back-end proxy service and initiating the back-end data acquisition request when the data acquisition task is received, the method further includes:
when the back-end data acquisition request is sent, entering a blocking state and waiting for the back-end proxy service to send back-end data;
starting a preset timer to start timing;
and throwing out abnormal information when the timing duration reaches the preset waiting duration and the back-end data is not received yet.
Optionally, after the step of starting the preset timer to start timing, the method further includes:
and when the back-end data is received and the timing time does not reach the preset waiting time, sending preset poison information to a transfer Actor, and when the transfer Actor receives the preset poison information, performing self-destruction and memory garbage recovery.
Optionally, the step of extracting backend data from the response information according to the data collection task includes:
determining data screening conditions according to the data acquisition task;
and extracting back-end data from the response information according to the data screening conditions.
In addition, to achieve the above object, the present invention also provides a data acquisition device, including:
the receiving module is used for starting a back-end proxy service and initiating a back-end data acquisition request when receiving a data acquisition task, wherein the back-end proxy service is newly added proxy service in a robot flow automation program;
the acquisition module is used for receiving response information corresponding to the back-end data acquisition request through the back-end proxy service;
and the extraction module is used for extracting the back-end data from the response information according to the data acquisition task.
In addition, to achieve the above object, the present invention also proposes a data acquisition device, the device comprising: a memory, a processor and a data acquisition program stored on the memory and executable on the processor, the data acquisition program configured to implement the steps of the data acquisition method as described above.
In addition, in order to achieve the above object, the present invention also proposes a storage medium having stored thereon a data acquisition program which, when executed by a processor, implements the steps of the data acquisition method as described above.
When a data acquisition task is received, a back-end proxy service is started, and a back-end data acquisition request is initiated, wherein the back-end proxy service is newly added proxy service in a robot flow automation program; receiving response information corresponding to the back-end data acquisition request through the back-end proxy service; and extracting back-end data from the response information according to the data acquisition task. The response information corresponding to the back-end data acquisition request is received through the back-end proxy service; and extracting back-end data from the response information according to the data acquisition task. Compared with the mode that the existing robot flow automation program can only acquire data from the user interface, the mode can acquire the back-end data through the back-end proxy service, and the data which cannot be acquired from the user interface is acquired.
Drawings
FIG. 1 is a schematic diagram of a data acquisition device of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart of a first embodiment of a data acquisition method according to the present invention;
FIG. 3 is a schematic diagram of a first embodiment of a data acquisition method according to the present invention;
FIG. 4 is a flow chart of a second embodiment of the data acquisition method of the present invention;
FIG. 5 is a flowchart of a third embodiment of a data acquisition method according to the present invention;
fig. 6 is a block diagram of a first embodiment of a data acquisition device according to the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a data acquisition device in a hardware operating environment according to an embodiment of the present invention.
As shown in fig. 1, the data acquisition device may include: a processor 1001, such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a Wireless interface (e.g., a Wireless-Fidelity (WI-FI) interface). The Memory 1005 may be a high-speed random access Memory (Random Access Memory, RAM) or a stable nonvolatile Memory (NVM), such as a disk Memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the structure shown in fig. 1 does not constitute a limitation of the data acquisition device and may include more or fewer components than shown, or may combine certain components, or may be arranged in different components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and a data collection program may be included in the memory 1005 as one type of storage medium.
In the data acquisition device shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the data acquisition device of the present invention may be disposed in the data acquisition device, where the data acquisition device invokes a data acquisition program stored in the memory 1005 through the processor 1001, and executes the data acquisition method provided by the embodiment of the present invention.
Based on the above data acquisition device, an embodiment of the present invention provides a data acquisition method, and referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the data acquisition method of the present invention.
In this embodiment, the data acquisition method includes the following steps:
step S10: and when receiving the data acquisition task, starting a back-end proxy service, and initiating a back-end data acquisition request, wherein the back-end proxy service is a newly added proxy service in the robot flow automation program.
It should be noted that, the execution body of the embodiment may be a computing service device with functions of data processing, network communication and program running, such as a mobile phone, a tablet computer, a personal computer, or an electronic device or a robot flow automation device capable of implementing the above functions. The present embodiment and the following embodiments will be described below by taking the above-described robot flow automation device as an example.
It should be noted that the data collection task may be a task for collecting specific backend data in a target program or an application, and includes information such as key feature information of data to be collected, callback operation after the data is collected, and the like. The back-end proxy service may be a proxy service thread newly created by the robotic flow automation device for capturing back-end data. The initiation back-end data acquisition request can be that the robot flow automation equipment initiates a network request through a robot flow automation main thread or through a network interface monitored by an agent by newly adding a robot flow automation sub-thread driving target program or an application user interface.
It should be appreciated that robotic process automation (Robotic Process Automation, RPA) can simulate human operation on a mouse-keyboard on a computer, can perform automated office work like a human, operate a computer program, and perform a process or a series of tasks automatically according to rules. In conventional RPA products, data collection is typically achieved by using parsing structured data or OCR electronic recognition, the source of which is still limited to the information presented on the user interface. For underlying data that the page does not exhibit, or is indirectly used to derive page data, there is currently no way for conventional RPA to extract directly. And traditional workflow-based RPA takes activity as a running unit, and logic execution inside the activity is generally performed in a single thread. The Actor model is a concurrent execution scheduling system, and can be used for encapsulating single-threaded programs by using an Actor. Because only the unchangeable information is transferred and shared between the two actors and the internal state cannot be changed mutually, a plurality of actors can be executed in parallel through message transfer without locks and are not in conflict with each other. The Actor system is used as a parallel architecture and has higher stability and maintainability.
In specific implementation, the user may preset key feature information of the back-end data to be acquired, for example, may match a wild card in call url, a part of url, or respond to a file content type, etc., use the key feature information as a screening condition capable of distinguishing other data calls or network requests, and assign the screening condition and a callback transformation operation to be executed subsequently to the robot flow automation main thread through RPA component attribute configuration. The configuration setting process may be performed within a UI box provided by the component. The callback transformation operation can be defined by a lambda function described by codes, and can also be preset common default transformation, such as replacing a server response result with a local file, returning different http state codes, modifying returned content types and the like. The specific steps can be as follows: a user firstly observes a network call mode of a target application in a mode of local data monitoring, a browser developer tool and the like, finds a network call request needing to be monitored or modified, and extracts key feature information of the call (such as a wild card in call url, a part of url or a response file content type and the like can be matched to be used as a screening condition capable of distinguishing other calls. And then the user endows the screening conditions and the callback transformation operation required to be executed subsequently to a robot flow automatic main thread through the RPA component attribute configuration, and the main thread can screen response information corresponding to a back-end data acquisition request acquired by a back-end proxy service according to the screening conditions to acquire target data and can trigger the rendering of a possible UI interface according to the callback transformation operation.
Further, in order to improve the efficiency of data collection, after the step S10, the method further includes: when the back-end data acquisition request is sent, entering a blocking state and waiting for the back-end proxy service to send back-end data;
starting a preset timer to start timing;
and throwing out abnormal information when the timing duration reaches the preset waiting duration and the back-end data is not received yet.
It should be noted that, after the main thread of the robot process automation device initiates the back-end data acquisition request through the driving user interface, if no response information sent by the back-end proxy service is received, the main thread actively enters a blocking state, and waits for the back-end proxy service to send back-end data acquired by the back-end proxy service. And the concurrent information informs the transfer Actor that the internal flow is executed until a return result is received. And if the back-end data is not received after the preset waiting time, throwing out the abnormal information. Meanwhile, when the back-end proxy service intercepts target data meeting screening conditions, the back-end proxy service sends the target data to a main thread of the robot flow automation equipment. If the data acquisition task comprises callback transformation operation, the target data can be modified through data modification information in the callback transformation operation, the modified data is sent to a target application, and user interface rendering on the target application is triggered. Modifying the target data may include: replacing a javascript script in the target data with preset document content to realize direct modification of the target application front-end logic; or setting the data request with the modification return type json and xml to realize the rapid loading and modification of complex data. The callback transformation operation may be defined by a lambda equation provided by the user from the RPA component layer. After the back-end proxy service makes the corresponding callback transformation, the modified information and the target data are recorded and packaged, and meanwhile, the modified information and the target data are packaged into a message and sent to a transfer Actor in addition to the normal transmission of the information to the target application which initiates the request. And the transfer Actor receives the message and forwards the message to the main thread of the robot flow automation equipment. At this time, if the main thread does not notify the transfer Actor that it has entered the blocking state, the transfer Actor will cache the received message. And the transfer Actor will not transfer the received message to the RPA main thread and release the blocking until receiving the information of the blocking of the UI operation ending sent by the main thread. After the main thread takes the message, the main thread can send poison information to tell the transfer Actor and the proxy service Actor to self-destroy, exit the proxy program and recover the memory garbage. The request return information may be in the form of variables that are maintained by the proxy component for use by subsequent RPA flows and components. If any error occurs in the middle or the whole process is overtime, the Actor system can perform internal asynchronous recording processing, a normal Actor system exit and memory recovery mechanism is started, and finally, the transfer Actor packages error information to the RPA main thread to prompt the RPA main thread to exit from a blocking state. And the main thread component throws out corresponding exception according to the exception handling method and executes the RPA exception handling flow defined by the user.
Step S20: and receiving response information corresponding to the back-end data acquisition request through the back-end proxy service.
It should be noted that, the receiving, by the back-end proxy service, the response information corresponding to the back-end data acquisition request may be intercepting, by the back-end proxy service, the data information sent to the target program by the external resource server.
Step S30: and extracting back-end data from the response information according to the data acquisition task.
It should be noted that, the extracting the back-end data from the response information according to the data acquisition task may be determining a data screening condition according to the data acquisition task, and performing data screening on the response information according to the data screening condition to obtain the back-end data.
In specific implementation, reference may be made to fig. 3, and fig. 3 is a schematic structural diagram of a first embodiment of the data acquisition method according to the present invention; the browser and desktop application shown in fig. 3 may be target programs, and the robotic process automation device mainly includes a back-end proxy service thread and an RPA main thread, corresponding to the local proxy service and the RPA thread in fig. 3, where the RPA thread may drive the UI in a non-invasive manner to trigger the browser and the desktop application to initiate a network request. The local proxy service and the RPA thread realize thread isolation through the information transfer Actor. The local proxy service may intercept response information corresponding to the network request sent by the external resource server to the browser and/or the desktop application, and then send the intercepted response information to the RPA through the message transfer Actor. And the RPA extracts back-end data from the response information according to the data screening conditions in the data acquisition task. Or the local proxy service screens the back-end data meeting the data screening condition after intercepting the response information, and then sends the back-end data to the RPA through the message transfer Actor.
When receiving a data acquisition task, the embodiment starts a back-end proxy service and initiates a back-end data acquisition request, wherein the back-end proxy service is newly added proxy service in a robot flow automation program; receiving response information corresponding to the back-end data acquisition request through the back-end proxy service; and extracting back-end data from the response information according to the data acquisition task. Because the response information corresponding to the back-end data acquisition request is received through the back-end proxy service; and extracting back-end data from the response information according to the data acquisition task. Compared with the mode that the existing robot flow automation program can only acquire data from the user interface, the mode of the embodiment can acquire the back-end data through the back-end proxy service, and the data which cannot be acquired from the user interface is realized.
The embodiment can supplement the short boards for the RPA flow to acquire the network bottom layer information. Meanwhile, a multithreading scheduling mode of an Actor model is utilized, and a control information synchronization state machine is introduced into the RPA process. On the premise of not influencing the real-time interaction of the RPA main thread to the UI, the lock-free parallel data interaction of the bottom agent and the RPA flow in one process is realized. And finally, inserting necessary data transformation into the data intercepted by the proxy layer, and providing the data to the RPA proxy component in an asynchronous mode for subsequent flow. And the proxy layer can modify the request and the reply according to the rule defined by the RPA component layer, and the upper network application (such as a BS browser or a CS client) is transparent and insensitive to the related operation in the whole process, so that any processing and modification are not needed.
Referring to fig. 4, fig. 4 is a flowchart of a second embodiment of the data acquisition method according to the present invention.
Based on the first embodiment, in this embodiment, the step S20 includes:
step S101: and initializing a transfer initiator when a data acquisition task is received, and actively entering a blocking state.
In specific implementation, when the robot flow automation device receives a data acquisition task, an RPA main thread in the robot flow automation device starts the Actor system information first, initializes a transfer Actor, waits for the transfer Actor to further initialize a network proxy instance and transmits screening condition information in the data acquisition task to the proxy instance. At this point the main thread is actively blocking and waiting for the transfer Actor to reply to the proxy service to start up normally and ready to receive the network request generated by the UI operation. The proxy instance is the back-end proxy service.
Step S102: and creating and starting the back-end proxy service through the transfer Actor.
In specific implementation, the information transfer Actor enters a preparation working state after being initialized, a local proxy service Actor is created, and screening condition information and/or callback transformation operation in a data acquisition task are transmitted to the proxy service Actor for registration. The proxy service Actor starts the proxy service in a form of dependent injection according to the screening conditions. The agent takes screening conditions such as url, content type and the like as a trigger, when a request is successfully returned, and the response value meets the screening conditions, the agent executes the information callback hook transformation logic, and transmits the request information to the Actor system. The proxy here is simply a back-end program with no graphical user interface.
After the proxy service is successfully started and registers the screening condition and callback transformation operation, the proxy service Actor sends successful information to the information transfer Actor. And the transfer Actor switches the behavior, enters a formal working state, and transmits an information notification to the RPA main thread to complete the proxy binding process. If the proxy binding process fails, the proxy service Actor will send failure information to the transfer Actor, which will start the system closing and recovering mechanism and throw abnormal information to the main thread to remind the user to make debugging modification.
Step S103: and after the back-end proxy service is successfully started, exiting the blocking state, and initiating a back-end data acquisition request.
In specific implementation, after receiving a message that the back-end proxy service is successfully started, a main thread in the robot flow automation equipment exits from a blocking state, and initiates a pre-nested business operation from the UI, namely, triggers a data acquisition request. Where the corresponding network request operation will drive the object to initiate the corresponding network request.
When receiving a data acquisition task, the embodiment initializes a transfer initiator and actively enters a blocking state; creating and starting a back-end proxy service through the transfer Actor; and after the back-end proxy service is successfully started, exiting the blocking state, and initiating a back-end data acquisition request. When receiving a data acquisition task, the embodiment creates and starts a back-end proxy service through a transfer Actor; and a back-end data acquisition request is initiated, and the back-end data is acquired through a back-end proxy service, so that the back-end data which cannot be acquired from a user interface by the RPA can be obtained.
Referring to fig. 5, fig. 5 is a flowchart of a third embodiment of the data acquisition method according to the present invention.
Based on the above embodiments, in this embodiment, after step S30, the method further includes:
step S40: and determining a callback strategy according to the data acquisition task.
It should be noted that, the callback policy may be information preset by the user to modify the UI interface or the backend data, for example, the user defines some processing rules and callback logic at the RPA component layer, and implants the processing rules and callback logic at the proxy layer to implement modification to the network request. If the code module can be set to replace the javascript script request on the webpage with the local document content, the modification of the target website/application front-end logic can be directly realized. And a modification rule can be set, and the data request with json and xml return types is changed, so that the complex data rapid loading and modification can be directly realized. On the premise of not changing the target program service end code, the control capability of the RPA developer on the front end of the application is greatly enriched.
Step S50: and modifying the back-end data according to the callback strategy to obtain target data.
It should be noted that, the modifying the backend data according to the callback policy may be modifying the backend data according to information in the callback policy to obtain modified target data. When the back-end data is abnormal, the callback strategy is added to display the abnormal information on the user interface, so that the main thread can acquire the abnormal information of the back-end service from the user interface. When the data reading and writing or the data acquisition is completed, the information of the completion of the task displayed on the front-end interface is added by modifying the back-end data, so that the robot flow automation equipment can capture the back-end information of the completion of the data reading and writing or the data acquisition.
Step S60: and performing data rendering on the user interface based on the target data.
It should be noted that, the rendering of the data on the user interface based on the target data may be that the modified target data is displayed on the user interface, so that the RPA main thread can obtain the abnormal information of the back-end service from the user interface.
The synchronous control of the workflow by the existing RPA technology depends on the time delay of the component and the detection of the UI interface element. For example, when an application driven by the RPA robot reads and writes the back-end database, the RPA flow can only rely on actively waiting for a fixed time or detecting the change of the UI of the target application to detect whether the read-write operation is completed or not. If the target application does not change the interface element when the read and write operations are completed, the conventional RPA technique has no way to know that the backend service is completed. Let alone when an exception occurs in the back-end request, a specific exception is obtained and a different exception handling operation is taken. Anomaly tracking information is not typically displayed directly on the ui unless the front end of the target application is not well developed or does not place importance on security and user experience. The conventional RPA information acquisition component has no way to acquire anomaly information for the backend service from the front-end interface. The embodiment can overcome the defects of the prior RPA technology, can monitor the back-end request initiated by the front end of the target application, and enables the back-end information flow to be completely transparent to the proxy process. Through proxy callback, when callback return information meets the conditions set by a user, the end of a request can be accurately perceived, the bottom abnormal information returned by the callback can be captured, and the control capability of the RPA program on the service flow is enhanced.
Referring to fig. 6, fig. 6 is a block diagram illustrating a first embodiment of a data acquisition device according to the present invention.
As shown in fig. 6, a data acquisition device according to an embodiment of the present invention includes:
the receiving module 10 is configured to start a back-end proxy service when receiving a data acquisition task, and initiate a back-end data acquisition request, where the back-end proxy service is a newly added proxy service in a robot flow automation program;
the acquisition module 20 is configured to receive response information corresponding to the back-end data acquisition request through the back-end proxy service;
and the extracting module 30 is used for extracting the back-end data from the response information according to the data acquisition task.
When receiving a data acquisition task, the embodiment starts a back-end proxy service and initiates a back-end data acquisition request, wherein the back-end proxy service is newly added proxy service in a robot flow automation program; receiving response information corresponding to the back-end data acquisition request through the back-end proxy service; and extracting back-end data from the response information according to the data acquisition task. Because the response information corresponding to the back-end data acquisition request is received through the back-end proxy service; and extracting back-end data from the response information according to the data acquisition task. Compared with the mode that the existing robot flow automation program can only acquire data from the user interface, the mode of the embodiment can acquire the back-end data through the back-end proxy service, and the data which cannot be acquired from the user interface is realized.
It should be noted that the above-described working procedure is merely illustrative, and does not limit the scope of the present invention, and in practical application, a person skilled in the art may select part or all of them according to actual needs to achieve the purpose of the embodiment, which is not limited herein.
In addition, technical details not described in detail in this embodiment may refer to the data acquisition method provided in any embodiment of the present invention, and are not described herein.
Based on the first embodiment of the data acquisition device of the present invention, a second embodiment of the data acquisition device of the present invention is provided.
In this embodiment, the receiving module 10 is further configured to initialize a transitional Actor and actively enter a blocking state when receiving a data acquisition task;
creating and starting a back-end proxy service through the transfer Actor;
and after the back-end proxy service is successfully started, exiting the blocking state, and initiating a back-end data acquisition request.
Further, the receiving module 10 is further configured to start a backend proxy service when receiving a data acquisition task;
and initiating a back-end data acquisition request through a robot flow automatic sub-thread.
Further, the extracting module 30 is further configured to determine a callback policy according to the data collection task;
modifying the back-end data according to the callback strategy to obtain target data;
and performing data rendering on the user interface based on the target data.
Further, the receiving module 10 is further configured to enter a blocking state when the sending of the back-end data acquisition request is completed, and wait for the back-end proxy service to send back-end data;
starting a preset timer to start timing;
and throwing out abnormal information when the timing duration reaches the preset waiting duration and the back-end data is not received yet.
Further, the receiving module 10 is further configured to send preset poison information to a transfer Actor when the back-end data is received and the timing duration does not reach the preset waiting duration, and when the transfer Actor receives the preset poison information, perform self-destruction and memory garbage collection.
Further, the extracting module 30 is further configured to determine a data screening condition according to the data acquisition task;
and extracting back-end data from the response information according to the data screening conditions.
Other embodiments or specific implementation manners of the data acquisition device of the present invention may refer to the above method embodiments, and are not described herein again.
In addition, the embodiment of the invention also provides a storage medium, wherein the storage medium is stored with a data acquisition program, and the data acquisition program realizes the steps of the data acquisition method when being executed by a processor.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. read-only memory/random-access memory, magnetic disk, optical disk), comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (10)
1. The data acquisition method is characterized by comprising the following steps of:
when a data acquisition task is received, starting a back-end proxy service, and initiating a back-end data acquisition request, wherein the back-end proxy service is a newly added proxy service in a robot flow automation program;
receiving response information corresponding to the back-end data acquisition request through the back-end proxy service;
and extracting back-end data from the response information according to the data acquisition task.
2. The data collection method according to claim 1, wherein the step of starting a back-end proxy service and initiating a back-end data collection request upon receiving a data collection task comprises:
when a data acquisition task is received, initializing a transfer Actor, and actively entering a blocking state;
creating and starting a back-end proxy service through the transfer Actor;
and after the back-end proxy service is successfully started, exiting the blocking state, and initiating a back-end data acquisition request.
3. The data collection method according to claim 1, wherein the step of starting a back-end proxy service and initiating a back-end data collection request upon receiving a data collection task comprises:
when receiving a data acquisition task, starting a back-end proxy service;
and initiating a back-end data acquisition request through a robot flow automatic sub-thread.
4. The data acquisition method according to claim 1, further comprising, after the step of extracting back-end data from the response information according to the data acquisition task:
determining a callback strategy according to the data acquisition task;
modifying the back-end data according to the callback strategy to obtain target data;
and performing data rendering on the user interface based on the target data.
5. The data collection method according to any one of claims 1 to 4, wherein after the step of starting the back-end proxy service and initiating the back-end data collection request when the data collection task is received, the method further comprises:
when the back-end data acquisition request is sent, entering a blocking state and waiting for the back-end proxy service to send back-end data;
starting a preset timer to start timing;
and throwing out abnormal information when the timing duration reaches the preset waiting duration and the back-end data is not received yet.
6. The data acquisition method of claim 5, wherein after the step of starting the preset timer to begin timing, further comprising:
and when the back-end data is received and the timing time does not reach the preset waiting time, sending preset poison information to a transfer Actor, and when the transfer Actor receives the preset poison information, performing self-destruction and memory garbage recovery.
7. The data acquisition method according to any one of claims 1 to 4, wherein the step of extracting back-end data from the response information according to the data acquisition task includes:
determining data screening conditions according to the data acquisition task;
and extracting back-end data from the response information according to the data screening conditions.
8. A data acquisition device, the data acquisition device comprising:
the receiving module is used for starting a back-end proxy service and initiating a back-end data acquisition request when receiving a data acquisition task, wherein the back-end proxy service is newly added proxy service in a robot flow automation program;
the acquisition module is used for receiving response information corresponding to the back-end data acquisition request through the back-end proxy service;
and the extraction module is used for extracting the back-end data from the response information according to the data acquisition task.
9. A data acquisition device, the device comprising: a memory, a processor and a data acquisition program stored on the memory and executable on the processor, the data acquisition program being configured to implement the steps of the data acquisition method of any one of claims 1 to 7.
10. A storage medium having stored thereon a data acquisition program which, when executed by a processor, implements the steps of the data acquisition method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310914146.6A CN116974785A (en) | 2023-07-24 | 2023-07-24 | Data acquisition method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310914146.6A CN116974785A (en) | 2023-07-24 | 2023-07-24 | Data acquisition method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116974785A true CN116974785A (en) | 2023-10-31 |
Family
ID=88472472
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310914146.6A Pending CN116974785A (en) | 2023-07-24 | 2023-07-24 | Data acquisition method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116974785A (en) |
-
2023
- 2023-07-24 CN CN202310914146.6A patent/CN116974785A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3215900B1 (en) | Robotic process automation | |
| US10838838B2 (en) | Method and apparatus for dealing with abnormality of application program and storage medium | |
| US10852898B2 (en) | System for interactive help | |
| US11734104B2 (en) | Screen response validation of robot execution for robotic process automation | |
| CN111291384B (en) | Vulnerability scanning method and device and electronic equipment | |
| WO2018188380A1 (en) | Method and device for adding control identification | |
| EP3905134B1 (en) | Active element detection | |
| US20150169152A1 (en) | Automatically recovering and maintaining focus | |
| CN112379993A (en) | Robot process automation processing system, method and device | |
| US20220044050A1 (en) | Text detection, caret tracking, and active element detection | |
| CN116974785A (en) | Data acquisition method, device, equipment and storage medium | |
| CN104471531A (en) | Capturing an application state in a conversation | |
| CN110032503A (en) | Data processing system, method, equipment and device based on UI automation and OCR | |
| EP3905131A1 (en) | Text detection, caret tracking, and active element detection | |
| CN112612469A (en) | Interface element processing method and device and electronic equipment | |
| CN111414525A (en) | Data acquisition method and device for small program, computer equipment and storage medium | |
| US11977435B2 (en) | Access method, communication system, and non-transitory computer readable memory | |
| CN112839100B (en) | Method, equipment and storage medium for suspending http requests in batches | |
| EP3905038A1 (en) | Screen response validation of robot execution for robotic process automation | |
| US20240012896A1 (en) | System and method for classifying human and non-human traffice using behaviour authenticity | |
| EP3905132A1 (en) | Text detection and active element detection | |
| CN117555662A (en) | Page jump method and device between application programs | |
| CN115904899A (en) | Operation record generation method, operation record acquisition method, operation record generation device, operation record acquisition device and operation record acquisition medium | |
| CN115599637A (en) | Memory overflow detection method, device, system, storage medium and computer equipment | |
| CN117828158A (en) | Webpage data acquisition device and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |