CN116302621A - Method and system for picking up information-created environment elements based on QT interface library - Google Patents
Method and system for picking up information-created environment elements based on QT interface library Download PDFInfo
- Publication number
- CN116302621A CN116302621A CN202310569094.3A CN202310569094A CN116302621A CN 116302621 A CN116302621 A CN 116302621A CN 202310569094 A CN202310569094 A CN 202310569094A CN 116302621 A CN116302621 A CN 116302621A
- Authority
- CN
- China
- Prior art keywords
- mouse
- information
- rpa
- interface control
- interface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 155
- 230000008569 process Effects 0.000 claims abstract description 106
- 238000001514 detection method Methods 0.000 claims abstract description 36
- 238000002347 injection Methods 0.000 claims abstract description 24
- 239000007924 injection Substances 0.000 claims abstract description 24
- 230000007246 mechanism Effects 0.000 claims abstract description 24
- 238000005516 engineering process Methods 0.000 claims abstract description 21
- 238000004891 communication Methods 0.000 claims description 44
- 239000000872 buffer Substances 0.000 claims description 25
- 238000004458 analytical method Methods 0.000 claims description 5
- 238000007906 compression Methods 0.000 claims description 3
- 230000006835 compression Effects 0.000 claims description 3
- 230000007613 environmental effect Effects 0.000 abstract description 4
- 238000013144 data compression Methods 0.000 description 9
- 238000013500 data storage Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 238000012790 confirmation Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007935 neutral effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000004801 process automation Methods 0.000 description 1
- 238000011165 process development Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000000547 structure data Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/542—Event management; Broadcasting; Multicasting; Notifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/544—Buffers; Shared memory; Pipes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/547—Remote procedure calls [RPC]; Web services
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Stored Programmes (AREA)
Abstract
The invention belongs to the technical field of RPA element pickup, and particularly relates to a method and a system for pickup of a credit-invasive environmental element based on a QT interface library. The method comprises the following steps: s1, rpa assists in loading a target app application process, and a detection service library is injected into the target app application process through a gdb debugging injection technology; s2, receiving the operation period information of the current mouse position through a qt mouse event detection mechanism; transmitting the current position information of the mouse and the interface control element information under the current mouse to an rpa service process through a socket; s3, inquiring the current position of the interface control according to the interface control information sent by the rpa service process, and returning the current position to the rpa service process; the rpa service process moves the mouse to the corresponding position according to the interface control position information, and performs mouse clicking or keyboard input operation; s4, picking up is completed, and a qt mouse event detection mechanism is restored.
Description
Technical Field
The invention belongs to the technical field of RPA element pickup, and particularly relates to a method and a system for pickup of a credit-invasive environmental element based on a QT interface library.
Background
RPA (robotic flow automation, robotic Process Automation) is a currently rapidly evolving computer software automation technology. The process editor provided by the RPA software can design and configure the business operation process capable of being automatically executed, package the business operation process into a form of a software robot or a virtual robot, deploy the business operation process to a production environment and a business system for execution, and simulate a series of operations of a person on a computer.
QT is a cross-platform C++ application development framework which provides a complete set of tools including GUI, network, database, XML, multimedia and other modules, and can help developers to quickly construct high-quality cross-platform applications. The name QT is interpreted as a "Cross-platform toolkit" (english all called "QT Cross-platform Application and UI Framework").
However, the existing information creation environment (in linux desktop environment) RPA robot element positioning and picking mode and related technology have the following disadvantages:
1. interface provided by system incapable of being directly used
The system only provides the definition of the bottom api interface, does not provide specific implementation, cannot be directly used, and cannot be used for automation.
Accessibility Toolkit is part of the GNOME project and is an open source software library that provides an application programming interface for implementing accessibility support in software. There is no way to directly use its interface for interface element positioning and pick-up.
App vendors typically do not have power to provide specific implementations
app application vendors typically do not provide specific implementations, resulting in an inability to perform accurate automated, procedural operations on app applications. Based on the consideration of manpower development cost, technical difficulty realization and the like, manufacturers have no power or will to provide open app pick-up and positioning functions for app applications of the manufacturers. Thus, the app application cannot position the interface control element in a native, accurate and quick manner under a credit and debit desktop environment (linux desktop environment).
3. The accuracy of the picture pick-up mode is not high
Based on the mode of realizing the picture analysis visual technology, the accuracy is not high during picking up, and the positioning operation efficiency is low during operation. For a complete technical scheme without general use, part of app manufacturers adopt element positioning and picking based on picture vision technology. Although most of the problems can be solved, the accuracy and the execution speed of the execution effect are not the same as the original interface library analysis mode of picking and positioning. At the same time, it is easily affected by the picture environment and the related display resolution.
Therefore, it is very important to design a method and a system for picking up elements of a created environment based on a QT interface library, which can accurately and rapidly locate and pick up elements of an app application developed based on the QT interface library in the created environment (linux desktop environment).
Disclosure of Invention
The invention provides a method and a system for picking up elements of a created environment based on a QT interface library, which can accurately and rapidly locate and pick up app applications developed on the basis of the QT interface library in the created environment (linux desktop environment) by aiming at solving the problems that the element location and the picking up cannot be accurately and rapidly carried out in the existing created environment (linux desktop environment) by using an RPA robot element location picking-up mode and related technologies in the prior art.
In order to achieve the aim of the invention, the invention adopts the following technical scheme:
a method for picking up information-created environmental elements based on a QT interface library comprises the following steps of;
s1, rpa assists in loading a target app application process, and a detection service library is injected into the target app application process through a gdb debugging injection technology, so that socket services are created to carry out inter-process communication;
s2, in the picking-up stage, receiving the running period information of the current mouse position through a qt mouse event detection mechanism; transmitting the current position information of the mouse and the interface control element information under the current mouse to an rpa service process through a socket;
s3, in the execution stage, according to the interface control information sent by the rpa service process, inquiring the current position of the interface control, and returning to the rpa service process; the rpa service process moves the mouse to the corresponding position according to the interface control position information, and performs mouse clicking or keyboard input operation;
s4, picking up is completed, and a qt mouse event detection mechanism is restored.
Preferably, in step S1, the gdb debug injection technology is to implement process injection through gdb and a shared object file; the shared object file is a program library; gdb is the debugger.
Preferably, in step S1, the socket service is used for communication between different processes in the same system or for communication between different processes in different systems; the socket service is used for transmitting element pickup and element update information.
Preferably, in the process of step S2 and step S3, a protocol buffer protocol mechanism is adopted to perform data communication protocol serialization compression and parsing during communication.
Preferably, the step S2 further includes the steps of:
along with the movement of the mouse, the information of the interface control element under the current mouse is updated in real time, and the position frame of the interface control is highlighted and drawn for prompting the user of the current specific interface control element information.
Preferably, before the step S1 is executed, determining a process id of the current target app application running by using a linux system tool; the process id is used for the unique identification of the communication between the subsequent processes.
The invention also provides a system for picking up the created environmental elements based on the QT interface library, which comprises;
the process injection auxiliary module is used for rpa to assist in loading the target app application process and injecting the detection service library into the target app application process through the gdb debugging injection technology;
the pick-up module is used for receiving the running period information of the current mouse position through a qt mouse event detection mechanism; transmitting the current position information of the mouse and the interface control element information under the current mouse to an rpa service process through a socket;
the execution module is used for inquiring the current position of the interface control according to the interface control information sent by the rpa service process and returning the current position to the rpa service process; the rpa service process moves the mouse to the corresponding position according to the interface control position information, and performs mouse clicking or keyboard input operation;
and the restoring module is used for restoring the mouse event detection mechanism of qt.
Compared with the prior art, the invention has the beneficial effects that: (1) According to the invention, the gdb debugging process injection technology, the inter-process communication technology and the data compression coding mode Protocol buffer (Protocol buffers) are combined with RPA element positioning pickup; (2) The invention firstly injects the detection service library into the target app through the gdb, and can acquire the related information of the running period of the target app process, mainly the information of the qt open source interface library inheritance class system, the mouse position, the interface control element position and the like in the running period; secondly, through an inter-process communication technology, an rpa service process acquires mouse and interface control related information of a target app application process in a request/response mode, and updates corresponding interface control position information in a later period; meanwhile, by means of a Protocol buffer (Protocol buffers) of a data compression coding mode, the execution speed of the sequential data compression and analysis is improved, and the front-back compatibility of a communication Protocol in later development iteration is improved; (3) By the technical scheme, the method and the device can realize accurate and rapid element positioning and pickup of the app application developed based on the QT interface library under the information creation environment (linux desktop environment).
Drawings
FIG. 1 is a schematic flow chart of a method for picking up elements in a created environment based on a QT interface library;
FIG. 2 is a block diagram of a QT interface library-based system for picking up elements of a created environment in accordance with the present invention;
FIG. 3 is a sequence diagram of an execution of the QT interface library-based system for picking up elements of a created environment;
fig. 4 is a flowchart of a method for picking up a created environment element based on QT interface library according to an embodiment of the present invention.
Detailed Description
In order to more clearly illustrate the embodiments of the present invention, specific embodiments of the present invention will be described below with reference to the accompanying drawings. It is evident that the drawings in the following description are only examples of the invention, from which other drawings and other embodiments can be obtained by a person skilled in the art without inventive effort.
As shown in fig. 1, the method for picking up the created environment elements based on QT interface library can be seen, the rpa correlation involves an auxiliary loading process, an rpa service process (communication client), and a detection service library and a target app application, and the specific procedures are as follows:
first, using the linux related systems tool, the process id of the current target runtime is determined. The process id is used for the unique identification of the communication between the subsequent processes.
rpa assists in loading the target app application process and injecting the detection service library into the target app application process through the gdb debugging injection technique. After the detection service library is injected into the target app application, a socket service can be created in the process of the detection service library for subsequent inter-process communication.
In the picking-up stage, the current mouse position and other runtime information are received through a qt mouse event detection mechanism. And transmitting the current position information of the mouse and the related information of the interface control under the current mouse to a communication client (rpa service process) through a socket. Along with the movement of the mouse, the information of the interface control element under the current mouse is updated in real time, and the position frame of the interface control is highlighted and drawn for prompting the user of the current specific interface control element information. The QT mouse event detection mechanism is a mechanism self-contained in the QT interface library.
In the execution stage, the current position of the interface control information sent by the client is queried, and the information is returned to the client (rpa service process). And the rpa service process moves the mouse into the relevant position according to the position information, and performs operations of simulating people such as mouse clicking or keyboard input.
Finally, after the related logic is completed, the normal operation of the target app is not affected by a mouse event detection mechanism for restoring qt.
The gdb debugging injection technology is to realize process injection through the gdb and the shared object file; the shared object file is a program library; gdb is the debugger.
socket service is used for communication between different processes in the same system or for communication between different processes in different systems; the socket service is used to transmit element pick-up and element update information.
And during communication, a protocol buffer protocol mechanism is adopted to perform data communication protocol serialization compression and analysis.
As shown in fig. 2 and 3, the present invention further provides a system for picking up a created environment element based on QT interface library, including: the servce module (injected into the target app process) is detected, and the process is executed to inject the auxiliary module and the communication client (rpa service process) module.
Detecting a service module: the socket service and qt mouse event detection module is mainly included. The socket service sub-module serves as a general socket service, waits for a client (rpa service process) socket to establish connection, and then provides an inter-process communication mechanism. After establishing the connection, receiving a request sent by a client:
1. and acquiring the element information under the current mouse, carrying out relevant drawing on returned data according to the condition by the client according to the rectangular area of the screen, and finally confirming the target element according to the operation of the client. And serializing the information of the confirmation target element by using a Protocol buffer (Protocol Buffers), storing the client into a related file, and using the information when the actual process is carried out in a later execution period.
2. The client loads the picked interface control element information from the originally stored file and inversely sequences the information into a Protocol buffer (Protocol Buffers) format; then sending information to a detection service module to request to update the rectangular related information of the current position of the element; and executing subsequent operations such as a system mouse and a keyboard according to the returned latest position related information.
Process injection auxiliary module: the key library module detection service module is assisted to be injected into the target app application through the gdb debugging technology, and the detection service module and the target app application are in the same process space after the successful injection. Then the detection service module can access related information of a QT interface library and mouse position information and the like of the current running of the target app.
Communication client (rpa service process) module: the method is used as a communication client and is also a main body of rpa flow control. The method mainly comprises the steps that the detection service module picks up the acquired qt interface control operation period information through an inter-process communication mechanism, receives and stores the acquired qt interface control operation period information, reads and sends the stored information to the detection service module to update the latest position information when the operation period is later, and then carries out actual rpa flow automatic execution.
In addition, the system for picking up the created environment elements based on the QT interface library also comprises a picking up module, which is used for receiving the running period information of the current mouse position through a QT mouse event detection mechanism; transmitting the current position information of the mouse and the interface control element information under the current mouse to an rpa service process through a socket;
the execution module is used for inquiring the current position of the interface control according to the interface control information sent by the rpa service process and returning the current position to the rpa service process; the rpa service process moves the mouse to the corresponding position according to the interface control position information, and performs mouse clicking or keyboard input operation;
and the restoring module is used for restoring the mouse event detection mechanism of qt.
The technical scheme applied to each module is specifically described as follows:
1. process injection technology based on gdb debugging
The corresponding module is injected into the target process through the gdb debugging technology, so that the purpose is to acquire the related information of the running period of the target app process, wherein the related information is mainly the running period information such as QT open source interface library inheritance class system, mouse position, interface control element position and the like. Process injection is implemented through gdb and shared object files (i.e., libraries), and is a specific process injection technique used in the present invention.
2. Inter-process communication technique
Inter-process communication refers to some technique or method by which at least two processes communicate data or information. The invention relates to an inter-process communication mode, which is socket, and is different from other inter-process communication mechanisms, and can be used for communication between different processes in the same system and between different processes in different systems. The invention adopts inter-process communication, and is mainly used for transmitting element pickup and element update information. The socket is adopted, so that clients (rpa processes) in communication can be facilitated, deployment and expansion can be performed, and the positions of specific processes and corresponding running systems are not concerned. While isolating the potentially generated interference between the target app application and rpa service processes. Avoiding any impact on the functionality of the target app application itself.
3. Data compression coding mode Protocol buffer (Protocol buffers)
Inter-process communication necessarily requires defining the protocol content between applications. In practical applications, very well-defined formatting of the protocol can complicate the new protocol. Because the developer must ensure that all servers between the request originator and the actual server handling the request understand the new protocol before switching to begin using the new protocol. Each developer encounters problems associated with low version compatibility, new and old protocol compatibility.
Protocol buffers (protobufs), which are a language neutral, platform independent, extensible format of serialized data, can be used for communication protocols, data storage, etc.
It is flexible and efficient in serializing data. Compared with XML, the Protocol buffers are smaller, faster and simpler. Once the data structure of the data to be processed is defined, the code generation tool of the protocol buffer may be utilized to generate the relevant code. The data structure may even be updated without the need to redeploy the program. The structured data can be easily read and written by using various different languages or from various different data streams by only using Protobuf to describe the data structure once.
Protocol buffers are well suited for data storage or RPC data exchange formats. The method can be used for language-independent, platform-independent and extensible serial structure data formats in the fields of communication protocols, data storage and the like.
The protocol buffer is mainly characterized in that:
1. new fields can be easily introduced and intermediate servers that do not need to examine the data can simply parse and pass the data on without having to know all of the fields.
2. The data format is more self-descriptive and can be handled in a variety of languages (c++, java, python, etc.).
Aiming at the app application developed by the rpa robot based on the qt interface library in the information creation environment (linux desktop environment), the element can not locate and pick up the problem. The method adopts the gdb debugging injection technology, an inter-process communication mechanism and a Protocol buffer (Protocol Buffers) data compression analysis mechanism.
As shown in fig. 4, taking the following RPA robot as an example, in the environment of a signal creation linux desktop, picking and positioning nails (linux version), the operation and specific implementation flow are as follows:
1. in the preparation stage of the system, the normal starting running state of the nailing application needs to be determined, and the current process id of the nailing is acquired by using related system tools, wherein the process id is used for a related unique identification part of the communication between the following processes.
2. And injecting the detection service library module into a process space where the nails run through the auxiliary injection module.
3. In the process development stage of the RPA, the RPA service process is used as a client role and sends a request element picking command to the detection service module.
4. And the detection service module acquires the information of the interface control element under the mouse according to the current position of the mouse, and sends the information to the rpa service process in a serialization package mode.
And 5, drawing a prompt box by the rpa service process according to the received interface control position information, and highlighting prompt operation user current effective interface control element information.
6. The user intuitively displays the element information of the interface control at the corresponding position by operating and moving the mouse position, and when the element information is displayed as the required element information, sends pickup confirmation information to the rpa service process, such as pickup operation of a 'login' button of a pin login interface and pickup operation of other account number input boxes. After the operations of the required account number input box, password input box, login button and other pickup elements are completed, the pickup flow can be ended according to the requirements.
7. In the process execution stage of the RPA, the RPA service process reads the stored original pickup information and reverse-serialization related information, sequentially sends updated element latest position information, such as a current account input box, a password input box and login button position information, to the detection service module, and receives the latest interface control element position information.
And 8, the rpa service process moves the corresponding mouse according to the received latest interface control element position information to perform rpa automation flow clicking and keyboard input operation. For example, the nailing flow example can locate and pick up element controls such as an account number input box, a password input box and a login button, and can acquire the control position when the control is executed, execute account number information automatic input and password automatic filling, and automatically click the login button, thereby completing the automatic nailing login operation of the RPA robot and comprising subsequent related operations.
The invention adopts a gdb debugging process injection and inter-process communication architecture, and combines a high-efficiency data compression coding mode Protocol buffer (Protocol buffers) technology to solve the problem that elements cannot be positioned and picked up in an app application developed based on a QT interface library under a credit-creation environment (linux desktop environment).
The innovation points of the invention are as follows:
1. the innovative combination of gdb debugging process injection, inter-process communication and data compression coding mode Protocol buffer (protocols) solves the problem that element positioning pickup cannot be performed under a credit-creation environment (linux desktop environment) by the traditional RPA technology.
2. The method adopts an inter-process communication mode to isolate the possible interference between the target app application and the rpa service process, and avoid any influence on the function of the target app application.
3. The inter-process communication of the invention adopts a Protocol buffer (Protocol buffers) of a data compression coding mode, and the invention has the advantages of high-efficiency and rapid data compression coding serialization and deserialization modes, thereby improving the transmission efficiency between processes and reducing the delay in picking and positioning.
The foregoing is only illustrative of the preferred embodiments and principles of the present invention, and changes in specific embodiments will occur to those skilled in the art upon consideration of the teachings provided herein, and such changes are intended to be included within the scope of the invention as defined by the claims.
Claims (7)
1. The method for picking up the created environment elements based on the QT interface library is characterized by comprising the following steps of;
s1, rpa assists in loading a target app application process, and a detection service library is injected into the target app application process through a gdb debugging injection technology, so that socket services are created to carry out inter-process communication;
s2, in the picking-up stage, receiving the running period information of the current mouse position through a qt mouse event detection mechanism; transmitting the current position information of the mouse and the interface control element information under the current mouse to an rpa service process through a socket;
s3, in the execution stage, according to the interface control information sent by the rpa service process, inquiring the current position of the interface control, and returning to the rpa service process; the rpa service process moves the mouse to the corresponding position according to the interface control position information, and performs mouse clicking or keyboard input operation;
s4, picking up is completed, and a qt mouse event detection mechanism is restored.
2. The method for picking up a created environment element based on QT interface library of claim 1, wherein in step S1, the gdb debug injection technique is to implement process injection by gdb and a shared object file; the shared object file is a program library; gdb is the debugger.
3. The method for picking up a created environment element based on QT interface library according to claim 1, wherein in step S1, the socket service is used for communication between different processes in the same system or for communication between different processes in different systems; the socket service is used for transmitting element pickup and element update information.
4. The method for picking up information and creating environment elements based on QT interface library according to claim 1 is characterized in that in the process of step S2 and step S3, a protocol buffer protocol mechanism is adopted for carrying out data communication protocol serialization compression and analysis during communication.
5. The method for picking up a created environment element based on QT interface library of claim 1, wherein step S2 further includes the steps of:
along with the movement of the mouse, the information of the interface control element under the current mouse is updated in real time, and the position frame of the interface control is highlighted and drawn for prompting the user of the current specific interface control element information.
6. The method for picking up a created environment element based on QT interface library according to claim 1, wherein before step S1 is performed, a linux system tool is used to determine a process id when the current target app application is running; the process id is used for the unique identification of the communication between the subsequent processes.
7. A QT interface library-based credit-creating environment element pickup system for implementing the QT interface library-based credit-creating environment element pickup method of any one of claims 1 to 6, characterized in that the QT interface library-based credit-creating environment element pickup system includes;
the process injection auxiliary module is used for rpa to assist in loading the target app application process and injecting the detection service library into the target app application process through the gdb debugging injection technology;
the pick-up module is used for receiving the running period information of the current mouse position through a qt mouse event detection mechanism; transmitting the current position information of the mouse and the interface control element information under the current mouse to an rpa service process through a socket;
the execution module is used for inquiring the current position of the interface control according to the interface control information sent by the rpa service process and returning the current position to the rpa service process; the rpa service process moves the mouse to the corresponding position according to the interface control position information, and performs mouse clicking or keyboard input operation;
and the restoring module is used for restoring the mouse event detection mechanism of qt.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310569094.3A CN116302621A (en) | 2023-05-19 | 2023-05-19 | Method and system for picking up information-created environment elements based on QT interface library |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310569094.3A CN116302621A (en) | 2023-05-19 | 2023-05-19 | Method and system for picking up information-created environment elements based on QT interface library |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116302621A true CN116302621A (en) | 2023-06-23 |
Family
ID=86781961
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310569094.3A Pending CN116302621A (en) | 2023-05-19 | 2023-05-19 | Method and system for picking up information-created environment elements based on QT interface library |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116302621A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117667285A (en) * | 2023-12-04 | 2024-03-08 | 九科信息技术(深圳)有限公司 | Element selection method, system, terminal equipment and readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105607985A (en) * | 2015-09-29 | 2016-05-25 | 中国船舶重工集团公司第七0九研究所 | Qt graphical interface software-based operation log recording method and system |
| CN112445994A (en) * | 2020-11-12 | 2021-03-05 | 广东电网有限责任公司 | Method for carrying out automatic operation in opened Chrome browser |
| CN113391871A (en) * | 2021-08-17 | 2021-09-14 | 杭州实在智能科技有限公司 | RPA element intelligent fusion picking method and system |
| CN114637662A (en) * | 2022-05-13 | 2022-06-17 | 杭州未斯科技有限公司 | RPA process automatic recording control method and system |
| CN114816401A (en) * | 2022-04-13 | 2022-07-29 | 上海弘玑信息技术有限公司 | Interface element positioning method, electronic equipment and storage medium |
| US20230052452A1 (en) * | 2021-08-12 | 2023-02-16 | International Business Machines Corporation | Socket transferring for hpc networks using kernel tracing |
-
2023
- 2023-05-19 CN CN202310569094.3A patent/CN116302621A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105607985A (en) * | 2015-09-29 | 2016-05-25 | 中国船舶重工集团公司第七0九研究所 | Qt graphical interface software-based operation log recording method and system |
| CN112445994A (en) * | 2020-11-12 | 2021-03-05 | 广东电网有限责任公司 | Method for carrying out automatic operation in opened Chrome browser |
| US20230052452A1 (en) * | 2021-08-12 | 2023-02-16 | International Business Machines Corporation | Socket transferring for hpc networks using kernel tracing |
| CN113391871A (en) * | 2021-08-17 | 2021-09-14 | 杭州实在智能科技有限公司 | RPA element intelligent fusion picking method and system |
| CN114816401A (en) * | 2022-04-13 | 2022-07-29 | 上海弘玑信息技术有限公司 | Interface element positioning method, electronic equipment and storage medium |
| CN114637662A (en) * | 2022-05-13 | 2022-06-17 | 杭州未斯科技有限公司 | RPA process automatic recording control method and system |
Non-Patent Citations (1)
| Title |
|---|
| 张金;邬晶;陈卓宁;: "基于界面可视信息提取的软件互动集成", 计算机系统应用, no. 08 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117667285A (en) * | 2023-12-04 | 2024-03-08 | 九科信息技术(深圳)有限公司 | Element selection method, system, terminal equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8196108B2 (en) | Breakpoint management and reconciliation for embedded scripts in a business integration language specified program process | |
| EP0752653B1 (en) | Method and system for synchronizing the execution of events during software testing | |
| US8156478B2 (en) | Optimized program analysis | |
| US8024721B2 (en) | System and method for implementing a safe framework | |
| CN104424095A (en) | Automatic testing method and system of mobile terminal | |
| CN110716853A (en) | Test script recording method, application program testing method and related device | |
| US8250554B2 (en) | Systems and methods for generating and distributing executable procedures for technical desk-side support | |
| CN111782513B (en) | Satellite ground universal automatic test method based on DLL | |
| CN112732576B (en) | Automatic testing method, device and equipment based on user interface | |
| CN116302621A (en) | Method and system for picking up information-created environment elements based on QT interface library | |
| CN115757183A (en) | Agent-based task level debugging method under weather embedded operating system | |
| US20110061042A1 (en) | Synchronized java debugger | |
| CN115658496A (en) | Extensible Web automatic test method, system, equipment and storage medium | |
| CN112988588B (en) | Client software debugging method and device, storage medium and electronic equipment | |
| CN111694744B (en) | Test development method based on monkey source code | |
| US20080059949A1 (en) | System and method for implementing a safe framework | |
| CN116248526A (en) | Method and device for deploying container platform and electronic equipment | |
| KR101418390B1 (en) | system for remotely debugging an application programming and controlling method therefor | |
| CN111061522A (en) | Method, device, equipment and storage medium for generating front-end card assembly based on back end | |
| CN115421770A (en) | Resource information processing method and device, storage medium and electronic equipment | |
| CN110515849B (en) | Breakpoint debugging method, device, system, equipment and storage medium | |
| CN110554966B (en) | Driving debugging method, behavior analysis method and driving debugging system | |
| CN117130717B (en) | Element positioning method and system of HTMLayout application program in RPA scene | |
| CN116991380B (en) | Application program construction method and device, electronic equipment and storage medium | |
| CN111611161B (en) | Implementation method of lightweight debugging tool applied to avionic software |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20230623 |