CN111309617A - Application program control method and device, storage medium and electronic equipment - Google Patents
Application program control method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN111309617A CN111309617A CN202010108528.6A CN202010108528A CN111309617A CN 111309617 A CN111309617 A CN 111309617A CN 202010108528 A CN202010108528 A CN 202010108528A CN 111309617 A CN111309617 A CN 111309617A
- Authority
- CN
- China
- Prior art keywords
- touch instruction
- screen image
- current screen
- target process
- application program
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 152
- 230000008569 process Effects 0.000 claims abstract description 109
- 238000004088 simulation Methods 0.000 claims description 77
- 238000004590 computer program Methods 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 12
- 230000006870 function Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 230000009191 jumping Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The disclosure relates to a control method and device of an application program, a storage medium and an electronic device. The method comprises the following steps: under the condition that a target application program to be controlled is in a foreground running state, a current screen image is obtained through a target process running in a terminal, wherein the target process has Shell permission, the current screen image is used for determining whether a corresponding simulated touch instruction is automatically generated or not, and the simulated touch instruction is used for controlling the target application program to execute target operation; after the target process obtains the simulated touch instruction, the target process sends the simulated touch instruction to an operating system of the terminal, so that the operating system runs the simulated touch instruction. Therefore, manual operation of a user is not needed, automatic control over the target application program can be achieved only by the Shell permission of the target process, the Root permission of the system does not need to be acquired, potential risks caused by the Root permission of the system are avoided, and therefore the safety of the terminal system is greatly improved.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method and an apparatus for controlling an application program, a storage medium, and an electronic device.
Background
When using a mobile phone, a tablet computer, or other terminal device, there are many application scenarios that require a user to repeatedly operate the terminal device. For example, testing before a website or application releases requires a technician to repeatedly click on a button in a page to test whether the page can jump normally. For another example, each time a prompt window pops up in the terminal, the user needs to click the prompt window to open the prompt message. At present, the mode of manually and repeatedly operating the terminal equipment by a user wastes time and labor, and the user experience is poor.
Disclosure of Invention
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
In a first aspect, the present disclosure provides a method for controlling an application, the method including:
under the condition that a target application program to be controlled is in a foreground running state, acquiring a current screen image through a target process running in a terminal, wherein the target process has Shell permission, the current screen image is used for determining whether a corresponding simulated touch instruction is automatically generated or not, and the simulated touch instruction is used for controlling the target application program to execute target operation;
after the target process obtains the simulated touch instruction, the target process sends the simulated touch instruction to an operating system of the terminal, so that the operating system runs the simulated touch instruction.
In a second aspect, the present disclosure provides an apparatus for controlling an application, the apparatus comprising:
the system comprises an information acquisition module and an operating system;
the information acquisition module runs a target process, the target process has Shell permission, and the target process is used for acquiring a current screen image under the condition that a target application program to be controlled is in a foreground running state, wherein the current screen image is used for determining whether a corresponding simulated touch instruction is automatically generated or not, and the simulated touch instruction is used for controlling the target application program to execute target operation;
the target process is further used for sending the simulated touch instruction to the operating system after the simulated touch instruction is obtained;
the operating system is used for operating the simulation touch instruction.
In a third aspect, the present disclosure provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processing apparatus, performs the steps of the method provided by the first aspect of the present disclosure.
In a fourth aspect, the present disclosure provides an electronic device comprising: a storage device having a computer program stored thereon; processing means for executing the computer program in the storage means to implement the steps of the method provided by the first aspect of the present disclosure.
In the technical scheme, a current screen image of the terminal is obtained through a target process with Shell permission running in the terminal, the current screen image is used for determining whether to automatically generate a corresponding simulated touch instruction, and the simulated touch instruction can be used for automatically controlling a target application program to execute target operation. After the target process acquires the simulated touch instruction, the simulated touch instruction can be sent to an operating system of the terminal, so that the operating system runs the simulated touch instruction. Therefore, the purpose of performing touch operation on the terminal instead of a user can be achieved, automatic control over the target application program can be achieved without manual operation of the user, and user experience is improved. Moreover, according to the technical scheme, the target application program can be automatically controlled only by the Shell permission of the target process, the Root permission of the system does not need to be acquired, the Shell permission is lower than the Root permission, potential risks caused by acquiring the Root permission of the system are effectively avoided, and therefore the safety of the terminal system is greatly improved.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows.
Drawings
The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.
In the drawings:
fig. 1 is a flowchart illustrating a control method of an application according to an exemplary embodiment.
Fig. 2 is a block diagram illustrating a control device for an application according to an exemplary embodiment.
Fig. 3 is a schematic structural diagram of an electronic device according to an exemplary embodiment.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.
The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.
It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.
It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.
The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.
As described in the background art, some application scenarios require a user to repeatedly operate a terminal device, which is time-consuming and labor-consuming for the user. In the related art, these repetitive operations may be performed by an application installed in the terminal instead of the user. However, an application program in the related art can normally operate only when acquiring an administrator permission (Root permission) of the system, the Root permission is the highest permission of the system, and after acquiring the Root permission, the application program has the capability of performing important operations such as addition, deletion, modification and check on files in the terminal system, so that potential risks such as leakage of user privacy data and malicious fee deduction exist, and the security is low.
In view of this, the present disclosure provides a control method and apparatus for an application, a storage medium, and an electronic device, which can achieve the purpose of automatically performing touch operation on a terminal only through a Shell permission of the terminal, and a Root permission of the terminal does not need to be acquired, where the Shell permission is lower than the Root permission, so that security of a terminal system is greatly improved.
Fig. 1 is a flowchart illustrating a control method of an application according to an exemplary embodiment. The method can be applied to terminals such as smart phones, tablet computers, Personal Computers (PCs), notebook computers and other devices. As shown in fig. 1, the method may include S101 and S102.
In S101, in a case where a target application to be controlled is in a foreground operating state, a current screen image is acquired by a target process running in the terminal.
The target application to be controlled may be any application in the terminal, such as a desktop application, a system application, a driver application, a network application downloaded in the terminal, and the like. And under the condition that the target application program to be controlled is in a foreground running state, acquiring a current screen image through a target process running in the terminal.
Wherein the target process has Shell rights. The target process may be created and started in the terminal by an external device connected to the terminal, and the external device may be a personal computer, a notebook computer, or the like connected to the terminal. The terminal and the external device may establish connection through a wired communication manner (e.g., a USB connection line), after the terminal and the external device establish connection, the external device may send an installation package for creating a target process to the terminal, and after the terminal is installed, the external device may send a start instruction for starting the target process to the terminal, so that the terminal runs the target process. It should be noted that, after the target process is started by the external device, even if the connection between the terminal and the external device is disconnected, the normal operation of the target process in the terminal is not affected.
The current screen image refers to an image of a front-end page currently displayed to a user by the terminal, namely, an image of a page currently displayed by the target application program in a foreground running state. The target process has the Shell permission, so that the target process has the capability of acquiring the current screen image of the terminal. In the present disclosure, the target process may obtain the current screen image of the terminal according to a Java reflection mechanism, for example, obtain the current screen image by way of a reflection call function.
The current screen image may be used to determine whether to automatically generate a corresponding simulated touch instruction, which may be used to control the target application to perform a target operation, which may include, for example, a page jump operation, a page refresh operation, etc. of the target application. The simulation touch instruction can be used for simulating manual touch operation of a user by using a finger, a mouse, a touch pen and the like, so that the manual operation of the user can be replaced, and the purposes of performing touch operation on the terminal and automatically controlling a target application program are achieved.
The simulated touch instruction corresponding to the current screen image refers to touch operation required to be performed in a front-end page currently displayed by the terminal. The simulated touch command may include a click command and a slide command. For example, the click command may include click position information, click times, click strength information, and the like, wherein the click position information may be coordinate information of a click position point, the click times are, for example, single click and double click, and the click strength information is, for example, heavy press and light press. For example, the swipe instruction may include swipe start point position information, swipe stop point position information, and the like, the swipe start point position information may be coordinate information of the swipe start position point, and the swipe stop point position information may be coordinate information of the swipe stop position point.
For example, a front-end page currently displayed by the terminal is a first page of the target application program, and the current screen image is an image of the first page, for example, a preset button in the first page needs to be clicked to jump to a second page, the simulated touch instruction corresponding to the current screen image may be a click instruction for clicking the preset button, where the click instruction may include position information of the preset button, the click frequency is click, the click strength is tap, and the like, so as to control the target application program to execute a target operation for jumping from the first page to the second page. Or, for example, a refresh button in the first page needs to be clicked to refresh the first page, the simulated touch instruction corresponding to the current screen image may be a click instruction for clicking the refresh button, so as to control the target application program to execute the target operation for refreshing the first page.
For another example, the front-end page currently displayed by the terminal is a third page of the target application, the current screen image is an image of the third page, and for example, the third page needs to be slid rightward to jump to a fourth page, the simulated touch instruction corresponding to the current screen image may be a sliding instruction sliding rightward, and the sliding instruction may include position information of a sliding start point and position information of a sliding stop point after sliding rightward, so as to control the target application to execute a target operation of jumping from the third page to the fourth page.
In S102, after the target process obtains the simulated touch instruction, the target process sends the simulated touch instruction to the operating system of the terminal, so that the operating system runs the simulated touch instruction.
The target process has the Shell permission, so that the target process has the capability of sending the simulated touch instruction to the operating system of the terminal. In this disclosure, the target process may send the simulated touch instruction to the operating system of the terminal according to the Java reflection mechanism, for example, send the simulated touch instruction by reflecting a call function.
After receiving the simulated touch instruction sent by the target process, the operating system of the terminal can run the simulated touch instruction in the front-end page currently displayed by the terminal, so that the touch operation of the terminal and the automatic control of the target application program can be realized, for example, the touch operation of clicking a preset button in the first page, the touch operation of clicking a refresh button in the first page, or the touch operation of sliding to the right in the third page is realized.
In the technical scheme, a current screen image of the terminal is obtained through a target process with Shell permission running in the terminal, the current screen image is used for determining whether to automatically generate a corresponding simulated touch instruction, and the simulated touch instruction can be used for automatically controlling a target application program to execute target operation. After the target process acquires the simulated touch instruction, the simulated touch instruction can be sent to an operating system of the terminal, so that the operating system runs the simulated touch instruction. Therefore, the purpose of performing touch operation on the terminal instead of a user can be achieved, automatic control over the target application program can be achieved without manual operation of the user, and user experience is improved. Moreover, according to the technical scheme, the target application program can be automatically controlled only by the Shell permission of the target process, the Root permission of the system does not need to be acquired, the Shell permission is lower than the Root permission, potential risks caused by acquiring the Root permission of the system are effectively avoided, and therefore the safety of the terminal system is greatly improved.
In this disclosure, a touch simulation application may be run on the terminal, and the touch simulation application may be used for a user to preset a touch operation to be performed on a target application, such as a user-defined touch operation of clicking a preset button on a first page. The touch simulation application program can generate a script program according to the setting of the user, and the script program can automatically realize the user-defined touch operation on the target application program when running. The user may also set an automatic runtime of the script program, such as starting to run at nine am to automatically control the target application. In addition, the user can control whether the script program runs or not by clicking a start or stop button of the script program.
In the above S101, a specific implementation of acquiring the current screen image through the target process running in the terminal may be: and the target process responds to the received screen image acquisition instruction and acquires the current screen image.
The screen image acquisition instruction is used for requesting the target process to acquire the current screen image of the terminal. The screen image capturing instruction may be sent to the target process by the touch simulation application program. For example, in the case that the current time meets the automatic running time of the script program set by the user, or the user triggers a start button of the script program, the touch simulation application may send a screen image acquisition instruction to the target process.
For example, the screen image obtaining instruction may indicate to request the target process to obtain one current screen image, and may also indicate to request the target process to obtain multiple current screen images, where the screen image obtaining instruction is used to instruct the target process to send the current screen image of the terminal once every preset time length.
In this disclosure, the target process may obtain the simulated touch instruction corresponding to the current screen image in the following two ways.
Firstly, the target process determines whether a corresponding simulated touch instruction needs to be generated according to the current screen image, and generates the simulated touch instruction under the condition that the corresponding simulated touch instruction needs to be generated. That is, the target process itself may generate the simulated touch instruction.
Secondly, the target process sends the current screen image to the touch simulation application program and receives a simulation touch instruction corresponding to the current screen image returned by the touch simulation application program. That is, the target process may obtain the simulated touch instruction from the touch simulation application.
In one embodiment, the target process may send the current screen image of the terminal to the touch simulation application program upon receiving a screen image acquisition instruction sent by the touch simulation application program. After receiving a current screen image sent by a target process, a touch simulation application program can determine whether a corresponding simulation touch instruction needs to be generated according to the current screen image, generate the simulation touch instruction under the condition that the simulation touch instruction needs to be generated, and send the simulation touch instruction to the target process.
In another embodiment, after receiving the current screen image sent by the target process, the touch simulation application program may send the current screen image to the server, so that the server determines whether a corresponding simulated touch instruction needs to be generated according to the current screen image. And under the condition that the simulation touch instruction needs to be generated, the server generates the simulation touch instruction and sends the simulation touch instruction to the touch simulation application program. After receiving the simulated touch instruction returned by the server, the touch simulation application program can send the simulated touch instruction to the target process.
In the above technical solution, the simulated touch instruction may be generated by a target process, may also be generated by a touch simulation application, and may also be generated by a server. In this disclosure, the specific implementation of determining whether to generate a corresponding simulated touch instruction according to the current screen image may be: and if the preset image meeting the similarity condition with the current screen image exists, determining that a corresponding simulation touch instruction needs to be generated. Accordingly, generating the simulated touch instruction may include: and taking the simulated touch instruction corresponding to the pre-stored preset image as the simulated touch instruction corresponding to the current screen image.
The script program generated by the touch simulation application program according to the setting of the user may store a corresponding relationship between a preset image and a simulation touch instruction, where the preset image may be an image corresponding to any front-end page in the target application program, and the simulation touch instruction corresponding to the preset image is a touch operation that the user wants to perform in the front-end page corresponding to the preset image. The target process may store a script program, and for example, a technician may store the script program in an installation package of the target process in advance, and the target process may store the script program after the target process is installed in the terminal. The server may also store a script program, and for example, the touch simulation application may send the generated script program to the server.
After the target process, the touch simulation application program and the server acquire the current screen image of the terminal, the current screen image can be identified, for example, pixel points in the current screen image are extracted for identification, so as to identify whether a preset image meeting the similarity condition with the current screen image exists. The current screen image and the preset image meet the similarity condition, and the similarity between the current screen image and the preset image can be represented to be the same or higher. For example, the similarity condition may include at least one of:
(1) the number of the pixel points with the same characteristics in the current screen image and the preset image is larger than a preset number threshold.
For example, the pixel points with the same characteristics may be pixel points with the same RGB three-channel values, or pixel points with the same hue information, saturation information, and brightness information. The predetermined number threshold may be pre-calibrated. The fact that the number of the pixels with the same characteristics in the current screen image and the preset image is larger than the preset number threshold indicates that the number of the pixels with the same characteristics in the current screen image and the preset image is large, and the two images are the same or have high similarity.
In one embodiment, all pixels in the current screen image may be identified to determine the number of pixels having the same characteristics as those in the preset image.
In another embodiment, in order to improve the recognition efficiency, only a part of the pixel points in the current screen image may be recognized. For example, a plurality of pixel points located at preset positions may be extracted from the current screen image for identification, so as to determine the number of pixel points with the same characteristics as those in the preset image. Therefore, the recognition speed of the current screen image can be improved, and the processing efficiency can be improved.
(2) The Hamming distance between the current screen image and the preset image is smaller than a preset distance threshold.
The way in which the hamming distance between two images is calculated can be referred to the correlation technique. The smaller the hamming distance between two images, the more similar the two images can be characterized. The Hamming distance between the current screen image and the preset image is smaller than a preset distance threshold value, which indicates that the Hamming distance between the current screen image and the preset image is the same as or has higher similarity with the preset image, wherein the preset distance threshold value can be calibrated in advance.
In the present disclosure, the similarity condition may include one or both of the above-described condition (1) and condition (2). When the similarity condition includes both the condition (1) and the condition (2), the current screen image and the preset image may satisfy one of the conditions, for example, the condition (1) or the condition (2), that is, the similarity condition is considered to be satisfied. Or, in order to ensure the accuracy of the similarity determination, the similarity condition may be considered to be satisfied when the current screen image and the preset image satisfy both of the two conditions.
Because the current screen image is the same as or similar to the preset image, the current screen image and the preset image can be considered to correspond to the same front-end page, and therefore, the simulated touch instruction corresponding to the preset image stored in advance can be used as the simulated touch instruction corresponding to the current screen image.
Based on the same inventive concept, the disclosure also provides a control device of the application program. Fig. 2 is a block diagram illustrating an apparatus for controlling an application according to an exemplary embodiment, and as shown in fig. 2, the apparatus 200 may include:
an information acquisition module 201 and an operating system 202;
a target process runs in the information obtaining module 201, the target process has a Shell right, and the target process is used for obtaining a current screen image when a target application program to be controlled is in a foreground running state, wherein the current screen image is used for determining whether a corresponding simulated touch instruction is automatically generated or not, and the simulated touch instruction is used for controlling the target application program to execute a target operation;
the target process is further configured to send the simulated touch instruction to the operating system 202 after the simulated touch instruction is obtained;
the operating system 202 is configured to run the simulated touch instruction.
In the technical scheme, a current screen image of the terminal is obtained through a target process with Shell permission running in the terminal, the current screen image is used for determining whether to automatically generate a corresponding simulated touch instruction, and the simulated touch instruction can be used for automatically controlling a target application program to execute target operation. After the target process acquires the simulated touch instruction, the simulated touch instruction can be sent to an operating system of the terminal, so that the operating system runs the simulated touch instruction. Therefore, the purpose of performing touch operation on the terminal instead of a user can be achieved, automatic control over the target application program can be achieved without manual operation of the user, and user experience is improved. Moreover, according to the technical scheme, the target application program can be automatically controlled only by the Shell permission of the target process, the Root permission of the system does not need to be acquired, the Shell permission is lower than the Root permission, potential risks caused by acquiring the Root permission of the system are effectively avoided, and therefore the safety of the terminal system is greatly improved.
Optionally, the target process is configured to determine whether a corresponding simulated touch instruction needs to be generated according to the current screen image, and generate the simulated touch instruction when it is determined that the corresponding simulated touch instruction needs to be generated.
Optionally, a touch simulation application is run on the device 200; the target process is used for sending the current screen image to the touch simulation application program;
the touch simulation application program is used for determining whether a corresponding simulation touch instruction needs to be generated or not according to the current screen image, generating the simulation touch instruction under the condition that the simulation touch instruction needs to be generated, and sending the simulation touch instruction to the target process;
or the touch simulation application program is used for sending the current screen image to a server, so that the server determines whether a corresponding simulation touch instruction needs to be generated according to the current screen image, and the touch simulation application program is used for sending the simulation touch instruction to the target process after receiving the simulation touch instruction returned by the server.
Optionally, a touch simulation application is run on the device 200; the target process is used for responding to a received screen image acquisition instruction and acquiring the current screen image, wherein the screen image acquisition instruction is sent by the touch simulation application program.
Referring now to FIG. 3, a block diagram of an electronic device 300 suitable for use in implementing embodiments of the present disclosure is shown. The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in fig. 3 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 3, the electronic device 300 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 301 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)302 or a program loaded from a storage means 308 into a Random Access Memory (RAM) 303. In the RAM 303, various programs and data necessary for the operation of the electronic apparatus 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input/output (I/O) interface 305 is also connected to bus 304.
Generally, the following devices may be connected to the I/O interface 305: input devices 306 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 307 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage devices 308 including, for example, magnetic tape, hard disk, etc.; and a communication device 309. The communication means 309 may allow the electronic device 300 to communicate wirelessly or by wire with other devices to exchange data. While fig. 3 illustrates an electronic device 300 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication means 309, or installed from the storage means 308, or installed from the ROM 302. The computer program, when executed by the processing device 301, performs the above-described functions defined in the methods of the embodiments of the present disclosure.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some implementations, the clients may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: under the condition that a target application program to be controlled is in a foreground running state, acquiring a current screen image through a target process running in a terminal, wherein the target process has Shell permission, the current screen image is used for determining whether a corresponding simulated touch instruction is automatically generated or not, and the simulated touch instruction is used for controlling the target application program to execute target operation; after the target process obtains the simulated touch instruction, the target process sends the simulated touch instruction to an operating system of the terminal, so that the operating system runs the simulated touch instruction.
Computer program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present disclosure may be implemented by software or hardware. The name of a module does not in some cases constitute a limitation of the module itself, and for example, an information acquisition module may also be described as an "image acquisition module".
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
Example 1 provides a method of operating a terminal, the method comprising: under the condition that a target application program to be controlled is in a foreground running state, acquiring a current screen image through a target process running in a terminal, wherein the target process has Shell permission, the current screen image is used for determining whether a corresponding simulated touch instruction is automatically generated or not, and the simulated touch instruction is used for controlling the target application program to execute target operation; after the target process obtains the simulated touch instruction, the target process sends the simulated touch instruction to an operating system of the terminal, so that the operating system runs the simulated touch instruction.
Example 2 provides the method of example 1, the target process being created and initiated in the terminal by an external device connected to the terminal, according to one or more embodiments of the present disclosure.
Example 3 provides the method of example 1, further comprising, in accordance with one or more embodiments of the present disclosure: and the target process determines whether a corresponding simulated touch instruction needs to be generated or not according to the current screen image, and generates the simulated touch instruction under the condition that the corresponding simulated touch instruction needs to be generated.
Example 4 provides the method of example 1, the terminal having a touch simulation application running thereon; the method further comprises the following steps: the target process sends the current screen image to the touch simulation application program; the touch simulation application program determines whether a corresponding simulation touch instruction needs to be generated or not according to the current screen image, generates the simulation touch instruction under the condition that the simulation touch instruction needs to be generated, and sends the simulation touch instruction to the target process; or the touch simulation application program sends the current screen image to a server, so that the server determines whether a corresponding simulation touch instruction needs to be generated according to the current screen image, and the touch simulation application program sends the simulation touch instruction to the target process after receiving the simulation touch instruction returned by the server.
Example 5 provides the method of example 3 or example 4, the determining whether a corresponding simulated touch instruction needs to be generated from the current screen image according to one or more embodiments of the present disclosure, including: if a preset image meeting the similarity condition with the current screen image exists, determining that a corresponding simulation touch instruction needs to be generated; accordingly, the generating the simulated touch instruction includes: and taking the pre-stored simulated touch instruction corresponding to the preset image as the simulated touch instruction corresponding to the current screen image.
Example 6 provides the method of example 5, the similarity condition comprising at least one of: the number of pixel points with the same characteristics in the current screen image and the preset image is larger than a preset number threshold; and the Hamming distance between the current screen image and the preset image is smaller than a preset distance threshold value.
Example 7 provides the method of example 1, the terminal having a touch simulation application running thereon, in accordance with one or more embodiments of the present disclosure; the acquiring of the current screen image through the target process running in the terminal includes: and the target process responds to a received screen image acquisition instruction to acquire the current screen image, wherein the screen image acquisition instruction is sent by the touch simulation application program.
Example 8 provides an apparatus for controlling an application, the apparatus including: the system comprises an information acquisition module and an operating system; the information acquisition module runs a target process, the target process has Shell permission, and the target process is used for acquiring a current screen image under the condition that a target application program to be controlled is in a foreground running state, wherein the current screen image is used for determining whether a corresponding simulated touch instruction is automatically generated or not, and the simulated touch instruction is used for controlling the target application program to execute target operation; the target process is further used for sending the simulated touch instruction to the operating system after the simulated touch instruction is obtained; the operating system is used for operating the simulation touch instruction.
Example 9 provides a computer-readable medium having stored thereon a computer program that, when executed by a processing apparatus, implements the steps of the methods of examples 1-7, in accordance with one or more embodiments of the present disclosure.
Example 10 provides, in accordance with one or more embodiments of the present disclosure, an electronic device comprising: a storage device having a computer program stored thereon; processing means for executing the computer program in the storage means to implement the steps of the methods of examples 1-7.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.
Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Claims (10)
1. A method for controlling an application, the method comprising:
under the condition that a target application program to be controlled is in a foreground running state, acquiring a current screen image through a target process running in a terminal, wherein the target process has Shell permission, the current screen image is used for determining whether a corresponding simulated touch instruction is automatically generated or not, and the simulated touch instruction is used for controlling the target application program to execute target operation;
after the target process obtains the simulated touch instruction, the target process sends the simulated touch instruction to an operating system of the terminal, so that the operating system runs the simulated touch instruction.
2. The method according to claim 1, wherein the target process is created and initiated in the terminal by an external device connected to the terminal.
3. The method of claim 1, further comprising:
and the target process determines whether a corresponding simulated touch instruction needs to be generated or not according to the current screen image, and generates the simulated touch instruction under the condition that the corresponding simulated touch instruction needs to be generated.
4. The method according to claim 1, wherein a touch simulation application is run on the terminal; the method further comprises the following steps:
the target process sends the current screen image to the touch simulation application program;
the touch simulation application program determines whether a corresponding simulation touch instruction needs to be generated or not according to the current screen image, generates the simulation touch instruction under the condition that the simulation touch instruction needs to be generated, and sends the simulation touch instruction to the target process;
or the touch simulation application program sends the current screen image to a server, so that the server determines whether a corresponding simulation touch instruction needs to be generated according to the current screen image, and the touch simulation application program sends the simulation touch instruction to the target process after receiving the simulation touch instruction returned by the server.
5. The method according to claim 3 or 4, wherein the determining whether the corresponding simulated touch instruction needs to be generated according to the current screen image comprises:
if a preset image meeting the similarity condition with the current screen image exists, determining that a corresponding simulation touch instruction needs to be generated;
accordingly, the generating the simulated touch instruction includes:
and taking the pre-stored simulated touch instruction corresponding to the preset image as the simulated touch instruction corresponding to the current screen image.
6. The method of claim 5, wherein the similarity condition comprises at least one of:
the number of pixel points with the same characteristics in the current screen image and the preset image is larger than a preset number threshold;
and the Hamming distance between the current screen image and the preset image is smaller than a preset distance threshold value.
7. The method according to claim 1, wherein a touch simulation application is run on the terminal;
the acquiring of the current screen image through the target process running in the terminal includes:
and the target process responds to a received screen image acquisition instruction to acquire the current screen image, wherein the screen image acquisition instruction is sent by the touch simulation application program.
8. An apparatus for controlling an application, the apparatus comprising:
the system comprises an information acquisition module and an operating system;
the information acquisition module runs a target process, the target process has Shell permission, and the target process is used for acquiring a current screen image under the condition that a target application program to be controlled is in a foreground running state, wherein the current screen image is used for determining whether a corresponding simulated touch instruction is automatically generated or not, and the simulated touch instruction is used for controlling the target application program to execute target operation;
the target process is further used for sending the simulated touch instruction to the operating system after the simulated touch instruction is obtained;
the operating system is used for operating the simulation touch instruction.
9. A computer-readable medium, on which a computer program is stored, characterized in that the program, when being executed by processing means, carries out the steps of the method of any one of claims 1 to 7.
10. An electronic device, comprising:
a storage device having a computer program stored thereon;
processing means for executing the computer program in the storage means to carry out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010108528.6A CN111309617A (en) | 2020-02-21 | 2020-02-21 | Application program control method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010108528.6A CN111309617A (en) | 2020-02-21 | 2020-02-21 | Application program control method and device, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111309617A true CN111309617A (en) | 2020-06-19 |
Family
ID=71145589
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010108528.6A Pending CN111309617A (en) | 2020-02-21 | 2020-02-21 | Application program control method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111309617A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113407933A (en) * | 2021-06-16 | 2021-09-17 | 成都欧珀通信科技有限公司 | Kernel function reinforcing method and device, storage medium and electronic equipment |
| CN113507500A (en) * | 2021-06-04 | 2021-10-15 | 上海闻泰信息技术有限公司 | Terminal control method, terminal control device, computer equipment and computer-readable storage medium |
| CN114265535A (en) * | 2021-12-21 | 2022-04-01 | 百度在线网络技术（北京）有限公司 | Push message landing page display method, device, equipment and medium |
| CN114296825A (en) * | 2021-12-29 | 2022-04-08 | 杭州盈高科技有限公司 | Starting method and device of mobile terminal, storage medium and processor |
| CN114327185A (en) * | 2021-12-29 | 2022-04-12 | 盯盯拍（深圳）技术股份有限公司 | Vehicle screen control method and device, medium and electronic equipment |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102841712A (en) * | 2012-07-11 | 2012-12-26 | 广州市久邦数码科技有限公司 | Method and system for identifying and editing gestures |
| CN104462879A (en) * | 2014-11-28 | 2015-03-25 | 北京奇虎科技有限公司 | Root-free running control method and device of application program |
| CN106021108A (en) * | 2016-05-19 | 2016-10-12 | 努比亚技术有限公司 | Application program testing device, mobile terminal and method |
| CN106406914A (en) * | 2016-11-11 | 2017-02-15 | 武汉斗鱼网络科技有限公司 | Method and device for obtaining Android system window data |
| CN106445816A (en) * | 2016-09-07 | 2017-02-22 | 美的智慧家居科技有限公司 | Control execution method and apparatus |
| CN106598251A (en) * | 2016-12-23 | 2017-04-26 | 重庆墨希科技有限公司 | Smartband-based gesture control method and system, and smartband |
| CN106843729A (en) * | 2017-01-20 | 2017-06-13 | 宇龙计算机通信科技(深圳)有限公司 | A kind of terminal control method and terminal |
| CN107844250A (en) * | 2016-09-18 | 2018-03-27 | 中兴通讯股份有限公司 | A kind of control method, device and terminal |
| CN109117358A (en) * | 2017-06-23 | 2019-01-01 | 百度在线网络技术（北京）有限公司 | test method and test device for electronic equipment |
| CN109165062A (en) * | 2018-07-24 | 2019-01-08 | 苏宁易购集团股份有限公司 | A kind of terminal remote assists control method and system |
| CN109766275A (en) * | 2018-12-29 | 2019-05-17 | 火烈鸟网络(广州)股份有限公司 | Automated testing method, system, storage medium and the equipment of mobile application |
| CN110543789A (en) * | 2018-05-29 | 2019-12-06 | 腾讯科技（深圳）有限公司 | method and device for adapting handle and third-party application program and storage medium |
-
2020
- 2020-02-21 CN CN202010108528.6A patent/CN111309617A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102841712A (en) * | 2012-07-11 | 2012-12-26 | 广州市久邦数码科技有限公司 | Method and system for identifying and editing gestures |
| CN104462879A (en) * | 2014-11-28 | 2015-03-25 | 北京奇虎科技有限公司 | Root-free running control method and device of application program |
| CN106021108A (en) * | 2016-05-19 | 2016-10-12 | 努比亚技术有限公司 | Application program testing device, mobile terminal and method |
| CN106445816A (en) * | 2016-09-07 | 2017-02-22 | 美的智慧家居科技有限公司 | Control execution method and apparatus |
| CN107844250A (en) * | 2016-09-18 | 2018-03-27 | 中兴通讯股份有限公司 | A kind of control method, device and terminal |
| CN106406914A (en) * | 2016-11-11 | 2017-02-15 | 武汉斗鱼网络科技有限公司 | Method and device for obtaining Android system window data |
| CN106598251A (en) * | 2016-12-23 | 2017-04-26 | 重庆墨希科技有限公司 | Smartband-based gesture control method and system, and smartband |
| CN106843729A (en) * | 2017-01-20 | 2017-06-13 | 宇龙计算机通信科技(深圳)有限公司 | A kind of terminal control method and terminal |
| CN109117358A (en) * | 2017-06-23 | 2019-01-01 | 百度在线网络技术（北京）有限公司 | test method and test device for electronic equipment |
| CN110543789A (en) * | 2018-05-29 | 2019-12-06 | 腾讯科技（深圳）有限公司 | method and device for adapting handle and third-party application program and storage medium |
| CN109165062A (en) * | 2018-07-24 | 2019-01-08 | 苏宁易购集团股份有限公司 | A kind of terminal remote assists control method and system |
| CN109766275A (en) * | 2018-12-29 | 2019-05-17 | 火烈鸟网络(广州)股份有限公司 | Automated testing method, system, storage medium and the equipment of mobile application |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113507500A (en) * | 2021-06-04 | 2021-10-15 | 上海闻泰信息技术有限公司 | Terminal control method, terminal control device, computer equipment and computer-readable storage medium |
| CN113407933A (en) * | 2021-06-16 | 2021-09-17 | 成都欧珀通信科技有限公司 | Kernel function reinforcing method and device, storage medium and electronic equipment |
| CN114265535A (en) * | 2021-12-21 | 2022-04-01 | 百度在线网络技术（北京）有限公司 | Push message landing page display method, device, equipment and medium |
| CN114265535B (en) * | 2021-12-21 | 2024-02-27 | 百度在线网络技术（北京）有限公司 | Floor page display method, device, equipment and medium for push message |
| CN114296825A (en) * | 2021-12-29 | 2022-04-08 | 杭州盈高科技有限公司 | Starting method and device of mobile terminal, storage medium and processor |
| CN114327185A (en) * | 2021-12-29 | 2022-04-12 | 盯盯拍（深圳）技术股份有限公司 | Vehicle screen control method and device, medium and electronic equipment |
| CN114327185B (en) * | 2021-12-29 | 2024-02-09 | 盯盯拍（深圳）技术股份有限公司 | Vehicle-mounted screen control method and device, medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111309617A (en) | Application program control method and device, storage medium and electronic equipment | |
| CN111459364B (en) | Icon updating method and device and electronic equipment | |
| CN110211030B (en) | Image generation method and device | |
| CN110059623B (en) | Method and apparatus for generating information | |
| CN113377366A (en) | Control editing method, device, equipment, readable storage medium and product | |
| CN111596991A (en) | Interactive operation execution method and device and electronic equipment | |
| CN111163324A (en) | Information processing method and device and electronic equipment | |
| CN111324376B (en) | Function configuration method, device, electronic equipment and computer readable medium | |
| CN112257478A (en) | Code scanning method, device, terminal and storage medium | |
| CN111309416A (en) | Information display method, device and equipment of application interface and readable medium | |
| CN110673886B (en) | Method and device for generating thermodynamic diagrams | |
| CN113766303B (en) | Multi-screen interaction method, device, equipment and storage medium | |
| CN114417782A (en) | Display method and device and electronic equipment | |
| CN114489891A (en) | Control method, system, device, readable medium and equipment of cloud application program | |
| EP4113446A1 (en) | Sticker processing method and apparatus | |
| CN112558933B (en) | Component rendering method and device, readable medium and electronic equipment | |
| CN113628097A (en) | Image special effect configuration method, image recognition method, image special effect configuration device and electronic equipment | |
| CN114786069B (en) | Video generation method, device, medium and electronic equipment | |
| CN115390776A (en) | Screen projection control method, device, terminal and storage medium | |
| CN111290692B (en) | Picture display method and device, electronic equipment and computer readable medium | |
| CN111290812B (en) | Display method, device, terminal and storage medium of application control | |
| CN114089891A (en) | Display control method and device and electronic equipment | |
| CN113222050A (en) | Image classification method and device, readable medium and electronic equipment | |
| CN114520928A (en) | Display information generation method, information display method and device and electronic equipment | |
| CN110991312A (en) | Method, apparatus, electronic device, and medium for generating detection information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200619 |