CN111767176A - Method and device for remotely controlling terminal equipment - Google Patents
Method and device for remotely controlling terminal equipment Download PDFInfo
- Publication number
- CN111767176A CN111767176A CN201911172739.XA CN201911172739A CN111767176A CN 111767176 A CN111767176 A CN 111767176A CN 201911172739 A CN201911172739 A CN 201911172739A CN 111767176 A CN111767176 A CN 111767176A
- Authority
- CN
- China
- Prior art keywords
- terminal device
- terminal
- user
- terminal equipment
- remote
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000004044 response Effects 0.000 claims abstract description 17
- 238000004891 communication Methods 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 9
- 230000001360 synchronised effect Effects 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 238000012544 monitoring process Methods 0.000 claims description 5
- 238000011084 recovery Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 abstract description 12
- 238000010586 diagram Methods 0.000 description 16
- 230000006870 function Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 4
- 239000008186 active pharmaceutical agent Substances 0.000 description 3
- 238000009434 installation Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 239000003795 chemical substances by application Substances 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/2273—Test methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/2294—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing by remote test
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Telephonic Communication Services (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a method and a device for remotely controlling terminal equipment, and relates to the technical field of computers. One embodiment of the method comprises: acquiring remote control information of a user end to a terminal device through a web application; generating a control instruction of the terminal equipment according to the remote control information, and sending the control instruction to the terminal equipment through a remote drive proxy service; acquiring a stream file generated by the terminal equipment in response to the control instruction, and converting the stream file into a compressed picture in a preset format; and publishing the compressed picture in the preset format to the web application. The implementation mode can reduce the response time of a remote real machine screen, one hardware device can be connected with a plurality of terminal devices, remote multi-control can be realized, and the compatibility test cost is reduced.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method and a device for remotely controlling terminal equipment.
Background
In a terminal device testing scene, various types of terminal devices need to be controlled to perform compatibility testing, function testing and the like, because real terminals (called real terminals for short) such as apple devices and the like are very high in price, the compatibility testing cost is also very high, although some open iOS remote projects such as iosminicip and the like exist at present, the iosminicip adopts video stream return, the remote real terminals are slow in response, in addition, currently, one hardware device (such as a mac notebook computer) can only be connected with one iOS (mobile operating system developed by apple company) device and cannot achieve the multi-control effect, and iOS devices of different types need to be purchased to perform compatibility testing and function testing.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art:
the screen of the remote real machine has slow response, one hardware device can only be connected with one terminal device, and multi-control cannot be realized.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for remotely controlling a terminal device, which can reduce the response time of a remote real machine screen, and a hardware device can be connected to multiple terminal devices, and can implement remote multi-control, thereby reducing the cost of compatibility testing.
To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a method of remotely controlling a terminal device.
A method of remotely controlling a terminal device, comprising: acquiring remote control information of a user end to a terminal device through a web application; generating a control instruction of the terminal equipment according to the remote control information, and sending the control instruction to the terminal equipment through a remote drive proxy service; acquiring a stream file generated by the terminal equipment in response to the control instruction, and converting the stream file into a compressed picture in a preset format; and publishing the compressed picture in the preset format to the web application.
Optionally, before sending the control instruction to the terminal device through a remote driver proxy service, the method includes: and pre-configuring communication port information for starting the remote driving proxy service, wherein the communication port information for starting the remote driving proxy service corresponding to each terminal device is respectively configured when the terminal devices are multiple.
Optionally, the method further comprises: and detecting whether a new terminal device is connected or not at regular time, synchronizing the basic information of the new terminal device to a database and broadcasting the basic information to all the user sides under the condition that the connection of the new terminal device is detected.
Optionally, the method further comprises: and under the condition that the disconnection of the terminal equipment or the user side is monitored, the recovery of the disconnected connection is waited within a preset time period, and under the condition that the disconnection of the terminal equipment or the user side is not recovered within the preset time period, the state of each terminal equipment is synchronized to all the connected user sides.
Optionally, when a first user terminal performs remote control on a first terminal device, a message that the first terminal device is occupied is broadcast to other user terminals except the first user terminal, and the first terminal device is released for the remote control of the other user terminals when the time that the first user terminal does not remotely control the first terminal device reaches a preset time threshold, where the first user terminal is one of the currently connected user terminals, and the first terminal device is one of the currently connected terminal devices.
Optionally, the obtaining, by the web application, remote control information of the user terminal to the terminal device includes: monitoring a mouse event of the user side, and acquiring the remote control information comprising one or more of the following information according to the type of the mouse event: the relative coordinates of the elements clicked by the mouse, the relative coordinates of the elements pressed and released by the mouse, the duration and the relative coordinates of the elements clicked by the mouse, and the information returned to the main screen interface of the terminal equipment.
Optionally, the terminal device is an iOS terminal device.
According to another aspect of the embodiments of the present invention, an apparatus for remotely controlling a terminal device is provided.
An apparatus for remotely controlling a terminal device, comprising: the remote control information acquisition module is used for acquiring remote control information of the user terminal to the terminal equipment through the web application; the control instruction sending module is used for generating a control instruction of the terminal equipment according to the remote control information and sending the control instruction to the terminal equipment through a remote drive proxy service; the stream file conversion module is used for acquiring a stream file generated by the terminal equipment in response to the control instruction and converting the stream file into a compressed picture in a preset format; and the compressed picture publishing module is used for publishing the compressed picture in the preset format to the web application.
Optionally, the system further comprises a configuration module, configured to: and pre-configuring communication port information for starting the remote driving proxy service, wherein the communication port information for starting the remote driving proxy service corresponding to each terminal device is respectively configured when the terminal devices are multiple.
Optionally, the system further includes a terminal device connection detection module, configured to: and detecting whether a new terminal device is connected or not at regular time, synchronizing the basic information of the new terminal device to a database and broadcasting the basic information to all the user sides under the condition that the connection of the new terminal device is detected.
Optionally, the apparatus further comprises a listening module, configured to: and under the condition that the disconnection of the terminal equipment or the user side is monitored, the recovery of the disconnected connection is waited within a preset time period, and under the condition that the disconnection of the terminal equipment or the user side is not recovered within the preset time period, the state of each terminal equipment is synchronized to all the connected user sides.
Optionally, the system further includes a terminal device information management module, configured to broadcast, when a first user terminal performs remote control on a first terminal device, a message that the first terminal device is occupied to other user terminals except the first user terminal, and release the first terminal device for the remote control of the other user terminals when a time when the first user terminal does not remotely control the first terminal device reaches a preset time threshold, where the first user terminal is one of currently connected user terminals, and the first terminal device is one of currently connected terminal devices.
Optionally, the remote control information obtaining module is further configured to: monitoring a mouse event of the user side, and acquiring the remote control information comprising one or more of the following information according to the type of the mouse event: the relative coordinates of the elements clicked by the mouse, the relative coordinates of the elements pressed and released by the mouse, the duration and the relative coordinates of the elements clicked by the mouse, and the information returned to the main screen interface of the terminal equipment.
Optionally, the terminal device is an iOS terminal device.
According to yet another aspect of an embodiment of the present invention, an electronic device is provided.
An electronic device, comprising: one or more processors; a memory for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the method of remotely controlling a terminal device provided by the embodiment of the present invention.
According to yet another aspect of an embodiment of the present invention, a computer-readable medium is provided.
A computer-readable medium, on which a computer program is stored, which, when executed by a processor, implements a method of remotely controlling a terminal device provided by an embodiment of the present invention.
One embodiment of the above invention has the following advantages or benefits: the method comprises the steps of obtaining remote control information of a user terminal to the terminal equipment through a web application to generate a control instruction of the terminal equipment, sending the control instruction to the terminal equipment through a remote driving proxy service, converting a stream file generated by the terminal equipment responding to the control instruction into a compressed picture in a preset format, and issuing the compressed picture to the web application, so that the response time of a screen of a remote real machine can be reduced.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
fig. 1 is a schematic diagram of the main steps of a method of remotely controlling a terminal device according to an embodiment of the present invention;
fig. 2 is a schematic diagram of the overall architecture of a remote control terminal device according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of synchronizing terminal device usage information and updating a terminal device list according to an embodiment of the present invention;
4a, 4b, 4c are schematic diagrams of web application pages according to embodiments of the present invention;
FIG. 5 is a flow chart illustrating remote control of a single terminal device according to an embodiment of the present invention;
fig. 6 is a schematic diagram of main blocks of an apparatus for remotely controlling a terminal device according to an embodiment of the present invention;
FIG. 7 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 8 is a schematic structural diagram of a computer system suitable for implementing a terminal device or a server according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of main steps of a method of remotely controlling a terminal device according to an embodiment of the present invention.
As shown in fig. 1, the method for remotely controlling a terminal device according to an embodiment of the present invention mainly includes the following steps S101 to S104.
Step S101: and acquiring remote control information of the user terminal to the terminal equipment through the web application.
Step S102: and generating a control instruction of the terminal equipment according to the remote control information, and sending the control instruction to the terminal equipment through the remote drive proxy service.
Step S103: and acquiring a stream file generated by the terminal equipment in response to the control instruction, and converting the stream file into a compressed picture in a preset format.
Step S104: and publishing the compressed picture in the preset format to the web application.
The terminal device of the embodiment of the present invention may be an iOS terminal device, and the method for remotely controlling a terminal device of the embodiment of the present invention may be executed by a hardware device connected to the iOS terminal device.
The hardware device may be a Mac (apple operating system) computer, such as a Mac notebook computer. The hardware device comprises a web application and a remote driving proxy service, and simultaneously serves as a back-end server of the web application. The hardware equipment of the embodiment of the invention supports the simultaneous connection of a plurality of terminal equipment.
The remote driver proxy service may be a webdriver agent (WDA), which is an iOS mobile test framework.
In one embodiment, before the control command is sent to the terminal device by the remote driving proxy service, the communication port information started by the remote driving proxy service may be configured in advance, where in the case that there are a plurality of terminal devices, the communication port information started by the remote driving proxy service corresponding to each terminal device is configured separately.
The embodiment of the invention adds a parameterization process for starting the designated communication port by WDA, and specifically, can add configuration attributes as follows:
the USE _ PORT represents a communication PORT of the server, and when a plurality of terminal devices are provided, the communication PORT information corresponding to the remote drive proxy service start of each terminal device is configured, that is, different communication PORT information is configured for the WDA of each terminal device, so that one hardware device can be connected to a plurality of terminal devices, for example, one Mac laptop is connected to a plurality of iOS terminal devices.
In one embodiment, whether a new terminal device is connected or not can be detected at regular time, and in the case that the new terminal device is detected to be connected, the basic information of the new terminal device is synchronized to the database and broadcasted to all the user terminals. For example, a timer may be added to the Web application background, and a determination may be made once per second as to whether a new iOS terminal device is connected, and if a new iOS terminal device is connected, the basic information (which may include a model, a resolution, a system version, a unique identification number, and the like) of the new iOS terminal device is synchronized to the database and broadcast to all online clients, that is, clients currently establishing network connection with the Mac computer, and the Web application background is located in the Mac computer.
In one embodiment, when it is monitored that the terminal device or the user terminal is disconnected, the disconnected connection is waited to be recovered within a preset time period, and when the disconnected connection is not recovered within the preset time period, the states of the terminal devices are synchronized to all the connected user terminals. For example, a certain iOS terminal device is disconnected accidentally, or a certain client is disconnected temporarily from the websocket of the Mac computer due to a network reason, in this case, the backend WDA service may be retained for a period of time (the specific time length may be set as required) to wait for the disconnected iOS terminal device to be connected, or the websocket reconnection is performed with the disconnected client, and if the set time length is reached and the disconnected connection is still not recovered, the states of the iOS terminal devices are synchronized and broadcast to all online clients.
In one embodiment, when a first user terminal performs remote control on a first terminal device, a message that the first terminal device is occupied is broadcast to other user terminals except the first user terminal, and the first terminal device is released for the remote control of the other user terminals when the time that the first user terminal does not remotely control the first terminal device reaches a preset time threshold, where the first user terminal is one of the currently connected user terminals and the first terminal device is one of the currently connected terminal devices. For example, when a user terminal X remotely controls an iOS terminal Y, the Mac computer broadcasts to other online user terminals that the iOS terminal Y is occupied, and when the user terminal X does not perform a remote control operation on the iOS terminal Y within 10 minutes, the iOS terminal Y is automatically released for the other online user terminals to perform a remote control.
In one embodiment, remote control information of a user terminal to a terminal device is acquired through a web application, specifically, a mouse event of the user terminal is monitored, and the remote control information including one or more of the following is acquired according to the type of the mouse event: the relative coordinates of the elements clicked by the mouse, the relative coordinates when the mouse is pressed and released, the duration and the relative coordinates of the elements clicked by the mouse, and the information returned to the main screen interface of the terminal equipment. The elements are specifically terminal device interface elements, including but not limited to application icons, buttons, and the like.
In an embodiment, after obtaining a stream file generated by a terminal device in response to a control instruction, the stream file may be converted into a compressed picture in JPEG format, in an embodiment of the present invention, a picture return processing logic file of WDA is modified: XCUIDevice + fbhelpers.m, change the format of the data transmitted back by the WDA to the web application to JPEG picture format, and compress the size of the transmitted picture to a specified proportion, e.g. 90%.
Fig. 2 is a schematic diagram of the overall architecture of a remote control terminal device according to an embodiment of the present invention.
As shown in fig. 2, the overall architecture of the remote control terminal device according to an embodiment of the present invention includes a web application, a backend API (application program interface), a WDA API, and the web application, the backend API, and the WDA API may be located in a Mac computer. The web application establishes connection with each user side through the websocket, synchronizes the equipment state of the terminal equipment to each user side, and can also synchronize user information (namely information of the user at the user side), time the user, load a screen of the terminal equipment, monitor mouse operation of the user side, and perform operations such as uploading files, loading installation packages or unloading installation packages, automatic disconnection, terminal equipment switching and the like. The back-end API is an API of the back end of the web application, and includes various APIs for installing, uninstalling, obtaining a third-party installation package, monitoring, data warehousing, starting a service, terminating a service, and the like. The WDA API is an interface provided by the WDA, and can realize operations of screen capture, clicking, long pressing, inputting, sliding, resolution acquisition, session identification acquisition, Home interface (main interface) return and the like on the terminal equipment through the WDAAPI.
In the embodiment of the invention, when the terminal device is remotely controlled, the websocket synchronizes the use information of the terminal device and updates the terminal device list, and the flow of synchronizing the use information of the terminal device and updating the terminal device list is shown in fig. 3.
As shown in fig. 3, a web application page is loaded, when a user enters the web application page, a websocket connection (or called websocket link) is established with the user, if the link is established successfully, the number of online users is increased by 1, that is, the number of current online users is increased by 1, a message that the link is established successfully is returned, and current user information, that is, user information that the current link is successfully established, and device information and status are displayed, that is, current terminal device information and status are displayed to the user, and device status in the library is updated and broadcasted to all online users. The devices in the library are the terminal devices, such as mobile phones, to which the Mac computers are currently connected. As a preferred embodiment, before establishing the websocket link, it may be determined whether the current page supports websocket, if not, the unsupported information is prompted, and the process is ended. In addition, if the websocket link is not successfully established, an error prompt message is called back, and then the process is ended.
When the terminal device information is broadcast to the online user side, the online user side may display a web application page on its user device, and fig. 4a, 4b, and 4c are schematic diagrams of the web application page according to an embodiment of the present invention, where fig. 4a is a schematic diagram of a front-end style of the web application according to the embodiment of the present invention, where three terminal devices are taken as an example. Fig. 4b is a schematic front-end page view of a single terminal device remote control according to an embodiment of the present invention. Fig. 4c is a schematic front-end page of a remote control of multiple terminal devices according to an embodiment of the present invention. Reference numerals "4-1" and "4-2" in fig. 4b and 4c represent real terminal device screen interfaces, respectively.
Fig. 5 is a flowchart illustrating remote control of a single terminal device according to an embodiment of the present invention.
As shown in fig. 5, a mouse event of a user side is monitored through a web application, remote control information of the user side to a terminal device is obtained according to the type of the mouse event, specifically, under the condition of clicking, a relative coordinate of a clicked element is obtained, under the condition of sliding, a relative coordinate when the mouse is pressed down and a relative coordinate when the mouse is released are obtained, under the condition of long pressing, the duration and the relative coordinate of the clicked element are obtained, remote control information of clicking, sliding, long pressing, returning to a Home main interface and the like is obtained, the obtained remote control information is sent to the back end of the web application through a back-end specified API, so that a corresponding control instruction of the terminal device is generated according to the remote control information, and the control instruction is sent to the terminal device through a WDA API. And subsequently, converting the stream file generated by the terminal equipment responding to the control instruction into a compressed picture in a JPEG format, and then issuing the compressed picture to the web application.
In a scenario of remote control of multiple terminal devices, the control page (see fig. 4b) of a single terminal device may be switched to the device list page (see fig. 4a) again, and then another terminal device is selected for use, where the page is a multi-machine control page (see fig. 4 c). By modifying the Webdriveragent (WDA) bottom layer and packaging some APIs (open source APIs of WDA), the requirement of terminal equipment on-line remote operation is met, webdriveragent multi-open parameters are increased, a Mac computer is supported to be connected with a plurality of iOS terminal equipment, simultaneous on-line is supported, picture returning logic is optimized, the problems that the iOS terminal equipment is slow in response and cannot be controlled more are solved, the multi-terminal equipment is shared in spare time, and team compatibility cost is reduced.
Fig. 6 is a schematic diagram of main blocks of an apparatus for remotely controlling a terminal device according to an embodiment of the present invention.
As shown in fig. 6, an apparatus 600 for remotely controlling a terminal device according to an embodiment of the present invention mainly includes: a remote control information acquisition module 601, a control instruction sending module 602, a stream file conversion module 603, and a compressed picture publishing module 604.
The remote control information obtaining module 601 is configured to obtain remote control information of the terminal device from the user end through the web application.
And a control instruction sending module 602, configured to generate a control instruction of the terminal device according to the remote control information, and send the control instruction to the terminal device through the remote driving proxy service.
The stream file conversion module 603 is configured to obtain a stream file generated by the terminal device in response to the control instruction, and convert the stream file into a compressed picture in a preset format.
And the compressed picture publishing module 604 is configured to publish the compressed picture in the preset format to the web application.
The terminal equipment of the embodiment of the invention can be iOS terminal equipment. The apparatus 600 for remotely controlling a terminal device according to an embodiment of the present invention may be located in a hardware device connected to an iOS terminal device.
The hardware device may be a Mac (apple operating system) computer, such as a Mac notebook computer. The hardware device comprises a web application and a remote driving proxy service, and simultaneously serves as a back-end server of the web application.
The remote driver proxy service may be a webdriver agent (WDA), which is an iOS mobile test framework.
The remote control information obtaining module 601 may specifically be configured to: monitoring a mouse event of a user side, and acquiring remote control information comprising one or more of the following information according to the type of the mouse event: the relative coordinates of the elements clicked by the mouse, the relative coordinates when the mouse is pressed and released, the duration and the relative coordinates of the elements clicked by the mouse, and the information returned to the main screen interface of the terminal equipment.
In one embodiment, the apparatus 600 for remotely controlling a terminal device further comprises a configuration module for: the method comprises the steps of configuring communication port information for starting the remote driving proxy service in advance, wherein the communication port information for starting the remote driving proxy service corresponding to each terminal device is configured respectively under the condition that a plurality of terminal devices are provided.
In one embodiment, the apparatus 600 for remotely controlling a terminal device further includes a terminal device connection detection module configured to: and detecting whether a new terminal device is connected or not at regular time, synchronizing the basic information of the new terminal device to the database and broadcasting the basic information to all the user sides under the condition that the connection of the new terminal device is detected.
In one embodiment, the apparatus 600 for remotely controlling a terminal device further includes a listening module configured to: and under the condition that the disconnection of the terminal equipment or the user side is monitored, the disconnection of the terminal equipment or the user side is waited for recovering within a preset time period, and under the condition that the disconnection of the terminal equipment or the user side does not recover within the preset time period, the state of each terminal equipment is synchronized to all the connected user sides.
In an embodiment, the apparatus 600 for remotely controlling a terminal device further includes a terminal device information management module, configured to broadcast a message that the first terminal device is occupied to other user terminals except the first user terminal when the first user terminal remotely controls the first terminal device, and release the first terminal device for the other user terminals to perform remote control when a time when the first user terminal does not remotely control the first terminal device reaches a preset time threshold, where the first user terminal is one of currently connected user terminals, and the first terminal device is one of currently connected terminal devices.
The device for remotely controlling the terminal equipment can reduce the response time of a remote real machine screen, one hardware equipment can be connected with a plurality of terminal equipment, remote multi-control can be realized, and the compatibility test cost is reduced.
In addition, the detailed implementation of the apparatus for remotely controlling a terminal device in the embodiment of the present invention has been described in detail in the above method for remotely controlling a terminal device, so that the repeated description is omitted here.
Fig. 7 shows an exemplary system architecture 700 to which the method of remotely controlling a terminal device or the apparatus of remotely controlling a terminal device of the embodiments of the present invention may be applied.
As shown in fig. 7, the system architecture 700 may include terminal devices 701, 702, 703, a network 704, and a server 705. The network 704 serves to provide a medium for communication links between the terminal devices 701, 702, 703 and the server 705. Network 704 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use the terminal devices 701, 702, 703 to interact with a server 705 over a network 704, to receive or send messages or the like. The terminal devices 701, 702, 703 may have installed thereon various communication client applications, such as a shopping-like application, a web browser application, a search-like application, an instant messaging tool, a mailbox client, social platform software, etc. (by way of example only).
The terminal devices 701, 702, 703 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 705 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the terminal devices 701, 702, 703. The backend management server may analyze and perform other processing on the received data such as the product information query request, and feed back a processing result (for example, target push information, product information — just an example) to the terminal device.
It should be noted that the method for remotely controlling the terminal device provided by the embodiment of the present invention is generally executed by the server 705, and accordingly, the apparatus for remotely controlling the terminal device is generally disposed in the server 705.
It should be understood that the number of terminal devices, networks, and servers in fig. 7 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 8, shown is a block diagram of a computer system 800 suitable for use in implementing a terminal device or server of an embodiment of the present application. The terminal device or the server shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 8, the computer system 800 includes a Central Processing Unit (CPU)801 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data necessary for the operation of the system 800 are also stored. The CPU 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
The following components are connected to the I/O interface 805: an input portion 806 including a keyboard, a mouse, and the like; an output section 807 including a signal such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 808 including a hard disk and the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. A drive 810 is also connected to the I/O interface 805 as necessary. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as necessary, so that a computer program read out therefrom is mounted on the storage section 808 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 809 and/or installed from the removable medium 811. The computer program executes the above-described functions defined in the system of the present application when executed by the Central Processing Unit (CPU) 801.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor comprises a remote control information acquisition module, a control instruction sending module, a stream file conversion module and a compressed picture publishing module. The names of these modules do not constitute a limitation to the modules themselves in some cases, and for example, the remote control information acquisition module may also be described as a "module for acquiring remote control information of a user terminal to a terminal device through a web application".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: acquiring remote control information of a user end to a terminal device through a web application; generating a control instruction of the terminal equipment according to the remote control information, and sending the control instruction to the terminal equipment through a remote drive proxy service; acquiring a stream file generated by the terminal equipment in response to the control instruction, and converting the stream file into a compressed picture in a preset format; and publishing the compressed picture in the preset format to the web application.
According to the technical scheme of the embodiment of the invention, the remote control information of the user terminal to the terminal equipment is obtained through the web application to generate the control instruction of the terminal equipment, the control instruction is sent to the terminal equipment through the remote drive proxy service, the stream file generated by the terminal equipment responding to the control instruction is converted into the compressed picture with the preset format and is issued to the web application, the response time of a remote real machine screen can be reduced, under the condition that a plurality of terminal equipment are provided, the communication port information started by the remote drive proxy service corresponding to each terminal equipment is respectively configured, and the purpose that one hardware equipment can be connected with a plurality of terminal equipment, for example, one Mac computer is connected with a plurality of iOS terminal equipment, can be realized, in addition, the embodiment of the invention can realize remote multi-control and reduce the compatibility test cost.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for remotely controlling a terminal device, comprising:
acquiring remote control information of a user end to a terminal device through a web application;
generating a control instruction of the terminal equipment according to the remote control information, and sending the control instruction to the terminal equipment through a remote drive proxy service;
acquiring a stream file generated by the terminal equipment in response to the control instruction, and converting the stream file into a compressed picture in a preset format;
and publishing the compressed picture in the preset format to the web application.
2. The method of claim 1, wherein prior to sending the control instructions to the terminal device via a remote-driven proxy service, comprising:
and pre-configuring communication port information for starting the remote driving proxy service, wherein the communication port information for starting the remote driving proxy service corresponding to each terminal device is respectively configured when the terminal devices are multiple.
3. The method of claim 1, further comprising:
and detecting whether a new terminal device is connected or not at regular time, synchronizing the basic information of the new terminal device to a database and broadcasting the basic information to all the user sides under the condition that the connection of the new terminal device is detected.
4. The method of claim 1, further comprising: and under the condition that the disconnection of the terminal equipment or the user side is monitored, the recovery of the disconnected connection is waited within a preset time period, and under the condition that the disconnection of the terminal equipment or the user side is not recovered within the preset time period, the state of each terminal equipment is synchronized to all the connected user sides.
5. The method according to claim 1, wherein when a first user terminal performs remote control on a first terminal device, a message that the first terminal device is occupied is broadcasted to other user terminals except the first user terminal, and when a time when the first user terminal does not remotely control the first terminal device reaches a preset time threshold, the first terminal device is released for the other user terminals to perform remote control, wherein the first user terminal is one of currently connected user terminals, and the first terminal device is one of currently connected terminal devices.
6. The method of claim 1, wherein obtaining the remote control information of the user terminal to the terminal device through the web application comprises:
monitoring a mouse event of the user side, and acquiring the remote control information comprising one or more of the following information according to the type of the mouse event: the relative coordinates of the elements clicked by the mouse, the relative coordinates of the elements pressed and released by the mouse, the duration and the relative coordinates of the elements clicked by the mouse, and the information returned to the main screen interface of the terminal equipment.
7. The method of claim 1, wherein the terminal device is an iOS terminal device.
8. An apparatus for remotely controlling a terminal device, comprising:
the remote control information acquisition module is used for acquiring remote control information of the user terminal to the terminal equipment through the web application;
the control instruction sending module is used for generating a control instruction of the terminal equipment according to the remote control information and sending the control instruction to the terminal equipment through a remote drive proxy service;
the stream file conversion module is used for acquiring a stream file generated by the terminal equipment in response to the control instruction and converting the stream file into a compressed picture in a preset format;
and the compressed picture publishing module is used for publishing the compressed picture in the preset format to the web application.
9. An electronic device, comprising:
one or more processors;
a memory for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method recited in any of claims 1-7.
10. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911172739.XA CN111767176B (en) | 2019-11-26 | 2019-11-26 | Method and device for remotely controlling terminal equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911172739.XA CN111767176B (en) | 2019-11-26 | 2019-11-26 | Method and device for remotely controlling terminal equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111767176A true CN111767176A (en) | 2020-10-13 |
| CN111767176B CN111767176B (en) | 2024-04-12 |
Family
ID=72718457
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911172739.XA Active CN111767176B (en) | 2019-11-26 | 2019-11-26 | Method and device for remotely controlling terminal equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111767176B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113688043A (en) * | 2021-08-25 | 2021-11-23 | 北京三快在线科技有限公司 | Application program testing method, device, server, iOS device and medium |
| CN115514750A (en) * | 2021-06-21 | 2022-12-23 | 北京字节跳动网络技术有限公司 | Electronic device remote control method, server, system, medium, and device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007096722A2 (en) * | 2006-02-24 | 2007-08-30 | Nokia Corporation | System, apparatus and method for transmitting event information from a remote terminal to subscribers over a network |
| CN109660665A (en) * | 2018-12-21 | 2019-04-19 | Oppo广东移动通信有限公司 | A kind of screen control method, screen control device and terminal device |
-
2019
- 2019-11-26 CN CN201911172739.XA patent/CN111767176B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007096722A2 (en) * | 2006-02-24 | 2007-08-30 | Nokia Corporation | System, apparatus and method for transmitting event information from a remote terminal to subscribers over a network |
| CN109660665A (en) * | 2018-12-21 | 2019-04-19 | Oppo广东移动通信有限公司 | A kind of screen control method, screen control device and terminal device |
Non-Patent Citations (2)
| Title |
|---|
| B WEI: "A Radio Remote Control System and Its Application to the Compress Type Garbage Truck", 《CONSTRUCTION MACHINERY AND EQUIPMENT》, 30 November 2018 (2018-11-30) * |
| 万春旭;: "一种基于网络的新型远程监控系统的设计", 微计算机信息, no. 33, 25 November 2008 (2008-11-25) * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115514750A (en) * | 2021-06-21 | 2022-12-23 | 北京字节跳动网络技术有限公司 | Electronic device remote control method, server, system, medium, and device |
| CN113688043A (en) * | 2021-08-25 | 2021-11-23 | 北京三快在线科技有限公司 | Application program testing method, device, server, iOS device and medium |
| CN113688043B (en) * | 2021-08-25 | 2022-07-26 | 北京三快在线科技有限公司 | Application program testing method, device, server, iOS device and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111767176B (en) | 2024-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20160323647A1 (en) | Video sharing method and system in smart tv | |
| CN107766509B (en) | Method and device for static backup of webpage | |
| US11360737B2 (en) | Method and apparatus for providing speech service | |
| CN109218041B (en) | Request processing method and device for server system | |
| CN111913884A (en) | Distributed test method, device, equipment, system and readable storage medium | |
| CN111246238A (en) | Live broadcast method, device, storage medium and live broadcast system | |
| CN111181765A (en) | Task processing method and device | |
| CN111767176B (en) | Method and device for remotely controlling terminal equipment | |
| CN112596920A (en) | Message processing method and device, electronic equipment and storage medium | |
| CN110324432B (en) | Data processing method, device and medium applied to terminal and computing equipment | |
| CN110912948A (en) | Method and device for reporting problems | |
| CN112579447A (en) | Browser testing method and device | |
| CN106933449B (en) | Icon processing method and device | |
| CN112149392A (en) | Rich text editing method and device | |
| CN113761433B (en) | Service processing method and device | |
| CN115454666A (en) | Data synchronization method and device among message queue clusters | |
| CN112953810B (en) | Processing method and device of network request | |
| CN110768855B (en) | Method and device for testing linkmzation performance | |
| CN110048991B (en) | Method and device for realizing signal interaction reliability | |
| CN110300004B (en) | Method and device for switching network services | |
| CN113572704A (en) | Information processing method, production end, consumption end and server | |
| CN112883103A (en) | Method and device for data transfer between clusters | |
| CN111176982A (en) | Test interface generation method and device | |
| CN113094211B (en) | Backup data processing method and device | |
| CN112583614B (en) | Host cluster management method, device and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |