CN111767176B - Method and device for remotely controlling terminal equipment - Google Patents
Method and device for remotely controlling terminal equipment Download PDFInfo
- Publication number
- CN111767176B CN111767176B CN201911172739.XA CN201911172739A CN111767176B CN 111767176 B CN111767176 B CN 111767176B CN 201911172739 A CN201911172739 A CN 201911172739A CN 111767176 B CN111767176 B CN 111767176B
- Authority
- CN
- China
- Prior art keywords
- terminal device
- terminal
- terminal equipment
- remote
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000004044 response Effects 0.000 claims abstract description 19
- 238000004891 communication Methods 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 238000012544 monitoring process Methods 0.000 claims description 5
- 238000012360 testing method Methods 0.000 abstract description 12
- 238000010586 diagram Methods 0.000 description 18
- 230000006870 function Effects 0.000 description 7
- 239000008186 active pharmaceutical agent Substances 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000001360 synchronised effect Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- -1 i.e. Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/2273—Test methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/2294—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing by remote test
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Telephonic Communication Services (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a method and a device for remotely controlling terminal equipment, and relates to the technical field of computers. One embodiment of the method comprises the following steps: acquiring remote control information of a user side to terminal equipment through web application; generating a control instruction of the terminal equipment according to the remote control information, and sending the control instruction to the terminal equipment through a remote driving proxy service; acquiring a stream file generated by the terminal equipment in response to the control instruction, and converting the stream file into a compressed picture with a preset format; and publishing the compressed picture with the preset format to the web application. According to the embodiment, the response time of a remote real machine screen can be reduced, one hardware device can be connected with a plurality of terminal devices, remote multi-control can be realized, and the compatibility testing cost is reduced.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for remotely controlling a terminal device.
Background
In the scenario of terminal equipment testing, control needs to be performed on various types of terminal equipment to perform compatibility testing and function testing, and the like, as the price of terminal equipment such as apple equipment is quite high, the compatibility testing cost is quite high, although some open-source iOS remote projects such as iosminiicap and the like exist at present, the iosminiicap adopts video streaming back, the remote real equipment responds slowly, in addition, the current hardware equipment (such as a mac notebook computer) can only be connected with one iOS (mobile operating system) equipment developed by apple corporation, and the effect of multi-control cannot be achieved, and iOS equipment of different types needs to be purchased to perform compatibility testing and function testing.
In the process of implementing the present invention, the inventor finds that at least the following problems exist in the prior art:
the remote real machine screen has slow response, one hardware device can only be connected with one terminal device, and multiple control cannot be realized.
Disclosure of Invention
In view of this, the embodiments of the present invention provide a method and an apparatus for remotely controlling a terminal device, which can reduce the response time of a remote real machine screen, and one hardware device can connect a plurality of terminal devices, and can realize remote multi-control, thereby reducing the compatibility testing cost.
To achieve the above object, according to one aspect of the embodiments of the present invention, there is provided a method of remotely controlling a terminal device.
A method of remotely controlling a terminal device, comprising: acquiring remote control information of a user side to terminal equipment through web application; generating a control instruction of the terminal equipment according to the remote control information, and sending the control instruction to the terminal equipment through a remote driving proxy service; acquiring a stream file generated by the terminal equipment in response to the control instruction, and converting the stream file into a compressed picture with a preset format; and publishing the compressed picture with the preset format to the web application.
Optionally, before the control instruction is sent to the terminal device through a remote driving proxy service, the method includes: and pre-configuring communication port information started by the remote drive proxy service, wherein when a plurality of terminal devices are provided, the communication port information started by the remote drive proxy service corresponding to each terminal device is respectively configured.
Optionally, the method further comprises: and detecting whether a new terminal device is connected or not at regular time, synchronizing the basic information of the new terminal device to a database and broadcasting the basic information to all the user terminals under the condition that the new terminal device is detected to be connected.
Optionally, the method further comprises: and under the condition that the disconnection of the terminal equipment or the user terminal is monitored, waiting for the restoration of the disconnected connection in a preset time period, and under the condition that the disconnected connection is not restored in the preset time period, synchronizing the states of all the terminal equipment to all the connected user terminals.
Optionally, when the first user side remotely controls the first terminal device, broadcasting a message that the first terminal device is occupied to other user sides except the first user side, and releasing the first terminal device for the other user sides to remotely control if the time of the first user side not remotely controlling the first terminal device reaches a preset time threshold, wherein the first user side is one of the user sides currently connected, and the first terminal device is one of the terminal devices currently connected.
Optionally, the obtaining, by the web application, remote control information of the user end to the terminal device includes: monitoring a mouse event of the user side, and acquiring the remote control information including one or more of the following according to the type of the mouse event: and the relative coordinates of the elements clicked by the mouse, the relative coordinates when the mouse is pressed and released, the time length and the relative coordinates of the elements clicked by the mouse and the information of the main interface of the screen of the terminal equipment are returned.
Optionally, the terminal device is an iOS terminal device.
According to another aspect of the embodiments of the present invention, there is provided an apparatus for remotely controlling a terminal device.
An apparatus for remotely controlling a terminal device, comprising: the remote control information acquisition module is used for acquiring the remote control information of the user side to the terminal equipment through the web application; the control instruction sending module is used for generating a control instruction of the terminal equipment according to the remote control information and sending the control instruction to the terminal equipment through a remote driving proxy service; the stream file conversion module is used for obtaining a stream file generated by the terminal equipment in response to the control instruction and converting the stream file into a compressed picture with a preset format; and the compressed picture publishing module is used for publishing the compressed picture with the preset format to the web application.
Optionally, the method further comprises a configuration module for: and pre-configuring communication port information started by the remote drive proxy service, wherein when a plurality of terminal devices are provided, the communication port information started by the remote drive proxy service corresponding to each terminal device is respectively configured.
Optionally, the terminal device connection detection module is further included, and is used for: and detecting whether a new terminal device is connected or not at regular time, synchronizing the basic information of the new terminal device to a database and broadcasting the basic information to all the user terminals under the condition that the new terminal device is detected to be connected.
Optionally, the device further comprises a monitoring module for: and under the condition that the disconnection of the terminal equipment or the user terminal is monitored, waiting for the restoration of the disconnected connection in a preset time period, and under the condition that the disconnected connection is not restored in the preset time period, synchronizing the states of all the terminal equipment to all the connected user terminals.
Optionally, the terminal equipment information management module is further configured to broadcast, when the first user side performs remote control on the first terminal equipment, a message that the first terminal equipment is occupied to other user sides except the first user side, and release the first terminal equipment for the other user sides to perform remote control if the time of the first user side, which is not in remote control on the first terminal equipment, reaches a preset time threshold, where the first user side is one of the currently connected user sides, and the first terminal equipment is one of the currently connected terminal equipment.
Optionally, the remote control information acquisition module is further configured to: monitoring a mouse event of the user side, and acquiring the remote control information including one or more of the following according to the type of the mouse event: and the relative coordinates of the elements clicked by the mouse, the relative coordinates when the mouse is pressed and released, the time length and the relative coordinates of the elements clicked by the mouse and the information of the main interface of the screen of the terminal equipment are returned.
Optionally, the terminal device is an iOS terminal device.
According to yet another aspect of an embodiment of the present invention, an electronic device is provided.
An electronic device, comprising: one or more processors; and the memory is used for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are enabled to realize the method for remotely controlling the terminal equipment provided by the embodiment of the invention.
According to yet another aspect of an embodiment of the present invention, a computer-readable medium is provided.
A computer readable medium having stored thereon a computer program which when executed by a processor implements a method of remotely controlling a terminal device provided by an embodiment of the present invention.
One embodiment of the above invention has the following advantages or benefits: the method comprises the steps of obtaining remote control information of a user side to terminal equipment through web application to generate a control instruction of the terminal equipment, sending the control instruction to the terminal equipment through remote drive proxy service, converting a stream file generated by the terminal equipment in response to the control instruction into a compressed picture in a preset format, and issuing the compressed picture to the web application, so that the response time of a remote real machine screen can be reduced.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
fig. 1 is a schematic diagram of main steps of a method of remotely controlling a terminal device according to an embodiment of the present invention;
fig. 2 is a schematic diagram of the overall architecture of a remote control terminal device according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of synchronizing terminal device usage information and updating a terminal device list according to an embodiment of the present invention;
FIGS. 4a, 4b, 4c are schematic diagrams of web application pages according to embodiments of the present invention;
fig. 5 is a schematic flow chart of a remote control of a single terminal device according to an embodiment of the present invention;
fig. 6 is a schematic diagram of main modules of an apparatus for remotely controlling a terminal device according to an embodiment of the present invention;
FIG. 7 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
fig. 8 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of main steps of a method for remotely controlling a terminal device according to an embodiment of the present invention.
As shown in fig. 1, the method for remotely controlling a terminal device according to an embodiment of the present invention mainly includes the following steps S101 to S104.
Step S101: and acquiring remote control information of the user side to the terminal equipment through the web application.
Step S102: and generating a control instruction of the terminal equipment according to the remote control information, and sending the control instruction to the terminal equipment through a remote driving proxy service.
Step S103: and acquiring a stream file generated by the terminal equipment in response to the control instruction, and converting the stream file into a compressed picture with a preset format.
Step S104: and publishing the compressed picture with the preset format to the web application.
The terminal device of the embodiment of the invention can be an iOS terminal device, and the method for remotely controlling the terminal device of the embodiment of the invention can be executed by a hardware device connected with the iOS terminal device.
The hardware device may be a Mac (apple operating system) computer, such as a Mac notebook. The hardware device comprises a web application and a remote drive proxy service, and simultaneously serves as a back-end server of the web application. The hardware equipment of the embodiment of the invention supports the simultaneous connection of a plurality of terminal equipment.
The remote driven proxy service may be webdriveagent (WDA for short, or WDA service), which is an iOS mobile test framework.
In one embodiment, before the control instruction is sent to the terminal device through the remote driving proxy service, communication port information started by the remote driving proxy service may be preconfigured, where in a case that there are a plurality of terminal devices, communication port information started by the remote driving proxy service corresponding to each terminal device is respectively configured.
The embodiment of the invention increases the parameterization process of the WDA for starting the appointed communication port, and specifically, the configuration attribute can be increased as follows:
the use_port represents a communication PORT of the server, and in the case that the number of terminal devices is multiple, communication PORT information started by the remote drive proxy service corresponding to each terminal device is configured respectively, that is, different communication PORT information is configured corresponding to WDA of each terminal device respectively, so that one hardware device can be connected with multiple terminal devices, for example, one Mac notebook computer is connected with multiple iOS terminal devices.
In one embodiment, whether a new terminal device is connected or not can be detected regularly, and when the new terminal device is detected to be connected, basic information of the new terminal device is synchronized to a database and broadcasted to all user terminals. For example, a timer may be added to the Web application background, and it may be detected once a second whether a new iOS terminal device is connected, and if a new iOS terminal device is connected, the basic information (which may include a model, a resolution, a system version, a unique identifier, etc.) of the new iOS terminal device is synchronized to a database and broadcast to all online clients, where the online clients are clients that currently establish a network connection with a Mac computer, and the Web application background is located in the Mac computer.
In one embodiment, when the disconnection of the terminal device or the user terminal is monitored, the disconnection is waited for to be recovered within a preset time period, and when the disconnection is not recovered within the preset time period, the state of each terminal device is synchronized to all the connected user terminals. For example, a certain iOS terminal device is disconnected accidentally, or a certain user terminal is disconnected temporarily from the websocket connection of the Mac computer due to network reasons, in this case, the back-end WDA service may be reserved for a period of time (a specific time length may be set according to needs), so as to wait for the disconnected iOS terminal device to connect, or a websocket reconnection is performed with the disconnected user terminal, if the set time length is reached, the state of each iOS terminal device is synchronized and broadcasted to all online user terminals.
In one embodiment, when the first user terminal remotely controls the first terminal device, a message that the first terminal device is occupied is broadcasted to other user terminals except the first user terminal, and when the time that the first user terminal does not remotely control the first terminal device reaches a preset time threshold, the first terminal device is released for the other user terminal to remotely control, wherein the first user terminal is one of the user terminals currently connected, and the first terminal device is one of the terminal devices currently connected. For example, when a certain ue X remotely controls a certain iOS terminal device Y, the Mac computer broadcasts to other online ues that the iOS terminal device Y is occupied, and when the ue X does not remotely control the iOS terminal device Y within 10 minutes, the iOS terminal device Y is automatically released for remote control by other online ues.
In one embodiment, the remote control information of the user side to the terminal device is obtained through the web application, specifically, a mouse event of the user side is monitored, and the remote control information including one or more of the following is obtained according to the type of the mouse event: the relative coordinates of the elements clicked by the mouse, the relative coordinates when the mouse is pressed and released, the time length and the relative coordinates of the elements clicked by the mouse, and the information of the main interface of the screen of the terminal equipment are returned. The elements are specifically terminal device interface elements including, but not limited to, application icons, buttons, and the like.
In one embodiment, after the stream file generated by the terminal device in response to the control instruction is obtained, the stream file may be converted into a compressed picture in JPEG format, and the embodiment of the present invention modifies the picture feedback processing logic file of WDA: xcuidevice+fbhelpers.m, changes the format of WDA data returned to web applications to JPEG picture format, and compresses the transferred picture size to a specified proportion, e.g., 90%.
Fig. 2 is a schematic diagram of the overall architecture of a remote control terminal device according to an embodiment of the present invention.
As shown in fig. 2, the overall architecture of the remote control terminal device according to an embodiment of the present invention includes a web application, a back-end API (application program interface), a WDA API, which may be located in one Mac computer. The web application establishes connection with each user terminal through websocket, synchronizes the equipment state of the terminal equipment to each user terminal, can synchronize user information (namely information of users of the user terminals) through the web application, counts the use of the user, loads a screen of the terminal equipment, monitors the mouse operation of the user terminal, and performs operations such as uploading files, loading and unloading installation packages, automatically disconnecting and switching the terminal equipment. The backend APIs, i.e., APIs of the web application backend, include various APIs that install, uninstall, acquire third party installation packages, monitor, data warehouse entry, start services, terminate services, and the like. The WDA API is an interface provided by the WDA externally, and through the WDA API, operations such as screenshot, clicking, long-pressing, inputting, sliding, obtaining resolution, obtaining session id, returning to Home interface (main interface) and the like can be implemented on the terminal device.
When the embodiment of the invention realizes the remote control of the terminal equipment, the terminal equipment use information is synchronized through websocket and the terminal equipment list is updated, and the flow of the terminal equipment use information synchronization and the terminal equipment list updating is shown in figure 3.
As shown in fig. 3, a web application page is loaded, when a user side enters the web application page, websocket connection (or websocket link) is established with the user side, if the link establishment is successful, the number of online people is increased by 1, namely the number of current online user sides is increased by 1, a message of successful link establishment is returned, current user information is displayed, namely the user side information of successful current link is displayed, and equipment information and state are displayed, namely the current terminal equipment information and state are displayed to the user side, and the equipment state in a library is updated and broadcasted to all online user sides. The devices in the library are terminal devices, such as cell phones, to which the current Mac computer is connected. As a preferred embodiment, before the websocket link is established, whether the current page supports websocket may be judged first, if not, information of the non-support is prompted, and the flow is ended. In addition, if the websocket link is not successfully established, the error prompt information is called back, and then the flow is ended.
When terminal device information is broadcast to an online user terminal, the online user terminal may display a web application page on its own user device, and fig. 4a, fig. 4b, and fig. 4c are schematic diagrams of web application pages according to an embodiment of the present invention, where fig. 4a is a schematic diagram of a front end style of a web application in an embodiment of the present invention, and three terminal devices are taken as an example. Fig. 4b is a schematic diagram of a front-end page of a remote control of a single terminal device according to an embodiment of the present invention. Fig. 4c is a schematic diagram of a front-end page remotely controlled by a plurality of terminal devices according to an embodiment of the present invention. The reference numerals "4-1" and "4-2" in fig. 4b and 4c, respectively, represent the actual terminal device screen interfaces.
Fig. 5 is a schematic flow chart of a remote control of a single terminal device according to an embodiment of the present invention.
As shown in fig. 5, a web application monitors a mouse event of a user terminal, acquires remote control information of the user terminal to a terminal device according to a type of the mouse event, specifically, acquires relative coordinates of a clicked element in the case of clicking, acquires relative coordinates when the mouse is pressed and relative coordinates when released in the case of sliding, acquires duration and relative coordinates of the clicked element in the case of long pressing, acquires remote control information of clicking, sliding, long pressing, returning to a Home main interface and the like, and transmits the acquired remote control information to a rear end of the web application through a rear-end designated API so as to generate a corresponding control instruction of the terminal device according to the remote control information, and transmits the control instruction to the terminal device through a WDA API. And converting the stream file generated by the terminal equipment in response to the control instruction into a compressed picture in a JPEG format, and then issuing the compressed picture to the web application.
In the scenario of remote control of multiple terminal devices, the control page (as shown in fig. 4 b) of a single terminal device may be switched to the device list page (as shown in fig. 4 a) again, and then another terminal device is selected for use, where the page is a multiple-machine control page (as shown in fig. 4 c). Through modifying Webdriveragent (WDA) bottom layer and packaging some APIs (open source APIs of WDA), the requirement of online remote operation of terminal equipment is met, webdrivegent multi-opening parameters are increased, one Mac computer is supported to be connected with a plurality of iOS terminal equipment, the simultaneous online operation is supported, the picture feedback logic is optimized, the problems that the response of the iOS terminal equipment is slow and the multi-control cannot be realized are solved, the multi-terminal equipment is shared in the idle state, and the team compatibility cost is reduced.
Fig. 6 is a schematic diagram of main modules of an apparatus for remotely controlling a terminal device according to an embodiment of the present invention.
As shown in fig. 6, an apparatus 600 for remotely controlling a terminal device according to an embodiment of the present invention mainly includes: the remote control information acquisition module 601, the control instruction sending module 602, the stream file conversion module 603 and the compressed picture issuing module 604.
The remote control information obtaining module 601 is configured to obtain remote control information of the user terminal to the terminal device through a web application.
The control instruction sending module 602 is configured to generate a control instruction of the terminal device according to the remote control information, and send the control instruction to the terminal device through the remote driving proxy service.
The stream file conversion module 603 is configured to obtain a stream file generated by the terminal device in response to the control instruction, and convert the stream file into a compressed picture in a preset format.
The compressed picture publishing module 604 is configured to publish the compressed picture in the preset format to the web application.
The terminal device of the embodiment of the invention can be an iOS terminal device. The apparatus 600 for remotely controlling a terminal device according to the embodiment of the present invention may be located in a hardware device connected to an iOS terminal device.
The hardware device may be a Mac (apple operating system) computer, such as a Mac notebook. The hardware device comprises a web application and a remote drive proxy service, and simultaneously serves as a back-end server of the web application.
The remote driven proxy service may be webdriveagent (WDA for short, or WDA service), which is an iOS mobile test framework.
The remote control information acquisition module 601 may specifically be configured to: monitoring a mouse event of a user terminal, and acquiring remote control information comprising one or more of the following according to the type of the mouse event: the relative coordinates of the elements clicked by the mouse, the relative coordinates when the mouse is pressed and released, the time length and the relative coordinates of the elements clicked by the mouse, and the information of the main interface of the screen of the terminal equipment are returned.
In one embodiment, the apparatus 600 for remotely controlling a terminal device further comprises a configuration module for: and pre-configuring communication port information started by the remote drive proxy service, wherein when a plurality of terminal devices are provided, the communication port information started by the remote drive proxy service corresponding to each terminal device is respectively configured.
In one embodiment, the apparatus 600 for remotely controlling a terminal device further includes a terminal device connection detection module for: and detecting whether a new terminal device is connected or not at regular time, synchronizing the basic information of the new terminal device to a database and broadcasting the basic information to all user terminals under the condition that the new terminal device is detected to be connected.
In one embodiment, the apparatus 600 for remotely controlling a terminal device further includes a listening module for: and under the condition that the disconnection of the terminal equipment or the user terminal is monitored, waiting for the restoration of the disconnected connection in a preset time period, and under the condition that the disconnected connection is not restored in the preset time period, synchronizing the states of all the terminal equipment to all the connected user terminals.
In one embodiment, the apparatus 600 for remotely controlling a terminal device further includes a terminal device information management module, configured to, when the first user terminal performs remote control on the first terminal device, broadcast a message that the first terminal device is occupied to other user terminals except the first user terminal, and release the first terminal device for the other user terminal to perform remote control if a time when the first user terminal does not remotely control the first terminal device reaches a preset time threshold, where the first user terminal is one of the currently connected user terminals, and the first terminal device is one of the currently connected terminal devices.
The device for remotely controlling the terminal equipment can reduce the response time of a remote real machine screen, one hardware equipment can be connected with a plurality of terminal equipment, remote multi-control can be realized, and the compatibility testing cost is reduced.
In addition, the specific implementation of the apparatus for remotely controlling a terminal device in the embodiment of the present invention has been described in detail in the method for remotely controlling a terminal device above, so that the description is not repeated here.
Fig. 7 shows an exemplary system architecture 700 of a method of remotely controlling a terminal device or an apparatus of remotely controlling a terminal device to which embodiments of the present invention may be applied.
As shown in fig. 7, a system architecture 700 may include terminal devices 701, 702, 703, a network 704, and a server 705. The network 704 is the medium used to provide communication links between the terminal devices 701, 702, 703 and the server 705. The network 704 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 705 via the network 704 using the terminal devices 701, 702, 703 to receive or send messages or the like. Various communication client applications such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only) may be installed on the terminal devices 701, 702, 703.
The terminal devices 701, 702, 703 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 705 may be a server providing various services, such as a background management server (by way of example only) providing support for shopping-type websites browsed by users using the terminal devices 701, 702, 703. The background management server may analyze and process the received data such as the product information query request, and feedback the processing result (e.g., the target push information, the product information—only an example) to the terminal device.
It should be noted that, the method for remotely controlling the terminal device according to the embodiment of the present invention is generally executed by the server 705, and accordingly, the apparatus for remotely controlling the terminal device is generally disposed in the server 705.
It should be understood that the number of terminal devices, networks and servers in fig. 7 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 8, there is illustrated a schematic diagram of a computer system 800 suitable for use in implementing a terminal device or server in accordance with embodiments of the present application. The terminal device or server illustrated in fig. 8 is merely an example, and should not impose any limitation on the functionality and scope of use of the embodiments of the present application.
As shown in fig. 8, the computer system 800 includes a Central Processing Unit (CPU) 801 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the system 800 are also stored. The CPU 801, ROM 802, and RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804.
The following components are connected to the I/O interface 805: an input portion 806 including a keyboard, mouse, etc.; an output portion 807 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage section 808 including a hard disk or the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. The drive 810 is also connected to the I/O interface 805 as needed. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as needed so that a computer program read out therefrom is mounted into the storage section 808 as needed.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 809, and/or installed from the removable media 811. The above-described functions defined in the system of the present application are performed when the computer program is executed by a Central Processing Unit (CPU) 801.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: the processor comprises a remote control information acquisition module, a control instruction sending module, a stream file conversion module and a compressed picture release module. The names of these modules do not limit the module itself in some cases, and for example, the remote control information acquisition module may also be described as "a module for acquiring remote control information of a user terminal to a terminal device through a web application".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include: acquiring remote control information of a user side to terminal equipment through web application; generating a control instruction of the terminal equipment according to the remote control information, and sending the control instruction to the terminal equipment through a remote driving proxy service; acquiring a stream file generated by the terminal equipment in response to the control instruction, and converting the stream file into a compressed picture with a preset format; and publishing the compressed picture with the preset format to the web application.
According to the technical scheme of the embodiment of the invention, the remote control information of the user side to the terminal equipment is obtained through the web application to generate the control instruction of the terminal equipment, the control instruction is sent to the terminal equipment through the remote drive proxy service, the stream file generated by the terminal equipment in response to the control instruction is converted into the compressed picture with the preset format and is issued to the web application, the response time of a remote true screen can be reduced, the communication port information started by the remote drive proxy service corresponding to each terminal equipment is respectively configured under the condition that the terminal equipment is multiple, one hardware equipment can be connected with multiple terminal equipment, such as one Mac computer is connected with multiple iOS terminal equipment, and in addition, the embodiment of the invention can realize remote multi-control and reduce the compatibility test cost.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (9)
1. A method of remotely controlling a terminal device, comprising:
acquiring remote control information of a user side to terminal equipment through web application;
generating a control instruction of the terminal equipment according to the remote control information, and sending the control instruction to the terminal equipment through a remote driving proxy service;
acquiring a stream file generated by the terminal equipment in response to the control instruction, and converting the stream file into a compressed picture with a preset format;
publishing the compressed picture with the preset format to the web application;
before the control instruction is sent to the terminal equipment through a remote driving proxy service, the method comprises the following steps: and pre-configuring communication port information started by the remote drive proxy service, wherein when a plurality of terminal devices are provided, the communication port information started by the remote drive proxy service corresponding to each terminal device is respectively configured.
2. The method as recited in claim 1, further comprising:
and detecting whether a new terminal device is connected or not at regular time, synchronizing the basic information of the new terminal device to a database and broadcasting the basic information to all the user terminals under the condition that the new terminal device is detected to be connected.
3. The method as recited in claim 1, further comprising: and under the condition that the disconnection of the terminal equipment or the user terminal is monitored, waiting for the restoration of the disconnected connection in a preset time period, and under the condition that the disconnected connection is not restored in the preset time period, synchronizing the states of all the terminal equipment to all the connected user terminals.
4. The method according to claim 1, wherein when a first user terminal remotely controls a first terminal device, a message that the first terminal device is occupied is broadcasted to other user terminals except the first user terminal, and when a time for which the first user terminal does not remotely control the first terminal device reaches a preset time threshold, the first terminal device is released for the other user terminal to remotely control, wherein the first user terminal is one of currently connected user terminals, and the first terminal device is one of currently connected terminal devices.
5. The method according to claim 1, wherein the obtaining, by the web application, the remote control information of the user side to the terminal device includes:
monitoring a mouse event of the user side, and acquiring the remote control information including one or more of the following according to the type of the mouse event: and the relative coordinates of the elements clicked by the mouse, the relative coordinates when the mouse is pressed and released, the time length and the relative coordinates of the elements clicked by the mouse and the information of the main interface of the screen of the terminal equipment are returned.
6. The method of claim 1, wherein the terminal device is an iOS terminal device.
7. An apparatus for remotely controlling a terminal device, comprising:
the remote control information acquisition module is used for acquiring the remote control information of the user side to the terminal equipment through the web application;
the control instruction sending module is used for generating a control instruction of the terminal equipment according to the remote control information and sending the control instruction to the terminal equipment through a remote driving proxy service;
the stream file conversion module is used for obtaining a stream file generated by the terminal equipment in response to the control instruction and converting the stream file into a compressed picture with a preset format;
the compressed picture publishing module is used for publishing the compressed picture with the preset format to the web application;
before the control instruction is sent to the terminal equipment through a remote driving proxy service, the method comprises the following steps: and pre-configuring communication port information started by the remote drive proxy service, wherein when a plurality of terminal devices are provided, the communication port information started by the remote drive proxy service corresponding to each terminal device is respectively configured.
8. An electronic device, comprising:
one or more processors;
a memory for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-6.
9. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911172739.XA CN111767176B (en) | 2019-11-26 | 2019-11-26 | Method and device for remotely controlling terminal equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911172739.XA CN111767176B (en) | 2019-11-26 | 2019-11-26 | Method and device for remotely controlling terminal equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111767176A CN111767176A (en) | 2020-10-13 |
| CN111767176B true CN111767176B (en) | 2024-04-12 |
Family
ID=72718457
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911172739.XA Active CN111767176B (en) | 2019-11-26 | 2019-11-26 | Method and device for remotely controlling terminal equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111767176B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115514750A (en) * | 2021-06-21 | 2022-12-23 | 北京字节跳动网络技术有限公司 | Electronic device remote control method, server, system, medium, and device |
| CN113688043B (en) * | 2021-08-25 | 2022-07-26 | 北京三快在线科技有限公司 | Application program testing method, device, server, iOS device and medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007096722A2 (en) * | 2006-02-24 | 2007-08-30 | Nokia Corporation | System, apparatus and method for transmitting event information from a remote terminal to subscribers over a network |
| CN109660665A (en) * | 2018-12-21 | 2019-04-19 | Oppo广东移动通信有限公司 | A kind of screen control method, screen control device and terminal device |
-
2019
- 2019-11-26 CN CN201911172739.XA patent/CN111767176B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007096722A2 (en) * | 2006-02-24 | 2007-08-30 | Nokia Corporation | System, apparatus and method for transmitting event information from a remote terminal to subscribers over a network |
| CN109660665A (en) * | 2018-12-21 | 2019-04-19 | Oppo广东移动通信有限公司 | A kind of screen control method, screen control device and terminal device |
Non-Patent Citations (2)
| Title |
|---|
| A Radio Remote Control System and Its Application to the Compress Type Garbage Truck;B Wei;《Construction Machinery and Equipment》;20181130;全文 * |
| 一种基于网络的新型远程监控系统的设计;万春旭;;微计算机信息;20081125(第33期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111767176A (en) | 2020-10-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20160323647A1 (en) | Video sharing method and system in smart tv | |
| US20150117513A1 (en) | Bandwidth reduction system and method | |
| CN109510754B (en) | Online document generation method, device and system and electronic equipment | |
| CN111767176B (en) | Method and device for remotely controlling terminal equipment | |
| CN111478781B (en) | Message broadcasting method and device | |
| CN112306851A (en) | Automatic testing method and device | |
| CN104092815A (en) | Method and device for inter-application information interaction based on Android system | |
| CN112416396B (en) | Application program updating method and system | |
| CN111246238A (en) | Live broadcast method, device, storage medium and live broadcast system | |
| CN109218041B (en) | Request processing method and device for server system | |
| CN111181765A (en) | Task processing method and device | |
| CN112084042B (en) | Message processing method and device | |
| CN110912948B (en) | Method and device for reporting problems | |
| CN112838935A (en) | Electronic data exchange method and system | |
| CN110795328A (en) | Interface testing method and device | |
| CN110324432B (en) | Data processing method, device and medium applied to terminal and computing equipment | |
| WO2015000417A1 (en) | Method, apparatus, application interactive platform, terminal, and system for uploading an image | |
| CN109088907B (en) | File transfer method and device | |
| CN106933449B (en) | Icon processing method and device | |
| CN112131095B (en) | Pressure testing method and device | |
| CN110048991B (en) | Method and device for realizing signal interaction reliability | |
| CN112732728A (en) | Data synchronization method and system | |
| CN105472452B (en) | Method and system for realizing data sharing among multiple terminals through intermediary equipment | |
| CN112583614B (en) | Host cluster management method, device and computer readable medium | |
| CN113194277B (en) | Method, device and storage medium for multi-screen video of conference terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |