CN115202779A - Remote control method and related equipment - Google Patents
Remote control method and related equipment Download PDFInfo
- Publication number
- CN115202779A CN115202779A CN202210658244.3A CN202210658244A CN115202779A CN 115202779 A CN115202779 A CN 115202779A CN 202210658244 A CN202210658244 A CN 202210658244A CN 115202779 A CN115202779 A CN 115202779A
- Authority
- CN
- China
- Prior art keywords
- controlled
- control
- window
- equipment
- control instruction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
- G06F9/452—Remote windowing, e.g. X-Window System, desktop virtualisation
Abstract
The embodiment of the application provides a remote control method and related equipment, and relates to the technical field of computers. The remote control method comprises the following steps: the control equipment receives and displays content data of a controlled window sent by the controlled equipment, and the controlled window is a program window running in the background of the controlled equipment. The control device receives a user operation for the controlled window and determines a control instruction according to the user operation. And the control device sends the control instruction to the controlled device so that the controlled device executes the control operation aiming at the controlled window according to the control instruction. In the embodiment of the application, the controlled window does not need to be displayed on the foreground screen. The controlled equipment can also perform other operations in the foreground when being remotely controlled, thereby improving the operating efficiency of the controlled equipment.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a remote control method and a related device.
Background
With the development of mobile internet, the application of remote control technology in enterprise office is more and more extensive. The remote control technology is a technology in which a network control apparatus remotely controls another apparatus. In the prior art, remote education, remote maintenance, remote assistance, equipment remote control and the like are typical application scenes of remote control.
The remote control is mainly realized by remote control software. Specifically, the controlled device intercepts the content on the screen thereof and sends the content to the control device. And the control equipment responds to the intercepted content, determines and sends a control instruction to the controlled equipment. And finally, the controlled equipment executes related operations on a screen according to the control instruction to complete the whole remote control flow.
In the above process, since the controlled device captures the screen content displayed in the foreground, the control device needs to display the controlled window in the foreground to complete the control of the controlled window. And the controlled device is to complete the control operation for the controlled window on the foreground screen. This results in that the controlled device, when remotely controlled, can only perform control operations, and cannot perform other user operations. This will seriously affect the efficiency of operation of the controlled equipment.
Disclosure of Invention
In view of this, the present application provides a remote control method and a related device, where a control device determines a control instruction by acquiring content of a controlled window running in a background, and then sends the control instruction to a controlled device, so that the controlled device executes an operation corresponding to the control instruction. In this way, the controlled window need not be displayed on the foreground screen. The controlled device can also perform other operations in the foreground when remotely controlled.
A first aspect of an embodiment of the present application provides a remote control method, including:
the control equipment receives and controls the content data of the controlled window sent by the equipment, and the controlled window is a program window running in the background of the controlled equipment.
The control device receives a control operation for the controlled window and determines a control instruction according to the control operation.
And the control device sends the control instruction to the controlled device so that the controlled device executes the control operation aiming at the controlled window according to the control instruction.
In an alternative embodiment, the method further comprises:
the control device receives the controlled window identification sent by the controlled device.
And the control equipment sends a window intercepting request to the controlled equipment according to the identifier of the controlled window, wherein the window intercepting request is used for indicating the controlled equipment to carry out screenshot on the content data of the controlled window in the background.
In an optional embodiment, the control device sends the control instruction to the controlled device, including:
the control device presents the content data of the controlled window in the foreground.
The control equipment receives control operation aiming at the controlled window and determines a control instruction according to the control operation, and the control equipment comprises:
the control device receives user control operation aiming at the controlled window at the foreground and determines the control instruction according to the user control operation.
A second aspect of the embodiments of the present application provides another remote control method, including:
the controlled device determines a controlled window in at least one background running window.
And the controlled equipment captures the controlled window in the background to obtain the content data of the controlled window.
The controlled device transmits the content data of the controlled window to the control device.
And the controlled equipment receives a control instruction sent by the control equipment, wherein the control instruction corresponds to the content data of the controlled window.
And the controlled equipment executes the control operation aiming at the controlled window according to the control instruction.
In an optional embodiment, the method further comprises:
the controlled device sends the identity of the controlled window to the controlling device.
And the controlled equipment receives a window intercepting request sent by the control equipment, wherein the window intercepting request corresponds to the identifier of the controlled window.
The controlled equipment carries out screen capture on the controlled window in the background, and the screen capture comprises the following steps:
and the controlled equipment intercepts the request according to the window and performs screen interception on the controlled window in the background.
In an alternative embodiment, the controlled device transmits the content data of the controlled window to the control device, including:
the controlled device performs compression encoding processing on the content data of the controlled window.
And the controlled equipment transmits the content data of the controlled window after the compression coding processing to the control equipment.
In an optional embodiment, the receiving, by the controlled device, the control instruction sent by the control device includes:
and the controlled equipment sends the control instruction to a controlled window, and the controlled window comprises a message interface.
The controlled device executes control operation aiming at the controlled window according to the control instruction, and the control operation comprises the following steps:
and the controlled equipment executes the control operation aiming at the controlled window in the background according to the control instruction.
In an optional embodiment, the receiving, by the controlled device, the control instruction sent by the control device includes:
and the controlled equipment sends the control instruction to a task queue operated by the foreground.
The controlled equipment executes the control operation aiming at the controlled window according to the control instruction, and the control operation comprises the following steps:
the controlled equipment controls the controlled window to run in the foreground;
and the controlled equipment executes the control operation aiming at the controlled window in the foreground according to the control instruction.
In an optional embodiment, the method further comprises:
and the controlled equipment queries the task queue operated by the foreground and acquires all control instructions corresponding to the controlled window.
The controlled equipment executes control operation aiming at the controlled window in the foreground according to the control instruction, and the control operation comprises the following steps:
and the controlled equipment sequentially executes the control operation corresponding to each control instruction in all the control instructions in the foreground.
In an optional embodiment, after the controlled device performs the control operation for the controlled window in the foreground, the method further includes:
and the controlled equipment switches the controlled window to the background and controls the controlled window to run at the background.
A third aspect of embodiments of the present application provides a control apparatus, including:
and the receiving unit is used for receiving the content data of the controlled window sent by the controlled equipment, and the controlled window is a program window running in the background of the controlled equipment.
And the receiving unit is also used for receiving the control operation aiming at the controlled window.
And the determining unit is used for determining the control instruction according to the control operation.
And the sending unit is used for sending the control instruction to the controlled equipment so that the controlled equipment executes the control operation aiming at the controlled window according to the control instruction.
In an optional embodiment, the receiving unit is further configured to receive an identifier of a controlled window sent by the controlled device.
And the sending unit is further used for sending a window intercepting request to the controlled equipment according to the identifier of the controlled window, wherein the window intercepting request is used for indicating the controlled equipment to capture the content data of the controlled window in the background.
In an alternative embodiment of the method according to the invention,
and the receiving unit is also used for displaying the content data of the controlled window in the foreground. User control operations for the controlled window are received at the foreground.
And the determining unit is specifically used for determining the control instruction according to the user control operation.
A fourth aspect of the embodiments of the present application provides a controlled device, including:
a determining unit for determining a controlled window in at least one background running window.
And the processing unit is used for carrying out screen capture on the controlled window in the background to obtain the content data of the controlled window.
And a transmitting unit for transmitting the content data of the controlled window to the control device.
And the receiving unit is used for receiving a control instruction sent by the control equipment, and the control instruction corresponds to the content data of the controlled window.
And the execution unit is used for executing the control operation aiming at the controlled window according to the control instruction.
In an optional embodiment, the sending unit is further configured to send an identifier of the controlled window to the control device.
And the receiving unit is also used for receiving a window intercepting request sent by the control equipment, wherein the window intercepting request corresponds to the identifier of the controlled window.
And the processing unit is specifically used for carrying out screen capture on the controlled window in the background according to the window capture request.
In an optional embodiment, the sending unit is specifically configured to perform a compression encoding process on the content data of the controlled window. And transmitting the content data of the controlled window after the compression coding processing to the control device.
In an optional embodiment, the receiving unit is specifically configured to send the control instruction to a controlled window, and the controlled window includes a message interface.
And the execution unit is specifically used for executing the control operation aiming at the controlled window in the background according to the control instruction.
In an optional embodiment, the receiving unit is specifically configured to send the control instruction to a task queue running in a foreground.
And the execution unit is specifically used for controlling the controlled window to run in the foreground. And according to the control instruction, executing control operation aiming at the controlled window in the foreground.
In an optional embodiment, the controlled device further comprises an acquisition unit.
And the acquisition unit is specifically used for inquiring the task queue operated by the foreground and acquiring all the control instructions corresponding to the controlled window.
And the execution unit is specifically used for sequentially executing the control operation corresponding to each control instruction in all the control instructions in the foreground.
In an optional embodiment, the execution unit is further configured to switch the controlled window to a background, and control the controlled window to run in the background.
A fifth aspect of the embodiments of the present application further provides a control device, including: a memory and a processor, the memory and the processor being coupled.
Wherein the memory is configured to store one or more computer instructions.
The processor is configured to execute one or more computer instructions to implement the remote control method according to the first aspect.
The sixth aspect of the embodiments of the present application further provides a controlled device, including: a memory and a processor, the memory and the processor being coupled.
Wherein the memory is configured to store one or more computer instructions.
The processor is configured to execute one or more computer instructions to implement the remote control method according to the second aspect.
A seventh aspect of the embodiments of the present application further provides a computer-readable storage medium, on which one or more computer instructions are stored, where the instructions are executed by a processor to implement the remote control method according to any one of the above-mentioned technical solutions.
According to the technical scheme provided by the embodiment of the application, the controlled device needs to perform background screenshot on the controlled window running in the background so as to obtain content data of the controlled window and send the content data to the control device. Then the control device receives the user operation aiming at the controlled window, converts the user operation into a control instruction and sends the control instruction to the controlled device, and then the controlled device executes the control operation aiming at the controlled window according to the control instruction. Thus, the controlled window can be remotely controlled without being operated in the foreground of the controlled device, and the controlled device can operate other application programs or perform other operations in the foreground even if the controlled window is remotely controlled. Meanwhile, the control device sends the control instruction to be executed only in a single independent window, so that a plurality of control devices can simultaneously control a plurality of background running windows of the same controlled device. Therefore, the control efficiency and the operation efficiency of the controlled equipment are greatly improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive exercise.
Fig. 1 is a network architecture diagram of a remote control system according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a remote control method according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a control device according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a controlled device provided in an embodiment of the present application;
fig. 5 is a schematic structural diagram of another control device provided in an embodiment of the present application;
fig. 6 is a schematic structural diagram of another controlled device provided in the embodiment of the present application.
Detailed Description
The application provides a remote control method and related equipment. In this way, the controlled window need not be displayed on the foreground screen. The controlled device can also perform other operations in the foreground when remotely controlled.
In order to enable those skilled in the art to better understand the technical solution of the present application, the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. This application is capable of embodiments in many different forms than those described above and it is therefore intended that all such other embodiments, which would be within the scope of the present application and which are obtained by a person of ordinary skill in the art based on the embodiments provided herein without the exercise of inventive faculty, be covered by the present application.
It should be noted that the terms "first," "second," "third," and the like in the claims, the description, and the drawings of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. The data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
With the development of the mobile internet, the application of the remote control technology in the business office is more and more extensive. The remote control technology is a technology in which a network control apparatus remotely controls another apparatus. In the prior art, remote education, remote maintenance, remote assistance, equipment remote control and the like are typical application scenes of remote control.
Currently, the common remote control software is mainly sunflower, teamViewer, etc. Part of the software such as the stem, the QQ, the English WEIDA display card driver and the like also provides remote control functions. The basic scheme of the current remote control comprises two parts, namely, a control end device acquires the real-time state of a controlled end device, and the real-time state generally comprises screen images and sound. And secondly, the control end equipment transmits control instructions, such as mouse movement, keyboard input and other operation instructions, to the controlled end equipment through the network to instruct the controlled end equipment to execute control operation.
The remote control is mainly realized by remote control software. Specifically, the controlled device intercepts the content on the screen and sends the content to the control device. And the control equipment responds to the intercepted content, determines and sends a control instruction to the controlled equipment. And finally, the controlled equipment executes related operations on a screen according to the control instruction to complete the whole remote control process.
In the above process, since the screen content displayed in the foreground is captured by the control device, the control device needs to complete the control of the controlled window, and the controlled window needs to be displayed in the foreground. And the controlled device is to complete the control operation for the controlled window on the foreground screen. This results in that the controlled device, when being remotely controlled, can only perform control operations, and cannot perform other user operations. This will seriously affect the efficiency of the operation of the controlled device.
In order to solve the above problems, the present application provides a remote control method and related device. The controlled device may intercept content data of the controlled window running in the background and then transmit the content data to the controlling device. The control device determines a control instruction for the controlled window based on the content data of the controlled window. And the controlled device executes the control operation aiming at the controlled window after receiving the control instruction. In this way, the controlled window does not need to be displayed on the foreground screen, and therefore the controlled device can perform other operations in the foreground while being remotely controlled. The method, apparatus, terminal, and computer-readable storage medium described in this application are described in further detail below with reference to specific embodiments and accompanying drawings.
Before the embodiments of the present application are described, a remote control system is briefly described as follows:
fig. 1 is a network architecture diagram of a remote control system according to an embodiment of the present application. As shown in fig. 1, the control-side device and the controlled-side device are connected through a communication network. And then the control end can utilize the remote control software to realize the remote control of the controlled end equipment. In the embodiment of the application, a plurality of control devices can be allowed to simultaneously control the same controlled-end device. This is because each control device can control one program window of the controlled end device, and each controlled program window can run in the background. In this way, the control operations required by each control device are performed only in a single program window. Because the program windows are independent and run in the background, a plurality of control devices can control the same controlled end device at the same time. Moreover, the foreground of the controlled device can also execute normal user operation, thereby greatly improving the operating efficiency of the controlled device.
Fig. 2 is a schematic flowchart of a remote control method according to an embodiment of the present application, with reference to a network architecture diagram of the remote control system shown in fig. 1. It should be noted that the steps shown in the flowchart may be performed in a computer system such as a set of computer-executable instructions, and in some cases, the steps shown may be performed in a different logical order than that shown in the flowchart.
As shown in fig. 2, the remote control method includes the following steps:
201. the controlled device determines a controlled window in at least one background running window.
When a controlled device needs to be remotely controlled, it is first necessary to establish an active network connection with the controlling device. Illustratively, the controlled device receives a user operation, and sends a connection request to the control device according to information such as a connection mode, a network identifier of the control device, and an access password input by the user. The network identifier of the control device may be an IP address of the control device, a computer name of the control device, or a login user name, which is not limited herein.
In this embodiment, the control device is configured to control a background-running window on the controlled device, and therefore the controlled device needs to determine a controlled window that needs to be controlled by the controlled device from the background-running windows, and then send an identifier of the controlled window to the control device. It can be understood that one control device remotely controls one background running window on the controlled device, so that the controlled device can allow a plurality of control devices to simultaneously control different background running windows thereof. Thus, the remote control efficiency can be greatly improved.
After receiving a connection request initiated by the controlled device, the control device can verify the user authority according to an access password and the like, and establish connection according to a verification result. The control device may then identify the controlled window in the background running window of the controlled device based on the unique identification of the target window specified by the controlled device.
202. The controlled device sends an identification of the controlled window to the controlling device.
For example, the controlled device may send the identifier of the controlled window to the control device in the connection request, or may send the identifier of the controlled window separately. And is not particularly limited.
203. And the control equipment sends a window intercepting request to the controlled equipment according to the identifier of the controlled window.
After receiving the identifier of the controlled window, the control device needs to send a window capture request to the controlled device, and instruct the controlled device to perform screenshot operation on the controlled window running in the background, so as to obtain the content data of the controlled window. It can be understood that the window interception request is used for instructing the controlled device to start a remote control process, and the controlled device needs to respond to the window interception request to provide the window content of the controlled window for the control device.
204. And the controlled equipment captures the request according to the window and captures the content data of the controlled window in the background.
The controlled device may utilize the window API to make continuous screenshots of the controlled window to obtain real-time content data of the controlled window. Then, the real-time content data is encoded according to a specific encoding/decoding format specification, and the encoded content data is sent to the control device. The encoding/decoding format specification may be selected according to actual requirements, and may be a JPEG picture stream, an h.264 video stream, an h.265 video stream specification, or the like, which is not limited specifically.
The control device can also control the screenshot frequency of the controlled device. Illustratively, the window intercept request includes a preset time period. And after receiving the window interception request, the controlled equipment periodically intercepts the content data of the controlled window according to a preset time period.
The controlled device can use the window API to perform continuous screenshot on the controlled window to obtain real-time content data of the controlled window. For example, for a background controlled window displayed in a GDI manner in a Windows system, a background screenshot may be performed in a gdigrab manner.
It can be understood that in the existing remote control process, the controlled device outputs the screenshot of the whole screen displayed in the foreground, but the control device usually only needs part of the effective service data part, so in order to increase the data processing rate and reduce the waste of transmission bandwidth, the window API needs to intercept the content of the most effective area in the background window as much as possible. The effective utilization of transmission bandwidth is ensured, the data volume can be reduced, and the data processing speed of the control equipment is improved.
In the embodiment of the application, the controlled device does not intercept the screen content displayed in the foreground, but intercepts the window content of the controlled window running in the background. Therefore, the controlled window does not need to be displayed on the foreground, and occupies the foreground. When the controlled equipment is remotely controlled, other operations can be carried out by using the foreground, so that the operating efficiency of the controlled equipment is improved. Meanwhile, because the controlled windows run in the background, a plurality of control devices can simultaneously control different background running windows of one controlled device, and because the operation of each control device is only carried out in a single background window of the controlled device, the plurality of control devices do not conflict with each other, so that the concurrent control of the plurality of control devices on the controlled device can be realized.
205. The controlled device transmits the content data of the controlled window to the control device.
After the controlled device captures the controlled window, the content data of the captured image needs to be sent to the control device. Illustratively, real-time content data needs to be encoded and transmitted according to a specific encoding/decoding format specification, so that the data transmission rate can be increased and the data security of the content data can be ensured.
206. The control device presents the content data of the controlled window transmitted by the controlled device.
After the control device receives the screen image coded data transmitted back from the controlled device, the control device needs to decode information according to an agreed coding/decoding mode, and then displays the content data of the controlled window. It can be understood that the remote control refers to how the control device operates, and the same operation is required to be performed on the controlled device, so that the control device needs to display the content data of the controlled window to receive the user operation of the user for the controlled window.
207. The control device receives a user operation directed to the controlled window.
The control device needs to receive the operation of the user aiming at the controlled window, and then convert the user operation into a control instruction and send the control instruction to the controlled device. Therefore, the controlled equipment can execute the same control operation as the user operation on the controlled window according to the control instruction so as to realize the purpose of remotely controlling the controlled window by the user.
208. The control device determines a control instruction according to a user operation.
The control device needs to convert the user operation into a control instruction and send the control instruction to the controlled device, so that the controlled device can execute relevant operations according to the control instruction to complete remote control.
209. And the control equipment sends the control instruction to the controlled equipment.
For example, if the controlled window supports the message API, the control device may send the control instruction directly to the controlled window. In this way, the controlled window can execute the control operation in the background according to the control instruction. Thus, the controlled window does not need to be operated in the foreground of the controlled device, and the controlled device can operate other application programs in the foreground to execute other operations even if the controlled device is remotely controlled. Meanwhile, because the control instruction is directly executed in the independent background window, a plurality of control devices can simultaneously control a plurality of background running windows of the same controlled device. Thereby greatly improving the operating efficiency of the controlled equipment.
If the controlled window does not support the message API, the control operation is still performed by means of the foreground of the controlled device. The control device needs to send the control instruction to the task queue running in the foreground. Therefore, the controlled device needs to execute the control operation corresponding to the control instruction on the foreground to complete the remote control of the controlled window.
210. And the controlled equipment executes the control operation aiming at the controlled window according to the control instruction.
The controlled device needs to perform a control operation according to the state of the controlled window. In the above example, if the controlled window supports the message API, the controlled device performs the control operation in the controlled window running in the background according to the control instruction. In this way, the controlled window does not need to run in the foreground of the controlled device, and the controlled window is remotely controlled in the background.
If the controlled window does not support the message API, the control operation still needs to be performed by means of the foreground of the controlled device. The controlled device needs to call the controlled window to operate in the foreground, then obtains the control instruction in the task queue operated in the foreground, and executes the control operation aiming at the controlled window in the foreground according to the control instruction, so as to complete the remote control of the controlled window.
In the above embodiment, the controlled device needs to perform a background screenshot on the controlled window running in the background to obtain content data of the controlled window and send the content data to the control device. Then the control device receives the user operation aiming at the controlled window, converts the user operation into a control instruction and sends the control instruction to the controlled device, and then the controlled device executes the control operation aiming at the controlled window according to the control instruction. Thus, the controlled window can be remotely controlled without being operated in the foreground of the controlled device, and the controlled device can operate other application programs or perform other operations in the foreground even if the controlled window is remotely controlled. Meanwhile, the control device sends the control instruction to be executed only in a single independent window, so that a plurality of control devices can simultaneously control a plurality of background running windows of the same controlled device. Therefore, the control efficiency and the operation efficiency of the controlled equipment are greatly improved.
The following is a detailed description of a scenario in which the controlled window does not support the message API:
it will be appreciated that if the controlled window supports the message API, then the control device need only send the control instructions directly to the controlled window. Thus, the controlled window can execute the control operation in the controlled window in the background according to the control instruction. The controlled window does not need to run in the foreground of the controlled device, and the controlled window can be remotely controlled in the background. Thus, the foreground can be released, and other applications can be run or other operations can be executed even in the foreground of the controlled device in the remote control mode. Meanwhile, because the control instruction is directly executed in the independent background windows, a plurality of control devices can simultaneously control a plurality of background running windows of the same controlled device.
If the controlled window does not support the message API, the foreground of the controlled device needs to be relied on to execute the control operation corresponding to the control instruction. At this time, the control device needs to send the control instruction to the message queue running in the foreground. When the message queue receives the control instruction of the controlled window, the controlled device calls the controlled window to execute and display in the foreground, and then the control instruction in the message queue is used for executing the operation aiming at the controlled window in the foreground. It can be understood that a plurality of background running windows of the controlled device can be controlled by different control devices at the same time, and therefore, the message queue receives control instructions of a plurality of controlled windows. At this time, the controlled window needs to alternately run a plurality of controlled windows in the foreground to complete the control operations corresponding to the plurality of controlled windows.
It will be appreciated that the order in which the foreground of the controlled windows runs may be determined by the temporal order in which the control instructions are received by the message queue. Specifically, the controlled device determines a control instruction to be executed according to a receiving sequence of the control instructions in the message queue, then determines a controlled window corresponding to the control instruction, then calls the controlled window to a foreground for operation, finally executes a control operation aiming at the controlled window in the foreground according to the control instruction, and switches the controlled window to operate in the background after the control operation is executed. To execute the next control instruction in the message queue.
In order to reduce the operation delay, the controlled device may also reduce the number of foreground switching times by rearranging or the like. Illustratively, the task queue has a plurality of control instructions for the controlled window a and a plurality of control instructions for the controlled window B, and the execution sequence of the control instructions is alternated after the control instructions are arranged according to the message receiving time. At this time, the controlled device may execute all the control instructions of the controlled window a first, and then execute all the control instructions of the controlled window a after all the control instructions of the controlled window a are executed. In this way, the foreground only needs to switch the controlled window once, so that the operation delay can be reduced, and the data processing burden of the controlled equipment is relieved.
The entire remote control process is described below with reference to a specific example:
the control device A remotely controls the program window 1 of the controlled device, the control device A and the program window 1 of the controlled device establish remote connection, and the controlled device continuously transmits the content data of the window 1 to the control device A.
The control device B remotely controls the program window 2 of the controlled device, and the control device B and the program window 2 of the controlled device both establish a remote connection and continuously transmit the content data of the window 2 to the control device B.
Similarly, the control device N remotely controls the program window N of the controlled device, both the control device N and the program window N of the controlled device establish a remote connection, and the controlled device continuously transmits the content data of the window N to the control device N.
The control device A receives a remote operation action 1 sent by a user, the control device A converts the remote operation action 1 into a control instruction 1 and sends the control instruction 1 to the controlled device, and the controlled device receives the control instruction 1 and puts the control instruction into an instruction execution queue. And then the controlled equipment foreground the program window 1, sends a control instruction 1 to an operating system of the controlled equipment, and instructs the operating system of the controlled equipment to execute the operation aiming at the program window 1 in the foreground according to the control instruction 1.
The control device B receives a remote operation action 2 sent by a user, the control device B converts the remote operation action 2 into a control instruction 2 and sends the control instruction 2 to the controlled device, and the controlled device receives the control instruction 2 and puts the control instruction into an instruction execution queue. And then the controlled equipment foreground the program window 2, sends a control instruction 2 to an operating system of the controlled equipment, and instructs the operating system of the controlled equipment to execute the operation aiming at the program window 2 in the foreground according to the control instruction 1.
Similarly, the control device N receives a remote operation action N sent by a user, the control device N converts the remote operation action N into a control instruction N and sends the control instruction N to the controlled device, and the controlled device receives the control instruction N and puts the control instruction N into an instruction execution queue. And then the controlled equipment carries out background processing on the program window n-1 and foreground processing on the program window n, sends a control instruction n to an operating system of the controlled equipment, and instructs the operating system of the controlled equipment to execute the operation aiming at the program window n in the foreground according to the control instruction n.
In the process of alternately switching the background window to the foreground to realize remote control, in order to avoid frequently switching the window so as to reduce operation delay and reduce CPU load, all the operation instructions aiming at the current window in the instruction queue can be preferentially executed instead of being executed according to the issuing time sequence of the operation instructions in the instruction execution queue.
In this embodiment, when the controlled device executes the remote control operation of the controlled window that does not support the message API, the controlled windows running in the background need to be alternately run in the foreground, and the foreground is used to complete the remote control operation for the controlled windows. The controlled device determines a control instruction to be executed according to the receiving sequence of the control instructions in the message queue, then determines a controlled window corresponding to the control instruction, then calls the controlled window to a foreground for operation, finally executes the control operation aiming at the controlled window in the foreground according to the control instruction, and switches the controlled window to operate in the background after the control operation is executed. In order to reduce the operation delay, the controlled device may also reduce the number of foreground switching times by rearrangement or the like. Therefore, a plurality of control devices can simultaneously control a plurality of background running windows of the same controlled device. Therefore, the control efficiency and the operation efficiency of the controlled equipment are greatly improved.
Fig. 3 is a schematic structural diagram of a control device according to an embodiment of the present application, and details provided in the embodiment are described below with reference to fig. 3. The following description refers to examples for explaining the technical solution of the present application, and is not intended to limit the practical use.
The control apparatus includes:
the receiving unit 301 is configured to receive and display content data of a controlled window sent by a controlled device, where the controlled window is a program window running in a background of the controlled device.
The receiving unit 301 is further configured to receive a user operation for the controlled window.
A determining unit 302, configured to determine a control instruction according to a user operation.
A sending unit 303, configured to send a control instruction to the controlled device, so that the controlled device executes a control operation for the controlled window according to the control instruction.
In an optional embodiment, the receiving unit 301 is further configured to receive an identifier of a controlled window sent by a controlled device.
The sending unit 303 is further configured to send a window capture request to the controlled device according to the identifier of the controlled window, where the window capture request is used to instruct the controlled device to capture a screenshot of the content data of the controlled window in the background.
In an alternative embodiment, the sending unit 303 is specifically configured to send the control instruction directly to the controlled window. Or sending the control instruction to a task queue operated by the foreground of the controlled equipment.
Fig. 4 is a schematic structural diagram of a controlled device provided in an embodiment of the present application, and the detailed description provided in this embodiment is described below with reference to fig. 4. The following description refers to examples for the purpose of illustrating the technical solutions of the present application, and is not intended to limit the practical use thereof
The controlled device includes:
a determining unit 401, configured to determine a controlled window in at least one background running window.
And the processing unit 402 is configured to perform screen capturing on the controlled window in the background to obtain content data of the controlled window.
A sending unit 403, configured to send the content data of the controlled window to the control device.
A receiving unit 404, configured to receive a control instruction sent by the control device, where the control instruction corresponds to content data of the controlled window.
And an execution unit 405, configured to execute a control operation for the controlled window according to the control instruction.
In an optional embodiment, the sending unit 403 is further configured to send an identifier of the controlled window to the control device.
The receiving unit 404 is further configured to receive a window intercept request sent by the control device, where the window intercept request corresponds to an identifier of the controlled window.
The processing unit 402 is specifically configured to capture a screen of the controlled window in the background according to the window capture request.
In an optional implementation, the sending unit 403 is specifically configured to perform compression encoding processing on the content data of the controlled window. And transmitting the content data of the controlled window after the compression coding processing to the control device.
In an optional embodiment, the receiving unit 404 is specifically configured to send the control instruction to a controlled window, and the controlled window includes a message interface.
The execution unit 405 is specifically configured to execute a control operation for the controlled window in the background according to the control instruction.
In an optional embodiment, the receiving unit 404 is specifically configured to send the control instruction to a task queue running in a foreground.
The execution unit 405 is specifically configured to control the controlled window to run in the foreground. And according to the control instruction, executing control operation aiming at the controlled window in the foreground.
In an optional embodiment, the controlled device further comprises an obtaining unit 406.
The obtaining unit 406 is specifically configured to query a task queue running in a foreground, and obtain all control instructions corresponding to the controlled window.
The execution unit 405 is specifically configured to sequentially execute, in the foreground, a control operation corresponding to each control instruction in all the control instructions.
In an optional embodiment, the execution unit 405 is further configured to switch the controlled window to a background, and control the controlled window to run in the background.
It should be noted that, the information interaction and execution process between the modules/units in the apparatus are based on the same concept as that of the method embodiments corresponding to fig. 1 to fig. 2 in the present application, and specific contents may refer to the description in the foregoing method embodiments in the present application, and are not described herein again.
In the above embodiment, the controlled device needs to perform a background screenshot on the controlled window running in the background to obtain content data of the controlled window and send the content data to the control device. And then the control equipment receives the user operation aiming at the controlled window, converts the user operation into a control instruction and sends the control instruction to the controlled equipment, and then the controlled equipment executes the control operation aiming at the controlled window according to the control instruction. Thus, the controlled window can be remotely controlled without being operated in the foreground of the controlled device, and the controlled device can operate other application programs or perform other operations in the foreground even if the controlled window is remotely controlled. Meanwhile, the control device sends the control instruction to be executed only in a single independent window, so that a plurality of control devices can simultaneously control a plurality of background running windows of the same controlled device. Therefore, the control efficiency and the operation efficiency of the controlled equipment are greatly improved.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a control device provided in an embodiment of the present application, and the control device 800 may be embodied as a virtual reality VR device, a mobile phone, a tablet, a notebook computer, an intelligent wearable device, a monitoring data processing device, or a radar data processing device, which is not limited herein. The control device 800 may be deployed with the control device described in the embodiment corresponding to fig. 3, and is configured to implement the functions in the embodiments corresponding to fig. 1 to fig. 2. Specifically, the control device 800 includes: a receiver 801, a transmitter 802, a processor 803 and a memory 804 (wherein the number of processors 803 in the execution device 800 may be one or more, for example, one processor in fig. 5), wherein the processor 803 may include an application processor 8031 and a communication processor 8032. In some embodiments of the present application, the receiver 801, the transmitter 802, the processor 803, and the memory 804 may be connected by a bus or other means.
The memory 804 may include a read-only memory and a random access memory, and provides instructions and data to the processor 803. A portion of the memory 804 may also include non-volatile random access memory (NVRAM). The memory 804 stores the processor and control operations, which may include various control operations for implementing various operations, an executable module or data structure, or a subset thereof, or an expanded set thereof.
The processor 803 controls the operation of the execution apparatus. In a particular application, the various components of the execution device are coupled together by a bus system that may include a power bus, a control bus, a status signal bus, etc., in addition to a data bus. For clarity of illustration, the various buses are referred to in the figures as a bus system.
The method disclosed in the embodiments of the present application can be applied to the processor 803 or implemented by the processor 803. The processor 803 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 803. The processor 803 may be a general-purpose processor, a Digital Signal Processor (DSP), a microprocessor or a microcontroller, and may further include an Application Specific Integrated Circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, or discrete hardware components. The processor 803 may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 804, and the processor 803 reads the information in the memory 804 to complete the steps of the method in combination with the hardware thereof.
The receiver 801 may be used to receive input numeric or character information and generate signal inputs related to performing device related settings and function control. The transmitter 802 may be used to output numeric or character information through a first interface; the transmitter 802 may also be configured to send instructions to the disk groups via the first interface to modify data in the disk groups; the transmitter 802 may also include a display device such as a display screen.
In this embodiment, the application processor 8031 in the processor 803 is configured to execute the remote control method in the embodiment corresponding to fig. 2. It should be noted that, the specific manner of executing each step by the application processor 8031 is based on the same concept as that of each method embodiment corresponding to fig. 2 in the present application, and the technical effect brought by the specific manner is the same as that of each method embodiment corresponding to fig. 2 in the present application, and specific contents may refer to the description in the foregoing method embodiments in the present application, and are not described herein again.
Referring to fig. 6, fig. 6 is a schematic structural diagram of the controlled device provided in the embodiment of the present application, and the controlled device 900 may be disposed with the controlled device described in the embodiment corresponding to fig. 4, and is used to implement the function of the controlled device in the embodiment corresponding to fig. 2. In particular, the controlled device 900 is implemented by one or more servers, and the controlled device 900 may have relatively large differences according to different configurations or performances, and may include one or more Central Processing Units (CPUs) 922 (e.g., one or more processors) and a memory 932, one or more storage media 930 (e.g., one or more mass storage devices) storing applications 942 or data 944. Memory 932 and storage media 930 can be, among other things, transient storage or persistent storage. The program stored on storage medium 930 may include one or more modules (not shown), each of which may include a sequence of instructions for operating on the exercise device. Still further, central processor 922 may be configured to communicate with storage medium 930 to execute a series of instruction operations in storage medium 930 on controlled device 900.
The controlled device 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input output interfaces 958, and/or one or more operating systems 941, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, etc.
In this embodiment, in one case, the central processor 922 is configured to execute the steps performed by the controlled device described in the corresponding embodiment of fig. 2. The specific manner in which the central processing unit 922 executes the foregoing steps is based on the same concept as that of each method embodiment corresponding to fig. 2 in the present application, and the technical effect brought by the specific manner is the same as that of each method embodiment corresponding to fig. 2 in the present application, and specific contents may refer to the description in the foregoing method embodiments in the present application, and are not repeated here.
The embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium includes computer instructions, and the computer instructions, when executed by a processor, are used to implement a technical solution of any one of the remote control methods in the embodiment of the present application.
An embodiment of the present application further provides a computer program product, which when executed on a computer causes the computer to execute the steps in the method as described in the foregoing embodiment shown in fig. 2.
The control device and the controlled device provided by the embodiment of the application may be chips, and the chips include: a processing unit, which may be, for example, a processor, and a communication unit, which may be, for example, an input/output interface, a pin or a circuit, etc. The processing unit may execute the computer executable instructions stored by the storage unit to cause the chip to perform the method described in the embodiment of fig. 2 above. Alternatively, the storage unit may be a storage unit in the chip, such as a register, a cache, and the like, and the storage unit may also be a storage unit located outside the chip in the radio access device, such as a read-only memory (ROM) or another type of static storage device that may store static information and instructions, a Random Access Memory (RAM), and the like.
Although the present application has been described with reference to the preferred embodiments, it is not intended to limit the present application, and those skilled in the art can make variations and modifications without departing from the spirit and scope of the present application, therefore, the scope of the present application should be determined by the claims that follow.
Claims (15)
1. A remote control method, characterized in that the remote control method comprises:
the control equipment receives the content data of the controlled window sent by the controlled equipment; the controlled window is a program window running in the background of the controlled equipment;
the control equipment receives control operation aiming at the controlled window and determines a control instruction according to the control operation;
the control equipment sends the control instruction to the controlled equipment; so that the controlled equipment executes the control operation aiming at the controlled window according to the control instruction.
2. The method of claim 1, further comprising:
the control device receives the identifier of the controlled window sent by the controlled device;
the control equipment sends a window interception request to the controlled equipment according to the identifier of the controlled window; and the window interception request is used for indicating the controlled equipment to carry out screenshot on the content data of the controlled window in the background.
3. The method according to claim 1 or 2, characterized in that the method further comprises:
the control equipment displays the content data of the controlled window in the foreground;
the control equipment receives control operation aiming at the controlled window and determines a control instruction according to the control operation, and the control equipment comprises:
and the control equipment receives user control operation aiming at the controlled window at the foreground and determines the control instruction according to the user control operation.
4. A remote control method, characterized in that the remote control method comprises:
the controlled device determines a controlled window in at least one background running window;
the controlled device captures the controlled window in the background to obtain the content data of the controlled window;
the controlled equipment sends the content data of the controlled window to the control equipment;
the controlled equipment receives a control instruction sent by the control equipment; the control instruction corresponds to the content data of the controlled window;
and the controlled equipment executes the control operation aiming at the controlled window according to the control instruction.
5. The method of claim 4, further comprising:
the controlled device sends the identifier of the controlled window to the control device;
the controlled equipment receives a window interception request sent by the control equipment; the window interception request corresponds to the identifier of the controlled window;
the controlled device performs screen capture on the controlled window in the background, and the screen capture comprises the following steps:
and the controlled equipment captures the screen of the controlled window in the background according to the window capturing request.
6. The method of any of claims 4 to 5, wherein the controlled device sends the content data of the controlled window to a control device, comprising:
the controlled equipment performs compression coding processing on the content data of the controlled window;
and the controlled equipment sends the content data of the controlled window after compression coding processing to the control equipment.
7. The method according to any one of claims 4 to 6, wherein the controlled device receives the control instruction sent by the control device, and comprises:
the controlled equipment sends the control instruction to the controlled window; wherein the controlled window comprises a message interface;
the controlled equipment executes the control operation aiming at the controlled window according to the control instruction, and the control operation comprises the following steps:
and the controlled equipment executes the control operation aiming at the controlled window in a background according to the control instruction.
8. The method according to any one of claims 4 to 6, wherein the controlled device receives the control instruction sent by the control device, and comprises:
the controlled equipment sends the control instruction to a task queue operated by a foreground;
the controlled device executes control operation aiming at the controlled window according to the control instruction, and the control operation comprises the following steps:
the controlled device controls the controlled window to run in the foreground;
and the controlled equipment executes the control operation aiming at the controlled window in a foreground according to the control instruction.
9. The method of claim 8, further comprising:
the controlled equipment inquires the task queue operated by the foreground to obtain all control instructions corresponding to the controlled window;
the controlled device executes the control operation aiming at the controlled window in a foreground according to the control instruction, and the control operation comprises the following steps:
and the controlled equipment sequentially executes the control operation corresponding to each control instruction in all the control instructions in the foreground.
10. The method according to any one of claims 8 to 9, further comprising, after the controlled device performs the control operation for the controlled window in a foreground:
and the controlled equipment switches the controlled window to a background and controls the controlled window to run in the background.
11. A control apparatus, characterized in that the control apparatus comprises:
a receiving unit, configured to receive content data of a controlled window transmitted by a controlled device; the controlled window is a program window running in the background of the controlled equipment;
the receiving unit is further used for receiving control operation aiming at the controlled window;
the determining unit is used for determining a control instruction according to the control operation;
a sending unit, configured to send the control instruction to the controlled device; so that the controlled device executes the control operation aiming at the controlled window according to the control instruction.
12. A controlled device, characterized in that the controlled device comprises:
a determining unit, configured to determine a controlled window in at least one background-running window;
the processing unit is used for carrying out screen capture on the controlled window in the background to obtain content data of the controlled window;
a transmitting unit configured to transmit content data of the controlled window to a control device;
the receiving unit is used for receiving a control instruction sent by the control equipment; the control instruction corresponds to the content data of the controlled window;
and the execution unit is used for executing the control operation aiming at the controlled window according to the control instruction.
13. A control apparatus, characterized by comprising: a memory and a processor, the memory and the processor coupled;
the memory is to store one or more computer instructions;
the processor is configured to execute the one or more computer instructions to implement the remote control method according to any one of claims 1 to 3.
14. A controlled device, comprising: a memory and a processor, the memory and the processor coupled;
the memory is to store one or more computer instructions;
the processor is configured to execute the one or more computer instructions to implement the remote control method according to any one of claims 4 to 10.
15. A computer-readable storage medium having stored thereon one or more computer instructions, the instructions being executable by a processor to implement a remote control method as claimed in any one of claims 1 to 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210658244.3A CN115202779A (en) | 2022-06-10 | 2022-06-10 | Remote control method and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210658244.3A CN115202779A (en) | 2022-06-10 | 2022-06-10 | Remote control method and related equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115202779A true CN115202779A (en) | 2022-10-18 |
Family
ID=83576608
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210658244.3A Pending CN115202779A (en) | 2022-06-10 | 2022-06-10 | Remote control method and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115202779A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117311892A (en) * | 2023-11-30 | 2023-12-29 | 深圳富士伟业科技有限公司 | Remote assistance method, system, electronic device and storage medium |
-
2022
- 2022-06-10 CN CN202210658244.3A patent/CN115202779A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117311892A (en) * | 2023-11-30 | 2023-12-29 | 深圳富士伟业科技有限公司 | Remote assistance method, system, electronic device and storage medium |
| CN117311892B (en) * | 2023-11-30 | 2024-03-08 | 深圳富士伟业科技有限公司 | Remote assistance method, system, electronic device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111803940B (en) | Game processing method and device, electronic equipment and computer-readable storage medium | |
| CN110928511A (en) | Screen projection method and device and electronic equipment | |
| CN110012328B (en) | Method and device for interacting with smart television | |
| US20140082052A1 (en) | Data redirection system and method for providing data redirection service | |
| EP3637771A1 (en) | Cloud desktop system, and image sequence compression and encoding method, and medium therefor | |
| CN112165520B (en) | Screen projection control method, screen projection receiving end and storage medium | |
| CN109739786B (en) | DMA controller and heterogeneous acceleration system | |
| CN110442450B (en) | Image processing device, method and device and material calculating and rendering system | |
| CN112905289A (en) | Application picture display method, device, terminal, screen projection system and medium | |
| US9801146B2 (en) | Terminal and synchronization control method among terminals | |
| CN110933075B (en) | Service calling method and device, electronic equipment and storage medium | |
| US11671678B2 (en) | Method and device, equipment, and storage medium for data processing | |
| CN111427524A (en) | Remote control method, device, computer equipment and storage medium | |
| US20170272545A1 (en) | Method and system for transmitting remote screen | |
| CN111414208A (en) | Application program starting method, device and equipment | |
| US20120166585A1 (en) | Apparatus and method for accelerating virtual desktop | |
| CN115202779A (en) | Remote control method and related equipment | |
| CN115767092A (en) | Image transmission method, cloud desktop, image transmission system, image transmission equipment and product | |
| CN109788251B (en) | Video processing method, device and storage medium | |
| CN115445194A (en) | Rendering method, device and equipment of game and storage medium | |
| CN114051067A (en) | Image acquisition method, apparatus, device, storage medium, and program product | |
| CN111143017B (en) | Cloud operation system interaction processing method, client and cloud operation system | |
| CN110012003B (en) | Cloud application screen capturing method and device | |
| CN112583741A (en) | Multi-rate mode data transmission control method and device based on cache queue | |
| CN112770358B (en) | Multi-rate mode data transmission control method and device based on service data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |